openstack/charm-deployment-guide
Code Issues Proposed changes

Merge "Make Swift appendix more general"

Browse Source
This commit is contained in:
Zuul 2020-03-10 23:12:27 +00:00 committed by Gerrit Code Review
commit bf69e80554
3 changed files with 507 additions and 283 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

282
deploy-guide/source/app-swift-gc.rst
View File

@ -1,282 +0,0 @@
================================
Appendix R: Swift Global Cluster
================================
Overview
--------
As of the 20.02 charm release, with OpenStack Newton or later, support for
a global cluster in swift is available. See the `Upstream Overview`_ for
an overview of the feature.
.. warning::
The Swift Global Cluster feature is in a preview state (still under
development).
Swift global cluster is primarily useful when there are groups of Swift
storage and proxy nodes in different physical regions creating a
geographically-distributed cluster. From a Juju perspective these regions
typically exist in distinct models. To join the regions together the Juju
`Cross model relations`_
feature is needed. The example below assumes distinct models but Swift Global
Cluster can also work within a single model.
Basic configuration
+++++++++++++++++++
To enable the Swift global cluster feature a few extra charm configuration
options need to be set. The feature needs to be enables by setting
``enable-multi-region`` to ``True`` in the 'swift-proxy' charm. The storage nodes
need to be allocated to storage regions, this is done by the ``storage-region``
option in the 'swift-storage' charm. Usually all storage nodes in the same
geographic location will be placed in the same storage zone.
.. note::
The name of a swift storage region must be of the form 'rN'
where N in an integer. This means that the name of the storage region may
be different from the name of an OpenStack region.
Tuning configuration
++++++++++++++++++++
The 'swift-proxy' charm exposes configuration options for tuning the
deployment. The ``read-affinity`` option can be used to control what order
the regions and zones are examined when searching for an object. A common
approach would be to put the local region first on the search path for a proxy.
For instance, in the deployment example below the Swift proxy in New York is
configured to read from the New York storage nodes first. Similarly the San
Francisco proxy prefers storage nodes in San Francisco.
The ``write-affinity`` option allows nodes to be stored locally before being
eventually distributed globally. This write_affinity setting is useful only when
you do not read objects immediately after writing them.
The ``write-affinity-node-count`` option is used to further configure
``write-affinity``. This option dictates how many local storage servers will
be tried before falling back to remote ones.
For more details see the `Upstream Overview`_.
Deployment
++++++++++
This example assumes there are two datacenters, one in San Francisco (SF) and
one in New York (NY). These contain the respective Juju models called 'swift-sf'
and 'swift-ny'. 'swift-ny' contains the OpenStack region 'RegionOne' and the swift
storage region '1'. 'swift-sf' contains the OpenStack region 'RegionTwo' and the
swift storage region '2'.
Bundle overlays need to be used to encapsulate Juju cross-model relations. So
the deployment in each region consists of a bundle and an overlay.
Below is the contents of bundle ``swift-ny.yaml``
.. code-block:: yaml
series: bionic
applications:
swift-proxy-region1:
charm: cs:~gnuoy/swift-proxy-12
num_units: 1
options:
region: RegionOne
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "global-cluster"
read-affinity: "r1=100, r2=200"
write-affinity: "r1, r2"
write-affinity-node-count: '1'
openstack-origin: cloud:bionic-train
swift-storage-region1-zone1:
charm: cs:~gnuoy/swift-storage-6
num_units: 1
options:
storage-region: 1
zone: 1
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region1-zone2:
charm: cs:~gnuoy/swift-storage-6
num_units: 1
options:
storage-region: 1
zone: 2
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region1-zone3:
charm: cs:~gnuoy/swift-storage-6
num_units: 1
options:
storage-region: 1
zone: 3
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
percona-cluster:
charm: cs:~openstack-charmers-next/percona-cluster
num_units: 1
options:
dataset-size: 25%
max-connections: 1000
source: cloud:bionic-train
keystone:
expose: True
charm: cs:~openstack-charmers-next/keystone
num_units: 1
options:
openstack-origin: cloud:bionic-train
glance:
expose: True
charm: cs:~openstack-charmers-next/glance
num_units: 1
options:
openstack-origin: cloud:bionic-train
relations:
- - swift-proxy-region1:swift-storage
- swift-storage-region1-zone1:swift-storage
- - swift-proxy-region1:swift-storage
- swift-storage-region1-zone2:swift-storage
- - swift-proxy-region1:swift-storage
- swift-storage-region1-zone3:swift-storage
- - keystone:shared-db
- percona-cluster:shared-db
- - glance:shared-db
- percona-cluster:shared-db
- - glance:identity-service
- keystone:identity-service
- - swift-proxy-region1:identity-service
- keystone:identity-service
- - glance:object-store
- swift-proxy-region1:object-store
Below is the contents of the overlay bundle ``swift-ny-offers.yaml``
.. code-block:: yaml
applications:
keystone:
offers:
keystone-offer:
endpoints:
- identity-service
swift-proxy-region1:
offers:
swift-proxy-region1-offer:
endpoints:
- swift-storage
- rings-distributor
swift-storage-region1-zone1:
offers:
swift-storage-region1-zone1-offer:
endpoints:
- swift-storage
swift-storage-region1-zone2:
offers:
swift-storage-region1-zone2-offer:
endpoints:
- swift-storage
swift-storage-region1-zone3:
offers:
swift-storage-region1-zone3-offer:
endpoints:
- swift-storage
Below is the contents of bundle ``swift-sf.yaml``
.. code-block:: yaml
series: bionic
applications:
swift-proxy-region2:
charm: cs:~gnuoy/swift-proxy-12
num_units: 1
options:
region: RegionTwo
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "global-cluster"
read-affinity: "r1=100, r2=200"
write-affinity: "r1, r2"
write-affinity-node-count: '1'
openstack-origin: cloud:bionic-train
swift-storage-region2-zone1:
charm: cs:~gnuoy/swift-storage-6
num_units: 1
options:
storage-region: 2
zone: 1
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region2-zone2:
charm: cs:~gnuoy/swift-storage-6
num_units: 1
options:
storage-region: 2
zone: 2
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region2-zone3:
charm: cs:~gnuoy/swift-storage-6
num_units: 1
options:
storage-region: 2
zone: 3
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
relations:
- - swift-proxy-region2:swift-storage
- swift-storage-region2-zone1:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region2-zone2:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region2-zone3:swift-storage
Below is the contents of the overlay bundle ``swift-sf-consumer.yaml``.
.. code-block:: yaml
relations:
- - swift-proxy-region2:identity-service
- keystone:identity-service
- - swift-proxy-region2:swift-storage
- swift-storage-region1-zone1:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region1-zone2:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region1-zone3:swift-storage
- - swift-storage-region2-zone1:swift-storage
- swift-proxy-region1:swift-storage
- - swift-storage-region2-zone2:swift-storage
- swift-proxy-region1:swift-storage
- - swift-storage-region2-zone3:swift-storage
- swift-proxy-region1:swift-storage
- - swift-proxy-region2:rings-consumer
- swift-proxy-region1:rings-distributor
saas:
keystone:
url: admin/swift-ny.keystone-offer
swift-proxy-region1:
url: admin/swift-ny.swift-proxy-region1-offer
swift-storage-region1-zone1:
url: admin/swift-ny.swift-storage-region1-zone1-offer
swift-storage-region1-zone2:
url: admin/swift-ny.swift-storage-region1-zone2-offer
swift-storage-region1-zone3:
url: admin/swift-ny.swift-storage-region1-zone3-offer
In this case swift-ny.yaml must be deployed first as it contains the Juju
offers that swift-sf.yaml will consume:
.. code-block:: none
juju deploy -m swift-ny ./swift-ny.yaml --overlay ./swift-ny-offers.yaml
juju deploy -m swift-sf ./swift-sf.yaml --overlay ./swift-sf-consumer.yaml
.. LINKS
.. _Upstream Overview: https://docs.openstack.org/swift/latest/overview_global_cluster.html
.. _Cross model relations: https://jaas.ai/docs/cross-model-relations

506
deploy-guide/source/app-swift.rst Normal file
View File

@ -0,0 +1,506 @@
=======================
Appendix R: Swift usage
=======================
Overview
--------
There are two fundamental ways to deploy a Swift cluster with charms. Each
method differs in how storage nodes get assigned to storage zones.
As of the 20.02 charm release, with OpenStack Newton or later, support for a
multi-region (global) cluster, which is an extension of the single-region
scenario, is available. See the upstream documentation on `Global clusters`_
for background information.
.. warning::
Charmed Swift global cluster functionality is in a preview state and is
ready for testing. It is not production-ready.
Any Swift deployment relies upon the `swift-proxy`_ and `swift-storage`_
charms. Refer to those charms to learn about the various configuration options
used throughout this guide.
Single-region cluster
---------------------
Manual zone assignment
~~~~~~~~~~~~~~~~~~~~~~
The 'manual' method (the default) allows the cluster to be designed by
explicitly assigning a storage zone to a storage node. This zone gets
associated with the swift-storage application. This means that this method
involves multiple uniquely-named swift-storage applications.
Let file ``swift.yaml`` contain the configuration:
.. code-block:: yaml
swift-proxy:
zone-assignment: manual
replicas: 3
swift-storage-zone1:
zone: 1
block-device: /dev/sdb
swift-storage-zone2:
zone: 2
block-device: /dev/sdb
swift-storage-zone3:
zone: 3
block-device: /dev/sdb
Deploy the proxy and storage nodes:
.. code-block:: none
juju deploy --config swift.yaml swift-proxy
juju deploy --config swift.yaml swift-storage swift-storage-zone1
juju deploy --config swift.yaml swift-storage swift-storage-zone2
juju deploy --config swift.yaml swift-storage swift-storage-zone3
Add relations between the proxy node and all storage nodes:
.. code-block:: none
juju add-relation swift-proxy:swift-storage swift-storage-zone1:swift-storage
juju add-relation swift-proxy:swift-storage swift-storage-zone2:swift-storage
juju add-relation swift-proxy:swift-storage swift-storage-zone3:swift-storage
This will result in a three-zone cluster, with each zone consisting of a single
storage node, thereby satisfying the replica requirement of three.
Storage capacity is increased by adding swift-storage units to a zone. For
example, to add two storage nodes to zone '3':
.. code-block:: none
juju add-unit -n 2 swift-storage-zone3
.. note::
When scaling out ensure the candidate machines are equipped with the block
devices currently configured for the associated application.
This charm will not balance the storage ring until there are enough storage
zones to meet its minimum replica requirement, in this case three.
Auto zone assignment
~~~~~~~~~~~~~~~~~~~~
The 'auto' method automatically assigns storage zones to storage nodes. There
is only one swift-storage application and only one relation between it and the
swift-proxy application. The relation sets up the initial storage node in zone
'1' (by default). Newly-added nodes get assigned to zone '2', '3', and so on,
until the number of stated replicas is reached. Zone numbering then falls back
to '1' and the zone-assignment cycle repeats. In this way, storage nodes get
distributed evenly across zones.
Let file ``swift.yaml`` contain the configuration:
.. code-block:: yaml
swift-proxy:
zone-assignment: auto
replicas: 3
swift-storage:
block-device: /dev/sdb
Deploy the proxy node and the storage application:
.. code-block:: none
juju deploy --config swift.yaml swift-proxy
juju deploy --config swift.yaml swift-storage
The first storage node gets assigned to zone '1' when the initial relation is
added:
.. code-block:: none
juju add-relation swift-proxy:swift-storage swift-storage:swift-storage
The second and third units get assigned to zones '2' and '3', respectively,
during scale-out operations:
.. code-block:: none
juju add-unit -n 2 swift-storage
.. note::
When scaling out ensure the candidate machines are equipped with the block
devices currently configured for the associated application.
At this point the replica requirement is satisfied and the ring is balanced.
The ring is extended by continuing to add more units to the single application.
Multi-region cluster
--------------------
The previous configurations provided a single-region cluster. Generally a
region is composed of a group of nodes with high-bandwidth, low-latency links
between them. This almost always translates to the same geographical location.
Multiple such clusters can be meshed together to create a multi-region (global)
cluster. The goal is to achieve greater data resiliency by spanning zones
across geographically dispersed regions.
This section includes two configurations for implementing a Swift global
cluster: minimal and comprehensive.
A global cluster is an extension of the single cluster scenario. Refer to
the `Single-region cluster`_ section for information on essential options.
Minimal configuration
~~~~~~~~~~~~~~~~~~~~~
The proxy and storage nodes for a global cluster require extra configuration:
On the proxy node,
* option ``enable-multi-region`` is set to 'true'
* option ``region`` is defined
* option ``swift-hash`` is defined (unique value for each region)
On the storage nodes,
* option ``storage-region`` is set
The below example has two storage regions, a single zone, one storage node per
storage region, and a replica requirement of two. Manual zone assignment will
be used.
Let file ``swift.yaml`` contain the configuration:
.. code-block:: yaml
swift-proxy-region1:
region: RegionOne
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "efcf2102-b9e9-4d71-afe6-000000111111"
swift-proxy-region2:
region: RegionTwo
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "efcf2102-b9e9-4d71-afe6-111111000000"
swift-storage-region1:
storage-region: 1
zone: 1
block-device: /dev/sdb
swift-storage-region2:
storage-region: 2
zone: 1
block-device: /dev/sdb
The value of ``swift-hash`` is arbitrary. It is provided here in the form of a
UUID.
.. important::
The name of a storage region must be an integer. Here, OpenStack region
'RegionOne' corresponds to storage region '1', and OpenStack region
'RegionTwo' corresponds to storage region '2'.
Deploy in RegionOne:
.. code-block:: none
juju deploy --config swift.yaml swift-proxy swift-proxy-region1
juju deploy --config swift.yaml swift-storage swift-storage-region1
Deploy in RegionTwo:
.. code-block:: none
juju deploy --config swift.yaml swift-proxy swift-proxy-region2
juju deploy --config swift.yaml swift-storage swift-storage-region2
Add relations between swift-proxy in RegionOne and swift-storage in both
RegionOne and RegionTwo:
.. code-block:: none
juju add-relation swift-proxy-region1:swift-storage swift-storage-region1:swift-storage
juju add-relation swift-proxy-region1:swift-storage swift-storage-region2:swift-storage
Add relations between swift-proxy in RegionTwo and swift-storage in both
RegionOne and RegionTwo:
.. code-block:: none
juju add-relation swift-proxy-region2:swift-storage swift-storage-region1:swift-storage
juju add-relation swift-proxy-region2:swift-storage swift-storage-region2:swift-storage
Add a relation between swift-proxy in RegionOne and swift-proxy in RegionTwo:
.. code-block:: none
juju add-relation swift-proxy-region1:rings-distributor swift-proxy-region2:rings-consumer
More than one proxy can be deployed per OpenStack region, and each must have a
relation to every proxy in all other OpenStack regions. Only one proxy can act
as a "rings-distributor" at any one time; the proxy in RegionOne was
arbitrarily chosen.
Comprehensive configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~
A global cluster is primarily useful when there are groups of storage nodes and
proxy nodes in different physical regions, creating a
geographically-distributed cluster. These regions typically reside in distinct
Juju models, making `Cross model relations`_ a necessity. In addition, there
are configuration options available for tuning read and write behaviour. The
next example demonstrates how to implement these features and options in a
realistic scenario.
Refer to the `Minimal configuration`_ section for basic settings.
Tuning configuration
^^^^^^^^^^^^^^^^^^^^
The ``read-affinity`` option is used to control what order the regions and
zones are examined when searching for an object. A common approach would be to
put the local region first on the search path for a proxy. For instance, in the
deployment example below the Swift proxy in New York is configured to read from
the New York storage nodes first. Similarly the San Francisco proxy prefers
storage nodes in San Francisco.
The ``write-affinity`` option allows nodes to be stored locally before being
eventually distributed globally. This write_affinity setting is useful only
when you do not read objects immediately after writing them.
The ``write-affinity-node-count`` option is used to further configure
``write-affinity``. This option dictates how many local storage servers will be
tried before falling back to remote ones.
Storage regions are referred to by prepending an 'r' to their names. Hence 'r1'
refers to storage region '1'. Similarly for zones. Zone '1' is referred to by
'z1'.
For more details on these options see the upstream `Global clusters`_ document.
Deployment
^^^^^^^^^^
This example assumes there are two data centres, one in San Francisco (SF) and
one in New York (NY). These contain Juju models 'swift-sf' and 'swift-ny'
respectively. Model 'swift-ny' contains OpenStack region 'RegionOne' and
storage region '1'. Model 'swift-sf' contains OpenStack region 'RegionTwo' and
storage region '2'.
Bundle overlays are needed for encapsulating cross-model relations. So the
deployment in each OpenStack region consists of both a bundle and an overlay.
This is the contents of bundle ``swift-ny.yaml``:
.. code-block:: yaml
series: bionic
applications:
swift-proxy-region1:
charm: cs:swift-proxy
num_units: 1
options:
region: RegionOne
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "efcf2102-b9e9-4d71-afe6-000000111111"
read-affinity: "r1=100, r2=200"
write-affinity: "r1, r2"
write-affinity-node-count: '1'
openstack-origin: cloud:bionic-train
swift-storage-region1-zone1:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 1
zone: 1
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region1-zone2:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 1
zone: 2
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region1-zone3:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 1
zone: 3
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
percona-cluster:
charm: cs:~openstack-charmers-next/percona-cluster
num_units: 1
options:
dataset-size: 25%
max-connections: 1000
source: cloud:bionic-train
keystone:
expose: True
charm: cs:~openstack-charmers-next/keystone
num_units: 1
options:
openstack-origin: cloud:bionic-train
glance:
expose: True
charm: cs:~openstack-charmers-next/glance
num_units: 1
options:
openstack-origin: cloud:bionic-train
relations:
- - swift-proxy-region1:swift-storage
- swift-storage-region1-zone1:swift-storage
- - swift-proxy-region1:swift-storage
- swift-storage-region1-zone2:swift-storage
- - swift-proxy-region1:swift-storage
- swift-storage-region1-zone3:swift-storage
- - keystone:shared-db
- percona-cluster:shared-db
- - glance:shared-db
- percona-cluster:shared-db
- - glance:identity-service
- keystone:identity-service
- - swift-proxy-region1:identity-service
- keystone:identity-service
- - glance:object-store
- swift-proxy-region1:object-store
This is the contents of overlay bundle ``swift-ny-offers.yaml``:
.. code-block:: yaml
applications:
keystone:
offers:
keystone-offer:
endpoints:
- identity-service
swift-proxy-region1:
offers:
swift-proxy-region1-offer:
endpoints:
- swift-storage
- rings-distributor
swift-storage-region1-zone1:
offers:
swift-storage-region1-zone1-offer:
endpoints:
- swift-storage
swift-storage-region1-zone2:
offers:
swift-storage-region1-zone2-offer:
endpoints:
- swift-storage
swift-storage-region1-zone3:
offers:
swift-storage-region1-zone3-offer:
endpoints:
- swift-storage
This is the contents of bundle ``swift-sf.yaml``:
.. code-block:: yaml
series: bionic
applications:
swift-proxy-region2:
charm: cs:swift-proxy
num_units: 1
options:
region: RegionTwo
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "efcf2102-b9e9-4d71-afe6-111111000000"
read-affinity: "r1=100, r2=200"
write-affinity: "r1, r2"
write-affinity-node-count: '1'
openstack-origin: cloud:bionic-train
swift-storage-region2-zone1:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 2
zone: 1
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region2-zone2:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 2
zone: 2
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region2-zone3:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 2
zone: 3
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
relations:
- - swift-proxy-region2:swift-storage
- swift-storage-region2-zone1:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region2-zone2:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region2-zone3:swift-storage
This is the contents of overlay bundle ``swift-sf-consumer.yaml``:
.. code-block:: yaml
relations:
- - swift-proxy-region2:identity-service
- keystone:identity-service
- - swift-proxy-region2:swift-storage
- swift-storage-region1-zone1:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region1-zone2:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region1-zone3:swift-storage
- - swift-storage-region2-zone1:swift-storage
- swift-proxy-region1:swift-storage
- - swift-storage-region2-zone2:swift-storage
- swift-proxy-region1:swift-storage
- - swift-storage-region2-zone3:swift-storage
- swift-proxy-region1:swift-storage
- - swift-proxy-region2:rings-consumer
- swift-proxy-region1:rings-distributor
saas:
keystone:
url: admin/swift-ny.keystone-offer
swift-proxy-region1:
url: admin/swift-ny.swift-proxy-region1-offer
swift-storage-region1-zone1:
url: admin/swift-ny.swift-storage-region1-zone1-offer
swift-storage-region1-zone2:
url: admin/swift-ny.swift-storage-region1-zone2-offer
swift-storage-region1-zone3:
url: admin/swift-ny.swift-storage-region1-zone3-offer
With the current configuration, ``swift-ny.yaml`` must be deployed first as it
contains the Juju "offers" that ``swift-sf.yaml`` will consume:
.. code-block:: none
juju deploy -m swift-ny ./swift-ny.yaml --overlay ./swift-ny-offers.yaml
juju deploy -m swift-sf ./swift-sf.yaml --overlay ./swift-sf-consumer.yaml
.. LINKS
.. _Global clusters: https://docs.openstack.org/swift/latest/overview_global_cluster.html
.. _Cross model relations: https://jaas.ai/docs/cross-model-relations
.. _swift-proxy: https://jaas.ai/swift-proxy
.. _swift-storage: https://jaas.ai/swift-storage

2
deploy-guide/source/app.rst
View File

@ -22,4 +22,4 @@ Appendices
app-ovn.rst
app-managing-power-events.rst
app-manila-ganesha.rst
app-swift-gc.rst
app-swift.rst