Merge "Add instructions for upgrading OpenStack"
This commit is contained in:
Zuul
Gerrit Code Review
commit
dfe4f7c37e
266
deploy-guide/source/app-upgrade-openstack.rst
Normal file
266
deploy-guide/source/app-upgrade-openstack.rst
Normal file
@ -0,0 +1,266 @@
|
|||||||
|
Appendix B: OpenStack Upgrades
|
||||||
|
==============================
|
||||||
|
|
||||||
|
Overview
|
||||||
|
--------
|
||||||
|
|
||||||
|
This document outlines approaches to upgrading OpenStack using the charms.
|
||||||
|
|
||||||
|
Definitions and Terms
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
Charm Upgrade
|
||||||
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This is an upgrade of the charm software which is used to deploy and manage
|
||||||
|
OpenStack. This will include charms that manage applications which are not
|
||||||
|
part of the OpenStack project such as Rabbitmq and MySQL.
|
||||||
|
|
||||||
|
OpenStack Upgrade
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This is an upgrade of the Openstack software (packages) that are installed
|
||||||
|
and managed by the charms.
|
||||||
|
|
||||||
|
Ubuntu Server Package Upgrade
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This is an upgrade of the Ubuntu packages on the server that are not part of
|
||||||
|
the OpenStack project such as kernel modules, QEMU binaries, KVM kernel module
|
||||||
|
etc.
|
||||||
|
|
||||||
|
Ubuntu Release Upgrade
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This is an upgrade from one Ubuntu release to the next.
|
||||||
|
|
||||||
|
Testing
|
||||||
|
-------
|
||||||
|
|
||||||
|
All procedures outlined below should be tested in a non-production environment
|
||||||
|
first.
|
||||||
|
|
||||||
|
Skipping Releases
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
The charms support triggering a multi-release upgrade and will handle stepping
|
||||||
|
through the upgrade. However, given that some OpenStack projects (including
|
||||||
|
Nova) only support there being a single release difference between components
|
||||||
|
this is not recommended.
|
||||||
|
|
||||||
|
1. Charm Upgrades
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
All charms should be upgraded to the latest stable release before performing
|
||||||
|
an OpenStack upgrade. It is recommended to upgrade the Keystone charm first.
|
||||||
|
The order of upgrading subsequent charms is usually not important but
|
||||||
|
check the release notes for each release to ensure there are no
|
||||||
|
special requirements.
|
||||||
|
|
||||||
|
To upgrade a charm that was deployed from the charm store:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
juju upgrade-charm <charm name>
|
||||||
|
|
||||||
|
|
||||||
|
The progress of the upgrade can be monitored by checking the workload status
|
||||||
|
of the charm which can been with **juju status**. Once the upgrade is complete
|
||||||
|
the charm status should contain the message 'Unit is ready'. The version of
|
||||||
|
the deployed software can also been seen from **juju status**:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
juju status
|
||||||
|
...
|
||||||
|
App Version Status Scale Charm Store Rev OS Notes
|
||||||
|
keystone 11.0.3 active 1 keystone local 0 ubuntu
|
||||||
|
...
|
||||||
|
|
||||||
|
This shows that the deployed version of keystone is 11.0.3 (Ocata)
|
||||||
|
|
||||||
|
If the Juju controller is resource constrained it may be beneficial to do the
|
||||||
|
charm upgrades in series rather than in parallel. After each charm upgrade
|
||||||
|
check for any unforeseen errors reported in **juju status** before proceeding.
|
||||||
|
|
||||||
|
2. Pre-Upgrade Tasks
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
2.1 Release Notes
|
||||||
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Check the release notes for the charm releases for any special instructions.
|
||||||
|
|
||||||
|
2.2 Check current deployment
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Check for any charm errors in **juju status**. If a monitor is in use like
|
||||||
|
Nagios then make sure any alerts have been cleared before proceeding. This is
|
||||||
|
to ensure that alerts after the upgrade are not pre-existing problems.
|
||||||
|
|
||||||
|
Also ensure that the current charms must not do not contain any customisations
|
||||||
|
since that is not supported and they will be overwritten by the upgrade.
|
||||||
|
|
||||||
|
2.3 Database row archiving
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
During the upgrade, database migrations will be run. These can be significantly
|
||||||
|
sped up by archiving any stale data (such as deleted instances). To perform the
|
||||||
|
archive of nova data run the nova-cloud-controller action:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
juju run-action nova-cloud-controller/0 archive-data
|
||||||
|
|
||||||
|
This action may need to be run multiple times until the action output reports
|
||||||
|
'Nothing was archived'
|
||||||
|
|
||||||
|
|
||||||
|
3. Upgrade Order
|
||||||
|
----------------
|
||||||
|
|
||||||
|
The charms are grouped together below. The ordering of upgrade within a group
|
||||||
|
does not matter but all the charms in each group should be upgraded before
|
||||||
|
moving on to the next group. Any release note guidance overrides the order
|
||||||
|
listed here.
|
||||||
|
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| Group | Charm Name | Charm Type |
|
||||||
|
+=======+=======================+===============+
|
||||||
|
| 1 | keystone | Core Identity |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 2 | ceph | Storage |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 2 | ceph-mon | Storage |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 2 | ceph-osd | Storage |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 2 | ceph-fs | Storage |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 2 | ceph-radosgw | Storage |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 2 | swift-proxy | Storage |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 2 | swift-storage | Storage |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | aodh | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | barbican | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | ceilometer | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | cinder | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | designate | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | designate-bind | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | glance | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | gnocchi | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | heat | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | manila | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | manila-generic | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | neutron-api | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | neutron-gateway | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | nova-cloud-controller | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | odl-controller | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 3 | openstack-dashboard | Control Plane |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
| 4 | nova-compute | Compute |
|
||||||
|
+-------+-----------------------+---------------+
|
||||||
|
|
||||||
|
4. Performing The Upgrade
|
||||||
|
-------------------------
|
||||||
|
|
||||||
|
If the service to be upgraded is in a highly-available cluster then the best
|
||||||
|
way to minimise service interruption is to follow the "HA with pause/resume"
|
||||||
|
instructions below. If there are multiple units of the service but they are
|
||||||
|
not clustered then follow the "Action managed" instructions. Finally, if there
|
||||||
|
is a single unit then follow "Application one-shot".
|
||||||
|
|
||||||
|
Some parts of the upgrade, like database migrations, only need to run once per
|
||||||
|
application and these tasks are handled by the lead unit. It is advisable that
|
||||||
|
these tasks are run first (this is not applicable for one-shot deployments). To
|
||||||
|
achieve this run the upgrade on the lead unit first. To check which unit is the
|
||||||
|
lead unit either check which unit has a '*' next to it in **juju status** or
|
||||||
|
run:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
juju run --application application-name is-leader
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
HA with pause/resume
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
The majority of charms support pause and resume actions. These actions can be
|
||||||
|
used to place units of a charm into a state where maintenance operations can
|
||||||
|
be carried out. Using these actions along with action managed upgrades allows
|
||||||
|
a charm to be removed from service, upgraded and returned to service.
|
||||||
|
|
||||||
|
|
||||||
|
For example to upgrade a three node nova-cloud-controller service from Ocata
|
||||||
|
to Pike where nova-cloud-controller/2 is the leader:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
juju config nova-cloud-controller action-managed-upgrade=True
|
||||||
|
juju config nova-cloud-controller openstack-origin='cloud:xenial-pike'
|
||||||
|
juju run-action nova-cloud-controller/2 --wait pause
|
||||||
|
juju run-action nova-cloud-controller/2 --wait openstack-upgrade
|
||||||
|
juju run-action nova-cloud-controller/2 --wait resume
|
||||||
|
juju run-action nova-cloud-controller/1 --wait pause
|
||||||
|
juju run-action nova-cloud-controller/1 --wait openstack-upgrade
|
||||||
|
juju run-action nova-cloud-controller/1 --wait resume
|
||||||
|
juju run-action nova-cloud-controller/0 --wait pause
|
||||||
|
juju run-action nova-cloud-controller/0 --wait openstack-upgrade
|
||||||
|
juju run-action nova-cloud-controller/0 --wait resume
|
||||||
|
|
||||||
|
Action managed
|
||||||
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
If there are multiple units of an application then each unit can be upgraded
|
||||||
|
one at a time using Juju actions. This allows for rolling upgrades. To use
|
||||||
|
this feature the charm configuration option action-managed-upgrade must be set
|
||||||
|
to True.
|
||||||
|
|
||||||
|
For example to upgrade a three node keystone service from Ocata to Pike where
|
||||||
|
keystone/1 is the leader:
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
juju config keystone action-managed-upgrade=True
|
||||||
|
juju config keystone openstack-origin='cloud:xenial-pike'
|
||||||
|
juju run-action keystone/1 --wait openstack-upgrade
|
||||||
|
juju run-action keystone/0 --wait openstack-upgrade
|
||||||
|
juju run-action keystone/2 --wait openstack-upgrade
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Application one-shot
|
||||||
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
This is the simplest and quickest way to perform the upgrade. Using this method
|
||||||
|
will cause all the units in the application to be upgraded at the same time.
|
||||||
|
This is likely to cause a service outage while the upgrade completes. If there
|
||||||
|
is only one unit in the application then this is the only option.
|
||||||
|
|
||||||
|
.. code:: bash
|
||||||
|
|
||||||
|
juju config keystone openstack-origin='cloud:xenial-pike'
|
||||||
|
|
||||||
|
|
||||||
|
5. Post-Upgrade Tasks
|
||||||
|
---------------------
|
||||||
|
|
||||||
|
Check **juju status** and any monitoring solution for errors.
|
||||||
@ -6,3 +6,4 @@ Appendices
|
|||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
|
|
||||||
app-ceph-migration.rst
|
app-ceph-migration.rst
|
||||||
|
app-upgrade-openstack.rst
|
||||||
|
|||||||
@ -1,4 +1,4 @@
|
|||||||
sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3 # BSD
|
sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3 # BSD
|
||||||
oslosphinx>=2.5.0,!=3.4.0 # Apache-2.0
|
oslosphinx>=2.5.0,!=3.4.0 # Apache-2.0
|
||||||
pbr>=1.6 # Apache-2.0
|
pbr>=1.6 # Apache-2.0
|
||||||
openstackdocstheme>=1.5.0 # Apache-2.0
|
openstackdocstheme>=1.5.0,<1.18.0 # Apache-2.0
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user