Move UCP to Airship documentation
Updates the Shipyard documentation to reflect changes related to moving to OpenStack Airship. Adds some additional information. Change-Id: I32a3aa24ce268cbc925a512aa2ae54eaf95acd2a
This commit is contained in:
Bryan Strassner
parent
b87569abe8
commit
c1539ae95d
@ -28,8 +28,7 @@ functionality in Shipyard.
|
||||
|
||||
Standards used by the API
|
||||
-------------------------
|
||||
See `UCP API
|
||||
conventions <https://github.com/att-comdev/ucp-integration/blob/master/docs>`__
|
||||
See `API Conventions`_
|
||||
|
||||
Notes on examples
|
||||
-----------------
|
||||
@ -258,20 +257,19 @@ Entity Structure
|
||||
^^^^^^^^^^^^^^^^
|
||||
The response will be the list of validations from all downstream systems that
|
||||
perform validation during the commit process. The structure will match the
|
||||
error response object described in the `UCP API
|
||||
conventions <https://github.com/att-comdev/ucp-integration/blob/master/docs>`__
|
||||
and will be an aggregation of each UCP component’s responses.
|
||||
error response object described in the `API Conventions`_ and will be an
|
||||
aggregation of each validating component’s responses.
|
||||
|
||||
POST /v1.0/commitconfigdocs
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Synchronous. Performs the commit of the Shipyard Buffer to the Committed
|
||||
Documents. This invokes each of the UCP components to examine the Shipyard
|
||||
Buffer version of the configuration documents and aggregate the responses.
|
||||
While performing this commit, further POSTing of configdocs, or other commits
|
||||
may not be invoked (Shipyard will block those requests with a 409 response). If
|
||||
there are any failures to validate, the Shipyard Buffer and Committed Documents
|
||||
will remain unchanged. If successful, the Shipyard Buffer will be cleared, and
|
||||
the Committed documents will be updated.
|
||||
Documents. This invokes each of the validating components to examine the
|
||||
Shipyard Buffer version of the configuration documents and aggregate the
|
||||
responses. While performing this commit, further POSTing of configdocs, or
|
||||
other commits may not be invoked (Shipyard will block those requests with a 409
|
||||
response). If there are any failures to validate, the Shipyard Buffer and
|
||||
Committed Documents will remain unchanged. If successful, the Shipyard Buffer
|
||||
will be cleared, and the Committed documents will be updated.
|
||||
|
||||
.. note::
|
||||
|
||||
@ -1033,3 +1031,6 @@ Example
|
||||
[2018-04-11 07:30:43,892] {{cli.py:374}} INFO - Running on host airflow-worker-0.airflow-worker-discovery.ucp.svc.cluster.local
|
||||
[2018-04-11 07:30:43,945] {{base_task_runner.py:98}} INFO - Subtask: [2018-04-11 07:30:43,944] {{python_operator.py:90}} INFO - Done. Returned value was: None
|
||||
[2018-04-11 07:30:43,992] {{base_task_runner.py:98}} INFO - Subtask: """)
|
||||
|
||||
|
||||
.. _API Conventions: https://airshipit.readthedocs.io/en/latest/api-conventions.html
|
||||
|
||||
@ -34,12 +34,9 @@ configuration documents. Steps, conceptually:
|
||||
Prevents concurrent site modifications by conflicting
|
||||
actions/workflows.
|
||||
#. Preflight checks
|
||||
Ensures all UCP components are in a responsive state.
|
||||
#. Get design version
|
||||
Uses Deckhand to discover the latest committed version of design for
|
||||
the site.
|
||||
Ensures all Airship components are in a responsive state.
|
||||
#. Validate design
|
||||
Asks each involved UCP component to validate the design. This ensures
|
||||
Asks each involved Airship component to validate the design. This ensures
|
||||
that the previously committed design is valid at the present time.
|
||||
#. Drydock build
|
||||
Orchestrates the Drydock component to configure hardware and the
|
||||
@ -92,12 +89,11 @@ Schemas
|
||||
DeploymentConfiguration_ schema - Provides for validation of the
|
||||
deployment-configuration documents
|
||||
|
||||
Documents
|
||||
~~~~~~~~~
|
||||
deployment-configuration - Allows for specification of configurable options
|
||||
used by the site deployment related workflows, including the timeouts used for
|
||||
various steps, and the name of the armada manifest that will be used during the
|
||||
deployment/update.
|
||||
Deployment Configuration
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Allows for specification of configurable options used by the site deployment
|
||||
related workflows, including the timeouts used for various steps, and the name
|
||||
of the armada manifest that will be used during the deployment/update.
|
||||
|
||||
A `sample deployment-configuration`_ shows a completely specified example.
|
||||
|
||||
@ -114,8 +110,8 @@ Supported values:
|
||||
|
||||
- deployment_strategy:
|
||||
|
||||
Not currently used, placeholder for specifying how server deployments
|
||||
should be grouped. (e.g. all-at-once, by-rack, by-label)
|
||||
The name of the deployment strategy document to be used. There is a default
|
||||
deployment strategy that is used if this field is not present.
|
||||
|
||||
- deploy_interval:
|
||||
|
||||
@ -201,6 +197,20 @@ Supported values:
|
||||
The name of the Armada manifest document that the workflow will use during
|
||||
site deployment activities. e.g.:'full-site'
|
||||
|
||||
.. _DeploymentConfiguration: https://github.com/att-comdev/shipyard/blob/master/shipyard_airflow/schemas/deploymentConfiguration.yaml
|
||||
.. _`sample deployment-configuration`: https://github.com/att-comdev/shipyard/blob/master/tests/unit/yaml_samples/deploymentConfiguration_full_valid.yaml
|
||||
.. _`Default configuration values`: https://github.com/att-comdev/shipyard/blob/a88a5cf15a81de32707fee49d7a626908a8c8f38/shipyard_airflow/plugins/deployment_configuration_operator.py#L47
|
||||
Deployment Strategy
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
The deployment strategy document is optionally specified in the Deployment
|
||||
Configuration and provides a way to group, sequence, and test the deployments
|
||||
of groups of hosts deployed using `Drydock`_. The `deployment strategy design`_
|
||||
provides details for the structures and usage of the deployment strategy.
|
||||
A `sample deployment-strategy`_ shows one possible strategy, in the context of
|
||||
the Shipyard unit testing.
|
||||
The `DeploymentStrategy`_ schema is a more formal definition of this document.
|
||||
|
||||
.. _`Default configuration values`: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/plugins/deployment_configuration_operator.py
|
||||
.. _DeploymentConfiguration: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/schemas/deploymentConfiguration.yaml
|
||||
.. _DeploymentStrategy: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/schemas/deploymentStrategy.yaml
|
||||
.. _`deployment strategy design`: https://airshipit.readthedocs.io/en/latest/blueprints/deployment-grouping-baremetal.html
|
||||
.. _Drydock: https://git.airshipit.org/cgit/airship-drydock
|
||||
.. _`sample deployment-configuration`: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/tests/unit/yaml_samples/deploymentConfiguration_full_valid.yaml
|
||||
.. _`sample deployment-strategy`: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/tests/unit/yaml_samples/deploymentStrategy_full_valid.yaml
|
||||
|
||||
@ -97,8 +97,7 @@ Setup
|
||||
Server Components
|
||||
~~~~~~~~~~~~~~~~~
|
||||
Use of the Shipyard client requires that a working installation of the Shipyard
|
||||
API is available. This may be accomplished by using the basic-ucp_ setup.
|
||||
Detailed instructions are not part of this guide.
|
||||
API is available. See :ref:`shipyard_deployment_guide`
|
||||
|
||||
Local Environment
|
||||
~~~~~~~~~~~~~~~~~
|
||||
@ -153,6 +152,8 @@ response::
|
||||
the target Shipyard API/Keystone environment.
|
||||
- The ``shipyard`` and ``password`` values are the insecure values used by
|
||||
default if not overridden by the installation of Shipyard.
|
||||
- The dev_minimal manifest deployment from Airship-in-a-bottle referenced in
|
||||
the deployment guide provides a set of credentials that can be used.
|
||||
|
||||
``Configure hosts file, if necessary``::
|
||||
|
||||
@ -196,9 +197,9 @@ The output will resemble the following::
|
||||
|
||||
Use Case: Ingest Site Design
|
||||
----------------------------
|
||||
Shipyard serves as the entrypoint for a deployment of the Undercloud Platform
|
||||
(UCP). One can imagine the following activities representing part of the
|
||||
lifecycle of a group of servers that the UCP would serve as the control plane:
|
||||
Shipyard serves as the entrypoint for a deployment of Airship. One can imagine
|
||||
the following activities representing part of the lifecycle of a group of
|
||||
servers for which Airship would serve as the control plane:
|
||||
|
||||
Definition
|
||||
A group of servers making up a ``site`` has been identified. Designs covering
|
||||
@ -207,10 +208,10 @@ Definition
|
||||
Preparation
|
||||
The site is assembled, racking, and wiring is completed, and the hardware is
|
||||
readied for operation. The ``Genesis Node`` is preinstalled with an
|
||||
(Ubuntu 16.04) image. The Promenade_ genesis process similar to the
|
||||
basic-ucp_ process is configured on and then run on the ``Genesis Node``.
|
||||
(Ubuntu 16.04) image. Airship is deployed; See
|
||||
:ref:`shipyard_deployment_guide`
|
||||
|
||||
At this point, the UCP is ready for use. This is the when the Shipyard API
|
||||
At this point, Airship is ready for use. This is the when the Shipyard API
|
||||
is available for use.
|
||||
|
||||
Load Configuration Documents
|
||||
@ -218,7 +219,7 @@ Load Configuration Documents
|
||||
with Shipyard, perhaps by using the CLI. The operator loads ``configdocs``
|
||||
which are a product of the definition step. These ``configdocs`` are
|
||||
declarative set of YAML documents using a format compatible with
|
||||
Deckhand_ and containing information usable by the other UCP components.
|
||||
Deckhand_ and containing information usable by the other Airship components.
|
||||
|
||||
The interaction with Shipyard could happen as follows::
|
||||
|
||||
@ -252,7 +253,7 @@ of an action::
|
||||
|
||||
$ shipyard commit configdocs
|
||||
|
||||
During this command, the other UCP components are contacted to validate the
|
||||
During this command, the other Airship components are contacted to validate the
|
||||
designs in Deckhand. If the validations are not successful, Shipyard will not
|
||||
mark the revision as committed.
|
||||
|
||||
@ -271,7 +272,6 @@ mark the revision as committed.
|
||||
site-specific collection that combines with the other two collections to
|
||||
fully define the site.
|
||||
|
||||
|
||||
Use Case: Deploy Site
|
||||
---------------------
|
||||
Continuing the lifecycle steps from the Ingest Site Design use case, the
|
||||
@ -314,14 +314,14 @@ action, the operator can query the status and results of the action::
|
||||
...
|
||||
|
||||
More information is returned than shown here - for sake of abbreviation. The
|
||||
process of maintenance is very similar to the process of deploying a site.
|
||||
process of maintenance (update_site) is very similar to the process of
|
||||
deploying a site.
|
||||
|
||||
|
||||
.. _api-client: https://github.com/att-comdev/shipyard/tree/master/shipyard_client/api_client
|
||||
.. _Armada: https://github.com/att-comdev/armada
|
||||
.. _basic-ucp: https://github.com/att-comdev/ucp-integration/tree/master/manifests/basic_ucp
|
||||
.. _Deckhand: http://deckhand.readthedocs.io/en/latest/
|
||||
.. _Divingbell: https://github.com/att-comdev/divingbell
|
||||
.. _Drydock: https://github.com/att-comdev/drydock
|
||||
.. _api-client: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_client
|
||||
.. _Armada: https://git.airshipit.org/cgit/airship-armada
|
||||
.. _Deckhand: https://git.airshipit.org/cgit/airship-deckhand
|
||||
.. _Divingbell: https://git.airshipit.org/cgit/airship-divingbell
|
||||
.. _Drydock: https://git.airshipit.org/cgit/airship-drydock
|
||||
.. _Keystone: https://developer.openstack.org/api-ref/identity/index.html
|
||||
.. _Promenade: https://github.com/att-comdev/promenade
|
||||
.. _Promenade: https://git.airshipit.org/cgit/airship-promenade
|
||||
|
||||
@ -23,109 +23,29 @@ Deployment Guide
|
||||
Shipyard is still under active development and this guide will evolve along
|
||||
the way
|
||||
|
||||
The current deployment makes use of the `ucp-integration`_ repository to set up
|
||||
the underlaying Kubernetes infrastructure, Ceph and UCP components. This approach
|
||||
sets up an 'All-In-One' UCP environment that allows developers to bring up Shipyard
|
||||
and the rest of the UCP components on a single Ubuntu 16.04 Virtual Machine.
|
||||
Deployment
|
||||
----------
|
||||
|
||||
.. note::
|
||||
The minimum recommended size of the VM is 4 vCPUs, 16GB of RAM with 64GB disk space
|
||||
The current deployment makes use of the `airship-in-a-bottle`_ project to
|
||||
set up the underlaying Kubernetes infrastructure, container networking
|
||||
(Calico), disk provisioner (Ceph or NFS), and Airship components that are used
|
||||
by Shipyard.
|
||||
|
||||
The `dev_minimal`_ manifest is the recommended manifest. Please see the
|
||||
README.txt that exists in that manifest's directory.
|
||||
|
||||
Pre-Deployment Preparations
|
||||
---------------------------
|
||||
|
||||
#. Set up ``etc/hosts`` on a freshly installed Ubuntu 16.04 Virtual Machine with the
|
||||
IP and hostname of the VM::
|
||||
|
||||
HOST_IFACE=$(ip route | grep "^default" | head -1 | awk '{ print $5 }')
|
||||
LOCAL_IP=$(ip addr | awk "/inet/ && /${HOST_IFACE}/{sub(/\/.*$/,\"\",\$2); print \$2}")
|
||||
cat << EOF | sudo tee -a /etc/hosts
|
||||
${LOCAL_IP} $(hostname)
|
||||
EOF
|
||||
|
||||
#. Clone the `ucp-integration`_ repository
|
||||
|
||||
|
||||
Production Deployment
|
||||
---------------------
|
||||
|
||||
We will use this approach if we are not making any changes to Shipyard and would like to bring up
|
||||
the UCP environment as it is
|
||||
|
||||
#. Switch to root user after performing the steps in the *Pre-Deployment Preparations* section::
|
||||
|
||||
sudo -i
|
||||
cd /home/ubuntu/ucp-integration/manifests/basic_ucp/
|
||||
|
||||
#. Export the variables that are unique to the environment. For instance, we can do the following
|
||||
to update the environment variables for a VM with hostname **node1** that is assigned an IP of
|
||||
**30.30.30.4** on interface **ens3** (network **30.30.30.0/24**)::
|
||||
|
||||
export GENESIS_NODE_NAME=node1
|
||||
export CEPH_CLUSTER_NET=30.30.30.0/24
|
||||
export CEPH_PUBLIC_NET=30.30.30.0/24
|
||||
export GENESIS_NODE_IP=30.30.30.4
|
||||
export NODE_NET_IFACE=ens3
|
||||
|
||||
#. Start the UCP deployment::
|
||||
|
||||
./deploy_ucp.sh
|
||||
|
||||
|
||||
Dev Environment Deployment
|
||||
--------------------------
|
||||
|
||||
We will use this approach if we want to bring up a dev environment that allows us to test our own
|
||||
dags and operators in Airflow. Changes to Shipyard API/CLI will require a rebuild of the Shipyard
|
||||
images with the updates. We will need to reference to the custom image in our environment variables
|
||||
so that it does not point to the image in the Master branch.
|
||||
|
||||
#. Create the following directories on the target host machine::
|
||||
|
||||
mkdir -p /var/tmp/airflow/dags
|
||||
mkdir -p /var/tmp/airflow/plugins
|
||||
mkdir -p /var/tmp/airflow/logs
|
||||
|
||||
#. Copy the `rest_api_plugin`_ into the newly created plugins directory, i.e. ``/var/tmp/airflow/plugins``
|
||||
so that it can be loaded by Airflow during startup. **Note** that other custom operators
|
||||
should be added to the directory as required.
|
||||
|
||||
**Note** that custom dags should be added into the newly created dags directory, i.e. ``/var/tmp/airflow/dags``
|
||||
|
||||
#. Switch to root user after performing Step 1 and 2::
|
||||
|
||||
sudo -i
|
||||
cd /home/ubuntu/ucp-integration/manifests/basic_ucp/
|
||||
|
||||
#. Export the variables that are unique to the environment. For instance, we can do the below
|
||||
to update the environment variables for a VM with hostname **node1** that is assigned an IP
|
||||
of **30.30.30.4** on interface **ens3** (network **30.30.30.0/24**).
|
||||
|
||||
Update image references in the environment variables if we want to test a new Shipyard and
|
||||
Airflow image, e.g. image v0.1.0 as a result of code changes::
|
||||
|
||||
export SHIPYARD_PROD_DEPLOY=false
|
||||
export GENESIS_NODE_NAME=node1
|
||||
export CEPH_CLUSTER_NET=30.30.30.0/24
|
||||
export CEPH_PUBLIC_NET=30.30.30.0/24
|
||||
export GENESIS_NODE_IP=30.30.30.4
|
||||
export NODE_NET_IFACE=ens3
|
||||
export SHIPYARD_IMAGE="attcomdev/shipyard:v0.1.0"
|
||||
export AIRFLOW_IMAGE="attcomdev/airflow:v0.1.0"
|
||||
|
||||
#. Start the UCP deployment::
|
||||
|
||||
./deploy_ucp.sh
|
||||
This approach sets up an 'All-In-One' Airship environment that allows
|
||||
developers to bring up Shipyard and the rest of the Airship components on a
|
||||
single Ubuntu Virtual Machine.
|
||||
|
||||
The deployment is fully automated and can take a while to complete (it can take
|
||||
30 minutes to an hour for a full deployment to complete)
|
||||
|
||||
Post Deployment
|
||||
---------------
|
||||
|
||||
#. The deployment is fully automated and can take a while to complete (it can take 30 minutes
|
||||
to an hour for a full deployment to complete)
|
||||
|
||||
#. The environment should resemble the following after executing the required steps::
|
||||
#. The environment should resemble the following after executing the required
|
||||
steps::
|
||||
|
||||
# sudo kubectl get pods -n ucp
|
||||
NAME READY STATUS RESTARTS AGE
|
||||
@ -182,5 +102,6 @@ Post Deployment
|
||||
ucp-ucp-memcached 1 Fri Dec 1 10:02:44 2017 DEPLOYED memcached-0.1.0 ucp
|
||||
|
||||
|
||||
.. _ucp-integration: https://github.com/att-comdev/ucp-integration
|
||||
.. _rest_api_plugin: https://github.com/att-comdev/shipyard/blob/master/shipyard_airflow/plugins/rest_api_plugin.py
|
||||
.. _airship-in-a-bottle: https://git.airshipit.org/cgit/airship-in-a-bottle
|
||||
.. _dev_minimal: https://git.airshipit.org/cgit/airship-in-a-bottle/tree/manifests/dev_minimal
|
||||
.. _rest_api_plugin: https://git.airshipit.org/cgit/airship-shipyard/tree/src/bin/shipyard_airflow/shipyard_airflow/plugins/rest_api_plugin.py
|
||||
|
||||
@ -19,7 +19,7 @@ Welcome to Shipyard's documentation!
|
||||
====================================
|
||||
|
||||
Shipyard is a directed acyclic graph controller for Kubernetes and OpenStack
|
||||
control plane life-cycle management.
|
||||
control plane life-cycle management, and is part of the `Airship`_ platform.
|
||||
|
||||
User's Guide
|
||||
============
|
||||
@ -37,3 +37,5 @@ Shipyard Configuration Guide
|
||||
CLI
|
||||
client_user_guide
|
||||
deployment_guide
|
||||
|
||||
.. _Airship: https://airshipit.org
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user