diff --git a/doc/source/conf.py b/doc/source/conf.py index 895dbd9..f6faf25 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -52,15 +52,6 @@ project = 'Airship Project' copyright = '2019, Airship Team' author = 'Airship Team' -# The version info for the project you're documenting, acts as replacement for -# |version| and |release|, also used in various other places throughout the -# built documents. -# -# The short X.Y version. -version = '0.0.1' -# The full version, including alpha/beta/rc tags. -release = '0.0.1' - # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. # diff --git a/doc/source/develop/conventions.rst b/doc/source/develop/conventions.rst index 059e7eb..f3e3e1b 100644 --- a/doc/source/develop/conventions.rst +++ b/doc/source/develop/conventions.rst @@ -18,11 +18,13 @@ Airship Conventions =================== -Airship components conform to a minimal set of conventions to -provide for reasonable levels of consistency. + +Airship components conform to a minimal set of conventions to provide for +reasonable levels of consistency. Language -------- + While these documents are not an IETF RFC, `RFC 2119`_ provides for useful language definitions. In this spirit: @@ -36,12 +38,13 @@ Conventions and Standards ------------------------- .. toctree:: - :maxdepth: 2 + :maxdepth: 2 + :hidden: - api-conventions - code-conventions - crd-conventions - documentation-conventions - service-logging-conventions + api-conventions + code-conventions + crd-conventions + documentation-conventions + service-logging-conventions .. _RFC 2119: https://www.ietf.org/rfc/rfc2119.txt diff --git a/doc/source/develop/developers.rst b/doc/source/develop/developers.rst new file mode 100644 index 0000000..4272b5d --- /dev/null +++ b/doc/source/develop/developers.rst @@ -0,0 +1,245 @@ +.. + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Getting Started - Airship 2 Development +======================================= + +Development is underway on Airship 2: the educated evolution of Airship 1, +designed with our experience using Airship in production. Airship 2 makes the +Airship control plane ephemeral, leverages entrenched upstream projects such as +the `Cluster API`_, `Metal Kubed`_, Kustomize_, and `kubeadm`_, and embraces +Kubernetes Custom Resource Definitions (CRDs). To learn more about the Airship +2.0 evolution, see the `Airship 2 evolution blog series`_. + +Each Airship 2 project has its own development guidelines. Join the ongoing Airship 2 +development by referencing the `airshipctl`_ or the `airshipui`_ documentation. + +.. _airshipctl: https://docs.airshipit.org/airshipctl/developers.html +.. _Airship 2 evolution blog series: https://www.airshipit.org/blog/airship-blog-series-1-evolution-towards-2.0 +.. _airshipui: https://docs.airshipit.org/airshipui/developers.html +.. _Cluster API: https://github.com/kubernetes-sigs/cluster-api +.. _kubeadm: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm +.. _Kustomize: https://github.com/kubernetes-sigs/kustomize +.. _Metal Kubed: https://metal3.io + +Getting Started - Airship 1 Development +======================================= + +Airship 1 is a collection of open source tools that automate cloud provisioning +and management. Airship provides a declarative framework for defining and +managing the life cycle of open infrastructure tools and the underlying +hardware. These tools include OpenStack for virtual machines, Kubernetes for +container orchestration, and MaaS for bare metal, with planned support for +OpenStack Ironic. + +Improvements are still made Airship 1 daily, and we welcome additional +contributors. We recommend that new contributors begin by reading the +high-level architecture overview included in our `treasuremap`_ documentation. +The architectural overview introduces each Airship component, their core +responsibilities, and their integration points. + +.. _treasuremap: https://docs.airshipit.org/treasuremap + +Deep Dive +--------- + +Each Airship component is accompanied by its own documentation that provides an +extensive overview of the component. With so many components, it can be +challenging to find a starting point. + +We recommend the following: + +Try an Airship environment +-------------------------- + +Airship provides two single-node environments for demo and development purpose. + +`Airship-in-a-Bottle`_ is a set of reference documents and shell scripts that +stand up a full Airship environment with the execution of a script. + +`Airskiff`_ is a light-weight development environment bundled with a set of +deployment scripts that provides a single-node Airship environment. Airskiff +uses minikube to bootstrap Kubernetes, so it does not include Drydock, MaaS, or +Promenade. + +Additionally, we provide a reference architecture for easily deploying a +smaller, demo site. + +`Airsloop`_ is a fully-authored Airship site that can be quickly deployed as a +baremetal, demo lab. + +.. _Airship-in-a-Bottle: https://opendev.org/airship/in-a-bottle + +.. _Airskiff: https://docs.airshipit.org/treasuremap/airskiff.html + +.. _Airsloop: https://docs.airshipit.org/treasuremap/airsloop.html + +Focus on a component +-------------------- + +When starting out, focusing on one Airship component allows you to become +intricately familiar with the responsibilities of that component and understand +its function in the Airship integration. Because the components are modeled +after each other, you will also become familiar with the same patterns and +conventions that all Airship components use. + +Airship source code lives in the `OpenDev Airship namespace`_. To clone an +Airship project, execute the following, replacing `` with the name +of the Airship component you want to clone. + +.. code-block bash:: + + git clone https://opendev.org/airship/.git + +Refer to the component's documentation to get started. A list of each +component's documentation is listed below for reference: + + * `Armada`_ + * `Deckhand`_ + * `Divingbell`_ + * `Drydock`_ + * `Pegleg`_ + * `Promenade`_ + * `Shipyard`_ + +.. _OpenDev Airship namespace: https://opendev.org/airship + +.. _Armada: https://airship-armada.readthedocs.io + +.. _Deckhand: https://airship-deckhand.readthedocs.io + +.. _Divingbell: https://airship-divingbell.readthedocs.io + +.. _Drydock: https://airship-drydock.readthedocs.io + +.. _Pegleg: https://airship-pegleg.readthedocs.io + +.. _Promenade: https://airship-promenade.readthedocs.io + +.. _Shipyard: https://airship-shipyard.readthedocs.io + +Testing Changes +--------------- + +Testing of Airship changes can be accomplished several ways: + + #. Standalone, single component testing + #. Integration testing + #. Linting, unit, and functional tests/linting + +.. note:: Testing changes to charts in Airship repositories is best + accomplished using the integration method describe below. + +Standalone Testing +------------------ + +Standalone testing of Airship components, i.e. using an Airship component as a +Python project, provides the quickest feedback loop of the three methods and +allows developers to make changes on the fly. We recommend testing initial code +changes using this method to see results in real-time. + +Each Airship component written in Python has pre-requisites and guides for +running the project in a standalone capacity. Refer to the documentation listed +below. + + * `Armada`_ + * `Deckhand`_ + * `Drydock`_ + * `Pegleg`_ + * `Promenade`_ + * `Shipyard`_ + +Integration Testing +------------------- + +While each Airship component supports individual usage, Airship components +have several integration points that should be exercised after modifying +functionality. + +We maintain several environments that encompass these integration points: + + #. `Airskiff`_: Integration of Armada, Deckhand, Shipyard, and Pegleg + #. `Airship-in-a-Bottle Multinode`: Full Airship integration + +For changes that merely impact software delivery components, exercising a full +Airskiff deployment is often sufficient. Otherwise, we recommend using the +Airship-in-a-Bottle Multinode environment. + +Each environment's documentation covers the process required to build and test +component images. + +.. _Airskiff: https://docs.airshipit.org/treasuremap/airskiff.html + +.. _Airship-in-a-Bottle Multinode: http://git.openstack.org/cgit/openstack/ + airship-in-a-bottle/tree/tools/multi_nodes_gate/README.rst + +Final Checks +------------ + +Airship projects provide Makefiles to run unit, integration, and functional +tests as well as lint Python code for PEP8 compliance and Helm charts for +successful template rendering. All checks are gated by Zuul before a change can +be merged. For more information on executing these checks, refer to +project-specific documentation. + +Third party CI tools, such as Jenkins, report results on Airship-in-a-Bottle +patches. These can be exposed using the "Toggle CI" button in the bottom +left-hand page of any gerrit change. + +Pushing code +------------ + +Airship uses the `OpenDev gerrit`_ for code review. Refer to the `OpenStack +Contributing Guide`_ for a tutorial on submitting changes to Gerrit code +review. + +.. _OpenDev gerrit: https://review.opendev.org + +.. _OpenStack Contributing Guide: https://docs.openstack.org/horizon/latest/contributor/contributing.html + +Next steps +---------- + +Upon pushing a change to gerrit, Zuul continuous integration will post job +results on your patch. Refer to the job output by clicking on the job itself to +determine if further action is required. If it's not clear why a job failed, +please reach out to a team member in IRC. We are happy to assist! + +Assuming all continuous integration jobs succeed, Airship community members and +core developers will review your patch and provide feedback. Many patches are +submitted to Airship projects each day. If your patch does not receive feedback +for several days, please reach out using IRC or the Airship mailing list. + +Merging code +------------ + +Like most OpenDev projects, Airship patches require two +2 code review votes +from core members to merge. Once you have addressed all outstanding feedback, +your change will be merged. + +Beyond +------ + +Congratulations! After your first change merges, please keep up-to-date with +the team. We hold two weekly meetings for project and design discussion: + +Our weekly #airshipit IRC meeting provides an opportunity to discuss project +operations. + +Our weekly design call provides an opportunity for in-depth discussion of new +and existing Airship features. + +For more information on the times of each meeting, refer to the `Airship +wiki`_. + +.. _Airship wiki: https://wiki.openstack.org/wiki/Airship diff --git a/doc/source/develop/getting-started.rst b/doc/source/develop/getting-started.rst deleted file mode 100644 index 4674316..0000000 --- a/doc/source/develop/getting-started.rst +++ /dev/null @@ -1,162 +0,0 @@ -.. - Copyright 2017 AT&T Intellectual Property. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -.. _getting-started: - -Getting Started -=============== -Airship uses many foundational concepts that should be understood by developers -wanting to get started. This documentation attempts to provide a survey of -those topics. - -Concepts --------- - -- Containers/Docker -- RESTful APIs -- YAML -- Security - -Containers/Docker -~~~~~~~~~~~~~~~~~ -Airship is, at its core, intended to be used in a containerized fashion. -Dockerfile resources exist in each of the project repositories that are used by -the build process to generate Docker images. Images are hosted on `quay.io`_ -under ``airshipit``. - -Each main component is responsible for generating one or more images (E.g.: -Shipyard produces a Shipyard image and an `Airflow`_ image). - -When running, nearly every aspect of Airship runs as a container, and Airship -(primarily Promenade + Armada) sets up many of the other foundational -components as containers, including many `Kubernetes`_ components, `etcd`_, -`Calico`_, and `Ceph`_. - -RESTful APIs -~~~~~~~~~~~~ -Each Airship component that runs as a service provides a RESTful API. -Some :ref:`api-conventions` exist explaining the basic format of requests and -responses and required endpoints that are exposed, such as health check and -and design validation. - -YAML -~~~~ -The `YAML`_ document format is used along with `JSON Schema`_ to define the -declarative site and software design inputs to the Airship components. - -Security -~~~~~~~~ -Security is a consideration from the ground-up for Airship components. Some -technologies in this space are TLS and `Keystone`_ auth. Airship APIs are -protected by RBAC policies implemented with `oslo.policy`_ (with some -exceptions for basic health checking and listing of API versions). Keystone -middleware serves as a layer in the pipeline of service layers for each -component, providing lookup of authenticated users, resolving their roles, -which are then checked. Access enforcement is within the Airship components, -using a decorator for each API that requires limited access. - -Environment ------------ - -- Helm -- Kubernetes -- Linux - -Helm -~~~~ -Airship components are deployed into Kubernetes using `Armada`_, which in turn -uses the Tiller component of `Helm`_. Helm charts are used to generate the -Kubernetes artifacts (deployments, jobs, configmaps, etc...). - -Kubernetes -~~~~~~~~~~ -Airship is thoroughly intertwined with Kubernetes: - -- Airship depends on Kubernetes as the orchestrator of the containers that make - up the platform. -- Airship sets up a single node Kubernetes instance during the `Promenade`_ - genesis process, with the necessary configuration to become the seed of a - resilient Kubernetes cluster during later stages of Airship. -- Airship's components run as containers inside the Kubernetes cluster. - -Linux -~~~~~ -Airship is targeted to a Linux platform. There are significant elements of -Airship that use shell scripts to drive processes. - -Coding ------- -Further information is available in :ref:`code-conventions`. - -Airship is primarily a combination of Python 3 and shell scripting. There are -several Python libraries that are used in common across many components: - -- Falcon: A service framework providing the API endpoints. -- uWSGI: The service container. -- oslo_config: Provides per-deployment, configuration file configurability. -- oslo_policy: Provides RBAC support for API endpoints (and more). -- Requests: A framework for making HTTP requests and receiving responses. -- Click: A CLI framework used to provide component-level Command Line - Interfaces. - -Each component also brings in their own dependencies as needed. - -Database(s) ------------ -Several of the Airship components require some data persistence. Some data -persistence is achieved by utilizing Kubernetes provided mechanisms, and the -Keystone software uses a MariaDB instance, but most is accomplished using a -containerized PostgreSQL database. - -Interaction with PostgreSQL uses the following: - -- SQLAlchemy: A python library providing most of the needed database - functionality. -- Alembic: Version management for database schemas and data. -- oslo_db: An OpenStack layer providing additional functionality over - SQLAlchemy. - -Testing -------- - -- Unit -- Functional -- Integration - -Unit and functional tests are used in the gating of changes before merging -code. Unit tests utilize combinations of `pytest`_ and `stestr`_. Functional -tests utilize `Gabbi`_. These tools are not exclusive of others, but are the -primary tools being used for unit and functional tests. - -Integration testing is orchestrated in the merge gates, and uses various means -of testing. - -.. _Airflow: https://airflow.apache.org/ -.. _Armada: https://airship-armada.readthedocs.io/ -.. _Calico: https://www.projectcalico.org/calico-networking-for-kubernetes/ -.. _Ceph: https://ceph.com -.. _etcd: https://coreos.com/etcd/ -.. _Gabbi: https://github.com/cdent/gabbi -.. _Helm: https://github.com/kubernetes/helm -.. _JSON Schema: http://json-schema.org/ -.. _Keystone: https://docs.openstack.org/keystone/latest/ -.. _Kubernetes: https://kubernetes.io/ -.. _oslo.policy: https://docs.openstack.org/oslo.policy/latest/ -.. _Promenade: https://airship-promenade.readthedocs.io/ -.. _pytest: https://docs.pytest.org/en/latest/ -.. _quay.io: https://quay.io/organization/airshipit -.. _stestr: https://github.com/mtreinish/stestr -.. _YAML: http://yaml.org/ diff --git a/doc/source/develop/index.rst b/doc/source/develop/index.rst index eba0520..d657a7c 100644 --- a/doc/source/develop/index.rst +++ b/doc/source/develop/index.rst @@ -14,9 +14,6 @@ Develop Airship =============== -Welcome -------- - Thank you for your interest in Airship. Our community is eager to help you contribute to the success of our project and welcome you as a member of our community! @@ -29,242 +26,8 @@ Welcome aboard! .. _Airship mailing list: http://lists.airshipit.org .. _Slack workspace: http://airshipit.org/slack -Getting Started - Airship 2 Development ---------------------------------------- - -Development is underway on Airship 2: the educated evolution of Airship 1, -designed with our experience using Airship in production. Airship 2 makes the -Airship control plane ephemeral, leverages entrenched upstream projects such as -the `Cluster API`_, `Metal Kubed`_, Kustomize_, and `kubeadm`_, and embraces -Kubernetes Custom Resource Definitions (CRDs). To learn more about the Airship -2.0 evolution, see the `Airship 2 evolution blog series`_. - -Each Airship 2 project has its own development guidelines. Join the ongoing Airship 2 -development by referencing the `airshipctl`_ or the `airshipui`_ documentation. - -.. _airshipctl: https://docs.airshipit.org/airshipctl/developers.html -.. _Airship 2 evolution blog series: https://www.airshipit.org/blog/airship-blog-series-1-evolution-towards-2.0 -.. _airshipui: https://docs.airshipit.org/airshipui/developers.html -.. _Cluster API: https://github.com/kubernetes-sigs/cluster-api -.. _kubeadm: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm -.. _Kustomize: https://github.com/kubernetes-sigs/kustomize -.. _Metal Kubed: https://metal3.io - -Getting Started - Airship 1 Development ---------------------------------------- - -Airship 1 is a collection of open source tools that automate cloud provisioning -and management. Airship provides a declarative framework for defining and -managing the life cycle of open infrastructure tools and the underlying -hardware. These tools include OpenStack for virtual machines, Kubernetes for -container orchestration, and MaaS for bare metal, with planned support for -OpenStack Ironic. - -Improvements are still made Airship 1 daily, and we welcome additional -contributors. We recommend that new contributors begin by reading the -high-level architecture overview included in our `treasuremap`_ documentation. -The architectural overview introduces each Airship component, their core -responsibilities, and their integration points. - -.. _treasuremap: https://docs.airshipit.org/treasuremap - -Deep Dive -~~~~~~~~~ - -Each Airship component is accompanied by its own documentation that provides an -extensive overview of the component. With so many components, it can be -challenging to find a starting point. - -We recommend the following: - -Try an Airship environment -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Airship provides two single-node environments for demo and development purpose. - -`Airship-in-a-Bottle`_ is a set of reference documents and shell scripts that -stand up a full Airship environment with the execution of a script. - -`Airskiff`_ is a light-weight development environment bundled with a set of -deployment scripts that provides a single-node Airship environment. Airskiff -uses minikube to bootstrap Kubernetes, so it does not include Drydock, MaaS, or -Promenade. - -Additionally, we provide a reference architecture for easily deploying a -smaller, demo site. - -`Airsloop`_ is a fully-authored Airship site that can be quickly deployed as a -baremetal, demo lab. - -.. _Airship-in-a-Bottle: https://opendev.org/airship/in-a-bottle - -.. _Airskiff: https://docs.airshipit.org/treasuremap/airskiff.html - -.. _Airsloop: https://docs.airshipit.org/treasuremap/airsloop.html - -Focus on a component -~~~~~~~~~~~~~~~~~~~~ - -When starting out, focusing on one Airship component allows you to become -intricately familiar with the responsibilities of that component and understand -its function in the Airship integration. Because the components are modeled -after each other, you will also become familiar with the same patterns and -conventions that all Airship components use. - -Airship source code lives in the `OpenDev Airship namespace`_. To clone an -Airship project, execute the following, replacing `` with the name -of the Airship component you want to clone. - -.. code-block bash:: - - git clone https://opendev.org/airship/.git - -Refer to the component's documentation to get started. A list of each -component's documentation is listed below for reference: - - * `Armada`_ - * `Deckhand`_ - * `Divingbell`_ - * `Drydock`_ - * `Pegleg`_ - * `Promenade`_ - * `Shipyard`_ - -.. _OpenDev Airship namespace: https://opendev.org/airship - -.. _Armada: https://airship-armada.readthedocs.io - -.. _Deckhand: https://airship-deckhand.readthedocs.io - -.. _Divingbell: https://airship-divingbell.readthedocs.io - -.. _Drydock: https://airship-drydock.readthedocs.io - -.. _Pegleg: https://airship-pegleg.readthedocs.io - -.. _Promenade: https://airship-promenade.readthedocs.io - -.. _Shipyard: https://airship-shipyard.readthedocs.io - -Testing Changes -~~~~~~~~~~~~~~~ - -Testing of Airship changes can be accomplished several ways: - - #. Standalone, single component testing - #. Integration testing - #. Linting, unit, and functional tests/linting - -.. note:: Testing changes to charts in Airship repositories is best - accomplished using the integration method describe below. - -Standalone Testing -~~~~~~~~~~~~~~~~~~ - -Standalone testing of Airship components, i.e. using an Airship component as a -Python project, provides the quickest feedback loop of the three methods and -allows developers to make changes on the fly. We recommend testing initial code -changes using this method to see results in real-time. - -Each Airship component written in Python has pre-requisites and guides for -running the project in a standalone capacity. Refer to the documentation listed -below. - - * `Armada`_ - * `Deckhand`_ - * `Drydock`_ - * `Pegleg`_ - * `Promenade`_ - * `Shipyard`_ - -Integration Testing -~~~~~~~~~~~~~~~~~~~ - -While each Airship component supports individual usage, Airship components -have several integration points that should be exercised after modifying -functionality. - -We maintain several environments that encompass these integration points: - - #. `Airskiff`_: Integration of Armada, Deckhand, Shipyard, and Pegleg - #. `Airship-in-a-Bottle Multinode`: Full Airship integration - -For changes that merely impact software delivery components, exercising a full -Airskiff deployment is often sufficient. Otherwise, we recommend using the -Airship-in-a-Bottle Multinode environment. - -Each environment's documentation covers the process required to build and test -component images. - -.. _Airskiff: https://docs.airshipit.org/treasuremap/airskiff.html - -.. _Airship-in-a-Bottle Multinode: http://git.openstack.org/cgit/openstack/ - airship-in-a-bottle/tree/tools/multi_nodes_gate/README.rst - -Final Checks -~~~~~~~~~~~~ - -Airship projects provide Makefiles to run unit, integration, and functional -tests as well as lint Python code for PEP8 compliance and Helm charts for -successful template rendering. All checks are gated by Zuul before a change can -be merged. For more information on executing these checks, refer to -project-specific documentation. - -Third party CI tools, such as Jenkins, report results on Airship-in-a-Bottle -patches. These can be exposed using the "Toggle CI" button in the bottom -left-hand page of any gerrit change. - -Pushing code -~~~~~~~~~~~~ - -Airship uses the `OpenDev gerrit`_ for code review. Refer to the `OpenStack -Contributing Guide`_ for a tutorial on submitting changes to Gerrit code -review. - -.. _OpenDev gerrit: https://review.opendev.org - -.. _OpenStack Contributing Guide: https://docs.openstack.org/horizon/latest/contributor/contributing.html - -Next steps -~~~~~~~~~~ - -Upon pushing a change to gerrit, Zuul continuous integration will post job -results on your patch. Refer to the job output by clicking on the job itself to -determine if further action is required. If it's not clear why a job failed, -please reach out to a team member in IRC. We are happy to assist! - -Assuming all continuous integration jobs succeed, Airship community members and -core developers will review your patch and provide feedback. Many patches are -submitted to Airship projects each day. If your patch does not receive feedback -for several days, please reach out using IRC or the Airship mailing list. - -Merging code -~~~~~~~~~~~~ - -Like most OpenDev projects, Airship patches require two +2 code review votes -from core members to merge. Once you have addressed all outstanding feedback, -your change will be merged. - -Beyond -~~~~~~ - -Congratulations! After your first change merges, please keep up-to-date with -the team. We hold two weekly meetings for project and design discussion: - -Our weekly #airshipit IRC meeting provides an opportunity to discuss project -operations. - -Our weekly design call provides an opportunity for in-depth discussion of new -and existing Airship features. - -For more information on the times of each meeting, refer to the `Airship -wiki`_. - -.. _Airship wiki: https://wiki.openstack.org/wiki/Airship - .. toctree:: - :hidden: - :maxdepth: 3 + :hidden: - getting-started - conventions + developers + conventions diff --git a/doc/source/learn/index.rst b/doc/source/learn/index.rst index 13807c7..a6dc924 100644 --- a/doc/source/learn/index.rst +++ b/doc/source/learn/index.rst @@ -17,4 +17,4 @@ Learn About Airship .. toctree:: :maxdepth: 3 - ../security/guide + vulnerabilities diff --git a/doc/source/security/vulnerabilities.rst b/doc/source/learn/vulnerabilities.rst similarity index 100% rename from doc/source/security/vulnerabilities.rst rename to doc/source/learn/vulnerabilities.rst diff --git a/doc/source/security/guide.rst b/doc/source/security/guide.rst deleted file mode 100644 index 72697cb..0000000 --- a/doc/source/security/guide.rst +++ /dev/null @@ -1,57 +0,0 @@ -.. - Copyright 2018 AT&T Intellectual Property. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -.. _security_guide: - -Airship Security Guide -====================== - -An undercloud environment deployed via Airship crosses many security domains. This -guide explains many of the security concerns that have been reviewed and considered -by the Airship developers. Because Airship is a highly configuration-driven platform, -there is some onus on the end-user to make good decisions with their configuration. - -Layout and Nomenclature ------------------------ - -Each topic in the security guide will provide some overview for scope of that topic -and then provide a list of tactical security items. For each item two statuses will -be listed as well as the project scope. - - * Project Scope: Which Airship projects address this security item. - * Solution: The solution is how this security concern is addressed in the platform - - * Remediated: The item is solved for automatically - * Configurable: The item is based on configuration. Guidance will be provided. - * Mitigated: The item currently mitigated while a permanent remediation is in progress. - * Pending: Addressing the item is in-progress - - * Audit: Auditing the item provides for ongoing monitoring to ensure there is no regression - - * Testing: The item is tested for in an automated test pipeline during development - * Validation: The item is reported on by a validation framework after a site deployment - * Pending: Auditing is in-progress - -Airship Security Topics ------------------------ - -.. toctree:: - :maxdepth: 1 - - template - haproxy - ubuntu - vulnerabilities diff --git a/doc/source/security/haproxy.rst b/doc/source/security/haproxy.rst deleted file mode 100644 index 7185020..0000000 --- a/doc/source/security/haproxy.rst +++ /dev/null @@ -1,55 +0,0 @@ -.. - Copyright 2018 AT&T Intellectual Property. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -.. _haproxy_security_guide: - -HAProxy Security Guide -====================== - -Updated: 13-AUG-2018 - -This guide covers configurations for HAProxy. Specifically, in ``mode tcp``. - -.. contents:: :depth: 2 - -Security Item List ------------------- - -TCP Mode -^^^^^^^^ - -The instance will work in pure TCP mode. A full-duplex connection will be -established between clients and servers, and no layer 7 examination will be -performed. This is the default mode. It should be used for TLS. - -Max Connections -^^^^^^^^^^^^^^^ - -Set ``maxconn`` in ``global`` to a reasonable level. HAProxy will queue -requests beyond that value. - -Set Headers -^^^^^^^^^^^ -"set-header" does the same as "add-header" except that the header name is first -removed if it existed. This is useful when passing security information to the -server, where the header must not be manipulated by external users. Note that -the new value is computed before the removal so it is possible to concatenate a -value to an existing header. - -References ----------- - -`HAProxy Configuration Guide `_ diff --git a/doc/source/security/template.rst b/doc/source/security/template.rst deleted file mode 100644 index 6d8f9c9..0000000 --- a/doc/source/security/template.rst +++ /dev/null @@ -1,70 +0,0 @@ -.. - Copyright 2018 AT&T Intellectual Property. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -.. _template_security_guide: - -Template for a Security Guide Topic -=================================== - -Updated: 1-AUG-2018 - -An overview of the scope of this topic. - -.. contents:: :depth: 2 - -Security Item List ------------------- - -Sensitive Data Security -^^^^^^^^^^^^^^^^^^^^^^^ - -Sensitive data should be encrypted at-rest. - - * Project Scope: Deckhand - * Solution *Remediated*: The ``storagePolicy`` metadata determines if Deckhand will persist - document data encrypted. - * Audit: *Testing*: Pipeline test checks that documents with a ``storagePolicy: encrypted`` - are not persisted to the database with an intact ``data`` section. - -Sensitive data should be encrypted in-transit. - - * Project Scope: Shipyard, Deckhand - * Solution *Pending*: Shipyard and Deckhand API endpoints should support - TLS. See data_security_. - * Audit: *Pending*: Expect to validate post-deployment that endpoints all support TLS - -Configuration Guidance ----------------------- - -For items that require guidance on configuration that impact a security item -please list an item here. Use RST anchors and links to link the security item solution -status to this guidance. - -Temporary Mitigation Status ---------------------------- - -.. _data_security: - -Data Security In-Transit -^^^^^^^^^^^^^^^^^^^^^^^^ - -Current work to support Deckhand enabling TLS termination, Shipyard enabling self-signing -CAs and Barbican supporting TLS termination. - -References ----------- - -`Transport Layer Security (TLS) `_ diff --git a/doc/source/security/ubuntu.rst b/doc/source/security/ubuntu.rst deleted file mode 100644 index aaf8737..0000000 --- a/doc/source/security/ubuntu.rst +++ /dev/null @@ -1,247 +0,0 @@ -.. - Copyright 2018 AT&T Intellectual Property. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -.. _ubuntu_security_guide: - -Canonical Ubuntu/MAAS Security Guide -==================================== - -Updated: 6-AUG-2018 - -This guide covers the configuration of MAAS to run securely and to deploy -secure installations of Ubuntu 16.04.x. Some items are above and beyond MAAS -when MAAS does not offer the functionality needed to fully secure a -newly provisioned server. - -.. contents:: :depth: 2 - -Security Item List ------------------- - -Filesystem Permissions -^^^^^^^^^^^^^^^^^^^^^^ - -Many files on the filesystem can contain sensitive data that can hasten a malignant -attack on a host. Ensure the below files have appropriate ownership and permissions - -================================== ========= ========= =============== - Filesystem Path Owner Group Permissions -================================== ========= ========= =============== -``/boot/System.map-*`` root root ``0600`` -``/etc/shadow`` root shadow ``0640`` -``/etc/gshadow`` root shadow ``0640`` -``/etc/passwwd`` root root ``0644`` -``/etc/group`` root root ``0644`` -``/var/log/kern.log`` root root ``0640`` -``/var/log/auth.log`` root root ``0640`` -``/var/log/syslog`` root root ``0640`` -================================== ========= ========= =============== - - - Project Scope: Drydock - - Solution *Configurable*: A bootaction will be run to enforce this on first boot - - Audit: *Pending*: This will be verified on an ongoing basis via a Sonobuoy plugin - -Filesystem Partitioning -^^^^^^^^^^^^^^^^^^^^^^^ - -The mounts ``/tmp``, ``/var``, ``/var/log``, ``/var/log/audit`` and ``/home`` should be -individual file systems. - - - Project Scope: Drydock - - Solution *Configurable*: Drydock supports user designed partitioning, see - `Filesystem Configuration`_. - - Audit: *Testing*: The Airship testing pipeline will validate that nodes are partitioned - as described in the site definition. - -Filesystem Hardening -^^^^^^^^^^^^^^^^^^^^ - -Disallow symlinks and hardlinks to files not owned by the user. Set ``fs.protected_symlinks`` and -``fs.protected_hardlinks`` to ``1``. - - - Project Scope: Diving Bell - - Solution *Configurable*: Diving Bell overrides will enforce this kernel tunable. By default - MAAS deploys nodes in compliance. - - Audit: *Pending*: This will be verified on an ongoing basis via a Sonobuoy plugin. - -Execution Environment Hardening -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The kernel tunable ``fs.suid_dumpable`` must be set to ``0`` and there must be a hard limit -disabling core dumps (``hard core 0``) - - - Project Scope: DivingBell, Drydock - - Solution *Configurable*: Diving Bell overrides will enforce this kernel tunable, by default - MAAS deploys nodes with ``fs.suid_dumpable = 2``. A boot action will put in place the hard - limit. - - Audit: *Pending*: This will be verified on an ongoing basis via a Sonobuoy plugin - -Randomizing stack space can make it harder to exploit buffer overflow vulnerabilities. Enable -the kernel tunable ``kernel.randomize_va_space = 2``. - - - Project Scope: DivingBell - - Solution *Configurable*: Diving Bell overrides will enforce this kernel tunable, by default - MAAS deploys nodes in compliance. - - Audit: *Pending*: This will be verified on an ongoing basis via a Sonobuoy plugin - -Mandatory Access Control -^^^^^^^^^^^^^^^^^^^^^^^^ - -Put in place the approved default AppArmor profile and ensure that Docker is configured -to use it. - - - Project Scope: Drydock, Promenade - - Solution *Configurable*: A bootaction will put in place the default AppArmor profile. Promenade - will deploy a Docker configuration to enforce the default policy. - - Audit: *Pending*: This will be verified on an ongoing basis via a Sonobuoy plugin probing - ``/proc//attr/current``. - -Put in place an approved AppArmor profile to be used by containers that will manipulate the -on-host AppArmor profiles. This allows an init container in Pods to put customized AppArmor -profile in place and load them. - - - Project Scope: Drydock - - Solution *Configurable*: A bootaction will put in place the profile-manager AppArmor profile and - load it on each boot. - - Audit: *Pending*: The availability of this profile will be verified by a Sonobuoy plugin. - -.. IMPORTANT:: - - All other AppArmor profiles must be delivered and loaded by an init container in the Pod - that requires them. The Pod must also be decorated with the appropriate annotation to specify - the custom profile. - -System Monitoring -^^^^^^^^^^^^^^^^^ - -Run `rsyslogd` to log events. - - - Project Scope: Drydock - - Solution *Remediated*: MAAS installs rsyslog by default. - - Audit: *Pending*: This will be verified on an ongoing basis via a Sonobuoy plugin. - -Run a monitor for logging kernel audit events such as auditd. - - - Project Scope: Non-Airship - - Solution *Remediated*: The `Sysdig Falco `_ will be used - and - - Audit: *Pending*: This will be verified on an ongoing basis via a Sonobuoy plugin. - -Watch the watchers. Ensure that monitoring services are up and responsive. - - - Project Scope: Non-Airship - - Solution *Remediated*: Nagios will monitor host services and Kubernetes resources - - Audit: *Validation*: Internal corporate systems track Nagios heartbeats to ensure Nagios is responsive - -Blacklisted Services -^^^^^^^^^^^^^^^^^^^^ - -The below services are deprecated and should not be enabled or installed on hosts. - -================ ==================== - Service Ubuntu Package -================ ==================== - telnet telnetd - inet telnet inetutils-telnetd - SSL telnet telnetd-ssl - NIS nis - NTP date ntpdate -================ ==================== - - - Project Scope: Drydock - - Solution *Configurable*: A boot action will be used to enforce this on first boot. - - Audit: *Pending*: This will be verified on an ongoing basis via Sonobuoy plugin. - -Required System Services -^^^^^^^^^^^^^^^^^^^^^^^^ - -``cron`` and ``ntpd`` **must** be installed and enabled on all hosts. Only administrative -accounts should have access to cron. ``ntpd -q`` should show time synchronization is active. - - - Project Scope: Drydock - - Solution *Remediated*: A MAAS deployed node runs cron and configured ntpd by default. - - Audit: *Pending*: This will be verified on an ongoing basis via Sonobuoy plugin. - -System Service Configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If ``sshd`` is enabled, ensure it is securely configured: - - - **Must** only support protocol version 2 (``Protocol 2``) - - **Must** disallow root SSH logins (``PermitRootLogin no``) - - **Must** disallow empty passwords (``PermitEmptyPasswords no``) - - **Should** set a idle timeout interval (``ClientAliveInterval 600`` and ``ClientAliveCountMax 0``) - - - Project Scope: Drydock - - Solution *Configurable*: A boot action will install an explicit configuration file - - Audit: *Pending*: This will be verified on an ongoing basis via Sonobuoy plugin. - -Network Security -^^^^^^^^^^^^^^^^ - -.. IMPORTANT:: - - Calico network policies will be used to secure host-level network access. Nothing will - be orchestrated outside of Calico to enforce host-level network policy. - -Secure the transport of traffic between nodes and MAAS/Drydock during node deployment. - - - Project Scope: Drydock, MAAS - - Solution *Pending*: The Drydock and MAAS charts will be updated to include an Ingress - port utilizing TLS 1.2 and a publicly signed certificate. Also the service will enable - TLS on the pod IP. - - Audit: *Testing*: The testing pipeline will validate the deployment is using TLS to - access the Drydock and MAAS APIs. - -.. DANGER:: - - Some traffic, such as iPXE, DHCP, TFTP, will utilize node ports and is not encrypted. This - is not configurable. However, this traffic traverses the private PXE network. - -Secure Accounts -^^^^^^^^^^^^^^^ - -Enforce a minimum password length of 8 characters - - - Project Scope: Drydock - - Solution *Configurable*: A boot action will update ``/etc/pam.d/common-password`` to specify ``minlen=8`` for ``pam_unix.so``. - - Audit: *Pending*: This will be verified on an ongoing basis via Sonobuoy plugin. - -Configuration Guidance ----------------------- - -Filesystem Configuration -^^^^^^^^^^^^^^^^^^^^^^^^ - -The filesystem partitioning strategy should be sure to protect the ability for the host to -log critical information, both for security and reliability. The log data should not risk -filling up the root filesystem (``/``) and non-critical log data should not risk crowding out -critical log data. If you are shipping log data to a remote store, the latter concern is -less critical. Because Airship nodes are built to **ONLY** run Kubernetes, isolating filesystems -such as ``/home`` is not as critical since there is no direct user access and applications -are running in a containerized environment. - -Temporary Mitigation Status ---------------------------- - - -References ----------- - - * `OpenSCAP for Ubuntu 16.04 `_ - * `Ubuntu 16.04 Server Guide `_ - * `Canonical MAAS 2.3 TLS `_ - * `Canonical MAAS 2.4 TLS `_