From 440015d591f13057a09ed968fe5686c9a9343583 Mon Sep 17 00:00:00 2001
From: Purnendu Ghosh <purnendu@gmail.com>
Date: Thu, 29 Nov 2018 17:25:03 +0530
Subject: [PATCH] Added base documentation
---
README.md | 9 ++--
doc/source/getting_started.rst | 74 +++++++++++++++++++++++---
doc/source/index.rst | 5 +-
doc/source/tugboat.rst | 97 ++++++++++++++++++++++++++++++++++
4 files changed, 174 insertions(+), 11 deletions(-)
create mode 100644 doc/source/tugboat.rst
diff --git a/README.md b/README.md
index b0c3c3e..4080d62 100644
--- a/README.md
+++ b/README.md
@@ -23,7 +23,10 @@ all the information that will be rendered to generate Airship site
manifests. This optional step will help the deployment engineer to
modify any data if required.
-Basic Usage
------------
+Getting Started
+---------------
+For more detailed installation and setup information, please refer to the
+`Getting Started`_ guide.
-TODO
+
+.. _`Getting Started`: ./doc/source/getting_started.rst
diff --git a/doc/source/getting_started.rst b/doc/source/getting_started.rst
index 1e502f4..892e476 100644
--- a/doc/source/getting_started.rst
+++ b/doc/source/getting_started.rst
@@ -19,7 +19,7 @@ Getting Started
===============
What is Spyglass?
-----------------
+-----------------
Spyglass is a data extraction tool which can interface with
different input data sources to generate site manifest YAML files.
@@ -28,6 +28,7 @@ for a site deployment. These site manifest YAML files generated
by spyglass will be saved in a Git repository, from where Pegleg
can access and aggregate them. This aggregated file can then be
fed to Shipyard for site deployment / updates.
+Reference: https://review.openstack.org/#/c/605227
Architecture
------------
@@ -62,6 +63,65 @@ Architecture
--
+Supported Features
+------------------
+1. Tugboat Plugin: Supports extracting site data from Excel files and
+ then generate site manifests for sitetype:airship-seaworthy.
+ Find more documentation for Tugboat, see :ref:`tugboatinfo`.
+
+2. Remote Data Source Plugin: Supports extracting site data from a REST
+ endpoint.
+
+3. YAML Editor for Intermediary YAML: Support runtime editing of missing
+ site parameters
+
+Future Work
+-----------
+1) Schema based manifest generation instead of Jinja2 templates. It shall
+be possible to cleanly transition to this schema based generation keeping a unique
+mapping between schema and generated manifests. Currently this is managed by
+considering a mapping of j2 templates with schemas and site type.
+
+List of Generated Site Manifests:
+---------------------------------
+The spyglass uses the plugin data source to generate the following site
+manifests:
+
+- site-definition.yaml
+- profile/
+- profile/region.yaml
+- baremetal/
+- baremetal/nodes.yaml
+- networks/
+- networks/common_addresses.yaml
+- networks/control-plane-addresses.yaml
+- networks/physical/
+- networks/physical/networks.yaml
+- software/
+- software/charts/
+- software/charts/osh/
+- software/charts/osh/openstack-tenant-ceph/
+- software/charts/osh/openstack-tenant-ceph/ceph-client.yaml
+- software/charts/ucp/
+- software/charts/ucp/divingbell/
+- software/charts/ucp/divingbell/divingbell.yaml
+- software/config/
+- software/config/corridor.yaml
+- software/config/common-software-config.yaml
+- deployment/
+- deployment/deployment-strategy.yaml
+- pki/
+- pki/kubelet-node-pkicatalog.yaml
+
+Spyglass maintains corresponding J2 templates for these files
+and then those are processed with site information obtained
+from plugin data source.
+
+In some cases, the site might require additional site
+manifests containing static information independent of the
+plugin data received. In such cases one can just place the
+corresponding J2 templates in the appropriate folder.
+
Basic Usage
-----------
@@ -72,7 +132,7 @@ Before using Spyglass you must:
.. code-block:: console
- git clone https://github.com/att-comdev/tugboat/tree/spyglass
+ git clone https://github.com/att-comdev/spyglass
2. Install the required packages in spyglass:
@@ -100,8 +160,7 @@ Options:
excel spec
-idir, --intermediary_dir PATH The path where intermediary file needs to be
generated
- -e, --edit_intermediary / -nedit, --no_edit_intermediary
- Flag to let user edit intermediary
+ -e, --edit_intermediary Flag to let user edit intermediary
-m, --generate_manifests Generate manifests from the generated
intermediary file
-mdir, --manifest_dir PATH The path where manifest files needs to be
@@ -115,6 +174,9 @@ Options:
20]
--help Show this message and exit.
+--------
+Examples
+--------
1. Running Spyglass with Remote Data Source Plugin
@@ -125,8 +187,8 @@ spyglass -mg --type formation -f <URL> -u <user_id> -p <password> -d <site_confi
spyglass -mg --type tugboat -x <Excel File> -e <Excel Spec> -d <Site Config> -s <Region> --template_dir=<j2 template dir>
for example:
-spyglass -mg -t tugboat -x SiteDesignSpec_v0.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=<j2 template dir>
+spyglass -mg -t tugboat -x SiteDesignSpec_v1.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=<j2 template dir>
+
Where sample 'excel_spec_upstream.yaml', 'SiteDesignSpec_v0.1.xlsx'
'site_config.yaml' and J2 templates can be found under 'spyglass/examples'
folder
-
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 4c43fa2..b0ec2b9 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -14,9 +14,9 @@
License for the specific language governing permissions and limitations
under the License.
-=====================
+======================
Spyglass Documentation
-=====================
+======================
Overview
--------
@@ -32,3 +32,4 @@ fed to Shipyard for site deployment / updates.
:maxdepth: 2
getting_started
+ tugboat
diff --git a/doc/source/tugboat.rst b/doc/source/tugboat.rst
new file mode 100644
index 0000000..3236128
--- /dev/null
+++ b/doc/source/tugboat.rst
@@ -0,0 +1,97 @@
+
+ Copyright 2018 AT&T Intellectual Property.
+ All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License"); you may
+ not use this file except in compliance with the License. You may obtain
+ a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+ License for the specific language governing permissions and limitations
+ under the License.
+
+.. _tugboatinfo:
+
+=======
+Tugboat
+=======
+
+What is Tugboat?
+----------------
+
+Tugboat is a Spyglass plugin to generate airship-seaworthy site manifest files
+from an excel based engineering spec. The plugin is configured with an Excel
+sheet and its corresponding excel specification as inputs. Spyglass uses this
+plugin to construct an intermediary yaml which is processed further using J2
+templates to generate site manifests.
+
+Excel specification
+-------------------
+Excel Spec is like an index to the Excel sheet to look for the data to be
+collected by the tool. Excel Spec Sample specifies all the details that
+need to be filled by the Deployment Engineer.
+
+Below is the definition for each key in the Excel spec
+
+::
+
+
+ ipmi_sheet_name - name of the sheet from where IPMI and host profile information is to be read
+ start_row - row number from where the IPMI and host profile information starts
+ end_row - row number from where the IPMI and host profile information ends
+ hostname_col - column number where the hostnames are to be read from
+ ipmi_address_col - column number from where the ipmi addresses are to be read
+ host_profile_col - column number from where the host profiles are to be read
+ ipmi_gateway_col - column number from where the ipmi gateways are to be read
+ private_ip_sheet - name of the sheet which has the private IP information
+ net_type_col - column number from where the network type is to be read
+ vlan_col - column number from where the network vlan is to be read
+ vlan_start_row - row number from where the vlan information starts
+ vlan_end_row - row number from where the vlan information ends
+ net_start_row - row number from where the network information starts
+ net_end_row - row number from where the network information ends
+ net_col - column number where the IP ranges for network is to be read
+ net_vlan_col - column number where the vlan information is present in the pod wise network section
+ public_ip_sheet - name of the sheet which has the public IP information
+ oam_vlan_col - column number from where the OAM vlan information is to be read from
+ oam_ip_row - row number from where the OAM network information is to be read from
+ oam_ip_col - column number from where the OAM network information is to be read from
+ oob_net_row - row number which has the OOB network subnet ranges
+ oob_net_start_col - column number from where the OOB network ranges start
+ oob_net_end_col - column number from where the OOB network ranges end
+ ingress_ip_row - row number from where the Ingress network information is to be read from
+ dns_ntp_ldap_sheet - name of the sheet which has the DNS, NTP and LDAP information
+ login_domain_row - row number which has the ldap login domain
+ ldap_col - column number which has the all ldap related information
+ global_group - row number which has the ldap group information
+ ldap_search_url_row - row number which has the ldap url
+ ntp_row - row number which has the ntp information
+ ntp_col - column number which has the ntp information
+ dns_row - row number which has the dns information
+ dns_col - column number which has the dns information
+ domain_row - row number which has the domain information
+ domain_col - column number which has the domain information
+ location_sheet - name of the sheet which has the location information
+ column - column number which has all the information
+ corridor_row - row number which has the corridor information
+ site_name_row - row number which has the site name
+ state_name_row - row number which has the state name
+ country_name_row - row number which has the country name
+ clli_name_row - row number which has CLLI information
+
+Example: Tugboat Plugin Usage
+-----------------------------
+1. Required Input(Refer to 'spyglass/examples' folder to get these inputs)
+ a) Excel File: SiteDesignSpec_v0.1.xlsx
+ b) Excel Spec: excel_spec_upstream.yaml
+ c) Site Config: site_config.yaml
+ d) Template_dir: '../examples/templates'
+ c) Site name: airship-seaworthy
+
+2. Spyglass CLI Command:
+ spyglass -mg -t tugboat -x SiteDesignSpec_v0.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=<relative path to '../examples/templates'
+