Added base documentation

README.md

@@ -23,7 +23,10 @@ all the information that will be rendered to generate Airship site
 manifests. This optional step will help the deployment engineer to
 modify any data if required.
 
-Basic Usage
+Getting Started
------------
+---------------
+For more detailed installation and setup information, please refer to the
+`Getting Started`_ guide.
 
-TODO
+
+.. _`Getting Started`: ./doc/source/getting_started.rst

doc/source/getting_started.rst

@@ -19,7 +19,7 @@ Getting Started
 ===============
 
 What is Spyglass?
-----------------
+-----------------
 
 Spyglass is a data extraction tool which can interface with
 different input data sources to generate site manifest YAML files.
@@ -28,6 +28,7 @@ for a site deployment. These site manifest YAML files generated
 by spyglass will be saved in a Git repository, from where Pegleg
 can access and aggregate them. This aggregated file can then be
 fed to Shipyard for site deployment / updates.
+Reference: https://review.openstack.org/#/c/605227
 
 Architecture
 ------------
@@ -62,6 +63,65 @@ Architecture
 
 --
 
+Supported Features
+------------------
+1. Tugboat Plugin: Supports extracting site data from Excel files and
+   then generate site manifests for sitetype:airship-seaworthy.
+   Find more documentation for Tugboat, see :ref:`tugboatinfo`.
+
+2. Remote Data Source Plugin: Supports extracting site data from a REST
+   endpoint.
+
+3. YAML Editor for Intermediary YAML: Support runtime editing of missing
+   site parameters
+
+Future Work
+-----------
+1) Schema based manifest generation instead of Jinja2 templates. It shall
+be possible to cleanly transition to this schema based generation keeping a unique
+mapping between schema and generated manifests. Currently this is managed by
+considering a mapping of j2 templates with schemas and site type.
+
+List of Generated Site Manifests:
+---------------------------------
+The spyglass uses the plugin data source to generate the following site
+manifests:
+
+- site-definition.yaml
+- profile/
+- profile/region.yaml
+- baremetal/
+- baremetal/nodes.yaml
+- networks/
+- networks/common_addresses.yaml
+- networks/control-plane-addresses.yaml
+- networks/physical/
+- networks/physical/networks.yaml
+- software/
+- software/charts/
+- software/charts/osh/
+- software/charts/osh/openstack-tenant-ceph/
+- software/charts/osh/openstack-tenant-ceph/ceph-client.yaml
+- software/charts/ucp/
+- software/charts/ucp/divingbell/
+- software/charts/ucp/divingbell/divingbell.yaml
+- software/config/
+- software/config/corridor.yaml
+- software/config/common-software-config.yaml
+- deployment/
+- deployment/deployment-strategy.yaml
+- pki/
+- pki/kubelet-node-pkicatalog.yaml
+
+Spyglass maintains corresponding J2 templates for these files
+and then those are processed with site information obtained
+from plugin data source.
+
+In some cases, the site might require additional site
+manifests containing static information independent of the
+plugin data received. In such cases one can just place the
+corresponding J2 templates in the appropriate folder.
+
 Basic Usage
 -----------
 
@@ -72,7 +132,7 @@ Before using Spyglass you must:
 
    .. code-block:: console
 
-    git clone https://github.com/att-comdev/tugboat/tree/spyglass
+    git clone https://github.com/att-comdev/spyglass
 
 2. Install the required packages in spyglass:
 
@@ -100,8 +160,7 @@ Options:
                                   excel spec
   -idir, --intermediary_dir PATH  The path where intermediary file needs to be
                                   generated
-  -e, --edit_intermediary / -nedit, --no_edit_intermediary
+  -e, --edit_intermediary         Flag to let user edit intermediary
-                                  Flag to let user edit intermediary
   -m, --generate_manifests        Generate manifests from the generated
                                   intermediary file
   -mdir, --manifest_dir PATH      The path where manifest files needs to be
@@ -115,6 +174,9 @@ Options:
                                   20]
   --help                          Show this message and exit.
 
+--------
+Examples
+--------
 
 1. Running Spyglass with  Remote Data Source Plugin
 
@@ -125,8 +187,8 @@ spyglass -mg --type formation -f <URL> -u <user_id> -p <password> -d <site_confi
 spyglass -mg --type tugboat -x <Excel File> -e <Excel Spec> -d <Site Config> -s <Region> --template_dir=<j2 template dir>
 
 for example:
-spyglass -mg -t tugboat -x SiteDesignSpec_v0.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=<j2 template dir>
+spyglass -mg -t tugboat -x SiteDesignSpec_v1.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=<j2 template dir>
+
 Where sample 'excel_spec_upstream.yaml', 'SiteDesignSpec_v0.1.xlsx'
 'site_config.yaml' and J2 templates can be found under 'spyglass/examples'
 folder
-

doc/source/index.rst

@@ -14,9 +14,9 @@
       License for the specific language governing permissions and limitations
       under the License.
 
-=====================
+======================
 Spyglass Documentation
-=====================
+======================
 
 Overview
 --------
@@ -32,3 +32,4 @@ fed to Shipyard for site deployment / updates.
    :maxdepth: 2
 
    getting_started
+   tugboat

doc/source/tugboat.rst

@@ -0,0 +1,97 @@
+
+      Copyright 2018 AT&T Intellectual Property.
+      All Rights Reserved.
+
+      Licensed under the Apache License, Version 2.0 (the "License"); you may
+      not use this file except in compliance with the License. You may obtain
+      a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+      Unless required by applicable law or agreed to in writing, software
+      distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+      WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+      License for the specific language governing permissions and limitations
+      under the License.
+
+.. _tugboatinfo:
+
+=======
+Tugboat
+=======
+
+What is Tugboat?
+----------------
+
+Tugboat is a Spyglass plugin to generate airship-seaworthy site manifest files
+from an excel based engineering spec. The plugin is configured with an Excel
+sheet and its corresponding excel specification as inputs. Spyglass uses this
+plugin to construct an intermediary yaml which is processed further using J2
+templates to generate site manifests.
+
+Excel specification
+-------------------
+Excel Spec is like an index to the Excel sheet to look for the data to be
+collected by the tool. Excel Spec Sample specifies all the details that
+need to be filled by the Deployment Engineer.
+
+Below is the definition for each key in the Excel spec
+
+::
+
+
+     ipmi_sheet_name - name of the sheet from where IPMI and host profile information is to be read
+     start_row - row number from where the IPMI and host profile information starts
+     end_row - row number from where the IPMI and host profile information ends
+     hostname_col - column number where the hostnames are to be read from
+     ipmi_address_col - column number from where the ipmi addresses are to be read
+     host_profile_col - column number from where the host profiles are to be read
+     ipmi_gateway_col - column number from where the ipmi gateways are to be read
+     private_ip_sheet - name of the sheet which has the private IP information
+     net_type_col - column number from where the network type is to be read
+     vlan_col - column number from where the network vlan is to be read
+     vlan_start_row - row number from where the vlan information starts
+     vlan_end_row - row number from where the vlan information ends
+     net_start_row - row number from where the network information starts
+     net_end_row - row number from where the network information ends
+     net_col - column number where the IP ranges for network is to be read
+     net_vlan_col - column number where the vlan information is present in the pod wise network section
+     public_ip_sheet - name of the sheet which has the public IP information
+     oam_vlan_col - column number from where the OAM vlan information is to be read from
+     oam_ip_row - row number from where the OAM network information is to be read from
+     oam_ip_col - column number from where the OAM network information is to be read from
+     oob_net_row - row number which has the OOB network subnet ranges
+     oob_net_start_col - column number from where the OOB network ranges start
+     oob_net_end_col - column number from where the OOB network ranges end
+     ingress_ip_row - row number from where the Ingress network information is to be read from
+     dns_ntp_ldap_sheet - name of the sheet which has the DNS, NTP and LDAP information
+     login_domain_row - row number which has the ldap login domain
+     ldap_col - column number which has the all ldap related information
+     global_group - row number which has the ldap group information
+     ldap_search_url_row - row number which has the ldap url
+     ntp_row - row number which has the ntp information
+     ntp_col - column number which has the ntp information
+     dns_row - row number which has the dns information
+     dns_col - column number which has the dns information
+     domain_row - row number which has the domain information
+     domain_col - column number which has the domain information
+     location_sheet - name of the sheet which has the location information
+     column - column number which has all the information
+     corridor_row - row number which has the corridor information
+     site_name_row - row number which has the site name
+     state_name_row - row number which has the state name
+     country_name_row - row number which has the country name
+     clli_name_row - row number which has CLLI information
+
+Example: Tugboat Plugin Usage
+-----------------------------
+1. Required Input(Refer to 'spyglass/examples' folder to get these inputs)
+   a) Excel File: SiteDesignSpec_v0.1.xlsx
+   b) Excel Spec: excel_spec_upstream.yaml
+   c) Site Config: site_config.yaml
+   d) Template_dir: '../examples/templates'
+   c) Site name: airship-seaworthy
+
+2. Spyglass CLI Command:
+   spyglass -mg -t tugboat -x SiteDesignSpec_v0.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=<relative path to '../examples/templates'
+