Merge pull request #386 from bogdando/docs_resource_example_puppet

Add examples docs, reformat add indexes

docs/index.rst

@@ -13,7 +13,7 @@ Contents:
 
    source/glossary
    source/orchestration
-
+   source/examples
 
 
 Indices and tables

docs/source/examples.rst

@@ -0,0 +1,136 @@
+.. _examples:
+
+Examples
+========
+
+Create resource for the puppet handler
+--------------------------------------
+
+Let's create an example :ref:`resource-term` for the puppet :ref:`res-handler-term`
+version 1 [#]_. The resource should install and configure OpenStack Nova API
+service.
+
+.. [#] There is also puppet handler version 2 but it is out of the scope
+   of this example.
+
+Step 1: Find an apropriate puppet module
+++++++++++++++++++++++++++++++++++++++++
+
+The `Puppet OpenStack <https://wiki.openstack.org/wiki/Puppet>`_
+module for `Nova <https://github.com/openstack/puppet-nova>`_
+provides all of the required functionality.
+
+Step 2: Define granularity scope for a resource
++++++++++++++++++++++++++++++++++++++++++++++++
+
+One may want to implement resources as atomic entities doing each its only
+task, like running one and only puppet manifest [#]_. Or as a single entity doing
+all required tasks instead. In order to configure and run the Nova API service
+at least two manifests should be executed: the
+`init.pp <https://github.com/openstack/puppet-nova/blob/master/manifests/init.pp>`_
+and the `api.pp <https://github.com/openstack/puppet-nova/blob/master/manifests/api.pp>`_ [#]_.
+
+.. [#] Puppet manifests may contain references to externally defined classess
+   or services in the catalog. Keep that in mind then designing the resource.
+
+.. [#] This assumes configuring DB and messaging entities like user, password
+   database, vhost, access rights are left out of the scope of this example.
+
+Assuming the atomic tasks approach, the example resource for Nova API service
+should only use the `api.pp` manifest. Note that the puppet handler is normally
+executed in its own isolated puppet catalog with its specific hiera data only.
+This assumes every puppet manifest called by every action to be executed as a
+separate puppet run and shares nothing.
+
+Step 3: Define resource inputs
+++++++++++++++++++++++++++++++
+
+Once the granularity scope of the resource was clearly defined, one should
+define the resource's :ref:`res-input-term` data. The puppet class `nova::api`
+contains a lots of parameters. It looks reasonable to use them as the resource
+inputs as is.
+
+.. note ::
+  There is a `helper script <https://github.com/bogdando/convert_puppet_parameters>`_
+  to convert a puppet class parameters into the format expeted by the
+  `meta.yaml` inputs file.
+
+Step 4: Implement the basic action run
+++++++++++++++++++++++++++++++++++++++
+
+Each resource should have all of the mandatory actions defined. In this example
+we define only the :ref:`ref-action-term` `run`. For the example Nova API
+resource, the action run should:
+
+- fetch the resource inputs from the hiera [#]_ ::
+
+      $resource = hiera($::resource_name)
+      $ensure_package = $resource['input']['ensure_package']
+      $auth_strategy = $resource['input']['auth_strategy']
+
+.. [#] The syntax is the puppet handler v1 specific. The v2 allows to query
+   the hiera directly, like `$public_vip = hiera('public_vip')`
+
+- call the `class { 'nova::api': }` with the required parameters
+- implement workarounds for externally referenced entities, like ::
+
+     exec { 'post-nova_config':
+       command     => '/bin/echo "Nova config has changed"',
+     }
+
+     include nova::params
+
+     package { 'nova-common':
+       name   => $nova::params::common_package_name,
+       ensure => $ensure_package,
+     }
+
+.. note ::
+   Otherwise, the called class would assume the package and exec are
+   already included in the catalog by the `init.pp`. And would fail as
+   there is no `class { 'nova': }` call expected for the Nova API resource
+   action run.
+   In order to implement the resource without such workarounds, one should
+   rethink the granularity scope for the resource. And make sure the resource
+   contains required inputs for the main `nova` and `nova::api` classes and
+   call them both in the resource action run.
+
+Step 5: Think of the rest of the resource actions
++++++++++++++++++++++++++++++++++++++++++++++++++
+
+One should also design other actions for the resource. Mandatory are only
+`run`, `update` and `remove`. There might be additional ones like `on-fail`,
+`on-retry` or whichever are actually required to implement expected behavior.
+For the given API resource there is none specific actions expected and there
+is nothing to do for the action remove. For the action update, it is likely
+the same steps should be done as for the action run.
+
+Step 6: Design the high level functional test
++++++++++++++++++++++++++++++++++++++++++++++
+
+TODO(bogdando) provide details about test.py and writing tests for Nova API
+in order to verify if it works on the app level.
+
+Step 7: Think of the deployment composition
++++++++++++++++++++++++++++++++++++++++++++
+
+The deployment composition is which resources should be used and in which order
+it should be executed to achive the expected result, which is a successfull
+:ref:`deploy-plan-term`. For the given example, the composition may be as the
+following:
+
+- Install and configure MySQL DB [#]_
+- Install and configure RabbitMQ node
+- Install and configure dependency components like OpenStack Keystone
+- Create all of the required user/tenant/db/vhost entities and assign rights
+- Install and configure Nova main components, like packages, db sync, configs.
+- Install and configure Nova API. BINGO! A job for our resource, at last!
+
+.. [#] Omitted host related steps like OS provisioning, disks and network
+   configuration.
+
+Besides the execution plan, there is also data :ref:`res-connection-term`
+to be considered. For example, one might want to have all of the OpenStack
+services to use the common RabbitMQ virtualhost and user. Or have them
+separated instead. Or use the clustered RabbitMQ nodes. These decisions
+will directly impact how resources' inputs should be connected.

docs/source/glossary.rst

@@ -1,12 +1,18 @@
+.. _glossary:
+
 ==============
 Solar Glossary
 ==============
 
-Resources
+.. _resource-term:
-=========
 
-Inputs
+Resource
---------
+========
+
+.. _res-input-term:
+
+Input
+-----
 Resource configuration that will be used in actions, handlers and orchestration.
 All known inputs should be provided in meta.yaml ::
 
@@ -24,14 +30,18 @@ All known inputs should be provided in meta.yaml ::
         schema: str
         value: 'cinder'
 
-Connections
+.. _res-connection-term:
-------------
+
+Connection
+----------
 Allows to build hierarchy between inputs of several resources,
 parent value will be always used in child while connection is created.
 If connection will be removed - original value of child will be preserved.
 
-Actions
+.. _res-action-term:
---------
+
+Action
+------
 Solar wraps deployment code into actions with specific names.
 Several actions of resource are mandatory:
 - run
@@ -44,21 +54,29 @@ All actions should be provided in meta.yaml ::
       run: run.pp
       update: run.pp
 
-Tags
+.. _res-tag-term:
-------
+
+Tag
+---
 Used to create arbitrary groups of resources, later this groups will be
 used for different user operations.
 
-Handlers
+.. _res-handler-term:
-========
+
+Handler
+=======
+
 Layer that responsible for action execution and tracking result.
 Currently handler specified in resource meta.yaml and used for all resource
 actions ::
 
   handler: puppet
 
-Transports
+.. _res-transports-term:
------------
+
+Transport
+=========
+
 Used in handlers to communicate with managed by solar hosts. List of transports
 should be added to a node. Transports will be added to a resource by means
 of transports id.
@@ -68,7 +86,7 @@ Run transport - reponsible for running command on remote host.
 Sync transport - uploads required information.
 
 location_id
-------------
+-----------
 Used in transport layer to find ip address of a node. ::
 
   'location_id': '96bc779540d832284680785ecd948a2d'
@@ -80,12 +98,15 @@ Used to find transports array that will be used for transport selection. ::
   'transports_id': '3889e1790e68b80b4f255cf0e13494b1'
 
 BAT transport
---------------
+-------------
 According to preferences solar will choose best available transport for
 file uploading and command execution.
 
-Events
+.. _res-event-term:
-======
+
+Event
+=====
+
 Used in solar to describe all possible transitions between resources changes.
 Each event allows to specify two points of transitions, condition of this
 transition and type of event.
@@ -106,39 +127,49 @@ Example ::
   child_action: run
   state: success // condition
 
+.. _res-virtual-term:
+
+Virtual resource/template
+=========================
 
-Virtual resources/templates
-===========================
 Composition layer that allows to:
 
 - group resources
 - specify connections between inputs
 - add list of events
 
+.. _system-log-term:
+
 System log component
 ====================
+
 Component responsible for tracking changes and keeping ordered history of
 them.
 
 Staged log
------------
+----------
 Based on user changes - solar will create log of staged changes.
 This log will be used later to build deployment plan.
 
 History
---------
+-------
-After action that is related to change will be executed - it will be moved to history with same uuid.
+After action that is related to change will be executed - it will be moved to
+history with same uuid.
 
 Commited resource data
 ----------------------
 After each succesfull change commited copy of resource data will be updated
 with diff of that change.
 
+.. _orch-term:
+
 Orchestration component
-========================
+=======================
+
+.. _deploy-plan-term:
 
 Deployment plan
-----------------
+---------------
 Based on changes tracked by system log and configured events - solar build
 deployment plan. In general deployment plan is built with ::
 
@@ -157,4 +188,4 @@ Solar cli provides several commands to work with deployment plan.
 - stop
 - resume/restart/retry
 
-TODO link to ./orchestration.md
+See also :ref:`orchestration`

docs/source/orchestration.rst

@@ -1,3 +1,5 @@
+.. _orchestration:
+
 Deployment operations
 =====================
 
@@ -10,8 +12,7 @@ detect which resource requires changes with ::
     solar changes stage
 
 History
---------
+-------
-
 After changes are staged - they will be used to populate history which can be seen
 with command (*n* option used to limit number of items, -1 will return all changes) ::
 
@@ -32,8 +33,7 @@ All commands that are able to manipulate deployment graph located in
 *orch* namespace.
 
 Report
--------
+------
-
 Report will print all deployment tasks in topological order, with status,
 and error if status of task is *ERROR* ::
 
@@ -41,7 +41,6 @@ and error if status of task is *ERROR* ::
 
 Graphviz graph
 --------------
-
 To see picture of deployment dependencies one can use following command ::
 
     solar orch dg <uid>
@@ -52,7 +51,6 @@ order of traversal.
 
 Run deployment
 --------------
-
 Execute deployment ::
 
     solar orch run-once <uid>
@@ -60,28 +58,24 @@ Execute deployment ::
 
 Stop deployment
 ---------------
-
 Gracefully stop deployment, after all already scheduled tasks are finished ::
 
     solar orch stop <uid>
 
 Resume deployment
-------------------
+-----------------
-
 Reset SKIPPED tasks to PENDING and continue deployment ::
 
     solar orch resume <uid>
 
 Restart deployment
 ------------------
-
 All tasks will be returned to PENDING state, and deployment will be restarted ::
 
     solar orch restart <uid>
 
 Retry deployment
 ----------------
-
 Orchestrator will reset all ERROR tasks to PENDING state and restart deployment ::
 
     solar orch retry <uid>