Merge branch 'master' into docs_handler_ansible

docs/index.rst

@@ -14,6 +14,7 @@ Contents:
    source/glossary
    source/resource
    source/orchestration
+   source/transports
    source/handler_ansible
    source/examples
    source/deployment_plan

docs/source/deployment_plan.rst

@@ -3,8 +3,9 @@
 Preparing deployment plan
 =========================
 
-Solar allows to make transitions between different versions of infrastructure
+Solar allows you to make transitions between different versions of
-based on changes found by solar control plane and configured events between this changes.
+infrastructure based on changes found by solar control plane and events
+configured between this changes.
 
 Required information
 ----------------------
@@ -27,11 +28,11 @@ user (human/program). Examples of changes are:
 Staged changes
 ---------------
 
-On demand solar runs procedure that will find all changed from last deployment
+On demand solar runs procedure that will find all resources changed from last
-resources, and using transitions from solar state machine will determine
+deployment and will determine list of actions using transitions from solar
-list of actions.
+state machine .
 
-This procedure performed by ::
+This procedure is performed by ::
 
     solar changes stage -d
 
@@ -58,7 +59,7 @@ It prints information like ::
         ++ collation: utf8_general_ci
         ++ location_id: 694b35afa622da857f95e14a21599d81
 
-At this point information is stored as a list, and doensnt know anything
+At this point information is stored as a list, and user doesn't know anything
 about dependencies between found changes.
 
 Events usage
@@ -66,21 +67,22 @@ Events usage
 
 For events definition see :ref:`res-event-term`.
 
-Events used during this procedure to insert dependencies between found changes,
+Events are used during this procedure to insert dependencies between found
-and add new actions that are reactions for changes.
+changes, and to add new actions that are reactions for changes.
 
 Example of dependency between changes would be *nova service* that depends
 on successful creation of *database*.
 
-And for removal we might add dependencies that will allow reverse
+For removal we might add dependencies that will allow reverse order, e.g. when
-order, e.g when removing *nova service* and *database*, *database* will be
+removing *nova service* and *database*, *database* will be removed only after
-removed only after successfull *nova service*.
+successful *nova service* removal.
-Can be specified as ::
+
+This can be specified as ::
 
     Dependency database1.run -> nova1.run
     Dependency nova1.remove -> database1.remove
 
-Reaction allows to construct more advacned worklows that will take into
+Reaction allows to construct more advanced workflows that will take into
 account not only changes, but also arbitrary actions for resources in solar.
 
 Good example of usage is provisioning procedure, where reboot must be
@@ -100,9 +102,9 @@ of deployment plan for current version of infrastructure ::
 
     solar changes process
 
-After this command plan can be viewed by ::
+After this deployment command plan can be viewed by ::
 
-    # graphviz representataion
+    # graphviz representation
     solar orch dg last
 
     # text report

docs/source/examples.rst

@@ -6,29 +6,30 @@ Examples
 Create resource for the puppet handler
 --------------------------------------
 
-Let's create an example :ref:`resource-term` for the puppet :ref:`res-handler-term`
+Let's create an example :ref:`resource-term` for the puppet
-version 1 [#]_. The resource should install and configure OpenStack Nova API
+:ref:`res-handler-term` version 1 [#]_. The resource should install and
-service.
+configure OpenStack Nova API service.
 
 .. [#] There is also puppet handler version 2 but it is out of the scope
    of this example.
 
-Step 1: Find an apropriate puppet module
+Step 1: Find an appropriate puppet module
 ++++++++++++++++++++++++++++++++++++++++
 
 The `Puppet OpenStack <https://wiki.openstack.org/wiki/Puppet>`_
 module for `Nova <https://github.com/openstack/puppet-nova>`_
 provides all of the required functionality.
 
-Step 2: Define granularity scope for a resource
+Step 2: Define granularity level for a resource
 +++++++++++++++++++++++++++++++++++++++++++++++
 
-One may want to implement resources as atomic entities doing each its only
+One may want to implement resources as atomic entities doing their only single
-task, like running one and only puppet manifest [#]_. Or as a single entity doing
+task, like running one and only puppet manifest [#]_. Other option might be
-all required tasks instead. In order to configure and run the Nova API service
+single entity doing all required tasks instead. In order to configure and run
-at least two manifests should be executed: the
+the Nova API service at least two manifests should be executed:
 `init.pp <https://github.com/openstack/puppet-nova/blob/master/manifests/init.pp>`_
-and the `api.pp <https://github.com/openstack/puppet-nova/blob/master/manifests/api.pp>`_ [#]_.
+and
+`api.pp <https://github.com/openstack/puppet-nova/blob/master/manifests/api.pp>`_ [#]_.
 
 .. [#] Puppet manifests may contain references to externally defined classess
    or services in the catalog. Keep that in mind then designing the resource.
@@ -40,26 +41,26 @@ Assuming the atomic tasks approach, the example resource for Nova API service
 should only use the `api.pp` manifest. Note that the puppet handler is normally
 executed in its own isolated puppet catalog with its specific hiera data only.
 This assumes every puppet manifest called by every action to be executed as a
-separate puppet run and shares nothing.
+separate puppet run and shares nothing with other tasks.
 
 Step 3: Define resource inputs
 ++++++++++++++++++++++++++++++
 
-Once the granularity scope of the resource was clearly defined, one should
+Once the granularity level of the resource is clearly defined, one should
 define the resource's :ref:`res-input-term` data. The puppet class `nova::api`
-contains a lots of parameters. It looks reasonable to use them as the resource
+contains lots of parameters. It looks reasonable to use them as the resource
 inputs as is.
 
 .. note ::
   There is a `helper script <https://github.com/bogdando/convert_puppet_parameters>`_
-  to convert a puppet class parameters into the format expeted by the
+  to convert a puppet class parameters into the format expected by the
   `meta.yaml` inputs file.
 
-Step 4: Implement the basic action run
+Step 4: Implement basic action run
 ++++++++++++++++++++++++++++++++++++++
 
 Each resource should have all of the mandatory actions defined. In this example
-we define only the :ref:`ref-action-term` `run`. For the example Nova API
+we define only the :ref:`ref-action-term` `run`. With the example of Nova API
 resource, the action run should:
 
 - fetch the resource inputs from the hiera [#]_ ::
@@ -86,7 +87,7 @@ resource, the action run should:
      }
 
 .. note ::
-   Otherwise, the called class would assume the package and exec are
+   Otherwise, called class would assume the package and exec are
    already included in the catalog by the `init.pp`. And would fail as
    there is no `class { 'nova': }` call expected for the Nova API resource
    action run.
@@ -101,7 +102,7 @@ Step 5: Think of the rest of the resource actions
 One should also design other actions for the resource. Mandatory are only
 `run`, `update` and `remove`. There might be additional ones like `on-fail`,
 `on-retry` or whichever are actually required to implement expected behavior.
-For the given API resource there is none specific actions expected and there
+For the given API resource there are no specific actions expected and there
 is nothing to do for the action remove. For the action update, it is likely
 the same steps should be done as for the action run.
 
@@ -116,7 +117,7 @@ Step 7: Think of the deployment composition
 
 The deployment composition is which resources should be used and in which order
 it should be executed to achive the expected result, which is a successfull
-:ref:`deploy-plan-term`. For the given example, the composition may be as the
+:ref:`deploy-plan-term`. For the given example, the composition may be as
 following:
 
 - Install and configure MySQL DB [#]_

docs/source/glossary.rst

@@ -9,28 +9,33 @@ Solar Glossary
 Resource
 ========
 
-You can learn more about it in :ref:`Resource Details <resource_details>`
+Resource is an abstraction of item in system managed by Solar. It's a basic
+building block used to assemble your system. Almost every entity in Solar
+is a resource.
+
+You can learn more about it in :ref:`resource details <resource_details>`
 
 .. _res-input-term:
 
 Input
 -----
-Resource configuration that will be used in actions, handlers and orchestration.
+Resource configuration that will be used in actions, handlers and
-All known inputs should be provided in meta.yaml
+orchestration. All known inputs for a resource should be defined in meta.yaml
 
 .. _res-connection-term:
 
 Connection
 ----------
-Allows to build hierarchy between inputs of several resources,
+Allows to build hierarchy between inputs of several resources, parent value
-parent value will be always used in child while connection is created.
+will be always used in child while connection is created. If connection is
-If connection will be removed - original value of child will be preserved.
+removed - original value of child will be preserved.
 
 .. _res-action-term:
 
 Action
 ------
-Solar wraps deployment code into actions with specific names.
+Solar wraps deployment code into actions with specific names. Actions are
+executed from the resource.
 
 .. _res-tag-term:
 
@@ -44,20 +49,20 @@ used for different user operations.
 Handler
 =======
 
-Layer that responsible for action execution and tracking result.
+Layer responsible for action execution and tracking results.
 
 .. _res-transports-term:
 
 Transport
 =========
 
-Used in handlers to communicate with managed by solar hosts. List of transports
+Used in handlers to communicate with hosts managed by Solar.
-should be added to a node. Transports will be added to a resource by means
-of transports id.
 
-Two different types of transports are used: run and sync.
+.. seealso::
-Run transport - reponsible for running command on remote host.
+
-Sync transport - uploads required information.
+   :ref:`More details about transports <transports_details>`
+
+.. _location-id-term:
 
 location_id
 -----------
@@ -65,12 +70,15 @@ Used in transport layer to find ip address of a node. ::
 
   'location_id': '96bc779540d832284680785ecd948a2d'
 
+.. _transports-id-term:
+
 transports_id
 -------------
 Used to find transports array that will be used for transport selection. ::
 
   'transports_id': '3889e1790e68b80b4f255cf0e13494b1'
 
+
 BAT transport
 -------------
 According to preferences solar will choose best available transport for
@@ -85,12 +93,11 @@ Used in solar to describe all possible transitions between resources changes.
 Each event allows to specify two points of transitions, condition of this
 transition and type of event.
 
-Right now we are supporting 2 types of events.
+Right now we are supporting 2 types of events:
 
-1. Dependency
+1. Dependency - inserts edge between 2 changes into the deployment plan.
-Inserts edge between 2 changes into the deployment plan.
+2. Reaction - inserts change specified in reaction and makes edge between
-2. Reaction
+parent and child.
-Inserts change specified in reaction and makes edge between parent and child.
 
 Example ::
 
@@ -106,7 +113,7 @@ Example ::
 Virtual resource/template
 =========================
 
-Composition layer that allows to:
+Composition layer that allows user to:
 
 - group resources
 - specify connections between inputs
@@ -130,9 +137,9 @@ History
 After action that is related to change will be executed - it will be moved to
 history with same uuid.
 
-Commited resource data
+Committed resource data
 ----------------------
-After each succesfull change commited copy of resource data will be updated
+After each successful change committed copy of resource data will be updated
 with diff of that change.
 
 .. _orch-term:

docs/source/orchestration.rst

@@ -5,7 +5,6 @@ Deployment operations
 
 Stage changes
 -------------
-
 After user created all required resource - it is possible to automatically
 detect which resource requires changes with ::
 
@@ -14,30 +13,30 @@ detect which resource requires changes with ::
 
 History
 -------
-After changes are staged - they will be used to populate history which can be seen
+After changes are staged - they will be used to populate history which can be
-with command (*n* option used to limit number of items, -1 will return all changes) ::
+previewed with command (*n* option used to limit number of items, -1 will
+return all changes) ::
 
     solar changes history -n 5
 
 Prepare deployment plan
 -----------------------
-
+User is able to generate deployment scenario based on changes found by system
-User is able to generate deployment scenario based on changes found by system log. ::
+log. ::
 
     solar changes process
 
-
+This command will prepare deployment graph, and return uid of deployment graph
-This command will prepare deployment graph, and return uid of deployment graph to
+to work with.
-work with.
 
 All commands that are able to manipulate deployment graph located in
 *orch* namespace.
 
 
 .. tip::
-   Solar writes returned deployment graph uid into special file (`.solar_cli_uids`), it
+   Solar writes returned deployment graph uid into special file
-   allows you to use `last` instead of full returned uid:
+   (`.solar_cli_uids`), it allows you to use `last` instead of full returned
-   `solar orch report <uid>` becomes `solar orch report last`
+   uid: `solar orch report <uid>` becomes `solar orch report last`
 
 
 Report
@@ -84,6 +83,7 @@ All tasks will be returned to PENDING state, and deployment will be restarted ::
 
 Retry deployment
 ----------------
-Orchestrator will reset all ERROR tasks to PENDING state and restart deployment ::
+Orchestrator will reset all ERROR tasks to PENDING state and restart
+deployment ::
 
     solar orch retry <uid>

docs/source/resource.rst

@@ -3,7 +3,15 @@
 Resource
 ========
 
-Resource is one of the key Solar components. Resoruce definition takes place in ``meta.yaml`` file.
+Resource is one of the key Solar components, almost every entity in Solar is a
+resource. Examples are:
+
+* packages
+* services
+
+Resources are defined in ``meta.yaml`` file. This file is responsible for basic
+configuration of given resource. Below is an explanation what constitutes
+typical resource. 
 
 
 Basic resource structure
@@ -23,14 +31,34 @@ Handler
 
 .. TODO: add link to handlers doc there
 
-Layer that is responsible for action execution. You need to specify handler per resource, definition in ``meta.yaml`` looks like ::
+Pluggable layer that is responsible for executing an action on resource. You
+need to specify handler per every resource. Handler is defined in ``meta.yaml``
+as below ::
 
   handler: puppet
 
+Solar currently supports following handlers:
+
+* puppet - first version of puppet handler (legacy, will be deprecated soon)
+* puppetv2 - second, improved version of puppet, supporting hiera integration 
+* ansible_playbook - first version of ansible handler (legacy, will be
+deprecated soon)
+* ansible_template - second generation of ansible implementation, includes
+transport support
+
+Handlers are pluggable, so you can write your own easily to extend
+functionality of Solar. Interesting examples might be Chef, SaltStack,
+CFEngine etc. Using handlers allows Solar to be quickly implemented in various
+environments and integrate with already used configuration management tools.
 
 Input
 -----
-Treat them as values that your resouce have. All needed inputs should be provided in ``meta.yaml`` for example ::
+Inputs are essentially values that given resource can accept. Exact usage
+depends on handler and actions implementation. If your handler is puppet,
+inputs are basically parameters that can be accepted by puppet manifest
+underneath.
+
+All needed inputs should be defined in ``meta.yaml`` for example: ::
 
   input:
       keystone_password:
@@ -48,7 +76,9 @@ Treat them as values that your resouce have. All needed inputs should be provide
 
 Input schema
 ~~~~~~~~~~~~
-It allows to validate if all values are correct. ``!`` at the end of a type means that this is required (``null`` value is not valid).
+Input definition contains basic schema validation that allows to validate if
+all values are correct. ``!`` at the end of a type means that it is required
+(``null`` value is not valid).
 
 * string type ``str``, ``str!``
 * integer type ``int``, ``int!``
@@ -63,14 +93,18 @@ It allows to validate if all values are correct. ``!`` at the end of a type mean
 
 Action
 ------
-Solar wraps deployment code into actions with specific names. Actions are executed by :ref:`res-handler-term`
+Solar wraps deployment code into actions with specific names. Actions are
+executed by :ref:`res-handler-term`
 
 Several actions of resource are mandatory:
+
 - run
 - remove
 - update
 
-You can just put files into ``actions`` subdir in your resource or solar will detect them automaticaly, you can also provide actions in ``meta.yaml`` ::
+You can just put files into ``actions`` subdir in your resource and solar will
+detect them automatically based on their names, or you can also customize
+action file names in ``meta.yaml`` ::
 
     actions:
       run: run.pp
@@ -79,6 +113,7 @@ You can just put files into ``actions`` subdir in your resource or solar will de
 Tag
 ---
 
-You can attach as many tags to resource as you want, later you can use those tags for grouping etc ::
+Tags are used for flexible grouping of resources. You can attach as many tags
+to resource as you want, later you can use those tags for grouping etc ::
 
   tags: [resource=hosts_file, tag_name=tag_value, just_some_label]

docs/source/transports.rst

@@ -0,0 +1,50 @@
+.. _transports_details:
+
+
+Transports
+==========
+
+Transports are used by Solar to communicate with managed nodes.
+Transports are also resources, so they have all resources features and
+flexibility.
+Transports should be added to a node, but if you need you can add different
+transports for different resources.
+
+How it works
+------------
+
+Each resource in solar has a random :ref:`transports-id-term` generated,
+when resources are connected to each other. Solar will ensure that correct
+`transport_id` is used. Then using this `transport_id` a correct real value is
+fetched. Changing transports contents will not cause `resource.update` action
+for related resources.
+
+Sync transport
+--------------
+
+This transport uploads required information to target node.
+
+Currently there are following sync transports available:
+
+* ssh
+* rsync
+* solar_agent
+* torrent
+
+Run transport
+-------------
+
+This transport is responsible for running commands on remote host.
+
+Currently there are following run transports available:
+
+* ssh
+* solar_agent
+
+BAT transport
+-------------
+
+A transport that will automatically select best available transport (BAT) that
+is available for a given resource. Currently it's default transport in the
+system, so when you add more transports, everything should configure
+automatically.

tox.ini

@@ -21,6 +21,12 @@ deps = -r{toxinidir}/requirements.txt
        sphinx
 commands = {posargs:}
 
+[testenv:cover]
+commands =
+  coverage erase
+  python setup.py testr --coverage \
+    --testr-args='--concurrency=1 {posargs}'
+
 [testenv:devenv]
 envdir = devenv
 usedevelop = True

utils/docker_build.sh

@@ -0,0 +1,14 @@
+#!/bin/bash
+set -e
+
+# should be executed from directory with required Dockerfile
+name_w_tags=$1
+
+if [[ -z "$name_w_tags" ]]; then
+    name_w_tags='solarproject/solar-celery:latest'
+fi
+
+echo "Building image with name $name_w_tags"
+docker build -t "$name_w_tags" .
+docker push "$name_w_tags"
+