x/stacktach-sandbox
Code Issues Proposed changes

Some stuff was pretty clunky, fixed it up.

Browse Source 
Change-Id: I6133e26d242d6d4a89219e746f03f9808cb13685
This commit is contained in:
Sandy Walsh 2015-03-05 17:47:49 -08:00
parent ddda9be458
commit 23375de029
2 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/about.html
View File

@ -132,7 +132,7 @@ max_messages = 100
</pre>
<p>The important part of this configuration is the <code>[event_worker]</code> section. This says we want to use the RabbitMQ data source. The RabbitMQ connectivity information is stored in the <code>[rabbit_broker]</code> section. The name of each rabbitmq queue to consume from is specified in the <code>[consumers]</code> section. For every queue you define there, you will need a <code>[consumer:&lt;queue_name&gt;]</code> section. This last section is where there real magic happens. Beyond defining the exchange, routing_key and durability characteristics, it defines the chain of <code>Yagi Handlers</code> that will run on every notification that gets consumed. </p>
<p>You can write your own Yagi handlers if you like, but there are a number that ship with StackTach.v3 to do some interesting things. The most important of these is the <code>winchester.yagi_handler:WinchesterHandler</code>. This handler is your entry point into StackTach.v3 stream processing. But first, we need to convert those messy notifications into events ...</p>
<p>You can write your own Yagi handlers if you like, but there are a number that ship with StackTach.v3 to do some interesting things. The most important of these is the <a href='https://github.com/stackforge/stacktach-winchester/blob/4875e419a66974e416dbe3b43ed286017bad1ec4/winchester/yagi_handler.py#L18'>winchester.yagi_handler:WinchesterHandler</a>. This handler is your entry point into StackTach.v3 stream processing. But first, we need to convert those messy notifications into events ...</p>
<h3>Distilling Notifications to Events</h3>
<p>Now we have notifications coming into Winchester. But, as we hinted at above, we need to take the larger notification and <i>distill</i> it down into a, more manageable, event. The stack-distiller module makes this happen. Within StackTach.v3, this is part of <code>winchester.yagi_handler:WinchesterHandler</code>.</p>
<p>A notification is a large, nested JSON data structure. But we don't need all of that data for stream processing. In fact, we generally only require a few <a href='glossary.html#trait'>Traits</a> from the notification. That's what distilling does. It pulls out the important traits, scrubs the data and uses that. Distillations are done via the distillation configuration file (specified in winchester.conf). </p>

5
docs/install.html
View File

@ -83,19 +83,20 @@
<li>The pipeline-worker workers look for ready streams and does pipeline processing on them.</li>
<li>The user can use the klugman cmdline tool to talk to the REST API to perform stream and event operations/queries</li>
</ol>
<p>In order to do all this, there are a number of configation files required. Refer to the general documentation here or the particular libaries README file for configuration details. The main configuration files include:</p>
<p>In order to do all this, there are a number of configation files required. Refer to the general documentation here or the particular libaries README file for configuration details. Of course, the names could be anything, these are just the onces we've settled on. The main configuration files include:</p>
<ul>
<li><b>yagi.conf</b> - the configuration file that tells yagi how to connect to the quque and what to do with the notifications consumed.</li>
<li><b>winchester.yaml</b> - the master configuration file for winchester. It specifies the pipeline configuration to use, the stream definitions and the triggering rules.</p>
<li><b>triggers.yaml</b> - the detailed stream definitions and pipeline triggering rules</b></li>
<li><b>pipelines.yaml</b> - the pipeline handler definitions</p>
<li><b>event_definitions.yaml</b> - how the distiller should map notifications to events. Which traits to select or transform.
</ul>
<p>Where each of these files is used is illustrated in the diagram below:</p>
<img src='sandbox-2.gif' class="img-rounded"/>
<p>This will get you going for a minimal StackTach.v3 installation. Especially handy for dev environments. Read up on the "build" command, below, for instructions on For larger deployments, you may want to look at how we deploy StackTach.v3 within Rackspace Public Cloud, below:</p>
<p>This will get you going for a minimal StackTach.v3 installation. It's especially handy for dev environments as well as a way of just playing around. For low-volume/non-mission critical evironments, this would be appropriate. Read up on the "build" command, below, for instructions on launching the sandbox environment. For larger deployments, you may want to look at how we deploy StackTach.v3 within Rackspace Public Cloud, below:</p>
<h3>How StackTach.v3 is deployed at Rackspace</h3>
<p>For Rackspace Public Cloud, OpenStack is deployed in many different regions and each region is comprised of many cells.</p>