From 23375de02971f901ab857374fe769813678c83e0 Mon Sep 17 00:00:00 2001 From: Sandy Walsh Date: Thu, 5 Mar 2015 17:47:49 -0800 Subject: [PATCH] Some stuff was pretty clunky, fixed it up. Change-Id: I6133e26d242d6d4a89219e746f03f9808cb13685 --- docs/about.html | 2 +- docs/install.html | 5 +++-- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/about.html b/docs/about.html index b27cb53..51afdb7 100644 --- a/docs/about.html +++ b/docs/about.html @@ -132,7 +132,7 @@ max_messages = 100

The important part of this configuration is the [event_worker] section. This says we want to use the RabbitMQ data source. The RabbitMQ connectivity information is stored in the [rabbit_broker] section. The name of each rabbitmq queue to consume from is specified in the [consumers] section. For every queue you define there, you will need a [consumer:<queue_name>] section. This last section is where there real magic happens. Beyond defining the exchange, routing_key and durability characteristics, it defines the chain of Yagi Handlers that will run on every notification that gets consumed.

-

You can write your own Yagi handlers if you like, but there are a number that ship with StackTach.v3 to do some interesting things. The most important of these is the winchester.yagi_handler:WinchesterHandler. This handler is your entry point into StackTach.v3 stream processing. But first, we need to convert those messy notifications into events ...

+

You can write your own Yagi handlers if you like, but there are a number that ship with StackTach.v3 to do some interesting things. The most important of these is the winchester.yagi_handler:WinchesterHandler. This handler is your entry point into StackTach.v3 stream processing. But first, we need to convert those messy notifications into events ...

Distilling Notifications to Events

Now we have notifications coming into Winchester. But, as we hinted at above, we need to take the larger notification and distill it down into a, more manageable, event. The stack-distiller module makes this happen. Within StackTach.v3, this is part of winchester.yagi_handler:WinchesterHandler.

A notification is a large, nested JSON data structure. But we don't need all of that data for stream processing. In fact, we generally only require a few Traits from the notification. That's what distilling does. It pulls out the important traits, scrubs the data and uses that. Distillations are done via the distillation configuration file (specified in winchester.conf).

diff --git a/docs/install.html b/docs/install.html index 14b9582..828f15d 100644 --- a/docs/install.html +++ b/docs/install.html @@ -83,19 +83,20 @@
  • The pipeline-worker workers look for ready streams and does pipeline processing on them.
  • The user can use the klugman cmdline tool to talk to the REST API to perform stream and event operations/queries
    • -

    In order to do all this, there are a number of configation files required. Refer to the general documentation here or the particular libaries README file for configuration details. The main configuration files include:

    +

    In order to do all this, there are a number of configation files required. Refer to the general documentation here or the particular libaries README file for configuration details. Of course, the names could be anything, these are just the onces we've settled on. The main configuration files include:

    Where each of these files is used is illustrated in the diagram below:

    -

    This will get you going for a minimal StackTach.v3 installation. Especially handy for dev environments. Read up on the "build" command, below, for instructions on For larger deployments, you may want to look at how we deploy StackTach.v3 within Rackspace Public Cloud, below:

    +

    This will get you going for a minimal StackTach.v3 installation. It's especially handy for dev environments as well as a way of just playing around. For low-volume/non-mission critical evironments, this would be appropriate. Read up on the "build" command, below, for instructions on launching the sandbox environment. For larger deployments, you may want to look at how we deploy StackTach.v3 within Rackspace Public Cloud, below:

    How StackTach.v3 is deployed at Rackspace

    For Rackspace Public Cloud, OpenStack is deployed in many different regions and each region is comprised of many cells.