85 Commits

Author SHA1 Message Date
Sandy Walsh
f6b34bb7f7 Fix klugman call in screen script for new cmdline ordering.
klugman now requires the URL as a mandatory param.

Change-Id: I51b5a2750c85e02580d023f2d8291ec8ec76f571
0.9.2
2015-04-27 11:55:41 -07:00
Sandy Walsh
8995b349a7 Status utility now support different vhosts per cell.
Minor tweaks also include:
- shoebox handler now wraps notification with region/cell info
- pip_freeze_versions.txt is updated after each ./build and
  copied into the VENV directory for inclusion in tarball.

Change-Id: Ibe4027770b859ea7878c9f0952fccf94fb496c43
2015-04-16 12:04:47 -07:00
Sandy Walsh
f4b9b7758b Whoops, old .venv used for pip_freeze. Fixed.
Change-Id: I0fee078d57d4795d40073fffcf34cb1635d5a016
0.9.1
2015-04-14 05:41:25 -07:00
Sandy Walsh
f96a5f4ab8 pip freeze of versions for production release tag.
Change-Id: I6f2ea33ac77abb3900dbe5ee05807fdfedf1cde4
0.9.0
2015-04-14 05:18:01 -07:00
Sandy Walsh
c99d0c3980 Change status utility to reflect new pipeline worker log format.
Change-Id: I45c3919da7fb34dee3564e8578b428cb0b5ee1c3
2015-04-14 05:13:06 -07:00
Levi Blackstone
c7b7cf7876 Correct a bunch of typos in docs and add contributor link
Change-Id: I18769378a3c14ca0771da9c1fbd5d3fb81959e3b
2015-04-03 09:48:55 -05:00
Jenkins
d3ad114596 Merge "More docs around debugging streams." 2015-04-03 14:13:57 +00:00
Sandy Walsh
3c41a33bb7 More docs around debugging streams.
And minor patch to status utility.

Change-Id: I3eb9f8028b0140a9f4dae22137f137c37a80e6c9
2015-04-03 07:12:32 -07:00
Levi Blackstone
1efff2974a Update example timestamps to comply with ISO 8601
Change-Id: I066d7bc916fcb1cb42d325bb7530fabf45ab69f9
2015-04-03 09:09:34 -05:00
Sandy Walsh
97edc50ae5 Fix docs to reflect Kilo notification driver changes
For OpenStack deployments Kilo-release and beyond.

Change-Id: I80f135248161f585f0763dfd2335ed46c0c082ee
2015-03-23 08:59:55 -07:00
Sandy Walsh
a5aad8bfce UsageHandler and NotabeneHandler docs.
Includes a description of failure conditions.

Change-Id: I312c1d4685ba1138d1478ffc1d424fc2216a450c
2015-03-20 09:15:47 -07:00
Sandy Walsh
1d5f7d21fd Better explain how ansible scripts work
And fix some wonky things in the deploy

Change-Id: I681c7be1223e81617b3e2912aa87e72cc9514d49
2015-03-08 20:42:00 -07:00
Sandy Walsh
23375de029 Some stuff was pretty clunky, fixed it up.
Change-Id: I6133e26d242d6d4a89219e746f03f9808cb13685
2015-03-05 17:47:49 -08:00
Sandy Walsh
ddda9be458 Remove oahu mentions ... and fix up README
Change-Id: I9c077b6e5ed10be76ef6b702d10a30f33aff6fa2
2015-03-05 12:12:48 -08:00
Sandy Walsh
c30f3db329 More docs on sandbox setup
Explains how a sandbox dev deploy looks and what goes where.

Change-Id: I32d1ddf5b893b6650ddd5ff926978ef23eb94ed4
2015-03-05 11:59:25 -08:00
Sandy Walsh
becac34178 Fix API doc titles
Change-Id: I0884f72a4eb1f16fca19e58859c8b17412182708
2015-03-04 14:15:41 -08:00
Sandy Walsh
b82f147b8b Added missing docs for streams and added events api docs.
Also added support for quick build (does not reinstall everything in .venv)
which is useful when you have to hand-tweak a venv for testing. Generally
though, it shouldn't be needed.

Change-Id: I4b93c8e256f49d16d2b0fa821b37355526885ce8
2015-03-04 09:31:53 -08:00
Sandy Walsh
c438c215c9 Fix bad links on about (docs) page
Whoops

Change-Id: I6b52313905d52faf3f4e29e536f9d7dfdf962ef0
2015-02-27 13:05:39 -08:00
Jenkins
7623a2dc80 Merge "Expanded event_definitions to UsageHandler required traits." 2015-02-27 18:12:49 +00:00
Sandy Walsh
d1cacbcee4 API docs
Started on API docs. Particularly the /streams resource.

Change-Id: I082486fd62158fb175f5a22a3cfba141bb7d1fa1
2015-02-27 07:45:27 -08:00
Sandy Walsh
a7e600585e Adds glossary and fixes some mistakes
Change-Id: I679ffa1383bf5214ebbf1ac273f9179bb825c09d
2015-02-27 05:27:55 -08:00
Sandy Walsh
f0a178b54c More docs: install guide, contributing and screencasts.
Still need the terminology section.

Change-Id: Ic3976231ad9fbf2eb16ea708011aa906d5c5e781
2015-02-26 13:43:04 -08:00
Sandy Walsh
5b17594ce2 Expanded event_definitions to UsageHandler required traits.
Otherwise the CUF Yagi handler will not be able to create and
publish the CUF records.

Change-Id: Iaac5aaee7d6c17ee58a26f9a64d8fd7148b44b80
2015-02-25 11:03:01 -08:00
Sandy Walsh
74fd693f51 Supports the new tarball style of WritingJSONRollManager.
Updates the ansible and local dev configs to archive
events to tarballs via shoebox.

Change-Id: I1952aea79f1021aae5094f6ca45e325eb9b43485
2015-02-24 11:29:25 -08:00
Sandy Walsh
73ef32b2b6 Start on STv3 docs
This is not only the docs, but the stacktach.com website.

Change-Id: I6b1f71a2fa90966f55f4cf21181d729093a53718
2015-02-19 10:24:47 -08:00
Levi Blackstone
5b3293c146 Update Vagrant script to avoid needing sudo for build.sh
* Update Vagrantfile to run git clone command as unprivileged to avoid having root own the directory
* Update README to reflect changed command

Change-Id: Ibfde4f9cce2367ee44afb43dbafbaf6d626d7381
2015-02-19 08:44:59 -06:00
Sandy Walsh
b21ee4ef6d Fix instance_type distilling.
Was trying to parse string flavor id as int and failing. This fixes that.

Change-Id: I078d5860a385a1c2f0098b23af9c416017b59c1c
2015-02-18 09:25:43 -08:00
Sandy Walsh
363dfd5966 Include cell name in event filename template
For larger deployments, the region name should be parameterized too.

Change-Id: Iac7e589c4b58a74b5730a436a77cac31d7fc1fe1
2015-02-17 15:10:47 -08:00
Sandy Walsh
4cb647a7f3 Updates yagi.conf to write json files of every event.
For uploading raw notifications to archive (like an HDFS store).

Also, removes msgpack-python (installed by oslo), which currently
conflict with carrot. This is a temp fix.

Change-Id: I973949c83c9018235169646ce0370d71a2260241
2015-02-17 15:00:45 -08:00
Levi Blackstone
0833226b93 Add Vagrant support for easy local testing
* Add simple Vagrantfile to provision an Ubuntu 14.04 VM with the bootstrap.sh script
* Update README with instructions for use

Change-Id: I94cd69742aeffab4c01a27510c25f381151bfefe
2015-02-17 15:24:43 -06:00
Sandy Walsh
d9abe7ceae Don't purge completed streams and fix pipeline expiries
Just a more variances from production to local dev.

Change-Id: I24611c3dd4e816ac78c77930e6fd85ecafe82d56
2015-02-16 12:33:32 -08:00
Sandy Walsh
5d5f8ca105 Fix up defaults to use notabene handler and usage
Also backports some tweaks from ops:
- fixes logging.conf
- fixes pipeline_worker first time start

Change-Id: I6ab1e567594097bc23d857c05d0c1fc84262babf
2015-02-10 12:57:28 -08:00
Sandy Walsh
acd342b38a Switch to "nova" exchange.
Whoops. Was assuming a "monitor" exchange, but it's the "nova" exchange.

Change-Id: Ib0b921bf92724693eaf666762022a5245af8ef11
2015-02-06 11:03:27 -08:00
Sandy Walsh
72c8f298e4 Ansible scripts now default monitor.*3
In order to run StackTach.v2 and StackTach.v2 in parallel,
we need to have two queue from OpenStack (vs. having to change
--notification_topics to include a new queue).

This approach makes a new queue with the same routing_key off the
same exchange, resulting in mirrored queues which can be consumed
separately.

Exchange "nova" -> (routing key "monitor.info") -> Queue "monitor.info"
                -> (routing key "monitor.info") -> Queue "monitor.info3"

Change-Id: Icce87c7eb43025672759d442553feb325e186128
2015-02-04 15:10:09 -08:00
Sandy Walsh
4c2672e8c1 Operators Guide etherpad link
So we can make quick edits without requiring commits.

Change-Id: I83cae960f05c12a5b201df33e1a109bd4bb7874c
2015-01-30 13:10:18 -08:00
Sandy Walsh
85380c4f74 Fix silly mistake with api results.
Was using the worker information. Not the api nodes.

Change-Id: I66f36931ba4c53df49a62670e3cd60f17f22c7c2
2015-01-28 12:14:43 -08:00
Sandy Walsh
b636a67497 Include monitor.error queue
Also buffed stv3_status.py to handle api nodes.

Change-Id: I70bfc55007d8ced37f47c27c3e31753d42e83492
2015-01-28 06:50:36 -08:00
Sandy Walsh
c6e111f645 Let gunicorn accept from all addresses
Still running on 8000 since we're running at stv3 user.

Change-Id: I50cb731852523d1a08b45e51536e4d178b42975b
2015-01-26 07:00:24 -08:00
Sandy Walsh
a28a545e47 stv3_status tweaks
More efficient calls. Proper outut.

Change-Id: I2511e60cf084db48b88874f43764ce20b8f361c9
2015-01-23 11:14:01 -08:00
Jenkins
46bdb0f2e5 Merge "stv3_status.py" 2015-01-23 18:15:16 +00:00
Sandy Walsh
0e5a5ab9ab Sync ansible files/templates with usage files
Looks like we were using the default config files in the ansible
scripts when we should be using the Usage related ones.

Change-Id: Id30152ab7762aff85f3258552de0b5a3e0a6e876
2015-01-22 09:38:12 -08:00
Sandy Walsh
fbcd9218b5 stv3_status.py
Utility for extracting worker and rabbit information from
large STv3 deployments. Easier than having to check many
different hosts. See etc/status.yaml for sample config file.

Change-Id: If1b39ce3b34235f3bf0eac4ea707be2087ca14ad
2015-01-22 07:14:16 -08:00
Jenkins
804db37954 Merge "my.cnf was going to wrong location." 2015-01-07 20:31:35 +00:00
Sandy Walsh
f21724afb9 Missing libmysqlclient-dev
fixes 'EnvironmentError: mysql_config not found' error.

Change-Id: I558c66a87fa89435c2d1942f584c810235df6f20
2015-01-07 12:29:29 -08:00
Sandy Walsh
b99a74198f my.cnf was going to wrong location.
Got schooled on the difference between .my.cnf and my.cnf. This
should be correct now.

Change-Id: I5f23ae5dc9e824153fb9e85a11e09e422ed1e0f2
2015-01-07 11:26:28 -08:00
Sandy Walsh
ff7a73ba86 Fix missing timex library
Wasn't git cloning the timex library. It was working because timex
was on pypi.

Change-Id: If5072d17f48ecf82b0216eacca433cb636b1961a
2015-01-05 05:37:29 -08:00
Sandy Walsh
618cdcdd63 Cell Support
Yagi-workers are now created per cell. One rabbit per cell.
Also renamed the common role to stv3-common to prevent conflicts
with other playbooks.

Take note of ansible/roles/stv3-workers/defaults/main.yaml to
see an example of the expected format of the calling facts file.
Each entry in this list-of-dictionaries represents a rabbit server
and Swift container for that cell.

Also, multiple pipeline_workers are created based on the value
of num_pipeline_workers (defaults to 2). This is so a machine
can better utilize multi-core architectures.

Change-Id: I09c4d364878493546d6e23e019f01f5f4f38c37c
2014-12-22 10:14:37 -08:00
Sandy Walsh
51bfd3f12a Fix services not starting on first run.
Services now run properly on first run (thanks Joel). Didn't
need to check the changed status of the command after all.

Also, spelling mistake in the playbook launch fixed.

Change-Id: I5b6cb276b066844c934da4ffbd2513a006033214
2014-12-16 07:53:17 -08:00
Sandy Walsh
f4484a9a9d Remove cgit from branching.
Seems to be a change in how infra defines the stackforge repo.

Also removed a stv3-api restart from common, which was undefined.

Change-Id: I3c33dd6db467225444a0a9da137bf8f002629e3a
2014-12-15 08:46:55 -08:00
Sandy Walsh
24f118f9ac tox verification and tarball venv packaging support.
Optionally run tox on each git repo (-t) and package resultant
venv into a tarball on success (-p)

tarball filename includes the short-sha1 of the repo.

If tarball is created, screen is not run.

if the -d option is given the cmdline incantation for kicking
off the ansible deploy playbook will be provided. Later, this
could be automated.

A mysql role playbook is also included for single node db.

stv3-api role is also included.

Change-Id: Ib9f97a0035c228d4a163d7794ab7ce60043d572b
2014-12-08 13:15:43 -08:00