12 Commits

Author SHA1 Message Date
Levi Blackstone
3b7e69c972 Add PEP8 check and fix related issues
- Add PEP8 section to tox.ini
- Add hacking to requirements to enforce OpenStack style requirements
- Fix formatting issues flagged by flake8 check
- Add copyright notices to all remaining files
- Update .gitignore file

Change-Id: I7e9a0203ddf2002c08ac96800fe30c1c46ebba88
2015-05-05 07:43:17 -05:00
Sandy Walsh
8fc4b88519 Skip tarball and just gzip the archive file.
It's just a single file, so it makes sense to just gzip it.

Change-Id: I0103269a9e365fdbcb30c1389cf6f3eabfce1a23
2015-04-15 13:40:31 -07:00
Sandy Walsh
7a600792db One notification per file line in tarball.
JsonRollManager also now rotates after time (15min default).
The contents of the tarball are a single file that contains
one notification per file line.

The tarball is created after reaching a certain size or after
a fixed amount of time. Any working files are also tarball'ed
when the service restarts.

Change-Id: I6e9d620086cf24bc1468b638e7b148af7230e2ac
2015-04-10 09:31:45 -07:00
Sandy Walsh
aecc00d485 use in-memory count for directory size.
Only check the directory size when we first start the
RollManager. From then on, incrementally add the payload
size. This will result in a difference between on-disk size
vs. file sizes. But, it's much faster than checking every N
times ... those stats() checks are very expensive.

Change-Id: I71b1455db0ea073a4f563f115a693027de50638b
2015-03-19 12:28:55 -07:00
Sandy Walsh
5f91c87739 Tarball archiving of JSON messages.
Changed the JSON RollManager to write JSON payloads
to a working directory until a certain directory
size is met. Then the directory is tar/gzipped and
moved into a destination directory.

No CRC is available for the tarball.

Change-Id: I119da646104eeaea86f7153a9bcf123d3143a0ba
2015-02-24 11:28:20 -08:00
Sandy Walsh
852c1853e5 Simple per-event file output.
Records the event as a json payload in a file that
may include a CRC of the payload. This is typically used
when storing events in something like HDFS.

Change-Id: Iaecca2397afa3501e437e27d698f3573760f1ac0
2015-02-17 07:14:02 -08:00
Sandy Walsh
a3e80502f6 Get tox tests working again.
Had to upgrade to using template-based notigen. Unfortunately this
means copy-pasting the templates from notigen (for now).

Also bump version and change repo to stackforge.

Change-Id: Idfd97ec5c6d2cf43c842e001946d8fa8801c8dfb
2014-10-22 11:32:26 -07:00
Sandy Walsh
3b65cab8c2 Moved utils to notification_utils project 2014-05-21 12:11:50 +00:00
Sandy Walsh
12878c8ce3 reading works and integration end-to-end tests in place 2014-05-14 20:59:48 +00:00
Sandy Walsh
2bb11b0d31 added ArchiveCallback, WritingRollManager working 2014-05-14 14:08:39 +00:00
Sandy Walsh
1c6fae4b3e Started on integration testing.
Brought over and refactored event generator code from TwoBillion.
Now supports real-time and fast-time event generation.
Also in an easier-to-use library now.
2014-05-13 19:52:07 +00:00
Sandy Walsh
e69b617a24 working on tests for disk_storage 2014-05-13 14:02:27 +00:00