33 Commits

Author SHA1 Message Date
Levi Blackstone
3b7e69c972 Add PEP8 check and fix related issues
- Add PEP8 section to tox.ini
- Add hacking to requirements to enforce OpenStack style requirements
- Fix formatting issues flagged by flake8 check
- Add copyright notices to all remaining files
- Update .gitignore file

Change-Id: I7e9a0203ddf2002c08ac96800fe30c1c46ebba88
2015-05-05 07:43:17 -05:00
Sandy Walsh
8fc4b88519 Skip tarball and just gzip the archive file.
It's just a single file, so it makes sense to just gzip it.

Change-Id: I0103269a9e365fdbcb30c1389cf6f3eabfce1a23
2015-04-15 13:40:31 -07:00
Sandy Walsh
7a600792db One notification per file line in tarball.
JsonRollManager also now rotates after time (15min default).
The contents of the tarball are a single file that contains
one notification per file line.

The tarball is created after reaching a certain size or after
a fixed amount of time. Any working files are also tarball'ed
when the service restarts.

Change-Id: I6e9d620086cf24bc1468b638e7b148af7230e2ac
2015-04-10 09:31:45 -07:00
Sandy Walsh
2be257bbfd Remove Swift handler (for now)
Pyrax is deprecated and causing installation problems.
Nuclear option.

Change-Id: I2d27a541e6617e468862fd13c473ea0ce7bc774b
2015-03-27 09:42:17 -07:00
Sandy Walsh
346fd39d24 Only compute directory size when delete is done.
Gah. Was doing this on every delete operation.

Change-Id: Ie207fd94b713c550265676716a7d206fc20763de
2015-03-26 13:14:11 -07:00
Sandy Walsh
aecc00d485 use in-memory count for directory size.
Only check the directory size when we first start the
RollManager. From then on, incrementally add the payload
size. This will result in a difference between on-disk size
vs. file sizes. But, it's much faster than checking every N
times ... those stats() checks are very expensive.

Change-Id: I71b1455db0ea073a4f563f115a693027de50638b
2015-03-19 12:28:55 -07:00
Sandy Walsh
5f91c87739 Tarball archiving of JSON messages.
Changed the JSON RollManager to write JSON payloads
to a working directory until a certain directory
size is met. Then the directory is tar/gzipped and
moved into a destination directory.

No CRC is available for the tarball.

Change-Id: I119da646104eeaea86f7153a9bcf123d3143a0ba
2015-02-24 11:28:20 -08:00
Sandy Walsh
852c1853e5 Simple per-event file output.
Records the event as a json payload in a file that
may include a CRC of the payload. This is typically used
when storing events in something like HDFS.

Change-Id: Iaecca2397afa3501e437e27d698f3573760f1ac0
2015-02-17 07:14:02 -08:00
Sandy Walsh
01cdd7ab7d create container 2014-06-03 03:33:48 +00:00
Sandy Walsh
29ec822b67 CloudFiles upload handler and setup.cfg tweak 2014-06-03 02:15:48 +00:00
Sandy Walsh
1ef14d82bf Start of swift support 2014-05-30 19:33:46 +00:00
Sandy Walsh
40c3ac1920 Proper unicode support for metadata. Json doesn't need it 2014-05-29 20:13:51 +00:00
Sandy Walsh
a180835028 CallbackList now correctly uses objects, not classes 2014-05-27 01:00:05 +00:00
Sandy Walsh
8facfff750 Chain of callback support using simport 2014-05-26 19:04:27 +00:00
Sandy Walsh
3b65cab8c2 Moved utils to notification_utils project 2014-05-21 12:11:50 +00:00
Sandy Walsh
6753b9baba Some play callback handlers. Now uses notigen 2014-05-16 00:31:04 +00:00
Sandy Walsh
12878c8ce3 reading works and integration end-to-end tests in place 2014-05-14 20:59:48 +00:00
Sandy Walsh
2bb11b0d31 added ArchiveCallback, WritingRollManager working 2014-05-14 14:08:39 +00:00
Sandy Walsh
331c9e0caa pulled in datetime/decimal code, fixed up json encoding, started tying into disk_storage in rolling test 2014-05-14 03:50:12 +00:00
Sandy Walsh
1c6fae4b3e Started on integration testing.
Brought over and refactored event generator code from TwoBillion.
Now supports real-time and fast-time event generation.
Also in an easier-to-use library now.
2014-05-13 19:52:07 +00:00
Sandy Walsh
159054a53d 100% coverage 2014-05-13 14:15:05 +00:00
Sandy Walsh
e69b617a24 working on tests for disk_storage 2014-05-13 14:02:27 +00:00
Sandy Walsh
2a2bd5e54c tests work again and no temp files 2014-05-13 02:48:47 +00:00
Sandy Walsh
2dee06e43d split out shoebox into smaller files 2014-05-13 01:42:46 +00:00
Sandy Walsh
72a603e58b moved experiment into shoebox code 2014-05-12 21:23:42 +00:00
Sandy Walsh
ef8fc4237e Back to looking at raw binary 2014-03-22 01:56:47 -03:00
Sandy Walsh
366e84d2fd finish tests. Starting on writing 2014-03-12 15:09:29 -03:00
Sandy Walsh
4c0e33229a more tests 2014-01-13 19:58:44 +00:00
Sandy Walsh
cf27ef257b fleshing out tests 2014-01-10 20:50:44 +00:00
Sandy Walsh
49f05ca5ac packaging working 2014-01-10 16:31:30 +00:00
Sandy Walsh
19b4898244 starting on tests 2014-01-10 16:24:05 +00:00
Sandy Walsh
11193f894b starting to flesh it out a little more 2014-01-09 22:48:05 +00:00
Sandy Walsh
08cb1c0718 sketching out the api 2014-01-09 20:01:14 +00:00