x/turbo-hipster
Code Issues Proposed changes
165 Commits 1 Branch 0 Tags
690db7e166
Commit Graph

165 Commits

Author SHA1 Message Date
Michael Still
690db7e166 Regenerate historical data, add auto generated data config 
Change-Id: I04a2d7117338dad49e84b7d7d91c81ed405e6674
 2014-01-01 18:36:44 +11:00
Michael Still
81ebd10bdf Implement historical graphing. 
Change-Id: Ieaf347baec7ccbbbd7029d4b2e671414ab7425fb
 2014-01-01 09:37:16 +11:00
Michael Still
3c65b637a4 Historical analysis of migration times. 
Break out the log parser so I can reuse it, add a couple of tools
to analyse test logs to generate historical timings.

Fixes bug: 1263873
DocImpact: we should document the methodology here.

Change-Id: I23a66e17612daa20bcb1beda59af1e6bd4f1b276
 2014-01-01 09:37:16 +11:00
Jenkins
ec19f69f9c Merge "More worker build documentation." 2013-12-31 22:05:42 +00:00
Michael Still
bae0304995 Always build a new git repo. 
This will only slow down test runs a tiny bit, but ensures we are
never polluted with something from another test run.

Change-Id: I3a9e4c13b289b14f82a6a8c9a821a0abc36876d2
Closes-bug: #1264947
 2013-12-30 21:07:59 +11:00
Michael Still
9e814216d1 More worker build documentation. 
Also a small tweak to how to setup venvs to put them somewhere
more obvious.

Change-Id: I140de9aad4e7609edf0d1ab8888e3d224bb3606b
 2013-12-29 22:02:20 +11:00
Jenkins
26be80f7bb Merge "Check the return code for the test script." 2013-12-27 00:35:39 +00:00
Michael Still
5231d4c50b Check the return code for the test script. 
Rework the error reporting from the dataset log analysis to take
into account global failures as well, and only report success if
both global success and dataset success have been achieved.

Fixes bug: 1263828

Change-Id: I53aa5ab949293a02f93f9981c0f98246a77d825f
 2013-12-25 08:56:36 +11:00
Michael Still
16028a9543 Check if we built the venv correctly 
A simple check for venv creation.

Partial fix bug: 1263840

Change-Id: Ic585cd97a53871df1058a99b0d3db5ff553342f4
 2013-12-24 19:42:59 +11:00
Michael Still
49cf70e748 Use netns to block network access from nova-manage. 
This is done for security reasons as we don't trust the code running
inside migrations.

To run this:
 - bind mysql to 0.0.0.0
 - run makenetnamespace.sh as root at boot, passing in the mysql root
   password so that user perms can be setup
 - add this to the sudo config:
   turbo-hipster ALL=(root) NOPASSWD: /sbin/ip netns exec nonet *

Change-Id: I86190fbd515ecf7683194923df14e5b707ab21c5
 2013-12-24 19:42:50 +11:00
Joshua Hesketh
61daba46bb Fix paramaters passed to database downgrades 
Change-Id: I8f9e7373640b0bf9953f3bc2363391d1e001ca01
 2013-12-18 13:55:42 +11:00
Joshua Hesketh
a2276ace7b Only test downgrades to the last version in stable 
Change-Id: I75a43667279ee82ca7f3c63c640460e20f430c34
 2013-12-18 12:57:12 +11:00
Joshua Hesketh
47caf5f3f3 fix dash in warning message 
Change-Id: I2347fed8ad571b7e7cb17dffabe5de010c34c40f
 2013-12-17 15:29:59 +11:00
Joshua Hesketh
18ac7140ef Add a space after FAILURE message 
Change-Id: I47c6968f55b60ab479f951fa5079d1b82d074535
 2013-12-17 13:02:59 +11:00
Joshua Hesketh
a581752a93 Return the actual error message back to zuul 
Change-Id: Ib615be2bfb16cd92914626c89fb72db0e3c6a382
 2013-12-17 11:50:21 +11:00
Joshua Hesketh
dce07dfd44 Add back in multiple dataset support 
Since most of the code is there we should at least try and support multiple datasets.

Change-Id: I83a4994919b376335ffbfb1fa40c4ecc9c7f9aeb
 2013-12-17 11:40:30 +11:00
Joshua Hesketh
d56dff89b3 Update requirements for swift/keystone 
Change-Id: I843b3abfb0088c16f5fb24a6cc4849e6154d5f79
 2013-12-16 12:30:39 +11:00
Joshua Hesketh
d8b23ad1d6 Return the swift object name for the log url 
Change-Id: I43bdbd8d8f05c26de4f9961646ec672095befea3
 2013-12-16 12:26:49 +11:00
Michael Still
a497d3a48f Fix swift push 
Change-Id: I86c814330b32aa9f27868d5b815f969f5545171f
 2013-12-12 14:21:23 +11:00
Michael Still
09258822b2 Script to upload old logs to swift 
Change-Id: I07ea7d0394bde0e2508d87b22ce64d2822f02649
 2013-12-12 14:21:20 +11:00
Michael Still
4e100ca3b4 Use the last log URL. 
Change-Id: Ic5e6c1d72128dceff0b8789da9f4085a4fd0a260
 2013-12-12 14:20:36 +11:00
Michael Still
32a267ef58 Stop generating index files. 
The index files are doubling the number of inodes we take, and we can
generate them on the fly with the log serving shim.

Change-Id: I2adcc4b4736892fbfa4afead83d04bc17aceb4a1
 2013-12-12 14:20:31 +11:00
Joshua Hesketh
b21b8c38c1 Add in some comments for thoughts 
Change-Id: If902f8bd723ac273dff6cf1b1014e9ce2d402109
 2013-12-12 14:12:28 +11:00
Joshua Hesketh
0df6286925 Ensure working dir is in correct state for checks 
Remove the cleanup so that the branch is in the correct state for the hanlde_results to look at the repository.

Change-Id: I94160a82600c15fe0b3acb6409f6837ea43c2312
 2013-12-12 14:12:28 +11:00
Michael Still
99c285411d Pass all the args to db_sync. 
Change-Id: I1359a72768a4c6311b3d7ecaf1516b3b50f86c4c
 2013-12-12 14:12:28 +11:00
Michael Still
d0ccf15382 Havana upgrade should use Havana. 
Change-Id: I3e5e6e282bea8941bafb5b30b19f8dffab0fcd32
 2013-12-12 14:12:26 +11:00
Michael Still
66db25e785 Log where we tried to find a pre-entrypoint nova-manage. 
Change-Id: I90468a2f14338afed741886c092b7521139f0c54
 2013-12-12 14:11:19 +11:00
Michael Still
7601a6363f nova-manage should be verbose 
Change-Id: I8e026fee48458767f51b65cb65973b6635e42203
 2013-12-12 14:11:14 +11:00
Michael Still
ca869abb93 Support aborting early. 
Change-Id: I399936406f16b0357400cae20a1ee1061a61d9b3
 2013-12-12 14:11:09 +11:00
Michael Still
ef0f711e35 Only walk back to the start of trunk 
Change-Id: I54b326f0dfb7ec82aa6ac4d58863adf3608a42e2
 2013-12-12 14:11:04 +11:00
Michael Still
2401b92c76 Fix path to old style nova manage 
Change-Id: I125056b7c8ddfc41ecb4bf74b4536cba3b9a5acf
 2013-12-12 14:11:00 +11:00
Michael Still
e0782599ac Pass the DB name as well, get the schema version check command right. 
Change-Id: I3fdab7ec46aff6f9ba395701fbe6d47b66d741bc
 2013-12-12 14:10:55 +11:00
Michael Still
30bee6a6cd You have to walk old dbs through stable releases. 
Change-Id: Ibec1b0277846f40c0a8dcc08dd7128c3033b8449
 2013-12-12 14:10:51 +11:00
Michael Still
e85d1e0e99 Make cleanup optional via an env var. 
Change-Id: Ifcaecd22f6d57cd844ed33423ec01035ee9ca490
 2013-12-12 14:10:47 +11:00
Michael Still
f1bd4dbcec More logging in the testing script. 
Change-Id: I79fc36b1c780250eb6d0343947a0186561ff5721
 2013-12-12 14:10:42 +11:00
Michael Still
732d25cfaf Fix typo'ed variable. 
Change-Id: I15f9b6193d2192fc4ffc7369949a2bfc1c0b85c6
 2013-12-12 14:01:51 +11:00
Joshua Hesketh
4d5ba72081 Check the logs for migrations that take too long 
Change-Id: I38cbce3771afea8dc2ffecc243d727bc1759431e
 2013-12-12 13:39:07 +11:00
Joshua Hesketh
b39f8840d6 Make sure we wait for a gearman server 
Change-Id: I4cc5f5b6fb50a77116aeede4a450bd8281a5a5a9
 2013-11-21 13:12:00 +11:00
Joshua Hesketh
3baaa0d160 Update log format 
Change-Id: I5c8673ef713da43802837093753491ca31d07dd2
 2013-11-21 11:05:37 +11:00
Joshua Hesketh
fc44906a7b Fix typo in debug logging 
Change-Id: I562b0c13c660bd65f562d3e5003c8f0e8c6a7a04
 2013-11-21 10:37:27 +11:00
Joshua Hesketh
8d0f241ead Remove old thread code 
Change-Id: I8aaf92d6fe3ad21853fb21d11c98d889d54953d5
 2013-11-21 10:37:27 +11:00
Joshua Hesketh
6eb5fdcd8d Add a bunch of debugging info 
Change-Id: Idda0e7e79fa29aa458947356ddaed9d47d0a1d46
 2013-11-21 10:37:27 +11:00
Joshua Hesketh
b305b4e3e6 update install requirements 
Change-Id: I4bfe853dc2c00358e070d09bcb477597a680a655
 2013-11-21 10:37:27 +11:00
Joshua Hesketh
8326907163 Fix config setup order 
Change-Id: I8cb584e6918797d9c7b197d629db7dad08073059
 2013-11-21 10:37:27 +11:00
Joshua Hesketh
d61c805dae Fix typo in mysql database loading 
Change-Id: I85c96900aef64674fdf11f6821bafa06ad393e97
 2013-11-21 10:37:27 +11:00
Joshua Hesketh
4343b956c5 Refactor th to have one gearman worker thread 
Change-Id: I78f95a67b80ce0627b4a3bbb20578f3d16028714
 2013-11-21 10:37:27 +11:00
Joshua Hesketh
527966b2e4 Have a different worker name for each thread. 
This makes each thread create their own git repo so threads don't
compete and cause race conditions

Change-Id: Ifaf1faf153d9b26a0ff302b00603573b04205b04
 2013-11-21 10:37:27 +11:00
Joshua Hesketh
3b7765de0d Add .gitreview file and fix documentation build 
Change-Id: Ic95b7a2c04e0578f89d76bd77ca34301776658c8
 2013-11-21 10:31:33 +11:00
Joshua Hesketh
e8a5b83528 Revert "capture stderr into stdout" 
This reverts commit 94ca7c271fe6d36d2eff7f0ad24dadc7fad1caea.
 2013-10-09 16:13:21 +11:00
Joshua Hesketh
94ca7c271f capture stderr into stdout 2013-10-07 10:29:46 +11:00