Go to file

Michael Still fa30f12e0d Handle the new format for migrations from the LogParser.

This patch just converts the world to the new format for the
migrations found by LogParser. I've also moved the migration time
check method to a more generic name, as I am about to use it to
test other attributes of migrations.

Change-Id: Ibf11f3ea0932c1a05caa060ceaa48308b8b5e2e5

2014-01-15 13:26:50 +11:00

datasets

Regenerate historical data, add auto generated data config

2014-01-01 18:36:44 +11:00

doc

Add unit tests for execute_to_log.

2014-01-10 14:09:10 +11:00

etc

Better cost metrics.

2014-01-14 11:45:49 +11:00

tests

Handle the new format for migrations from the LogParser.

2014-01-15 13:26:50 +11:00

tools

Script to upload old logs to swift

2013-12-12 14:21:20 +11:00

turbo_hipster

Handle the new format for migrations from the LogParser.

2014-01-15 13:26:50 +11:00

.gitignore

Check the logs for migrations that take too long

2013-12-12 13:39:07 +11:00

.gitreview

Add .gitreview file and fix documentation build

2013-11-21 10:31:33 +11:00

.testr.conf

Move tests to root folder

2013-08-12 10:40:27 +10:00

LICENSE

add in license

2013-08-12 23:25:35 +10:00

README.md

turbo-hipster

A set of CI tools.

worker_server.py is a worker server that loads and runs task_plugins.

Each task_plugin is a zuul gearman worker that implements, handles, executes a job, uploads/post-processes the logs and sends back the results to zuul.

Plugins

gate_real_db_upgrade: Runs the db_sync migrations on each dataset available in the datasets subdir.

Installation

boot a fresh Ubuntu image
setup ssh authentication for your admin team
apt-get update; apt-get dist-upgrade
adduser th
apt-get install vim git python-pip python-setuptools python-keystoneclient virtualenvwrapper python-eventlet python-numpy python-mysqldb python-git python-gitdb python-netaddr python-pkg-resources libxml2-dev libxml2-utils libxslt-dev git-review libxml2-dev libxml2-utils libxslt-dev libmysqlclient-dev pep8 postgresql-server-dev-9.1 python2.7-dev python-coverage python-netaddr
pip install -U pip
apt-get purge python-pip
cd /home/th; git clone http://github.com/stackforge/turbo-hipster
apply any patches you need
python setup.py install
cp turbo_hipster/task_plugins/gate_real_db_upgrade/*.sh /usr/local/lib/python2.7/dist-packages/turbo_hipster/task_plugins/gate_real_db_upgrade/
cp -R etc/* /etc/
mkdir /var/lib/turbo-hipster
chown -R th.th /var/lib/turbo-hipster
mkdir /var/log/turbo-hipster
chown -R th.th /var/log/turbo-hipster
install your chosen MySQL-like database engine (percona, maria, mysql)
mysql -u root --password=$1 -e "create user 'nova'@'localhost' identified by 'tester';"
mysql -u root --password=$1 -e "create user 'nova'@'172.16.0.2' identified by 'tester';"
mysql -u root --password=$1 -e "grant all privileges on . to 'nova'@'localhost' with grant option;"
mysql -u root --password=$1 -e "grant all privileges on . to 'nova'@'172.16.0.2' with grant option;"
/etc/rc.local
rsync the datasets from the master
logrotate -f /etc/logrotate.conf
chmod -R ugo+r /var/log/*
chmod ugo+rx /var/log/mysql
mkdir /var/cache/pip
chmod -R ugo+rwx /var/cache/pip
touch /var/log/mysql/slow-queries.log
/etc/init.d/turbo-hipster start