x/kloudbuster
Code Issues Proposed changes

Cleanup logic enhancements, document enhancements

Browse Source 
1. Enhance the cleanup logic;
2. Add the missing keypair into the resource logger;
3. Document enhancements on installation and contribution;
4. Fix the bug when parsing the image version string in DIB;
5. Fix the typo in Rest API specification;

Change-Id: Ibf4a1c7828ee4522938b4b55f9a8af5e6c6a133b
This commit is contained in:
Yichen Wang 2015-09-02 11:42:20 -07:00
parent 525010a2d5
commit d7d9575dda
15 changed files with 372 additions and 165 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
CONTRIBUTING.rst
View File

@ -1,23 +0,0 @@
Feedbacks and contributions to KloudBuster are welcome.
KloudBuster follows the same workflow as any other OpenStack project.
If you would like to contribute to the development of OpenStack, you must
follow the steps in this page:
`<http://docs.openstack.org/infra/manual/developers.html>`_
If you already have a good understanding of how the system works and your
OpenStack accounts are set up, you can skip to the development workflow
section of this documentation to learn how changes to OpenStack should be
submitted for review via the Gerrit tool:
`<http://docs.openstack.org/infra/manual/developers.html#development-workflow>`_
Pull requests submitted through GitHub will be ignored.
Feedbacks and Bugs should be filed on Launchpad, not GitHub:
`<https://bugs.launchpad.net/kloudbuster>`_

23
README.rst
View File

@ -1,6 +1,6 @@
=============================== ========
KloudBuster Overview
=============================== ========
How good is your OpenStack data plane under real heavy load? How good is your OpenStack data plane under real heavy load?
@ -50,15 +50,9 @@ If you have any feedbacks or would like to make small or large contributions, si
openstack-dev@lists.openstack.org with a '[kloudbuster]' tag in the subject. openstack-dev@lists.openstack.org with a '[kloudbuster]' tag in the subject.
Links
-----
* Source: `<http://git.openstack.org/cgit/openstack/kloudbuster>`_
* Bugs: `<http://bugs.launchpad.net/kloudbuster>`_
* Documentation: `<http://kloudbuster.readthedocs.org>`_
Licensing Licensing
--------- ---------
KloudBuster is licensed under the Apache License, Version 2.0 (the "License"); KloudBuster is licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License. you may not use this file except in compliance with the License.
You may obtain a copy of the License at `<http://www.apache.org/licenses/LICENSE-2.0>`_ You may obtain a copy of the License at `<http://www.apache.org/licenses/LICENSE-2.0>`_
@ -71,3 +65,12 @@ limitations under the License.
KloudBuster VM images contain multi-open source license components KloudBuster VM images contain multi-open source license components
Links
-----
* Documentation: `<http://kloudbuster.readthedocs.org>`_
* Source: `<http://git.openstack.org/cgit/openstack/kloudbuster>`_
* Supports/Bugs: `<http://launchpad.net/kloudbuster>`_
* Mailing List: kloudbuster-core@lists.launchpad.net

91
doc/source/contributing.rst
View File

@ -1,4 +1,93 @@
============ ============
Contributing Contributing
============ ============
.. include:: ../../CONTRIBUTING.rst
Contribute to KloudBuster
-------------------------
Below are a simplified version of the workflow to work on KloudBuster. For complete instructions, you have to follow the Developer's Guide in OpenStack official documents. Refer to :ref:`below section <developer_guide_of_openstack>` for links.
Start working
^^^^^^^^^^^^^
Before starting, a GitHub/OpenStack respository based installation must be done. Refer :ref:`here <git_installation>` for detailed documentation.
1. From the root of your workspace, check out a new branch to work on::
$ git checkout -b <TOPIC-BRANCH>
2. Happy working on your code for features or bugfixes;
Before Commit
^^^^^^^^^^^^^
There are some criteria that are enforced to commit to KloudBuster. Below commands will perform the check and make sure your code complys with it.
3. PEP 8::
$ tox -epep8
.. note:: The first run usually takes longer, as tox will create a new virtual environment and download all dependencies. Once that is the done, further run will be very fast.
4. Run the test suite::
$ tox -epython27
5. If you made a documentation change (i.e. changes to .rst files), make sure the documentation is built as you expected::
$ cd <kloudbuster-ws-root>/doc
$ make html
Once finished, the documentation in HTML format will be ready at <kloudbuster-ws-root>/doc/build/html.
Submit Review
^^^^^^^^^^^^^
6. Commit the code::
$ git commit -a
.. note:: For a feature commit, please supply a clear commit message indicating what the feature is; for a bugfix commit, please also containing a launchpad link to the bug you are working on.
7. Submit the review::
$ git review <TOPIC-BRANCH>
The members in the KloudBuster team will get notified once the Jenkin verification is passed. So watch your email from the review site, as it will contain the updates for your submission.
8. If the code is approved with a +2 review, Gerrit will automatically merge your code.
File Bugs
---------
Bugs should be filed on Launchpad, not GitHub:
https://bugs.launchpad.net/kloudbuster
.. _developer_guide_of_openstack:
Developer's Guide of OpenStack
------------------------------
Feedbacks and contributions to KloudBuster are welcome.
KloudBuster follows the same workflow as any other OpenStack project.
If you would like to contribute to the development of OpenStack, you must
follow the steps in this page:
`<http://docs.openstack.org/infra/manual/developers.html>`_
If you already have a good understanding of how the system works and your
OpenStack accounts are set up, you can skip to the development workflow
section of this documentation to learn how changes to OpenStack should be
submitted for review via the Gerrit tool:
`<http://docs.openstack.org/infra/manual/developers.html#development-workflow>`_
Pull requests submitted through GitHub will be ignored.

2
doc/source/index.rst
View File

@ -4,7 +4,7 @@
contain the root `toctree` directive. contain the root `toctree` directive.
Welcome to the KloudBuster documentation! Welcome to the KloudBuster documentation!
======================================================== =========================================
Contents: Contents:

258
doc/source/installation.rst
View File

@ -2,153 +2,243 @@
Installation Installation
============ ============
Binary Installation There are two ways to install and run KloudBuster tool. Users of KloudBuster should use regular PyPI based installation, while developers of KloudBuster should use GitHub/OpenStack Repository based installation. Normally, PyPI based installation will satisfy most of use cases, and it is the recommended way for running KloudBuster under production environments, or through an automated or scheduled job. A git repository based installation gives more flexibility, and it is a must for developers of KloudBuster.
-------------------
This is the recommended way to install KloudBuster for non-development use. .. note:: Installation from PyPI will only have the latest stable version.
KloudBuster is available in the Python Package Index (PyPI)::
`KloudBuster PyPI <https://pypi.python.org/pypi/KloudBuster>`_ PyPI based Installation
-----------------------
You will need to have python 2.7 and pip installed before installing KloudBuster. This is the recommended way to install KloudBuster for non-development use, and KloudBuster is available in the Python Package Index (PyPI): `KloudBuster PyPI <https://pypi.python.org/pypi/KloudBuster>`_
At the command line:: Step 1
^^^^^^
$ pip install kloudbuster You will need to have python 2.7, pip, and some dependencies installed before installing KloudBuster, run the command based on your distro.
Or, if you have `virtualenv <https://pypi.python.org/pypi/virtualenv>`_ installed:: Ubuntu/Debian based:
.. code-block:: bash
$ sudo apt-get install python-dev python-pip python-virtualenv
$ sudo apt-get install libxml2-dev libxslt-dev libffi-dev libz-dev libyaml-dev libssl-dev
RHEL/Fedora/CentOS based:
.. code-block:: bash
$ sudo yum install python-devel python-pip python-virtualenv
$ sudo yum install libxml2-devel libxslt-devel libffi-devel libyaml-devel openssl-devel
MacOSX:
.. code-block:: bash
$ # Download the XCode command line tools from Apple App Store
$ xcode-select --install
$ sudo easy_install pip
$ sudo pip install virtualenv
Step 2
^^^^^^
Create a virtual environment for Python, and install KloudBuster:
.. code-block:: bash
$ virtualenv vkb $ virtualenv vkb
$ source vkb/bin/activate $ source vkb/bin/activate
$ pip install kloudbuster $ pip install kloudbuster
Or, if you have `virtualenvwrapper <https://virtualenvwrapper.readthedocs.org>`_ installed:: Alternatively, if you have `virtualenvwrapper <https://virtualenvwrapper.readthedocs.org>`_ installed:
.. code-block:: bash
$ mkvirtualenv kloudbuster $ mkvirtualenv kloudbuster
$ pip install kloudbuster $ pip install kloudbuster
.. note::
"A Virtual Environment is a tool to keep the dependencies required by different projects in separate places, by creating virtual Python environments for them." It is optional but recommended. We could use::
To verify kloudbuster is installed, just type $ sudo pip install kloudbuster
.. code:: instead if isolation among multiple Python projects is not needed.
kloudbuster --help
Source Installation To verify kloudbuster is installed, just type::
-------------------
For code development, clone the kloudbuster git repository::
git clone https://github.com/openstack/kloudbuster.git kloudbuster --help
Then install dependencies (after optionally creating and activating a virtual env):: .. _git_installation:
cd kloudbuster GitHub/OpenStack Repository based Installation
pip install -r requirements.txt ----------------------------------------------
pip install -r test-requirements.txt
To verify kloudbuster is installed, just type It is recommended to run KloudBuster inside a virtual environment. However, it can be skipped if installed in a dedicated VM.
.. code::
python kloudbuster/kloudbuster.py --help
VM Image Upload Super quick installation on Ubuntu/Debian
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: bash
$ sudo apt-get install python-dev python-virtualenv git git-review qemu-utils
$ sudo apt-get install libxml2-dev libxslt-dev libffi-dev libz-dev libyaml-dev libssl-dev
$ # create a virtual environment
$ virtualenv ./vkb
$ source ./vkb/bin/activate
$ git clone https://github.com/openstack/kloudbuster.git
$ cd kloudbuster
$ pip install -r requirements-dev.txt
Super quick installation on RHEL/Fedora/CentOS
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: bash
$ sudo yum install python-devel python-virtualenv git qemu-img
$ sudo yum install libxml2-devel libxslt-devel libffi-devel libyaml-devel openssl-devel
$ # create a virtual environment
$ virtualenv ./vkb
$ source ./vkb/bin/activate
$ git clone https://github.com/openstack/kloudbuster.git
$ cd kloudbuster
$ pip install -r requirements-dev.txt
Super quick installation on MacOSX
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KloudBuster can run natively on MacOSX. These instructions have been verified to work on MacOSX 10.10 (Yosemite).
First, download XCode from App Store, then execute below commands:
.. code-block:: bash
$ # Download the XCode command line tools
$ xcode-select --install
$ # Install pip
$ sudo easy_install pip
$ # Install python virtualenv
$ sudo pip install virtualenv
$ # create a virtual environment
$ virtualenv ./vkb
$ source ./vkb/bin/activate
$ git clone https://github.com/openstack/kloudbuster.git
$ cd kloudbuster
$ pip install -r requirements-dev.txt
Verify installation
^^^^^^^^^^^^^^^^^^^
To verify kloudbuster is installed, just type:
.. code-block:: bash
$ python kloudbuster/kloudbuster.py --help
Upload VM Image
--------------- ---------------
.. note::
If your OpenStack Glance can access the Internet, you can skip this section and you are done with the installation.
KloudBuster needs one "universal" test VM image (referred to as "KloudBuster image") that contains the necessary test software. The KloudBuster image is then instantiated in potentially large number of VMs by the KloudBuster application using the appropriate role (HTTP server, HTTP traffic generator...). KloudBuster needs one "universal" test VM image (referred to as "KloudBuster image") that contains the necessary test software. The KloudBuster image is then instantiated in potentially large number of VMs by the KloudBuster application using the appropriate role (HTTP server, HTTP traffic generator...).
Pre-built images are available for download from the `OpenStack App Catalog <http://apps.openstack.org>`_ (preferred method) or can be built from MacOSX using Vagrant or from any Linux server. Pre-built images are available for download from the `OpenStack App Catalog <http://apps.openstack.org>`_ (preferred method). For whatever reason the pre-built version doesn't work for you, the image can be re-built from MacOSX using Vagrant or from any Linux server. See :ref:`below section <build_vm_image>` for more details.
If your OpenStack Glance can access the Internet, you can skip the following section and **you are done with the installation**.
Manual Upload of the KloudBuster VM image Manual upload of the KloudBuster VM image
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
If Glance does not have access to http://storage.apps.openstack.org on the Internet, the KloudBuster VM image must be downloaded from the OpenStack App Catalog to an intermediate location then uploaded to Glance using either a Glance CLI command or using the Horizon dashboard. If Glance does not have access to http://storage.apps.openstack.org on the Internet, the KloudBuster VM image must be downloaded from the OpenStack App Catalog to an intermediate location then uploaded to Glance using either a Glance CLI command or via Horizon dashboard.
The KloudBuster VM image can be downloaded from `<http://apps.openstack.org/#tab=glance-images>`_ and look for an image named with the "kloudbuster_v" prefix and download the one that has the latest version.
KloudBuster VM images are qcow2 images named "kloudbuster_v<version>.qcow2" (e.g. "kloudbuster_v3.qcow2"). The KloudBuster VM image can be downloaded from `<http://apps.openstack.org/#tab=glance-images>`_. Look for an image named with the "kloudbuster_v" prefix and download the one that has the latest version.
The name of the image in Glance must match exactly the image name in the App Catalog (without the .qcow2 extension), for example to upload the image from a local copy of that image: KloudBuster VM images are qcow2 images named "kloudbuster_v<version>.qcow2" (e.g. "kloudbuster_v3.qcow2"). The name of the image in Glance must match exactly the image name in the App Catalog (without the .qcow2 extension), for example to upload the image from a local copy of that image:
.. code:: .. code-block:: bash
glance image-create --file kloudbuster_v3.qcow2 --disk-format qcow2 --container-format bare --name kloudbuster_v3 $ glance image-create --file kloudbuster_v3.qcow2 --disk-format qcow2 --container-format bare --is-public True --name kloudbuster_v3
Rebuild the Image .. _build_vm_image:
^^^^^^^^^^^^^^^^^
Only if using the pre-built version does not work (for whatever reason). Build VM image
--------------
The official pre-built image from OpenStack App Catalog should satisfy most of use cases. This option is just for development purposes, and for whatever reason the pre-built version does not work for you.
MacOSX with Vagrant Build on MacOSX
~~~~~~~~~~~~~~~~~~~ ^^^^^^^^^^^^^^^
You need to install first: You need to install first:
* `Virtualbox <https://cisco.jiveon.com/external-link.jspa?url=https://www.virtualbox.org/wiki/Downloads>`_ * `Virtualbox <https://www.virtualbox.org/wiki/Downloads>`_
* `Vagrant <https://cisco.jiveon.com/external-link.jspa?url=https://www.vagrantup.com/downloads.html>`_ * `Vagrant <https://www.vagrantup.com/downloads.html>`_
.. code:: And build the image with below commands:
# clone the kloudbuster repository if you have not done so .. code-block:: bash
git clone https://github.com/openstack/kloudbuster.git
# go to the dib directory $ # Clone the kloudbuster repository if you have not done so
cd kloudbuster/kloudbuster/dib $ git clone https://github.com/openstack/kloudbuster.git
# run vagrant and start building the image $ # Go to the dib directory
vagrant up $ cd kloudbuster/kb_dib
$ # Run vagrant and start building the image
$ vagrant up
After a few minutes (depending on virtualbox overhead), the qcow2 image will be built and available in the same directory. You can then upload it to OpenStack using the glance CLI, destroy the vagrant VM ("vagrant destroy") and dispose of the kloudbuster directory (if no longer needed). After a few minutes (depending on virtualbox overhead), the qcow2 image will be built and available in the same directory. You can then upload it to OpenStack using the glance CLI, destroy the vagrant VM ("vagrant destroy") and dispose of the kloudbuster directory (if no longer needed).
Buid on Linux Build on Linux
~~~~~~~~~~~~~ ^^^^^^^^^^^^^^
A generally faster build method than with MacOSX/Vagrant. A generally faster build method than with MacOSX/Vagrant. Your Linux server must have python, git and qemu utilities installed. You can skip the package installation if you've installed KloudBuster using the GitHub/OpenStack repository based installation.
Your Linux server must have python, git and qemu utilities installed.
Ubuntu/Debian:: Ubuntu/Debian based:
$ sudo apt-get install python-dev git qemu-utils .. code-block:: bash
Redhat/Fedora/CentOS:: $ sudo apt-get install python-dev git qemu-utils
$ # Source the virtual environment if you have one
$ pip install PyYAML
sudo yum install python-devel git qemu-img Redhat/Fedora/CentOS based:
Furthermore, the python PyYAML package must be installed (use "pip install PyYAML" in your virtual environment if you have one). .. code-block:: bash
Then build the image: $ sudo yum install python-devel git qemu-img
$ # Source the virtual environment if you have one
.. code:: $ pip install PyYAML
# clone the kloudbuster repository Build the image with below commands:
git clone https://github.com/openstack/kloudbuster.git
# go to the dib directory .. code-block:: bash
cd kloudbuster/kloudbuster/dib
# run the build image script, will install DIB and start the build $ # Clone the kloudbuster repository if you have not done so
sh build-image.sh $ git clone https://github.com/openstack/kloudbuster.git
$ # Go to the dib directory
After a few minutes, the qcow2 image will be built and available in the same directory. You can then upload it to OpenStack using the glance CLI), $ cd kloudbuster/kb_dib
$ # Run the build image script, which will install DIB and start the build
$ ./build-image.sh
If you get an error message saying that import yaml fails (seems to happen only on Ubuntu): After a few minutes, the qcow2 image will be built and available in the same directory. You can then upload it to OpenStack using the glance CLI.
.. code:: Trouble-shooting
^^^^^^^^^^^^^^^^
dib-run-parts Thu Jul 2 09:27:50 PDT 2015 Running /tmp/image.ewtpa5DW/hooks/extra-data.d/99-squash-package-install If you get an error message saying that import yaml fails (seems to happen only on Ubuntu)::
"/tmp/image.ewtpa5DW/hooks/extra-data.d/../bin/package-installs-squash",
line 26, in <module>
import yaml
ImportError: No module named yaml
You need to comment out the secure_path option in your /etc/sudoers file (use "sudo visudo" to edit that file):
.. code:: dib-run-parts Thu Jul 2 09:27:50 PDT 2015 Running /tmp/image.ewtpa5DW/hooks/extra-data.d/99-squash-package-install
#Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" "/tmp/image.ewtpa5DW/hooks/extra-data.d/../bin/package-installs-squash",
line 26, in <module>
import yaml
ImportError: No module named yaml
You need to comment out the secure_path option in your /etc/sudoers file (use "sudo visudo" to edit that file)::
#Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

6
kb_dib/Vagrantfile vendored
View File

@ -17,9 +17,9 @@ git clone git://github.com/openstack/dib-utils.git
# install kloudbuster # install kloudbuster
git clone git://github.com/openstack/kloudbuster.git git clone git://github.com/openstack/kloudbuster.git
kb_root=kloudbuster/kloudbuster kb_root=kloudbuster
# Extract image version number '__version__ = 2.0' becomes '__version__=2_0' # Extract image version number '__version__ = 2.0' becomes '__version__=2_0'
ver=`grep '^__version__' $kb_root/kb_vm_agent.py | tr -d ' ' | tr '.' '_'` ver=`grep '^__version__' $kb_root/kloudbuster/kb_vm_agent.py | tr -d ' ' | tr '.' '_'`
eval $ver eval $ver
kb_image_name=kloudbuster_v$__version__ kb_image_name=kloudbuster_v$__version__
@ -30,7 +30,7 @@ echo "Building $kb_image_name.qcow2..."
export PATH=$PATH:`pwd`/diskimage-builder/bin:`pwd`/dib-utils/bin export PATH=$PATH:`pwd`/diskimage-builder/bin:`pwd`/dib-utils/bin
# Add the kloudbuster elements directory to the DIB elements path # Add the kloudbuster elements directory to the DIB elements path
export ELEMENTS_PATH=`pwd`/$kb_root/dib/elements export ELEMENTS_PATH=`pwd`/$kb_root/kb_dib/elements
time disk-image-create -o $kb_image_name ubuntu kloudbuster time disk-image-create -o $kb_image_name ubuntu kloudbuster
mv $kb_image_name.qcow2 /vagrant mv $kb_image_name.qcow2 /vagrant

2
kb_dib/build-image.sh
View File

@ -18,7 +18,7 @@ export PATH=$PATH:`pwd`/diskimage-builder/bin:`pwd`/dib-utils/bin
export ELEMENTS_PATH=`pwd`/elements export ELEMENTS_PATH=`pwd`/elements
# Extract image version number '__version__ = 2.0' becomes '__version__=2_0' # Extract image version number '__version__ = 2.0' becomes '__version__=2_0'
ver=`grep '^__version__' ../kb_vm_agent.py | tr -d ' ' | tr '.' '_'` ver=`grep '^__version__' ../kloudbuster/kb_vm_agent.py | tr -d ' ' | tr '.' '_'`
eval $ver eval $ver
kb_image_name=kloudbuster_v$__version__ kb_image_name=kloudbuster_v$__version__

1
kb_dib/elements/kloudbuster/static/kb_test/kb_vm_agent.py
View File

@ -100,6 +100,7 @@ class KB_Instance(object):
report_interval): report_interval):
if not rate_limit: if not rate_limit:
rate_limit = 65535 rate_limit = 65535
cmd = '%s -t%d -c%d -R%d -d%ds -p%ds --timeout %ds -D2 -e %s' % \ cmd = '%s -t%d -c%d -R%d -d%ds -p%ds --timeout %ds -D2 -e %s' % \
(dest_path, threads, connections, rate_limit, duration, (dest_path, threads, connections, rate_limit, duration,
report_interval, timeout, target_url) report_interval, timeout, target_url)

2
kb_server/kloudbuster-swagger.yaml
View File

@ -158,7 +158,7 @@ paths:
404: 404:
description: The session_id is not found or invalid description: The session_id is not found or invalid
/kloudbuster/console_log/{session_id}: /kloudbuster/log/{session_id}:
get: get:
description: | description: |
Get KloudBuster console log for a given session Get KloudBuster console log for a given session

10
kloudbuster/base_compute.py
View File

@ -170,13 +170,18 @@ class SecGroup(object):
Sometimes this maybe in use if instance is just deleted Sometimes this maybe in use if instance is just deleted
Add a retry mechanism Add a retry mechanism
""" """
if not self.secgroup:
return True
for _ in range(10): for _ in range(10):
try: try:
self.novaclient.security_groups.delete(self.secgroup) self.novaclient.security_groups.delete(self.secgroup)
break return True
except Exception: except Exception:
time.sleep(2) time.sleep(2)
LOG.error('Failed while deleting security group %s.' % self.secgroup.id)
return False
class KeyPair(object): class KeyPair(object):
@ -208,7 +213,8 @@ class KeyPair(object):
""" """
Remove the keypair created by KloudBuster Remove the keypair created by KloudBuster
""" """
self.novaclient.keypairs.delete(self.keypair) if self.keypair:
self.novaclient.keypairs.delete(self.keypair)
class Flavor(object): class Flavor(object):

84
kloudbuster/base_network.py
View File

@ -35,6 +35,7 @@ def create_floating_ip(neutron_client, ext_net):
""" """
Function that creates a floating ip and returns it Function that creates a floating ip and returns it
Accepts the neutron client and ext_net Accepts the neutron client and ext_net
Module level function since this is not associated with a Module level function since this is not associated with a
specific network instance specific network instance
""" """
@ -51,8 +52,19 @@ def delete_floating_ip(neutron_client, fip):
Deletes the floating ip Deletes the floating ip
Module level function since this operation Module level function since this operation
is not associated with a network is not associated with a network
Sometimes this will fail if instance is just deleted
Add a retry mechanism
""" """
neutron_client.delete_floatingip(fip) for _ in range(10):
try:
neutron_client.delete_floatingip(fip)
return True
except Exception:
time.sleep(1)
LOG.error('Failed while deleting floating IP %s.' % fip['floatingip']['id'])
return False
def find_external_network(neutron_client): def find_external_network(neutron_client):
""" """
@ -141,26 +153,20 @@ class BaseNetwork(object):
Deletes the compute resources Deletes the compute resources
Security groups, keypairs and instances Security groups, keypairs and instances
""" """
flag = True
# Delete the instances first # Delete the instances first
for instance in self.instance_list: for instance in self.instance_list:
instance.delete_server() instance.delete_server()
if instance.fip: if instance.fip:
""" # Delete the Floating IP
Delete the Floating IP flag = flag &\
Sometimes this will fail if instance is just deleted delete_floating_ip(self.neutron_client, instance.fip['floatingip']['id'])
Add a retry mechanism
"""
for _ in range(10):
try:
delete_floating_ip(self.neutron_client, instance.fip['floatingip']['id'])
break
except Exception:
time.sleep(1)
# Delete all security groups # Delete all security groups
for secgroup_instance in self.secgroup_list: for secgroup_instance in self.secgroup_list:
secgroup_instance.delete_secgroup() flag = flag & secgroup_instance.delete_secgroup()
return flag
def create_network_and_subnet(self, network_name): def create_network_and_subnet(self, network_name):
""" """
@ -202,13 +208,18 @@ class BaseNetwork(object):
Deletes the network and associated subnet Deletes the network and associated subnet
retry the deletion since network may be in use retry the deletion since network may be in use
""" """
for _ in range(1, 5): if not self.network:
return True
for _ in range(10):
try: try:
self.neutron_client.delete_network(self.network['id']) self.neutron_client.delete_network(self.network['id'])
break return True
except NetworkInUseClient: except NetworkInUseClient:
time.sleep(1) time.sleep(1)
LOG.error('Failed while deleting network %s.' % self.network['id'])
return False
def get_all_instances(self): def get_all_instances(self):
return self.instance_list return self.instance_list
@ -269,21 +280,19 @@ class Router(object):
associated with a router associated with a router
""" """
flag = True
for network in self.network_list: for network in self.network_list:
# Now delete the compute resources and the network resources # Now delete the compute resources and the network resources
network.delete_compute_resources() flag = flag & network.delete_compute_resources()
if network.network: if network.network:
self.remove_router_interface(network) flag = flag & self.remove_router_interface(network)
network.delete_network() flag = flag & network.delete_network()
# Also delete the shared port and remove it from router interface # Also delete the shared port and remove it from router interface
if self.shared_network: if self.shared_network:
for _ in range(10): flag = flag & self.remove_router_interface(self.shared_network, use_port=True)
try: self.shared_network = None
self.remove_router_interface(self.shared_network, use_port=True)
self.shared_network = None return flag
break
except Exception:
time.sleep(1)
def create_router(self, router_name, ext_net): def create_router(self, router_name, ext_net):
""" """
@ -317,20 +326,22 @@ class Router(object):
Also delete the networks attached to this router Also delete the networks attached to this router
""" """
# Delete the network resources first and than delete the router itself # Delete the network resources first and than delete the router itself
self.delete_network_resources() if not self.router:
return True
network_flag = self.delete_network_resources()
router_flag = False
for _ in range(10): for _ in range(10):
try: try:
self.neutron_client.remove_gateway_router(self.router['router']['id']) self.neutron_client.remove_gateway_router(self.router['router']['id'])
self.shared_network = None
break
except Exception:
time.sleep(1)
for _ in range(10):
try:
self.neutron_client.delete_router(self.router['router']['id']) self.neutron_client.delete_router(self.router['router']['id'])
router_flag = True
break break
except Exception: except Exception:
time.sleep(1) time.sleep(1)
if not router_flag:
LOG.error('Failed while deleting router %s.' % self.router['router']['id'])
return network_flag & router_flag
def _port_create_neutron(self, network_instance): def _port_create_neutron(self, network_instance):
""" """
@ -377,8 +388,15 @@ class Router(object):
body = { body = {
'subnet_id': network_instance.network['subnets'][0] 'subnet_id': network_instance.network['subnets'][0]
} }
self.neutron_client.remove_interface_router(self.router['router']['id'], body)
for _ in range(10):
try:
self.neutron_client.remove_interface_router(self.router['router']['id'], body)
return True
except Exception:
time.sleep(1)
return False
class NeutronQuota(object): class NeutronQuota(object):

6
kloudbuster/force_cleanup.sh
View File

@ -62,6 +62,7 @@ if [ "$1" == "--file" ] && [ -f "$2" ]; then
INSTANCE_LIST=`grep "instances" $2 | cut -d'|' -f3` INSTANCE_LIST=`grep "instances" $2 | cut -d'|' -f3`
SEC_GROUP_LIST=`grep "sec_groups" $2 | cut -d'|' -f3` SEC_GROUP_LIST=`grep "sec_groups" $2 | cut -d'|' -f3`
FLAVOR_LIST=`grep "flavors" $2 | cut -d'|' -f3` FLAVOR_LIST=`grep "flavors" $2 | cut -d'|' -f3`
KEYPAIR_LIST=`grep "keypairs" $2 | cut -d'|' -f2`
ROUTER_LIST=`grep "routers" $2 | cut -d'|' -f3` ROUTER_LIST=`grep "routers" $2 | cut -d'|' -f3`
NETWORK_LIST=`grep "networks" $2 | cut -d'|' -f3` NETWORK_LIST=`grep "networks" $2 | cut -d'|' -f3`
TENANT_LIST=`grep "tenants" $2 | cut -d'|' -f3` TENANT_LIST=`grep "tenants" $2 | cut -d'|' -f3`
@ -72,6 +73,7 @@ else
INSTANCE_LIST=`nova list --all-tenants | grep KB | cut -d'|' -f2` INSTANCE_LIST=`nova list --all-tenants | grep KB | cut -d'|' -f2`
SEC_GROUP_LIST=`neutron security-group-list | grep KB | cut -d'|' -f2` SEC_GROUP_LIST=`neutron security-group-list | grep KB | cut -d'|' -f2`
FLAVOR_LIST=`nova flavor-list | grep kb | cut -d'|' -f3` FLAVOR_LIST=`nova flavor-list | grep kb | cut -d'|' -f3`
KEYPAIR_LIST=`nova keypair-list | grep KB | cut -d'|' -f2`
ROUTER_LIST=`neutron router-list | grep KB | cut -d'|' -f2` ROUTER_LIST=`neutron router-list | grep KB | cut -d'|' -f2`
NETWORK_LIST=`neutron net-list | grep KB | cut -d'|' -f2` NETWORK_LIST=`neutron net-list | grep KB | cut -d'|' -f2`
TENANT_LIST=`keystone tenant-list | grep KB | cut -d'|' -f2` TENANT_LIST=`keystone tenant-list | grep KB | cut -d'|' -f2`
@ -91,6 +93,10 @@ for line in $SEC_GROUP_LIST; do
neutron security-group-delete $line & neutron security-group-delete $line &
done; done;
for line in $KEYPAIR_LIST; do
nova keypair-delete "$line"
done;
if [ "$FLOATINGIP_LIST" == "" ]; then if [ "$FLOATINGIP_LIST" == "" ]; then
echo -e "`neutron floatingip-list | grep -E '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'`" | while read line; do echo -e "`neutron floatingip-list | grep -E '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+'`" | while read line; do
fid=`echo $line | cut -d'|' -f2 | xargs` fid=`echo $line | cut -d'|' -f2 | xargs`

18
kloudbuster/kloudbuster.py
View File

@ -114,8 +114,9 @@ class Kloud(object):
nova_client = self.tenant_list[0].user_list[0].nova_client nova_client = self.tenant_list[0].user_list[0].nova_client
except Exception: except Exception:
# NOVA Client is not yet initialized, so skip cleaning up... # NOVA Client is not yet initialized, so skip cleaning up...
return return True
flag = True
if not self.reusing_tenants: if not self.reusing_tenants:
flavor_manager = base_compute.Flavor(nova_client) flavor_manager = base_compute.Flavor(nova_client)
if self.testing_side: if self.testing_side:
@ -125,7 +126,9 @@ class Kloud(object):
flavor_manager.delete_flavor('kb.server') flavor_manager.delete_flavor('kb.server')
for tnt in self.tenant_list: for tnt in self.tenant_list:
tnt.delete_resources() flag = flag & tnt.delete_resources()
return flag
def get_first_network(self): def get_first_network(self):
if self.tenant_list: if self.tenant_list:
@ -341,6 +344,7 @@ class KloudBuster(object):
The runner for KloudBuster Tests The runner for KloudBuster Tests
""" """
vm_creation_concurrency = self.client_cfg.vm_creation_concurrency vm_creation_concurrency = self.client_cfg.vm_creation_concurrency
cleanup_flag = True
try: try:
tenant_quota = self.calc_tenant_quota() tenant_quota = self.calc_tenant_quota()
self.kloud.create_resources(tenant_quota['server']) self.kloud.create_resources(tenant_quota['server'])
@ -419,16 +423,22 @@ class KloudBuster(object):
# then testing side last (order is important because of the shared network) # then testing side last (order is important because of the shared network)
if self.server_cfg['cleanup_resources']: if self.server_cfg['cleanup_resources']:
try: try:
self.kloud.delete_resources() cleanup_flag = self.kloud.delete_resources()
except Exception: except Exception:
traceback.print_exc() traceback.print_exc()
KBResLogger.dump_and_save('svr', self.kloud.res_logger.resource_list) KBResLogger.dump_and_save('svr', self.kloud.res_logger.resource_list)
if not cleanup_flag:
LOG.warn('Some resources are not cleaned up properly.')
KBResLogger.dump_and_save('svr', self.kloud.res_logger.resource_list)
if self.client_cfg['cleanup_resources']: if self.client_cfg['cleanup_resources']:
try: try:
self.testing_kloud.delete_resources() cleanup_flag = self.testing_kloud.delete_resources()
except Exception: except Exception:
traceback.print_exc() traceback.print_exc()
KBResLogger.dump_and_save('clt', self.testing_kloud.res_logger.resource_list) KBResLogger.dump_and_save('clt', self.testing_kloud.res_logger.resource_list)
if not cleanup_flag:
KBResLogger.dump_and_save('clt', self.testing_kloud.res_logger.resource_list)
def dump_logs(self, offset=0): def dump_logs(self, offset=0):
if not self.fp_logfile: if not self.fp_logfile:

5
kloudbuster/tenant.py
View File

@ -124,10 +124,13 @@ class Tenant(object):
Delete all user resources and than Delete all user resources and than
deletes the tenant deletes the tenant
""" """
flag = True
# Delete all the users in the tenant along with network and compute elements # Delete all the users in the tenant along with network and compute elements
for user in self.user_list: for user in self.user_list:
user.delete_resources() flag = flag & user.delete_resources()
if not self.reusing_users: if not self.reusing_users:
# Delete the tenant (self) # Delete the tenant (self)
self.kloud.keystone.tenants.delete(self.tenant_id) self.kloud.keystone.tenants.delete(self.tenant_id)
return flag

6
kloudbuster/users.py
View File

@ -107,18 +107,21 @@ class User(object):
def delete_resources(self): def delete_resources(self):
LOG.info("Deleting all user resources for user %s" % self.user_name) LOG.info("Deleting all user resources for user %s" % self.user_name)
flag = True
# Delete key pair # Delete key pair
if self.key_pair: if self.key_pair:
self.key_pair.remove_public_key() self.key_pair.remove_public_key()
# Delete all user routers # Delete all user routers
for router in self.router_list: for router in self.router_list:
router.delete_router() flag = flag & router.delete_router()
if not self.tenant.reusing_users: if not self.tenant.reusing_users:
# Finally delete the user # Finally delete the user
self.tenant.kloud.keystone.users.delete(self.user.id) self.tenant.kloud.keystone.users.delete(self.user.id)
return flag
def update_tenant_quota(self, tenant_quota): def update_tenant_quota(self, tenant_quota):
nova_quota = base_compute.NovaQuota(self.nova_client, self.tenant.tenant_id) nova_quota = base_compute.NovaQuota(self.nova_client, self.tenant.tenant_id)
nova_quota.update_quota(**tenant_quota['nova']) nova_quota.update_quota(**tenant_quota['nova'])
@ -209,6 +212,7 @@ class User(object):
if config_scale.public_key_file: if config_scale.public_key_file:
self.key_pair = base_compute.KeyPair(self.nova_client) self.key_pair = base_compute.KeyPair(self.nova_client)
self.key_name = self.user_name + '-K' self.key_name = self.user_name + '-K'
self.res_logger.log('keypairs', self.key_name, "")
self.key_pair.add_public_key(self.key_name, config_scale.public_key_file) self.key_pair.add_public_key(self.key_name, config_scale.public_key_file)
# Find the external network that routers need to attach to # Find the external network that routers need to attach to