Update README
Our README is woefully out of date. Bring it more in line with reality. Update setup.cfg as well. Change-Id: I83f7c2e776983d92c11a691076d2a3d25b7ed30c
This commit is contained in:
Artom Lifshitz
parent
c74bb93b6d
commit
9eb8cb2d28
130
README.rst
130
README.rst
@ -1,112 +1,80 @@
|
|||||||
Whitebox Tempest plugin
|
Whitebox Tempest plugin
|
||||||
=======================
|
=======================
|
||||||
|
|
||||||
This repo is a Tempest plugin that contains scenario tests ran against
|
This is a Tempest plugin for whitebox testing. While Tempest's scope is limited
|
||||||
TripleO/Director-based deployments.
|
to only the REST APIs, whitebox allows tests to peak behind the curtain,
|
||||||
|
similar to how a cloud admin might. Examining things on the compute host(s)
|
||||||
|
and/or the controller(s) is not only allowed, it's required for a test to be in
|
||||||
|
whitebox's scope. Whitebox tests must still be REST API-driven, however their
|
||||||
|
assertions can involve things like the instance XML (if the Nova libvirt driver
|
||||||
|
is in use) or the database.
|
||||||
|
|
||||||
.. important::
|
* Bugs: https://storyboard.openstack.org/#!/project/1162
|
||||||
|
* IRC: #openstack-qa on freenode
|
||||||
This is still a work in progress.
|
|
||||||
|
|
||||||
* Free software: Apache license
|
|
||||||
* Documentation: n/a
|
|
||||||
* Source: https://review.rdoproject.org/r/gitweb?p=openstack/whitebox-tempest-plugin.git
|
|
||||||
* Bugs: n/a
|
|
||||||
|
|
||||||
Requirements
|
Requirements
|
||||||
------------
|
------------
|
||||||
|
|
||||||
The tests assume a TripleO/Director-based deployment with an undercloud and
|
While Tempest is cloud-agnostic because all clouds expose the same OpenStack
|
||||||
overcloud. The tests will be run from the undercloud therefore Tempest should
|
APIs (with some caveats around extensions), whitebox peaks behind the curtain,
|
||||||
be installed and configured on the undercloud node. It's assumed that the Unix
|
and thus is coupled to the way the cloud was deployed. Currently, devstack and
|
||||||
user running the tests, generally *stack*, has SSH access to all the compute
|
TripleO (with undercloud and overcloud) are supported, with only devstack being
|
||||||
nodes running in the overcloud.
|
tested in CI.
|
||||||
|
|
||||||
Most tests have specific hardware requirements. These are documented in the
|
Some tests have specific hardware requirements. These should be documented as
|
||||||
tests themselves and the tests should fast-fail if these hardware requirements
|
config options, and tests are expected to skip if their hardware requirements
|
||||||
are not met. You will require multiple nodes to run these tests and will need
|
are not declared in the configration.
|
||||||
to manually specify which test to run on which node. For more information on
|
|
||||||
our plans here, refer to :ref:`roadmap`.
|
|
||||||
|
|
||||||
For more information on TripleO/Director, refer to the `Red Hat OpenStack
|
|
||||||
Platform documentation`__.
|
|
||||||
|
|
||||||
__ https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/11/html/director_installation_and_usage/chap-introduction
|
|
||||||
|
|
||||||
Install, configure and run
|
Install, configure and run
|
||||||
--------------------------
|
--------------------------
|
||||||
|
|
||||||
|
0. Tempest needs to be installed and configured.
|
||||||
|
|
||||||
1. Install the plugin.
|
1. Install the plugin.
|
||||||
|
|
||||||
This should be done from source. ::
|
This should be done from source. ::
|
||||||
|
|
||||||
WORKSPACE=/some/directory
|
WORKSPACE=/some/directory
|
||||||
cd $WORKSPACE
|
cd $WORKSPACE
|
||||||
git clone https://github.com/redhat-openstack/whitebox-tempest-plugin
|
git clone https://opendev.org/openstack/whitebox-tempest-plugin
|
||||||
sudo pip install whitebox-tempest-plugin
|
sudo pip install whitebox-tempest-plugin
|
||||||
|
|
||||||
2. Configure Tempest.
|
2. Configure Tempest.
|
||||||
|
|
||||||
Add the following lines at the end of your ``tempest.conf`` file. These
|
The exact configuration will depend on the deployment. There is no
|
||||||
determine how your undercloud node, which is running Tempest, should connect
|
configuration reference yet, have a look at
|
||||||
to the compute nodes in the overcloud and vice versa. For example::
|
``whitebox_tempest_plugin/config.py`` instead. As an example, here is a
|
||||||
|
configuration for a multinode TripleO deployment::
|
||||||
|
|
||||||
[whitebox]
|
[whitebox]
|
||||||
hypervisors = compute-0.localdomain:192.168.24.6,compute-1.localdomain:192.168.24.12
|
ctlplane_addresses = compute-0.localdomain:192.168.24.6,compute-1.localdomain:192.168.24.12
|
||||||
# Only set the following if different from the defaults listed
|
ctlplane_ssh_username = heat-admin
|
||||||
# ctlplane_ssh_username = heat-admin
|
ctlplane_ssh_private_key_path = /home/stack/.ssh/id_rsa
|
||||||
# ctlplane_ssh_private_key_path = /home/stack/.ssh/id_rsa
|
containers = true
|
||||||
containers = true
|
max_compute_nodes = 2 # Some tests depend on there being a single
|
||||||
max_compute_nodes = 2 # Some tests depend on there being a single
|
# (available) compute node
|
||||||
# (available) compute node
|
|
||||||
|
|
||||||
3. Execute the tests. ::
|
3. Execute the tests. ::
|
||||||
|
|
||||||
tempest run --regex whitebox_tempest_plugin.
|
tempest run --serial --regex whitebox_tempest_plugin.
|
||||||
|
|
||||||
|
.. important::
|
||||||
|
|
||||||
|
Whitebox expects its tests to run one at a time. Make sure to pass
|
||||||
|
`--serial` or `--concurrency 1` to `tempest run`.
|
||||||
|
|
||||||
|
|
||||||
How to add a new test
|
How to add a new test
|
||||||
---------------------
|
---------------------
|
||||||
|
|
||||||
New tests should be added to the ``whitebox_tempest_plugin/tests`` directory.
|
Tests should fit whitebox's scope. If a test intereacts with REST APIs and
|
||||||
|
nothing else, it is better suited for Tempest itself. New tests should be added
|
||||||
According to the plugin interface doc__, you should mainly import "stable" APIs
|
in their respective subdirectories. For example, tests that use the compute API
|
||||||
which usually are:
|
live in ``whitebox_tempest_plugin/api/compute``. Test code does not need unit
|
||||||
|
tests, but helpers or utilities do. Unit tests live in
|
||||||
* ``tempest.lib.*``
|
``whitebox_tempest_plugin/tests``. Whitebox does not adhere to the `Tempest
|
||||||
* ``tempest.config``
|
plugin interface <https://docs.openstack.org/tempest/latest/plugin.html>`. As
|
||||||
* ``tempest.test_discover.plugins``
|
mentioned, whitebox tests run one at a time, so it's safe for a test to modify
|
||||||
* ``tempest.common.credentials_factory``
|
the environment and/or be destructive, as long as it cleans up after itself.
|
||||||
* ``tempest.clients``
|
For example, changing Nova configuration values and/or restarting services is
|
||||||
* ``tempest.test``
|
acceptable, as long as the original values and service state are restored.
|
||||||
|
|
||||||
Importing classes from ``tempest.api.*`` could be dangerous since future
|
|
||||||
version of Tempest could break.
|
|
||||||
|
|
||||||
__ http://docs.openstack.org/tempest/latest/plugin.html
|
|
||||||
|
|
||||||
.. _roadmap:
|
|
||||||
|
|
||||||
Roadmap
|
|
||||||
-------
|
|
||||||
|
|
||||||
The different tests found here all have different hardware requirements, and
|
|
||||||
these requirements often conflict. For example, a test that requires a host
|
|
||||||
without HyperThreading enabled cannot be used for a test that requires
|
|
||||||
HyperThreading. As a result, it's not possible to have one "master
|
|
||||||
configuration" that can be used to run all tests. Instead, different tests must
|
|
||||||
be run on different nodes.
|
|
||||||
|
|
||||||
At present, this plugin exists in isolation and the running of individual tests
|
|
||||||
on nodes, along with the configuration of said nodes, remains a manual process.
|
|
||||||
However, the end goal for this project is to be able to kick run this test
|
|
||||||
suite of against *N* overcloud nodes, where each node has a different hardware
|
|
||||||
configuration and *N* is the total number of different hardware configurations
|
|
||||||
required (one for real-time, one for SR-IOV, etc.). Each node would have a
|
|
||||||
different profile__ and host aggregates would likely be used to ensure each
|
|
||||||
test runs on its preferred hardware. To get here, we should probably provide a
|
|
||||||
recipe along with hardware configuration steps.
|
|
||||||
|
|
||||||
This being said, the above is way off yet. For now, we're focussed on getting
|
|
||||||
the tests in place so we can stop doing all this stuff by hand.
|
|
||||||
|
|
||||||
__ http://tripleo.org/install/advanced_deployment/profile_matching.html
|
|
||||||
|
|||||||
15
setup.cfg
15
setup.cfg
@ -1,20 +1,23 @@
|
|||||||
[metadata]
|
[metadata]
|
||||||
name = whitebox-tempest-plugin
|
name = whitebox-tempest-plugin
|
||||||
summary = Tempest plugin for whitebox testing of TripleO/Director-based OpenStack deployments
|
summary = Tempest plugin for whitebox testing. For testing things not exposed through the REST APIs.
|
||||||
description-file =
|
description-file =
|
||||||
README.rst
|
README.rst
|
||||||
author = Red Hat, Inc.
|
author = OpenStack
|
||||||
author-email = rhos-ci@redhat.com
|
author-email = openstack-discuss@lists.openstack.org
|
||||||
home-page = https://github.com/redhat-openstack/whitebox-tempest-plugin
|
home-page = https://opendev.org/openstack/whitebox-tempest-plugin
|
||||||
classifier =
|
classifier =
|
||||||
Environment :: OpenStack
|
|
||||||
Intended Audience :: Information Technology
|
Intended Audience :: Information Technology
|
||||||
Intended Audience :: System Administrators
|
Intended Audience :: System Administrators
|
||||||
|
Intended Audience :: Developers
|
||||||
License :: OSI Approved :: Apache Software License
|
License :: OSI Approved :: Apache Software License
|
||||||
Operating System :: POSIX :: Linux
|
Operating System :: POSIX :: Linux
|
||||||
Programming Language :: Python
|
Programming Language :: Python
|
||||||
Programming Language :: Python :: 2
|
|
||||||
Programming Language :: Python :: 2.7
|
Programming Language :: Python :: 2.7
|
||||||
|
Programming Language :: Python :: 3
|
||||||
|
Programming Language :: Python :: 3.6
|
||||||
|
Programming Language :: Python :: 3.7
|
||||||
|
Programming Language :: Python :: 3.8
|
||||||
|
|
||||||
[global]
|
[global]
|
||||||
setup-hooks =
|
setup-hooks =
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user