Issue: Some of our more complex tests require injecting "custom" data
into the heat stacks in order to run correctly. For example in order to
run certain Contrail based tests we need to set "contrail-asn:(some
asn number)". The asn number varies between cloud deployments, so we
dynamically set this field in a Heat environment file. However this is
a very specific field that other Shaker users might not need or
understand it, so this commit allows for a more generic approach.
This commit aims to let the user specify "custom" user defined options
similar to the matrix config parameter. Since this opts is not
directly referenced anywhere in code it's main use is to inject data
into heat environment files. The YAML format allows flexibility for any
object the user might want to add during their scenario
run or it can be left blank without any issues.
Change-Id: I96b6e578eb59813e5e0c8a2fe7a14c5ecc369be7
Issue: At AT&T we have large complex test stacks that make putting
everything into a single heat template and environment file very
cumbersome. Large monolithic templates make it harder to debug failures,
maintain, extend, and organize these tests. In order to solve this issue
we have enhanced Shaker to support specifying support templates with
environment files.
This commit enhances Shaker to add the ability to define
support_templates with env_files in test definitions.
Support templates spin up "support type" resources before the actual
test stack is spun up. This could range from networks, to volumes, to
anything Heat can create. The support resources do not have any reliance
on resources created in the test stack, they set up a "foundation" for
the test stack. The test stack can then reference these resources by
name. (e.g. assume they exist by the time the test stack is spun up)
While the example provided with this commit is simple, and the support
networks that get created are not directly used in the test, it
shows the basic principles of how support templates work.
As a real-world example and to give an idea of the complexity this
enhancement is trying to solve, we have a test definition that looks
like this:
support_templates:
-
Base:
template: templates/module_1_base.yaml
env_file: env/module_1_base.env
-
SI_L2:
template: templates/module_2_si_l2.yaml
env_file: env/module_2_si_l2.env
-
SI_L3:
template: templates/module_3_si_l3.yaml
env_file: env/module_3_si_l3.env
template: templates/module_4_master_servant.yaml
env_file: env/module_4_master_servant.env
The first support stack (module_1) sets up some "base" network
resources. This stack provides some network resources used by the SI_L2
and SI_L3 support stacks.
SI_L2 is a support stack with 2 VMs that do Contrail service chaining
on an L2 network.
SI_L3 is a support stack with 2 VMs that do Contrail service chaining
on an L3 network.
Then the test stack (module 4) gets spun up on N amount of computes and
runs traffic across the SI_L2 and SI_L3 service chained networks.
After the test run all stacks are cleaned up
Using the concept of support stacks allows us to beter organize and
maintain our complex tests and allows for faster debugging due to the
"layered" nature of the setup.
Support templates also allow us to spin up more Shaker test threads
that use the same support templates simultaneously to better simulate
real-world network traffic. It also reduces the set up time of certain
tests we have since the support stacks already exist.
This enhancement does not alter existing Shaker functionality and
is fully backwards compatible.
Change-Id: Ife51bc55874c6ec4faac221bab8f9f0eea175fdc
Scenarios under `test/` subfolder are used for integration testing
or as samples. Skip them in CLI help and keep only production-ready.
Change-Id: I69a428480055c825aa3b590f02b4c09b346a6012
Problem: We have large compute deployments (400+ computes) but different
groups are constantly working in the same lab/deployments so it becomes
unrealistic to try and run tests using all computes because the Shaker
tests could disrupt other work. I know the accomdation is set in test
definitions, and they can changed, but that is also cumbersome when
you're running a large number of tests in a single run.
This commit aims to solve the problem by allowing Shaker users to have
more control over the availaibity_zone and compute_nodes accommodation
by specifying the fields in either the cfg input file or as cli
arguments.
This change would allow Shaker to "override" what's been set in the test
definition and act like a "global" setting for all tests in a
single run to use a particular AZ and/or fixed number of computes_nodes
without having to alter N amount of tests.
Change-Id: I5459150a0eac9bed6a6a62a126bd9ec0648941fe
At AT&T our Openstack Airship deployments use identity api v3, but
require the os-interface parameter to be passed, in order to correctly
authenticate.
This commit adds support for specifying and using os-interface in Shaker
Change-Id: Icad7fc823dc302632bebe5d18f5cfe34bcc00b06
OpenStack profiling can be enabled by setting parameter --os-profile.
The feature requires osprofiler library. If library is not present
a warning message is shown.
Change-Id: I2c1b0cbd99450236b3720e19700f99cfdc14e378
Closes-Bug: 1666585
This commit adds image builder support for CentOS by generalizing the
image_elements using package-installs.yaml ad pkg-map. A few other
executables have been modified to ensure compatibility. Ubuntu is
kept as default.
Change-Id: I4122155d82ad64867efdb6d88536624f677c207d
Shaker-image-builder can build images in one of two modes:
* "heat" - classic approach with help of Heat, but it also
requires Glance v1
* "dib" - make local image using diskimage-builder and then
upload into Glance
The mode can be selected automatically (falls to "dib" if no
Glance v1 present) or specified manually (--image-builder-mode).
Change-Id: Ibbcb99e85012ecab012612dccea38d40100625f5
* Deprecate pyshaker-agent and use full pyshaker package instead.
* Upgrade Ubuntu image to Xenial and remove Trusty-based
Change-Id: I01762f648a79cc5e90eb55e757975563ed1fc06d
Option --scenario now accepts list of scenarios. They are executed
one-by-one, results are aggregated and stored in a single json
file or report. New option --artifacts-dir allows to store individual
results as well.
Change-Id: If9b8f0650635a8e2534f757de158a24461f045ba
Now Shaker can be run in Docker container! The container
does the whole work: it creates image, runs the scenario
and cleans everything up.
Change-Id: I9cbcb7e79d08f9526cf1ee8f3ccafab34d3935c2
Currently the nameserver for the subnets is set by default to
Google Public DNS by the heat templates. This patch makes the
dns nameserver configurable keeping the defaults standard. All
the heat templates are fixed to reflect the same.
+ Implementing Ilya's suggestions
Change-Id: I029e39407fccf528e164181539021978bd32c44b
Closes-Bug: #1591365
The flavor can be configured by image builder:
--flavor-ram - image RAM in MB
--flavor-vcpus - number of cores
--flavor-disk - disk size in GB
Change-Id: I243583ab8725d0a862ba66dae7828c23b9b20de0
* Replace all home-brewn code to work with OpenStack clients
with os-client-config lib. No need to monitor token expiration
for heat client anymore!
* Add 'os-project-name' parameter as it becomes standard
* Sync requirements to the latest
Closes-Bug: 1573504
Change-Id: I7520b9aed075074b4b47551eb22d18e568da83dd
Rename location of all OpenStack scenarios from "networking"
to "openstack". The old name is deprecated, but is automatically
translated into the new one.
Change-Id: I59ab4b7659199216bfdddcf2c57c254ddb1a19c8
Existing north-south scenarios actually do double path from tenant network
to external and then back. New scenarios allow to test against target host
located outside of the cloud. There are 2 options: instances with floating
IPs and without.
Change-Id: I91308465e9bbf261b0c919405b012d3d9b9f07da
With new option --book Shaker is able to generate report
as book in ReST format. The report contains necessary data only.
Change-Id: I3bc575214da5cabb7ec9eee7f497a8d5cbb67097
A new CLI parameter --matrix is introduced. The parameter allows to override
values specified in test cases.
Change-Id: I384b5f8c8963d8704ef9d31ce6f209be3ec8e184
Added new entry-point shaker-spot, which allows to execute scenarios
from the local node. Example of such scenario is spot/tcp which uses
iperf3 to measure bandwidth against one of public hosts.
Change-Id: I9303785501c3af7212ca590b8de63218ca877cd6
Now host name is not only taken from slave agent, but can
be specified directly in the test case. As example of usage
a new static agent networking scenario is added. The scenario
uses iperf3 to test connectivity to one of public iperf3 servers.
Change-Id: I660c91e1607a538217b81213abaaf9d57f4978a3
When using SSL in connections to the registry server, do not require
validation via a certifying authority. This is the registry's
equivalent of specifying --insecure on the command line using
Openstack clients for the API.
Change-Id: Id8164ccc5c8eee6e2da179c7552a667e10e61215
Use case: as a user, I would like to run networking tests when
master nodes are in one AZ and slave nodes in the other.
This patch removes hardcoded AZ name from config files, allows
to specify list of active AZs and enable cross AZ testing (for
case of 2 AZs). A new example of scenario is added into
networking/cross_az/full_l2.
Change-Id: I416f0f790d859f046525084ddc307952cbcc41d8
Added heat template for implementing centos target vms referenced by their alias.
Co-Authored-By: Joe Talerico <jtaleric@redhat.com>
Related-Bug: #1460726
Change-Id: I99c698506da36bd731951a58554f99701b49b4c9
UDP tests produce significant load on controllers, they should
be configured accordingly to scale of the cloud. This patch reduces
amount of UDP traffic in stock scenarios, tests with old values are
extracted to udp_* files.
Partial bug 1470892
Change-Id: I410bb3be0f7e267cd4aba4e95a3759e9ad6bc5c0
TLS/SSL support is enabled by adding os-cacert option to point
to a CA cert file. All Openstack client commands now pass
the cacert location as a parameter.
Change-Id: I72d9582d82381fd990e8d35b6320d0bea7cc203e
Cleans the heat stack in case of any error during scenario execution.
Configurable, with default action to clean the stack on any error
Closes-Bug: 1464315
Change-Id: I4a097ed92127b7eed6fdab1e80f3821002312bb3
Report template can be specified by full path or via alias.
List of available aliases can be found in the docs and CLI tools
help.
Change-Id: I22eb933d57a52b4eec24bba53b783e224a0a651b
Now Shaker package can contain several templates for image builder. Users may
refer to these templates by short name (alias), no full path knowledge is required.
All available aliases are shown in help message and in the docs.
Added Debian-based template as an example.
Closes bug 1460726
Change-Id: I6109be99acfa802abc185f92b15f0540007935be
