abe9fbd877
Issue: At AT&T we have large complex test stacks that make putting
everything into a single heat template and environment file very
cumbersome. Large monolithic templates make it harder to debug failures,
maintain, extend, and organize these tests. In order to solve this issue
we have enhanced Shaker to support specifying support templates with
environment files.
This commit enhances Shaker to add the ability to define
support_templates with env_files in test definitions.
Support templates spin up "support type" resources before the actual
test stack is spun up. This could range from networks, to volumes, to
anything Heat can create. The support resources do not have any reliance
on resources created in the test stack, they set up a "foundation" for
the test stack. The test stack can then reference these resources by
name. (e.g. assume they exist by the time the test stack is spun up)
While the example provided with this commit is simple, and the support
networks that get created are not directly used in the test, it
shows the basic principles of how support templates work.
As a real-world example and to give an idea of the complexity this
enhancement is trying to solve, we have a test definition that looks
like this:
support_templates:
-
Base:
template: templates/module_1_base.yaml
env_file: env/module_1_base.env
-
SI_L2:
template: templates/module_2_si_l2.yaml
env_file: env/module_2_si_l2.env
-
SI_L3:
template: templates/module_3_si_l3.yaml
env_file: env/module_3_si_l3.env
template: templates/module_4_master_servant.yaml
env_file: env/module_4_master_servant.env
The first support stack (module_1) sets up some "base" network
resources. This stack provides some network resources used by the SI_L2
and SI_L3 support stacks.
SI_L2 is a support stack with 2 VMs that do Contrail service chaining
on an L2 network.
SI_L3 is a support stack with 2 VMs that do Contrail service chaining
on an L3 network.
Then the test stack (module 4) gets spun up on N amount of computes and
runs traffic across the SI_L2 and SI_L3 service chained networks.
After the test run all stacks are cleaned up
Using the concept of support stacks allows us to beter organize and
maintain our complex tests and allows for faster debugging due to the
"layered" nature of the setup.
Support templates also allow us to spin up more Shaker test threads
that use the same support templates simultaneously to better simulate
real-world network traffic. It also reduces the set up time of certain
tests we have since the support stacks already exist.
This enhancement does not alter existing Shaker functionality and
is fully backwards compatible.
Change-Id: Ife51bc55874c6ec4faac221bab8f9f0eea175fdc
140 lines
8.9 KiB
Plaintext
140 lines
8.9 KiB
Plaintext
usage: shaker-spot [-h] [--artifacts-dir ARTIFACTS_DIR] [--book BOOK]
|
|
[--config-dir DIR] [--config-file PATH] [--debug]
|
|
[--log-config-append PATH] [--log-date-format DATE_FORMAT]
|
|
[--log-dir LOG_DIR] [--log-file PATH] [--matrix MATRIX]
|
|
[--no-report-on-error] [--nodebug] [--nono-report-on-error]
|
|
[--nouse-journal] [--nouse-json] [--nouse-syslog]
|
|
[--nowatch-log-file] [--output OUTPUT] [--report REPORT]
|
|
[--report-template REPORT_TEMPLATE] [--scenario SCENARIO]
|
|
[--scenario_availability_zone SCENARIO_AVAILABILITY_ZONE]
|
|
[--scenario_compute_nodes SCENARIO_COMPUTE_NODES]
|
|
[--subunit SUBUNIT]
|
|
[--syslog-log-facility SYSLOG_LOG_FACILITY] [--use-journal]
|
|
[--use-json] [--use-syslog] [--watch-log-file]
|
|
|
|
optional arguments:
|
|
-h, --help show this help message and exit
|
|
--artifacts-dir ARTIFACTS_DIR
|
|
If specified, directs Shaker to store there all its
|
|
artifacts (output, report, subunit and book). Defaults
|
|
to env[SHAKER_ARTIFACTS_DIR].
|
|
--book BOOK Generate report in ReST format and store it into the
|
|
specified folder, defaults to env[SHAKER_BOOK].
|
|
--config-dir DIR Path to a config directory to pull `*.conf` files
|
|
from. This file set is sorted, so as to provide a
|
|
predictable parse order if individual options are
|
|
over-ridden. The set is parsed after the file(s)
|
|
specified via previous --config-file, arguments hence
|
|
over-ridden options in the directory take precedence.
|
|
--config-file PATH Path to a config file to use. Multiple config files
|
|
can be specified, with values in later files taking
|
|
precedence. Defaults to None.
|
|
--debug, -d If set to true, the logging level will be set to DEBUG
|
|
instead of the default INFO level.
|
|
--log-config-append PATH, --log-config PATH, --log_config PATH
|
|
The name of a logging configuration file. This file is
|
|
appended to any existing logging configuration files.
|
|
For details about logging configuration files, see the
|
|
Python logging module documentation. Note that when
|
|
logging configuration files are used then all logging
|
|
configuration is set in the configuration file and
|
|
other logging configuration options are ignored (for
|
|
example, logging_context_format_string).
|
|
--log-date-format DATE_FORMAT
|
|
Defines the format string for %(asctime)s in log
|
|
records. Default: None . This option is ignored if
|
|
log_config_append is set.
|
|
--log-dir LOG_DIR, --logdir LOG_DIR
|
|
(Optional) The base directory used for relative
|
|
log_file paths. This option is ignored if
|
|
log_config_append is set.
|
|
--log-file PATH, --logfile PATH
|
|
(Optional) Name of log file to send logging output to.
|
|
If no default is set, logging will go to stderr as
|
|
defined by use_stderr. This option is ignored if
|
|
log_config_append is set.
|
|
--matrix MATRIX Set the matrix of parameters for the scenario. The
|
|
value is specified in YAML format. E.g. to override
|
|
the scenario duration one may provide: "{time: 10}",
|
|
or to override list of hosts: "{host:[ping.online.net,
|
|
iperf.eenet.ee]}". When several parameters are
|
|
overridden all combinations are tested
|
|
--no-report-on-error Do not generate report for failed scenarios
|
|
--nodebug The inverse of --debug
|
|
--nono-report-on-error
|
|
The inverse of --no-report-on-error
|
|
--nouse-journal The inverse of --use-journal
|
|
--nouse-json The inverse of --use-json
|
|
--nouse-syslog The inverse of --use-syslog
|
|
--nowatch-log-file The inverse of --watch-log-file
|
|
--output OUTPUT File for output in JSON format, defaults to
|
|
env[SHAKER_OUTPUT]. If it is empty, then output will
|
|
be saved to /tmp/shaker_<time_now>.json
|
|
--report REPORT Report file name, defaults to env[SHAKER_REPORT].
|
|
--report-template REPORT_TEMPLATE
|
|
Template for report. Can be a file name or one of
|
|
aliases: "interactive", "json". Defaults to
|
|
"interactive".
|
|
--scenario SCENARIO Comma-separated list of scenarios to play. Each entity
|
|
can be a file name or one of aliases:
|
|
"misc/instance_metadata", "misc/static_agent",
|
|
"misc/static_agents_pair",
|
|
"openstack/cross_az/full_l2",
|
|
"openstack/cross_az/full_l3_east_west",
|
|
"openstack/cross_az/full_l3_north_south",
|
|
"openstack/cross_az/perf_l2",
|
|
"openstack/cross_az/perf_l3_east_west",
|
|
"openstack/cross_az/perf_l3_north_south",
|
|
"openstack/cross_az/udp_l2",
|
|
"openstack/cross_az/udp_l2_mss8950",
|
|
"openstack/cross_az/udp_l3_east_west",
|
|
"openstack/dense_l2", "openstack/dense_l3_east_west",
|
|
"openstack/dense_l3_north_south",
|
|
"openstack/external/dense_l3_north_south_no_fip",
|
|
"openstack/external/dense_l3_north_south_with_fip",
|
|
"openstack/external/full_l3_north_south_no_fip",
|
|
"openstack/external/full_l3_north_south_with_fip",
|
|
"openstack/external/perf_l3_north_south_no_fip",
|
|
"openstack/external/perf_l3_north_south_with_fip",
|
|
"openstack/full_l2", "openstack/full_l3_east_west",
|
|
"openstack/full_l3_north_south", "openstack/perf_l2",
|
|
"openstack/perf_l3_east_west",
|
|
"openstack/perf_l3_north_south",
|
|
"openstack/qos/perf_l2", "openstack/udp_l2",
|
|
"openstack/udp_l3_east_west",
|
|
"openstack/udp_l3_north_south", "spot/ping",
|
|
"spot/tcp", "spot/udp". Defaults to
|
|
env[SHAKER_SCENARIO].
|
|
--scenario_availability_zone SCENARIO_AVAILABILITY_ZONE
|
|
Comma-separated list of availability_zone. If
|
|
specified this setting will override the
|
|
availability_zone accomodation setting in the scenario
|
|
test definition.Defaults to SCENARIO_AVAILABILITY_ZONE
|
|
--scenario_compute_nodes SCENARIO_COMPUTE_NODES
|
|
Number of compute_nodes. If specified this setting
|
|
will override the compute_nodes accomodation setting
|
|
in the scenario test definition. Defaults to
|
|
SCENARIO_COMPUTE_NODES
|
|
--subunit SUBUNIT Subunit stream file name, defaults to
|
|
env[SHAKER_SUBUNIT].
|
|
--syslog-log-facility SYSLOG_LOG_FACILITY
|
|
Syslog facility to receive log lines. This option is
|
|
ignored if log_config_append is set.
|
|
--use-journal Enable journald for logging. If running in a systemd
|
|
environment you may wish to enable journal support.
|
|
Doing so will use the journal native protocol which
|
|
includes structured metadata in addition to log
|
|
messages.This option is ignored if log_config_append
|
|
is set.
|
|
--use-json Use JSON formatting for logging. This option is
|
|
ignored if log_config_append is set.
|
|
--use-syslog Use syslog for logging. Existing syslog format is
|
|
DEPRECATED and will be changed later to honor RFC5424.
|
|
This option is ignored if log_config_append is set.
|
|
--watch-log-file Uses logging handler designed to watch file system.
|
|
When log file is moved or removed this handler will
|
|
open a new log file with specified path
|
|
instantaneously. It makes sense only if log_file
|
|
option is specified and Linux platform is used. This
|
|
option is ignored if log_config_append is set.
|