abe9fbd877
Issue: At AT&T we have large complex test stacks that make putting
everything into a single heat template and environment file very
cumbersome. Large monolithic templates make it harder to debug failures,
maintain, extend, and organize these tests. In order to solve this issue
we have enhanced Shaker to support specifying support templates with
environment files.
This commit enhances Shaker to add the ability to define
support_templates with env_files in test definitions.
Support templates spin up "support type" resources before the actual
test stack is spun up. This could range from networks, to volumes, to
anything Heat can create. The support resources do not have any reliance
on resources created in the test stack, they set up a "foundation" for
the test stack. The test stack can then reference these resources by
name. (e.g. assume they exist by the time the test stack is spun up)
While the example provided with this commit is simple, and the support
networks that get created are not directly used in the test, it
shows the basic principles of how support templates work.
As a real-world example and to give an idea of the complexity this
enhancement is trying to solve, we have a test definition that looks
like this:
support_templates:
-
Base:
template: templates/module_1_base.yaml
env_file: env/module_1_base.env
-
SI_L2:
template: templates/module_2_si_l2.yaml
env_file: env/module_2_si_l2.env
-
SI_L3:
template: templates/module_3_si_l3.yaml
env_file: env/module_3_si_l3.env
template: templates/module_4_master_servant.yaml
env_file: env/module_4_master_servant.env
The first support stack (module_1) sets up some "base" network
resources. This stack provides some network resources used by the SI_L2
and SI_L3 support stacks.
SI_L2 is a support stack with 2 VMs that do Contrail service chaining
on an L2 network.
SI_L3 is a support stack with 2 VMs that do Contrail service chaining
on an L3 network.
Then the test stack (module 4) gets spun up on N amount of computes and
runs traffic across the SI_L2 and SI_L3 service chained networks.
After the test run all stacks are cleaned up
Using the concept of support stacks allows us to beter organize and
maintain our complex tests and allows for faster debugging due to the
"layered" nature of the setup.
Support templates also allow us to spin up more Shaker test threads
that use the same support templates simultaneously to better simulate
real-world network traffic. It also reduces the set up time of certain
tests we have since the support stacks already exist.
This enhancement does not alter existing Shaker functionality and
is fully backwards compatible.
Change-Id: Ife51bc55874c6ec4faac221bab8f9f0eea175fdc
264 lines
16 KiB
Plaintext
264 lines
16 KiB
Plaintext
usage: shaker-all-in-one [-h] [--agent-join-timeout AGENT_JOIN_TIMEOUT]
|
|
[--agent-loss-timeout AGENT_LOSS_TIMEOUT]
|
|
[--artifacts-dir ARTIFACTS_DIR] [--book BOOK]
|
|
[--cleanup] [--cleanup-on-error] [--config-dir DIR]
|
|
[--config-file PATH] [--debug]
|
|
[--dns-nameservers DNS_NAMESERVERS]
|
|
[--external-net EXTERNAL_NET]
|
|
[--flavor-disk FLAVOR_DISK]
|
|
[--flavor-name FLAVOR_NAME] [--flavor-ram FLAVOR_RAM]
|
|
[--flavor-vcpus FLAVOR_VCPUS]
|
|
[--image-builder-distro IMAGE_BUILDER_DISTRO]
|
|
[--image-builder-mode IMAGE_BUILDER_MODE]
|
|
[--image-builder-template IMAGE_BUILDER_TEMPLATE]
|
|
[--image-name IMAGE_NAME] [--log-config-append PATH]
|
|
[--log-date-format DATE_FORMAT] [--log-dir LOG_DIR]
|
|
[--log-file PATH] [--matrix MATRIX]
|
|
[--no-report-on-error] [--nocleanup]
|
|
[--nocleanup-on-error] [--nodebug]
|
|
[--nono-report-on-error] [--noos-insecure]
|
|
[--nouse-journal] [--nouse-json] [--nouse-syslog]
|
|
[--nowatch-log-file] [--os-auth-url <auth-url>]
|
|
[--os-cacert <auth-cacert>]
|
|
[--os-identity-api-version <identity-api-version>]
|
|
[--os-insecure] [--os-interface <os-interface>]
|
|
[--os-password <auth-password>]
|
|
[--os-profile <hmac-key>]
|
|
[--os-project-domain-name <auth-project-domain-name>]
|
|
[--os-project-name <auth-project-name>]
|
|
[--os-region-name <auth-region-name>]
|
|
[--os-tenant-name <auth-tenant-name>]
|
|
[--os-user-domain-name <auth-user-domain-name>]
|
|
[--os-username <auth-username>] [--output OUTPUT]
|
|
[--polling-interval POLLING_INTERVAL]
|
|
[--report REPORT] [--report-template REPORT_TEMPLATE]
|
|
[--scenario SCENARIO]
|
|
[--scenario_availability_zone SCENARIO_AVAILABILITY_ZONE]
|
|
[--scenario_compute_nodes SCENARIO_COMPUTE_NODES]
|
|
[--server-endpoint SERVER_ENDPOINT]
|
|
[--subunit SUBUNIT]
|
|
[--syslog-log-facility SYSLOG_LOG_FACILITY]
|
|
[--use-journal] [--use-json] [--use-syslog]
|
|
[--watch-log-file]
|
|
|
|
optional arguments:
|
|
-h, --help show this help message and exit
|
|
--agent-join-timeout AGENT_JOIN_TIMEOUT
|
|
Timeout to treat agent as join failed in seconds,
|
|
defaults to env[SHAKER_AGENT_JOIN_TIMEOUT] (time
|
|
between stack deployment and start of scenario
|
|
execution).
|
|
--agent-loss-timeout AGENT_LOSS_TIMEOUT
|
|
Timeout to treat agent as lost in seconds, defaults to
|
|
env[SHAKER_AGENT_LOSS_TIMEOUT]
|
|
--artifacts-dir ARTIFACTS_DIR
|
|
If specified, directs Shaker to store there all its
|
|
artifacts (output, report, subunit and book). Defaults
|
|
to env[SHAKER_ARTIFACTS_DIR].
|
|
--book BOOK Generate report in ReST format and store it into the
|
|
specified folder, defaults to env[SHAKER_BOOK].
|
|
--cleanup Cleanup the image and the flavor.
|
|
--cleanup-on-error Clean up the heat-stack upon any error occurred during
|
|
scenario execution.
|
|
--config-dir DIR Path to a config directory to pull `*.conf` files
|
|
from. This file set is sorted, so as to provide a
|
|
predictable parse order if individual options are
|
|
over-ridden. The set is parsed after the file(s)
|
|
specified via previous --config-file, arguments hence
|
|
over-ridden options in the directory take precedence.
|
|
--config-file PATH Path to a config file to use. Multiple config files
|
|
can be specified, with values in later files taking
|
|
precedence. Defaults to None.
|
|
--debug, -d If set to true, the logging level will be set to DEBUG
|
|
instead of the default INFO level.
|
|
--dns-nameservers DNS_NAMESERVERS
|
|
Comma-separated list of IPs of the DNS nameservers for
|
|
the subnets. If no value is provided defaults to
|
|
Google Public DNS.
|
|
--external-net EXTERNAL_NET
|
|
Name or ID of external network, defaults to
|
|
env[SHAKER_EXTERNAL_NET]. If no value provided then
|
|
Shaker picks any of available external networks.
|
|
--flavor-disk FLAVOR_DISK
|
|
Shaker image disk size in GB, defaults to
|
|
env[SHAKER_FLAVOR_DISK]
|
|
--flavor-name FLAVOR_NAME
|
|
Name of image flavor. The default is created by
|
|
shaker-image-builder.
|
|
--flavor-ram FLAVOR_RAM
|
|
Shaker image RAM size in MB, defaults to
|
|
env[SHAKER_FLAVOR_RAM]
|
|
--flavor-vcpus FLAVOR_VCPUS
|
|
Number of cores to allocate for Shaker image, defaults
|
|
to env[SHAKER_FLAVOR_VCPUS]
|
|
--image-builder-distro IMAGE_BUILDER_DISTRO
|
|
Operating System Distribution for shaker image when
|
|
using diskimage-builder, defaults to ubuntu Allowed
|
|
values: ubuntu, centos7
|
|
--image-builder-mode IMAGE_BUILDER_MODE
|
|
Image building mode: "heat" - using Heat template
|
|
(requires Glance v1 for base image upload); "dib" -
|
|
using diskimage-builder elements (requires qemu-utils
|
|
and debootstrap). If not set, switches to "dib" if
|
|
Glance v1 is not available. Can be specified as
|
|
env[SHAKER_IMAGE_BUILDER_MODE] Allowed values: heat,
|
|
dib
|
|
--image-builder-template IMAGE_BUILDER_TEMPLATE
|
|
Heat template containing receipt of building the
|
|
image. Can be a file name or one of aliases: "centos",
|
|
"debian", "ubuntu". Defaults to "ubuntu".
|
|
--image-name IMAGE_NAME
|
|
Name of image to use. The default is created by
|
|
shaker-image-builder.
|
|
--log-config-append PATH, --log-config PATH, --log_config PATH
|
|
The name of a logging configuration file. This file is
|
|
appended to any existing logging configuration files.
|
|
For details about logging configuration files, see the
|
|
Python logging module documentation. Note that when
|
|
logging configuration files are used then all logging
|
|
configuration is set in the configuration file and
|
|
other logging configuration options are ignored (for
|
|
example, logging_context_format_string).
|
|
--log-date-format DATE_FORMAT
|
|
Defines the format string for %(asctime)s in log
|
|
records. Default: None . This option is ignored if
|
|
log_config_append is set.
|
|
--log-dir LOG_DIR, --logdir LOG_DIR
|
|
(Optional) The base directory used for relative
|
|
log_file paths. This option is ignored if
|
|
log_config_append is set.
|
|
--log-file PATH, --logfile PATH
|
|
(Optional) Name of log file to send logging output to.
|
|
If no default is set, logging will go to stderr as
|
|
defined by use_stderr. This option is ignored if
|
|
log_config_append is set.
|
|
--matrix MATRIX Set the matrix of parameters for the scenario. The
|
|
value is specified in YAML format. E.g. to override
|
|
the scenario duration one may provide: "{time: 10}",
|
|
or to override list of hosts: "{host:[ping.online.net,
|
|
iperf.eenet.ee]}". When several parameters are
|
|
overridden all combinations are tested
|
|
--no-report-on-error Do not generate report for failed scenarios
|
|
--nocleanup The inverse of --cleanup
|
|
--nocleanup-on-error The inverse of --cleanup-on-error
|
|
--nodebug The inverse of --debug
|
|
--nono-report-on-error
|
|
The inverse of --no-report-on-error
|
|
--noos-insecure The inverse of --os-insecure
|
|
--nouse-journal The inverse of --use-journal
|
|
--nouse-json The inverse of --use-json
|
|
--nouse-syslog The inverse of --use-syslog
|
|
--nowatch-log-file The inverse of --watch-log-file
|
|
--os-auth-url <auth-url>
|
|
Authentication URL, defaults to env[OS_AUTH_URL].
|
|
--os-cacert <auth-cacert>
|
|
Location of CA Certificate, defaults to
|
|
env[OS_CACERT].
|
|
--os-identity-api-version <identity-api-version>
|
|
Identity API version, defaults to
|
|
env[OS_IDENTITY_API_VERSION].
|
|
--os-insecure When using SSL in connections to the registry server,
|
|
do not require validation via a certifying authority,
|
|
defaults to env[OS_INSECURE].
|
|
--os-interface <os-interface>
|
|
Interface type. Valid options are public, admin and
|
|
internal. defaults to env[OS_INTERFACE].
|
|
--os-password <auth-password>
|
|
Authentication password, defaults to env[OS_PASSWORD].
|
|
--os-profile <hmac-key>
|
|
HMAC key for encrypting profiling context data,
|
|
defaults to env[OS_PROFILE].
|
|
--os-project-domain-name <auth-project-domain-name>
|
|
Authentication project domain name. Defaults to
|
|
env[OS_PROJECT_DOMAIN_NAME].
|
|
--os-project-name <auth-project-name>
|
|
Authentication project name. This option is mutually
|
|
exclusive with --os-tenant-name. Defaults to
|
|
env[OS_PROJECT_NAME].
|
|
--os-region-name <auth-region-name>
|
|
Authentication region name, defaults to
|
|
env[OS_REGION_NAME].
|
|
--os-tenant-name <auth-tenant-name>
|
|
Authentication tenant name, defaults to
|
|
env[OS_TENANT_NAME].
|
|
--os-user-domain-name <auth-user-domain-name>
|
|
Authentication username. Defaults to
|
|
env[OS_USER_DOMAIN_NAME].
|
|
--os-username <auth-username>
|
|
Authentication username, defaults to env[OS_USERNAME].
|
|
--output OUTPUT File for output in JSON format, defaults to
|
|
env[SHAKER_OUTPUT]. If it is empty, then output will
|
|
be saved to /tmp/shaker_<time_now>.json
|
|
--polling-interval POLLING_INTERVAL
|
|
How frequently the agent polls server, in seconds
|
|
--report REPORT Report file name, defaults to env[SHAKER_REPORT].
|
|
--report-template REPORT_TEMPLATE
|
|
Template for report. Can be a file name or one of
|
|
aliases: "interactive", "json". Defaults to
|
|
"interactive".
|
|
--scenario SCENARIO Comma-separated list of scenarios to play. Each entity
|
|
can be a file name or one of aliases:
|
|
"misc/instance_metadata", "misc/static_agent",
|
|
"misc/static_agents_pair",
|
|
"openstack/cross_az/full_l2",
|
|
"openstack/cross_az/full_l3_east_west",
|
|
"openstack/cross_az/full_l3_north_south",
|
|
"openstack/cross_az/perf_l2",
|
|
"openstack/cross_az/perf_l3_east_west",
|
|
"openstack/cross_az/perf_l3_north_south",
|
|
"openstack/cross_az/udp_l2",
|
|
"openstack/cross_az/udp_l2_mss8950",
|
|
"openstack/cross_az/udp_l3_east_west",
|
|
"openstack/dense_l2", "openstack/dense_l3_east_west",
|
|
"openstack/dense_l3_north_south",
|
|
"openstack/external/dense_l3_north_south_no_fip",
|
|
"openstack/external/dense_l3_north_south_with_fip",
|
|
"openstack/external/full_l3_north_south_no_fip",
|
|
"openstack/external/full_l3_north_south_with_fip",
|
|
"openstack/external/perf_l3_north_south_no_fip",
|
|
"openstack/external/perf_l3_north_south_with_fip",
|
|
"openstack/full_l2", "openstack/full_l3_east_west",
|
|
"openstack/full_l3_north_south", "openstack/perf_l2",
|
|
"openstack/perf_l3_east_west",
|
|
"openstack/perf_l3_north_south",
|
|
"openstack/qos/perf_l2", "openstack/udp_l2",
|
|
"openstack/udp_l3_east_west",
|
|
"openstack/udp_l3_north_south", "spot/ping",
|
|
"spot/tcp", "spot/udp". Defaults to
|
|
env[SHAKER_SCENARIO].
|
|
--scenario_availability_zone SCENARIO_AVAILABILITY_ZONE
|
|
Comma-separated list of availability_zone. If
|
|
specified this setting will override the
|
|
availability_zone accomodation setting in the scenario
|
|
test definition.Defaults to SCENARIO_AVAILABILITY_ZONE
|
|
--scenario_compute_nodes SCENARIO_COMPUTE_NODES
|
|
Number of compute_nodes. If specified this setting
|
|
will override the compute_nodes accomodation setting
|
|
in the scenario test definition. Defaults to
|
|
SCENARIO_COMPUTE_NODES
|
|
--server-endpoint SERVER_ENDPOINT
|
|
Address for server connections (host:port), defaults
|
|
to env[SHAKER_SERVER_ENDPOINT].
|
|
--subunit SUBUNIT Subunit stream file name, defaults to
|
|
env[SHAKER_SUBUNIT].
|
|
--syslog-log-facility SYSLOG_LOG_FACILITY
|
|
Syslog facility to receive log lines. This option is
|
|
ignored if log_config_append is set.
|
|
--use-journal Enable journald for logging. If running in a systemd
|
|
environment you may wish to enable journal support.
|
|
Doing so will use the journal native protocol which
|
|
includes structured metadata in addition to log
|
|
messages.This option is ignored if log_config_append
|
|
is set.
|
|
--use-json Use JSON formatting for logging. This option is
|
|
ignored if log_config_append is set.
|
|
--use-syslog Use syslog for logging. Existing syslog format is
|
|
DEPRECATED and will be changed later to honor RFC5424.
|
|
This option is ignored if log_config_append is set.
|
|
--watch-log-file Uses logging handler designed to watch file system.
|
|
When log file is moved or removed this handler will
|
|
open a new log file with specified path
|
|
instantaneously. It makes sense only if log_file
|
|
option is specified and Linux platform is used. This
|
|
option is ignored if log_config_append is set.
|