performa/shaker
Code Issues Proposed changes

Update scenarios and documentation

Browse Source 
Change-Id: I99556bf7143a04b712ebb6686061c05b5c39ebc3
This commit is contained in:
Ilya Shakhat 2015-12-24 16:34:53 +03:00
parent 9ba58fb214
commit f882160812
33 changed files with 366 additions and 188 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

29
README.rst
View File

@ -1,31 +1,35 @@
Shaker
======
The distributed data-plane testing tool for OpenStack.
**The distributed data-plane testing tool built for OpenStack.**
Features
--------
Shaker wraps around popular system network testing tools like
`iperf <https://iperf.fr/>`_, `iperf3 <https://iperf.fr/>`_
and netperf (with help of `flent <https://flent.org/>`_).
Shaker is able to deploy OpenStack instances and networks in different
topologies. Shaker scenario specifies the deployment and list of tests
to execute. Additionally tests may be tuned dynamically in command-line.
Features:
* User-defined topology via Heat templates
* Simultaneously test execution on multiple instances
* Pluggable tools
* Interactive report with stats and charts
* Built-in SLA verification
Requirements
------------
Requirements:
* Shaker server routable from OpenStack cloud
* Admin-user access to OpenStack API
Setup
-----
Setup:
1. ``pip install pyshaker`` - installs the tool and all its python dependencies
2. ``shaker-image-builder`` - builds shaker image and stores it in Glance
Run
---
Run:
``shaker --server-endpoint <host:port> --scenario <scenario.yaml> --report <report.html>``
@ -35,8 +39,7 @@ Run
* ``<report.html>`` - file to store the report
Links
-----
* Launchpad - https://launchpad.net/shaker/
* Docs - http://pyshaker.readthedocs.org/
Links:
* PyPi - https://pypi.python.org/pypi/pyshaker/
* Docs - http://pyshaker.readthedocs.org/
* Bugtracker - https://launchpad.net/shaker/

12
doc/source/examples.rst
View File

@ -1,6 +1,6 @@
========
Examples
========
===================
OpenStack Scenarios
===================
L2 Same Domain
--------------
@ -15,7 +15,7 @@ How To Run
^^^^^^^^^^
.. code::
shaker --server-endpoint <host:port> --scenario <full_l2.yaml> --report <full_l2.html>
shaker --server-endpoint <host:port> --scenario networking/full_l2 --report <full_l2.html>
Scenario
^^^^^^^^
@ -36,7 +36,7 @@ How To Run
^^^^^^^^^^
.. code::
shaker --server-endpoint <host:port> --scenario <full_l3_east_west.yaml> --report <full_l3_east_west.html>
shaker --server-endpoint <host:port> --scenario networking/full_l3_east_west --report <full_l3_east_west.html>
Scenario
^^^^^^^^
@ -59,7 +59,7 @@ How To Run
^^^^^^^^^^
.. code::
shaker --server-endpoint <host:port> --scenario <full_l3_north_south.yaml> --report <full_l3_north_south.html>
shaker --server-endpoint <host:port> --scenario networkingfull_l3_north_south --report <full_l3_north_south.html>
Scenario
^^^^^^^^

60
doc/source/examples_spot.rst Normal file
View File

@ -0,0 +1,60 @@
==============
Spot Scenarios
==============
TCP
---
This scenario tests TCP bandwidth to the destination host. By default it sends traffic to one
of public iperf servers. This can be overridden via parameter ``--matrix "{host:<host>}"``.
How To Run
^^^^^^^^^^
1. Run the scenario with defaults and generate interactive report into file ``report.html``::
.. code::
shaker-spot --scenario spot/tcp --report report.html
2. Run the scenario with overridden target host (10.0.0.2) and store raw result::
.. code::
shaker-spot --scenario spot/tcp --matrix "{host:10.0.0.2}" --output report.json
3. Run the scenario with overridden target host (10.0.0.2) and store SLA verification results in `subunit <https://launchpad.net/subunit>`_ stream file::
.. code::
shaker-spot --scenario spot/tcp --matrix "{host:10.0.0.2}" --subunit report.subunit
4. Run the scenario against the list of target hosts and store report::
.. code::
shaker-spot --scenario spot/tcp --matrix "{host:[10.0.0.2, 10.0.0.3]}" --output report.html
Scenario
^^^^^^^^
.. literalinclude:: ../../shaker/scenarios/spot/tcp.yaml
UDP
---
This scenario tests UDP packets per second to the destination host. By default it sends traffic to one
of public iperf servers. This can be overridden via parameter ``--matrix "{host:<host>}"``.
How To Run
^^^^^^^^^^
.. code::
shaker-spot --scenario spot/udp --report report.html
Scenario
^^^^^^^^
.. literalinclude:: ../../shaker/scenarios/spot/udp.yaml

10
doc/source/index.rst
View File

@ -1,7 +1,14 @@
Welcome to Shaker!
==================
The distributed data-plane testing tool for OpenStack.
**The distributed data-plane testing tool built for OpenStack.**
Shaker wraps around popular system network testing tools like
`iperf <https://iperf.fr/>`_, `iperf3 <https://iperf.fr/>`_
and netperf (with help of `flent <https://flent.org/>`_).
Shaker is able to deploy OpenStack instances and networks in different
topologies. Shaker scenario specifies the deployment and list of tests
to execute. Additionally tests may be tuned dynamically in command-line.
.. toctree::
:maxdepth: 2
@ -10,6 +17,7 @@ The distributed data-plane testing tool for OpenStack.
usage
tools
examples
examples_spot
architecture
contributing

13
doc/source/installation.rst
View File

@ -33,6 +33,11 @@ Installation on fresh system requires additional libraries needed by some of dep
Deployment
^^^^^^^^^^
Shaker may be used to run scenarios against OpenStack or hardware nodes.
OpenStack Deployment
~~~~~~~~~~~~~~~~~~~~
.. image:: images/architecture.*
Requirements:
@ -43,7 +48,7 @@ Requirements:
First Run
^^^^^^^^^
~~~~~~~~~
Build the master image. The process downloads Ubuntu cloud image, installs all necessary packages and stores
snapshot into Glance. This snapshot is used by ``shaker`` as base of instances.
@ -52,3 +57,9 @@ snapshot into Glance. This snapshot is used by ``shaker`` as base of instances.
$ shaker-image-builder
Non-OpenStack Deployment (aka Spot mode)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To run scenarios against remote nodes (``shaker-spot`` command) install shaker on the local host.
Make sure all necessary system tools are installed too.

8
doc/source/tools/shaker-spot.txt
View File

@ -75,16 +75,15 @@ optional arguments:
"interactive".
--scenario SCENARIO Scenario to play. Can be a file name or one of
aliases: "misc/instance_metadata",
"misc/static_agent", "misc/static_agent_networking",
"misc/static_agents_pair",
"misc/static_agent", "misc/static_agents_pair",
"networking/cross_az/full_l2",
"networking/cross_az/full_l3_east_west",
"networking/cross_az/full_l3_north_south",
"networking/cross_az/perf_l2",
"networking/cross_az/perf_l3_east_west",
"networking/cross_az/perf_l3_north_south",
"networking/cross_az/udp_full_l2",
"networking/cross_az/udp_l2",
"networking/cross_az/udp_l2_mss8950",
"networking/cross_az/udp_l3_east_west",
"networking/dense_l2",
"networking/dense_l3_east_west",
@ -95,8 +94,7 @@ optional arguments:
"networking/perf_l3_north_south", "networking/udp_l2",
"networking/udp_l3_east_west",
"networking/udp_l3_north_south", "spot/tcp",
"spot/tcp_bandwidth". Defaults to
env[SHAKER_SCENARIO].
"spot/udp". Defaults to env[SHAKER_SCENARIO].
--subunit SUBUNIT Subunit stream file name, defaults to
env[SHAKER_SUBUNIT].
--syslog-log-facility SYSLOG_LOG_FACILITY

8
doc/source/tools/shaker.txt
View File

@ -124,16 +124,15 @@ optional arguments:
"interactive".
--scenario SCENARIO Scenario to play. Can be a file name or one of
aliases: "misc/instance_metadata",
"misc/static_agent", "misc/static_agent_networking",
"misc/static_agents_pair",
"misc/static_agent", "misc/static_agents_pair",
"networking/cross_az/full_l2",
"networking/cross_az/full_l3_east_west",
"networking/cross_az/full_l3_north_south",
"networking/cross_az/perf_l2",
"networking/cross_az/perf_l3_east_west",
"networking/cross_az/perf_l3_north_south",
"networking/cross_az/udp_full_l2",
"networking/cross_az/udp_l2",
"networking/cross_az/udp_l2_mss8950",
"networking/cross_az/udp_l3_east_west",
"networking/dense_l2",
"networking/dense_l3_east_west",
@ -144,8 +143,7 @@ optional arguments:
"networking/perf_l3_north_south", "networking/udp_l2",
"networking/udp_l3_east_west",
"networking/udp_l3_north_south", "spot/tcp",
"spot/tcp_bandwidth". Defaults to
env[SHAKER_SCENARIO].
"spot/udp". Defaults to env[SHAKER_SCENARIO].
--server-endpoint SERVER_ENDPOINT
Address for server connections (host:port), defaults
to env[SHAKER_SERVER_ENDPOINT].

59
doc/source/usage.rst
View File

@ -5,12 +5,43 @@ Usage
Configuration
-------------
The connection to OpenStack is configured using standard ``openrc`` file. (Refer to
http://docs.openstack.org/cli-reference/content/cli_openrc.html on how to retrieve it)
For OpenStack scenarios the connection is configured using standard ``openrc`` file. (Refer to
http://docs.openstack.org/cli-reference/content/cli_openrc.html on how to retrieve it).
The config can be passed to Shaker rather by sourcing into system env ``source openrc``
or via set of CLI parameters ``--os-tenant-name``, ``--os-username``, ``--os-password``,
``--os-auth-url`` and ``--os-region-name``. Note that Shaker requires a user with admin privileges.
Connection to SSL endpoints is configuredby parameters ``--os-cacert`` and ``--os-insecure``
(to disable strict verifications of certificates).
Common Parameters
-----------------
The following parameters are applicable for both OpenStack mode (`shaker`) and spot mode (`shaker-spot`).
1. Run the scenario with defaults and generate interactive report into file `report.html`::
.. code::
shaker --scenario <scenario> --report report.html
2. Run the scenario and store raw result::
.. code::
shaker --scenario <scenario> --output output.json
3. Run the scenario and store SLA verification results in `subunit <https://launchpad.net/subunit>`_ stream file::
.. code::
shaker --scenario <scenario> --subunit report.subunit
4. Generate report from the raw data::
.. code::
shaker-report --input output.json --output report.html
Scenario Explained
@ -88,14 +119,16 @@ Test classes
Tools are configured via key-value attributes in test definition. For all networking tools Shaker offers unified parameters, that are translated
automatically.
iperf_graph, iperf:
~~~~~~~~~~~~~~~~~~~
iperf3, iperf, iperf_graph:
~~~~~~~~~~~~~~~~~~~~~~~~~~~
* ``time`` - time in seconds to transmit for, defaults to `60`
* ``udp`` - use UDP instead of TCP, defaults to `TCP`
* ``interval`` - seconds between periodic bandwidth reports, defaults to `1 s`
* ``bandwidth`` - for UDP, bandwidth to send at in bits/sec, defaults to `1 Mbit/s`
* ``threads`` - number of parallel client threads to run
* ``host`` - the address of destination host to run the tool against, defaults to IP address of slave agent
* ``datagram_size`` - the size of UDP datagrams
* ``mss`` - set TCP maximum segment size
flent:
~~~~~~
@ -115,3 +148,19 @@ shell:
~~~~~~
* ``program`` - run single program
* ``script`` - run bash script
SLA validation
^^^^^^^^^^^^^^
Test case can contain SLA rules that are calculated upon test completion.
Every rule has 2 parts: record selector and condition. The record selector allows
to filter only subset of all records, e.g. of type `agent` to filter records produced
by a single agent. The condition applies to particular statistics.
SLA examples:
* ``[type == 'agent'] >> (stats.bandwidth.min > 1000)`` - require min bandwidth on every agent be at least 1000 Mbit
* ``[type == 'agent'] >> (stderr == '')`` - require stderr to be empty
Results of SLA validation can be obtained by generating output in subunit format.
To do this a file name should be provided via `--subunit` parameter.

11
etc/shaker.conf
View File

@ -87,7 +87,7 @@
# List of logger=LEVEL pairs. This option is ignored if log_config_append is
# set. (list value)
#default_log_levels = amqp=WARN,amqplib=WARN,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=INFO,iso8601=WARN,requests.packages.urllib3.connectionpool=WARN,urllib3.connectionpool=WARN,websocket=WARN,requests.packages.urllib3.util.retry=WARN,urllib3.util.retry=WARN,keystonemiddleware=WARN,routes.middleware=WARN,stevedore=WARN,taskflow=WARN
#default_log_levels = amqp=WARN,amqplib=WARN,boto=WARN,qpid=WARN,sqlalchemy=WARN,suds=INFO,oslo.messaging=INFO,iso8601=WARN,requests.packages.urllib3.connectionpool=WARN,urllib3.connectionpool=WARN,websocket=WARN,requests.packages.urllib3.util.retry=WARN,urllib3.util.retry=WARN,keystonemiddleware=WARN,routes.middleware=WARN,stevedore=WARN,taskflow=WARN,keystoneauth=WARN
# Enables or disables publication of error events. (boolean value)
#publish_errors = false
@ -159,20 +159,19 @@
#cleanup_on_error = true
# Scenario to play. Can be a file name or one of aliases:
# "misc/instance_metadata", "misc/static_agent",
# "misc/static_agent_networking", "misc/static_agents_pair",
# "misc/instance_metadata", "misc/static_agent", "misc/static_agents_pair",
# "networking/cross_az/full_l2", "networking/cross_az/full_l3_east_west",
# "networking/cross_az/full_l3_north_south", "networking/cross_az/perf_l2",
# "networking/cross_az/perf_l3_east_west",
# "networking/cross_az/perf_l3_north_south", "networking/cross_az/udp_full_l2",
# "networking/cross_az/udp_l2", "networking/cross_az/udp_l3_east_west",
# "networking/cross_az/perf_l3_north_south", "networking/cross_az/udp_l2",
# "networking/cross_az/udp_l2_mss8950", "networking/cross_az/udp_l3_east_west",
# "networking/dense_l2", "networking/dense_l3_east_west",
# "networking/dense_l3_north_south", "networking/full_l2",
# "networking/full_l3_east_west", "networking/full_l3_north_south",
# "networking/perf_l2", "networking/perf_l3_east_west",
# "networking/perf_l3_north_south", "networking/udp_l2",
# "networking/udp_l3_east_west", "networking/udp_l3_north_south", "spot/tcp",
# "spot/tcp_bandwidth". Defaults to env[SHAKER_SCENARIO]. (string value)
# "spot/udp". Defaults to env[SHAKER_SCENARIO]. (string value)
#scenario = <None>
# Set the matrix of parameters for the scenario. The value is specified in YAML

14
shaker/scenarios/networking/cross_az/full_l2.yaml
View File

@ -10,10 +10,20 @@ execution:
progression: quadratic
tests:
-
title: TCP download
title: Download
class: flent
method: tcp_download
sla:
- "[type == 'agent'] >> (stats.tcp_download.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: TCP bi-directional
title: Upload
class: flent
method: tcp_upload
sla:
- "[type == 'agent'] >> (stats.tcp_upload.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: Bi-directional
class: flent
method: tcp_bidirectional

23
shaker/scenarios/networking/cross_az/full_l3_east_west.yaml
View File

@ -10,19 +10,20 @@ execution:
progression: quadratic
tests:
-
title: Iperf TCP
class: iperf_graph
time: 60
-
title: TCP download
title: Download
class: flent
method: tcp_download
sla:
- "[type == 'agent'] >> (stats.tcp_download.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: TCP bi-directional
title: Upload
class: flent
method: tcp_upload
sla:
- "[type == 'agent'] >> (stats.tcp_upload.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: Bi-directional
class: flent
method: tcp_bidirectional
-
title: Iperf UDP
class: iperf
udp: 1
bandwidth: 1000M

23
shaker/scenarios/networking/cross_az/full_l3_north_south.yaml
View File

@ -11,19 +11,20 @@ execution:
progression: quadratic
tests:
-
title: Iperf TCP
class: iperf_graph
time: 60
-
title: TCP download
title: Download
class: flent
method: tcp_download
sla:
- "[type == 'agent'] >> (stats.tcp_download.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: TCP bi-directional
title: Upload
class: flent
method: tcp_upload
sla:
- "[type == 'agent'] >> (stats.tcp_upload.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: Bi-directional
class: flent
method: tcp_bidirectional
-
title: Iperf UDP
class: iperf
udp: 1
bandwidth: 1000M

18
shaker/scenarios/networking/cross_az/perf_l2.yaml
View File

@ -9,16 +9,16 @@ deployment:
execution:
tests:
-
title: Iperf TCP
class: iperf_graph
title: TCP
class: iperf3
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 900)"
- "[type == 'agent'] >> (stats.retransmits.max < 50)"
-
title: Iperf UDP
class: iperf_graph
udp: 1
title: UDP
class: iperf3
udp: on
bandwidth: 1000M
datagram_size: 32
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.packets.mean > 20000)"

18
shaker/scenarios/networking/cross_az/perf_l3_east_west.yaml
View File

@ -9,16 +9,16 @@ deployment:
execution:
tests:
-
title: Iperf TCP
class: iperf_graph
title: TCP
class: iperf3
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 900)"
- "[type == 'agent'] >> (stats.retransmits.max < 50)"
-
title: Iperf UDP
class: iperf_graph
udp: 1
title: UDP
class: iperf3
udp: on
bandwidth: 1000M
datagram_size: 32
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.packets.mean > 20000)"

18
shaker/scenarios/networking/cross_az/perf_l3_north_south.yaml
View File

@ -10,16 +10,16 @@ deployment:
execution:
tests:
-
title: Iperf TCP
class: iperf_graph
title: TCP
class: iperf3
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 900)"
- "[type == 'agent'] >> (stats.retransmits.max < 50)"
-
title: Iperf UDP
class: iperf_graph
udp: 1
title: UDP
class: iperf3
udp: on
bandwidth: 1000M
datagram_size: 32
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.packets.mean > 20000)"

2
shaker/scenarios/networking/cross_az/udp_l2.yaml
View File

@ -10,7 +10,7 @@ execution:
progression: quadratic
tests:
-
title: Iperf UDP
title: UDP
class: iperf3
udp: on
bandwidth: 1000M

2
shaker/scenarios/networking/cross_az/udp_full_l2.yaml → shaker/scenarios/networking/cross_az/udp_l2_mss8950.yaml
View File

@ -10,7 +10,7 @@ execution:
progression: quadratic
tests:
-
title: Iperf UDP
title: UDP
class: iperf
udp: 1
threads: 2

2
shaker/scenarios/networking/cross_az/udp_l3_east_west.yaml
View File

@ -10,7 +10,7 @@ execution:
progression: quadratic
tests:
-
title: Iperf UDP
title: UDP
class: iperf3
udp: on
bandwidth: 1000M

9
shaker/scenarios/networking/dense_l2.yaml
View File

@ -10,10 +10,6 @@ deployment:
execution:
progression: linear
tests:
-
title: Iperf TCP
class: iperf_graph
time: 60
-
title: TCP download
class: flent
@ -22,8 +18,3 @@ execution:
title: TCP bi-directional
class: flent
method: tcp_bidirectional
-
title: Iperf UDP
class: iperf
udp: 1
bandwidth: 1000M

9
shaker/scenarios/networking/dense_l3_east_west.yaml
View File

@ -9,10 +9,6 @@ deployment:
execution:
progression: linear
tests:
-
title: Iperf TCP
class: iperf_graph
time: 60
-
title: TCP download
class: flent
@ -21,8 +17,3 @@ execution:
title: TCP bi-directional
class: flent
method: tcp_bidirectional
-
title: Iperf UDP
class: iperf
udp: 1
bandwidth: 1000M

9
shaker/scenarios/networking/dense_l3_north_south.yaml
View File

@ -10,10 +10,6 @@ deployment:
execution:
progression: linear
tests:
-
title: Iperf TCP
class: iperf_graph
time: 60
-
title: TCP download
class: flent
@ -22,8 +18,3 @@ execution:
title: TCP bi-directional
class: flent
method: tcp_bidirectional
-
title: Iperf UDP
class: iperf
udp: 1
bandwidth: 1000M

23
shaker/scenarios/networking/full_l2.yaml
View File

@ -10,19 +10,20 @@ execution:
progression: quadratic
tests:
-
title: Iperf TCP
class: iperf_graph
time: 60
-
title: TCP download
title: Download
class: flent
method: tcp_download
sla:
- "[type == 'agent'] >> (stats.tcp_download.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: TCP bi-directional
title: Upload
class: flent
method: tcp_upload
sla:
- "[type == 'agent'] >> (stats.tcp_upload.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: Bi-directional
class: flent
method: tcp_bidirectional
-
title: Iperf UDP
class: iperf
udp: 1
bandwidth: 1000M

23
shaker/scenarios/networking/full_l3_east_west.yaml
View File

@ -10,19 +10,20 @@ execution:
progression: quadratic
tests:
-
title: Iperf TCP
class: iperf_graph
time: 60
-
title: TCP download
title: Download
class: flent
method: tcp_download
sla:
- "[type == 'agent'] >> (stats.tcp_download.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: TCP bi-directional
title: Upload
class: flent
method: tcp_upload
sla:
- "[type == 'agent'] >> (stats.tcp_upload.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: Bi-directional
class: flent
method: tcp_bidirectional
-
title: Iperf UDP
class: iperf
udp: 1
bandwidth: 1000M

23
shaker/scenarios/networking/full_l3_north_south.yaml
View File

@ -11,19 +11,20 @@ execution:
progression: quadratic
tests:
-
title: Iperf TCP
class: iperf_graph
time: 60
-
title: TCP download
title: Download
class: flent
method: tcp_download
sla:
- "[type == 'agent'] >> (stats.tcp_download.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: TCP bi-directional
title: Upload
class: flent
method: tcp_upload
sla:
- "[type == 'agent'] >> (stats.tcp_upload.mean > 900)"
- "[type == 'agent'] >> (stats.ping_icmp.max < 0.5)"
-
title: Bi-directional
class: flent
method: tcp_bidirectional
-
title: Iperf UDP
class: iperf
udp: 1
bandwidth: 1000M

18
shaker/scenarios/networking/perf_l2.yaml
View File

@ -9,16 +9,16 @@ deployment:
execution:
tests:
-
title: Iperf TCP
class: iperf_graph
title: TCP
class: iperf3
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 900)"
- "[type == 'agent'] >> (stats.retransmits.max < 50)"
-
title: Iperf UDP
class: iperf_graph
udp: 1
title: UDP
class: iperf3
udp: on
bandwidth: 1000M
datagram_size: 32
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.packets.mean > 20000)"

18
shaker/scenarios/networking/perf_l3_east_west.yaml
View File

@ -9,16 +9,16 @@ deployment:
execution:
tests:
-
title: Iperf TCP
class: iperf_graph
title: TCP
class: iperf3
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 900)"
- "[type == 'agent'] >> (stats.retransmits.max < 50)"
-
title: Iperf UDP
class: iperf_graph
udp: 1
title: UDP
class: iperf3
udp: on
bandwidth: 1000M
datagram_size: 32
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.packets.mean > 20000)"

18
shaker/scenarios/networking/perf_l3_north_south.yaml
View File

@ -10,16 +10,16 @@ deployment:
execution:
tests:
-
title: Iperf TCP
class: iperf_graph
title: TCP
class: iperf3
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 900)"
- "[type == 'agent'] >> (stats.retransmits.max < 50)"
-
title: Iperf UDP
class: iperf_graph
udp: 1
title: UDP
class: iperf3
udp: on
bandwidth: 1000M
datagram_size: 32
sla:
- "[type == 'agent'] >> (stats.bandwidth.min > 1000)"
- "[type == 'agent'] >> (stats.bandwidth.mean > 2000)"
- "[type == 'agent'] >> (stats.packets.mean > 20000)"

2
shaker/scenarios/networking/udp_l2.yaml
View File

@ -10,7 +10,7 @@ execution:
progression: quadratic
tests:
-
title: Iperf UDP
title: UDP
class: iperf3
udp: on
bandwidth: 1000M

2
shaker/scenarios/networking/udp_l3_east_west.yaml
View File

@ -10,7 +10,7 @@ execution:
progression: quadratic
tests:
-
title: Iperf UDP
title: UDP
class: iperf3
udp: on
bandwidth: 1000M

2
shaker/scenarios/networking/udp_l3_north_south.yaml
View File

@ -11,7 +11,7 @@ execution:
progression: quadratic
tests:
-
title: Iperf UDP
title: UDP
class: iperf3
udp: on
bandwidth: 1000M

5
shaker/scenarios/spot/tcp.yaml
View File

@ -3,7 +3,7 @@ title: TCP bandwidth
description:
This scenario uses iperf3 to measure TCP bandwidth between local host and
ping.online.net (or against hosts provided via CLI). SLA check is verified
and expects the speed to be at least 90Mbit.
and expects the speed to be at least 90Mbit and at most 20 retransmitts.
execution:
tests:
@ -11,6 +11,7 @@ execution:
title: TCP
class: iperf3
host: ping.online.net
time: 10
time: 20
sla:
- "[type == 'agent'] >> (stats.bandwidth.mean > 90)"
- "[type == 'agent'] >> (stats.retransmits.max < 20)"

18
shaker/scenarios/spot/udp.yaml Normal file
View File

@ -0,0 +1,18 @@
title: UDP bandwidth
description:
This scenario uses iperf3 to measure UDP bandwidth between local host and
ping.online.net (or against hosts provided via CLI). SLA check is verified
and requires at least 10 000 packets per second.
execution:
tests:
-
title: UDP
class: iperf3
host: ping.online.net
udp: on
time: 20
bandwidth: 1000M
sla:
- "[type == 'agent'] >> (stats.packets.mean > 10000)"

45
shaker/tests/test_scenarios.py Normal file
View File

@ -0,0 +1,45 @@
# Copyright (c) 2015 Mirantis Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import functools
import os
import six
import testtools
import yaml
class TestReport(testtools.TestCase):
def _read_raw_file(self, file_name):
if six.PY3:
opener = functools.partial(open, encoding='utf8')
else:
opener = open
with opener(file_name, 'r') as content_file:
return content_file.read()
def test_yaml_valid(self):
for dir_data in os.walk('shaker/scenarios'):
dir_path, dir_names, file_names = dir_data
for file_name in file_names:
if not file_name.endswith('.yaml'):
continue
cnt = self._read_raw_file(os.path.join(dir_path, file_name))
try:
yaml.safe_load(cnt)
except Exception as e:
self.fail('File %s is invalid: %s' % (file_name, e))