performa/shaker
Code Issues Proposed changes

Implement agent-less execution mode (aka spot)

Browse Source 
Added new entry-point shaker-spot, which allows to execute scenarios
from the local node. Example of such scenario is spot/tcp which uses
iperf3 to measure bandwidth against one of public hosts.

Change-Id: I9303785501c3af7212ca590b8de63218ca877cd6
This commit is contained in:
Ilya Shakhat 2015-12-21 16:33:59 +03:00
parent 0fd0c7e02b
commit 59723a8099
13 changed files with 288 additions and 54 deletions

8
doc/source/tools.rst

@ -10,6 +10,14 @@ Executes specified scenario in OpenStack cloud, stores results and generates HTM
.. literalinclude:: tools/shaker.txt
shaker-spot
-----------
Executes specified scenario from the local node, stores results and generates HTML report.
.. literalinclude:: tools/shaker-spot.txt
shaker-image-builder
--------------------

116
doc/source/tools/shaker-spot.txt Normal file

@ -0,0 +1,116 @@
usage: shaker-spot [-h] [--config-dir DIR] [--config-file PATH] [--debug]
[--log-config-append PATH] [--log-date-format DATE_FORMAT]
[--log-dir LOG_DIR] [--log-file PATH] [--log-format FORMAT]
[--no-report-on-error] [--nodebug] [--nono-report-on-error]
[--nouse-syslog] [--nouse-syslog-rfc-format] [--noverbose]
[--nowatch-log-file] [--output OUTPUT] [--report REPORT]
[--report-template REPORT_TEMPLATE] [--scenario SCENARIO]
[--subunit SUBUNIT]
[--syslog-log-facility SYSLOG_LOG_FACILITY] [--use-syslog]
[--use-syslog-rfc-format] [--verbose] [--version]
[--watch-log-file]
optional arguments:
-h, --help show this help message and exit
--config-dir DIR Path to a config directory to pull *.conf files from.
This file set is sorted, so as to provide a
predictable parse order if individual options are
over-ridden. The set is parsed after the file(s)
specified via previous --config-file, arguments hence
over-ridden options in the directory take precedence.
--config-file PATH Path to a config file to use. Multiple config files
can be specified, with values in later files taking
precedence. The default files used are: None.
--debug, -d Print debugging output (set logging level to DEBUG
instead of default INFO level).
--log-config-append PATH, --log_config PATH
The name of a logging configuration file. This file is
appended to any existing logging configuration files.
For details about logging configuration files, see the
Python logging module documentation. Note that when
logging configuration files are used then all logging
configuration is set in the configuration file and
other logging configuration options are ignored (for
example, log_format).
--log-date-format DATE_FORMAT
Format string for %(asctime)s in log records. Default:
None . This option is ignored if log_config_append is
set.
--log-dir LOG_DIR, --logdir LOG_DIR
(Optional) The base directory used for relative --log-
file paths. This option is ignored if
log_config_append is set.
--log-file PATH, --logfile PATH
(Optional) Name of log file to output to. If no
default is set, logging will go to stdout. This option
is ignored if log_config_append is set.
--log-format FORMAT DEPRECATED. A logging.Formatter log message format
string which may use any of the available
logging.LogRecord attributes. This option is
deprecated. Please use logging_context_format_string
and logging_default_format_string instead. This option
is ignored if log_config_append is set.
--no-report-on-error Do not generate report for failed scenarios
--nodebug The inverse of --debug
--nono-report-on-error
The inverse of --no-report-on-error
--nouse-syslog The inverse of --use-syslog
--nouse-syslog-rfc-format
The inverse of --use-syslog-rfc-format
--noverbose The inverse of --verbose
--nowatch-log-file The inverse of --watch-log-file
--output OUTPUT File for output in JSON format, defaults to
env[SHAKER_OUTPUT].
--report REPORT Report file name, defaults to env[SHAKER_REPORT].
--report-template REPORT_TEMPLATE
Template for report. Can be a file name or one of
aliases: "interactive", "json". Defaults to
"interactive".
--scenario SCENARIO Scenario to play. Can be a file name or one of
aliases: "misc/instance_metadata",
"misc/static_agent", "misc/static_agent_networking",
"misc/static_agents_pair",
"networking/cross_az/full_l2",
"networking/cross_az/full_l3_east_west",
"networking/cross_az/full_l3_north_south",
"networking/cross_az/perf_l2",
"networking/cross_az/perf_l3_east_west",
"networking/cross_az/perf_l3_north_south",
"networking/cross_az/udp_full_l2",
"networking/cross_az/udp_l2",
"networking/cross_az/udp_l3_east_west",
"networking/dense_l2",
"networking/dense_l3_east_west",
"networking/dense_l3_north_south",
"networking/full_l2", "networking/full_l3_east_west",
"networking/full_l3_north_south",
"networking/perf_l2", "networking/perf_l3_east_west",
"networking/perf_l3_north_south", "networking/udp_l2",
"networking/udp_l3_east_west",
"networking/udp_l3_north_south", "spot/tcp",
"spot/tcp_bandwidth". Defaults to
env[SHAKER_SCENARIO].
--subunit SUBUNIT Subunit stream file name, defaults to
env[SHAKER_SUBUNIT].
--syslog-log-facility SYSLOG_LOG_FACILITY
Syslog facility to receive log lines. This option is
ignored if log_config_append is set.
--use-syslog Use syslog for logging. Existing syslog format is
DEPRECATED and will be changed later to honor RFC5424.
This option is ignored if log_config_append is set.
--use-syslog-rfc-format
(Optional) Enables or disables syslog rfc5424 format
for logging. If enabled, prefixes the MSG part of the
syslog message with APP-NAME (RFC5424). The format
without the APP-NAME is deprecated in Kilo, and will
be removed in Mitaka, along with this option. This
option is ignored if log_config_append is set.
--verbose, -v If set to false, will disable INFO logging level,
making WARNING the default.
--version show program's version number and exit
--watch-log-file (Optional) Uses logging handler designed to watch file
system. When log file is moved or removed this handler
will open a new log file with specified path
instantaneously. It makes sense only if log-file
option is specified and Linux platform is used. This
option is ignored if log_config_append is set.

3
doc/source/tools/shaker.txt

@ -137,7 +137,8 @@ optional arguments:
"networking/perf_l2", "networking/perf_l3_east_west",
"networking/perf_l3_north_south", "networking/udp_l2",
"networking/udp_l3_east_west",
"networking/udp_l3_north_south". Defaults to
"networking/udp_l3_north_south", "spot/tcp",
"spot/tcp_bandwidth". Defaults to
env[SHAKER_SCENARIO].
--server-endpoint SERVER_ENDPOINT
Address for server connections (host:port), defaults

10
etc/shaker.conf

@ -171,14 +171,17 @@
# "networking/full_l3_east_west", "networking/full_l3_north_south",
# "networking/perf_l2", "networking/perf_l3_east_west",
# "networking/perf_l3_north_south", "networking/udp_l2",
# "networking/udp_l3_east_west", "networking/udp_l3_north_south". Defaults to
# env[SHAKER_SCENARIO]. (string value)
# "networking/udp_l3_east_west", "networking/udp_l3_north_south", "spot/tcp",
# "spot/tcp_bandwidth". Defaults to env[SHAKER_SCENARIO]. (string value)
#scenario = <None>
# File for output in JSON format, defaults to env[SHAKER_OUTPUT]. (string
# value)
#output = <None>
# Do not generate report for failed scenarios (boolean value)
#no_report_on_error = false
# Timeout to treat agent as lost in seconds (integer value)
#agent_loss_timeout = 60
@ -186,9 +189,6 @@
# and start of scenario execution). (integer value)
#agent_join_timeout = 600
# Do not generate report for failed scenarios (boolean value)
#no_report_on_error = false
# Template for report. Can be a file name or one of aliases: "interactive",
# "json". Defaults to "interactive". (string value)
#report_template = interactive

1
setup.cfg

@ -29,6 +29,7 @@ console_scripts =
shaker-agent = shaker.agent.agent:main
shaker-image-builder = shaker.engine.image_builder:build_image
shaker-cleanup = shaker.engine.image_builder:cleanup
shaker-spot = shaker.engine.spot:main
oslo.config.opts =
oslo_log = oslo_log._options:list_opts

23
shaker/engine/config.py

@ -104,7 +104,18 @@ OPENSTACK_OPTS = [
]
SERVER_OPTS = [
SERVER_AGENT_OPTS = [
cfg.IntOpt('agent-loss-timeout',
default=utils.env('SHAKER_AGENT_LOSS_TIMEOUT') or 60,
help='Timeout to treat agent as lost in seconds'),
cfg.IntOpt('agent-join-timeout',
default=utils.env('SHAKER_AGENT_JOIN_TIMEOUT') or 600,
help='How long to wait for agents to join in seconds (time '
'between stack deployment and start of scenario '
'execution).'),
]
SCENARIO_OPTS = [
cfg.StrOpt('scenario',
default=utils.env('SHAKER_SCENARIO'),
required=True,
@ -117,19 +128,13 @@ SERVER_OPTS = [
default=utils.env('SHAKER_OUTPUT'),
help='File for output in JSON format, '
'defaults to env[SHAKER_OUTPUT].'),
cfg.IntOpt('agent-loss-timeout',
default=utils.env('SHAKER_AGENT_LOSS_TIMEOUT') or 60,
help='Timeout to treat agent as lost in seconds'),
cfg.IntOpt('agent-join-timeout',
default=utils.env('SHAKER_AGENT_JOIN_TIMEOUT') or 600,
help='How long to wait for agents to join in seconds (time '
'between stack deployment and start of scenario '
'execution).'),
cfg.BoolOpt('no-report-on-error',
default=(utils.env('SHAKER_NO_REPORT_ON_ERROR') or False),
help='Do not generate report for failed scenarios'),
]
SERVER_OPTS = SCENARIO_OPTS + SERVER_AGENT_OPTS
REPORT_OPTS = [
cfg.StrOpt('report-template',
default=(utils.env('SHAKER_REPORT_TEMPLATE') or 'interactive'),

16
shaker/engine/deploy.py

@ -30,6 +30,10 @@ from shaker.openstack.clients import openstack
LOG = logging.getLogger(__name__)
class DeploymentException(Exception):
pass
def prepare_for_cross_az(compute_nodes, zones):
if len(zones) != 2:
LOG.warn('cross_az is specified, but len(zones) is not 2')
@ -235,10 +239,15 @@ class Deployment(object):
def deploy(self, deployment, base_dir=None, server_endpoint=None):
agents = {}
if not deployment:
# local mode, create fake agent
agents.update(dict(local=dict(id='local', mode='alone')))
if deployment.get('template'):
if not self.openstack_client:
LOG.error('OpenStack client is not initialized. Template '
'deployment is ignored.')
raise DeploymentException(
'OpenStack client is not initialized. '
'Template-based deployment is ignored.')
else:
# deploy topology specified by HOT
agents.update(self._deploy_from_hot(
@ -254,6 +263,3 @@ class Deployment(object):
if self.stack_created and cfg.CONF.cleanup_on_error:
LOG.debug('Cleaning up the stack: %s', self.stack_name)
self.openstack_client.heat.stacks.delete(self.stack_name)
else:
LOG.info('No Heat Stack clean-up as no cleanup on error'
'requested or static agents were deployed')

18
shaker/engine/quorum.py

@ -218,6 +218,20 @@ class Quorum(object):
return self._run(ExecuteOperation(executors))
class LocalQuorum(object):
def execute(self, executors):
operation = ExecuteOperation(executors)
agent_ids = operation.get_active_agent_ids()
result = {}
for agent_id in agent_ids:
task = operation.get_reply(agent_id, None)
command_res = agent_process.run_command(task.get('command'))
result[agent_id] = operation.process_reply(agent_id, command_res)
return result
def make_quorum(agent_ids, server_endpoint, polling_interval,
agent_loss_timeout, agent_join_timeout):
message_queue = messaging.MessageQueue(server_endpoint)
@ -239,3 +253,7 @@ def make_quorum(agent_ids, server_endpoint, polling_interval,
raise Exception('Agents failed to join: %s' % failed)
return quorum
def make_local_quorum():
return LocalQuorum()

47
shaker/engine/server.py

@ -106,6 +106,14 @@ def execute(quorum, execution, agents):
return records
def _under_openstack():
required = ['os_username', 'os_password', 'os_tenant_name', 'os_auth_url']
for param in required:
if param not in cfg.CONF or not cfg.CONF[param]:
return False
return True
def play_scenario(scenario):
deployment = None
output = dict(scenario=scenario, records={}, agents={})
@ -115,8 +123,7 @@ def play_scenario(scenario):
try:
deployment = deploy.Deployment()
if (cfg.CONF.os_username and cfg.CONF.os_password and
cfg.CONF.os_tenant_name and cfg.CONF.os_auth_url):
if _under_openstack():
deployment.connect_to_openstack(
cfg.CONF.os_username, cfg.CONF.os_password,
cfg.CONF.os_tenant_name, cfg.CONF.os_auth_url,
@ -125,8 +132,12 @@ def play_scenario(scenario):
cfg.CONF.os_cacert, cfg.CONF.os_insecure)
base_dir = os.path.dirname(scenario['file_name'])
agents = deployment.deploy(scenario['deployment'], base_dir=base_dir,
server_endpoint=cfg.CONF.server_endpoint)
scenario_deployment = scenario.get('deployment', {})
server_endpoint = (cfg.CONF.server_endpoint
if 'server_endpoint' in cfg.CONF else None)
agents = deployment.deploy(scenario_deployment, base_dir=base_dir,
server_endpoint=server_endpoint)
agents = _extend_agents(agents)
output['agents'] = agents
@ -135,10 +146,14 @@ def play_scenario(scenario):
if not agents:
raise Exception('No agents deployed.')
quorum = quorum_pkg.make_quorum(
agents.keys(), cfg.CONF.server_endpoint,
cfg.CONF.polling_interval, cfg.CONF.agent_loss_timeout,
cfg.CONF.agent_join_timeout)
if scenario_deployment:
quorum = quorum_pkg.make_quorum(
agents.keys(), server_endpoint,
cfg.CONF.polling_interval, cfg.CONF.agent_loss_timeout,
cfg.CONF.agent_join_timeout)
else:
# local
quorum = quorum_pkg.make_local_quorum()
output['records'] = execute(quorum, scenario['execution'], agents)
@ -163,12 +178,7 @@ def play_scenario(scenario):
return output
def main():
utils.init_config_and_logging(
config.COMMON_OPTS + config.OPENSTACK_OPTS + config.SERVER_OPTS +
config.REPORT_OPTS
)
def act():
output = dict(records={}, agents={}, scenarios={}, tests={})
for scenario_param in [cfg.CONF.scenario]:
@ -202,5 +212,14 @@ def main():
cfg.CONF.report, cfg.CONF.subunit)
def main():
utils.init_config_and_logging(
config.COMMON_OPTS + config.OPENSTACK_OPTS + config.SERVER_OPTS +
config.REPORT_OPTS
)
act()
if __name__ == "__main__":
main()

35
shaker/engine/spot.py Normal file

@ -0,0 +1,35 @@
# Copyright (c) 2015 Mirantis Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from oslo_log import log as logging
from shaker.engine import config
from shaker.engine import server
from shaker.engine import utils
LOG = logging.getLogger(__name__)
def main():
utils.init_config_and_logging(
config.SCENARIO_OPTS + config.REPORT_OPTS
)
server.act()
if __name__ == "__main__":
main()

20
shaker/scenarios/misc/static_agent_networking.yaml

@ -1,20 +0,0 @@
title: Static Networking
description:
This scenario runs tests on pre-deployed static agents
deployment:
agents:
-
id: the-agent
mode: alone
execution:
tests:
-
title: TCP
class: iperf3
host: ping.online.net
time: 10
sla:
- "[type == 'agent'] >> (stats.bandwidth.mean > 1000)"

16
shaker/scenarios/spot/tcp.yaml Normal file

@ -0,0 +1,16 @@
title: TCP bandwidth
description:
This scenario uses iperf3 to measure TCP bandwidth between local host and
ping.online.net (or against hosts provided via CLI). SLA check is verified
and expects the speed to be at least 90Mbit.
execution:
tests:
-
title: TCP
class: iperf3
host: ping.online.net
time: 10
sla:
- "[type == 'agent'] >> (stats.bandwidth.mean > 90)"

29
shaker/tests/test_deploy.py

@ -438,3 +438,32 @@ class TestDeploy(testtools.TestCase):
]
observed = deploy.prepare_for_cross_az(source, ['nova', 'vcenter'])
self.assertEqual(expected, observed)
# Deployment class unit tests
def test_deploy_local(self):
deployment = deploy.Deployment()
expected = {
'local': {'id': 'local', 'mode': 'alone'}
}
agents = deployment.deploy({})
self.assertEqual(expected, agents)
def test_deploy_static(self):
deployment = deploy.Deployment()
expected = {
'agent': {'id': 'agent', 'mode': 'alone'}
}
agents = deployment.deploy({'agents':
[{'id': 'agent', 'mode': 'alone'}]})
self.assertEqual(expected, agents)
def test_deploy_template_error_when_non_initialized(self):
deployment = deploy.Deployment()
self.assertRaises(deploy.DeploymentException,
deployment.deploy, {'template': 'foo'})