Release 0.7.0
This release includes: * Documentation improvements * ostestr: unrecognized arguemnts are passedthrough to testr * ostestr: bugfix to always treat xfail as success * add a --version flag to all commands * subunit-trace: split out functionality from main() to be directly callable * regex builder logic from ostestr is broken out into a new command -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAABCgAGBQJXR1EtAAoJEP0SoPIUyeF3lQkP/13BYqHLeEulyoMCT9s3k8RF U3zUDSHp3WfH6+iyqaDAB8fREaIu7BzS0SN7DRkAb4vloQdI+ZlhMehUM5myVvuz 1ui1rXS3Wf+115gF8jPO/IO9RGvhpbf8ooibH65A8Hh/F2OrypSZMHEtpwF/UK3J 87DQWNP7RIje3Vfo/GeWp6PeZ7i07nvOQsGUInimvaAwACkVHvvfO5a1CK4wk3Ag nqVtt2NyrChW4dZfPzSfNqG6Ioo7S90KQni+62qBef4VdS4H8TPN1rj3itQwLYCL 6tFpPxFjbauYxLPDk5zI3Gq0gOX8iOBWPFemk0IONl8Rckmv3RXdcjN88k3mbuhg O+jFxc8h2zMTQ7JjdYcN5tc5tho2FKlQZR8T1RHln9fCWO3UWAdvAjk5EG4gj9Ed 27wvbL7ZppfpNJF8Xee3PJzy1Vm24NJqx4ASc4Lixg0NYViyw4rQ5ip4ecEHYSS0 KruzeV+3mgi/tgLpkyIf2CoQrJFTdGASV7jR1+whMjvPPg/ocuEuiiboU5DsmEXi k4vq2u7Ju2CBKMqLwrqGw1LagnIAI0ixzSOCW3XUOPUXiwUyK3WQ4vyUWhUoiCGr TZJJSNcNflMJj+fX5c3WjQHibsZwY+FPvbwM45VbPJyzid6STj7e1BjiIcdyIE4+ Ar39hjarGwjFdPWsop9z =qWLS -----END PGP SIGNATURE----- Merge tag '0.7.0' into debian/newton Release 0.7.0 This release includes: * Documentation improvements * ostestr: unrecognized arguemnts are passedthrough to testr * ostestr: bugfix to always treat xfail as success * add a --version flag to all commands * subunit-trace: split out functionality from main() to be directly callable * regex builder logic from ostestr is broken out into a new command
This commit is contained in:
Corey Bryant
commit
0fc12e210e
@ -2,6 +2,14 @@
|
||||
os-testr
|
||||
========
|
||||
|
||||
.. image:: https://img.shields.io/pypi/v/os-testr.svg
|
||||
:target: https://pypi.python.org/pypi/os-testr/
|
||||
:alt: Latest Version
|
||||
|
||||
.. image:: https://img.shields.io/pypi/dm/os-testr.svg
|
||||
:target: https://pypi.python.org/pypi/os-testr/
|
||||
:alt: Downloads
|
||||
|
||||
A testr wrapper to provide functionality for OpenStack projects.
|
||||
|
||||
* Free software: Apache license
|
||||
|
||||
@ -14,6 +14,7 @@ default behavior might change in future version.
|
||||
Summary
|
||||
-------
|
||||
ostestr [-b|--blacklist_file <blacklist_file>] [-r|--regex REGEX]
|
||||
[-w|--whitelist_file <whitelist_file>]
|
||||
[-p|--pretty] [--no-pretty] [-s|--subunit] [-l|--list]
|
||||
[-n|--no-discover <test_id>] [--slowest] [--no-slowest]
|
||||
[--pdb <test_id>] [--parallel] [--serial]
|
||||
@ -25,6 +26,9 @@ Options
|
||||
--blacklist_file BLACKLIST_FILE, -b BLACKLIST_FILE
|
||||
Path to a blacklist file, this file contains a
|
||||
separate regex exclude on each newline
|
||||
--whitelist_file WHITELIST_FILE, -w WHITELIST_FILE
|
||||
Path to a whitelist file, this file contains a
|
||||
separate regex on each newline
|
||||
--regex REGEX, -r REGEX
|
||||
A normal testr selection regex. If a blacklist file is
|
||||
specified, the regex will be appended to the end of
|
||||
@ -36,12 +40,12 @@ Options
|
||||
Disable the pretty output with subunit-trace
|
||||
--subunit, -s
|
||||
output the raw subunit v2 from the test run this is
|
||||
mutuall exclusive with --pretty
|
||||
mutually exclusive with --pretty
|
||||
--list, -l
|
||||
List all the tests which will be run.
|
||||
--no-discover TEST_ID, -n TEST_ID
|
||||
Takes in a single test to bypasses test discover and
|
||||
just excute the test specified
|
||||
just execute the test specified
|
||||
--slowest
|
||||
After the test run print the slowest tests
|
||||
--no-slowest
|
||||
@ -114,7 +118,7 @@ exposed via the --regex option. For example::
|
||||
This will do a straight passthrough of the provided regex to testr.
|
||||
Additionally, ostestr allows you to specify a blacklist file to define a set
|
||||
of regexes to exclude. You can specify a blacklist file with the
|
||||
--blacklist-file/-b option, for example::
|
||||
--blacklist_file/-b option, for example::
|
||||
|
||||
$ ostestr --blacklist_file $path_to_file
|
||||
|
||||
@ -134,6 +138,22 @@ regex test selection options can not be used in conjunction with the
|
||||
because the regex selection requires using testr under the covers to actually
|
||||
do the filtering, and those 2 options do not use testr.
|
||||
|
||||
The dual of the blacklist file is the whitelist file which works in the exact
|
||||
same manner, except that instead of excluding regex matches it includes them.
|
||||
You can specify the path to the file with --whitelist_file/-w, for example::
|
||||
|
||||
$ ostestr --whitelist_file $path_to_file
|
||||
|
||||
The format for the file is more or less identical to the blacklist file::
|
||||
|
||||
# Whitelist File
|
||||
^regex1 # Include these tests
|
||||
.*regex2 # include those tests
|
||||
|
||||
However, instead of excluding the matches it will include them. Note that a
|
||||
blacklist file can not be used at the same time as a whitelist file, they
|
||||
are mutually exclusive.
|
||||
|
||||
It's also worth noting that you can use the test list option to dry run any
|
||||
selection arguments you are using. You just need to use --list/-l with your
|
||||
selection options to do this, for example::
|
||||
|
||||
@ -11,7 +11,7 @@ Summary
|
||||
-------
|
||||
|
||||
subunit-trace [--fails|-f] [--failonly] [--perc-diff|-d] [--no-summary]
|
||||
[--diff-threshold|-t <threshold>]
|
||||
[--diff-threshold|-t <threshold>] [--color]
|
||||
|
||||
Options
|
||||
-------
|
||||
@ -31,6 +31,8 @@ Options
|
||||
change will always be displayed.
|
||||
--no-summary
|
||||
Don't print the summary of the test run after completes
|
||||
--color
|
||||
Print result with colors
|
||||
|
||||
Usage
|
||||
-----
|
||||
|
||||
@ -1,6 +0,0 @@
|
||||
[DEFAULT]
|
||||
|
||||
# The list of modules to copy from oslo-incubator.git
|
||||
|
||||
# The base module to hold the copy of openstack.common
|
||||
base=os_testr
|
||||
@ -17,11 +17,19 @@
|
||||
import datetime
|
||||
import sys
|
||||
|
||||
import pbr.version
|
||||
import subunit
|
||||
from subunit import iso8601
|
||||
|
||||
|
||||
__version__ = pbr.version.VersionInfo('os_testr').version_string()
|
||||
|
||||
|
||||
def main():
|
||||
if '--version' in sys.argv:
|
||||
print(__version__)
|
||||
exit(0)
|
||||
|
||||
start_time = datetime.datetime.fromtimestamp(float(sys.argv[1])).replace(
|
||||
tzinfo=iso8601.UTC)
|
||||
elapsed_time = datetime.timedelta(seconds=int(sys.argv[2]))
|
||||
|
||||
@ -19,13 +19,21 @@ import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pbr.version
|
||||
from subunit import run as subunit_run
|
||||
from testtools import run as testtools_run
|
||||
|
||||
from os_testr import regex_builder as rb
|
||||
|
||||
|
||||
__version__ = pbr.version.VersionInfo('os_testr').version_string()
|
||||
|
||||
|
||||
def get_parser(args):
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Tool to run openstack tests')
|
||||
parser.add_argument('--version', action='version',
|
||||
version='%s' % __version__)
|
||||
list_files = parser.add_mutually_exclusive_group()
|
||||
list_files.add_argument('--blacklist_file', '-b',
|
||||
help='Path to a blacklist file, this file '
|
||||
@ -44,7 +52,7 @@ def get_parser(args):
|
||||
help='A file name or directory of tests to run.')
|
||||
group.add_argument('--no-discover', '-n', metavar='TEST_ID',
|
||||
help="Takes in a single test to bypasses test "
|
||||
"discover and just excute the test specified. "
|
||||
"discover and just execute the test specified. "
|
||||
"A file name may be used in place of a test "
|
||||
"name.")
|
||||
pretty = parser.add_mutually_exclusive_group()
|
||||
@ -86,96 +94,12 @@ def get_parser(args):
|
||||
'prints the comment from the same line and all '
|
||||
'skipped tests before the test run')
|
||||
parser.set_defaults(pretty=True, slowest=True, parallel=True)
|
||||
return parser.parse_args(args)
|
||||
|
||||
|
||||
def _get_test_list(regex, env=None):
|
||||
env = env or copy.deepcopy(os.environ)
|
||||
proc = subprocess.Popen(['testr', 'list-tests', regex], env=env,
|
||||
stdout=subprocess.PIPE)
|
||||
out = proc.communicate()[0]
|
||||
raw_test_list = out.split('\n')
|
||||
bad = False
|
||||
test_list = []
|
||||
exclude_list = ['OS_', 'CAPTURE', 'TEST_TIMEOUT', 'PYTHON',
|
||||
'subunit.run discover']
|
||||
for line in raw_test_list:
|
||||
for exclude in exclude_list:
|
||||
if exclude in line:
|
||||
bad = True
|
||||
break
|
||||
elif not line:
|
||||
bad = True
|
||||
break
|
||||
if not bad:
|
||||
test_list.append(line)
|
||||
bad = False
|
||||
return test_list
|
||||
|
||||
|
||||
def print_skips(regex, message):
|
||||
test_list = _get_test_list(regex)
|
||||
if test_list:
|
||||
if message:
|
||||
print(message)
|
||||
else:
|
||||
print('Skipped because of regex %s:' % regex)
|
||||
for test in test_list:
|
||||
print(test)
|
||||
# Extra whitespace to separate
|
||||
print('\n')
|
||||
|
||||
|
||||
def path_to_regex(path):
|
||||
root, _ = os.path.splitext(path)
|
||||
return root.replace('/', '.')
|
||||
|
||||
|
||||
def get_regex_from_whitelist_file(file_path):
|
||||
lines = []
|
||||
for line in open(file_path).read().splitlines():
|
||||
split_line = line.strip().split('#')
|
||||
# Before the # is the regex
|
||||
line_regex = split_line[0].strip()
|
||||
if line_regex:
|
||||
lines.append(line_regex)
|
||||
return '|'.join(lines)
|
||||
|
||||
|
||||
def construct_regex(blacklist_file, whitelist_file, regex, print_exclude):
|
||||
if not blacklist_file:
|
||||
exclude_regex = ''
|
||||
else:
|
||||
black_file = open(blacklist_file, 'r')
|
||||
exclude_regex = ''
|
||||
for line in black_file:
|
||||
raw_line = line.strip()
|
||||
split_line = raw_line.split('#')
|
||||
# Before the # is the regex
|
||||
line_regex = split_line[0].strip()
|
||||
if len(split_line) > 1:
|
||||
# After the # is a comment
|
||||
comment = split_line[1].strip()
|
||||
else:
|
||||
comment = ''
|
||||
if line_regex:
|
||||
if print_exclude:
|
||||
print_skips(line_regex, comment)
|
||||
if exclude_regex:
|
||||
exclude_regex = '|'.join([line_regex, exclude_regex])
|
||||
else:
|
||||
exclude_regex = line_regex
|
||||
if exclude_regex:
|
||||
exclude_regex = "^((?!" + exclude_regex + ").)*$"
|
||||
if regex:
|
||||
exclude_regex += regex
|
||||
if whitelist_file:
|
||||
exclude_regex += '%s' % get_regex_from_whitelist_file(whitelist_file)
|
||||
return exclude_regex
|
||||
return parser.parse_known_args(args)
|
||||
|
||||
|
||||
def call_testr(regex, subunit, pretty, list_tests, slowest, parallel, concur,
|
||||
until_failure, color):
|
||||
until_failure, color, others=None):
|
||||
others = others or []
|
||||
if parallel:
|
||||
cmd = ['testr', 'run', '--parallel']
|
||||
if concur:
|
||||
@ -199,7 +123,7 @@ def call_testr(regex, subunit, pretty, list_tests, slowest, parallel, concur,
|
||||
# This workaround is necessary because of lp bug 1411804 it's super hacky
|
||||
# and makes tons of unfounded assumptions, but it works for the most part
|
||||
if (subunit or pretty) and until_failure:
|
||||
test_list = _get_test_list(regex, env)
|
||||
test_list = rb._get_test_list(regex, env)
|
||||
count = 0
|
||||
failed = False
|
||||
if not test_list:
|
||||
@ -237,11 +161,13 @@ def call_testr(regex, subunit, pretty, list_tests, slowest, parallel, concur,
|
||||
exit(0)
|
||||
# If not until-failure special case call testr like normal
|
||||
elif pretty and not list_tests:
|
||||
cmd.extend(others)
|
||||
ps = subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE)
|
||||
proc = subprocess.Popen(subunit_trace_cmd,
|
||||
env=env, stdin=ps.stdout)
|
||||
ps.stdout.close()
|
||||
else:
|
||||
cmd.extend(others)
|
||||
proc = subprocess.Popen(cmd, env=env)
|
||||
proc.communicate()
|
||||
return_code = proc.returncode
|
||||
@ -268,7 +194,7 @@ def call_subunit_run(test_id, pretty, subunit):
|
||||
testtools_run.main([sys.argv[0], test_id], sys.stdout)
|
||||
|
||||
|
||||
def _select_and_call_runner(opts, exclude_regex):
|
||||
def _select_and_call_runner(opts, exclude_regex, others):
|
||||
ec = 1
|
||||
if not os.path.isdir('.testrepository'):
|
||||
subprocess.call(['testr', 'init'])
|
||||
@ -276,17 +202,20 @@ def _select_and_call_runner(opts, exclude_regex):
|
||||
if not opts.no_discover and not opts.pdb:
|
||||
ec = call_testr(exclude_regex, opts.subunit, opts.pretty, opts.list,
|
||||
opts.slowest, opts.parallel, opts.concurrency,
|
||||
opts.until_failure, opts.color)
|
||||
opts.until_failure, opts.color, others)
|
||||
else:
|
||||
if others:
|
||||
print('Unexpected arguments: ' + ' '.join(others))
|
||||
return 2
|
||||
test_to_run = opts.no_discover or opts.pdb
|
||||
if test_to_run.find('/') != -1:
|
||||
test_to_run = path_to_regex(test_to_run)
|
||||
test_to_run = rb.path_to_regex(test_to_run)
|
||||
ec = call_subunit_run(test_to_run, opts.pretty, opts.subunit)
|
||||
return ec
|
||||
|
||||
|
||||
def main():
|
||||
opts = get_parser(sys.argv[1:])
|
||||
opts, others = get_parser(sys.argv[1:])
|
||||
if opts.pretty and opts.subunit:
|
||||
msg = ('Subunit output and pretty output cannot be specified at the '
|
||||
'same time')
|
||||
@ -306,14 +235,14 @@ def main():
|
||||
print(msg)
|
||||
exit(5)
|
||||
if opts.path:
|
||||
regex = path_to_regex(opts.path)
|
||||
regex = rb.path_to_regex(opts.path)
|
||||
else:
|
||||
regex = opts.regex
|
||||
exclude_regex = construct_regex(opts.blacklist_file,
|
||||
opts.whitelist_file,
|
||||
regex,
|
||||
opts.print_exclude)
|
||||
exit(_select_and_call_runner(opts, exclude_regex))
|
||||
exclude_regex = rb.construct_regex(opts.blacklist_file,
|
||||
opts.whitelist_file,
|
||||
regex,
|
||||
opts.print_exclude)
|
||||
exit(_select_and_call_runner(opts, exclude_regex, others))
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
102
os_testr/regex_builder.py
Normal file
102
os_testr/regex_builder.py
Normal file
@ -0,0 +1,102 @@
|
||||
# Copyright 2016 Hewlett-Packard Development Company, L.P.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import copy
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
|
||||
def _get_test_list(regex, env=None):
|
||||
env = env or copy.deepcopy(os.environ)
|
||||
proc = subprocess.Popen(['testr', 'list-tests', regex], env=env,
|
||||
stdout=subprocess.PIPE)
|
||||
out = proc.communicate()[0]
|
||||
raw_test_list = out.split('\n')
|
||||
bad = False
|
||||
test_list = []
|
||||
exclude_list = ['OS_', 'CAPTURE', 'TEST_TIMEOUT', 'PYTHON',
|
||||
'subunit.run discover']
|
||||
for line in raw_test_list:
|
||||
for exclude in exclude_list:
|
||||
if exclude in line:
|
||||
bad = True
|
||||
break
|
||||
elif not line:
|
||||
bad = True
|
||||
break
|
||||
if not bad:
|
||||
test_list.append(line)
|
||||
bad = False
|
||||
return test_list
|
||||
|
||||
|
||||
def print_skips(regex, message):
|
||||
test_list = _get_test_list(regex)
|
||||
if test_list:
|
||||
if message:
|
||||
print(message)
|
||||
else:
|
||||
print('Skipped because of regex %s:' % regex)
|
||||
for test in test_list:
|
||||
print(test)
|
||||
# Extra whitespace to separate
|
||||
print('\n')
|
||||
|
||||
|
||||
def path_to_regex(path):
|
||||
root, _ = os.path.splitext(path)
|
||||
return root.replace('/', '.')
|
||||
|
||||
|
||||
def get_regex_from_whitelist_file(file_path):
|
||||
lines = []
|
||||
for line in open(file_path).read().splitlines():
|
||||
split_line = line.strip().split('#')
|
||||
# Before the # is the regex
|
||||
line_regex = split_line[0].strip()
|
||||
if line_regex:
|
||||
lines.append(line_regex)
|
||||
return '|'.join(lines)
|
||||
|
||||
|
||||
def construct_regex(blacklist_file, whitelist_file, regex, print_exclude):
|
||||
if not blacklist_file:
|
||||
exclude_regex = ''
|
||||
else:
|
||||
black_file = open(blacklist_file, 'r')
|
||||
exclude_regex = ''
|
||||
for line in black_file:
|
||||
raw_line = line.strip()
|
||||
split_line = raw_line.split('#')
|
||||
# Before the # is the regex
|
||||
line_regex = split_line[0].strip()
|
||||
if len(split_line) > 1:
|
||||
# After the # is a comment
|
||||
comment = split_line[1].strip()
|
||||
else:
|
||||
comment = ''
|
||||
if line_regex:
|
||||
if print_exclude:
|
||||
print_skips(line_regex, comment)
|
||||
if exclude_regex:
|
||||
exclude_regex = '|'.join([line_regex, exclude_regex])
|
||||
else:
|
||||
exclude_regex = line_regex
|
||||
if exclude_regex:
|
||||
exclude_regex = "^((?!" + exclude_regex + ").)*$"
|
||||
if regex:
|
||||
exclude_regex += regex
|
||||
if whitelist_file:
|
||||
exclude_regex += '%s' % get_regex_from_whitelist_file(whitelist_file)
|
||||
return exclude_regex
|
||||
@ -60,10 +60,12 @@ import sys
|
||||
import traceback
|
||||
from xml.sax import saxutils
|
||||
|
||||
import pbr.version
|
||||
import subunit
|
||||
import testtools
|
||||
|
||||
__version__ = '0.1'
|
||||
|
||||
__version__ = pbr.version.VersionInfo('os_testr').version_string()
|
||||
|
||||
|
||||
class TemplateData(object):
|
||||
@ -701,6 +703,10 @@ class FileAccumulator(testtools.StreamResult):
|
||||
|
||||
|
||||
def main():
|
||||
if '--version' in sys.argv:
|
||||
print(__version__)
|
||||
exit(0)
|
||||
|
||||
if len(sys.argv) < 2:
|
||||
print("Need at least one argument: path to subunit log.")
|
||||
exit(1)
|
||||
|
||||
@ -26,6 +26,7 @@ import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
import pbr.version
|
||||
import subunit
|
||||
import testtools
|
||||
|
||||
@ -191,7 +192,7 @@ def show_outcome(stream, test, print_failures=False, failonly=False,
|
||||
if not print_failures:
|
||||
print_attachments(stream, test, all_channels=True)
|
||||
elif not failonly:
|
||||
if status == 'success':
|
||||
if status == 'success' or status == 'xfail':
|
||||
if abbreviate:
|
||||
color.write('.', 'green')
|
||||
else:
|
||||
@ -313,8 +314,13 @@ def print_summary(stream, elapsed_time):
|
||||
stream.write(out_str)
|
||||
|
||||
|
||||
__version__ = pbr.version.VersionInfo('os_testr').version_string()
|
||||
|
||||
|
||||
def parse_args():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument('--version', action='version',
|
||||
version='%s' % __version__)
|
||||
parser.add_argument('--no-failure-debug', '-n', action='store_true',
|
||||
dest='print_failures', help='Disable printing failure '
|
||||
'debug information in realtime')
|
||||
@ -344,21 +350,22 @@ def parse_args():
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main():
|
||||
args = parse_args()
|
||||
def trace(stdin, stdout, print_failures=False, failonly=False,
|
||||
enable_diff=False, abbreviate=False, color=False, post_fails=False,
|
||||
no_summary=False):
|
||||
stream = subunit.ByteStreamToStreamResult(
|
||||
sys.stdin, non_subunit_name='stdout')
|
||||
stdin, non_subunit_name='stdout')
|
||||
outcomes = testtools.StreamToDict(
|
||||
functools.partial(show_outcome, sys.stdout,
|
||||
print_failures=args.print_failures,
|
||||
failonly=args.failonly,
|
||||
enable_diff=args.enable_diff,
|
||||
abbreviate=args.abbreviate,
|
||||
enable_color=args.color))
|
||||
functools.partial(show_outcome, stdout,
|
||||
print_failures=print_failures,
|
||||
failonly=failonly,
|
||||
enable_diff=enable_diff,
|
||||
abbreviate=abbreviate,
|
||||
enable_color=color))
|
||||
summary = testtools.StreamSummary()
|
||||
result = testtools.CopyStreamResult([outcomes, summary])
|
||||
result = testtools.StreamResultRouter(result)
|
||||
cat = subunit.test_results.CatFiles(sys.stdout)
|
||||
cat = subunit.test_results.CatFiles(stdout)
|
||||
result.add_rule(cat, 'test_id', test_id=None)
|
||||
start_time = datetime.datetime.utcnow()
|
||||
result.startTestRun()
|
||||
@ -371,18 +378,25 @@ def main():
|
||||
|
||||
if count_tests('status', '.*') == 0:
|
||||
print("The test run didn't actually run any tests")
|
||||
exit(1)
|
||||
if args.post_fails:
|
||||
print_fails(sys.stdout)
|
||||
if not args.no_summary:
|
||||
print_summary(sys.stdout, elapsed_time)
|
||||
return 1
|
||||
if post_fails:
|
||||
print_fails(stdout)
|
||||
if not no_summary:
|
||||
print_summary(stdout, elapsed_time)
|
||||
|
||||
# NOTE(mtreinish): Ideally this should live in testtools streamSummary
|
||||
# this is just in place until the behavior lands there (if it ever does)
|
||||
if count_tests('status', '^success$') == 0:
|
||||
print("\nNo tests were successful during the run")
|
||||
exit(1)
|
||||
exit(0 if summary.wasSuccessful() else 1)
|
||||
return 1
|
||||
return 0 if summary.wasSuccessful() else 1
|
||||
|
||||
|
||||
def main():
|
||||
args = parse_args()
|
||||
exit(trace(sys.stdin, sys.stdout, args.print_failures, args.failonly,
|
||||
args.enable_diff, args.abbreviate, args.color, args.post_fails,
|
||||
args.no_summary))
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
||||
117
os_testr/tests/test_ostestr.py
Normal file
117
os_testr/tests/test_ostestr.py
Normal file
@ -0,0 +1,117 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
"""
|
||||
test_os_testr
|
||||
----------------------------------
|
||||
|
||||
Tests for `os_testr` module.
|
||||
"""
|
||||
|
||||
import mock
|
||||
|
||||
from os_testr import ostestr as os_testr
|
||||
from os_testr.tests import base
|
||||
|
||||
|
||||
class TestGetParser(base.TestCase):
|
||||
def test_pretty(self):
|
||||
namespace = os_testr.get_parser(['--pretty'])
|
||||
self.assertEqual(True, namespace[0].pretty)
|
||||
namespace = os_testr.get_parser(['--no-pretty'])
|
||||
self.assertEqual(False, namespace[0].pretty)
|
||||
self.assertRaises(SystemExit, os_testr.get_parser,
|
||||
['--no-pretty', '--pretty'])
|
||||
|
||||
def test_slowest(self):
|
||||
namespace = os_testr.get_parser(['--slowest'])
|
||||
self.assertEqual(True, namespace[0].slowest)
|
||||
namespace = os_testr.get_parser(['--no-slowest'])
|
||||
self.assertEqual(False, namespace[0].slowest)
|
||||
self.assertRaises(SystemExit, os_testr.get_parser,
|
||||
['--no-slowest', '--slowest'])
|
||||
|
||||
def test_parallel(self):
|
||||
namespace = os_testr.get_parser(['--parallel'])
|
||||
self.assertEqual(True, namespace[0].parallel)
|
||||
namespace = os_testr.get_parser(['--serial'])
|
||||
self.assertEqual(False, namespace[0].parallel)
|
||||
self.assertRaises(SystemExit, os_testr.get_parser,
|
||||
['--parallel', '--serial'])
|
||||
|
||||
|
||||
class TestCallers(base.TestCase):
|
||||
def test_no_discover(self):
|
||||
namespace = os_testr.get_parser(['-n', 'project.tests.foo'])
|
||||
|
||||
def _fake_exit(arg):
|
||||
self.assertTrue(arg)
|
||||
|
||||
def _fake_run(*args, **kwargs):
|
||||
return 'project.tests.foo' in args
|
||||
|
||||
with mock.patch.object(os_testr, 'exit', side_effect=_fake_exit), \
|
||||
mock.patch.object(os_testr, 'get_parser', return_value=namespace), \
|
||||
mock.patch.object(os_testr,
|
||||
'call_subunit_run',
|
||||
side_effect=_fake_run):
|
||||
os_testr.main()
|
||||
|
||||
def test_no_discover_path(self):
|
||||
namespace = os_testr.get_parser(['-n', 'project/tests/foo'])
|
||||
|
||||
def _fake_exit(arg):
|
||||
self.assertTrue(arg)
|
||||
|
||||
def _fake_run(*args, **kwargs):
|
||||
return 'project.tests.foo' in args
|
||||
|
||||
with mock.patch.object(os_testr, 'exit', side_effect=_fake_exit), \
|
||||
mock.patch.object(os_testr, 'get_parser', return_value=namespace), \
|
||||
mock.patch.object(os_testr,
|
||||
'call_subunit_run',
|
||||
side_effect=_fake_run):
|
||||
os_testr.main()
|
||||
|
||||
def test_pdb(self):
|
||||
namespace = os_testr.get_parser(['--pdb', 'project.tests.foo'])
|
||||
|
||||
def _fake_exit(arg):
|
||||
self.assertTrue(arg)
|
||||
|
||||
def _fake_run(*args, **kwargs):
|
||||
return 'project.tests.foo' in args
|
||||
|
||||
with mock.patch.object(os_testr, 'exit', side_effect=_fake_exit), \
|
||||
mock.patch.object(os_testr, 'get_parser', return_value=namespace), \
|
||||
mock.patch.object(os_testr,
|
||||
'call_subunit_run',
|
||||
side_effect=_fake_run):
|
||||
os_testr.main()
|
||||
|
||||
def test_pdb_path(self):
|
||||
namespace = os_testr.get_parser(['--pdb', 'project/tests/foo'])
|
||||
|
||||
def _fake_exit(arg):
|
||||
self.assertTrue(arg)
|
||||
|
||||
def _fake_run(*args, **kwargs):
|
||||
return 'project.tests.foo' in args
|
||||
|
||||
with mock.patch.object(os_testr, 'exit', side_effect=_fake_exit), \
|
||||
mock.patch.object(os_testr, 'get_parser', return_value=namespace), \
|
||||
mock.patch.object(os_testr,
|
||||
'call_subunit_run',
|
||||
side_effect=_fake_run):
|
||||
os_testr.main()
|
||||
@ -12,17 +12,11 @@
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
"""
|
||||
test_os_testr
|
||||
----------------------------------
|
||||
|
||||
Tests for `os_testr` module.
|
||||
"""
|
||||
|
||||
import mock
|
||||
|
||||
import six
|
||||
|
||||
from os_testr import os_testr
|
||||
from os_testr import regex_builder as os_testr
|
||||
from os_testr.tests import base
|
||||
|
||||
|
||||
@ -35,98 +29,6 @@ class TestPathToRegex(base.TestCase):
|
||||
self.assertEqual("openstack.tests.network.v2", result)
|
||||
|
||||
|
||||
class TestGetParser(base.TestCase):
|
||||
def test_pretty(self):
|
||||
namespace = os_testr.get_parser(['--pretty'])
|
||||
self.assertEqual(True, namespace.pretty)
|
||||
namespace = os_testr.get_parser(['--no-pretty'])
|
||||
self.assertEqual(False, namespace.pretty)
|
||||
self.assertRaises(SystemExit, os_testr.get_parser,
|
||||
['--no-pretty', '--pretty'])
|
||||
|
||||
def test_slowest(self):
|
||||
namespace = os_testr.get_parser(['--slowest'])
|
||||
self.assertEqual(True, namespace.slowest)
|
||||
namespace = os_testr.get_parser(['--no-slowest'])
|
||||
self.assertEqual(False, namespace.slowest)
|
||||
self.assertRaises(SystemExit, os_testr.get_parser,
|
||||
['--no-slowest', '--slowest'])
|
||||
|
||||
def test_parallel(self):
|
||||
namespace = os_testr.get_parser(['--parallel'])
|
||||
self.assertEqual(True, namespace.parallel)
|
||||
namespace = os_testr.get_parser(['--serial'])
|
||||
self.assertEqual(False, namespace.parallel)
|
||||
self.assertRaises(SystemExit, os_testr.get_parser,
|
||||
['--parallel', '--serial'])
|
||||
|
||||
|
||||
class TestCallers(base.TestCase):
|
||||
def test_no_discover(self):
|
||||
namespace = os_testr.get_parser(['-n', 'project.tests.foo'])
|
||||
|
||||
def _fake_exit(arg):
|
||||
self.assertTrue(arg)
|
||||
|
||||
def _fake_run(*args, **kwargs):
|
||||
return 'project.tests.foo' in args
|
||||
|
||||
with mock.patch.object(os_testr, 'exit', side_effect=_fake_exit), \
|
||||
mock.patch.object(os_testr, 'get_parser', return_value=namespace), \
|
||||
mock.patch.object(os_testr,
|
||||
'call_subunit_run',
|
||||
side_effect=_fake_run):
|
||||
os_testr.main()
|
||||
|
||||
def test_no_discover_path(self):
|
||||
namespace = os_testr.get_parser(['-n', 'project/tests/foo'])
|
||||
|
||||
def _fake_exit(arg):
|
||||
self.assertTrue(arg)
|
||||
|
||||
def _fake_run(*args, **kwargs):
|
||||
return 'project.tests.foo' in args
|
||||
|
||||
with mock.patch.object(os_testr, 'exit', side_effect=_fake_exit), \
|
||||
mock.patch.object(os_testr, 'get_parser', return_value=namespace), \
|
||||
mock.patch.object(os_testr,
|
||||
'call_subunit_run',
|
||||
side_effect=_fake_run):
|
||||
os_testr.main()
|
||||
|
||||
def test_pdb(self):
|
||||
namespace = os_testr.get_parser(['--pdb', 'project.tests.foo'])
|
||||
|
||||
def _fake_exit(arg):
|
||||
self.assertTrue(arg)
|
||||
|
||||
def _fake_run(*args, **kwargs):
|
||||
return 'project.tests.foo' in args
|
||||
|
||||
with mock.patch.object(os_testr, 'exit', side_effect=_fake_exit), \
|
||||
mock.patch.object(os_testr, 'get_parser', return_value=namespace), \
|
||||
mock.patch.object(os_testr,
|
||||
'call_subunit_run',
|
||||
side_effect=_fake_run):
|
||||
os_testr.main()
|
||||
|
||||
def test_pdb_path(self):
|
||||
namespace = os_testr.get_parser(['--pdb', 'project/tests/foo'])
|
||||
|
||||
def _fake_exit(arg):
|
||||
self.assertTrue(arg)
|
||||
|
||||
def _fake_run(*args, **kwargs):
|
||||
return 'project.tests.foo' in args
|
||||
|
||||
with mock.patch.object(os_testr, 'exit', side_effect=_fake_exit), \
|
||||
mock.patch.object(os_testr, 'get_parser', return_value=namespace), \
|
||||
mock.patch.object(os_testr,
|
||||
'call_subunit_run',
|
||||
side_effect=_fake_run):
|
||||
os_testr.main()
|
||||
|
||||
|
||||
class TestConstructRegex(base.TestCase):
|
||||
def test_regex_passthrough(self):
|
||||
result = os_testr.construct_regex(None, None, 'fake_regex', False)
|
||||
@ -14,13 +14,16 @@
|
||||
# under the License.
|
||||
|
||||
from datetime import datetime as dt
|
||||
import io
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from ddt import data
|
||||
from ddt import ddt
|
||||
from ddt import unpack
|
||||
from mock import patch
|
||||
import six
|
||||
|
||||
from os_testr import subunit_trace
|
||||
from os_testr.tests import base
|
||||
@ -79,3 +82,15 @@ class TestSubunitTrace(base.TestCase):
|
||||
with open(regular_stream, 'rb') as stream:
|
||||
p.communicate(stream.read())
|
||||
self.assertEqual(0, p.returncode)
|
||||
|
||||
def test_trace(self):
|
||||
regular_stream = os.path.join(
|
||||
os.path.dirname(os.path.abspath(__file__)),
|
||||
'sample_streams/successful.subunit')
|
||||
bytes_ = io.BytesIO()
|
||||
with open(regular_stream, 'rb') as stream:
|
||||
bytes_.write(six.binary_type(stream.read()))
|
||||
bytes_.seek(0)
|
||||
stdin = io.TextIOWrapper(io.BufferedReader(bytes_))
|
||||
returncode = subunit_trace.trace(stdin, sys.stdout)
|
||||
self.assertEqual(0, returncode)
|
||||
|
||||
0
os_testr/tests/utils/__init__.py
Normal file
0
os_testr/tests/utils/__init__.py
Normal file
76
os_testr/tests/utils/test_colorizer.py
Normal file
76
os_testr/tests/utils/test_colorizer.py
Normal file
@ -0,0 +1,76 @@
|
||||
# Copyright 2016 Hewlett Packard Enterprise Development LP
|
||||
# All Rights Reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import six
|
||||
import sys
|
||||
|
||||
from ddt import data
|
||||
from ddt import ddt
|
||||
from ddt import unpack
|
||||
|
||||
from os_testr.tests import base
|
||||
from os_testr.utils import colorizer
|
||||
|
||||
|
||||
@ddt
|
||||
class TestNullColorizer(base.TestCase):
|
||||
|
||||
@data(None, "foo", sys.stdout, )
|
||||
def test_supported_always_true(self, stream):
|
||||
self.assertTrue(colorizer.NullColorizer.supported(stream))
|
||||
|
||||
@data(("foo", "red"), ("foo", "bar"))
|
||||
@unpack
|
||||
def test_write_string_ignore_color(self, text, color):
|
||||
output = six.StringIO()
|
||||
c = colorizer.NullColorizer(output)
|
||||
c.write(text, color)
|
||||
self.assertEqual(text, output.getvalue())
|
||||
|
||||
@data((None, "red"), (None, None))
|
||||
@unpack
|
||||
def test_write_none_exception(self, text, color):
|
||||
c = colorizer.NullColorizer(sys.stdout)
|
||||
self.assertRaises(TypeError, c.write, text, color)
|
||||
|
||||
|
||||
@ddt
|
||||
class TestAnsiColorizer(base.TestCase):
|
||||
|
||||
def test_supported_false(self):
|
||||
# NOTE(masayukig): This returns False because our unittest env isn't
|
||||
# interactive
|
||||
self.assertFalse(colorizer.AnsiColorizer.supported(sys.stdout))
|
||||
|
||||
@data(None, "foo")
|
||||
def test_supported_error(self, stream):
|
||||
self.assertRaises(AttributeError,
|
||||
colorizer.AnsiColorizer.supported, stream)
|
||||
|
||||
@data(("foo", "red", "31"), ("foo", "blue", "34"))
|
||||
@unpack
|
||||
def test_write_string_valid_color(self, text, color, color_code):
|
||||
output = six.StringIO()
|
||||
c = colorizer.AnsiColorizer(output)
|
||||
c.write(text, color)
|
||||
self.assertIn(text, output.getvalue())
|
||||
self.assertIn(color_code, output.getvalue())
|
||||
|
||||
@data(("foo", None), ("foo", "invalid_color"))
|
||||
@unpack
|
||||
def test_write_string_invalid_color(self, text, color):
|
||||
output = six.StringIO()
|
||||
c = colorizer.AnsiColorizer(output)
|
||||
self.assertRaises(KeyError, c.write, text, color)
|
||||
@ -2,8 +2,8 @@
|
||||
# of appearance. Changing the order has an impact on the overall integration
|
||||
# process, which may cause wedges in the gate later.
|
||||
|
||||
pbr>=1.3,<2.0
|
||||
Babel>=1.3
|
||||
testrepository>=0.0.18
|
||||
python-subunit>=0.0.18
|
||||
testtools>=1.4.0
|
||||
pbr>=1.6 # Apache-2.0
|
||||
Babel>=2.3.4 # BSD
|
||||
testrepository>=0.0.18 # Apache-2.0/BSD
|
||||
python-subunit>=0.0.18 # Apache-2.0/BSD
|
||||
testtools>=1.4.0 # MIT
|
||||
|
||||
@ -15,9 +15,7 @@ classifier =
|
||||
Programming Language :: Python
|
||||
Programming Language :: Python :: 2
|
||||
Programming Language :: Python :: 2.7
|
||||
Programming Language :: Python :: 2.6
|
||||
Programming Language :: Python :: 3
|
||||
Programming Language :: Python :: 3.3
|
||||
Programming Language :: Python :: 3.4
|
||||
|
||||
[files]
|
||||
@ -27,7 +25,7 @@ packages =
|
||||
[entry_points]
|
||||
console_scripts =
|
||||
subunit-trace = os_testr.subunit_trace:main
|
||||
ostestr = os_testr.os_testr:main
|
||||
ostestr = os_testr.ostestr:main
|
||||
subunit2html = os_testr.subunit2html:main
|
||||
generate-subunit = os_testr.generate_subunit:main
|
||||
|
||||
|
||||
3
setup.py
Executable file → Normal file
3
setup.py
Executable file → Normal file
@ -1,4 +1,3 @@
|
||||
#!/usr/bin/env python
|
||||
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
@ -26,5 +25,5 @@ except ImportError:
|
||||
pass
|
||||
|
||||
setuptools.setup(
|
||||
setup_requires=['pbr>=1.3'],
|
||||
setup_requires=['pbr>=1.8'],
|
||||
pbr=True)
|
||||
|
||||
@ -2,13 +2,13 @@
|
||||
# of appearance. Changing the order has an impact on the overall integration
|
||||
# process, which may cause wedges in the gate later.
|
||||
|
||||
hacking<0.11,>=0.10.0
|
||||
hacking<0.11,>=0.10.2 # Apache-2.0
|
||||
|
||||
coverage>=3.6
|
||||
discover
|
||||
sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3
|
||||
oslosphinx>=2.2.0 # Apache-2.0
|
||||
oslotest>=1.2.0 # Apache-2.0
|
||||
testscenarios>=0.4
|
||||
ddt>=0.4.0
|
||||
six>=1.9.0
|
||||
coverage>=3.6 # Apache-2.0
|
||||
discover # BSD
|
||||
sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
|
||||
oslosphinx!=3.4.0,>=2.5.0 # Apache-2.0
|
||||
oslotest>=1.10.0 # Apache-2.0
|
||||
testscenarios>=0.4 # Apache-2.0/BSD
|
||||
ddt>=1.0.1 # MIT
|
||||
six>=1.9.0 # MIT
|
||||
|
||||
4
tox.ini
4
tox.ini
@ -1,6 +1,6 @@
|
||||
[tox]
|
||||
minversion = 1.6
|
||||
envlist = py33,py34,py26,py27,pypy,pep8
|
||||
envlist = py34,py27,pypy,pep8
|
||||
skipsdist = True
|
||||
|
||||
[testenv]
|
||||
@ -22,7 +22,7 @@ commands = flake8
|
||||
commands = {posargs}
|
||||
|
||||
[testenv:cover]
|
||||
commands = python setup.py testr --coverage --coverage-package-name='os_testr' --testr-args='{posargs}'
|
||||
commands = python setup.py test --coverage --coverage-package-name='os_testr' --testr-args='{posargs}'
|
||||
|
||||
[testenv:docs]
|
||||
commands = python setup.py build_sphinx
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user