With this patch, a new pytest optional arg has been added:
--skipregex
When it is used, tests are skipped when the regex is found in their test
names. Fully qualified test names are used for this search. Example:
tobiko.tests.unit.test_exception.TestException.test_init
It can be used from CLI with pytest like this:
$ .tox/py3/bin/pytest \
--skipregex='test_join_chunks_.*_bytes|test_join_chunks_with_unicodes' \
tobiko/tests/unit
It can be used from CLI with tox like this:
$ PYTEST_ADDOPTS="--skipregex='test_join_chunks_.*_bytes|test_join_chunks_with_unicodes'" \
TOX_PYTHON=python3.9 \
tox -e py3
It can be used from the Tobiko Infrared plugin by using the following
option (limitation: the skipregex is applied to all the steps from the
the executed Tobiko workflow):
$ infrared tobiko ... --pytest-addopts "--skipregex='test_join_chunks_.*_bytes|test_join_chunks_with_unicodes'"
It can be used from a zuul job by configuring the following variable
within the zuul job yaml file (limitation: the skipregex is applied to
all the steps from the executed Tobiko workflow):
pytest_addopts_global: "--skipregex='test_join_chunks_.*_bytes|test_join_chunks_with_unicodes'"
Change-Id: I8ee32ba467bd70142816953598d1736fa353d3d0
The tox env values defined at step N are reused at step N+1
These values should be reset at the end of every step
Change-Id: Ied76f789a071e50917ce6752ecea17140775a99f
Tobiko test run output was printed twice:
- during the execution of the tests
- during the debug task "show test cases output" right after the
previous one
This patch removes the second task
Change-Id: I5336660baece1fe4df64d0aa09bd575f6e9b98fc
This allows to specify PyTest options depending on the workflow step
For instance in the faults workflow step below option is used:
--maxfail 1
The option can be forced for all speps with below InfraRed plugin option:
--pytest-maxfail <N>
For more details about pytest options please have a look to pytest help page:
tox -e py3 -- --help
Change-Id: I3146819b5f80fa4768ea38032cc8e69dc7b44f18
- Remove support for constrains file from infrared plugin
- Use file stored on tobiko repository by default
Change-Id: I0f835d753fb9d3a1670e8eb9b196e02e989d4262
After some given seconds of timeout test runner execution self-interrupt
to avoid hitting jobs timeout.
The problem is that when it occurs all remaining test cases that didn't started
yet are missing from the reports.
This changes things by marking such test cases as skipped, so that we
can see it from test results.
Change-Id: Idd0064812fc5a2da295712914c4531bdef466c24
Because upstream OpenStack requirements master branch
(Victoria) removed any support for Python 2.7 upstream
jobs started failing. We where only using Python 2
because infrared installation process fails on
RHEL/CentOS 7 and Python 3 due to lacking SE Linux
Python bindings.
This workaround this issue by removing support
for Python 2.7 and CentOS/RHEL 7. The support for
CentOS 7 could be restored once the IR issue has
been fixed.
Change-Id: I09d2577a0e5cb4c7efbb6863294f63b2dd88de0a
Tox venv commands are expected to return a code that indicates
test cases final outcome:
- 0 => any test vase has failed
- 1 => some test case has failed
- not in [0, 1] => any other error
This change handles the special case when some test cases have
failed. In any of below cases:
- JENKINS_URL environment variable is defined on controller node
- --ignore-test-failure flag is passed to IR plugin
then the plugin runs without treating it as an error.
This has been introduced because when running the plugin from
a Jenkins slave host, Jenkins itself is expected to check test
case have failed looking at produced Junit XML artifacts.
Change-Id: Ia7d7f736ae18a7561dced2271228d86ca658e892
- Acreate a new tools/run_tests.py script to
execute test cases
- Move report generation to such script and
remove 'report' tox environment
- Running test cases generates report files
(.log, .html, .xml) into 'report/{envname}'
directory
- Test cases failures will returns exit code 1
Other script test runner failures returns
exit code 2
Big refactory to tox.ini file to try semplifying it:
- create the new 'py3' tox environment for running unit
tests as default platform python 3 interpreter
- use same as py3 '{envdir}' for scenario, functional,
neutron and faults tox environments
Change-Id: Id09425245cc86b84b41e6b3b1c1db759cc686f3a
Create a set of roles to be composed into a new infrared plugin. The
final goal for these roles is to be used for either downstream and
upstream CI
Change-Id: I48d5f500c2e694c3f94ee497a0df1d92b6c411b3
