2260 Commits

Author SHA1 Message Date
Zuul
0b4bb29b5a Merge "Revert "Add no_log to the log uploading module parameters"" 2021-11-16 19:18:20 +00:00
Artem Goncharov
2f1951c23c Revert "Add no_log to the log uploading module parameters"
This reverts commit 10ee6ebf016fa00c9eb8cd15a09832982b08fe48.

Reason for revert: log url get broken

Change-Id: Ifb8db77c2534a92bbb19fde1bfbf31562ed04daf
2021-11-16 19:06:33 +00:00
Zuul
88e8bf4b2e Merge "Add no_log to the log uploading module parameters" 2021-11-16 16:45:20 +00:00
Zuul
6ff50fb0d4 Merge "Fix zuul-jobs-test-upload-logs-s3 job" 2021-11-16 16:34:48 +00:00
Artem Goncharov
10ee6ebf01 Add no_log to the log uploading module parameters
Use no_log parameter for potentially sensitive log uploading modules

Change-Id: I6264f1b028db0e4f1f74ae557847c146c00295da
2021-11-16 16:29:31 +00:00
Artem Goncharov
e7d7575eaf Fix zuul-jobs-test-upload-logs-s3 job
Since the security update we are not able to modify
zuul.executor.log_root variable in the role testing.
Replace this with explicit log folder creation on the test node.

Change-Id: Ie6cff6f9e9c5594167ddda6cd345f9c3d9f2f470
2021-11-16 16:20:33 +00:00
Gonéri Le Bouder
3cba437a15
enable-fips: Fedora also support FIPS
Fedora supports FIPS. There is no version check because the feature is
available on all the supported Fedora releases.

Change-Id: I924fb565a4d70e7c93a215e9e0a5b2b80bced52a
2021-10-28 15:20:59 -04:00
Gonéri Le Bouder
72581dcaac
use-buildset-registry: remove useless import
The module was not used and Python does not provide any `test` module since
py39.

Change-Id: Ia69a632bd4fff05da644c58d557912bc877a8135
2021-10-28 12:48:06 -04:00
Zuul
0bfa518718 Merge "ensure-dstat-graph: pull updated branch" 2021-10-25 23:31:59 +00:00
Zuul
9ff0c50c6e Merge "ensure-zookeeper: eliminate the tmpfs size limit" 2021-10-24 23:14:42 +00:00
Zuul
89bf334822 Merge "Update dstat to support bionic and others" 2021-10-24 23:01:36 +00:00
James E. Blair
5a83a49dac ensure-zookeeper: eliminate the tmpfs size limit
Zuul's use of this role has exceeded the 500M cap.  Let's go ahead
and remove it since it's arbitrary anyway, and Zuul is likely the
primary user of the role.

Change-Id: Ib0bc72f6c909e44bc04af7a8fbe40ef8415addcf
2021-10-22 15:32:03 -07:00
James E. Blair
aeeb946ebf Update dstat to support bionic and others
Change-Id: Ida88ebbc6c169878adbb8e4f06da05eac813a04b
2021-10-22 12:51:41 -07:00
Zuul
d453753fe6 Merge "ensure-rust: verify cryptography build on Ubuntu" 2021-10-22 16:13:06 +00:00
Zuul
24aea0f5c5 Merge "ensure-docker: remove Debian Stretch testing" 2021-10-22 15:19:12 +00:00
Ian Wienand
94d7d42f14 ensure-dstat-graph: pull updated branch
This updated branch fixes a few issues and should ensure display with
pcp-dstat (I9234b226242f145db9c235797649202aa530ec74)

Add an override so we pull it

Change-Id: I0d5e1b567c364a9e6c7aa0b95de17abffaef0434
2021-10-22 18:31:10 +11:00
James E. Blair
61d00fec6f Use pcp instead of dstand and return dstat graph artifact
Dstat is dead, but pcp supplies a compatible interface.

Installation can be unreliable, so let's ignore failures since
this isn't generally critical.

Change-Id: I9234b226242f145db9c235797649202aa530ec74
2021-10-21 15:39:54 -07:00
Zuul
bd60fbd744 Merge "Add FIPS enable multinode job definition" 2021-10-20 23:54:04 +00:00
Zuul
984147663c Merge "build-python-release: Add flag for whether to build a wheel or not" 2021-10-20 20:32:00 +00:00
Douglas Viroel
9107f3ee7d Add FIPS enable multinode job definition
This patch adds a new multinode job definition that enables
FIPS mode prior to multinode configuration.
In order to enable FIPS mode, the OS boot procedure need to be
changed to enable the appropriate kernel flag. This modification
has effect only after system reboot.
The default behavior of this job is to always enable FIPS mode.

Change-Id: I6f1365837d9ed2ba82c391a20f9094c9ef0e6c4e
Signed-off-by: Douglas Viroel <dviroel@redhat.com>
2021-10-20 11:20:52 -03:00
Zuul
e3519ce469 Merge "multi-node-bridge: repos to install ovs in C9" 2021-10-20 12:45:36 +00:00
Sandeep Yadav
a795073493 multi-node-bridge: repos to install ovs in C9
With this patch, Adding repos to install openvswitch
in Centos9 stream.

For Centos9 stream nodes we need to enable 2 repos:-
* nfv repo - Ussuri+ openvswitch is shipped as part of NFV SIG
* rdo xena repo - rdo-openvswitch(which provides openvswitch wrapper)
is available in OpenStack repo

Closes-Bug: #1947665
Change-Id: If5a32b30637296a070f2e75ba8feb6aa45d8ecac
2021-10-20 11:17:41 +05:30
Ian Wienand
a3f3458566 ensure-docker: remove Debian Stretch testing
Remove Debian Stretch testing in preparation for node removal

Change-Id: I072d931f21a8a4eefacf75eb25dd717cbd7db7fd
2021-10-20 14:54:26 +11:00
Ian Wienand
db23e0af70 ensure-podman: support Debian bullseye
The package is included in the latest stable, Debian bullseye.  Add an
install match, and test it.

Change-Id: I62d2582e8adf2bedb2ea7d41b4b22a48c9b2f274
2021-10-15 11:38:58 +11:00
Ian Wienand
72d4e506d5 intermediate-registry: handle socat warning out
Zuul switched to a new base image, and it seems the new socat puts out
a warning (something like

 ...  socat[489590] W ioctl(5, IOCTL_VM_SOCKETS_GET_LOCAL_CID, ...): Inappropriate ioctl for device

for reference).

Grep the output so we only get the line about what port it is
listening on.

Change-Id: I74fb86a9158b45e6601ee1fbc199ba80cd4991fe
2021-10-14 16:04:46 +11:00
Bhagyashri Shewale
84b86437c5 Handled TypeError while installing any sibling python packages
The below error is coming on mol-centos7 jobs while
installing sibling for python2 packeges on centos7

Error: TypeError: initial_value must be unicode or None, not str

This patch handled the TypeError while installing sibling
for python packeges for centos7.

Closes-Bug: #1946641
Change-Id: Ie8058cca92d099e50af19b95b4c417c5a665da0d
2021-10-13 12:52:03 +05:30
Gonéri Le Bouder
ad56b0b4eb
build-container-image: improve the reliabilty
build-container-image may ends up pushing large files. Network is not
always as reliable as we want. This commit ensures we retry several
times in case of failure.

Change-Id: Ieaf92a8c3531e24ab5e41783b540de1df806c02f
2021-10-08 11:38:11 -04:00
Zuul
d5e4d55c15 Merge "ensure-rust: rework global install" 2021-10-08 01:40:58 +00:00
Zuul
08e37ff140 Merge "Revert "Revert "Include tox_extra_args in tox siblings tasks""" 2021-10-06 17:31:10 +00:00
Zuul
bbf4c1bbaa Merge "ensure-twine: Avoid Reinstalling twine if present" 2021-10-04 20:58:04 +00:00
Zuul
ec6b89b0d4 Merge "ensure-tox: Avoid Reinstalling tox venv if present" 2021-10-04 20:48:06 +00:00
Ian Wienand
5cd0182aac ensure-rust: verify cryptography build on Ubuntu
Just for sanity, ensure the cryptography build (that uses rust) works.

Change-Id: I2fc9dc1fd0f19191af1535dd660d2a5376139764
2021-10-04 13:44:19 +11:00
Ian Wienand
67ae84ce39 ensure-rust: rework global install
I think the secondary "rustup default" was intended to make sure the
Zuul user sets up to use the installed rust toolchain
(I32f9b285904a7036f9a80ada8a49fa9cf31b5163) but actually results in a
re-download of components and another local installation.  This isn't
really the intention, and also doubles the time spent installing.

From the linked comment, it seems like we're not doing our global
install correctly; even putting it in /usr doesn't avoid the need for
RUST_HOME to be set.  Take it's suggestion and install out-of-the-way
in /opt, use a small /usr/local/bin wrapper to call with correct env
vars set and then setup the installed global binary names to be called
via that.

Change-Id: I28ef747b809a17664305bfd9754022251390647b
2021-10-04 11:43:22 +11:00
Jeremy Stanley
c02c28a982 Revert "Revert "Include tox_extra_args in tox siblings tasks""
This reverts commit 3668acc723d5b54cdaf789842a3c6e236bb6adb7.

Change-Id: I643b82c7dc6fd78ff67678b56c639a6c45fd0061
2021-10-01 16:03:13 +00:00
Jeremy Stanley
1861faf567 More exact section matching for tox showconfig
With the original verbose output support in change
Iafeb88eaf9a596603ad4d2134a4574345d5189ab we looked for lines from
tox --showconfig output starting with an opening '[' but verbosity
also causes output from pip install activity to be included if a
tox.requires entry causes it to install packages before continuing
and these lines are prefixed by a process ID number wrapped in
brackets with the command string after that, which prematurely
triggered our search for the start of the INI content. Add a
stipulation that the first INI output line also end in ']' in order
to skip over those additional prepended lines.

Change-Id: If29e5a9abe3b92a145d87f5efc1b93350ea3908a
2021-10-01 16:03:12 +00:00
Jeremy Stanley
31a470245b Test tox.requires doesn't break sibling installs
Update the alternate tox.ini files in test-playbooks/python to
trigger configuration we saw expose a bug in the siblings lib's
showconfig parser when a tox.requires value resulted in pip
activity. Update its minversion to the earliest tox which provides
the tox.requires configuration option, for clarity.

While we're there, switch to using tox_config_file instead of
overriding zuul_work_dir since that's now an option for this role
and was previously not exercised in the test job.

Change-Id: I521580da978903cd116c872447b1c028c470e63d
2021-10-01 12:46:14 +00:00
Jeremy Stanley
3668acc723 Revert "Include tox_extra_args in tox siblings tasks"
This reverts commit 8b1cc73ee30428517f9524624b9eb309e9b14c1f.

Apparently tox.requires adds lines like this to verbose showconfig
output:

    using tox-3.24.4 from /.../tox/__init__.py (pid 2919)

Which is confusing the current parser in the siblings module. Roll
this back while we work on a fix and regression test.

Change-Id: If3b1d48b36a5d32fddfdabd9c0ec1b81dd6453f2
2021-09-30 17:54:17 +00:00
Zuul
7fcb90c59d Merge "Deprecate EOL Python releases and OS versions" 2021-09-30 17:33:30 +00:00
Zuul
266f3e3642 Merge "Explicit tox_extra_args in zuul-jobs-test-tox" 2021-09-30 15:51:44 +00:00
Zuul
f0d72b612b Merge "Include tox_extra_args in tox siblings tasks" 2021-09-30 15:51:42 +00:00
Zuul
e6cabd2f69 Merge "Support verbose showconfig in tox siblings" 2021-09-30 15:51:41 +00:00
Zuul
8c35f29ae1 Merge "Add tox_config_file rolevar to tox" 2021-09-30 15:37:15 +00:00
Zuul
8e85bcdce8 Merge "Pin to funcparserlib prerelease for new SetupTools" 2021-09-30 15:35:13 +00:00
Zuul
8b86f270d7 Merge "Stop testing playbooks/roles on Ubuntu Xenial" 2021-09-30 14:51:03 +00:00
Zuul
3c18483b01 Merge "Make default tox run more strict about interpreter version" 2021-09-29 17:44:02 +00:00
Jeremy Stanley
483838ceca Deprecate EOL Python releases and OS versions
Update the deprecation policy to indicate that zuul-jobs is no
longer tested with EOL platforms. Also explicitly switch the minimum
Python 3 documented to 3.6, and add a note to the tox-py34 and
tox-py35 jobs mentioning that they're no longer directly tested.
Move those jobs to the deprecated jobs list as well, to help
reinforce the point that their continued use is not recommended.

Change-Id: I2edbf8ea010caf7a7641e0d88f360965fc0b96ab
2021-09-23 17:35:32 +00:00
Zuul
91468c150c Merge "Pin protobuf<3.18 for Python<3.6" 2021-09-18 07:03:28 +00:00
Jeremy Stanley
22689ba524 Explicit tox_extra_args in zuul-jobs-test-tox
In change Iafeb88eaf9a596603ad4d2134a4574345d5189ab we fixed a bug
with handling of `tox --showconfig -vv` in the siblings tasks for
the tox role. This arose because of the desire to add tox_extra_args
to all tox invocations in Ibfe77f67e43135ae5af7588d6859b8b3dbd4c3ca
because we currently set a default of '-vv' for the tox_extra_args
rolevar. This fix could easily regress in the future if we were to
ever change the default value, so explicitly test with it in the
zuul-jobs-test-tox job by adding it to the python/tox test playbook.

Change-Id: Ib6ddf7d188904193a5ecd520acac1d676e5d78ae
2021-09-17 20:34:37 +00:00
Jeremy Stanley
8b1cc73ee3 Include tox_extra_args in tox siblings tasks
When the tox role was introduced, a tox_extra_args rolevar was
included allowing the tox command line to be extended with arbitrary
options. When siblings functionality was added, tox_extra_args did
not get included in its separate tox invocations. If a project has a
particular situation where some aspect of tox's functionality must
be overridden in order to work, doing so through tox_extra_args
needs to apply to every tox invocation, including siblings installs.

Change-Id: Ibfe77f67e43135ae5af7588d6859b8b3dbd4c3ca
2021-09-17 20:34:37 +00:00
Jeremy Stanley
bc8776bad4 Support verbose showconfig in tox siblings
Unfortunately, when tox combines --showconfig with verbosity options
like -vv, some non-config output gets streamed to stdout before the
configuration is emitted. Filter this preamble in
tox_install_sibling_packages by discarding any initial lines of
output before the first section heading.

Also extend get_envlist() to deal with the fact that additional
verbosity adds a [tox] section in the --showconfig output, which it
was previously relying on to determine whether the config had been
filtered to a subset of env sections. Instead also check the
tox.args string to determine whether a -e option was passed on the
command line.

Change-Id: Iafeb88eaf9a596603ad4d2134a4574345d5189ab
2021-09-17 20:34:37 +00:00