2051 Commits

Author SHA1 Message Date
Sorin Sbarnea
3febd1423a Enable progressive mode with ansible-lint
Makes use of newer feature which produces failures only when current
commit is adding new violations, exiting ones being considered
as already known.

To prove it works, it also removes the E208 from warn, making it
an error. Still, the final linting result is a success because
these violations were already present before current commit.

Change-Id: Ia858f2a3e71d9634e9d90e890d82714105e8f8c9
2020-11-07 03:20:19 +01:00
Sorin Sbarnea
90d263941a More E208
Change-Id: I8c3d224682d7d6780d6ad7823e390c47114e702b
2020-11-05 08:33:28 +00:00
Zuul
88ef27b414 Merge "More E208 fixes" 2020-11-04 03:14:09 +00:00
Mohammed Naser
fbe7ae16ad Install gnupg if keys need to be imported
The apt_key module currently requires that gnupg is in place
in order to be able to succesfully import keys.  It is possible
that the system running this role does not have this, so we
should make sure it is installed if we have any keys to install.

Change-Id: Ia5ff4d35dc4faed1fb8dfc6bd32670dae394e5c7
2020-11-03 12:10:54 -05:00
Sorin Sbarnea
d31aa166f4 Update ensure-docker for new releases
Docker just released official packages for centos-8 so
we need to remove the temporary workaround that installs the
previous release version.

Change-Id: I787ce82b895069f4846fcdb35f55f0c9ad53652c
2020-11-03 14:06:13 +00:00
Sorin Sbarnea
086d1a9284 More E208 fixes
Change-Id: Iddda539a24e03ae33ef5e37630f98a7268f18dbe
2020-11-03 10:04:27 +00:00
Zuul
b91fb9e066 Merge "Allow bindep role to read from more then one bindep file" 2020-11-02 22:37:57 +00:00
Zuul
d43cfdbc70 Merge "More E208 fixes" 2020-11-02 20:17:25 +00:00
Zuul
c6e3be065e Merge "emit-job-header: Print username in node information" 2020-11-02 20:04:29 +00:00
Sorin Sbarnea
b5fffff87b More E208 fixes
Change-Id: I95dca97c8f79a96d7710b33cb0ee243d3234b7ca
2020-11-02 18:38:54 +00:00
Michal
522a78385a Allow bindep role to read from more then one bindep file
Extend bindep role to be able to install packages from
multiple paths at once. This behavior is fully compatible
with current implementation, however it allows to pass
number of bindep file paths.

Change-Id: I70d42f25837ff282aad534f9ead965d12858ac9b
2020-11-02 17:17:47 +01:00
Vitaliy Lotorev
28796a9a90 emit-job-header: Print username in node information
Knowing username for node is required for jobs running on static nodes.
as static node can have multiple connections with different users
(see [1] and [2]).

[1] https://zuul-ci.org/docs/nodepool/configuration.html#attr-providers.[static].pools.nodes.username
[2] https://review.opendev.org/#/c/659209/

Change-Id: I3c726c40ee024de18e65b7888b73f505232422c2
2020-10-31 22:06:55 +03:00
Sorin Sbarnea
856866fdde More E208 mode fixes
Change-Id: I8157ec1f31b8c5a064b63002e8311b91ef9ce9ab
See: https://ansible-lint.readthedocs.io/en/latest/default_rules.html#file-permissions-not-mentioned
2020-10-31 07:34:03 +00:00
Zuul
73a2da3b12 Merge "fetch-sphinx-tarball: don't run merge-output-to-logs" 2020-10-30 20:03:58 +00:00
Zuul
2223c8835d Merge "fetch-sphinx-tarball: explain what is happening" 2020-10-30 19:46:11 +00:00
Zuul
2efb34d589 Merge "Improve errors from update-test-platforms" 2020-10-25 23:26:48 +00:00
Sorin Sbarnea
d27c7ff9db Improve errors from update-test-platforms
Instead of getting a confusing stacktrace when someone adds a new
file that is missing the final project entry we now give a friendlier
error that also recommends way to address the issue.

Change-Id: I5a84d3a20e847eeb2d5d843ef970b5790532683c
2020-10-25 23:18:30 +00:00
Zuul
c5d8494e2f Merge "Install openswitch and firewall if need a bridge only" 2020-10-24 21:25:24 +00:00
Ian Wienand
6f43acde41 fetch-sphinx-tarball: don't run merge-output-to-logs
The merge-output-to-logs role is not doing anything in this test.  As
described inline, this merges files in
zuul.executor.work_root/<docs,artifacts> into the
zuul.executor.log_root directory so they are available in change
results.

Since this job doesn't publish anything there, this role is unused.

merge-output-to-logs currently can't run because it tries to run shell
scripts on the executor.  Thus we can remove this unused role and
restore the job.

Change-Id: I1afc905aa8d9c420bed316e99760ad7ad1d838ce
2020-10-15 14:44:03 +11:00
Ian Wienand
be12793c20 fetch-sphinx-tarball: explain what is happening
This documents the steps in the fetch-sphinx-tarball test to explain
what is happening.

Change-Id: I91d03beeb15395cf4f243988434112ae250b1742
2020-10-15 13:34:49 +11:00
Zuul
ac6119dd94 Merge "Revert "Refactor fetch-sphinx-tarball to be executor safe"" 2020-10-14 21:07:18 +00:00
James E. Blair
5603eb2291 Revert "Refactor fetch-sphinx-tarball to be executor safe"
This reverts commit 51a8ed8e95a2240547b0128701cc8acf6ba8bbcc.

This has a typo ("exector").  The fix is obvious, but the bigger
issue is that it was not caught in testing, even though the main
purpose of the change was to re-enable tests.  We should understand
why it wasn't caught in testing and resolve that before fixing and
unreverting.

Change-Id: I3ed407546fecc52d4a039f7959c0521511e6a00b
2020-10-14 13:54:22 -07:00
Tobias Henkel
e854c90f1f
Remove unneeded gce from upload_utils
This is gce specific and already existing in the gce specific part and
can be just removed.

Change-Id: I8838e3587cf5a2044bd4945f5ecea3f3290deddc
2020-10-14 22:43:04 +02:00
Zuul
4b19def07d Merge "Refactor fetch-sphinx-tarball to be executor safe" 2020-10-14 18:51:05 +00:00
Zuul
5d1662b56b Merge "Consolidate common log upload code into module_utils" 2020-10-14 18:28:17 +00:00
Sagi Shnaidman
7975e874e8 Install openswitch and firewall if need a bridge only
In case of nodeset which doesn't have peers and switch there is
no need to install openswitch and firewall rules, because nothing
will be installed. Let's skip it if no need.
Change-Id: I98cf5ec390ee22e538baa076c9ab87eea6a44c9e
2020-10-13 14:25:37 +03:00
Zuul
7e150858e4 Merge "Use ansible_distribution* facts instead of ansible_lsb" 2020-10-12 18:24:58 +00:00
Zuul
d7c6088d62 Merge "validate-host: skip linux nly tasks on windows machines" 2020-10-07 20:13:34 +00:00
Zuul
aba28fa570 Merge "Add CentOS 8 Stream testing" 2020-10-07 07:39:05 +00:00
Carlos Goncalves
3f743e00fd Add CentOS 8 Stream testing
Add CentOS 8 Stream nodes to the testing regime.

Add repositories for CentOS 8 Stream to configure-mirrors

Depends-On: https://review.opendev.org/#/c/734788/
Change-Id: Ia2f2b22461b4f4eca19d294ae06f6e1c90ea8599
2020-10-07 07:11:35 +11:00
Zuul
ffbfbebb02 Merge "ensure-docker: < 1500 MTU workaround" 2020-10-06 17:10:46 +00:00
Zuul
8d75b3d91c Merge "Pin openstacksdk to <0.48.0" 2020-10-06 16:53:21 +00:00
Tobias Henkel
9ff9fa5827
Pin openstacksdk to <0.48.0
The openstacksdk dropped py35 support in 0.48.0. Since we still have
py35 support we need to pin it temporarily until we've deprecated and
removed it.

Change-Id: Ia50ed54f43605fc1e12c95a2218b73abcadc66ed
2020-10-06 17:12:11 +02:00
Ian Wienand
664d068ef7 ensure-docker: < 1500 MTU workaround
As described inline, we should lower the MTU in the docker
configuration when we see the interface has a MTU lower than 1500 so
things "just work".  This particularly affects the Linaro ARM64 cloud
in OpenDev, but it is a generic issue.

Change-Id: I338616c41a65b007d56648fdab6da2a6a6b909f4
Story: https://storyboard.openstack.org/#!/story/2008230
2020-10-06 11:26:15 +00:00
Sorin Sbarnea
a1742afb60 ensure-docker: validate network connectivity
Verifies that installed docker can download and run containers
that need network access. This should prevent bugs where
service was installed but in a broken state.

Fixes bug which failed to run tests when tests were modified.

Change-Id: I309168719fd3cb7488bc2d0f4fec7785e1eb5d53
Story: https://storyboard.openstack.org/#!/story/2008215
2020-10-06 15:42:28 +11:00
Zuul
e59b68d909 Merge "update-json-file: add role to combine values into a .json" 2020-10-05 06:43:23 +00:00
Ian Wienand
0b9fad9583 update-json-file: add role to combine values into a .json
Ansible doens't really have a great built-in way to modify a json file
(unlike ini files).  The extant docker role does what seems to be the
usual standard, which is slurp in the file, parse it and then write it
back out.

In a follow-on change (I338616c41a65b007d56648fdab6da2a6a6b909f4) we
need to set some more values in the docker configuration .json file,
which made me think it's generic enough that we can have a role to
basically run read the file, |combine and write it back out.

This adds such a role with various options, and converts the existing
json configuration update in ensure-docker to use it.

Change-Id: I155a409945e0175249cf2dc630b839c7a97fb452
2020-10-05 15:18:58 +11:00
Ian Wienand
51a8ed8e95 Refactor fetch-sphinx-tarball to be executor safe
This reverts commit 69a238df46ca81e8890ebb2ace7addcbb4852911.

The role is re-written with executor-safe methods.

Depends-On: https://review.opendev.org/753222
Change-Id: I0b52eff66bfdca776e0e5c426bf1fc57deb3fc49
2020-10-05 15:14:31 +11:00
Zuul
ee7ff55598 Merge "Partial address ansible-lint E208" 2020-10-02 17:18:23 +00:00
Tobias Henkel
eb19f437f7
Consolidate common log upload code into module_utils
All log upload modules use a large portion of common code for finding
files and creating indexes. Move this into module_utils so this can be
shared between all.

Change-Id: I16c105bc70e07b0c8a4aa8e96119ab7451e00346
2020-10-02 12:40:14 +02:00
Zuul
1dbdc0e3fa Merge "Fix certificate issue with use buildset registry" 2020-10-02 00:26:58 +00:00
Ian Wienand
6f76d3f68a ensure-twine: remove readme_renderer
This dependency was added with
Iaf4da5aedaa3814b2ecebed4391da2324d3e388d to prevent a warning when
using "twine check" with text/markdown descriptions.

Per [1], this check doesn't really do anything and will never fail.

We are now having issues using this role with with Python 3.8 on the
executor as the md dependency pulls in cmarkgfm which has binary
dependencies but does not publish any wheels; meaning the executor
fails to install as it does not have a toolchain setup.  Unfortunately
upstream is not particularly active, so building good manylinux wheels
is not something we're going to be able to deal with in a timely
fashion.

Given that this is not a failure, and isn't really doing anything and
blocks Python 3.8, it seems our best course is to remove this
dependency.

[1] https://github.com/pypa/twine/pull/421

Change-Id: Iac5c9f63d41375889e4fdad67b9a45a24a644341
2020-10-02 09:08:58 +10:00
Zuul
effff59852 Merge "Merge upload logs modules into common role" 2020-10-01 16:00:31 +00:00
Zuul
5e5df2ae70 Merge "shake-build: add shake_target variable" 2020-09-30 15:41:35 +00:00
Andrii Ostapenko
fd2dc3bc91
Fix certificate issue with use buildset registry
Use buildset_registry_alias for certificate file name to address
'unknown certificate authority' issue on consumer side when using standalone
use-buildset-registry role.

Issue is not visible with opendev parent jobs because certificate is handled
in pull-from-intermediate-registry role there.

Change-Id: Ieaf43be0dd4ccb92a8240a493ee6636a23c9d484
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-29 10:24:23 -05:00
Clark Boylan
d49893f894 Partial address ansible-lint E208
- replaces ignore with a warn, which displays issue without affecting
  the linting outcome (allowing gradual fixing)
- bumps linter to enable the warn_list feature
- fixes a set of issues, others will be fixed in follow-up

Change-Id: I7d6f8c156b06f68f681943e88860930968e7c9f9
2020-09-29 10:29:01 +01:00
Zuul
cbe1b4a5aa Merge "Fix promote cleanup" 2020-09-29 06:42:44 +00:00
Zuul
d69c9f75b1 Merge "Make sure that we pass list in loop" 2020-09-29 01:35:06 +00:00
Dmitriy Rabotyagov
39776cfdd9 Make sure that we pass list in loop
map returns a generator that needs to be converted to a list for
the loop.

We also don't need unique as this is iterating over a list of
directory names that must already be unique.

Change-Id: Ibd22d79be29aaa9d3a7924319c59929e665f9cbc
2020-09-28 22:45:13 +00:00
Andrii Ostapenko
a8084e54ef
Fix promote cleanup
Change-Id: I1b639e5ed221301219f808a53dcb938cccefa019
Signed-off-by: Andrii Ostapenko <andrii.ostapenko@att.com>
2020-09-28 16:58:19 -05:00