1284 Commits

Author SHA1 Message Date
Zuul
5ea9bac2df Merge "Add prepare-workspace-openshift role" 2020-01-23 16:19:37 +00:00
Andreas Jaeger
ba12fb98a0 Fix ansible-2.9 skipped problem
ansible-2.9 does not allow "|skipped" anymore, it needs "is skipped"
instead, fix the only usage.

See also Ic0c93f2ea896ead76922694e6150882d8a280daf and
Id6c62da8f8a94c270d8a5d5981683a9d8c4e7ad6.

Change-Id: I597c92d2f6dce31dab1ac4bf52bb534c8146b33b
2020-01-23 07:47:49 +01:00
Zuul
bf083b71fd Merge "Revert "Make ara-report role to zuul_return an artifact"" 2020-01-23 06:44:59 +00:00
Paul Belanger
0f0d9bd2e2 Revert "Make ara-report role to zuul_return an artifact"
This seems to introduce the following error with Ansible 2.9

 "The conditional check 'not ara_generated | skipped' failed. 
 The error was: template error while templating string: no filter named 'skipped'. 

This reverts commit 262c16b30583f634588f5a52bd7aab944687a8a4.

Change-Id: Id6c62da8f8a94c270d8a5d5981683a9d8c4e7ad6
2020-01-23 00:39:47 +00:00
Zuul
0ed3f04dd1 Merge "Make ara-report role to zuul_return an artifact" 2020-01-22 10:33:15 +00:00
Zuul
b37367637e Merge "ensure-tox: Output tox version" 2020-01-21 21:32:25 +00:00
Ian Wienand
9e5907f4cc ensure-tox: fix pipe race
The

 command -v pip pip3 | head -n1

introduced with Ie50928c9b782ea84db916bb1441567e1206ff466 has a very
subtle race; if "pip" and "pip3" exists and there are two lines
output, the "head -n1" will exit and depending on scheduling the
"command" might write to a broken pipe (this manifests as exit code
141).

Move this to a more explicit if statement.

Co-Authored-By: Jens Harbott <j.harbott@x-ion.de>
Change-Id: I80823a7bc6351925d6f0b20bdebca3eafef0b27d
2020-01-22 07:49:35 +11:00
David Shrewsbury
4538583b1a ensure-tox: Output tox version
This can be useful for debugging.

Change-Id: Ic5ffb46cf2f583cd6dd04289e55f40f8acd3d9bf
2020-01-21 15:00:05 -05:00
Andreas Jaeger
5eb4c257f5 fetch-sphinx: Exclude doctrees directory
The sphinx .doctrees directory is not needed for publishing. By default
sphinx puts it below html. Some projects add to sphinx-build a "-d"
option to put the content elsewhere and not publish it.

This change assures that a .doctrees directory below "html" will not put
stored as artifact and published later.

Statistics: In all of OpenDev, we have:
* 511 repos putting .doctrees elsewhere and thus not publishing it
* 631 repos  no caring and thus publishing .doctrees

The size of .doctrees directory is not neglectable: 2MB for
system-config, 14 MB for nova.

Change-Id: I63b581164aab84352d9e80278cf4a8d634ff28af
2020-01-21 08:32:26 +01:00
Andreas Jaeger
f2680ebc70 Remove trusty testing
OpenDev is removing trusty nodes, remove test jobs.

Change-Id: Ibdd903daeefcec40990dc777d53af5cf3ea7844e
2020-01-17 10:44:38 +01:00
Sorin Sbarnea
2410119ac9 ensure-tox: improve pip detection
Fixed issues failing to install tox on python3 only systems which have
only pip3 executable and not the pip one.

Needed-By: https://review.rdoproject.org/r/#/c/24584/
Change-Id: Ie50928c9b782ea84db916bb1441567e1206ff466
2020-01-16 21:06:07 +00:00
Zuul
fc9d36ffe4 Merge "Stop compressing files during intermediate steps part 2" 2020-01-15 16:48:02 +00:00
Sagi Shnaidman
4d168ed016 Rename vars/ directory to prevent conflicts
/vars directories from 'tox' playbook and roles are merged, which
creates conflicts and wrong variables are loaded into the playbook
Prevent merge of /vars directories from roles and 'tox' playbook
by renaming variables directory of playbook to molecule-vars.
It should fix failing job tripleo-ansible-centos-7-role-addition

Change-Id: Iee6768a27db36a3a18c0728040ab9c09857d55b6
2020-01-13 02:30:17 +02:00
Zuul
33fc86060b Merge "Fix typo in helm role" 2020-01-11 08:45:30 +00:00
Zuul
2513e322e2 Merge "apply-helm-charts: Job to apply Helm charts" 2020-01-11 08:45:29 +00:00
Zuul
5fb6ba9dfe Merge "helm-template: Add role to run 'helm template'" 2020-01-11 08:45:29 +00:00
Zuul
b864698116 Merge "collect-container-logs: add role" 2020-01-10 21:56:26 +00:00
James E. Blair
3748623564 Fix typo in helm role
s/go/helm.

Change-Id: I840fff4e556df4b99994830755daddbe54222bb2
2020-01-10 13:26:51 -08:00
Zuul
e5f39018e4 Merge "install-go: bump version to 1.13.5" 2020-01-10 17:42:50 +00:00
Mohammed Naser
5833a6f8e3 apply-helm-charts: Job to apply Helm charts
This job deploys a Kubernetes cluster and then runs all the
Helm charts provided.

Change-Id: If6c4a6d5bce912f55549d16175554154c86dd25a
2020-01-10 17:31:38 +00:00
Mohammed Naser
8ae49163f3 helm-template: Add role to run 'helm template'
This role runs Helm template of a specific chart and then makes
sure that all resources become ready afterwards.

Change-Id: I62e160eac673d13bd1018b58fd36cae54e1482b4
2020-01-10 17:31:05 +00:00
Zuul
982ab1c6c7 Merge "Make pre-molecule tox playbook platform agnostic" 2020-01-10 15:38:40 +00:00
Mohammed Naser
228033e78a collect-container-logs: add role
We use this in a few different places and it's really useful
to collect all the logs of all containers.

Change-Id: Idc46a47f444bf48cd040f4f9724f3a6ee8bc8f8e
2020-01-09 19:52:01 -05:00
Zuul
96300f3543 Merge "Move chart-testing to a role and disable version check" 2020-01-09 22:31:14 +00:00
Zuul
14478de03f Merge "Add basic Helm jobs" 2020-01-09 21:19:44 +00:00
James E. Blair
ffb6406a40 Move chart-testing to a role and disable version check
Move the chart-testing task to a role (so that it is easily
reconsumable) and update the default arguments to disable checking
for version bumps (instead, we expect to rely on setting the version
when building a release).

Change-Id: I85482dc704c189222c92fd4174f733382ebf69d1
2020-01-09 11:46:41 -08:00
Zuul
1e378f4071 Merge "Add cri-o support to use-buildset registry" 2020-01-09 16:49:54 +00:00
Clark Boylan
a6f2ca827e Stop compressing files during intermediate steps part 2
Stop compressing files that would otherwise be handled uncompressed in
intermediate log handling. The reason for this is final log/artifact
upload should decide if a file need to be compressed or not as some file
hosting services (like swift) may not return the expected data if we get
this wrong.

This was noticed when we used swift to store intermediate tar.gz
tarballs before final upload to permanent storage. When retrieving the
tar.gz files from swift we got them back as uncompressed .tar files
(because swift was being helpful) and then later when we try to
uncompress these files we break.

Instead we'll let the upload to swift (or other storage) decide if files
should be compressed further than their existing state.

This is the second change in the stack and update the stage-output role.

Change-Id: I2fa6f123d23568ba1537a7c26d2672f04110c35b
2020-01-08 09:24:45 -08:00
Clark Boylan
bbca430417 Stop compressing files during intermediate steps part 1
Stop compressing files that would otherwise be handled uncompressed in
intermediate log handling. The reason for this is final log/artifact
upload should decide if a file need to be compressed or not as some file
hosting services (like swift) may not return the expected data if we get
this wrong.

This was noticed when we used swift to store intermediate tar.gz
tarballs before final upload to permanent storage. When retrieving the
tar.gz files from swift we got them back as uncompressed .tar files
(because swift was being helpful) and then later when we try to
uncompress these files we break.

Instead we'll let the upload to swift (or other storage) decide if files
should be compressed further than their existing state.

This is the first change in the stack that updates this behavior for
fetch-subunit-output.

Change-Id: I5727b270d6d34c256fe78a443baa1ad6c3474108
2020-01-08 09:24:43 -08:00
Mohammed Naser
b476e34ab0 Add basic Helm jobs
This adds a simple job which runs the `chart-testing` linter against
a repository.

Change-Id: I974c64343e88c235a6dc887d5378f7db2ee3b1fc
2020-01-07 17:12:15 +00:00
James E. Blair
b6cb191295 Add cri-o support to use-buildset registry
This lets use-buildset-registry notify cri-o about the new
registries.conf file if it is being used as the container backend
for k8s.

Change-Id: Ia1805519ab4b6bb5f79df0492f702effc6a3e024
2020-01-07 07:35:28 -08:00
Clark Boylan
0e23325a12 Swift upload logs without encoding type for gzip files
When retrieving gzipped files out of swift with gzip encoding set some
swift implementations return a decompressed version of the file if your
client cannot accept gzip encoding. This causes problems when the file
you want is actually compressed like a .tar.gz. Instead we avoid setting
the encoding type of these files forcing swift to give them back as is.

This change should only be approved after confirming its parent is
tested and working.

Change-Id: Ibcf478b572ba84273732e0ede17bf92bddd8c36f
2020-01-06 13:50:01 -08:00
Clark Boylan
9352276cc2 Test swift upload logs without encoding type for gzip files
When retrieving gzipped files out of swift with gzip encoding set some
swift implementations return a decompressed version of the file if your
client cannot accept gzip encoding. This causes problems when the file
you want is actually compressed like a .tar.gz. Instead we avoid setting
the encoding type of these files forcing swift to give them back as is.

This change will be followed by a change that updates the production
role.

Change-Id: Ica27cf070d50fcafd879b6f8ccc3964afa9bf5e6
2020-01-06 13:50:01 -08:00
Zuul
e5472ffa5a Merge "Ensure python interpreter is installed for tox-py3X" 2020-01-06 17:22:03 +00:00
Zuul
e21bed0c48 Merge "Add pypi_fqdn to differentiate it package mirrors" 2020-01-06 16:36:58 +00:00
Kevin Carter
7f156f22c7
Make pre-molecule tox playbook platform agnostic
This change updates the tox molecule playbook to make it platform
agnostic by moving the OS specific package lists into var files
that will be dynamically loaded based on the operating system
the playbook is being executed against. This is being done so
that we can use molecule on OS's like CentOS, Ubuntu, SUSE,
Debian, etc. While the only platforms supported at this time
are Red Hat based, this change will allow additional var files
to be added enabling additional platforms without needing any
chanages to our task structure.

> The linters job has been updated to exclude vars files from the
  ansible lint playbook commands. This is done because the playbook
  commands assume all found files are playbooks.

Change-Id: I88f3551838e3676374d0a795631c8769ba40fbf4
Signed-off-by: Kevin Carter <kecarter@redhat.com>
2020-01-02 09:49:28 -06:00
Andreas Jaeger
25cc095e53 Ensure python interpreter is installed for tox-py3X
Add tox-py38 and ensure that python interpreter is installed for tox-py3
jobs. The job includes the ensure-python role that checks and installs the
interpreter if python_version is set.

Test py37 to ensure that this works, it's the version currently used for
next OpenStack release, so let's test changes to roles and jobs here.

Run tox-py35 on Xenial, since that Xenial has Python 3.5 - it was run
previously on the default (3.6 in our install) instead, this change
showed that we run the wrong interpreter for tox-py35 on this repo.

Change-Id: I46d6780a5a32c443c37ef040b2af2459641dd123
2020-01-02 15:51:30 +01:00
Mohammed Naser
e38f43abd9 install-go: bump version to 1.13.5
1.31.1 contained some critical fixes regarding go module
handling, let's just jump to the latest minor release.

Change-Id: I829d3a1550d343e778e79b519925099651392eb1
2019-12-23 23:20:12 -05:00
Clark Boylan
b62c488eab Fix ansible use of filters and tests
Ansible 2.9 requires the use of "version" as a filter and
"version_compare" has been removed. Also tests cannot be used as
filters which means we cannot do when: foo | test and must do
when: foo is test instead. Make these fixes.

Both changes should be backward compatible to ansible 2.5:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html#version-comparison
https://docs.ansible.com/ansible/latest/porting_guides/porting_guide_2.5.html#jinja-tests-used-as-filters

Change-Id: Id95f674a485877db2a7924994366d1c6c591a684
2019-12-17 10:49:57 -08:00
Clark Boylan
a554a401ea Use present for package state instead of installed
Ansible 2.9 does not accept the "installed" state so we need to update
occurences of that to "present".

Change-Id: I13c95794efc63697c2418334ded7c6912bd7b59f
2019-12-17 08:24:05 -08:00
Zuul
5b0644185f Merge "Adds go jobs and roles." 2019-12-13 17:50:51 +00:00
Zuul
f1ff15cdcd Merge "fetch-zuul-cloner: replace /home/zuul by ansible_user_dir" 2019-12-13 17:18:09 +00:00
Zuul
8a8adfb58f Merge "openshift speculative containers" 2019-12-11 19:17:27 +00:00
Clark Boylan
f418a8d533 Fix python3 compat in tox siblings handling
This fixes a byte vs str problem on python3 in the tox role's sibling
library.

As part of this fix add unit testing of the function that was failing
under python3 and functional testing of the role that covers siblings.

Change-Id: I2df4ddce91dfd53f5c4177a19288e97120bc3e3b
2019-12-10 15:27:41 -08:00
Zuul
641baedc4c Merge "build-docker-image: fix up siblings copy" 2019-12-09 23:49:16 +00:00
Zuul
939a09387d Merge "build-container-image: support sibling copy" 2019-12-09 23:42:30 +00:00
Zuul
7f6710385f Merge "Fixes tox log fetching when envlist is set to 'ALL'" 2019-12-09 00:18:55 +00:00
Ian Wienand
599954715e build-docker-image: fix up siblings copy
There are a number of issues with this.  Firstly, it needs to copy the
parent directories to make a heirarchy in the .zuul-siblings
directory.  The current "cp -r" was only copying the final directory.
Switch into the source directory and use "--parent" to do this.

Also, it should be copying into the context dir.  Add the
{{ item.context }} to the path where appropriate.

Make new testing image that copies in files from the siblings.
Because COPY will fail if the sources aren't there, this is like an
assert that we copied it correctly.

Change-Id: I9f3b0a1f71d20cf7511f224648dd2fa51a039015
2019-12-09 11:14:21 +11:00
Ian Wienand
29d55cbf17 build-container-image: support sibling copy
Add support for "siblings:" to the build-container-image role.  This
is a port of change I079d823e7194e15b1b496aea0f53f70f6b563f02 from the
build-docker-image role.

Note that we found a couple of issues with this role, fixed with
I9f3b0a1f71d20cf7511f224648dd2fa51a039015.  However,
build-container-image has merged before this fix, so thess fixes now
follow this change.  To avoid squashing this into something confusing
this update incorporates those fixes.  However testing is left to the
follow-on change because the extant build-docker-image role would fail
the corrected testing.  They should be merged roughly together.

One minor thing is the removal of an additional duplicated "tags:"
entry from the variable documentation.

Change-Id: I71e2f2ab29446bd113644ce8c75378b8d3ef8ba6
2019-12-09 11:13:42 +11:00
James E. Blair
6a9aa82fcc openshift speculative containers
Change-Id: I81f7be6065a74f45386ea735a5291d48b7e042ce
2019-12-06 13:50:25 -08:00