opendev/base-jobs
Code Issues Proposed changes
base-jobs/zuul.d/jobs.yaml
Andreas Jaeger 3ea745fc46 Add docs templates 
Add project-templates for:
* Building documents
* Building documents and publishing them after each change

Use the new publish project-template for this repo, put the config in a
separate file to find it easier.

Update description of opendev-promote-docs.

Change-Id: Id76a751feb152afe8fca0893d9e0013440c51401
2019-10-02 20:23:14 +02:00

470 lines
15 KiB
YAML
Raw Blame History

# Shared zuul config common to all opendev tenants.
# Contains definitions of trusted jobs
# Changes to this job require a special procedure, because they can
# not be tested before landing, and if they are faulty, they will
# break all jobs, meaning subsequent corrections will not be able to
# land. To make a change:
#
# 1) Ensure that base-test and its playbooks are identical to base.
# 2) Make the change to base-test and/or its playbooks.
# 3) Merge the change from step 2. No jobs normally use base-test, so
# this is safe.
# 4) Propose a change to a job to reparent it to base-test. Choose a
# job which will exercise whatever you are changing. The
# "unittests" job in zuul-jobs is a good choice. Use [DNM] in the
# commit subject so that people know not to merge the change. Set
# it to "Work in progress" so people don't review it.
# 5) Once test results arrive for the change in step 2, make a change
# which copies the job and/or playbooks of base-test to base. In
# the commit message, link to (without using Depends-On:) the
# change from step 4 so reviewers can see the test results.
# 6) Once the change in step 5 merges, abandon the change from step 4.
- job:
name: base
parent: null
abstract: true
description: |
The base job for OpenDev's installation of Zuul.
All jobs ultimately inherit from this. It runs a pre-playbook
which copies all of the job's prepared git repos on to all of
the nodes in the nodeset. It runs a post-playbook which copies
all of the files in the logs/ subdirectory of the executor
work directory to the logserver.
It also sets default timeout and nodeset values (which may be
overidden).
Responds to these variables:
.. zuul:jobvar:: base_serial
:default: Omitted
This sets the serial keyword in the pre and post playbooks
which can be an integer or percentage.
See ansible documentation for more information:
http://docs.ansible.com/ansible/latest/playbooks_delegation.html
pre-run: playbooks/base/pre.yaml
post-run:
- playbooks/base/post.yaml
- playbooks/base/post-logs.yaml
cleanup-run: playbooks/base/cleanup.yaml
roles:
- zuul: zuul/zuul-jobs
timeout: 1800
post-timeout: 1800
vars:
ara_report_type: html
ara_report_path: ara-report
ara_compress_html: false
nodeset: ubuntu-bionic
secrets: &log_clouds
- opendev_cloud_fn_one
- opendev_cloud_ovh_bhs1
- opendev_cloud_ovh_gra1
- opendev_cloud_rax_dfw
- opendev_cloud_rax_iad
- opendev_cloud_rax_ord
- opendev_cloud_vexxhost_ymq1
# See the procedure described above "base" before making changes to
# this job.
- job:
name: base-test
parent: null
ansible-version: 2.8
description: |
A job to test changes to the base job without disturbing the
main job in production. Not for general use.
pre-run: playbooks/base-test/pre.yaml
post-run:
- playbooks/base-test/post.yaml
- playbooks/base-test/post-logs.yaml
cleanup-run: playbooks/base-test/cleanup.yaml
roles:
- zuul: zuul/zuul-jobs
timeout: 1800
post-timeout: 1800
nodeset: ubuntu-bionic
secrets: *log_clouds
- job:
name: base-minimal
parent: null
abstract: true
description: |
A subset of what the 'base' job provides: the absolute minimum considered
required to run for any one job.
It doesn't set up cached git repositories, will not set up mirrors,
doesn't validate the node, etc.
It is meant to be used, amongst other things, to test roles and
playbooks that would otherwise be included by default as part of the
'base' job.
These tasks, if required, can be included by the dependant jobs
themselves on a need basis.
pre-run: playbooks/base-minimal/pre.yaml
post-run:
- playbooks/base-minimal/post.yaml
- playbooks/base-minimal/post-logs.yaml
cleanup-run: playbooks/base-minimal/cleanup.yaml
roles:
- zuul: zuul/zuul-jobs
timeout: 1800
post-timeout: 1800
allowed-projects: zuul/zuul-jobs
vars:
ara_report_type: html
ara_report_path: ara-report
ara_compress_html: false
nodeset: ubuntu-bionic
secrets: *log_clouds
- secret:
name: opendev-intermediate-registry
data:
host: insecure-ci-registry.opendev.org
port: 5000
username: zuul
password: !encrypted/pkcs1-oaep
- sN5wugpZqGCp8kwHLDLydHi7HUH5e5gLKA/Xhge0DdtcTLy4TLWASDvPwKkC+w3Y1CdQG
V0kPMJCKodsx3jz83zbcjptAWV90RMfgOmwy1FOs9/fnQtcnH2UDkoFhf0m3qYu05kbyB
yJ9ybIewEFt9v1GORcFSAr7Z8YIyhBNuJaRGb9dDW/T9WTBS+bhxMVq1hlOAq17458UXV
MhRTb5+nFhJxVaXYpV3rGjqr2TsryBk16iu5kf8I8/6QkwpttI1i/F6wFrjWBNg1FkDIN
OwcoGnqY9sTmCya54Ko03FroRP4pe09KcsFoGHMVYNT4CAbhrMEIBtyRk5q/MHls/o82p
fHu1Xr8dC/u8vHfYnL3TEtO87tucRzW9CJBCrrrq77a/i0BS6UFpMY63dnwI9M9h1v4Zp
y/ITLk/YNdqR8lxG/T7fQg1YmBjSAQCJKfHS4uulujrcbPp2X1RXAFooUPH+uZ4m+Ml4b
KlDb/XtMzYw2zjVIi0nMfPG0jfEgtzHBW23Nf62joUrmmtUWSq1aij5J+Vb2tBiuDFpN6
+v5Gg5Wb07OWvBjg0jcL9IvNwQXgWh39y4uq4TmKcYXHL6RnICtN5Hp4Gc7xDxNvp2+uC
Na/OOailVlvkWMcpO/l77/qFtNXoRxFTDWYqp5rE3fGmdq09hCiflFWgMMre08=
- job:
name: opendev-buildset-registry
description: |
Starts a buildset registry which interacts with the intermediate
CI registry to share speculative container images between
projects.
Configure any jobs which require the use of a buildset registry
to depend on this job using the "dependencies" job attribute.
This job will pause after starting the registry so that it is
available to any jobs which depend on it. Once all such jobs
are complete, this job will finish.
pre-run: playbooks/buildset-registry/pre.yaml
run: playbooks/buildset-registry/run.yaml
post-run: playbooks/buildset-registry/post.yaml
secrets:
- secret: opendev-intermediate-registry
name: intermediate_registry
requires: docker-image
- job:
name: opendev-build-docker-image
parent: opendev-buildset-registry
description: |
Starts a buildset registry (if one has not already been started,
e.g., by invoking :zuul:job:`opendev-buildset-registry` and
specifying it as a dependency) and builds one or more docker
images.
Analog of build-docker-image job, but with a buildset registry.
This job will pause after starting the registry so that it is
available to any jobs which depend on it. Once all such jobs
are complete, this job will finish.
.. include:: ../../playbooks/docker-image/README.rst
run: playbooks/docker-image/run.yaml
provides: docker-image
- job:
name: opendev-upload-docker-image
parent: opendev-build-docker-image
description: |
Starts a buildset registry and builds and uploads one or more
docker images to docker.io.
Analog of upload-docker-image job, but with a buildset registry.
.. include:: ../../playbooks/docker-image/README.rst
.. include:: ../../playbooks/docker-image/credentials.rst
post-run: playbooks/docker-image/upload.yaml
- job:
name: opendev-promote-docker-image
parent: promote-docker-image
description: |
Retag a previously-uploaded docker image.
Analog of promote-docker-image job.
.. include:: ../../playbooks/docker-image/README.rst
.. include:: ../../playbooks/docker-image/credentials.rst
- job:
name: opendev-tox-docs
# This is not parented to tox-docs because the post playbook
# differs.
description: |
Build documentation with "tox".
Uses tox with the ``docs`` environment.
vars:
tox_envlist: docs
bindep_profile: compile doc
pre-run: playbooks/tox-docs/pre.yaml
run: playbooks/tox-docs/run.yaml
post-run: playbooks/tox-docs/post.yaml
success-url: docs/
- job:
name: opendev-publish-tox-docs-base
# This is not parented to opendev-tox-docs because the post
# playbook differs.
description: |
Publish a ref-based documentation build.
Use this in the tag or release pipelines to publish a build
based on a newly-created tag.
This is an abstract job intended to be inherited from in an
OpenDev tenant and an appropriate secret added.
.. zuul:jobvar:: afs
:type: dict
This is expected to be a Zuul Secret with these keys:
.. zuul:jobvar:: keytab
The AFS keytab for the service principal.
.. zuul:jobvar:: service_name
The name of the service princpal.
.. zuul:jobvar:: docs_master_path
The full docs publication path to use if the job is run on
the master branch.
.. zuul:jobvar:: docs_branch_path
The full docs publication path to use if the job is run on
any other branch.
.. zuul:jobvar:: docs_tag_path
The full docs publication path to use if the job is run on
a tag.
abstract: True
vars:
tox_envlist: docs
bindep_profile: compile doc
pre-run: playbooks/tox-docs/pre.yaml
run: playbooks/tox-docs/run.yaml
post-run:
- playbooks/tox-docs/post.yaml
- playbooks/tox-docs/publish.yaml
- job:
name: opendev-publish-tox-docs
parent: opendev-publish-tox-docs-base
description: |
Publish a ref-based documentation build.
Use this in the tag or release pipelines to publish a build
based on a newly-created tag.
post-run: playbooks/tox-docs/publish.yaml
secrets:
- secret: opendev-zuul-docs
name: afs
pass-to-parent: true
- job:
name: opendev-promote-docs-base
description: |
Publish a previously built branch-tip documentation tarball.
Use this in the promote pipeline to publish a branch tip tarball
built in the gate pipeline.
This is an abstract job intended to be inherited from in an
OpenDev tenant and an appropriate secret added.
.. zuul:jobvar:: afs
:type: dict
This is expected to be a Zuul Secret with these keys:
.. zuul:jobvar:: keytab
The AFS keytab for the service principal.
.. zuul:jobvar:: service_name
The name of the service princpal.
.. zuul:jobvar:: docs_master_path
The full docs publication path to use if the job is run on
the master branch.
.. zuul:jobvar:: docs_branch_path
The full docs publication path to use if the job is run on
any other branch.
.. zuul:jobvar:: docs_tag_path
The full docs publication path to use if the job is run on
a tag.
.. zuul:jobvar:: docs_redirect_path
If this variable is present, a .htaccess redirect will be
created at this path when the job is run on the master
branch. For example, it can be used to redirect "project/"
to "project/latest".
.. zuul:jobvar:: docs_redirect_content
The contents of the .htaccess file in docs_redirect_path.
.. zuul:jobvar:: download_artifact_job
The name of the job which built the docs artifact which this
job should download and promote.
.. zuul:jobvar:: write_root_marker
If this is set to false, then the root marker file is not
written.
Warning: setting this parameter incorrectly can result in loss of published data.
abstract: True
run: playbooks/docs/promote.yaml
vars:
write_root_marker: true
nodeset:
nodes: []
- job:
name: opendev-promote-docs
parent: opendev-promote-docs-base
description: |
Publish a previously built branch-tip documentation tarball.
Use this in the promote pipeline to publish a branch tip tarball
built in the gate pipeline.
The documentation tarball is published to
https://docs.opendev.org/{{ zuul.project.name }}.
Publishes depending on branch to latest/ (for master), or the
basename of the branch like train (for stable/train).
vars:
download_artifact_job: opendev-tox-docs
secrets:
- secret: opendev-zuul-docs
name: afs
pass-to-parent: true
- job:
name: opendev-promote-artifact-base
description: |
Publish a previously built branch-tip artifact.
Use this in the promote pipeline to publish a branch tip artifact
built in the gate pipeline.
This is an abstract job intended to be inherited from in an
OpenDev tenant and an appropriate secret added.
.. zuul:jobvar:: afs
:type: dict
This is expected to be a Zuul Secret with these keys:
.. zuul:jobvar:: keytab
The AFS keytab for the service principal.
.. zuul:jobvar:: service_name
The name of the service princpal.
.. zuul:jobvar:: artifacts_path
The full publication path to use.
.. zuul:jobvar:: download_artifact_job
The name of the job which built the artifacts which this
job should download and promote.
.. zuul:jobvar:: download_artifact_type
The type of the artifact to download (as specified in the
``type`` attribute of the artifact metadata).
.. zuul:jobvar:: artifact_extra_name
The artifact will be renamed to PROJECT-BRANCH.ext; if this
argument is present, it will be PROJECT-EXTRA-BRANCH.ext.
abstract: True
run: playbooks/artifacts/promote.yaml
nodeset:
nodes: []
- job:
name: opendev-promote-python
parent: opendev-promote-artifact-base
description: |
Publish a previously built branch-tip sdist/wheels.
Use this in the promote pipeline to publish a branch tip
sdist and wheel(s) built in the gate pipeline.
vars:
download_artifact_job: build-python-release
download_artifact_type:
- python_sdist
- python_wheel
secrets:
- secret: opendev-zuul-tarballs
name: afs
pass-to-parent: true
- job:
name: opendev-promote-javascript-content
parent: opendev-promote-artifact-base
description: |
Publish a previously built branch-tip javascript content archive.
Use this in the promote pipeline to publish a branch tip
javascript content archive built in the gate pipeline.
vars:
download_artifact_job: build-javascript-content-tarball
download_artifact_type: javascript_content
artifact_extra_name: js-content
secrets:
- secret: opendev-zuul-tarballs
name: afs
pass-to-parent: true
- job:
name: opendev-release-python
description: Release python tarballs / wheels to pypi.
pre-run: playbooks/release-python/pre.yaml
run: playbooks/release-python/run.yaml
post-run: playbooks/release-python/post.yaml
secrets:
- secret: opendev-pypi
name: pypi_info
View Git Blame Copy Permalink