opendev/base-jobs
Code Issues Proposed changes
base-jobs/zuul.yaml
James E. Blair e8ad51ae41 Always pause docker build jobs 
The design idea for these jobs was that the buildset-registry job
would be used in systems where a dedicated registry job was desired
(e.g., because there are multiple image build jobs).  The image-build
job inherits from the buildset-registry job so that if a registry is
not already running, it runs one, but it overrides the playbook so
that it does not pause.  In other words, it was meant only to do what
is necessary to build its own image(s), not to serve them to other
jobs.

However, in the (arguably common) case that a repo has a single image
build job as well as jobs which could consume those images, it would
be efficient to have the image build job run the registry, build the
images, then pause for the registry to be used by the consuming jobs.

Rather than ask users to make a new playbook which restores the job
pause, let's have the image build job pause iff it is running a
buildset registry.  If there are no child jobs, Zuul will resume
the job immediately, so there won't be any change in behavior.  But
if there are child jobs, they will be able to use the registry
running on the image build job.

Change-Id: Iebd47305eeafbfaac0f699aae3e0a7d7a36f85ee
2019-04-02 15:12:34 -07:00

441 lines
17 KiB
YAML
Raw Blame History

# Shared zuul config common to all opendev tenants.
# Contains definitions of trusted jobs
# Changes to this job require a special procedure, because they can
# not be tested before landing, and if they are faulty, they will
# break all jobs, meaning subsequent corrections will not be able to
# land. To make a change:
#
# 1) Ensure that base-test and its playbooks are identical to base.
# 2) Make the change to base-test and/or its playbooks.
# 3) Merge the change from step 2. No jobs normally use base-test, so
# this is safe.
# 4) Propose a change to a job to reparent it to base-test. Choose a
# job which will exercise whatever you are changing. The
# "unittests" job in zuul-jobs is a good choice. Use [DNM] in the
# commit subject so that people know not to merge the change. Set
# it to "Work in progress" so people don't review it.
# 5) Once test results arrive for the change in step 2, make a change
# which copies the job and/or playbooks of base-test to base. In
# the commit message, link to (without using Depends-On:) the
# change from step 4 so reviewers can see the test results.
# 6) Once the change in step 5 merges, abandon the change from step 4.
- secret:
name: site_logs
data:
fqdn: logs.openstack.org
path: /srv/static/logs
ssh_known_hosts: |
logs.openstack.org,23.253.108.137,2001:4800:7817:104:be76:4eff:fe05:dbee ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDcvLuGLagUAZfc0BThLus8ufSPCrIhDtG0BdXvhblJjvIbkuELD3dRWRZVSYZAdzGZRY3t6vTAcguTrkbQg5ngXfdfF+OKPkaH8DiZwAX/1g/iRXhInkZTGBVqHo9pLAMeNNwviSy2JjpTqdD6fLEkHwW+uw4E2YZhYivctTSbOepMkzAtFV0w5cpyBzjAT/Hax2x5un6es8R0Iw3AAnUmtapn5e5NCrg2rPNpd0nve84wUavvbC2DeGDOZQdnIahwo60Sder5ZE/x6cG39bkSDdgFQArAzrNrH6BHmNGjfFPpnGmfc7P8gQwDPtMf02HvKapqATXpIxdbSGimWLL
ssh_username: jenkins
ssh_private_key: !encrypted/pkcs1-oaep
- t9SCvfU4po36HYV0yCxivgaDF+L6BQVUGramqW3dgARxP+Mdl51h1+K/8EdNke0wzfDWX
tdVL6Vsh4D5/evfLuBgeILjXT/pzozfhDksjz78TiWBnFQyiC3FHwVB6tZ9903fIiltw5
aXg9AB3iYxSE/XQUKU3ThCt7zDJ0FoTrASVlKWaGeeMUiLBSaXaNrRTEFWyUJn7OU3nrj
646ac7QJnkZ5j/kQbKDdWF73tCrL69fOoHHZtc0QbnizbBRjdVyECktVy3jvYfIAEdsKW
Apg1HCQBJETe64PQR1OKv18sC6MdfSVP//8mpOAMdVeJzfNqkk83V1IBWHWTQgIAAyt/4
wB0aXUjX2rwMkInJfO6g2b+tMUajqEntib6IRKKXMb7/kS7ZcXwDkMj6bxBmnKgMLSx89
+fhBnYLoaNv9keBlDLtGc62glO3B9TxcxNzOFuBp0mLPR28v6DXBn0uXJwzdqXf1WAUsQ
m6BKVE34J99vuzHFDn7J0ov/biZtJLAsD6q0enBm0nJQPuXfrW0c/jcUO4D+SjStBo/t+
ZLMzzJvoygXTBFkiDX+6icIzLJMbpS8rBrGj+NbE+k1Lzni9Gq9Wo2xgDnGPwWDD97eup
H3cCIfhcFCP9m9YINLxxsJzpK8+Xss7LNqN8NbEbLPAbDH7b+rqIjoBPEAfVPM=
- C/Oz2r1fTYChvAbFpOdCF7+ZmEzSDYphP7fY/ENTOlvhq98QS3fGxRqj+oNEEppnM1oS1
Cc/bR3kzSqgMK629H0qVVqJhR0ffNT6ip6CIP2BkAaqT/6yUY5tp0BjZyC+O7tV6QtWkq
gj6k/cJcgT7JKMLSN4zjdO1A9qeLpjc9y98lArIeYXFvJHpXC9J8Vj8Fd+ODhH/YUUEkQ
nqCXcBTd2k1RFEWvCVRN7tKkiuAa4HPPmj+In9TKw3j2grn3LMmkUrQn5G7bWyuzQGp2u
2pVwvYNSEKxJiMMA0pTNLDMKaA5kvCQsQdt61FVN3AYZyCEbXq/6Is+JKoiZjBeyfUurB
btEoPNpjVmPQysCrvakSfbMi+Pn3jrZToxRNC30r1LWdHfKo0ovVRN0CEfce3suRu7uP8
BXH7Ow4sYKF5FLjzwzCO6VuoDg+SrfjbBwnzoySIsB3CXXieMUj+0ytfG1FBmKg2IiLQ7
Eaz+G4gCMe+1dMG87cKmizz7vC21ZFyeF3C2jBmXMMRvFgLCphHZOPfUOcy2yCPPFYmsg
2DBxx2VrvcPljTW6woVbb4Kxrd7+2TRbT9mzWDQDDdKGveIqUnEURGacJ+WRc8ZlBpFwN
cmwbJal3VSo0sB/X25ZNnF7Y7JHrXI6a3s/ck2ppid+2h1sk1oE6br/DRjYCN4=
- k8yssVEnQr58u8krETfjnByeO6UmQL7+JfXSYHI79z9n3Fp3nIRrFoH177d47iHtcYxyP
8IsQD2HMIGuRhyKZk5ruYwod/yeXZBwBcs7YSsof0U5gJ4gh6gw+bLQamKEaI4smq+xQA
UxxoHDw5m96+VUBeLdnXDFkq0qXiWOMmrCnVGgnDeuPZfyDbu8ILZi6c4WUFwj5o0oqRZ
pWEls8IfULjBEDMfbWhMrUh7zKurUwDXycmTAv4PriUdMdoMacqz/brxZZKC07+mzFiMj
iJvwV6STxATXy78+wWrM7MReoGownI0M0DKh07w/DEG000NTQnRz42DbwGbQQb8ugj4ee
1sB3+pz3udnwffREtht2uf2C48dHFqMOKeGNV3MJv8Z93H6rpgdpuySZwXC3iL2ga8m4I
U8ypFoCXXR5rHRqAL8xmuUVoavYC4XLPN1QvKueZnQW5XntZxXH/lSe9OnEo6SVya4v8p
CEQ6+XIWQCKIFPXxFM+KCoh7c8FASmJ7Tw1WLw+DNdSKL8kewk0Z2FvkR6bTzzcKT3RCf
/xM/+N674GhkYRFCMsQxrT9e6cfB2FRbBrxR1GJQQrS9KHPGn7dgKNN4/0snbtypekhjl
7oDENP6sbflXAo3Zeuq/XlvW0uobBqdI6bbkdMISAd779hVT5eQWvftwozrjHI=
- VjHYrglFpBi8Apnb64NYiblBANVDC0tXgAOzC7/NhcZ9Vc4rI7oRPfc48hrxjFlC+Uvtg
yI9cwu9y4FDDgGQ6qLovzP/Dvcwoga0YOZ7RYxdsT7N0/okRlWPRyj2h/7nlhrIxwK8bN
xRi7t/JniQkMrWiDckgw0YflLboMYQg8ShtCy1bZL1m0ISuBbodeswOLTiKFk2IG3R58h
Xylmgi2iM1md5ZeM9PhyLd8DrhuuJiKvhIiszdQNJN5Gg2CymYBveMfglE9r/10qgOM21
3UC37hSArn7WTu9Rwbo9bdNVePNik/x2O3fgMGND6ySX9vG8npPjOaomTGpds/z7DUn6F
0B4RWDoYDD57BHviUSYDDEbfpNS6dk/K4RpArjpS7ZZcUIok5sXSV18zSI8Gaa32SKU59
MdHuBtGW6p6kUTnuMSNCVsKGNOvjHsfnWFomUddEwhNFJW+tangCSkNaTQq/Yaf394lw8
nOsautk56uoiZPhSzdBpR9s8z0z1z0eGzdeBWyV+IFF/UJCftDiOSu0zA28RgDIwIg690
jVFWkZZRprDU6/5zgZPTLHOfz00IoMbGBKWSfvuOhF5l6VpSC3JVvcRd6/bivUq/1XkzP
uMv41vSFc4Kac1KmgAi96zglyRkzQgYVtLVNYyKbuLhVfx4U34mal/05sU3/MI=
- secret:
name: opendev-zuul-docs
data:
keytab: !encrypted/pkcs1-oaep
- nFyEj6IMXdp7UZwOQz9qAi6IlcKILCCzUqoNbchto7g2Uv8D/ZGdDU3DT+KV6ziWtv2d/
WBXQFjsCjE66SXpAUvtcGxj/rVqFGlayOb2WbOeo4+WvrFusNycVIs55R7I0vYyFzidZl
FVNP3+5Uv8N54mhFEVMudZdBO8aCwNftea5A2lLdZL/bnK071bzCKNYZAuDI/2j1VOsKx
JT346KdP3vqmyokqo+OxFE6QfbFHLTMgZigId3bkK01lpI0TBY2Wiv4rF76ErQWe/eghB
yOwrO1Oh2kkHADsrjrJ3rKGI8ZkWpgYIxKExXH9IAedbYaWhz7unvVrjUVjR/QIySv/u7
dENCrZhx2zd429eOjUHD+NmHisWoOQnvOVfiYBFbEPL9uAC+ek7fDxB3/9z3ok6KPv9f8
XLMNS63cQgPjYJP6kOqjrV/FXLl30SS3ikV0wVI2ErqYn3R2ukOccKJaF4uV9HCf+/mKt
0Uz89b0sUTzL5JkFYz/PhdqRVGwjjRNYahQb+QzWkxw/AgNS8Pdl/ijffx5DXAY5oKnJD
Jpyp6oXA+W+qefPPqQlxa9EbP3emwVf/HWUdCjlCw+GjDz730P94Xd8ie6KiGq3ywYtFn
EuJvheVCAYjlq4lwshfNIysNA8WePvAze9T5DpVl1MayR7b1KC+R/7wM6xUCbw=
service_name: service/opendev-zuul@OPENSTACK.ORG
docs_master_path: "/afs/.openstack.org/project/opendev.org/docs/{{ zuul.project.name }}/latest"
docs_branch_path: "/afs/.openstack.org/project/opendev.org/docs/{{ zuul.project.name }}/{{ zuul.branch | default('_error') }}"
docs_tag_path: "/afs/.openstack.org/project/opendev.org/docs/{{ zuul.project.name }}/{{ zuul.tag | default('_error') }}"
docs_redirect_path: "/afs/.openstack.org/project/opendev.org/docs/{{ zuul.project.name }}/.htaccess"
docs_redirect_content: "RedirectMatch 302 ^/{{ zuul.project.name }}/?$ /{{ zuul.project.name }}/latest/"
- secret:
name: opendev-pypi
data:
username: openstackci
password: !encrypted/pkcs1-oaep
- GKuFExWG2LM9nDqRn/lR2sBYzGwxsmBPI7lVCazB6mHjczmrYSgJnTsNGxGz872lgJdZg
GzxiqdrW5Wbof86PY6rL9wzFvrCPY7i3JhdwhyN9/V2GEKN9KzsFFwjtbSw0Ac6lRWzBA
/tWHERUHU1pcNG65IuSgjhSs7oxgLE5Y6JTvrbfFW9WviPfsnttgMFUZmgvHOShavy5Es
3T5TKnq2Seb9yERI2VdfBgFt/0yfnDgqqRG6vm60IDy7l9xTmJgsUj0dbil9mXv5YiuNO
Us2PcrfjXj1fMmmJYDY6QXQ3/AEswzKiDEtsS4XFUkxJdJgrZTvu61OAPajKw5r9imWhN
os5ZMTT720pwCnzucpDjsaECHGKcKQ8fCFFShAGgmP+8xAQmlpsLap+j5gY9EG3sAAAoE
9v8DzT8DR+ROfddDP02sA8xAsfdML/nG58udvAlvJj5akjZVKR8NMCFSw0//qbIVZqxI3
PBw3D384HOUeCJByI7Mqq9OtEpmHaytEkzrjHhhC/VaTTO53zgHyc6tMhwKgB8PEXMNeP
upg/65SBgTFbwmV68TpUyi9VspCV5BvRPBmR/ivNKDOE34yfx8KIa1xbYFKpucInR1c/K
YsOYcrXLlQDSWcn1brel5/LIh0BUvfTThCIOUpgw3thdBXUvy2gR6INgfQwMXo=
- job:
name: base
parent: null
abstract: true
description: |
The base job for OpenDev's installation of Zuul.
All jobs ultimately inherit from this. It runs a pre-playbook
which copies all of the job's prepared git repos on to all of
the nodes in the nodeset. It runs a post-playbook which copies
all of the files in the logs/ subdirectory of the executor
work directory to the logserver.
It also sets default timeout and nodeset values (which may be
overidden).
Responds to these variables:
.. zuul:jobvar:: base_serial
:default: Omitted
This sets the serial keyword in the pre and post playbooks
which can be an integer or percentage.
See ansible documentation for more information:
http://docs.ansible.com/ansible/latest/playbooks_delegation.html
pre-run: playbooks/base/pre.yaml
post-run:
- playbooks/base/post.yaml
- playbooks/base/post-logs.yaml
roles:
- zuul: openstack-infra/zuul-jobs
vars:
ara_report_type: database
ara_report_path: ara-report
timeout: 1800
post-timeout: 1800
nodeset:
nodes:
- name: ubuntu-bionic
label: ubuntu-bionic
secrets:
- site_logs
# See the procedure described above "base" before making changes to
# this job.
- job:
name: base-test
parent: null
description: |
A job to test changes to the base job without disturbing the
main job in production. Not for general use.
pre-run: playbooks/base-test/pre.yaml
post-run:
- playbooks/base-test/post.yaml
- playbooks/base-test/post-logs.yaml
roles:
- zuul: openstack-infra/zuul-jobs
timeout: 1800
post-timeout: 1800
vars:
ara_report_type: database
ara_report_path: ara-report
nodeset:
nodes:
- name: ubuntu-bionic
label: ubuntu-bionic
secrets:
- site_logs
- secret:
name: opendev-intermediate-registry
data:
host: insecure-ci-registry.opendev.org
port: 5000
username: zuul
password: !encrypted/pkcs1-oaep
- Y38es0iMk5vIGNZ9/FQtSb65hUqAvfduUV3pPnhURbMbEMuZpPiKRSxRaGWOVOZ0VcxaP
0eVYUSIHm+1n3+FK10ivFCl+EzanyFL70vleUxqHcN5dwTuevmB9kNp9FH8K45OKRvd1g
t7cpjMfbDV2iFik1uUkevbLzJZI+efXI0KLwCUYFifEWcl2exrqw8mudbmjjfxe0Prz11
EBxMBxCjLi3WEVvrquB76jW7p+ifgKJQc4FqUjzmLMI2xOeD1s4f+23InJOoRHKNC2lZu
/N2WSHQWxkebZnavjQTlshlBygD3etgkkYEjand9vcwWqTB0xnDagEUcrjl0axKJmPzXb
fGeyHrqld+IDaGxZP+JHcCZS5RNfXUOUt97Kgs9yzBtLwS+Lp4mqXXHvH1N17WFrT8YTD
cNxiFwR/wuq1g7AZWs0ej7rMBDF2rDnVV6/+8RWlqIhIjtCm4C8IsX/vm2/VsLTuNWdAM
JepYSbDvSQ5X55Ed3cZlGk+iPbfNFPb+EMIj3P7bxUjErQeT/hAhD6uKipSnisz+L6+RI
Ry8sLIVUbzLpIJKfcvo6xQCnepVdkF9dZET3prfnCf40MjCGeAITvgg1WcGX+yTiSQajr
oNz3bbxNeb2+MOucogQBwiSUnRPhpk2e+oMBVXGvDBjaHG1W0xakwMgQ9fIspw=
- job:
name: opendev-buildset-registry
description: |
Starts a buildset registry which interacts with the intermediate
CI registry to share speculative container images between
projects.
Configure any jobs which require the use of a buildset registry
to depend on this job using the "dependencies" job attribute.
This job will pause after starting the registry so that it is
available to any jobs which depend on it. Once all such jobs
are complete, this job will finish.
pre-run: playbooks/buildset-registry/pre.yaml
run: playbooks/buildset-registry/run.yaml
post-run: playbooks/buildset-registry/post.yaml
secrets:
- secret: opendev-intermediate-registry
name: intermediate_registry
requires: docker-image
- job:
name: opendev-build-docker-image
parent: opendev-buildset-registry
description: |
Starts a buildset registry (if one has not already been started,
e.g., by invoking :zuul:job:`opendev-buildset-registry` and
specifying it as a dependency) and builds one or more docker
images.
Analog of build-docker-image job, but with a buildset registry.
This job will pause after starting the registry so that it is
available to any jobs which depend on it. Once all such jobs
are complete, this job will finish.
.. include:: ../../playbooks/docker-image/README.rst
run: playbooks/docker-image/run.yaml
provides: docker-image
- job:
name: opendev-upload-docker-image
parent: opendev-build-docker-image
description: |
Starts a buildset registry and builds and uploads one or more
docker images to docker.io.
Analog of upload-docker-image job, but with a buildset registry.
.. include:: ../../playbooks/docker-image/README.rst
.. include:: ../../playbooks/docker-image/credentials.rst
post-run: playbooks/docker-image/upload.yaml
- job:
name: opendev-promote-docker-image
parent: promote-docker-image
description: |
Retag a previously-uploaded docker image.
Analog of promote-docker-image job.
.. include:: ../../playbooks/docker-image/README.rst
.. include:: ../../playbooks/docker-image/credentials.rst
- job:
name: opendev-tox-docs
# This is not parented to tox-docs because the post playbook
# differs.
description: |
Build documentation with "tox".
Uses tox with the ``docs`` environment.
vars:
tox_envlist: docs
bindep_profile: compile doc
pre-run: playbooks/tox-docs/pre.yaml
run: playbooks/tox-docs/run.yaml
post-run: playbooks/tox-docs/post.yaml
success-url: docs/
- job:
name: opendev-publish-tox-docs-base
# This is not parented to opendev-tox-docs because the post
# playbook differs.
description: |
Publish a ref-based documentation build.
Use this in the tag or release pipelines to publish a build
based on a newly-created tag.
This is an abstract job intended to be inherited from in an
OpenDev tenant and an appropriate secret added.
.. zuul:jobvar:: afs
:type: dict
This is expected to be a Zuul Secret with these keys:
.. zuul:jobvar:: keytab
The AFS keytab for the service principal.
.. zuul:jobvar:: service_name
The name of the service princpal.
.. zuul:jobvar:: docs_master_path
The full docs publication path to use if the job is run on
the master branch.
.. zuul:jobvar:: docs_branch_path
The full docs publication path to use if the job is run on
any other branch.
.. zuul:jobvar:: docs_tag_path
The full docs publication path to use if the job is run on
a tag.
abstract: True
vars:
tox_envlist: docs
bindep_profile: compile doc
pre-run: playbooks/tox-docs/pre.yaml
run: playbooks/tox-docs/run.yaml
post-run:
- playbooks/tox-docs/post.yaml
- playbooks/tox-docs/publish.yaml
- job:
name: opendev-publish-tox-docs
parent: opendev-publish-tox-docs-base
description: |
Publish a ref-based documentation build.
Use this in the tag or release pipelines to publish a build
based on a newly-created tag.
post-run: playbooks/tox-docs/publish.yaml
secrets:
- secret: opendev-zuul-docs
name: afs
pass-to-parent: true
- job:
name: opendev-promote-docs-base
description: |
Publish a previously built branch-tip documentation tarball.
Use this in the promote pipeline to publish a branch tip tarball
built in the gate pipeline.
This is an abstract job intended to be inherited from in an
OpenDev tenant and an appropriate secret added.
.. zuul:jobvar:: afs
:type: dict
This is expected to be a Zuul Secret with these keys:
.. zuul:jobvar:: keytab
The AFS keytab for the service principal.
.. zuul:jobvar:: service_name
The name of the service princpal.
.. zuul:jobvar:: docs_master_path
The full docs publication path to use if the job is run on
the master branch.
.. zuul:jobvar:: docs_branch_path
The full docs publication path to use if the job is run on
any other branch.
.. zuul:jobvar:: docs_tag_path
The full docs publication path to use if the job is run on
a tag.
.. zuul:jobvar:: docs_redirect_path
If this variable is present, a .htaccess redirect will be
created at this path when the job is run on the master
branch. For example, it can be used to redirect "project/"
to "project/latest".
.. zuul:jobvar:: docs_redirect_content
The contents of the .htaccess file in docs_redirect_path.
.. zuul:jobvar:: download_artifact_job
The name of the job which built the docs artifact which this
job should download and promote.
abstract: True
run: playbooks/docs/promote.yaml
nodeset:
nodes: []
- job:
name: opendev-promote-docs
parent: opendev-promote-docs-base
description: |
Publish a previously built branch-tip documentation tarball.
Use this in the promote pipeline to publish a branch tip tarball
built in the gate pipeline.
vars:
download_artifact_job: opendev-tox-docs
secrets:
- secret: opendev-zuul-docs
name: afs
pass-to-parent: true
- job:
name: opendev-release-python
description: Release python tarballs / wheels to pypi.
pre-run: playbooks/release-python/pre.yaml
run: playbooks/release-python/run.yaml
post-run: playbooks/release-python/post.yaml
secrets:
- secret: opendev-pypi
name: pypi_info
- project:
check:
jobs:
- opendev-tox-docs
- openstack-zuul-jobs-linters
gate:
jobs:
- opendev-tox-docs
- openstack-zuul-jobs-linters
promote:
jobs:
- opendev-promote-docs
View Git Blame Copy Permalink