19247 Commits

Author SHA1 Message Date
Jeremy Stanley
245b820102 Allow all Ubuntu releases to use our OpenAFS PPA
We want to go ahead and install a custom OpenAFS package set on the
latest Ubuntu LTS too, so just drop the version restriction on that.
Also, the newer version of OpenAFS in our PPA should build properly on
ARM64 so we make the noble arm64 roles job voting again.

Change-Id: Iaa4ab95476dc7601ce45b25cb045ee990cdf1039
2024-11-22 08:53:08 -08:00
Clark Boylan
b5148387ba Upgrade and reboot test nodes before openafs installation
CentOS ARM in particular complains bitterly if we try to upgrade openafs
on it and the kernel it is running is stale compared to packges in the
mirrors. This happens if our images have lagged behind upstream package
updates. To mitigate this we upgrade and reboot the system prior to
testing openafs installation.

To make arm64 reboots onto the new kernel reliable we manually run
grubby --set-default to explicitly set the default kernel to boot to the
newly installed kernel. Then we remove the old kernel with grubby and
finally generate a new grub config. This is necessary likely due to:

  https://bugzilla.redhat.com/show_bug.cgi?id=2032680

Note we also do the reboot (but not grub(by) dance) on Debuntu for
symmetry.

Change-Id: If87a0b1d7dc063ac9122d85f65b6fe1c129d2052
2024-11-21 17:51:56 -08:00
James E. Blair
660a906c09 Add vexxhost connection to zuul-launcher
Thiss adds vexxhost as a second openstack cloud connection to the
zuul launcher.  This will help us benchmark image times for "raw"
images.

Change-Id: Ifdab5abb9915f384482eda844194f77e8d1b80ca
2024-11-16 10:35:04 -08:00
James E. Blair
d1dcda1410 Add basic docs on updating the OpenAFS ppa
This adds documentation that should at least remind us that we have
a repo with the .deb files for our openafs packages and roughly
how to go about updating them.

Change-Id: I4e52be22eef0cf9daf8e3d92d3df04fa3a523c2f
2024-11-13 15:12:15 -08:00
Zuul
1290609bd5 Merge "Update test arm64 mirror node" 2024-11-08 06:45:00 +00:00
Clark Boylan
f10050a81f Run letsencrypt twice in system-config-run-letsencrypt
We want to ensure multiple runs of acme.sh and the entire letsencrypt
machinery produce expected results so run the playbook twice in this
job.

Change-Id: I24afd27a07576bd8e30b4466d338310bdff89d49
2024-11-06 09:32:25 -08:00
Clark Boylan
2066403ed1 Only update acme.sh if necessary
We install acme.sh into /opt on our servers. Some of our servers rely on
/opt for data intensive activities and we can run out of disk space on
/opt/. When our daily Ansible runs fire and hit servers in this
situation we end up with corrupted acme.sh repos on those servers. Then
acme.sh roles fail.

Avoid this problem by only updating the git repo for acme.sh if it isn't
already up to date on the versions we expect. We can still fill the disk
but this won't affect acme.sh only server operations that rely on /opt
disk space.

This is an alternative to https://review.opendev.org/c/opendev/system-config/+/934247
which will try to force updates to occur regardless of git repo
corruption.

Change-Id: Ib0ad55de833a2c2d9e8cacec0493b8422e486789
2024-11-06 09:32:21 -08:00
Zuul
c94c6f8f82 Merge "Add meetbot and statusbot to openstack-eventlet-removal" 2024-11-06 08:30:40 +00:00
Goutham Pacha Ravi
26748a3dba Fix doc for including statusbot in channels
The statusbot list is in a different place than
we've documented since we merged [1]

[1] https://review.opendev.org/c/opendev/system-config/+/795007

Change-Id: I1958bce8fe77df8def4b40deed563aa5ef5b0b39
2024-11-05 14:24:55 -08:00
Goutham Pacha Ravi
24aaa9403a Add meetbot and statusbot to openstack-eventlet-removal
There are team meetings hosted on this channel.

Change-Id: I9c1b343b38942c63a633a3f5334e4d7285d6ff43
2024-11-05 17:58:34 +00:00
Takashi Kajinami
689b14e6da Fix missing puppet8 component
This is follow-up of c28f5682fc1009384c7541604ccc6afa6467adf3 and
fixes the missing component to ensure all puppet 8 packages are synced.

Change-Id: Ic9e202cdc5038857d22ff9ec6a2b85164e85f9d3
2024-11-03 00:47:36 +09:00
Zuul
cb502bd292 Merge "Add documentation about Django/Mailman super user" 2024-11-01 16:22:29 +00:00
Zuul
08732cefe6 Merge "Increase size limits for some Gerrit caches" 2024-10-30 19:15:29 +00:00
Jeremy Stanley
c2ba2b3177 Add documentation about Django/Mailman super user
The Django admin account is also a super user in Hyperkitty and
Postorius, providing a backdoor for systems administrators in a
pinch.

Change-Id: Ic2a8763186c8b6f689c7e8c2f016429944aad753
2024-10-29 16:42:50 +00:00
Clark Boylan
deb04db7f4 Update etherpad to v2.2.6
This appears to be a small update compared to other recent updates. In
particular the dockerfile has not changed and neither have the
settings.json files. The changelog can be found here:

  https://github.com/ether/etherpad-lite/blob/v2.2.6/CHANGELOG.md

The only notable update there is that pads can be deleted by their
creators. I suspect this is a noop for us since we don't do
authenticated access, but we should check with a held node.

Change-Id: I4e935828912d004eb7694e981926557db8b26e56
2024-10-28 16:13:36 -07:00
Clark Boylan
44f9246de7 Update test arm64 mirror node
When we bumped things up previously there was a bug in openafs that
prevented it from being built by dkms on newer platforms. This has since
been fixed in Debian and hopefully Noble as well. Bump up the node to
Noble to find out.

Change-Id: I954e02935ece1d348460316dcfea028452c19413
2024-10-23 13:58:23 -07:00
Zuul
2a6a0a4b3d Merge "Mirror puppetlabs packages for Ubuntu Noble" 2024-10-23 00:10:32 +00:00
Zuul
912f373c44 Merge "Add some documentation about the mirror volume config script" 2024-10-22 17:18:24 +00:00
Clark Boylan
046aad9276 Increase size limits for some Gerrit caches
When we restarted Gerrit recently there were a number of caches that
were over their default max sizes so were pruned. Gerrit prunes daily at
0100 or when restarted. This gives us a good indication for which caches
are currently configured to be too small for typically operation as we
restarted several hours before 0100.

All of the logged cache pruning can be found in this paste [0]. Many of
these caches were floating around their configured maximum and we leave
them alone. However four related caches are well above their default max
which is a good indication we need to increase their sizes. The four are
identified below with their documented purpose/function from the
upstream docs [1]:

 * cache "git_modified_files"
   Each item caches the list of git modified files between two git trees
   corresponding to two different commits. This cache does not read the
   actual file contents nor does it include the edits (modified regions)
   of the files.

 * cache "modified_files"
   Each item caches the list of modified files between two commits. This
   cache is similar to the git_modified_files cache but performs extra
   logic including filtering out files that are untouched by both
   commits because they were purely modified between the parent commits.

 * cache "git_file_diff"
   Each item caches the pure git diff between two git trees for a
   specific file path. The diff includes all the file attributes
   (old/new paths, change/patch types) as well as the list of edits
   corresponding to the modified regions in the file.

 * cache "gerrit_file_diff"
   Each item caches the diff between two git commits for a specific file
   path. This cache is similar to the git_file_diff cache but performs
   extra logic including identifying the edits that are due to rebase.
   The diff for the "commit message" and "merge list" can also be
   requested from this cache.

   Entries in this cache are relatively large, so memoryLimit is an
   estimate in bytes of memory used. Administrators should try to target
   cache.diff.memoryLimit to fit all changes users will view in a 1 or 2
   day span. The same applies for other diff caches:
   "git_modified_files", "modified_files" and "git_file_diff".

The note at the end of cache "gerrit_file_diff" is what we use to
determine these new sizes though we're more conservative with the memory
limits (default of which is 10m for each of these caches) as memory is
more scarce than disk.

[0] https://paste.opendev.org/show/bk4pTIuQLCsWaF3dVVF7/
[1] https://gerrit-review.googlesource.com/Documentation/config-gerrit.html#cache

Change-Id: I521b53c130892fc2152586da1c4858ea4099479f
2024-10-18 13:52:28 -07:00
Zuul
440ddedfea Merge "Update Gerrit images to 3.9.7 and 3.10.2" 2024-10-18 18:30:24 +00:00
Zuul
6d64697ca6 Merge "Update Gitea to 1.22.3" 2024-10-18 17:27:01 +00:00
Zuul
252cb4695c Merge "Cleanup python 3.10 bullseye images" 2024-10-18 15:00:17 +00:00
Zuul
e3945d2c45 Merge "Add rabbitmq caching mirrors" 2024-10-18 11:13:15 +00:00
James E. Blair
fd47d15196 Add clouds.yaml to zuul schedulers
Similar to I127ba4a2cf33f7ffcecdf81acfc5c38db565da82 we now need
to add the nodepool clouds.yaml to the zuul schedulers.  When validating
and generating the config for the launchers, they need to access the
clouds.yaml file to learn some information about the cloud (eg, image
format).  Therefore, they must have the same clouds.yaml that the
launchers use.

Change-Id: I4a7076e66c9f8eca936ca3a7902537d427a99323
2024-10-14 11:32:49 -07:00
Zuul
4b85924ccd Merge "Remove linuxcontainers mirrors" 2024-10-14 03:25:54 +00:00
Zuul
0a5ec125fa Merge "Remove copr mirrors from the list" 2024-10-14 03:25:51 +00:00
Tony Breeds
0eec8291b1 Update Gerrit images to 3.9.7 and 3.10.2
There are new bugfix updates for both of the Gerrit images we are building.
Bump up to these new releases. As far as I can tell all plugins with fixed
tags have just been retagged (for both v3.9.x and v3.10.x).

Change-Id: Ie57b11a48919bd3fb0f27beaf8d92a99397bb10a
2024-10-14 13:18:15 +11:00
James E. Blair
90be36e513 Fix raxflex connection entry for zuul-launcher
The connection entry omitted the cloud name.

Change-Id: I76a95d167cbd12ac01a069885aa6c3c96227fcab
2024-10-10 13:30:24 -07:00
Clark Boylan
23e5bd3f6a Cleanup python 3.10 bullseye images
These images were kept around because zuul-operator was still relying on
them. Now that zuul-operator has updated to bookworm [0] we no longer
need to build and keep these images up to date. Drop them as the extra
effort is unneeded.

[0] https://review.opendev.org/c/zuul/zuul-operator/+/896142

Change-Id: I8d9ee1e4fa2b33fa37f75b27301e7e9f5559510e
2024-10-10 09:50:27 -07:00
James E. Blair
439ba7bd3b Add some documentation about the mirror volume config script
And update the static page.

Change-Id: I13295f90c0f0792b0c3f55bfa55b9f2b82cf291b
2024-10-10 09:24:18 -07:00
James E. Blair
19cc385403 Install clouds.yaml on zuul-launcher
And configure it with a connection to rax-flex for testing.

Change-Id: I127ba4a2cf33f7ffcecdf81acfc5c38db565da82
2024-10-10 09:05:59 -07:00
James E. Blair
b255615c99 Configure zuul-launcher to use its logging config file
This file exists, but is not referenced in zuul.conf.

Change-Id: I8244dcffd43697a03b0f559ac63e2a185fb35537
2024-10-09 15:01:37 -07:00
James E. Blair
f3f3d8adee Remove zuul_ssh_private_key_contents from scheduler host vars
This is defined in the big zuul group (and used by the zuul role),
 so let's just rely on that definition.

Change-Id: I2c5b5d744a3506717d9c4ad15fa04344bb5890bd
2024-10-09 12:45:16 -07:00
Zuul
49d4eed605 Merge "Update Mailman containers to latest versions" 2024-10-09 19:36:58 +00:00
Zuul
550508440f Merge "Add an inventory entry for our first Zuul launcher" 2024-10-09 17:59:59 +00:00
Zuul
0e8bd76019 Merge "Drop more Xenial testing from system-config" 2024-10-09 17:35:19 +00:00
Jeremy Stanley
bd092b9552 Add an inventory entry for our first Zuul launcher
This adds zl01.opendev.org to the Ansible inventory, as an initial
Zuul launcher server.

Depends-On: https://review.opendev.org/931917
Depends-On: https://review.opendev.org/922680
Change-Id: I561a9873bc5a4636f8d717727816cb342b6d7aea
2024-10-09 17:09:46 +00:00
Clark Boylan
33ce67a234 Update Gitea to 1.22.3
This is a bugfix release with a number of improvements. The full
changelog can be found here:

  https://github.com/go-gitea/gitea/blob/v1.22.3/CHANGELOG.md

There were no template changes in the template we override compared to
v1.22.2 (the current deployed version). No other build changes appear to
exist either (same golang version etc).

Change-Id: Id8d7057596e2a067534076507ffa64e4a85ac5e3
2024-10-09 08:02:48 -07:00
Dmitriy Rabotyagov
b84ea6ee91 Add rabbitmq caching mirrors
This adds RabbitMQ/Erlang caching mirror to the list, as remote
connections tend to fail due to Timeouts lately. Below are just some
examples of failed jobs due to that

[1] https://review.opendev.org/c/openstack/openstack-ansible/+/931612
[2] https://review.opendev.org/c/openstack/openstack-ansible-os_octavia/+/930469
[3] https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/931801
[4] https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/931802

Change-Id: I1831b8fb8a7b5882953a0a66bf0136345739cad4
2024-10-09 12:36:52 +02:00
Dmitriy Rabotyagov
ee280a229a Remove linuxcontainers mirrors
linuxcontainers mirror is not used by any maintained branch, thus can
be safely removed from mirrors

Change-Id: I6e223325af7d9cca55c76cc71c93ea20ad89ebbe
2024-10-09 10:25:54 +02:00
Dmitriy Rabotyagov
2bb29286ff Remove copr mirrors from the list
It seems these mirrors/repos are not used anywhere as of today and
can be safely removed from the list.

Change-Id: I4aee98aa6c159bb4617b0b584dee943f1070303c
2024-10-09 10:24:01 +02:00
Takashi Kajinami
c28f5682fc Mirror puppetlabs packages for Ubuntu Noble
Puppet OpenStack project is now planning to start testing Ubuntu 24.04.

Change-Id: I57c870b56c128c774df798534e20137894735f96
2024-10-08 00:16:18 +09:00
Zuul
8fc9da4578 Merge "Explicitly down Jitsi-Meet containers on upgrade" 2024-10-04 18:52:33 +00:00
Zuul
60b3e322ff Merge "reprepro: mirror Ubuntu UCA Dalmatian for Ubuntu Noble" 2024-10-04 14:19:33 +00:00
Jeremy Stanley
24acb98c8d Update Mailman containers to latest versions
This upgrades our images to Alpine 3.20, Django 4.2, Mailman 3.3.10,
Postorius 1.3.13, Hyperkitty 1.3.12 and django-mailman3 1.3.15.
Files are re-synced with upstream (either container or project)
files, with versions and any alterations noted.

Change-Id: I78d37c0635d38ecfc1d1143a69892fe8d8685214
2024-10-02 16:37:08 +00:00
Jeremy Stanley
4f0bed6998 Explicitly down Jitsi-Meet containers on upgrade
We've been running into problems with Jitsi-Meet processes not
restarting cleanly if we only docker-compose up after image pulls.
Instead, check for whether the pull resulted in new image downloads
and then cleanly down and up the containers, which seems to address
the problem based on manual testing on the servers.

Change-Id: Id551767e72102a4b8667aa9dacc81755f332a278
2024-09-26 17:52:29 +00:00
Zuul
ab28acf052 Merge "Update Gitea to v1.22.2" 2024-09-25 19:27:05 +00:00
Jonathan Rosser
e701af7716 reprepro: mirror Ubuntu UCA Dalmatian for Ubuntu Noble
Change-Id: I3a692457af110c00f979df8c7c7288f012899539
2024-09-24 12:47:52 +00:00
Clark Boylan
1fd4deb76d Update Etherpad to v2.2.5
After the 2.2.4 release we upgraded to a development commit between
2.2.4 and the future (at that time non existant) 2.2.5 release to fix
integration between meetpad and etherpad. Now there is a proper 2.2.5
and we should update to get off the dev commit.

This release fixes a number of bugs, updates dependencies, and adds
proper swagger documentation for the API. The "complete" changelog can
be seen here:
  https://github.com/ether/etherpad-lite/blob/v2.2.5/CHANGELOG.md

Note that I don't believe the API has changed they are merely
documenting it properly using swagger. Our testing should confirm.
To expose the new swagger documentation we do add /api-docs/ and
/api-docs.json to our proxy exclusion list.

We also update our settings.json files to sync with upstream. This pulls
in a new `updateServer` key value pair to set the location that should
be checked by etherpad to determine if there is a newer version
available. I believe this behavior has existing for years they are just
now making it a bit more configurable. Unfortunately the way this value
is used I think we will do a local file lookup if we set the value to
"". I've stuck with the default since this shouldn't be a regression and
we can try to disable it later.

Change-Id: I73a09a0c79db18887cb1703c84f9aebae6f072eb
2024-09-23 10:29:15 -07:00
Clark Boylan
ddbfa8b162 Update Gitea to v1.22.2
We're running v1.22.1 currently and v1.22.2 fixes a number of bugs. The
full changelog can be seen here:
  https://github.com/go-gitea/gitea/blob/v1.22.2/CHANGELOG.md

The template files we override have not changed and neither has the
Dockerfile since we last upgraded. I expect this should be a fairly
straightforward bugfix upgrade. Reviewers please to read the changelog
to see if there is anything we should be concerned about.

Change-Id: I4983865c94429c4cbcb54329f0f83b2fb0f26404
2024-09-23 08:44:08 -07:00