58 Commits

Author SHA1 Message Date
Sergiy Markin
d2b1cd94db Airship testing gate updates
Change-Id: Icc9356fc9d21ee3411f731c43d5048f1b7c137ba
2025-01-13 21:48:51 +00:00
Sergiy Markin
fef241da50 Update calicoctl version in jammy image
Change-Id: Ib0d1b37219c76eb55d792f7a3640873011fba053
2024-09-25 21:53:05 +00:00
Sergiy Markin
b475486d96 Switch to py310 and new kubernetes 1.30.0
By switching to Python 3.10 we need to disable
focal and bionic jobs.

Change-Id: I4ecd3c633e4fc90fea1622beb2e822a0b4c87452
2024-09-23 15:18:21 +00:00
astebenkova
f543332804 [images] Upgrade distro version to Ubuntu Jammy
+ add python3-netifaces for successful pip packages compilation and
  orphan-sysvinit-scripts to install /etc/init.d/rsyslog script in Jammy
+ build specific image instead of all in airship-porthole-images-* jobs

Change-Id: I5c9b3bf81db95a03bd623d3c46f37f37bd595504
2024-05-20 18:18:59 +03:00
Sergiy Markin
30a7c4699b Updated dependences
This PS updates openstack modules versions to
Antelope 2023.1, ceph utilities version to Reef
18.2.1 and kubectl to 1.29.2

Change-Id: Ic8f5c0a7b03a33bcef667bb16291aea93ca3be3a
2024-03-04 22:13:01 +00:00
astebenkova
1c4248ae74 [zuul] Define run playbooks
Move the main playbooks to "run" section to allow
child jobs to include their own pre-run playbooks

Change-Id: Ie829a23d56df6a115fb3d06509f676aa94246b4d
2024-02-15 14:14:27 +02:00
Sergiy Markin
9993c54155 [deploy-env] Switch to kubeadm role
Change-Id: I269243ff6a8e4a9c8761aa518c149602d3cba8ef
2024-02-14 19:00:52 +00:00
astebenkova
755e8e5c6f [zuul] Move logs playbook to post-run
If testing playbooks fail, logs won't be collected.
We need to gather logs regardless of the job's result.
+ change StorageClassName to "standard"
Change-Id: I5eb5a5cef6e63df72a2ff8092d1d8fc756c9a692

Change-Id: I178291863c2c1b385825a51b5e16cd230180578d
2024-02-14 03:56:35 +00:00
astebenkova
7af0a01b89 [images] Install the latest rclone package
+ upgrade k8s and etcd patch versions
+ comment out bionic-related jobs since Portgresql
  doesn't suppport this distribution anymore
The rclone package from upstream repository is obsolete and contains a lot of golang vulnerabilities

Change-Id: I4f1577e7c00c4272ee5216e9f8c8b6ded98dab6f
2023-11-17 16:01:20 +02:00
Sergiy Markin
8c69a118df Add zuul gate variable
This PS adds zuul gate variable args.chdir to the jobs where it has been
missed. This variable helps to control relative path to the source code
folder in cases when  playbooks have been inherited.

Change-Id: Iea56ea4d00b9449c129afb34f4f91952fcc015c4
2023-06-26 16:43:37 +00:00
Sergiy Markin
04a8cb59b5 Porthole upgrades
This PS delivers the following changes:

- adds values_overrides folders into utility binaries charts for
  flexible testing the charts and images;
- values_overrides added to .helmignore files for every chart;
- added munch python package into bionic images for compartibility with
  newer openstack client dependences;
- improved Makefile clean section;
- uses treasuremap v1.9 deploy-k8s.sh script to deploy test k8s cluster;
- adds appropriate changes to deploy k8s script, setup apparmor;
- porthole utility charts deployment script have been enriched with and
  ability to control additional values from values_overrides folders of
  each chart;
- added env-variables.sh and get-values-overrides.sh to power up the
  changes describes in previous point;
- removed deploy.sh script for clarity of actions in playbooks;
- improved airship-porthole-gate-runner playbook;
- removed install-deps playbook that was used to pre-setup zuul gate;
- united make-unit-tests and make-feature-tests playbooks into one
  make-functional tests playbook;
- helm version has been bumped up to 3.11.1;
- improved helm_tk.sh script in order to solve issues with multiple
  charts build process;
- bumped up version of stable helm-toolkit used for charts build;
- airship-porthole-gate-runner playbook is now used as pre-run script
  for tox profiles gates and as main run playbook for functional tests
  gate;

Change-Id: I202c573ff85ed2b80191057b537a65a5f5d4600e
2023-06-23 18:48:26 +00:00
Sergiy Markin
5ead2930ae Restore bionic images
This PS restores building ans testing bionic images

Change-Id: Id88e1c096a8934e5974b9404cc2b0bfd748fc25f
2023-06-07 04:15:14 +00:00
Sergiy Markin
693f99363e Update porthole project
This PS performs the following changes:
- updates approach to freeze requirements.txt
- adds freeze tox profile
- upgrades helm to v3.9.4
- changes deployment scripts in accordance with new helm v3
- python code has been re-styled to pass pep8 tests
- added tox-docs zuul gate
- added tox-py38 zuul gate
- added tox-cover zuul gate
- added tox-pep8 zuul gate
- deprecated old unit-tests zuul gate
- added a dependency pre-run playbook to deliver zuul node setup needed
  for python tox gates to run unit tests
- added tox profiles for py38,pep8,docs and cover tests

Change-Id: I960326fb0ab8d98cc3f62ffa638286e4fdcbb7c7
2023-06-02 16:01:08 +00:00
Sergiy Markin
017902b5ce [quay] Fix for docker publish in Quay
This PS adds docker python module for ansible to be able to publish
images to Quay.

Change-Id: Id599e6a7536435b18d4dbae0d618c53a35dbfb2e
2023-03-17 23:47:20 +00:00
astebenkova
91d35fc47f [images] Upgrade distro version to Ubuntu Focal
Add focal-based docker recipes for porthole images. Besides, amend helm
charts and zuul variables related to distribution.

Change-Id: I1a6ac464a115a842dd7010fa8a28fbe0822b9598
2023-02-14 23:41:28 +00:00
Sergiy Markin
fa49a102c9 [dbutils] Increase timeout for ON_DEMAND_POD
This PS is intended to fix intermittent issue on some sites when
database backup ondemand job fails to detect ON_DEMAND_POD when it is
still creating.

Also this PS has fixes related to using focal as base build image.
tox.ini also fixed for compatibility with tox4. This PS also uplifts
latest stable HTK version and changes openstackutility version to
Yoga.

Change-Id: I36e55091adf7dc68758fb9b7145cfa34776522d6
2023-02-14 19:35:47 +00:00
Markin, Sergiy (sm515x)
1cb2ee3566 [CPID-354] Improve MariaDB Backup/Restore validation process
HTK has been updated to 0.2.48 version that includes local backup verification step before uploading to remote storage.

In mysqlclient-utility chart HTK is used to perform test backup/restore processes. This chart has been updated to match HTK changes.
The main difference that a mariadb verification server has to be running as a side container.

In etcdctl-utility chart .Values.manifests.secret_certificates value was set to False and a condition has been added to deployment-etcdctl-utility.yaml to avoid looking for etcdctl-etcd-secrets secret to mount.

Also in mysqlclient-utility in ensure_ondemand_pod_exists() function of dbutils.sh made the folloging adjustment: number of containers in *-ondemand pod in pod existence verification condition set to 2/2. This is because we added the second container to this pod with verification server.

Additionally, bumping up HTK version to 0.2.48 from a commit id obtained from merge of https://review.opendev.org/c/openstack/openstack-helm-infra/+/853027 and set proper commit id in this file: tools/helm_tk.sh

Change-Id: I9d3796500574da3582cf67d8aeeec8c796dd88dd
2022-09-10 14:55:40 -05:00
diwakar thyagaraj
80a2c07d52 Rename Zuul Test Job
Since this job runs both Unit and Feature tests, renaming.

Change-Id: Ic94aecf22ba0075f3a2803d96992929a7ec61b7a
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-08-19 17:45:28 +00:00
diwakar thyagaraj
36b5369a12 Reduce airship-porthole check jobs.
1) Zuul gates takes two hours to merge the change.
    This Reduces repeating of same Jobs.
 2) Consolidated deploy and apparmor scripts to Single Job.
 3) Consolidated unit and feature tests to Single Job.

Change-Id: I97eec8140a4e10fd1c70a0b732b3b3a28c45ab70
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-06-29 18:47:54 +00:00
Trung Thai
b0e8e0f478 Create an initial Automation Validation Testing framework
- Setting up a runtime environment
- Add to Zuul 'check' and 'gate' pipeline
- Validate concepts:
  1) List a few unit tests successfully
  2) Execute a few simple tests successfully (pass or fail)
  3) Generate docsbuild successfully
- Each UC will have their own test suite and test cases
and will be handled seperately
- add compute-utility deployment into the gate
- add logging collector of primary
- add false / positive test cases
- add unit-test and feature-test gates

Change-Id: I55a0dcf440e9694b041d5fb8eb75bd6f4adb8913
2020-06-12 13:43:58 -04:00
diwakar thyagaraj
942c19243b Enable CI for Porthole
1) Standardizing the usage of Deployment Scripts since porthole is cloned
to openstack-helm project as Per ReadMe as well.
https://review.opendev.org/gitweb?p=airship/porthole.git;a=blob;f=README.md;h=ec953ff18b5286213b8eee57eacbab0f2a331e79;hb=refs/heads/master#l30

2) Changed Apparmor Scripts to run on Demand to reduce resources
needed.

Change-Id: I5e682684cfd4bec70542f0951dbcf33852ad78b5
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-06-05 17:54:37 +00:00
diwakar thyagaraj
4a2d6deefa Enable Logs Output in Zuul for Deployment Scripts
This Fixes Logs for Deployment Validation.

Change-Id: I6cda8d0f0bd2d5e957abb8df9b512866f169ae59
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-03-04 18:24:16 +00:00
diwakar thyagaraj
756b81cc1a Fix helm Installation Script in Apparmor Scripts
Added helm dependecy update instead of directory logging
and using MAKE functionality. This way script supports
local deployments.


Change-Id: I40b916f21a11945f4e065983e863922c3f537bbf
Signed-off-by: diwakar thyagaraj <diwakar.chitoor.thyagaraj@att.com>
2020-03-03 13:51:40 +00:00
diwakar thyagaraj
d551da949d Postgresql UC Python and Ubuntu upgrade.
This Patchset updates ubuntu from xenial to Bionic,
     and Python from 2.0 to 3.7.

Change-Id: Id79cb21eec820f6505b0287616ed399c1898da4b
2019-12-17 18:12:47 +00:00
Zuul
31ec6ac551 Merge "Ceph-utility python and Ubuntu upgrade." 2019-12-17 17:45:59 +00:00
Rahul Khiyani
f2125faf4d Ceph-utility python and Ubuntu upgrade.
This Patchset updated Ubuntu to  Bionic
and Python from 2.0 to 3.0

Change-Id: I0e967fa7f7f4d4a5619489a9da79b6561da9c9cb
2019-12-16 14:44:09 +00:00
KAVVA, JAGAN MOHAN REDDY (jk330k)
04c1b9fab8 Etcdctl utility container image python and Ubuntu upgrades
Currently etcdctl utility container image is built using Ubuntu 16.04
so upgrading it to use Ubuntu Bionic and also upgrading Python 2.x to
3.x.

Change-Id: I24212878fcb004f7043d78b7cffb9364cb745e97
2019-12-16 14:31:16 +00:00
Zuul
d6b23767ba Merge "Enable runtime-default Apparmor Profile to Postgresql Utility Container." 2019-12-13 21:03:49 +00:00
Rahul Khiyani
72394708c9 Calicoctl-utility python and Ubuntu upgrade.
This Patchset updated Ubuntu to  Bionic
and Python from 2.0 to 3.0

Change-Id: Idd910fc74558ceb43ace48652e2c6bd6232aeb9e
2019-12-12 15:50:47 +00:00
diwakar thyagaraj
28bddef227 Enable runtime-default Apparmor Profile to MySqlClient Utility Container.
This also changes node to enabled to make it consistency across
 all nodes, and adds zuul gate jobs.

Change-Id: Idb8779d6b6d7f73d6450de59d5ed0575d91a1518
2019-12-11 20:14:34 +00:00
diwakar thyagaraj
0e87826b63 Enable runtime-default Apparmor Profile to Postgresql Utility Container.
This also takes care of Zuul gate Jobs.

Change-Id: I47ee4f62b4e3a6291453093a9f39593fe82897b5
2019-12-11 17:35:54 +00:00
Prateek Dodda
9c2038cb59 Compute UC python and Ubuntu upgrade.
This Patchset updated Ubuntu to  Bionic
and Python from 2.0 to 3.0.

Change-Id: Ife0b2041c6c5c8935eb11d9157f39a217b130ae1
2019-12-11 15:33:47 +00:00
Prateek Dodda
c8afd0480b Openstack UC python and Ubuntu upgrade.
This Patchset updated Ubuntu to  Bionic
and Python from 2.0 to 3.0.

Change-Id: Ia87bcb06a2544ed456bf7efebf151e53efe64711
2019-12-10 17:46:54 +00:00
diwakar thyagaraj
d67b5947e8 Enable runtime-default Apparmor to etcdctl-Utility Container.
Changed apparmor gate jobs to non-voting for sucessful Zuul merges.

 Change-Id: Ic23806baefdde651330029d78a73216d0850a5e9

Change-Id: I1f9e466838f0c762d5a623a0195134574a91bd24
2019-12-09 21:03:34 +00:00
Zuul
bca7e61d03 Merge "Enable runtime-default Apparmor Profile to Ceph Utility Container." 2019-12-06 21:31:07 +00:00
diwakar thyagaraj
32e8e36677 Mysqlclient UC Python and Ubuntu upgrade.
This Patchset updates from xenial to Bionic,
and Python from 2.0 to 3.0.

Change-Id: Ia3b75e51b685666e67b100925249c80e2343de4b
2019-12-06 19:36:53 +00:00
diwakar thyagaraj
0f4267b5c9 Enable runtime-default Apparmor Profile to Ceph Utility Container.
Change-Id: If814aba2ec5bd457f55e38ce5024922b72d32187
2019-12-06 19:05:06 +00:00
diwakar thyagaraj
7fcb7efe3d Enable runtime-default Apparmor Profile to Calicoctl Utility Container.
This also enables Container name to be set as static, as
profile is applied to container and Added zuul gate check to false.

Change-Id: I730c0d7850b14543d14e17c60f89cd52b11a0878
2019-12-04 15:17:49 +00:00
diwakar thyagaraj
d4d4d7f486 Enable runtime-default Apparmor Profile to Openstack-Utility Container.
This PS also adds Gate jobs to check docker default
apparmor for utility Containers.

Change-Id: I32fbf16291a4f4df06d0cc23c8235de0420a044a
2019-12-02 18:25:03 +00:00
diwakar thyagaraj
19e2fa9721 Update Calicoctl Utility Container to use Ubuntu-Xenial Image
This patchset also reverts nodeselector value for Utility label
 back to enabled and also adds repective gate jobs to build Xenial Image
 along with Installation Script and ReadMe file.

Change-Id: I4213c121f83dedb96b14c6d101f9ff2a6ae766f3
2019-11-20 20:34:17 +00:00
Phil Sphicas
13084e6407 Add xenial-based calicoctl-utility containers
The change adds new Dockerfiles to allow calicoctl-utility containers to
be built using ubuntu_xenial as a base, for consistency with the rest of
the utility containers.

Change-Id: Ib7cbb159a22e1a38645b747ad94dd30ebb25757a
2019-10-28 21:51:50 -07:00
Zuul
b7d00a795d Merge "[WIP] Change source of compute-utility images" 2019-10-14 21:34:29 +00:00
mm782t@att.com
cb5d6cfba3 [WIP] Change source of compute-utility images
Change value.yaml to use quay.io airshipit/porthole image

Cleaning up Dockerfile

Adding zuul gates to automate build and deploy images
to quay.io

Change-Id: I98278918bd5209b03981c1d26335bb4395de65b9
2019-10-14 13:01:18 -06:00
Prateek Dodda
6d24e2e45f Publish Image in quay.io. airshipit/porthole
and adding zuul gates

Change-Id: I2c58e60bc94f48b4d35bc70ffc4c4e67271489c0
2019-10-14 15:50:34 +00:00
KHIYANI, RAHUL (rk0850)
f4b34f9d69 Updating the image to quay.io airshipit/porthole repo
Adding zuul gates to calicoctl-utility

Change-Id: I5211aade73b642f43b1332a3b681d2ba4cc2e0ae
2019-10-11 20:31:34 +00:00
Varagini Karthik
c905ac41f9 Chart/Dockerfile for Postgresql Utility Container
This postgresql utility container will help the operation user
 to perform the operations on postgresql server in the K8s Cluster.

Change-Id: Ib6dc67f88e32b047531d0d07e5667ec7da73eb0f
2019-10-10 20:12:15 +05:30
Zuul
dd66fa51fb Merge "Chart/Dockerfile for etcdctl Utility Container" 2019-10-09 17:02:23 +00:00
Kavva, Jagan Mohan (jk330k)
9255a73837 Chart/Dockerfile for etcdctl Utility Container
This etcdctl utility container will help the Operation user
to check the state/stats of etcdctl resources in the K8s Cluster

Change-Id: If0c46b9c19bf8492be3e92c8c215cc9980886981
2019-10-09 11:35:12 -05:00
Roman Gorshunov
591f61462f Fix ceph-utility image name and distro tags
- Fix incorrect ceph-utility image name and tag
- Fix distributive-specific tags not being applied
- Adjust files pattern for job to run

Change-Id: Id17eaf8758b45491eb1181dd7137f79bb7e33d4e
2019-10-09 07:16:36 +00:00
KHIYANI, RAHUL (rk0850)
08388af8f3 Updating the image to quay.io airshipit/porthole and adding zuul gates
Change-Id: Ic784aab697a47a5258416eb1ee695ca9496154e6
2019-10-07 16:10:58 -05:00