239 Commits

Author SHA1 Message Date
Sergiy Markin
41199aee82 Update create db user queries
This commit changes the queries to use % instead
of %% in the Host field of CREATE USER and GRANT
ALL statements.

It also uplifts fresh jammy images for mariadb.

Change-Id: I6779f55d962bc9d8efc3b3bfe05b72cbe0b7f863
2025-01-28 22:15:22 +00:00
Vladimir Kozhukalov
672e488519 Update versions of all charts to 2024.2.0
As per agreement with
https://docs.openstack.org/openstack-helm/latest/specs/2025.1/chart_versioning.html

Change-Id: Ia064d83881626452dc3c0cf888128e152692ae77
2024-12-18 11:05:10 -06:00
Vladimir Kozhukalov
693d3a2686 Update Chart.yaml apiVersion to v2
Change-Id: I66dcaedefd0640f8a7b5343363354ba539d70627
2024-12-16 16:48:03 -06:00
Vladimir Kozhukalov
8b29037cec Move values overrides to a separate directory
This is the action item to implement the spec:
doc/source/specs/2025.1/chart_versioning.rst

Also add overrides env variables

- OSH_VALUES_OVERRIDES_PATH
- OSH_INFRA_VALUES_OVERRIDES_PATH

This commit temporarily disables all jobs that involve scripts
in the OSH git repo because they need to be updated to work
with the new values_overrides structure in the OSH-infra repo.
Once this is merged I4974785c904cf7c8730279854e3ad9b6b7c35498
all these disabled test jobs must be enabled.

Depends-On: I327103c18fc0e10e989a17f69b3bff9995c45eb4
Change-Id: I7bfdef3ea2128bbb4e26e3a00161fe30ce29b8e7
2024-12-13 12:04:44 -06:00
Vasyl Saienko
10e8b39a91 [mariadb] Add probes for exporter
Implement readiness/liveness probes for exporter

Change-Id: I7e73872dd35b8e6adf67d585e7d4d9250eca70c3
2024-11-19 14:28:01 +00:00
Vasyl Saienko
cae49d7455 Allow to use default storage class
When name of storage class is specified as default, do not add
storageClassName option to let kubernetes pick a default

Change-Id: I25c60e49ba770ce10ea2ec68c3555ffea49848fe
2024-11-19 07:49:37 +00:00
Vasyl Saienko
bb236e0a98 [mariadb] Add terminationGracePeriodSeconds
Allow to set terminationGracePeriodSeconds for server instace to let
more time to shutdown all clients gracefully.
Increase timeout to 600 seconds by default.

Change-Id: I1f4ba7d5ca50d1282cedfacffbe818af7ccc60f2
2024-11-19 07:49:33 +00:00
Vasyl Saienko
c2269d70a2 [mariadb] Use service IP to discover endpoints
It was observed that under certain circumstances
galera instances can use old IP address of the node
after pod restart. This patch changes the value of
wsrep_cluster_address variable - instead of listing
all dns names of the cluster nodes the discovery service
IP address is used. In this case cluster_node_address is set to IP
address instead of DNS name - otherwise SST method will fail.

Co-Authored-By: Oleksii Grudev <ogrudev@mirantis.com>

Change-Id: I8059f28943150785abd48316514c0ffde56dfde5
2024-11-19 07:49:29 +00:00
Vasyl Saienko
65ec71b939 [mariadb] Implement mariadb upgrade on start
Call mysql_upgrade during start to check and upgrade if needed

Change-Id: I9c4ac1a5ea5f492282bb6bb1ee9923b036faa998
2024-11-19 06:17:24 +00:00
Vasyl Saienko
f0ad9daa5a [mariadb] Avoid using deprecated isAlive
The method was deprecated and later dropped, switch to is_alive()

Co-Authored-By: dbiletskiy <dbiletskiy@mirantis.com>

Change-Id: Ie259d0e59c68c9884e85025b1e44bcd347f45eff
2024-11-18 07:41:35 +00:00
Vasyl Saienko
174f6f5bd5 [mariadb] Refactor liveness/readiness probes
* Move all probes into single script to reduce code duplication
* Check free disk percent, fail when we consume 99% to avoid
  data corruption
* Do not restart container when SST is in progress

Change-Id: I6efc7596753dc988aa9edd7ade4d57107db98bdd
2024-11-13 18:37:52 +00:00
Vasyl Saienko
842f0f11dc [mariadb] Give more time on resolving configmap update conflicts
Make 'data too old' timeout dependent on state report interval. Increase
timeout to 5 times of report interval.

Change-Id: I0c350f9e64b65546965002d0d6a1082fd91f6f58
2024-11-13 18:37:48 +00:00
Oleksii Grudev
cb3afe6f85 Prevent TypeError in get_active_endpoint function
Sometimes "endpoints_dict" var can be evaluated to None
resulting in "TypeError: 'NoneType' object is not iterable"
error. This patch catches the exception while getting
list of endpoints and checks the value of
endpoints_dict.  Also the amount of active endpoints is being logged
for debug purposes.

Change-Id: I79f6b0b5ced8129b9a28c120b61e3ee050af4336
2024-11-13 18:37:44 +00:00
Vasyl Saienko
13a683b9c2 [mariadb] Remove useless retries on conflics during cm update
The retries were originally added at [0] but they were never working.
We pass fixed revision that we would like to see during patch to avoid
race condition, into the safe_update_configmap. We can't organize retries
inside function as it will require change of the original revision which
may happen only at upper layer. Revert patch partially.

[0] https://review.opendev.org/c/openstack/openstack-helm-infra/+/788886

Change-Id: I81850d5e534a3cfb3c4993275757c244caec8be9
2024-11-13 18:37:39 +00:00
Vasyl Saienko
ef707fa3f3 [mariadb] Stop running threads on sigkill
Stop monitor cluster and leader election threads on sigkill.
This allows to terminate all threads from start.py and actually
exit earlier than terminationGracePeriod in statefulset.
Drop preStop hook which is redundant with stop_mysqld() function call.

Change-Id: Ibc4b7604f00b1c5b3a398370dafed4d19929fd7d
2024-11-13 18:36:49 +00:00
Vasyl Saienko
6d7fba0c43 [mariadb] Improve python3 compatibility
Decode byte sequence into string before printing log.

Change-Id: Icd61a1373f5c62afda0558dfadc2add9138cff6d
2024-11-11 11:37:26 +00:00
Vasyl Saienko
4aaa5fc778 [mariadb] Improve leader election on cold start
During cold start we pick leader node by seqno. When node is running
of finished non gracefully seqno may stay as -1 unless periodic task
update its based on local grastate.dat or will detect latest seqno via
wsrep_recover. This patch adds an unfinite waiter to leader election
function to wait unless all nodes report seqno different that -1 to make
sure we detect leader based on correct data.

Change-Id: Id042f6f4c915b21b905bde4d57d40e159d924772
2024-11-11 11:37:19 +00:00
Vasyl Saienko
4ee7ebda43 [mysql] Use constant for mysqld binary name
Change-Id: I996141242dac9978283e5d2086579c75d120ed8b
2024-11-11 09:45:03 +00:00
Sergiy Markin
f630c152e8 Mariadb chart updates
This PS is for improvements for wait_for_cluster mariadb job.

Change-Id: I46de32243e3aaa98b7e3e8c132a84d7b65d657cc
2024-11-01 22:34:58 +00:00
Vladimir Kozhukalov
489b87a975 Add 2024.2 overrides
Change-Id: Ic43f14e212f4de6616b4255bdd5ce562c5bcf9b0
2024-10-10 13:52:24 +00:00
Vasyl Saienko
9e5fea6e18 [mariadb] Add cluster wait job
Add job that waits when initial bootstrapping of cluster is completed
which is required to pause db creation and initialization when cluster
is not fully bootstrapped.

Change-Id: I705df1a1b1a34f464dc36a36dd7964f8a7bf72d9
2024-09-18 06:37:11 +00:00
Vasyl Saienko
243289aae3 [mariadb] Remove ingress deployment
Ingress deployment is not used for a while and there are
more elegant ways to provide same functionality based on
controller to pick up master service.
Remove ingress deployment completely.

Change-Id: Ica5d778f5122f8a4f0713353aa5e0ef4e21c77f8
2024-09-18 06:37:00 +00:00
Vasyl Saienko
475a0c4b44 [mariadb] Switch to controller deployment
Move primary node selector into mariadb controller, this patch
partially reverts 07bd8c92a259557d07119525c85bea4b8fc6006e

Change-Id: Id53a6503b177f0c46e89a7def2c0773a68b8d8e8
2024-09-18 06:36:50 +00:00
Vasyl Saienko
2f2ce5f28f [mariadb] Deploy exporter as sidecar
Deploy exporter as a sidecar to provide correct mysql metrics.

Co-Authored-By: Oleh Hryhorov <ohryhorov@mirantis.com>

Change-Id: I25cfeaf7f95f772d2b3c07a6a91220d0154b4eea
2024-09-15 09:14:53 +00:00
Vasyl Saienko
32b571ab2f [mariadb] Avoid using cluster endpoints
Switch to namespaced based endpoints to remove requirement
configure kubernetes internal cluster domain name which can't
be get from kubernetes API.

Change-Id: I8808153a83e3cec588765797d66d728bb6133a5c
2024-09-15 07:32:03 +00:00
Vasyl Saienko
954e338d17 [mariadb] Add mariadb controller support
This patch adds mairadb controller that is responsible to mark one
ready pod as mariadb_role: primary to forward all traffic to it.
This will allow to drop nginx ingress controller which adds extra
hops between client and server and uses heavy customized nginx templates.

Change-Id: I3b29bc2029bfd39754516e73a09e4e14c52ccc99
2024-09-14 18:07:38 +00:00
Sergiy Markin
43fd714348 Ingress-nginx controller upgrade for mariadb
This PS bumps up ingress-nginx controller version
to v1.11.2 in mariadb chart due to CVE
vulnerability.

nginx.tmpl from mariadb chart has been updated to
match the latest 1.11.2 ingress-controller image.

Change-Id: Ie2fd811f8123515f567afde62bbbb290d58dd1b2
2024-08-27 15:36:23 +00:00
Vladimir Kozhukalov
05f2f45971 Add 2024.1 overrides to some charts
- Add 2024.1 overrides to those charts where
  there are overrides for previous releases.
- Update some jobs to use 2024.1 overrides.
- Update default images in  grafana, postgresql,
  nagios, ceph-rgw, ceph-provisioners,
  kubernetes-node-problem-detector
- Install tzdata package on K8s nodes. This
  is necessary for kubernetes-node-problem-detector
  chart which mounts /etc/localtime from hosts.

Change-Id: I343995c422b8d35fa902d22abf8fdd4d0f6f7334
2024-07-23 07:35:50 -05:00
Vladimir Kozhukalov
8077898106 Update kubernetes-entrypoint image
Use quay.io/airshipit/kubernetes-entrypoint:latest-ubuntu_focal
by default instead of 1.0.0 which is v1 formatted and
not supported any more by docker.

Change-Id: I6349a57494ed8b1e3c4b618f5bd82705bef42f7a
2024-07-12 13:52:07 -05:00
Sergiy Markin
0e086e4c12 [mariadb] Switch to ingress-less mariadb
This PS switches mariadb to use primary service by
default instead of ingress based deployment. The
primary service that is getting created and
automatically updated based on the leader election
process in start.py entrypoint script.

Mariadb primary service was introduced by this PS:

https://review.opendev.org/c/openstack/openstack-helm-infra/+/905797

Change-Id: I4992276d0902d277a7a81f2730c22635b15794b0
2024-02-26 18:59:22 +00:00
Sergiy Markin
07bd8c92a2 [mariadb] Add mariadb-server-primary service
This PS adds mariadb-server-primary service that is getting created
and automatically updated based on the leader election process in
start.py entrypoint script.

Change-Id: I1d8a8db0ce8102e5e23f7efdeedd139726ffff28
Signed-off-by: Sergiy Markin <smarkin@mirantis.com>
2024-01-18 00:17:47 +00:00
Ritchie, Frank (fr801x)
f66bb53509 Update template for ingress 1.9
The names of a few configuration variables have changed in version 1.9.

EnableRealIp to EnableRealIP
HttpAccessLogPath to HTTPAccessLogPath
whitelist to allowlist
Whitelist to Allowlist

Additionally,

ajp_temp_path

is no longer valid.

Change-Id: I2ebb658bd237216c43306dab6cd7f7a1ca6388ac
2024-01-04 18:32:56 -05:00
Sergiy Markin
13c1d8cd38 [backups] Add throttlling of remote backups
This PS adds a possibility to limit (to throttle) the number of
simultaneously uploaded backups while keeping the logic on the client
side using flag files on remote side. The main idea is to have an
ability to limit number of simultaneous remote backups upload sessions.

Change-Id: I5464004d4febfbe20df9cd41ca62ceb9fd6f0c0d
2023-12-18 20:39:45 +00:00
Sergiy Markin
f66c924b2f [backups] Mariadb backups improvements
This PS removes mariadb-verify-server sidecar container from
mariadb-backup cronjob in order to make backup process more resilient.

Change-Id: I2517c2de435ead34397ca0483610f511c8035bdf
2023-12-15 16:18:35 +00:00
Sergiy Markin
4a95f75b6b [backups] Added staggered backups
This PS adds staggered backups possibility by adding anti-affinity rules
to backups cronjobs that can be followed across several namespaces to
decrease load on remote backup destination server making sure that at
every moment in time there is only one backup upload is in progress.

Change-Id: If49791f866a73a08fb98fa0e0b4854042d079c66
2023-12-05 04:10:22 +00:00
Sergiy Markin
29f2b616cc [mariadb-operator] Mariadb-cluster chart
This PS adds mariadb-cluster chart based on mariadb-operator. Also for
some backward compartibility this PS adds mariadb-backup chart and
prometheus-mysql-exporter chart as a separate ones.

Change-Id: I3f652375cce2e3b45e095e08d2e6f4ae73b8d8f0
2023-11-29 21:51:48 -06:00
Sergiy Markin
eae5c6d69a Uplift nginx ingress controller to v1.8.2
Change-Id: I4223f3f859833447f4045e7acea81bf4c7a8948a
2023-10-04 20:00:33 +00:00
Vladimir Kozhukalov
ae91cf3fc3 Use deploy-env role for all deployment jobs
To make it easier to maintain the jobs all experimental
jobs (those which are not run in check and gate pipelines)
are moved to a separate file. They will be revised later
to use the same deploy-env role.

Also many charts use Openstack images for testing this
PR adds 2023.1 Ubuntu Focal overrides for all these charts.

Change-Id: I4a6fb998c7eb1026b3c05ddd69f62531137b6e51
2023-09-22 15:02:07 -05:00
Mosher, Jaymes (jm616v)
5358aed591 Prevent liveness probe from killing mariadb pods during SST
Update liveness probe script to accept pods either sending
or receiving a SST, and avoid killing them.

Change-Id: I4ad95c45a7ab7e5e1cec2b4696671b6055cc10e7
2023-09-20 14:12:24 -06:00
Vladimir Kozhukalov
97ce6d7d8e Update kubernetes registry to registry.k8s.io
See this link for details
https://kubernetes.io/blog/2023/03/10/image-registry-redirect/

Change-Id: Ifc8b64825751933def16a1784fae987a1d7250ad
2023-04-11 04:54:26 +03:00
Samuel Liu
6034a00bf7 Replace node-role.kubernetes.io/master with control-plane
The master label is no longer present on kubeadm control plane nodes(v1.24). For new clusters, the label 'node-role.kubernetes.io/master' will no longer be added to control plane nodes, only the label 'node-role.kubernetes.io/control-plane' will be added. For more information, refer to KEP-2067[https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/kubeadm/2067-rename-master-label-taint]: Rename the kubeadm "master" label and taint.

the kubernetes pr: https://github.com/kubernetes/kubernetes/pull/107533

Change-Id: I3056b642db0a1799089998e3c020b4203c9a93ab
2023-03-20 13:38:07 +08:00
Anselme, Schubert (sa246v)
d30bbfbfe7
Uplift nginx ingress controller to v1.5.1
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
Change-Id: I8e3eb2ebd52c0dae1d0cc0ebaa23885b8c0cf83f
2022-12-14 15:19:51 -05:00
Markin, Sergiy (sm515x)
5c4056ad34 [DATABASE] Add verify databases backup
HTK - added verify_databases_backup_in_directory function that is
going to be defined inside mariadb/postgresql/etcd charts.

Mariadb chart - added verify_databases_backup_archives function
implementation.

Added mariadb-verify container to mariadb-backup cronjob to run
verification process.

Added remove backup verification pocess - comparition of local and remote file md5 hashes.

PostgreSQL chart - added empty implementation of verify_databases_backup_archives() function. This is a subject for future realization.

Change-Id: I361cdb92c66b0b27539997d697adfd1e93c9a29d
2022-09-09 01:41:00 +00:00
Thales Elero Cervi
111f41edf6 Fixing broken mariadb helmrelease for helmv3
In an environment with helmv3, it was noticed that the mariadb
helmrelease is failing to render properly due to unsupported map key
type (int).

This change quickly fix this problem by quoting the value, forcing it to
be rendered as a string.

Signed-off-by: Thales Elero Cervi <thaleselero.cervi@windriver.com>
Change-Id: I2f2be87d0f79ca439e731d07354bcd5f149790d5
2022-09-01 13:31:09 -03:00
Brian Haley
f31cfb2ef9 support image registries with authentication
Based on spec in openstack-helm repo,
support-OCI-image-registry-with-authentication-turned-on.rst

Each Helm chart can configure an OCI image registry and
credentials to use. A Kubernetes secret is then created with these
info. Service Accounts then specify an imagePullSecret specifying
the Secret with creds for the registry. Then any pod using one
of these ServiceAccounts may pull images from an authenticated
container registry.

Change-Id: Iebda4c7a861aa13db921328776b20c14ba346269
2022-07-20 14:28:47 -05:00
Markin, Sergiy (sm515x)
931ba39e87 [MariaDB] Add liveness probe to restart a pod that got stuck in a transfer wsrep_local_state_comment
Readiness probe that we currently have does not help with restarting a
pod that got stuck in a transfer state reported by
wsrep_local_state_comment.

root@mariadb-server-2:/# mysql_status_query wsrep_ready
OFF
root@mariadb-server-2:/# mysql_status_query wsrep_connected
ON
root@mariadb-server-2:/# mysql_status_query wsrep_cluster_status
non-Primary
root@mariadb-server-2:/# mysql_status_query wsrep_local_state_comment
Transfer

So the idea is to add a liveness probe that will take care of this.

Change-Id: I2ccecc75349667fe19c6f7f9dccc2dbbd17d0a5e
2022-06-21 20:32:52 +00:00
Schubert Anselme
fce7ca38ae
Uplift Mariadb-ingress image to v1.2.0
Change-Id: Ic368517f893c5016793ce5d65b882a43fb2381ec
2022-06-06 09:14:56 -04:00
Markin, Sergiy (sm515x)
1f1a2ff527 [MariaDB] Fix backup/restore scripts for MariaDB 10.6
This patch adds database sys to the list of databases
to be ignored by backup/restore scripts in mariadb chart.

Change-Id: Ida7965bc583ada2c7ca4800c8ff5d6761fb3913a
2022-05-19 00:26:49 +00:00
Markin, Sergiy (sm515x)
322e5b8ccb [MariaDB] Fix ingress cluster role privileges
This patchset is adding update priviledge to ingress cluster role in order to let it to update mariadb state configmap. The problem appeared after upgrading nginx controller up to v1.1.3 in https://review.opendev.org/c/openstack/openstack-helm-infra/+/840691

Change-Id: I962ac336bf6b3588db88b04e2259de1aa20b1221
2022-05-13 17:42:32 -05:00
Markin, Sergiy (sm515x)
9d9edbded5 [MariaDB] Fix privileges for mysql-exporter user used by prometheus exporter
Change-Id: I1a2ba8d2525d28d1179a64d5c815e2f32ef56744
2022-05-12 17:35:55 -05:00