150 Commits

Author SHA1 Message Date
Reed, Joshua
99d4652354 Correct trigger for metadata job in .zuul.yaml
The metadata job previously would only trigger when
a file in the python3-* repo was changed.  This means
one could get a change to the metadata.yaml submitted
without the proper pipeline job checking the file. This
change corrects that.

Test Plan:
PASS: Verify metadata job runs.

Story: 2010929
Task: 49489

Change-Id: Ie81411fbc6e8debc2f1b8f4050c7bb72b3b2c3fd
Signed-off-by: Reed, Joshua <Joshua.Reed@windriver.com>
2024-02-06 09:47:00 -07:00
Zuul
585fee6033 Merge "Adding semantic check on platform-integ-apps reapply operation" 2024-02-05 17:45:04 +00:00
Guilherme Costa
788991a76d Adding semantic check on platform-integ-apps reapply operation
The reapply operation needs to do the same checks done on the apply
operation. These checks prevent reapplying the application on
some ceph or system scenarios.

Moreover, on the method pre_auto_apply_check, it is possible that
conductor_object._ceph (CephOperator) is not initialized yet, and trying
to access its methods will raise an AttributeError exception. As this is
a possible scenario, this case is now tested, and a
LifecycleSemanticCheckException is raised.

Test Plan:

    PASS: (AIO-DX) Check that platform-integ-apps is on uploaded state
    PASS: (AIO-DX) Unmanage ceph-mon
    PASS: (AIO-DX) Stop ceph-mon service
    PASS: (AIO-DX) Restart sysinv-conductor service
    PASS: (AIO-DX) Check on sysinv logs that the new Exception is
          being logged and the platform-integ-apps still trying to
          be applied
    PASS: (AIO-DX) Manage ceph-mon
    PASS: (AIO-DX) Start ceph-mon service
    PASS: (AIO-DX) After the Ceph cluster has become responsive,
          check that platform-integ-apps is being successfully
          applied.

    PASS: (AIO-DX) Check that platform-integ-apps is on applied state
    PASS: (AIO-DX) Unmanage ceph-mon
    PASS: (AIO-DX) Stop ceph-mon service
    PASS: (AIO-DX) Restart sysinv-conductor service
    PASS: (AIO-DX) Check on sysinv logs that the new Exception is
          being logged and the platform-integ-apps still trying
          to be applied.
    PASS: (AIO-DX) Wait until platform-integ-apps is on
          apply-failed state

Closes-bug: 2052315

Change-Id: I8c3318cc6f002671cb2e4bf81b14238ac4467131
Signed-off-by: Guilherme Costa <guilherme.costa@windriver.com>
2024-02-05 14:12:35 +00:00
Igor Soares
f59aa662bb Add minimum Kubernetes version supported
Add the minimum Kubernetes version supported to the application metadata
file.

The minimum Kubernetes version is set to 1.24.4 and should be changed
accordingly for future application updates.

The "supported_k8s_version:minimum" field is optional but it will become
mandatory in the near future.

Test Plan
PASS: build-pkgs && build-image
PASS: Apply application

Story: 2010929
Task: 49489

Change-Id: Ie4e0c01ca8a0d4b0d71e510b486cf866d4a62745
Signed-off-by: Igor Soares <Igor.PiresSoares@windriver.com>
2024-02-01 11:02:04 -03:00
Gabriel de Araújo Cabral
855c7b1da9 Add check for csi-snapshotter container creation
This commit introduces a check to determine whether or not the
csi-snapshotter container will be created in each provisioner's
pod.

The only scenario in which the 'provisioner.snapshotter.enabled'
field will automatically be set to 'true' and the csi-snapshotter
container will be created during apply is with:
- CRDs created: volumesnapshotclasses, volumesnapshots and
  volumesnapshotcontents.
- Snapshot-controller pod created and using the correct image
  according to the K8s version.

In all other scenarios, 'provisioner.snapshotter.enabled' will be
set to 'false' during apply and the container will not be created.

Remembering that the user can create/remove the container manually
if desired via helm override.

Depends-on: https://review.opendev.org/c/starlingx/platform-armada-app/+/904875

Test Plan:
 PASS: Build a new app package with the code changes
 PASS: Successfully execute upload, apply, remove, and delete
       operations for platform-integ-apps
 PASS: In an environment with CRDs and snapshot-controller using
       the correct image version according to Kubernetes, apply the
       app and verify that the snaphotter field is 'enabled: true'
       and the container has been created in both provisioner pods
 PASS: In an environment with CRDs but with snapshot-controller
       using the incorrect image version according to Kubernetes,
       apply the app and verify that the snaphotter field is
       'enabled: false' and the container has not been created
 PASS: In an environment without CRDs and snapshot-controller, apply
       the app and verify that snaphotter field is 'enabled: false'
       and the container has not been created
 PASS: Manually create/remove csi-snapshotter container via
       helm-override

Partial-Bug: 2045897

Change-Id: I7dbcbba520c9758de84e8dae5a553ec1fee69518
Signed-off-by: Gabriel de Araújo Cabral <gabriel.cabral@windriver.com>
2024-01-25 13:28:02 -03:00
Hediberto Cavalcante da Silva
fbee8e217a Update ceph-csi helm charts to v3.10.1
Update RBD/CephFS provisioners to v3.10.1 based on
https://github.com/ceph/ceph-csi/releases/tag/v3.10.1

We use the latest versions of ceph-csi, csi-provisioner,
csi-attacher, csi-snapshotter, csi-node-driver-registrar,
csi-resizer:

- https://github.com/kubernetes-csi/external-provisioner/releases/tag/v3.6.2
- https://github.com/kubernetes-csi/external-attacher/releases/tag/v4.4.2
- https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v6.3.2
- https://github.com/kubernetes-csi/node-driver-registrar/releases/tag/v2.9.1
- https://github.com/kubernetes-csi/external-resizer/releases/tag/v1.9.2

Test Plan:
 PASS: App upload/apply/remove/delete/update
 PASS: Create PVC using storageclass general (rbd)
 PASS: Create pod using RBD PVC
 PASS: Resize RBD PVC
 PASS: Create RBD volume snapshot
 PASS: Create PVC using storageclass cephfs
 PASS: Create pod using cephfs PVC
 PASS: Resize cephfs PVC
 PASS: Create cephfs volume snapshot

* Tests were performed on SX/DX/Standard/Storage with K8s
  1.24, 1.25, 1.26 and 1.27

Story: 2010688
Task: 49420

Change-Id: I5d4b6d3583be0f60f5a15c5ace49951959bdd027
Signed-off-by: Hediberto Cavalcante da Silva <hediberto.cavalcantedasilva@windriver.com>
Signed-off-by: Gabriel de Araújo Cabral <gabriel.cabral@windriver.com>
2024-01-22 21:57:30 -03:00
Robert Church
829555e0d9 Auto-increment chart versions
Enable auto-versioning of helm charts to ensure the FluxCD helm
controller recognizes chart changes.

To guarantee the helm chart version is incremented when a helm chart
change is submitted, the following is implemented:
 - Provide a top level hierarchy for helm charts to differentiate
   between upstream and custom charts: helm-charts/{custom,upstream}
 - Move platform-helm package to helm-charts/upstream
 - Move ceph-pools-audit to helm-charts/custom.
 - stx-APP-helm now contains only manifests and final application
   packaging rules. No custom helm charts should be delivered here.
 - Use StarlingX debian git revcount packaging mechanisms to derive the
   semver PATCH version for custom helm charts. Chart
   version (MAJOR.MINOR.PATCH) is passed to 'helm package' command to
   force the version, where PATCH == 'git revcount'
 - Use StarlingX debian git revcount packaging mechanisms to derive the
   semver BUILD version for upstream helm charts which maintains the
   upstream chart version and adds a versioned BUILD extension.

     <version core> ::= <major> "." <minor> "." <patch>
     <valid semver> ::= <version core> "+" <build>

   Chart version (MAJOR.MINOR.PATCH+STX.REV) is passed to 'helm package'
   command to force the version, where REV == 'git revcount'
 - Establish a new package for the custom 'ceph-pools-audit' chart
   called 'ceph-pools-audit-helm'
 - Enforce a baseline version for both 'platform-helm' and
   'ceph-pools-audit-helm' based on existing pre-move chart changes.

   $ git rev-list --count HEAD platform-helm/
   16

   $ git rev-list --count HEAD \
       stx-platform-helm/stx-platform-helm/helm-charts/ceph-pools-audit
   19
 - Update 'stx-platform-helm' to:
   - Update the build dependencies to include the new helm chart package
     and remove dependency on helm
   - Update the rules to remove building the ceph-pools-audit chart and
     automatically update the chart versions in the fluxCD
     helmrelease.yaml files.
 - Fix ceph-csi-rbd storage-init.yaml Job chart label as it was
   generating an invalid label based on the new auto versioning scheme.

Additional changes and cleanup:
 - Remove 'helm dep' commands from chart Makefile as no chart
   dependencies are present
 - Fix formatting in 'platform-helm' and 'stx-platform-helm' changelogs
   to avoid dpkg-genchanges errors.

Test Plan:
PASS - Build all packages generating an application tarball verifying
       all versions on the charts and application make sense.
PASS - Introduce temporary chart changes and ensure that the versions
       increment as expected.
PASS - Validate basic application lifecycle operations:
       upload/apply/remove/delete.

Change-Id: Ie5d5b9c7bb080210be18b0d886631638e705657f
Story: 2010929
Task: 49363
Signed-off-by: Robert Church <robert.church@windriver.com>
vf/kernel-6.6
2024-01-17 16:26:23 -06:00
Erickson Silva de Oliveira
9195bfb887 Remove debug option from cephfs PVs
Since version v3.9.0 of ceph-csi, the "debug" option has been
removed from mountOptions of the cephfs storage class, however,
this option still exists on cephfs PVs created with the previous
version of ceph-csi, causing the pod to fail.

To resolve this, a check for existing cephfs PVs has been added
to the cephfs storage-init script to remove this parameter if
it exists.

Test Plan:
  PASS: Create a PVC and pod on AIO-SX with ceph-csi v3.6.2
  PASS: Build platform-integ-apps with changes (ceph-csi v3.9.0)
  PASS: Check that "mountOption: -debug" is not present in the
        cephfs storage class and pv.

Closes-Bug: 2047571

Change-Id: Id7c8f77d2bc0b4e4afc67966810d5d3c40fc1e06
Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
2024-01-16 15:05:38 -03:00
Zuul
396dfb7d34 Merge "Add SnapshotClass Creation for CephFS/RBD via Helm Override" 2024-01-08 16:42:44 +00:00
Gabriel de Araújo Cabral
313c3b1d1d Add SnapshotClass Creation for CephFS/RBD via Helm Override
This commit  introduces the capability to create a snapshot
class using helm overrides within "cephfs-provisioner"
and "rbd-provisioner" charts.

By default, upon applying platform-integ-apps the
'snapshotClass.create' field is set to 'false' in both charts,
creating the snapshot class(es) after being changed to 'true' via
helm overrides and the app reapplied.

This enhancement depends on the changes from the review below,
where in each installation the CRDs and the snapshot-controller
will be created by default when running bootstrap playbook.
This way, each CSI app will be able to implement this functionality
as support for PVC snapshots will be default during installations.

* This is a new review after the revert of:
https://review.opendev.org/c/starlingx/platform-armada-app/+/902722

Test Plan:
 PASS: Build a new app package with the code changes
 PASS: Successfully execute upload, apply, remove, and delete
       operations for 'platform-integ-apps' on both AIO-SX and
       AIO-DX environments.
 PASS: Upon initial application apply and update, check with
       'helm get values' that the 'snapshotClass.create' field
       is set to 'false' in both the cephFS and RBD charts.
       Additionally, confirm on K8s that the snapshotClasses are not
       created, as expected
 PASS: Use 'system helm-override-update' to change the
       'snapshotClass.create' field to 'true' in both charts.
       Reapply the app and validate on K8s that the snapshotClasses
       are indeed created.
 PASS: After creating the SnapshotClass, take a VolumeSnapshot
       from an existing PVC, proceed with K8s upgrade (K8s 1.25,
       1.26, and 1.27) with CRDs and snapshot controller update,
       take a VolumeSnapshot from another PVC and verify that
       all VolumeSnapshots are correct
 PASS: With an old version of the application, create user overrides
       for 'classes', then update to the application with the
       current changes and verify that the user overrides for
       'classes' have been transferred to 'storageClasses'

Partial-Bug: 2045897

Depends-on: https://review.opendev.org/c/starlingx/ansible-playbooks/+/904360

Change-Id: I6e2fe2009d4cce3e351142359c1f36465cf03ee3
Signed-off-by: Gabriel de Araújo Cabral <gabriel.cabral@windriver.com>
Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
2024-01-08 12:45:54 -03:00
Zuul
c810a9c187 Merge "Application versioning based on build release" 2024-01-08 15:05:46 +00:00
Erickson Silva de Oliveira
5255e744eb Add "checksum/config" annotation to nodeplugin-daemonset
With the mgmt network reconfiguration, the 'cephfs-nodeplugin'
and 'rbd-nodeplugin' pods don't recognize the change and are
not automatically restarted.

To resolve this, an annotation was added to each
nodeplugin-daemonset.yaml, monitoring any changes to
csiConfig values.

$ kubectl get configmaps -n kube-system cephfs-csiplugin-config -o yaml
apiVersion: v1
data:
  config.json: '[{"clusterID":"e36fa26b-bdd5-42a5-83b3-c36a4042109b",
  "monitors":["192.168.204.2:6789"]}]'
kind: ConfigMap
...

----------------------------------------------------------------------

$ kubectl get configmaps -n kube-system cephfs-csiplugin-config -o yaml
apiVersion: v1
data:
  config.json: '[{"clusterID":"e36fa26b-bdd5-42a5-83b3-c36a4042109b",
  "monitors":["192.168.208.3:6789"]}]'
kind: ConfigMap
...

As can be seen above, the monitor is modified after reconfiguration.
This information is obtained through Values.csiConfig in
csiplugin-configmap template:

data:
  config.json: |-
{{ toJson .Values.csiConfig | indent 4 -}}

Test Plan:
  - PASS: Create pod on AIO-SX with ceph backend
  - PASS: Reconfiguration of the mgmt network
  - PASS: Check if the previously created pod is running

Story: 2010722
Task: 49295

Change-Id: I9892f1ba7127fe381cba5727e34d16803f93b5bf
Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
2024-01-05 11:20:23 -03:00
Zuul
718d56a3d6 Merge "Update app Zuul Check Jobs." 2024-01-03 13:10:47 +00:00
Robert Church
ae4e5b15b2 Application versioning based on build release
This change will automatically adjust versioning of the application
tarball and python plugins to reflect the same version reported by
SW_VERION in /etc/build.info.

Test plan:
PASS: build-pkgs -a & build-image
PASS: Fresh AIO-SX install
PASS: Apply application

Story: 2010929
Task: 49361

Change-Id: I8e474341812c39112e9055c99630f54729ea3694
Signed-off-by: Robert Church <robert.church@windriver.com>
2023-12-29 15:00:06 -03:00
Erickson Silva de Oliveira
ae2da62c56 Revert "Add SnapshotClass Creation for CephFS/RBD via Helm Override"
This reverts commit 8b1f987add87d217f720f505b9f65f80fc8dbd20.

Reason for revert: This review possibly caused a sanity failure

Change-Id: I6d0958f793cd46abca552fa0f9b3188d87036db8
2023-12-22 18:03:41 +00:00
Gabriel de Araújo Cabral
8b1f987add Add SnapshotClass Creation for CephFS/RBD via Helm Override
This commit  introduces the capability to create a snapshot
class using helm overrides within "cephfs-provisioner"
and "rbd-provisioner" charts.

By default, upon applying platform-integ-apps the
'snapshotClass.create' field is set to 'false' in both charts,
creating the snapshot class(es) after being changed to 'true' via
helm overrides and the app reapplied.

This enhancement depends on the changes from the review below,
where in each installation the CRDs and the snapshot-controller
will be created by default when running bootstrap playbook.
This way, each CSI app will be able to implement this functionality
as support for PVC snapshots will be default during installations.

Depends-on: https://review.opendev.org/c/starlingx/ansible-playbooks/+/902811

Test Plan:
 PASS: Build a new app package with the code changes
 PASS: Successfully execute upload, apply, remove, and delete
       operations for 'platform-integ-apps' on both AIO-SX and
       AIO-DX environments.
 PASS: Upon initial application apply and update, check with
       'helm get values' that the 'snapshotClass.create' field
       is set to 'false' in both the cephFS and RBD charts.
       Additionally, confirm on K8s that the snapshotClasses are not
       created, as expected
 PASS: Use 'system helm-override-update' to change the
       'snapshotClass.create' field to 'true' in both charts.
       Reapply the app and validate on K8s that the snapshotClasses
       are indeed created.
 PASS: After creating the SnapshotClass, take a VolumeSnapshot
       from an existing PVC, proceed with K8s upgrade (K8s 1.25,
       1.26, and 1.27) with CRDs and snapshot controller update,
       take a VolumeSnapshot from another PVC and verify that
       all VolumeSnapshots are correct

Partial-Bug: 2045897

Change-Id: I47b897d179c4260fad9171586fe2fbd69f7145de
Signed-off-by: Gabriel de Araújo Cabral <gabriel.cabral@windriver.com>
2023-12-19 10:48:13 -03:00
Reed, Joshua
82d9009b0e Update app Zuul Check Jobs.
Modify code to conform to flake8 and pylint.

Jobs are now flake8, pylint, py39 and metadata.

Test Plan
PASS - All zuul jobs pass as expected.

Story: 2010929
Task: 49278

Change-Id: Idab7faf2a4957edd95f0985c3dd4a70a2a95ecba
Signed-off-by: Reed, Joshua <Joshua.Reed@windriver.com>
2023-12-15 12:44:37 -07:00
Gustavo Ornaghi Antunes
5d3806316f Properly disable platform-integ-apps charts
Pods were not being deleted when rbd-provisioner or cephfs-provisioner
charts are disabled and then reapplied the app.
We needed to add new value in helm.sh/hook annotation on deployment
resource to delete pods when chart is disabled and then reapplied
platform-integ-apps.

Test Plan:
PASS: Apply platform-integ-apps, disable rbd-provisioner chart,
then reapply App and Check if chart related pods, FluxCD helmrelease
and helm release are all removed.
PASS: Apply platform-integ-apps, disable cephfs-provisioner chart,
then reapply App and Check if chart related pods, FluxCD helmrelease
and helm release are all removed.

Closes-Bug: #2029496

Change-Id: Ie4353cab3cf93abf362dc1e55265d4eb6a9d017e
Signed-off-by: Gustavo Ornaghi Antunes <gustavo.ornaghiantunes@windriver.com>
vf/antelope
2023-08-09 12:24:31 +00:00
Gabriel de Araújo Cabral
a832208a85 Update ceph-csi helm charts to v3.9.0
Update RBD/CephFS provisioners to v3.9.0 based on
https://github.com/ceph/ceph-csi/releases/tag/v3.9.0

We use the latest versions of ceph-csi, csi-provisioner,
csi-attacher, csi-snapshotter, csi-node-driver-registrar,
csi-resizer:

- https://github.com/kubernetes-csi/external-provisioner/releases/tag/v3.5.0
- https://github.com/kubernetes-csi/external-attacher/releases/tag/v4.3.0
- https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v6.2.2
- https://github.com/kubernetes-csi/node-driver-registrar/releases/tag/v2.8.0
- https://github.com/kubernetes-csi/external-resizer/releases/tag/v1.8.0

Test Plan:
PASS: SX/DX/Storage app upload/apply/remove/delete/update
PASS: Create pvc using storageclass general (rbd) on SX/DX/Storage
PASS: Create pod using RBD pvc on SX/DX/Storage
PASS: Resize RBD pvc on SX/DX/Storage
PASS: Create RBD volume snapshot on SX/DX/Storage
PASS: Create pvc using storageclass cephfs on SX/DX/Storage
PASS: Create pod using cephfs pvc on SX/DX/Storage
PASS: Resize cephfs pvc on SX/DX/Storage
PASS: Create cephfs volume snapshot on SX/DX/Storage

* Tests were performed with K8s 1.24, 1.25 and 1.26

Story: 2010688
Task: 48477

Change-Id: I44d9a58c866ee901a343c6bf46eb3682a723953b
Signed-off-by: Gabriel de Araújo Cabral <gabriel.cabral@windriver.com>
2023-07-27 14:55:44 +00:00
Marcos Paulo Oliveira Silva
e6904c6f37 Remove Node Feature Discovery helm charts from platform packaging
Test Plan:

PASSED: The platform-integ-apps debian package was built succesfully

Story: 2010769
Task: 48235

Depends-On: https://review.opendev.org/c/starlingx/app-node-feature-discovery/+/885097

Change-Id: I14b739127ac1da9c794a7eb253a52690cf3da93f
Signed-off-by: Marcos Paulo Oliveira Silva <Marcos.PauloOliveiraSilva@windriver.com>
2023-06-14 16:04:56 -03:00
Erickson Silva de Oliveira
55f5e443fa Update ceph-csi helm charts to v3.8.0
Update RBD/CephFS provisioners to v3.8.0 based on
https://github.com/ceph/ceph-csi/releases/tag/v3.8.0

NOTE: We use the latest versions (released until April 11, 2023)
of csi-provisioner, csi-attacher, csi-snapshotter,
csi-node-driver-register, csi-resizer, which has fewer security flaws:

https://github.com/kubernetes-csi/external-provisioner/releases/tag/v3.4.1
https://github.com/kubernetes-csi/external-attacher/releases/tag/v4.2.0
https://github.com/kubernetes-csi/external-snapshotter/releases/tag/v6.2.1
https://github.com/kubernetes-csi/node-driver-registrar/releases/tag/v2.6.3
https://github.com/kubernetes-csi/external-resizer/releases/tag/v1.7.0

Test Plan:
PASS: SX/DX/Storage app upload/apply/remove/delete/update
PASS: Create pvc using storageclass general (rbd) on SX/DX/Storage
PASS: Create pod using RBD pvc on SX/DX/Storage
PASS: Resize RBD pvc on SX/DX/Storage
PASS: Create RBD volume snapshot on SX/DX/Storage
PASS: Create pvc using storageclass cephfs on SX/DX/Storage
PASS: Create pod using cephfs pvc on SX/DX/Storage
PASS: Resize cephfs pvc on SX/DX/Storage
PASS: Create cephfs volume snapshot on SX/DX/Storage

Depends-On: https://review.opendev.org/c/starlingx/config/+/883974

Story: 2010688
Task: 47807

Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
Change-Id: I259667de2e2149f9af2218893fadc834bff6b3a1
2023-05-30 11:08:06 +00:00
Davlet Panech
d4f0cb3286 Fix github mirroring for this repo
Updating the rsa ssh host key based on:
https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/

Note: In the future, StarlingX should have a zuul job and
secret setup for all repos so we do not need to do this
for every repo.

Needed to rename the secret, because zuul fails if like-named
secrets have diffent values in different branches of the same
repo.

Partial-Bug: #2015246
Change-Id: I1ebaccd444d78478d99450bf5a06a92ebae768f8
Signed-off-by: Davlet Panech <davlet.panech@windriver.com>
2023-04-28 12:38:52 -04:00
Erickson Silva de Oliveira
af181a31fd Improves cephfs and rbd storage-init scripts
According to Jim's comments in [1], the ceph-fs and rbd init_storage
scripts needed improvement. In this way, the return codes of some
commands were checked, indentation problems were corrected,
in addition to making the log more readable.

Test Plan:
PASS: Update platform-integ-apps
PASS: Check logs with
"kubectl logs -n kube-system rbd-storage-init-XXXXX"
PASS: Check logs with
"kubectl logs -n kube-system cephfs-storage-init-XXXXX"

Story: 2010688
Task: 47806

[1] https://review.opendev.org/c/starlingx/platform-armada-app/+/871789

Signed-off-by: Erickson Silva de Oliveira <Erickson.SilvadeOliveira@windriver.com>
Change-Id: I4191af4e5af541443744a20e43ace8d4e9e7b847
2023-04-24 15:18:59 +00:00
Felipe Sanches Zanoni
ef33b99009 Adjust platform docker images to get from registry.local
The platform-integ-apps docker images were pointing to the
public registry. This was caused because the path for each
image was defined in values.yml file for cephfs and rbd provisioners,
but not defined in the static overrides files.

This change adds the image paths to the static overrides files, so
Conductor can change the prefix to registry.local:9001 in the
deployments.

Test-Plan:
  PASS: Fresh install AIO-SX and apply platform-integ-apps
        application. Verify if the deployments have all images
        pointing to registry.local:9001 registry.

Closes-bug: 2015788

Change-Id: Icb04d4fb9c9073eeb8de4bd9507a8460f0fbda14
Signed-off-by: Felipe Sanches Zanoni <Felipe.SanchesZanoni@windriver.com>
2023-04-10 14:58:49 -03:00
Zuul
33d690327f Merge "Update Debian package versions to use git commits" 2023-03-21 16:29:49 +00:00
Hediberto Cavalcante da Silva
6033f687b1 Updating helmrelease and appversion for ceph-csi
This change is part of solution to add userID and userKey
to rbd/cephfs secrets. Updating helmrelease and appvesion for
stx-rbd-provisioner and stx-cephfs-provisioner allows changes made
to patches related to rbd-storage-init and cephfs-storage-init jobs
can be applied via application-update avoiding an extra manual step.

This is needed to trigger a new revision on stx-rbd-provisioner and
stx-cephfs-provisioner during helm upgrade.

Test Plan:
PASS: build platform-helm, python3-k8sapp-platform and stx-platform-helm
      packages successfully.
PASS: secrets updated after apply using application-update.
PASS: new revision was released.

Partial-bug: 2003907

Depends-On: https://review.opendev.org/c/starlingx/platform-armada-app/+/871789

Signed-off-by: Hediberto Cavalcante da Silva <hediberto.cavalcantedasilva@windriver.com>
Change-Id: Ie4427bdc801c07c170e87dddb35018c9edc64efa
2023-03-16 10:01:23 -04:00
Zuul
47fdcc3e7d Merge "Update CronJob to use k8s 1.25 supported API version" 2023-03-15 21:44:42 +00:00
Sachin Gopala Krishna
0ed0d9b003 Update CronJob to use k8s 1.25 supported API version
The batch/v1beta1 API version of CronJob is no longer served as of
v1.25.
Migrate API clients to use the batch/v1 API version instead which is
available since k8s 1.21.

Test Plan:
PASS: Install ISO with k8s 1.24 on AIO-SX
PASS: Install ISO with k8s 1.25 on AIO-SX
PASS: Apply platform-armada-app and verify if ceph-pools-audit is
      running
PASS: Verify that batch/v1 API version is being used

Story: 2010368
Task: 47291

Signed-off-by: Sachin Gopala Krishna <saching.krishna@windriver.com>
Change-Id: I29fb40a46ff8a8e9fb076667499fa795e07970b6
2023-03-15 16:30:48 -04:00
Zuul
326a1ff85a Merge "Updated variable names used in versioning" 2023-03-08 22:02:20 +00:00
Zuul
1000fc7e2c Merge "Enable automatic app update" 2023-03-08 21:19:33 +00:00
Pedro Vinícius Silva da Cruz
3d37e034fc Update Debian package versions to use git commits
The Debian packaging has been changed to reflect all the
git commits under the directory, and not just the commits
to the metadata folder.

This ensures that any new code submissions under those
directories will increment the versions.

Test Plan:
PASS: build-pkgs -p platform-helm

Story: 2010550
Task: 47547

Signed-by: Pedro Vinícius Silva da Cruz
<pedro.silvadacruz@windriver.com>

Change-Id: I0925205e0368acd08431c697682da7832544497c
2023-03-08 11:33:31 +00:00
Igor Soares
e322e8d6af Enable automatic app update
This allows platform-integ-apps to be automatically updated.
This is helpful when providing patches, removing the need for manual
intervention.

Test Plan:
PASS: Rebuild platform-integ-apps containing "auto_update: true" then
      manually update. Created another designer tarball with version
      bumped and observed the auto update process be triggered
      automatically and finished successfully.

Story: 2010628
Task: 47576

Signed-off-by: Igor Soares <Igor.PiresSoares@windriver.com>
Change-Id: I51143231d07c770113600d01b72052e8c2460882
2023-03-03 14:52:57 -05:00
Leonardo Fagundes Luz Serrano
738f00d325 Updated variable names used in versioning
Updated a couple variable names to better
represent their contents.

Note this change has no impact on running apps
or build process, it's just for clarity.

Tests:
pass build-pkgs

Story: 2010542
Task: 47574

Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I5e0c9a350fdb3a30f000cef10d88f386842d82ad
2023-03-02 15:13:38 -03:00
Leonardo Fagundes Luz Serrano
f5cf584d8d Standardize app configs
Fixed versioning and adjusted some build files
to bring them as close to a standard as possible.

- Removed centos files
- Added version tracking via GITREVCOUNT
- Fixed mismatch in plugin name, set to python3-k8sapp-<app>
- Standardized plugin debian files (rules, *.install)
- Plugin wheels saved to /plugin instead of /plugin/<app>

Test Plan:
PASS - Build-pkgs -a
PASS - Build-image
PASS - Install, bootstrap, unlock
PASS - app tarball contains wheel file
PASS - wheel versioning updated properly

Story: 2010542
Task: 47193

Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I601800969134661ff40d6fb6277ffad3ce403eb7
2023-03-01 16:32:28 -03:00
Zuul
4c765c3550 Merge "Cleanup tox and zuul" 2023-03-01 17:42:34 +00:00
Leonardo Fagundes Luz Serrano
aa03f479f2 Cleanup tox and zuul
tox changes:
 - remove python2.7 and python3.6 unit test targets
   from tox.ini since only python 3.9 is supported
 - tox jobs are now setup for python 3.9
 - Added additional suppressions for new pylint codes.

Zuul changes:
 - remove py27 job from zuul
 - set the parent zuul jobs to be the jobs defined under
   openstack-zuul-jobs rather than just zuul-jobs
 - Specify debian-bullseye (python3.9) as the nodeset.
   ubuntu-jammy is python3.10, ubuntu-focal is python3.8
 - override tox constraints file to use the STX constraints
   This means starlingx/root needs to be a required project

Notes:
- these changes do not affect runtime.

Test Plan:
pass tox
pass build-pkgs

Story: 2010531
Task: 47511

Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I4649c60e669b546ff8b31df832e6e16b348f2fee
2023-02-28 12:33:53 -03:00
Zuul
89b05333c5 Merge "Add userID and userKey to rbd/cephfs secrets" 2023-02-28 09:25:54 +00:00
Hediberto Cavalcante da Silva
cad2f45bec Add userID and userKey to rbd/cephfs secrets
To create a new RBD volume snapshot the userID and userKey fields
are required in the RBD secret. These fields are required by
csi-snapshotter container in the rbd-provisioner pod.

Official examples show that the RBD secrets need userID and userKey
fields, while cephfs needs adminID and adminKey required
for dynamically provisioned volumes and userID and userKey required
for statically provisioned volumes. That's why these fields were also
added in cephfs secret.

Test Plan:
PASS: Create RBD volume snapshot and check if it's ready to use.
PASS: Restore RBD volume snapshot.
PASS: Create Cephfs volume snapshot and check if it's ready to use.
PASS: Restore Cephfs volume snapshot.

Closes-bug: 2003907

Signed-off-by: Hediberto Cavalcante da Silva <hediberto.cavalcantedasilva@windriver.com>
Change-Id: If526d8412f71da936a66ab783c5919bba0c6611d
2023-02-22 13:22:17 -05:00
Igor Soares
770e5f01d7 Add kube-upgrade-complete trigger
Add kube-upgrade-complete trigger for reevaluating app reaplies

The kube-upgrade-complete trigger was introduced to resume the
app reapply evaluation process after a Kubernetes upgrade is completed.

Test Plan:
PASS: Full image build
PASS: AIO-SX deployment

Relates-to: https://review.opendev.org/c/starlingx/config/+/870990
Closes-Bug: 2003260

Signed-off-by: Igor Soares <Igor.PiresSoares@windriver.com>
Change-Id: I85973b076aa1d05ca60cc58d0d26dd93c84d7092
2023-01-26 07:46:20 -05:00
Dan Voiculeasa
e29fbbadbd Keep user overrides during app update
It was observed when updating the app part of platform upgrade that
it loses user overrides. I suspect a day 0 slip. All the other apps
already have a flag enabled to keep the user overrides during
app updates.

Enable 'maintain_user_overrides' flag, if present on N+1 version then it
will keep the N version user overrides.

If there are any kind of user overrides that need transformation
between N->N+1 update, these must be handled in the upgrade scripts,
helm charts or in the lifecycle hooks(maybe create new lifecycle
hook for this purpose).

Tests:
PASS: tested stx.5.0 to stx.6.0 upgrade, with patched
      platform-integ-apps, user overrides are kept

Closes-Bug: 2002301
Signed-off-by: Dan Voiculeasa <dan.voiculeasa@windriver.com>
Change-Id: I8d9563519129372e35d328942813256f1a3c1e38
vr/stx.8.0 __v.stx.test2
2023-01-10 14:25:54 +00:00
Al Bailey
1e58027156 Update tox.ini to work with tox 4
This change will allow this repo to pass zuul now
that this has merged:
https://review.opendev.org/c/zuul/zuul-jobs/+/866943

Tox 4 deprecated whitelist_externals.
Replace whitelist_externals with allowlist_externals

Zuul is pinned to use tox < 4 in order for the py39
unit tests to pass. Additional changes for skipsdist
may be required in order to work properly with tox 4.

Partial-Bug: #2000399

Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: Iab988c50b18d683a0d0967cbcc31bbad621005f8
2022-12-30 14:11:59 +00:00
Felipe Sanches Zanoni
8a9f6b3ddf Ceph-csi: Add capability to create multiple storageclasses
Ceph-csi, part of platform-integ-apps application, was not able
to create multiple StorageClasses like the older provisioner.

Code has been changed to accept multiple StorageClasses from
helm-override command.

Test Plan:
  PASS: On AIO-SX, create rbd pvc, create pod
        and verify pod is running with no errors
  PASS: On AIO-SX, create cephfs pvc, create pod
        and verify pod is running with no errors
  PASS: On AIO-SX, upgrade platform activation complete
  PASS: On Storage setup, create new storage tier,
        add new storage backend, add disk to storage-0 for this new
        backend. Unlock storage-0 and verify StorageClass is created
        for the new storage backend.
  PASS: Add helm override with multiple StorageClasses,
        reapply platform-integ-apps and verify creation of
        all StorageClasses. Add new pvc and pod to use the
        new StorageClasses and verify pod is running with no errors.

Closes-bug: 1999169

Signed-off-by: Felipe Sanches Zanoni <Felipe.SanchesZanoni@windriver.com>
Change-Id: I521450cf92fa914aec4c1b7fabd7840a3d984954
Signed-off-by: Hediberto Cavalcante da Silva <hediberto.cavalcantedasilva@windriver.com>
2022-12-15 14:05:22 -05:00
Leonardo Fagundes Luz Serrano
2080d05721 appfwk: set flux reconciliation interval to 1m
Set reconciliation interval for all flux helm resources to 1m
to allow it to manage resources by itself in a reasonable time
interval.

Test Plan (tested as part of [1]):
PASS: bootstrap
PASS: unlocked enabled available
PASS: apps applied
PASS: inspect flux pod logs for errors
PASS: re-test known trigger for 1996747 and 1995748
PASS: re-test known trigger 1997368

[1] https://review.opendev.org/c/starlingx/config/+/866862

Partial-Bug: 1999032

Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I4b7b663107a96427b87162fbf1c68b26e4e8fc01
2022-12-07 19:31:05 +00:00
Leonardo Fagundes Luz Serrano
f1524ae5d9 Remove unnecessary version restriction
One of the build configurations is listing '1.0' as the specific
version to use for one of its files, but this restriction is
unnecessary and causes the build to fail when the app version
is not 1.0.

Test Plan:
PASS build app using a version different than '1.0'
     (modify meta_data.yaml and changelog)

Closes-Bug: 1998637

Signed-off-by: Leonardo Fagundes Luz Serrano <Leonardo.FagundesLuzSerrano@windriver.com>
Change-Id: I9e77b570262658dbed4570fa378a4ceeb4634aa4
2022-12-02 18:12:08 +00:00
Yue Tao
f98ebdeb83 platform-armada-app: update debian_iso_image.inc
Move the packages of "platform-armada-app" from stx-std.lst
to debian_iso_image.inc

A subsequent commit will be sent to clean up stx-std.lst.

Test Plan:

Pass: build-pkgs -c -a
Pass: build-image
Pass: boot

Story: 2008862
Task: 46923

Signed-off-by: Yue Tao <yue.tao@windriver.com>
Change-Id: I95c74db378073714ffbc26638d44c571128fa9d2
2022-11-23 11:11:19 +08:00
Hediberto Cavalcante da Silva
69c37e9978 Migration to ceph-csi for RBD/CephFS provisioners
Remove old RBD/CephFS provisioners and replace with a currently
supported and evolving set of provisioners based on
https://github.com/ceph/ceph-csi version 3.6.2.

Test Plan:
PASS: AIO-SX app upload/apply/remove/delete/update
PASS: AIO-DX app upload/apply/remove/delete
PASS: Storage 2+2+2 app upload/apply/remove/delete
PASS: Create pvc using storageclass general (rbd) on SX/DX/Storage
PASS: Create pod using rbd pvc on SX/DX/Storage
PASS: Create pvc using storageclass cephfs on SX/DX/Storage
PASS: Create pod using cephfs pvc on SX/DX/Storage

Story: 2009987
Task: 45050

Signed-off-by: Hediberto Cavalcante da Silva <hediberto.cavalcantedasilva@windriver.com>
Change-Id: Iffcd56f689aa70788c4c2abbbf2c9a02b5a797cf
2022-11-09 20:23:32 -05:00
Al Bailey
baccc223f0 Fix zuul failures during setup
The ubuntu-jammy nodeset gets selected by default
and is causing problems during setup.

Collecting cffi>=1.1
Failed to build cffi

ubuntu-focal seem to work fine.
Will specify the nodeset to be focal to resolve this.

Need to update a file under python-k8sapp-platform in
order to trigger the failing zuul jobs.

Partial-Bug: 1994843

Signed-off-by: Al Bailey <al.bailey@windriver.com>
Change-Id: I18235473c04fbcf97dd5df676f054f34cd18b8b8
2022-10-28 14:02:38 +00:00
Enzo Candotti
48859ba190 Add pre-upgrade helm hooks to StorageClass
After the introduction of FluxCD, there have been issues on
platform-integ-apps auto re-apply on SX to DX migration.

This change adds the following helm hooks that were missing from
StorageClass:
- "helm.sh/hook": "pre-upgrade, pre-install"
- "helm.sh/hook-delete-policy": "before-hook-creation"

Test Plan:

PASS: CentOS - Create platform-integ-apps tgz with these changes and
test the migration.
PASS: Debian - Create platform-integ-apps tgz with these changes and
test the migration.

Closes-bug: 1970443

Signed-off-by: Enzo Candotti <enzo.candotti@windriver.com>
Change-Id: I3aa2e8f6cbb7504f9a4bf368d90f130dd5131106
2022-08-31 17:23:08 +00:00
Lucas Cavalcante
4fac26a7f5 Remove platform-integ-app armada application build
As part of Armada deprecation we need to remove all Armada application
builds for all applications that have been migrated to FluxCD.

This patch removes the armada app build from centos and debian.

TEST PLAN:
PASS: Build centos
PASS: Build debian
PASS: rpm package has no armada tarball
PASS: deb package has no armada tarball
PASS: fluxcd tarball is unchanged

Story: 2009138
Task: 46046

Signed-off-by: Lucas Cavalcante <lucasmedeiros.cavalcante@windriver.com>
Change-Id: Ic1d31c3aba2e6c43c436f679f0a3f4c8479757a5
2022-08-24 11:41:26 -03:00
Thiago Brito
a733c009ff FluxCD: Disable HelmReleases instead of charts
This commit makes the platform-app complies with the FluxCD capability
to disable HelmReleases instead of charts.

TEST PLAN
PASS Install stx-platform-helm rpm, update the app and
     enable/disable the charts via helm-chart-attribute-modify
PASS Verify resources created with app enabled and disabled

LOGS: https://paste.opendev.org/show/b2rRUhTWY6mESmiav6UO/

Story: 2009138
Task: 45615
Depends-On: https://review.opendev.org/c/starlingx/config/+/845155
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I282737ccc6139b36baccb097933fa9885e1b8d52
vr/stx.7.0
2022-06-13 13:48:31 -03:00