127 Commits

Author SHA1 Message Date
Daniel Caires
b5768ae359 Review LOG levels
Adjust LOG levels throughout the code.

Story: 2005051
Task: 48516

Change-Id: Ie54a0d6a50c373ebc8b1d272aa5b574d5821b399
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-09-25 12:24:55 +00:00
Zuul
a41fffdefc Merge "Fix worker VMs configurations" 2023-09-21 13:01:13 +00:00
Zuul
f0c9c0dfb6 Merge "Update mirror URL" 2023-09-21 12:52:02 +00:00
Zuul
f57a99f29c Merge "Automatized stx-openstack install for AIO-SX" 2023-09-08 18:08:07 +00:00
Bruno Muniz
0fd3ffeff5 Small parser argument fix
Additional changes were necessary. Now it's possible to use either
-y or --yes-to-all to get a 'y' answers on all yes/no questions.

The result of the parser is explicitly stored on a variable called
'y' on the namespace that the argparse returns.

Task: 48626
Story: 2005051
Change-Id: I4760ea2f1f77bee7c11cf7e0e76ee2566f383b19
2023-09-05 11:51:09 -03:00
Daniel Caires
1523b70ba2 Fix worker VMs configurations
Change throughout the code every mention of
"compute" as a node to "worker".

Test Plan:
PASS: worker nodes are being configurated as expected
PASS: worker nodes are being deleted when portion of
the code is executed

Story: 2005051
Task: 48726

Change-Id: I6dda10c3b188f14e03dba514cb2063c4c7d1dda4
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-09-05 12:45:37 +00:00
Tomás Barros
c56230011f Automatized stx-openstack install for AIO-SX
Users can now install stx-openstack in an automated manner
by using the --openstack-helm-location parameter when running
the script installation, and by increasing the node storage
to at least 360 GB.

Story: 2005051
Task: 48149
Change-Id: I2eade54dca3328fa5fe5282355e9a1b9f33b059f
Signed-off-by: Tomás Barros <tomas.barros@encora.com>
2023-09-04 18:48:05 -03:00
Daniel Caires
6eb3fe0a36 Update mirror URL
Update the ISO download URL in the README file from
mirror.starlingx.cengn.ca to mirror.starlingx.windriver.com
as StarlingX ISO images are no longer hosted in CENGN

Change-Id: I27998624fbafbbbbae961af93c54ef95e52e244c
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-09-04 08:38:48 -03:00
Tomás Barros
0c1a53cebd Small parser argument fix
This review aims to fix a little bug in the
"yes-to-all" argument in the parser file of
the installation script.

Task: 48626
Story: 2005051
Change-Id: Ib3e36a6735ec3a413b27a81c0039f2f03b978f3c
Signed-off-by: Tomás Barros <tomas.barros@encora.com>
2023-08-31 16:02:01 -03:00
Zuul
6fb9a120f3 Merge "Improves compatibility with VirtualBox 7.0" 2023-08-28 14:09:47 +00:00
CaioCorrea
d6d7180e57 Improves compatibility with VirtualBox 7.0
Fixes an issue with overwriting portfowarding rules.
Closes-Bug: 2031930
Task: 48624

Adds a new argument: --y
Automatically answers yes to all prompts improving the autonomy of the installation
Task: 48626

Story: 2005051
Change-Id: I426be7486c59f2163c32227da83924b61e40ea30
2023-08-28 10:40:40 -03:00
Zuul
fa4860c4c6 Merge "Script to show host's and lab's disk usage" 2023-08-25 14:12:21 +00:00
Bailey Henry
364c46f73e Script to show host's and lab's disk usage
Script that assists with supervising host's resources, in this case disk
usage. Lists usage per node and overall usage.

Test plan:
PASS: no tox errors
PASS: information displayed is valid
PASS: does not affect nodes

Story: 2010816
Task: 48637

Change-Id: If874ffc3f02faa26521d305b1b6f5fe62096f06c
Signed-off-by: Bailey Henry <Henry.Bailey@windriver.com>
2023-08-25 13:50:28 +00:00
Michel Thebeau
43de3e6910 libvirt: fix readconfig.sh usage and permissions
readconfig.sh intends to export variables to the current shell, and
should be sourced using the 'source' command.  It is ineffective when
run as a sub-process.

Fix the file permission, since the script should be sourced rather than
executed.

Test Plan:
PASS: regression of default and madcloud examples
PASS: documented usage is effective

Story: 2010816
Task: 48614

Change-Id: I7e35ccf253e27b8209f8bf17dca065c58b016e14
Signed-off-by: Michel Thebeau <michel.thebeau@windriver.com>
2023-08-16 09:53:38 -04:00
Michel Thebeau
14279bfcba libvirt: move xml files to an xml subdirectory
The libvirt/ directory is a bit cluttered.  Move xml files to a
subdirectory.

Test plan:
PASS: Regression for default and mad examples
PASS: unit test of the affected replacement values for all
      configurations

Story: 2010816
Task: 48613

Change-Id: I7a8552616cce9ac4639d3c2c67d5a019f7f06bdc
Signed-off-by: Michel Thebeau <michel.thebeau@windriver.com>
2023-08-16 09:52:28 -04:00
Michel Thebeau
5f1da7dbdc libvirt: standardize sed replacement strings
The original code inconsistently uses replacement strings either limited
to capital letters or with additional percentage symbols.  The use of
percentage symbols is preferred to mitigate the possibility of
accidental replacements.

Standardize the sed replacement strings to use percentage symbols.

Test plan:
PASS: Regression for default and mad examples
PASS: unit test the replacement values for all configurations

Story: 2010816
Task: 48611

Change-Id: I35959d7ae8cd46e613a1d0b3a1fcb44fc3f0b1d6
Signed-off-by: Michel Thebeau <michel.thebeau@windriver.com>
2023-08-16 09:47:31 -04:00
Bailey Henry
2066a90fa5 Libvirt: CPU and Memory value configurable
Add the ability for the user to change how many cores and memory are
allocated per node. Each node may have any number of cores or memory
set, with which their values are used by sourcing the file:
'source readconfig.sh <yaml file>'

Test plan:
PASS: regression tests passed
PASS: sanity tests passed
PASS: no tox, flake8 or pylint errors
PASS: value succesfully set from config file
PASS: defaults used when no config file is sourced

Story: 2010816
Task: 48398
Task: 48586

Change-Id: Ia2f7df44c872fac41ac6376ef3fb00062624ac22
Signed-off-by: Bailey Henry <Henry.Bailey@windriver.com>
2023-08-16 08:54:23 -04:00
Michel Thebeau
8c64195f67 libvirt: update QCOW image detection
The script enforces the kind of file it will delete to prevent
accidents.  Update the detection of QCOW disk images for QCOW2.

On an updated Ubuntu the file type is "QCOW2" instead of "QCOW".

Change-Id: If6ab7113358f96749dadae53b52ae31e8f452777
Signed-off-by: Michel Thebeau <michel.thebeau@windriver.com>
2023-08-14 14:18:19 +00:00
Zuul
dcef4d4bf9 Merge "Review lab-setup files" 2023-08-11 13:50:12 +00:00
Roger Ferraz
0389ebc90a Review lab-setup files
The lab-setup files are refactored, as detailed below.
In addition, recovery, administration and logging improvements are
implemented.

The following lab-setup files are removed:

- lab_setup1.sh
- lab_setup2.sh

The corresponding code, previously run locally in the VM, is now
integrated to the main Python code.

The files lab_setup.sh and lab_setup.conf are kept, because they
are useful to populate the stx-openStack application. These should be
reviewed by a new task under the context of stx-openStack.

Test Plan
  - AIO-SX Virtual Deployment (PASS)
  - AIO-DX Virtual Deployment (PASS)

Story: 2005051
Task: 48402

Change-Id: I940e5a16ea98a4325efe1ee0dd45127674d6b192
Signed-off-by: Roger Ferraz <rogerio.ferraz@encora.com>
2023-08-10 11:13:47 -03:00
Bailey Henry
02f67a5746 libvirt: Add override variable for DISK variables
Add an override variable for the DISK file location for each node in a
virtual lab. Introduced the ability to set disk file location for all
nodes in a config file. (was hardcoded to /var/lib/libvirt/images)

This enhancement enables better utilization of the host's disk
resources, allowing for more complex deployments such as
'Distributed Cloud'.

Test plan:
PASS: pylint, flake8, bashate
PASS: Regression for default and madcloud examples
PASS: Able to change what is storage and how many are applied
PASS: Copy default.yaml and modify values of disk directories for nodes
      and compare to default.yaml

Story: 2010816
Task: 48397

Change-Id: If19404ed151739c4abcbd5f2c3b9fb62132ed6fe
Signed-off-by: Bailey Henry <Henry.Bailey@windriver.com>
2023-08-09 12:04:03 -04:00
Zuul
2462c55035 Merge "Change TCP port value" vf/antelope 2023-08-08 14:13:11 +00:00
Daniel Caires
78497262e7 Change TCP port value
TCP port is being uptaded to a value based on the
ssh port of controller-0

Test Plan:
PASS: VMs created with TCP port based on the ssh port
of controller-0

Story: 2005051
Task: 48375

Change-Id: I67d284a4e41a42de641670eb3bc7326bdcc432c9
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-08-04 11:56:30 -03:00
Bailey Henry
2d8e2fbaae libvirt: Yaml validator
Test plan:
    PASS: No pylint errors
    PASS: No pep8 errors
    PASS: All data validation accurate
    PASS: Misc Fixes relating to efficiency

Story: 2010816
Task: 48424

Change-Id: I54b48ca0f05c48989c5ae8fb7d18c7ae12d2faad
Signed-off-by: Bailey Henry <Henry.Bailey@windriver.com>
2023-07-25 14:42:20 +00:00
Michel Thebeau
7ff7e09392 libvirt: add pep8/flake8 to tox job
Using config/sysinv/sysinv/sysinv as example of setup for pep8/flake8.

Fix reported errors for config.py.

Story: 2010816
Task: 48450

Change-Id: Ia0e511fd827b5d697a6b41b7064d13ab64efb779
Signed-off-by: Michel Thebeau <Michel.Thebeau@windriver.com>
2023-07-24 17:15:45 -04:00
Zuul
f5407b51a7 Merge "Changes to make AIO-DX setup work" 2023-07-21 13:08:28 +00:00
Daniel Caires
d3f4d23f59 Changes to make AIO-DX setup work
-Minimal changes so the script works for the
installation of a AIO-DX setup type.
-Addition of a lab_setup2 for the configuration
of the controller-1.
-Change of serial port configuration so VMs can boot
without the need of a socat connection.

Regression: AIO-SX successfully provisioned using
modified code.

Test Plan:
PASS: StarlingX is succesful deployed in a AIO Duplex
configuration.
PASS: Both controllers are set with TCP as Port Mode
in serial ports configuration.

Story: 2005051
Task: 48261
Task: 48275

Change-Id: I5fd5c6d413270867424a30768b0ad7ff91d296b8
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-07-20 14:15:40 -03:00
Zuul
af03b857e1 Merge "Lock Kubernetes Dashboard version to 2.7.0" 2023-07-19 20:05:40 +00:00
Bailey Henry
92d5562f6d libvirt: Add configuration files for mad and default
yaml files for mad and default

Test plan:
    PASS: Create configuration files with all needed data
    PASS: Ensure all data is usable
    PASS: No Pylint/Bashate errors
    PASS: Regression test succeeded

Story: 2010816
Task: 48350

Change-Id: Id7a380a64348501396b139ebb57ff0f42825ffce
Signed-off-by: Bailey Henry <Henry.Bailey@windriver.com>
2023-07-18 14:46:57 +00:00
Michel Thebeau
4eba315b7e add libvirt to tox.ini pylint
Story: 2010816
Task: 48394

Test Plan:
PASS: add errors in libvirt bash and py files and observe tox output

Change-Id: Ie8baba41075360044859dc3c60f44d9411f5cb10
Signed-off-by: Michel Thebeau <Michel.Thebeau@windriver.com>
2023-07-14 17:18:36 -04:00
Bruno Muniz
557b083e09 Lock Kubernetes Dashboard version to 2.7.0
- Lock Kubernetes Dashboard version in automated install
- Lock StarlingX ISO version to download on the README instructions

The new Kubernetes Dashboard version (3.0.0) has a breaking change
which is related to its dependencies of nginx and cert-manager.

This commit locks the Kubernetes Dashboard version to 2.7.0, which
corresponds to the Helm Chart version 6.0.8. There's a new Task,
number 48393, to investigate all the changes necessary to allow a
new version of the Kubernetes Dashboard to be used.

Test Plan:

PASS: Verify end-to-end installation for AIO-SX

Closes-Bug: 2027722
Change-Id: I1330a133f344746b791067b1b12c29029be238f5
Signed-off-by: Bruno Muniz <bruno.muniz@encora.com>
2023-07-14 16:32:05 -03:00
Zuul
84e1231f06 Merge "libvirt: Add DOMAIN_DIRECTORY" 2023-07-14 13:38:33 +00:00
Bailey Henry
52c25ec086 libvirt: Add DOMAIN_DIRECTORY
Removes the hardcoded value for the domain directory 'vms'
While having it default to 'vms'

Test plan:
    Value of DOMAIN_DIRECTORY is overridable
    Default value of DOMAIN_DIRECTORY is still 'vms'
    Regression tests pass

Story: 2010816
Task: 48349

Change-Id: Ic4e79ea6dfaa1134070b9d7dfff23b83c7c8a1e2
Signed-off-by: Bailey Henry <Henry.Bailey@windriver.com>
2023-07-14 13:15:02 +00:00
Bailey Henry
a4e128d890 libvirt: Regression tests for mad and default
README with instructions for regression test

Test plan:
    All information necessary is documented
    All information necessary described accurately
    All information is formatted correctly (.rst syntax)

Story: 2010816
Task: 48345

Change-Id: Ibc543debb235a2865242a07c2c0e08b49a36b2fb
Signed-off-by: Bailey Henry <Henry.Bailey@windriver.com>
2023-07-14 13:10:25 +00:00
Daniel Caires
708c2c1676 Update README
Update the link in the README file to point
to the new repository

Change-Id: Ibcddf12385e5d45bcb26cadfe8d993b90fbc0370
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-07-12 11:23:22 -03:00
Zuul
3a903d4899 Merge "Assigned local port doesn't stop installation" 2023-07-05 17:02:05 +00:00
Zuul
ee916b1f39 Merge "Update hardware values for VM" 2023-07-05 14:53:14 +00:00
Daniel Caires
7b07a8a6b2 Assigned local port doesn't stop installation
Ensure that if given local port is already in use,
rather than stopping installation, the script asks
if the user would like to overwrite or no the currently
rule using the local port.

Test Plan:
PASS: User press y to override the rule
PASS: User press n or any other key to ignore the creation
of a port forwarding rule and continue installation

Story: 2005051
Task: 48091

Change-Id: Iaabf0d5b713cf6d73a9fccdcf7da6962ac5620c2
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-07-05 11:24:17 -03:00
Lindley Werner
267e025a84 Fix yaml config file for kubernetes dashboard installation
Task: 48328
Story: 2005051

Change-Id: I7f004db01f72bb411c7140717bbc59ff5829b844
Signed-off-by: Lindley Werner <lindley.vieira@encora.com>
2023-07-04 15:12:17 -03:00
Zuul
e45f694bc6 Merge "Revert "Added enum type in the parser file"" 2023-07-04 15:27:06 +00:00
Bruno Drugowick Muniz
1081624635 Revert "Added enum type in the parser file"
This reverts commit 9ff33116f230caac0442de835462456ef4533eec.

Reason for revert: although test plan states that installation works,
it's currently failing after merging this.

Change-Id: I57e3c67c927529508e6472557b373f094d9c0f1b
2023-07-04 13:16:46 +00:00
Lindley Werner
d65811f2d5 Adding unit tests in pybox python scripts.
Enabling automatic tests with tox and zuul for each new patchset.

To see the unit test logs, go to:
  1- Zuul Summary
  2- tox-unittests
  3- Logs
  4- job-output.txt

Test Plan:
PASS: Run "tox -e unittests" in the terminal, this will:
  - Set the PYTHONPATH environment variable
  - Run the tests
  - Show the coverage report

Task: 47929
Story: 2005051

Change-Id: I7f527860f3498c53b28691c654035d017d70f68b
Signed-off-by: Lindley Werner <lindley.vieira@encora.com>
2023-07-03 15:37:12 -03:00
Zuul
0d5855a7d9 Merge "Initialize Ceph-Storage" 2023-07-03 15:12:45 +00:00
Daniel Caires
6bdeea2f40 Initialize Ceph-Storage
Add a step in lab_setup1 to initialize a
host-based Ceph solution

Test Plan:
PASS: Run STG_LAB_SETUP1 and initilize a
host-based ceph storage on controller0

Story: 2005051
Task: 47936

Change-Id: Iab89ef8be22e4940ae44dab2c8da28407145685d
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-07-03 11:50:59 -03:00
Tomás Barros
9ff33116f2 Added enum type in the parser file
added enum type with the aim to use a circuit-breaker logic implemented
by the argparser itself, with a fail-fast behavior when trying to cast
the input parameters with values different than the ones expected.

Test Plan:
PASS: Run install_vbox.py with expected values for modified parameters

Failure Path:
PASS: Run install_vbox.py with non-expected values fails right away

Task: 47907
Story: 2005051
Change-Id: I30d283947200019d2472a7fdd8804026db924155
Signed-off-by: Tomás Barros <tomas.barros@encora.com>
Signed-off-by: Bruno Muniz <bruno.muniz@encora.com>
2023-07-03 11:32:31 -03:00
Daniel Caires
252481411f Add --headless argument in Parser.py
Allow user to choose a headless start
for the VMs

Story: 2005051
Task: 47899
Change-Id: Iec6ab9ea2a2b0544f99cd6a4028173706a9c4896
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-07-03 10:44:34 -03:00
Daniel Caires
0fce663636 Kubernetes dashboard installation stage
-Add a new stage that install Kubernetes
dashboard and saves a file with the access
token in the $HOME of the user.
-Change config files folder structure.

Test Plan:
PASS: kubernetes dashboard is successfully initiated
PASS: token.txt is sent to $HOME

Story: 2005051
Task: 47937

Change-Id: Id5a872e14c2620e6064e8593ce28d177a113ffaf
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-06-30 15:59:07 -03:00
Daniel Caires
56aeb98f07 Set versions in requirements
Set the package versions in the requirement.txt file

Story: 2005051
Task: 48077

Change-Id: Ia68a64a35c1d5aa821529125f2bea760d0e17c5a
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-06-30 17:56:39 +00:00
Daniel Caires
e425d5f945 Eliminate parameter redundancies
-stx_config.ini_centos file and all
 code related to it deleted.
-localhost.yml updates ansible_become_pass
 value based on password parameter.
-Ip's used in the VM are now set from the
 localhost.yml
-ruamel.yaml added to requirementes.txt

Test Plan:
PASS: localhost.yml is succesfully overriten
PASS: ip variables are set from localhost.yml

Story: 2005051
Task: 47953

Change-Id: Ieb5fe5d6c0a7da5ef1b6aa7488d6edf48f4cc527
Signed-off-by: Daniel Caires <daniel.caires@encora.com>
2023-06-30 14:50:35 -03:00
Bruno Muniz
07051a09aa Remove all hard-coded passwords and require it
- Additionally, add password validation function.
- Breaks Parser.py into more manageable functions.
- Changes where defaults are set (Parser.py instead of class under
/consts)

The goal is to not have any reference to a password in the code
itself, only in configuration files or README files, if strictly
necessary.

The new password validation function, besides failing fast at the
argument parser, makes sure the password will be allowed by Debian
later on the installation.

This commit is not touching any Shell script because they will
probably be removed or change in follow-up commits.

Test Plan:
PASS: Operating system password should be set to value passed via
--password

Failure Path:
PASS: Script fails fast without the parameter --password
PASS: Script fails fast if password requirements are not met

Regression:
PASS: AIO-SX install works

Story: 2005051
Task: 47960
Task: 48230

Change-Id: Ibf42b792ef825cee61cc69d1b5afa807361037b7
Signed-off-by: Bruno Muniz <bruno.muniz@encora.com>
2023-06-30 14:28:02 -03:00