85 Commits

Author SHA1 Message Date
Chi Lo
446168b7bd Display Ranger error response messasge in tempest exception
When resource creation failed, the region's error message is
retrieved and included as part of the tempest exception
message.

Change-Id: I6309b29ad516651b9a89ffe54763d11fb56949a4
2020-11-10 07:14:33 -08:00
Chi Lo
5678f3e210 Refactor retry loop awaiting for image status
Change-Id: If102370ccadaccf9ee9b3309eb273c183a7f13ea
2020-10-02 11:41:07 -07:00
Chi Lo
154583a392 Expand the visibility enum to include shared and community
Change-Id: I25d2e9c369b5802d6b7d7070fc1ae02c0e9e117f
2020-10-02 08:23:25 -07:00
Chi Lo
e123abc10e Refactor image test cases for shared images
Change-Id: Icd8a6417d9c0cd328bda9c6412e43851574f268c
2020-10-01 12:04:43 -07:00
jh629g
bd5acac654 Remove Ranger region create, catch all errrors
Remove ranger region create in favor of
charts performing needed creation. All
errors when ranger tempest attempts to
get region from ranger now mean that Ranger
is not available and we should skip all
tests

Change-Id: Ideb968aeece89b177c591408dbba78a7c2f9e916
2020-09-10 19:11:09 +00:00
Chi Lo
37ea690c81 Catch ConnectionError exception when CH server unavailabe
Change-Id: Idbec6eafd56dea2fc92a5fb7c3bee58c46c00148
2020-09-08 13:29:32 -07:00
Chi Lo
6587fd6efe Remove dependency on RMS's domainName for CMS test cases
CMS will use its own customer domain name instead of
retrieving from RMS's region.  As a result, no RMS's domainName
update is required to alter the domain name prior to the
test case execution.

Change-Id: Ib28f705fe2ba9362da138d88aeacd18f26222daf
2020-09-01 10:16:59 -07:00
jh629g
00681d0deb Fix teardown steps falsely erroring out tests
teardown steps executed at the end
of threadrunners disrupt tests
sometimes. Removed teardown which
would harm other tests, improved
error messages when ranger not
available

Change-Id: I1f01f5dfd928fcb4a8ec77d1fff1f0c9d6fd2dbb
2020-08-29 18:34:18 +00:00
jh629g
8e0975c424 CLLI value required
dummy region requires clli value

Change-Id: Ia339ab4ae598e1a957df684bd44bb46837d2fce8
2020-08-21 03:06:50 -05:00
jh629g
91f3af3759 Update Region create and update
format strings in parenthesis in
create statement, update update region
command for domain name of ranger
region

Change-Id: I976f8972faefe87e21bd746a18037403719bb6cf
2020-08-21 06:00:49 +00:00
jh629g
11f1de34a4 Update Ranger Region Create statement
Ranger region create statement
missing mandatory parameter
"locationtype"

Change-Id: Ifd734ccacc0996149c6b9b7b10db836904f0d3d6
2020-08-20 17:26:49 +00:00
Chi Lo
4d2e23a4b0 Removed duplicate class objects during setup
This patch removed inheritance of region group test class from
region test class to avoid duplicate creation of test objects.

Change-Id: Ia21e21fa8940fc9d7521889fbf3d924fc1847e3a
2020-08-18 14:46:56 -07:00
Chi Lo
3db2d2c746 Added skip check when Ranger not reachable
When Ranger is not reachable, test cases will need to be
skipped.  In addition, remove the extra skip check for RMS and IMS
services.  If setup fails for these modules, let the test cases
fail as well.

Change-Id: Ibadb065ef9816cf1cb6ce44e5229753d679f8bf6
2020-08-11 14:07:03 -07:00
Chi Lo
0cf694ada3 Changed format string typo for skip check
This patch also:
(1) changed the order base resource setup is called for RMS
    and IMS test modules
(2) added extra skip check for the RMS and IMS test module

Change-Id: I24332c053250f36f0998dae089f327f3bf1ec34f
2020-08-10 13:29:23 -07:00
jh629g
2f53e76998 Added skipcheck for ranger site availability
Ranger tempest tests must skip safely when
ranger is not deployed at expected endpoint.

Creates ranger region for single site testing
if it is not present.

Skips if ranger alt region testing is chosen
and ranger alt region is not available, as
tempest conf only contains information of
main target for deployment of resources in
order to be able to build that region.

Change-Id: Ib30d6cd9db099b17d068301c52f20b7be5e30ac3
2020-08-06 14:57:43 +00:00
jh629g
8f36b2944f Update FMS to build all clients
moved some clients from test_flavors
to fms_base as all clients used by base
should be built in base

Change-Id: I09064dbb33f0f5785e64d436bc58d6c9618a4f27
2020-07-27 11:47:31 -05:00
jh629g
83f1fc13fd Improved cleanup to prevent project resource leaks
moved cleanup steps to before creation steps for
customer (openstack project) creation

Change-Id: I62d2fc31080f7ab59200543a7736356ce930047b
2020-07-24 18:29:20 +00:00
jh629g
16c59612f1 Update FMS customer creation to include teardown
rather than tearing down customer after customer
call is made, teardown customer used in fms during
customer creation method in fms_base

Change-Id: Ibef84961d6e481b3d06a926ef4008032dc3d25f0
2020-07-24 02:55:47 +00:00
Zuul
71bf8946ec Merge "Remove deprecated scenario image option" 2020-07-22 18:44:04 +00:00
jh629g
eb20502cd4 Added full teardown steps to FMS tests
FMS creates customer resources for 2
tests and must make use of full teardown
process for customer resources

Change-Id: I253d94856776c7e73a5d8ced02a6f77afd6fa6ca
2020-07-22 17:36:20 +00:00
jh629g
93917fba8a Added Project Cleanup in FMS
FMS generates projects for fms tests,
these projects are not properly cleaned
under certain conditions. Ensured projects
always cleaned no matter what.

Change-Id: Ic1eb693667204c5980bbed6bdc24e4098e097977
2020-07-20 21:57:59 +00:00
jh629g
dcb24efebf Update quota value in customer testing
Test fail due to expectation of ten
while receiving value 20. Updated to
10 in customer default and quota testing.

Change-Id: Ic7564904fb02777cb514db82f25587d01900605a
2020-07-17 18:31:24 +00:00
jh629g
8434876e2d Update Sync, update flavor delete tag
update sync to sync from base class for
alt_region tests, update flavor delete
tag test to ensure tag exists before delete

Change-Id: Iea42bc1566203a9959fe74a0ab99fb70f1360119
2020-07-16 19:15:49 +00:00
jh629g
0616e8e574 Updated sync for alt_region tests, improved status loops
Updated synchronization after gaining better
insights into how to utilize oslo_concurrency
module. Updated region resource updates to be
in resource_setup of base.py for ranger tempest
test suite. Improved region status and removal
loops to overall improve functionality.

Change-Id: I17310dcc46697d4a4541d3ee8355f0669c268e8a
2020-07-13 16:38:07 +00:00
Martin Kopec
4dbe3c2f69 Remove deprecated scenario image option
Removing deprecated CONF.scenario.img_dir option as it's beeing removed
from Tempest after ~4 year depreation period [1].

Currently Tempest is throwing a warning if img_dir is still used.
Both behaviors (img_dir and a full pathin img_file) are supported until
Tempest 25.0.0, starting Tempest 26.0.0 a full path in
CONF.scenario.img_file config option will have to be specified.

[1] https://review.opendev.org/#/c/710996/

Related-Bug: #1393881
Change-Id: Ifd02065de88eea1c75825be3178fc9f8d9721a16
2020-07-13 09:15:20 +00:00
jh629g
3ad54cf082 Adjusted FMS tests for fringe failures
Adjusted delete tag test to ensure tags
are correct before test begins. Updated
test build flavor with two regions to
account for an unordered list in results.
Adjusted build interval to a longer period
to allow for slow networks.

Change-Id: I188953a0aab18e0cbd797907bc8d98ee6759e142
2020-06-30 18:37:11 +00:00
Chi Lo
fec00c7117 Modfied create image in region 'down' state to be tested in alt_region
Change-Id: I5c304daf72e0d76e0fa92446318bdeaaf4e5acda
2020-06-29 21:48:15 +00:00
Zuul
6daea23d65 Merge "Fix flavor tag tests to update and restore tags" 2020-06-29 14:25:00 +00:00
jh629g
8bb57f7ed1 Fix flavor tag tests to update and restore tags
Tags were not being restored correctly,
causing 4 flavor tests to fail. Removes
out of date tag restoration method, use
correct method _exec_tag_function.

Change-Id: I855ba643ae212d5186c7a4fe8807265c65227e51
2020-06-26 17:40:51 -05:00
Zuul
4888c06873 Merge "Add more test coverage for Ranger services" 2020-06-26 21:01:35 +00:00
Chi Lo
3aee34ac5f Add more test coverage for Ranger services
Change-Id: Id517254464e50e36d95a26d49ddd1288819e5032
2020-06-25 13:58:39 -07:00
jh629g
67dd3d33bb Reduced vcpu testing limit
ranger of vcpu limit exceeds typical
maximum allowable range of ranger, reduced
range to fit ranger use cases

Change-Id: I3ad61cc6d24e104012e6b59f9ab016413b61056c
2020-06-24 16:24:30 +00:00
jh629g
6d356f4e16 Fix FMS Tempest Tags
Fixed removal of missing variable
for resetting tags in flavor tag
tests.

Change-Id: I18b92c6e7e8a22ef62cb6ce3308368d770b36c83
2020-06-24 15:32:34 +00:00
Zuul
c9dd2c3adb Merge "Adjust flavor tests for unordered list" 2020-06-18 20:19:13 +00:00
jh629g
9117193ee6 Adjust flavor tests for unordered list
List of regions can change between environments,
changes assert statements to allow for this case

Change-Id: I8d44b72940c9a732b617703a116634ed39e6c386
2020-06-18 19:27:14 +00:00
Chi Lo
fc92e28653 Added new test cases for CMS service
Change-Id: I6f09d000652ad3a4967cc16874e23d2cada70cb8
2020-06-18 15:51:47 +00:00
jh629g
a5237ba0bc Update ranger update region statement
Changed 'is' to '==' when comparing string
values due to wanting to compare the values
not the objects in the logical statement

Change-Id: I7e918e49aed0a6cde1adf2ff1c8c73a4e3382111
2020-06-04 14:47:16 -05:00
jh629g
7241704336 Minor fixes for Ranger flavor tests
Fixed test that gave false positives
and false negatives, added skipcheck
to that test. Updated config value to
boolean so that test skipchecks in flavor
tests worked as expected. Added function
for checking the status of individual
regions on a flavor.

Change-Id: I331e12a757b8eb1182cb88a19a8fd9607358143e
2020-06-04 07:55:10 -05:00
jh629g
a4eb88a69c Added skipchecks, updated region domain update setup
Skipcheck implemented for when a test uses multiple
ranger regions to execute a test and tempest did
not provide an additional region to complete these
tests with. Updated region domain update statement
in setup steps to ensure that ranger-tempest-plugin
does not attempt to update a secondary site when
it is not available

Change-Id: Ia19ef442107d7cc66d1f254860606245bf6c127f
2020-06-01 10:39:38 -05:00
jh629g
78f191ddbb Fixed region domainName update in class setup process
During setup of ranger tempest test runs, an update
statement is ran which updates ranger regions to
the appropriate testing domain as defined by
tempest.conf. This update statement has been updated
to target only the sites being used for that particular
ranger tempest test run. Also fixed typo in IMS test.

Change-Id: I4ebdc7fd21cebb23e8033a066bbd90d26f76bb50
2020-05-28 18:01:18 +00:00
jh629g
4ccd35516f TEST PATCHSET - REMOVE RBAC
Attempt to remove rbac tests to
resolve duplicate idempotent id
issues in tempest pipeline due
to import of patrole_tempest_plugin
utilities

Change-Id: Ife08863979e39ecfdabb6b07ac90dad3a5747c9a
2020-05-19 15:03:50 -05:00
Chi Lo
6d7c3c14f5 Add skip_checks for Ims test cases
Add skip_checks for Ims test cases when image_url is not specified
in tempest.conf.

Change-Id: If8149c744c735c972fa013bf860ba96df7bf7707
2020-05-13 12:36:42 -07:00
Zuul
58f63ed9f3 Merge "Revert "Remove RBAC tests"" 2020-04-14 16:20:45 +00:00
Jeremy Houser
0427e9385f Revert "Remove RBAC tests"
as it was a test revert and tests
now need to be fixed as they have been
excluded from cause of failure in testing
framework.

This reverts commit 67c882996e128e38bbe90fe915f9565e89f81600.

Change-Id: Iaecd1afb6662e4c749a7fc028c1fd865b893cbb8
2020-04-14 13:36:14 +00:00
Zuul
d740b8ee15 Merge "Fix Typo, Add Resource Lock" 2020-04-03 19:10:04 +00:00
jh629g
ff87b86724 Fix Typo, Add Resource Lock
Added resource locking to flavor tests
that utilize regions and fixed typo in
name of key for image dictionary in
image region resource status tests

Change-Id: I66e0930d28a4e0e03af347a10913c9fef1c351bf
2020-04-03 18:42:58 +00:00
jh629g
ab1437b2fa Fix bad client initialization
Testing of region groups had badly
named client. This resolves client
to be rms_client for region group
testing

Change-Id: If7bc501b450cdbfa47dcf90c5a97c9507ec5188d
2020-04-02 13:23:30 -05:00
jh629g
38de76fe6b Fix domain_name -> domainName
Fixed use of incorrect variable
in region dictionary for update
to testing domain

Change-Id: Ic522a2640cdf0bf8e88783aef1c6b4572aeb4d77
2020-04-01 12:06:20 -05:00
jh629g
b167877470 Updating region create to region update
tempest tests use region create statement to ensure
regions exist for testing purposes. ranger-agent
charts now include automation during deployment for
ranger regions, so these statements have been changed
to update statements to ensure region is pointing at
testing domain

Change-Id: I3024145c5b388d85b2aa78c223bd47e486fbd157
2020-03-26 17:34:10 -05:00
jh629g
67c882996e Remove RBAC tests
Removing rbac tests to see if cause of testing failures

Change-Id: I2ed22ddded8ad5ee5da11d5ffe1139cca1996d51
2020-03-24 15:09:29 -05:00