179 Commits

Author SHA1 Message Date
Gage Hugo
68a500b3de Pep8 and Ranger Image Build process update
Updated pep8 requirements and fixed resulting
codes to fix broken pep8 job.

Updated docker image build process to use roles
to ensure docker rather than using manual docker
install process to fix broken ranger image job.

Co-Authored-By: Jeremy Houser <jeremyhouser@protonmail.com>
Co-Authored-By: Chi Lo <cl566n@att.com>

Change-Id: I28df0a27e4b354dd53c17fbb1a9468cb7ff5bc16
2020-09-24 15:26:26 +00:00
Chi Lo
8366b10bef Add customer's domain attribute in CMS API call
When CMS API call is made for either create and update customer, a

A new optional customer domain field is introduced for CMS API call
to create or update customer. The customer domain value will be used
by Rds to set the customer and users domain value when generating the
heat template.  In the event customer domain is not provided by the
CMS API call, a default value will be retrieved from the config file.

Change-Id: I58dbae760d56b756b72c5d9eaf7c0b61cf193847
2020-09-01 09:15:04 -07:00
Zuul
88b95a0dcb Merge "RMS direct access to database for region resources" 2020-08-20 18:24:29 +00:00
Chi Lo
330c0ee6a9 RMS direct access to database for region resources
RMS can access database to verify if customer, flavor, and image
recources exist for a particular region.  This avoids the need
to invoke an api call to CMS, FMS, and IMS services while these
services also making api call to RMS itself.

Change-Id: Ic64e848b03a8fd8f078162f59a148b2dfb3322df
2020-08-20 16:56:16 +00:00
jh629g
94170b688e Update CMS for cross domain role assignment
When a user with a domain different than
the domain of the user authorizing heat
to create the resources that ranger defines
is added to a customer template, heat returns
stack validation as the user cannot be found
in the domain of the authorizing user.
Updated to check if use match auth user and
if not, use ranger conf domain as previous

Change-Id: I1406b61c695a5d9a3d94e732b95c2b683c94852b
2020-08-19 17:59:57 +00:00
jh629g
23b9d62dc5 Persist User Domain for role validation in Heat
Heat must have the domain of the user
in order to validate the user for role
assignment. Regardless of if roles are being
changed or assigned, heat always requires
and validates this data when heat receives
a project template. This work persists the user
domain when calls are made to CMS so that
RDS can add this data to project heat templates.

Change-Id: I5a8e72241e68dac730c3522d820a17d926fa3be8
2020-08-03 14:49:15 +00:00
jh629g
7eef9932fd Update Ranger Auth Process for Update Region action
Changed ranger rms authentication to use
auth of site in request rather than site
of the ranger region when authorizing
updating the site

Change-Id: I85899e319b47eb70b1a5569894097a21b57e92c0
2020-06-08 18:06:40 -05:00
Haider, Nafiz (nh532m)
efb8397228 Update activate/deactivate to single Active boolean property for image
Change-Id: Ie5e2d387c1aff7ee78c65bd5cbff18e81fa26c72
2020-05-18 21:27:11 +00:00
st6218
f9ce6832f2 Refactor uuid handling for Ranger service resources
This commit provides the following uuid management:

(1) Remove uuid record from database after the uuid owned by
a Ranger service (cms, fms, ims) resource is deleted.

(2) Remove stale uuid record from database when such
record is inadvertently created during unsuccessful Ranger
service resource creation.

(3) Add authentication for uuidgen service delete uuid API.

Change-Id: Ifebd68851ff6c0996e7b19d331f4dd99682bd7d2
2020-05-08 11:31:58 -07:00
hosingh000
36bacfb2cf Include attributes of Image
Change-Id: Ic602f4584646faaa3a1fbc7cd2663b46f3c60ba0
2020-04-27 14:04:07 +00:00
jh629g
42f1c52620 Collapse policy.json files
RMS has three rules which were not included into
the top level policy.json. These rules have been
moved to the top level in order to fix 403 against
uuid server. RMS specific policy.json was not
pointed at by any code, so with this fix
the file has been removed.

Change-Id: I8bf507a6336b8b07885e3e11490f1324bf29c5d4
2020-04-22 13:23:36 -05:00
st6218
c0b491797e Minor fix to create private flavor tenant
Change-Id: Ib4f4933e6f3a8ee296d58610cb29d97599632326
2020-04-17 13:46:59 -07:00
Zuul
93d0818457 Merge "Additional changes to private flavors" 2020-04-16 21:16:45 +00:00
st6218
385b4b7723 Additional changes to private flavors
add_tenants logic:
a) allow add tenant only if flavor already assigned to region(s)

b) each tenant in the request will be validated against the
   regions assigned to flavor - at least ONE tenant must
   pass validation:
   i. if NO tenant in tenant list is associated with any of the
      regions assigned to the flavor, Ranger will reject the
      request entirely and user will be prompted to submit new
      request with valid tenants.
  ii. only the tenants in tenant list that pass validation will
      be kept in flavor tenant list;  those that failed
      validation will be DROPPED from the tenant list.

delete_region logic (only for flavors with tenants):
a) if a tenant is associated only with the deleted region, the
   delete_region logic will delete the tenant from fms table
   and the tenant is dropped from the tenant list.

   However, if the tenant is associated with other regions still
   assigned to the flavor, the tenant stays in the list.

Change-Id: I31935477733c8597741cf7c7c57350ab1e2b4452
2020-04-16 13:38:41 -07:00
Zuul
b01a5e8384 Merge "Shorten length of event details during image creation" 2020-04-16 19:15:41 +00:00
jh629g
ebd3e448d6 Shorten length of event details during image creation
Fixed image create event_details to not be in excess
of allotted database size parameters.

Change-Id: Id8771f4be777d98ef3e058ed20a552d1398087c9
2020-04-16 16:57:43 +00:00
hosingh000
efd73bbd37 Change to use rms url
Change-Id: Ia505e68918f98a16253455d08c5db391510e3e0e
2020-04-15 13:04:15 -05:00
Chi Lo
4dbd0e4058 Allow i2 option to be independent option for flavor creation
Change-Id: I5d0f27544fa40d43d4b416d1d8e11d107ec44176
2020-03-20 13:35:37 -07:00
st6218
0d9596317e Update user domain for customer heat template
Change-Id: I1a03e58f96af57fb6737c7c2eedd519bafbefb6d
2020-03-17 12:59:56 -07:00
jh629g
3f76cb63b9 Fix Transactions table breaking when image url too long
When an image is created with a url that is in excess
of 240 characters, the transacations table in ranger
audit database throws error "sqlalchemy.exc.DataError:
(MySQLdb._exceptions.DataError) (1406, "Data too long
for column 'event_details' at row 1")". By removing
this url from the log, we prevent this error while
maintaining other important data. Also restored ranger
to allowed projects in ranger policy.json

Change-Id: I4af73beed9bd40054f8d1e7281bc6f9216fdd243
2020-03-05 15:26:59 -06:00
jh629g
a5aacaa045 Add update statement for updating old sites
Added update statement which will update
old ranger deployment databases to add
ranger region domain name

Change-Id: If0020b8d3e9c5ff2cabfa5d8d6f9656806f2bc6c
2020-02-25 12:55:43 -06:00
Zuul
308b9af12d Merge "Removed RDS service call (cont'd)" 2020-02-24 21:45:51 +00:00
Chi Lo
2cba5155c7 Removed RDS service call (cont'd)
Change-Id: I0c5bebd53faca231f58b0280ef1423ef0d5d50fb
2020-02-24 12:08:22 -08:00
jh629g
93c873bc96 Update RDS to create resources as specified in Ranger Region
Ranger region now includes a field which declares
which openstack domain a resource should be deployed
in. This change updates rds to create resources in
that domain via the heat template

Change-Id: I43144eb75a34661fae15399b9d32842d65327621
2020-02-21 18:07:49 +00:00
Zuul
d026199784 Merge "Removed RDS service call" 2020-02-21 17:45:34 +00:00
Chi Lo
305483dcc2 Removed RDS service call
This patch removes RDS service call to retreive resource status by
CMS, FMS, and IMS Ranger services.  These services retrieve the
status by accessing the resource status database directly.

Change-Id: I7695eb5dca697f3326fa2d19aa45681e4b5fc2ce
2020-02-20 18:03:12 -08:00
Chi Lo
c06ecba646 Purged unused records from uuids table
This patch also removed transaction id hooks when calling
RootController default get.  As a result, uuids record will
not be created.

Change-Id: I2ca0be02256ec62c8f9d48266867cc44121c1c4b
2020-02-19 14:15:41 -08:00
Zuul
c7c6cd725a Merge "Revert "Fix Audit Trail event_details db size"" 2020-02-18 16:45:45 +00:00
Jeremy Houser
1cbfa02d6a Revert "Fix Audit Trail event_details db size"
This reverts commit e5b04669c88868a1966392b7469b039cea9bb370

Change-Id: Ic5aec4848db35bb8a257742981615414b8777dcf
2020-02-18 16:19:10 +00:00
jh629g
5eb91bd5d5 Add Domain name to ranger region
Refactor ranger to allow domain
for resource deployment to be
defined in create region api call,
rather than only being retrieved
from ranger configuration.

Change-Id: I1b481c49d4402438c8aa91a8ebd8b6dbe797997b
2020-02-13 19:14:32 +00:00
Zuul
7ca26d96a5 Merge "Delete region logic and orm script changes" 2020-02-10 16:34:28 +00:00
st6218
9384d4eada Delete region logic and orm script changes
Update orm script to fix the incorrect shebang interpreter.
Cleanup resource_status entry whenever  delete region is executed
from cms (customer), fms (flavor), and ims (image).

Also remove unused programs from ranger.

Change-Id: Ie7b512db2f7d45d1c5beede709ab127d72283a5e
2020-02-10 15:22:54 +00:00
jh629g
e5b04669c8 Fix Audit Trail event_details db size
Current application of audit trail does
not allow for filenames of excess size.
This patchset doubles the allowed space
to allow for files with names of excess
length.

Change-Id: I4d064c67ecd0e77f6e280452eb8afd386445bb6c
2020-02-06 19:34:31 +00:00
jh629g
a41951c6f7 update policy.json
policy.json will be updated to become
monolithic under orm/ directory, maintaining
all previous rules. Also update policy.json to
allow for ranger project as an authorized project
to use for authing through ranger. Also removes
unused git classes.

Change-Id: I547cb601318b507f759c264644c781743673be72
2020-01-30 14:27:25 -06:00
Chi Lo
7767e6bc7f Added API to get and delete uuid from RMS service
Change-Id: I12b86e8b4264205ea7daa6729e37215a2d4dfb79
2020-01-27 11:40:36 -08:00
Chi Lo
5548613de6 Fix Erroneous 'Duplicate Entry' error message
Change-Id: Ic296c493b37c1689e4c24c9402830e6120b8a085
2020-01-23 15:14:30 +00:00
Chi Lo
8f4ffa31e7 Fixed multiple criteria search using metadata and user as filter
This patch fixes the search issue when listing customer with
multiple search criteria using metadata and user parameter as filter.

Change-Id: Ida44ea79df1d1a7186f939b1470f7da52d3cee23
2020-01-21 12:32:51 -08:00
st6218
4a5726139e Private flavor tenant minor fix
Issue error message when tenant list in add_tenant request does not
contain any valid tenant for flavor already assigned to region(s).
Also update corresponding unit tests.

Change-Id: Iea81e812ba26038a06fbb80f7a7ca6d4274c3884
2020-01-16 21:40:27 -08:00
st6218
93d6e9d115 Fix private flavor tnenant issues
Change-Id: I5a89c6e4c497c72ce1856d0dd9aff8799cec07fa
2019-12-20 21:39:47 +00:00
st6218
19b035bfb6 Update resource tables layout structure
Remove  previously-added foreign key in resource_status
table that causes issue with updating resource status table.

Also update resource_template_data table to include region field
and replace transaction_id with resource_id.

Modify RDS logic accordingly.

Change-Id: I71b61b0cfb76545058ef26c0117f007b6d6784d3
2019-12-19 10:39:51 -08:00
st6218
b3541d3a49 Ranger resource create 'group not found' fix
Fix for the issue where Ranger resource (CMS, FMS) create
request failing with 'group not found' error when it is
assigned with region group type.

Change-Id: Ibbd71148153d860dcff4a5512fd3277cb7b39d8c
2019-12-13 22:43:35 +00:00
stewie925
7fb28bbb3b Minor fix for remove git repo logic
Fixed update stack issue where template was not writing
to the database.

Change-Id: I432d6069366b1406422162e66d6a40b5efa4ac67
2019-11-27 19:10:04 +00:00
stewie925
eeb1976e63 Ranger - remove git logic
Update Ranger to remove logic to stage heat template to git repo.
Instead, pass heat template directly to ranger-agent.

Added logic to create/update/delete  heat template entry in rds database.
Remove git repo logic from ranger.
Ranger to pass token to ranger-agent for request authentication.

Reintroduce commit #693590 changes that were inadvertently removed.

Change-Id: If95076e8343bbb50a7231b74fd4a80906b2a0785
2019-11-25 13:03:01 -08:00
Zuul
b2f64e91a5 Merge "Remove Deprecated Features in Ranger" 2019-11-25 19:06:56 +00:00
jh629g
36ebd47daa Remove Deprecated Features in Ranger
As of Stein, injected_files, injected_file_content_bytes,
and injected_path_content_bytes have been deprecated.
This patchset removes those variables.

Change-Id: I0a7fa6860a8f5573ae2435c3740c7fb4ec4bb54e
2019-11-25 12:16:00 -06:00
stewie925
2ccdfdf103 Port Private Flavor Tenant python2 logic changes
Change-Id: Icb7aab24f15c7ec970330d84baf03a9c07f00897
2019-11-22 15:49:10 -08:00
Chi Lo
6f3fb47049 Port from python2: Remove circular dependency on delete resource
python2: https://review.opendev.org/#/c/693399

Change-Id: I0f68235685f03fc73fb848997d5eba5e8915b477
2019-11-21 15:44:50 +00:00
jh629g
9470cc0f0d Ranger fixes for python 3.6 and stein heat
Fixes number of errors so that ranger json
sent to heat matches the parameters that heat
expects for resource allocation

Change-Id: I23d77923756fb7f86e311a6fea89cc7724d63b5b
2019-11-08 14:49:13 -06:00
hosingh000
87d644675e Merge Python3 branch
This change is going to upgrade ranger to use Python 3.x

Change-Id: I563661e071c56c2df7e0e1a6e365aecd4158b6cd
2019-10-29 10:36:00 -05:00
Chi Lo
3f8c56e00b Git push retries when updating SoT
Git push may fail when concurrent git pushes to code cloud occurs.  This
patch provides retries capability when git push failed.

Change-Id: I6cab95cb4caf3bf914c47046a20007570ff634d0
2019-08-30 18:24:39 -07:00