airship/drydock
Code Issues Proposed changes

Merge "Add config generation to tox.ini Move sample config to etc/drydock Update docs to generate a config with tox"

Browse Source
This commit is contained in:
Bryan Strassner 2017-08-23 11:40:43 -04:00 committed by Gerrit Code Review
commit c71e76aac2
38 changed files with 1387 additions and 253 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.dockerignore Normal file
View File

@ -0,0 +1 @@
.tox

11
README.md
View File

@ -1,17 +1,22 @@
# drydock_provisioner
A python REST orchestrator to translate a YAML host topology to a provisioned set of hosts and provide a set of cloud-init post-provisioning instructions.
To build and run, first move into the root directory of the repo and run:
$ tox -e genconfig
$ tox -e genpolicy
$ sudo docker build . -t drydock
$ sudo docker run -d -v $(pwd)/examples:/etc/drydock -P --name='drydock' drydock
$ vi etc/drydock/drydock.conf # Customize configuration
$ sudo docker run -d -v $(pwd)/etc/drydock:/etc/drydock -P --name='drydock' drydock
$ DDPORT=$(sudo docker port drydock 8000/tcp | awk -F ':' '{ print $NF }')
$ curl -v http://localhost:${DDPORT}/api/v1.0/designs
To be useful, Drydock needs to operate in a realistic topology and has some required
downstream services.
See [Configuring Drydock](docs/configuration.rst) for details on customizing the configuration. To be useful, Drydock needs
to operate in a realistic topology and has some required downstream services.
* A VM running Canonical MaaS v2.2+
* A functional Openstack Keystone instance w/ the v3 API
* Docker running to start the Drydock image (can be co-located on the MaaS VM)
* A second VM or Baremetal Node to provision via Drydock
* Baremetal needs to be able to PXE boot

51
docs/configuration.rst Normal file
View File

@ -0,0 +1,51 @@
===================
Configuring Drydock
===================
Drydock uses an INI-like standard oslo_config file. A sample
file can be generated via tox::
$ tox -e genconfig
Customize your configuration based on the information below
Keystone Integration
====================
Drydock requires a service account to use for validating client
tokens::
$ openstack domain create 'ucp'
$ openstack project create --domain 'ucp' 'service'
$ openstack user create --domain ucp --project service --project-domain 'ucp' --password drydock drydock
$ openstack role add --project-domain ucp --user-domain ucp --user drydock --project service admin
The service account must then be included in the drydock.conf::
[keystone_authtoken]
auth_uri = http://<keystone_ip>:5000/v3
auth_version = 3
delay_auth_decision = true
auth_type = password
auth_section = keystone_authtoken_password
[keystone_authtoken_password]
auth_url = http://<keystone_ip>:5000
project_name = service
project_domain_name = ucp
user_name = drydock
user_domain_name = ucp
password = drydock
MaaS Integration
================
Drydock uses Canonical MaaS to provision new nodes. This requires a running MaaS
instance and providing Drydock with the address and credentials. The MaaS API
enforces authentication via a API key generated by MaaS and used to sign API calls.
Configure Drydock with the MaaS API URL and a valid API key.::
[maasdriver]
maas_api_url = http://<maas_ip>:<maas_port>/MAAS
maas_api_key = <valid API key>

13
docs/getting_started.rst
View File

@ -33,14 +33,14 @@ Clone the git repo and customize your configuration file
::
git clone https://github.com/att-comdev/drydock
mkdir /tmp/drydock-etc
cp drydock/examples/drydock.conf /tmp/drydock-etc/
cp -r drydock/examples/bootdata /tmp/drydock-etc/
cd drydock
tox -e genconfig
cp -r etc /tmp/drydock-etc
In `/tmp/drydock-etc/drydock.conf` customize your maas_api_url to be
In `/tmp/drydock-etc/drydock/drydock.conf` customize your maas_api_url to be
the URL you used when opening the web UI and maas_api_key.
When starting the Drydock container, /tmp/drydock-etc will be
When starting the Drydock container, /tmp/drydock-etc/drydock will be
mounted as /etc/drydock with your customized configuration.
Drydock
@ -51,7 +51,7 @@ You will need to customize and mount your configuration file
::
$ sudo docker run -v /tmp/drydock-etc:/etc/drydock -P -d drydock:latest
$ sudo docker run -v /tmp/drydock-etc/drydock:/etc/drydock -P -d drydock:latest
Configure Site
--------------
@ -77,4 +77,3 @@ Use the CLI to create tasks to deploy your site
$ drydock --token <token> --url <drydock_url> task create -d <design_id> -a prepare_site
$ drydock --token <token> --url <drydock_url> task create -d <design_id> -a prepare_node
$ drydock --token <token> --url <drydock_url> task create -d <design_id> -a deploy_node

1
drydock_provisioner/cli/design/actions.py
View File

@ -37,7 +37,6 @@ class DesignCreate(CliAction): # pylint: disable=too-few-public-methods
self.base_design = base_design
def invoke(self):
return self.api_client.create_design(base_design=self.base_design)

16
drydock_provisioner/config.py
View File

@ -36,6 +36,10 @@ import pkgutil
from oslo_config import cfg
import keystoneauth1.loading as loading
IGNORED_MODULES = ('drydock', 'config')
class DrydockConfig(object):
"""
Initialize all the core options
@ -54,12 +58,6 @@ class DrydockConfig(object):
cfg.StrOpt('control_logger_name', default='${global_logger_name}.control', help='Logger name for API server logging'),
]
# API Authentication options
auth_options = [
cfg.StrOpt('admin_token', default='bigboss', help='X-Auth-Token value to bypass backend authentication', secret=True),
cfg.BoolOpt('bypass_enabled', default=False, help='Can backend authentication be bypassed?'),
]
# Enabled plugins
plugin_options = [
cfg.MultiStrOpt('ingester',
@ -95,17 +93,15 @@ class DrydockConfig(object):
def register_options(self):
self.conf.register_opts(DrydockConfig.options)
self.conf.register_opts(DrydockConfig.logging_options, group='logging')
self.conf.register_opts(DrydockConfig.auth_options, group='authentication')
self.conf.register_opts(DrydockConfig.plugin_options, group='plugins')
self.conf.register_opts(DrydockConfig.timeout_options, group='timeouts')
self.conf.register_opts(loading.get_auth_plugin_conf_options('password'), group='keystone_authtoken')
IGNORED_MODULES = ('drydock', 'config')
config_mgr = DrydockConfig()
def list_opts():
opts = {'DEFAULT': DrydockConfig.options,
'logging': DrydockConfig.logging_options,
'authentication': DrydockConfig.auth_options,
'plugins': DrydockConfig.plugin_options,
'timeouts': DrydockConfig.timeout_options
}
@ -115,6 +111,8 @@ def list_opts():
module_names = _list_module_names(package_path, parent_module)
imported_modules = _import_modules(module_names)
_append_config_options(imported_modules, opts)
# Assume we'll use the password plugin, so include those options in the configuration template
opts['keystone_authtoken'] = loading.get_auth_plugin_conf_options('password')
return _tupleize(opts)
def _tupleize(d):

1
drydock_provisioner/control/api.py
View File

@ -28,6 +28,7 @@ def start_api(state_manager=None, ingester=None, orchestrator=None):
state persistence
:param ingester: Instance of drydock_provisioner.ingester.ingester.Ingester for handling design
part input
:param orchestrator: Instance of drydock_provisioner.orchestrator.Orchestrator for managing tasks
"""
control_api = falcon.API(request_type=DrydockRequest,
middleware=[AuthMiddleware(), ContextMiddleware(), LoggingMiddleware()])

59
drydock_provisioner/control/base.py
View File

@ -11,18 +11,19 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import falcon.request as request
import uuid
import json
import logging
import falcon
import falcon.request
import drydock_provisioner.error as errors
class BaseResource(object):
def __init__(self):
self.logger = logging.getLogger('control')
self.authorized_roles = []
def on_options(self, req, resp):
self_attrs = dir(self)
@ -36,18 +37,6 @@ class BaseResource(object):
resp.headers['Allow'] = ','.join(allowed_methods)
resp.status = falcon.HTTP_200
# For authorizing access at the Resource level. A Resource requiring
# finer grained authorization at the method or instance level must
# implement that in the request handlers
def authorize_roles(self, role_list):
authorized = set(self.authorized_roles)
applied = set(role_list)
if authorized.isdisjoint(applied):
return False
else:
return True
def req_json(self, req):
if req.content_length is None or req.content_length == 0:
return None
@ -101,8 +90,8 @@ class BaseResource(object):
class StatefulResource(BaseResource):
def __init__(self, state_manager=None):
super(StatefulResource, self).__init__()
def __init__(self, state_manager=None, **kwargs):
super(StatefulResource, self).__init__(**kwargs)
if state_manager is None:
self.error(None, "StatefulResource:init - StatefulResources require a state manager be set")
@ -115,10 +104,17 @@ class DrydockRequestContext(object):
def __init__(self):
self.log_level = 'ERROR'
self.user = None
self.roles = ['anyone']
self.user = None # Username
self.user_id = None # User ID (UUID)
self.user_domain_id = None # Domain owning user
self.roles = []
self.project_id = None
self.project_domain_id = None # Domain owning project
self.is_admin_project = False
self.authenticated = False
self.request_id = str(uuid.uuid4())
self.external_marker = None
self.external_marker = ''
self.policy_engine = None
def set_log_level(self, level):
if level in ['error', 'info', 'debug']:
@ -127,6 +123,9 @@ class DrydockRequestContext(object):
def set_user(self, user):
self.user = user
def set_project(self, project):
self.project = project
def add_role(self, role):
self.roles.append(role)
@ -138,7 +137,23 @@ class DrydockRequestContext(object):
if x != role]
def set_external_marker(self, marker):
self.external_marker = str(marker)[:20]
self.external_marker = marker
class DrydockRequest(request.Request):
context_type = DrydockRequestContext
def set_policy_engine(self, engine):
self.policy_engine = engine
def to_policy_view(self):
policy_dict = {}
policy_dict['user_id'] = self.user_id
policy_dict['user_domain_id'] = self.user_domain_id
policy_dict['project_id'] = self.project_id
policy_dict['project_domain_id'] = self.project_domain_id
policy_dict['roles'] = self.roles
policy_dict['is_admin_project'] = self.is_admin_project
return policy_dict
class DrydockRequest(falcon.request.Request):
context_type = DrydockRequestContext

2
drydock_provisioner/control/bootdata.py
View File

@ -23,7 +23,7 @@ from .base import StatefulResource
class BootdataResource(StatefulResource):
bootdata_options = [
cfg.StrOpt('prom_init', default=None, help='Path to file to distribute for prom_init.sh')
cfg.StrOpt('prom_init', default='/etc/drydock/bootdata/join.sh', help='Path to file to distribute for prom_init.sh')
]
def __init__(self, orchestrator=None, **kwargs):

37
drydock_provisioner/control/designs.py
View File

@ -1,5 +1,4 @@
# Copyright 2017 AT&T Intellectual Property. All other rights reserved.
#
# Copyright 2017 AT&T Intellectual Property. All other rights reserved. #
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
@ -16,6 +15,7 @@ import json
import uuid
import logging
import drydock_provisioner.policy as policy
import drydock_provisioner.objects as hd_objects
import drydock_provisioner.error as errors
@ -25,17 +25,25 @@ class DesignsResource(StatefulResource):
def __init__(self, **kwargs):
super(DesignsResource, self).__init__(**kwargs)
self.authorized_roles = ['user']
@policy.ApiEnforcer('physical_provisioner:read_data')
def on_get(self, req, resp):
ctx = req.context
state = self.state_manager
designs = list(state.designs.keys())
try:
designs = list(state.designs.keys())
resp.body = json.dumps(designs)
resp.status = falcon.HTTP_200
resp.body = json.dumps(designs)
resp.status = falcon.HTTP_200
except Exception as ex:
self.error(req.context, "Exception raised: %s" % str(ex))
self.return_error(resp, falcon.HTTP_500, message="Error accessing design list", retry=True)
@policy.ApiEnforcer('physical_provisioner:ingest_data')
def on_post(self, req, resp):
ctx = req.context
try:
json_data = self.req_json(req)
design = None
@ -67,8 +75,10 @@ class DesignResource(StatefulResource):
self.authorized_roles = ['user']
self.orchestrator = orchestrator
@policy.ApiEnforcer('physical_provisioner:read_data')
def on_get(self, req, resp, design_id):
source = req.params.get('source', 'designed')
ctx = req.context
try:
design = None
@ -93,6 +103,7 @@ class DesignsPartsResource(StatefulResource):
self.error(None, "DesignsPartsResource requires a configured Ingester instance")
raise ValueError("DesignsPartsResource requires a configured Ingester instance")
@policy.ApiEnforcer('physical_provisioner:ingest_data')
def on_post(self, req, resp, design_id):
ingester_name = req.params.get('ingester', None)
@ -108,12 +119,13 @@ class DesignsPartsResource(StatefulResource):
resp.status = falcon.HTTP_201
resp.body = json.dumps([x.obj_to_simple() for x in parsed_items])
else:
self.return_error(resp, falcon.HTTP_400, message="Empty body not supported", retry=False)
self.return_error(resp, falcon.HTTP_400, message="Empty body not supported", retry=False)
except ValueError:
self.return_error(resp, falcon.HTTP_500, message="Error processing input", retry=False)
except LookupError:
self.return_error(resp, falcon.HTTP_400, message="Ingester %s not registered" % ingester_name, retry=False)
@policy.ApiEnforcer('physical_provisioner:ingest_data')
def on_get(self, req, resp, design_id):
try:
design = self.state_manager.get_design(design_id)
@ -142,12 +154,16 @@ class DesignsPartsResource(StatefulResource):
class DesignsPartsKindsResource(StatefulResource):
def __init__(self, **kwargs):
super(DesignsPartsKindsResource, self).__init__(**kwargs)
self.authorized_roles = ['user']
@policy.ApiEnforcer('physical_provisioner:read_data')
def on_get(self, req, resp, design_id, kind):
pass
ctx = req.context
resp.status = falcon.HTTP_200
class DesignsPartResource(StatefulResource):
@ -156,7 +172,9 @@ class DesignsPartResource(StatefulResource):
self.authorized_roles = ['user']
self.orchestrator = orchestrator
@policy.ApiEnforcer('physical_provisioner:read_data')
def on_get(self, req , resp, design_id, kind, name):
ctx = req.context
source = req.params.get('source', 'designed')
try:
@ -188,3 +206,6 @@ class DesignsPartResource(StatefulResource):
except errors.DesignError as dex:
self.error(req.context, str(dex))
self.return_error(resp, falcon.HTTP_404, message=str(dex), retry=False)
except Exception as exc:
self.error(req.context, str(exc))
self.return_error(resp. falcon.HTTP_500, message=str(exc), retry=False)

87
drydock_provisioner/control/middleware.py
View File

@ -12,68 +12,73 @@
# See the License for the specific language governing permissions and
# limitations under the License.
import falcon
import logging
import uuid
import re
from oslo_config import cfg
from drydock_provisioner import policy
class AuthMiddleware(object):
def __init__(self):
self.logger = logging.getLogger('drydock')
# Authentication
def process_request(self, req, resp):
ctx = req.context
token = req.get_header('X-Auth-Token')
user = self.validate_token(token)
ctx.set_policy_engine(policy.policy_engine)
if user is not None:
ctx.set_user(user)
user_roles = self.role_list(user)
ctx.add_roles(user_roles)
for k, v in req.headers.items():
self.logger.debug("Request with header %s: %s" % (k, v))
auth_status = req.get_header('X-SERVICE-IDENTITY-STATUS')
service = True
if auth_status is None:
auth_status = req.get_header('X-IDENTITY-STATUS')
service = False
if auth_status == 'Confirmed':
# Process account and roles
ctx.authenticated = True
ctx.user = req.get_header('X-SERVICE-USER-NAME') if service else req.get_header('X-USER-NAME')
ctx.user_id = req.get_header('X-SERVICE-USER-ID') if service else req.get_header('X-USER-ID')
ctx.user_domain_id = req.get_header('X-SERVICE-USER-DOMAIN-ID') if service else req.get_header('X-USER-DOMAIN-ID')
ctx.project_id = req.get_header('X-SERVICE-PROJECT-ID') if service else req.get_header('X-PROJECT-ID')
ctx.project_domain_id = req.get_header('X-SERVICE-PROJECT-DOMAIN-ID') if service else req.get_header('X-PROJECT-DOMAIN-NAME')
if service:
ctx.add_roles(req.get_header('X-SERVICE-ROLES').split(','))
else:
ctx.add_roles(req.get_header('X-ROLES').split(','))
if req.get_header('X-IS-ADMIN-PROJECT') == 'True':
ctx.is_admin_project = True
else:
ctx.is_admin_project = False
self.logger.debug('Request from authenticated user %s with roles %s' % (ctx.user, ','.join(ctx.roles)))
else:
ctx.add_role('anyone')
ctx.authenticated = False
# Authorization
def process_resource(self, req, resp, resource, params):
ctx = req.context
if not resource.authorize_roles(ctx.roles):
raise falcon.HTTPUnauthorized('Authentication required',
('This resource requires an authorized role.'))
# Return the username associated with an authenticated token or None
def validate_token(self, token):
if token == '42':
return 'scott'
elif token == 'bigboss':
return 'admin'
else:
return None
# Return the list of roles assigned to the username
# Roles need to be an enum
def role_list(self, username):
if username == 'scott':
return ['user']
elif username == 'admin':
return ['user', 'admin']
class ContextMiddleware(object):
def __init__(self):
# Setup validation pattern for external marker
UUIDv4_pattern = '^[0-9A-F]{8}-[0-9A-F]{4}-4[0-9A-F]{3}-[89AB][0-9A-F]{3}-[0-9A-F]{12}$';
self.marker_re = re.compile(UUIDv4_pattern, re.I)
def process_request(self, req, resp):
ctx = req.context
requested_logging = req.get_header('X-Log-Level')
if (cfg.CONF.logging.log_level == 'DEBUG' or
(requested_logging == 'DEBUG' and 'admin' in ctx.roles)):
ctx.set_log_level('DEBUG')
elif requested_logging == 'INFO':
ctx.set_log_level('INFO')
ext_marker = req.get_header('X-Context-Marker')
ctx.set_external_marker(ext_marker if ext_marker is not None else '')
if ext_marker is not None and self.marker_re.fullmatch(ext_marker):
ctx.set_external_marker(ext_marker)
class LoggingMiddleware(object):

223
drydock_provisioner/control/tasks.py
View File

@ -16,6 +16,9 @@ import json
import threading
import traceback
from drydock_provisioner import policy
from drydock_provisioner import error as errors
import drydock_provisioner.objects.task as obj_task
from .base import StatefulResource
@ -23,38 +26,203 @@ class TasksResource(StatefulResource):
def __init__(self, orchestrator=None, **kwargs):
super(TasksResource, self).__init__(**kwargs)
self.authorized_roles = ['user']
self.orchestrator = orchestrator
@policy.ApiEnforcer('physical_provisioner:read_task')
def on_get(self, req, resp):
task_id_list = [str(x.get_id()) for x in self.state_manager.tasks]
resp.body = json.dumps(task_id_list)
def on_post(self, req, resp):
try:
json_data = self.req_json(req)
design_id = json_data.get('design_id', None)
action = json_data.get('action', None)
node_filter = json_data.get('node_filter', None)
if design_id is None or action is None:
self.info(req.context, "Task creation requires fields design_id, action")
self.return_error(resp, falcon.HTTP_400, message="Task creation requires fields design_id, action", retry=False)
return
task = self.orchestrator.create_task(obj_task.OrchestratorTask, design_id=design_id,
action=action, node_filter=node_filter)
task_thread = threading.Thread(target=self.orchestrator.execute_task, args=[task.get_id()])
task_thread.start()
resp.body = json.dumps(task.to_dict())
resp.status = falcon.HTTP_201
task_id_list = [str(x.get_id()) for x in self.state_manager.tasks]
resp.body = json.dumps(task_id_list)
resp.status = falcon.HTTP_200
except Exception as ex:
self.error(req.context, "Unknown error: %s\n%s" % (str(ex), traceback.format_exc()))
self.return_error(resp, falcon.HTTP_500, message="Unknown error", retry=False)
@policy.ApiEnforcer('physical_provisioner:create_task')
def on_post(self, req, resp):
# A map of supported actions to the handlers for tasks for those actions
supported_actions = {
'validate_design': TasksResource.task_validate_design,
'verify_site': TasksResource.task_verify_site,
'prepare_site': TasksResource.task_prepare_site,
'verify_node': TasksResource.task_verify_node,
'prepare_node': TasksResource.task_prepare_node,
'deploy_node': TasksResource.task_deploy_node,
'destroy_node': TasksResource.task_destroy_node,
}
try:
ctx = req.context
json_data = self.req_json(req)
action = json_data.get('action', None)
if action not in supported_actions:
self.error(req,context, "Unsupported action %s" % action)
self.return_error(resp, falcon.HTTP_400, message="Unsupported action %s" % action, retry=False)
else:
supported_actions.get(action)(self, req, resp)
except Exception as ex:
self.error(req.context, "Unknown error: %s\n%s" % (str(ex), traceback.format_exc()))
self.return_error(resp, falcon.HTTP_500, message="Unknown error", retry=False)
@policy.ApiEnforcer('physical_provisioner:validate_design')
def task_validate_design(self, req, resp):
json_data = self.req_json(req)
action = json_data.get('action', None)
if action != 'validate_design':
self.error(req.context, "Task body ended up in wrong handler: action %s in task_validate_design" % action)
self.return_error(resp, falcon.HTTP_500, message="Error - misrouted request", retry=False)
try:
task = self.create_task(json_data)
resp.body = json.dumps(task.to_dict())
resp.append_header('Location', "/api/v1.0/tasks/%s" % str(task.task_id))
resp.status = falcon.HTTP_201
except errors.InvalidFormat as ex:
self.error(req.context, ex.msg)
self.return_error(resp, falcon.HTTP_400, message=ex.msg, retry=False)
@policy.ApiEnforcer('physical_provisioner:verify_site')
def task_verify_site(self, req, resp):
json_data = self.req_json(req)
action = json_data.get('action', None)
if action != 'verify_site':
self.error(req.context, "Task body ended up in wrong handler: action %s in task_verify_site" % action)
self.return_error(resp, falcon.HTTP_500, message="Error - misrouted request", retry=False)
try:
task = self.create_task(json_data)
resp.body = json.dumps(task.to_dict())
resp.append_header('Location', "/api/v1.0/tasks/%s" % str(task.task_id))
resp.status = falcon.HTTP_201
except errors.InvalidFormat as ex:
self.error(req.context, ex.msg)
self.return_error(resp, falcon.HTTP_400, message=ex.msg, retry=False)
@policy.ApiEnforcer('physical_provisioner:prepare_site')
def task_prepare_site(self, req, resp):
json_data = self.req_json(req)
action = json_data.get('action', None)
if action != 'prepare_site':
self.error(req.context, "Task body ended up in wrong handler: action %s in task_prepare_site" % action)
self.return_error(resp, falcon.HTTP_500, message="Error - misrouted request", retry=False)
try:
task = self.create_task(json_data)
resp.body = json.dumps(task.to_dict())
resp.append_header('Location', "/api/v1.0/tasks/%s" % str(task.task_id))
resp.status = falcon.HTTP_201
except errors.InvalidFormat as ex:
self.error(req.context, ex.msg)
self.return_error(resp, falcon.HTTP_400, message=ex.msg, retry=False)
@policy.ApiEnforcer('physical_provisioner:verify_node')
def task_verify_node(self, req, resp):
json_data = self.req_json(req)
action = json_data.get('action', None)
if action != 'verify_node':
self.error(req.context, "Task body ended up in wrong handler: action %s in task_verify_node" % action)
self.return_error(resp, falcon.HTTP_500, message="Error - misrouted request", retry=False)
try:
task = self.create_task(json_data)
resp.body = json.dumps(task.to_dict())
resp.append_header('Location', "/api/v1.0/tasks/%s" % str(task.task_id))
resp.status = falcon.HTTP_201
except errors.InvalidFormat as ex:
self.error(req.context, ex.msg)
self.return_error(resp, falcon.HTTP_400, message=ex.msg, retry=False)
@policy.ApiEnforcer('physical_provisioner:prepare_node')
def task_prepare_node(self, req, resp):
json_data = self.req_json(req)
action = json_data.get('action', None)
if action != 'prepare_node':
self.error(req.context, "Task body ended up in wrong handler: action %s in task_prepare_node" % action)
self.return_error(resp, falcon.HTTP_500, message="Error - misrouted request", retry=False)
try:
task = self.create_task(json_data)
resp.body = json.dumps(task.to_dict())
resp.append_header('Location', "/api/v1.0/tasks/%s" % str(task.task_id))
resp.status = falcon.HTTP_201
except errors.InvalidFormat as ex:
self.error(req.context, ex.msg)
self.return_error(resp, falcon.HTTP_400, message=ex.msg, retry=False)
@policy.ApiEnforcer('physical_provisioner:deploy_node')
def task_deploy_node(self, req, resp):
json_data = self.req_json(req)
action = json_data.get('action', None)
if action != 'deploy_node':
self.error(req.context, "Task body ended up in wrong handler: action %s in task_deploy_node" % action)
self.return_error(resp, falcon.HTTP_500, message="Error - misrouted request", retry=False)
try:
task = self.create_task(json_data)
resp.body = json.dumps(task.to_dict())
resp.append_header('Location', "/api/v1.0/tasks/%s" % str(task.task_id))
resp.status = falcon.HTTP_201
except errors.InvalidFormat as ex:
self.error(req.context, ex.msg)
self.return_error(resp, falcon.HTTP_400, message=ex.msg, retry=False)
@policy.ApiEnforcer('physical_provisioner:destroy_node')
def task_destroy_node(self, req, resp):
json_data = self.req_json(req)
action = json_data.get('action', None)
if action != 'destroy_node':
self.error(req.context, "Task body ended up in wrong handler: action %s in task_destroy_node" % action)
self.return_error(resp, falcon.HTTP_500, message="Error - misrouted request", retry=False)
try:
task = self.create_task(json_data)
resp.body = json.dumps(task.to_dict())
resp.append_header('Location', "/api/v1.0/tasks/%s" % str(task.task_id))
resp.status = falcon.HTTP_201
except errors.InvalidFormat as ex:
self.error(req.context, ex.msg)
self.return_error(resp, falcon.HTTP_400, message=ex.msg, retry=False)
def create_task(self, task_body):
"""
Given the parsed body of a create task request, create the task
and start it in a thread
:param dict task_body: Dict representing the JSON body of a create task request
action - The action the task will execute
design_id - The design context the task will execute in
node_filter - A filter on which nodes will be affected by the task. The result is
an intersection of
applying all filters
node_names - A list of node hostnames
rack_names - A list of rack names that contain the nodes
node_tags - A list of tags applied to the nodes
:return: The Task object created
"""
design_id = task_body.get('design_id', None)
node_filter = task_body.get('node_filter', None)
action = task_body.get('action', None)
if design_id is None or action is None:
raise errors.InvalidFormat('Task creation requires fields design_id, action')
task = self.orchestrator.create_task(obj_task.OrchestratorTask, design_id=design_id,
action=action, node_filter=node_filter)
task_thread = threading.Thread(target=self.orchestrator.execute_task, args=[task.get_id()])
task_thread.start()
return task
class TaskResource(StatefulResource):
@ -64,7 +232,14 @@ class TaskResource(StatefulResource):
self.orchestrator = orchestrator
def on_get(self, req, resp, task_id):
ctx = req.context
policy_action = 'physical_provisioner:read_task'
try:
if not self.check_policy(policy_action, ctx):
self.access_denied(req, resp, policy_action)
return
task = self.state_manager.get_task(task_id)
if task is None:

13
drydock_provisioner/drydock.py
View File

@ -17,6 +17,7 @@ import os
from oslo_config import cfg
from drydock_provisioner import policy
import drydock_provisioner.config as config
import drydock_provisioner.objects as objects
import drydock_provisioner.ingester as ingester
@ -68,13 +69,23 @@ def start_drydock():
if 'MAAS_API_KEY' in os.environ:
cfg.CONF.set_override(name='maas_api_key', override=os.environ['MAAS_API_KEY'], group='maasdriver')
# Setup the RBAC policy enforcer
policy.policy_engine = policy.DrydockPolicy()
policy.policy_engine.register_policy()
wsgi_callable = api.start_api(state_manager=state, ingester=input_ingester, orchestrator=orchestrator)
# Ensure that the policy_engine is initialized before starting the API
wsgi_callable = api.start_api(state_manager=state, ingester=input_ingester,
orchestrator=orchestrator)
# Now that loggers are configured, log the effective config
cfg.CONF.log_opt_values(logging.getLogger(cfg.CONF.logging.global_logger_name), logging.DEBUG)
return wsgi_callable
# Initialization compatible with PasteDeploy
def paste_start_drydock(global_conf, **kwargs):
# At this time just ignore everything in the paste configuration and rely on oslo_config
return drydock
drydock = start_drydock()

84
drydock_provisioner/drydock_client/client.py
View File

@ -36,11 +36,9 @@ class DrydockClient(object):
resp = self.session.get(endpoint)
if resp.status_code != 200:
raise errors.ClientError("Received a %d from GET URL: %s" % (resp.status_code, endpoint),
code=resp.status_code)
else:
return resp.json()
self._check_response(resp)
return resp.json()
def get_design(self, design_id, source='designed'):
"""
@ -55,14 +53,10 @@ class DrydockClient(object):
resp = self.session.get(endpoint, query={'source': source})
if resp.status_code == 404:
raise errors.ClientError("Design ID %s not found." % (design_id), code=404)
elif resp.status_code != 200:
raise errors.ClientError("Received a %d from GET URL: %s" % (resp.status_code, endpoint),
code=resp.status_code)
else:
return resp.json()
self._check_response(resp)
return resp.json()
def create_design(self, base_design=None):
"""
Create a new design context for holding design parts
@ -77,12 +71,10 @@ class DrydockClient(object):
else:
resp = self.session.post(endpoint)
if resp.status_code != 201:
raise errors.ClientError("Received a %d from POST URL: %s" % (resp.status_code, endpoint),
code=resp.status_code)
else:
design = resp.json()
return design.get('id', None)
self._check_response(resp)
design = resp.json()
return design.get('id', None)
def get_part(self, design_id, kind, key, source='designed'):
"""
@ -99,13 +91,9 @@ class DrydockClient(object):
resp = self.session.get(endpoint, query={'source': source})
if resp.status_code == 404:
raise errors.ClientError("%s %s in design %s not found" % (key, kind, design_id), code=404)
elif resp.status_code != 200:
raise errors.ClientError("Received a %d from GET URL: %s" % (resp.status_code, endpoint),
code=resp.status_code)
else:
return resp.json()
self._check_response(resp)
return resp.json()
def load_parts(self, design_id, yaml_string=None):
"""
@ -120,15 +108,10 @@ class DrydockClient(object):
resp = self.session.post(endpoint, query={'ingester': 'yaml'}, body=yaml_string)
if resp.status_code == 400:
raise errors.ClientError("Invalid inputs: %s" % resp.text, code=resp.status_code)
elif resp.status_code == 500:
raise errors.ClientError("Server error: %s" % resp.text, code=resp.status_code)
elif resp.status_code == 201:
return resp.json()
else:
raise errors.ClientError("Uknown error. Received %d" % resp.status_code,
code=resp.status_code)
self._check_response(resp)
return resp.json()
def get_tasks(self):
"""
Get a list of all the tasks, completed or running.
@ -140,10 +123,9 @@ class DrydockClient(object):
resp = self.session.get(endpoint)
if resp.status_code != 200:
raise errors.ClientError("Server error: %s" % resp.text, code=resp.status_code)
else:
return resp.json()
self._check_response(resp)
return resp.json()
def get_task(self, task_id):
"""
@ -157,12 +139,9 @@ class DrydockClient(object):
resp = self.session.get(endpoint)
if resp.status_code == 200:
return resp.json()
elif resp.status_code == 404:
raise errors.ClientError("Task %s not found" % task_id, code=resp.status_code)
else:
raise errors.ClientError("Server error: %s" % resp.text, code=resp.status_code)
self._check_response(resp)
return resp.json()
def create_task(self, design_id, task_action, node_filter=None):
"""
@ -185,9 +164,14 @@ class DrydockClient(object):
resp = self.session.post(endpoint, data=task_dict)
if resp.status_code == 201:
return resp.json().get('task_id')
elif resp.status_code == 400:
raise errors.ClientError("Invalid inputs, received a %d: %s" % (resp.status_code, resp.text),
code=resp.status_code)
self._check_response(resp)
return resp.json().get('task_id')
def _check_response(self, resp):
if resp.status_code == 401:
raise errors.ClientUnauthorizedError("Unauthorized access to %s, include valid token." % resp.url)
elif resp.status_code == 403:
raise errors.ClientForbiddenError("Forbidden access to %s" % resp.url)
elif not resp.ok:
raise errors.ClientError("Error - received %d: %s" % (resp.status_code, resp.text), code=resp.status_code)

25
drydock_provisioner/error.py
View File

@ -16,27 +16,35 @@ import json
class DesignError(Exception):
pass
class StateError(Exception):
pass
class OrchestratorError(Exception):
pass
class TransientOrchestratorError(OrchestratorError):
pass
class PersistentOrchestratorError(OrchestratorError):
pass
class DriverError(Exception):
pass
class TransientDriverError(DriverError):
pass
class PersistentDriverError(DriverError):
pass
class ApiError(Exception):
def __init__(self, msg, code=500):
super().__init__(msg)
@ -47,13 +55,22 @@ class ApiError(Exception):
err_dict = {'error': msg, 'type': self.__class__.__name__}
return json.dumps(err_dict)
class InvalidFormat(ApiError):
def __init__(self, msg, code=400):
super(InvalidFormat, self).__init__(msg, code=code)
super(InvalidFormat, self).__init__(msg, code=code)
class ClientError(Exception):
class ClientError(ApiError):
def __init__(self, msg, code=500):
super().__init__(msg)
self.message = msg
self.status_code = code
class ClientUnauthorizedError(ClientError):
def __init__(self, msg):
super().__init__(msg, code=401)
class ClientForbiddenError(ClientError):
def __init__(self, msg):
super().__init__(msg, code=403)

114
drydock_provisioner/policy.py Normal file
View File

@ -0,0 +1,114 @@
# Copyright 2017 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import logging
import functools
from oslo_config import cfg
from oslo_policy import policy
# Global reference to a instantiated DrydockPolicy. Will be initialized by drydock.py
policy_engine = None
class DrydockPolicy(object):
"""
Initialize policy defaults
"""
# Base Policy
base_rules = [
policy.RuleDefault('admin_required', 'role:admin or is_admin:1', description='Actions requiring admin authority'),
]
# Orchestrator Policy
task_rules = [
policy.DocumentedRuleDefault('physical_provisioner:read_task', 'role:admin', 'Get task status',
[{'path': '/api/v1.0/tasks', 'method': 'GET'},
{'path': '/api/v1.0/tasks/{task_id}', 'method': 'GET'}]),
policy.DocumentedRuleDefault('physical_provisioner:validate_design', 'role:admin', 'Create validate_design task',
[{'path': '/api/v1.0/tasks', 'method': 'POST'}]),
policy.DocumentedRuleDefault('physical_provisioner:verify_site', 'role:admin', 'Create verify_site task',
[{'path': '/api/v1.0/tasks', 'method': 'POST'}]),
policy.DocumentedRuleDefault('physical_provisioner:prepare_site', 'role:admin', 'Create prepare_site task',
[{'path': '/api/v1.0/tasks', 'method': 'POST'}]),
policy.DocumentedRuleDefault('physical_provisioner:verify_node', 'role:admin', 'Create verify_node task',
[{'path': '/api/v1.0/tasks', 'method': 'POST'}]),
policy.DocumentedRuleDefault('physical_provisioner:prepare_node', 'role:admin', 'Create prepare_node task',
[{'path': '/api/v1.0/tasks', 'method': 'POST'}]),
policy.DocumentedRuleDefault('physical_provisioner:deploy_node', 'role:admin', 'Create deploy_node task',
[{'path': '/api/v1.0/tasks', 'method': 'POST'}]),
policy.DocumentedRuleDefault('physical_provisioner:destroy_node', 'role:admin', 'Create destroy_node task',
[{'path': '/api/v1.0/tasks', 'method': 'POST'}]),
]
# Data Management Policy
data_rules = [
policy.DocumentedRuleDefault('physical_provisioner:read_data', 'role:admin', 'Read loaded design data',
[{'path': '/api/v1.0/designs', 'method': 'GET'},
{'path': '/api/v1.0/designs/{design_id}', 'method': 'GET'}]),
policy.DocumentedRuleDefault('physical_provisioner:ingest_data', 'role:admin', 'Load design data',
[{'path': '/api/v1.0/designs', 'method': 'POST'},
{'path': '/api/v1.0/designs/{design_id}/parts', 'method': 'POST'}])
]
def __init__(self):
self.enforcer = policy.Enforcer(cfg.CONF)
def register_policy(self):
self.enforcer.register_defaults(DrydockPolicy.base_rules)
self.enforcer.register_defaults(DrydockPolicy.task_rules)
self.enforcer.register_defaults(DrydockPolicy.data_rules)
self.enforcer.load_rules()
def authorize(self, action, ctx):
target = {'project_id': ctx.project_id, 'user_id': ctx.user_id}
return self.enforcer.authorize(action, target, ctx.to_policy_view())
class ApiEnforcer(object):
"""
A decorator class for enforcing RBAC policies
"""
def __init__(self, action):
self.action = action
self.logger = logging.getLogger('drydock.policy')
def __call__(self, f):
@functools.wraps(f)
def secure_handler(slf, req, resp, *args):
ctx = req.context
policy_engine = ctx.policy_engine
self.logger.debug("Enforcing policy %s on request %s" % (self.action, ctx.request_id))
if policy_engine is not None and policy_engine.authorize(self.action, ctx):
return f(slf, req, resp, *args)
else:
if ctx.authenticated:
slf.info(ctx, "Error - Forbidden access - action: %s" % self.action)
slf.return_error(resp, falcon.HTTP_403, message="Forbidden", retry=False)
else:
slf.info(ctx, "Error - Unauthenticated access")
slf.return_error(resp, falcon.HTTP_401, message="Unauthenticated", retry=False)
return secure_handler
def list_policies():
default_policy = []
default_policy.extend(DrydockPolicy.base_rules)
default_policy.extend(DrydockPolicy.task_rules)
default_policy.extend(DrydockPolicy.data_rules)
return default_policy

3
entrypoint.sh
View File

@ -5,7 +5,8 @@ CMD="drydock"
PORT=${PORT:-9000}
if [ "$1" = 'server' ]; then
exec uwsgi --http :${PORT} -w drydock_provisioner.drydock --callable drydock --enable-threads -L --pyargv "--config-file /etc/drydock/drydock.conf"
# exec uwsgi --http :${PORT} -w drydock_provisioner.drydock --callable drydock --enable-threads -L --pyargv "--config-file /etc/drydock/drydock.conf"
exec uwsgi --http :${PORT} --paste config:/etc/drydock/api-paste.ini --enable-threads -L --pyargv "--config-file /etc/drydock/drydock.conf"
fi
exec ${CMD} $@

3
etc/drydock/.gitignore vendored Normal file
View File

@ -0,0 +1,3 @@
# Ignore generated samples
drydock.conf
policy.yaml

8
etc/drydock/api-paste.ini Normal file
View File

@ -0,0 +1,8 @@
[app:drydock-api]
paste.app_factory = drydock_provisioner.drydock:paste_start_drydock
[pipeline:main]
pipeline = authtoken drydock-api
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory

0
examples/bootdata/join.sh → etc/drydock/bootdata/join.sh
View File

7
etc/drydock/drydock-config-generator.conf Normal file
View File

@ -0,0 +1,7 @@
[DEFAULT]
output_file = etc/drydock/drydock.conf.sample
wrap_width = 80
namespace = drydock_provisioner
namespace = keystonemiddleware.auth_token
namespace = oslo.policy

5
etc/drydock/drydock-policy-generator.conf Normal file
View File

@ -0,0 +1,5 @@
[DEFAULT]
output_file = etc/drydock/policy.yaml.sample
wrap_width = 80
namespace = drydock_provisioner

50
examples/drydock.conf
View File

@ -1,50 +0,0 @@
# Copyright 2017 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
[DEFAULT]
# No global options yet
[logging]
log_level = 'DEBUG'
[authentication]
bypass_enabled = True
[plugins]
# All the config ingesters that are active
# Supports multiple values
ingester = 'drydock_provisioner.ingester.plugins.yaml.YamlIngester'
# OOB drivers that are enabled
# Supports multiple values
oob_driver = 'drydock_provisioner.drivers.oob.pyghmi_driver.PyghmiDriver'
oob_driver = 'drydock_provisioner.drivers.oob.manual_driver.driver.ManualDriver'
# Node driver that is enabled
node_driver = 'drydock_provisioner.drivers.node.maasdriver.driver.MaasNodeDriver'
[timeouts]
create_network_template = 2
identify_node = 10
configure_hardware = 30
apply_node_networking = 5
apply_node_platform = 5
deploy_node = 45
[maasdriver]
maas_api_url = 'http://localhost:8000/MAAS/api/2.0/'
maas_api_key = 'your:secret:key'
[bootdata]
prom_init = '/etc/drydock/bootdata/join.sh'

347
examples/drydock.conf.keystone Normal file
View File

@ -0,0 +1,347 @@
[DEFAULT]
#
# From drydock_provisioner
#
# Polling interval in seconds for checking subtask or downstream status (integer
# value)
#poll_interval = 10
[authentication]
#
# From drydock_provisioner
#
# Client request authentication strategy (string value)
#auth_strategy = keystone
# X-Auth-Token value to bypass backend authentication (string value)
#admin_token = bigboss
# Can backend authentication be bypassed? (boolean value)
#bypass_enabled = false
[bootdata]
#
# From drydock_provisioner
#
# Path to file to distribute for prom_init.sh (string value)
#prom_init = /etc/drydock/bootdata/join.sh
[keystone_authtoken]
#
# From keystonemiddleware.auth_token
#
# Complete "public" Identity API endpoint. This endpoint should not be an
# "admin" endpoint, as it should be accessible by all end users. Unauthenticated
# clients are redirected to this endpoint to authenticate. Although this
# endpoint should ideally be unversioned, client support in the wild varies.
# If you're using a versioned v2 endpoint here, then this should *not* be the
# same endpoint the service user utilizes for validating tokens, because normal
# end users may not be able to reach that endpoint. (string value)
auth_uri = http://172.20.0.3:5000/v3
# API version of the admin Identity API endpoint. (string value)
auth_version = 3
# Do not handle authorization requests within the middleware, but delegate the
# authorization decision to downstream WSGI components. (boolean value)
delay_auth_decision = true
# Request timeout value for communicating with Identity API server. (integer
# value)
#http_connect_timeout = <None>
# How many times are we trying to reconnect when communicating with Identity API
# Server. (integer value)
#http_request_max_retries = 3
# Request environment key where the Swift cache object is stored. When
# auth_token middleware is deployed with a Swift cache, use this option to have
# the middleware share a caching backend with swift. Otherwise, use the
# ``memcached_servers`` option instead. (string value)
#cache = <None>
# Required if identity server requires client certificate (string value)
#certfile = <None>
# Required if identity server requires client certificate (string value)
#keyfile = <None>
# A PEM encoded Certificate Authority to use when verifying HTTPs connections.
# Defaults to system CAs. (string value)
#cafile = <None>
# Verify HTTPS connections. (boolean value)
#insecure = false
# The region in which the identity server can be found. (string value)
#region_name = <None>
# Directory used to cache files related to PKI tokens. (string value)
#signing_dir = <None>
# Optionally specify a list of memcached server(s) to use for caching. If left
# undefined, tokens will instead be cached in-process. (list value)
# Deprecated group/name - [keystone_authtoken]/memcache_servers
#memcached_servers = <None>
# In order to prevent excessive effort spent validating tokens, the middleware
# caches previously-seen tokens for a configurable duration (in seconds). Set to
# -1 to disable caching completely. (integer value)
#token_cache_time = 300
# Determines the frequency at which the list of revoked tokens is retrieved from
# the Identity service (in seconds). A high number of revocation events combined
# with a low cache duration may significantly reduce performance. Only valid for
# PKI tokens. (integer value)
#revocation_cache_time = 10
# (Optional) If defined, indicate whether token data should be authenticated or
# authenticated and encrypted. If MAC, token data is authenticated (with HMAC)
# in the cache. If ENCRYPT, token data is encrypted and authenticated in the
# cache. If the value is not one of these options or empty, auth_token will
# raise an exception on initialization. (string value)
# Allowed values: None, MAC, ENCRYPT
#memcache_security_strategy = None
# (Optional, mandatory if memcache_security_strategy is defined) This string is
# used for key derivation. (string value)
#memcache_secret_key = <None>
# (Optional) Number of seconds memcached server is considered dead before it is
# tried again. (integer value)
#memcache_pool_dead_retry = 300
# (Optional) Maximum total number of open connections to every memcached server.
# (integer value)
#memcache_pool_maxsize = 10
# (Optional) Socket timeout in seconds for communicating with a memcached
# server. (integer value)
#memcache_pool_socket_timeout = 3
# (Optional) Number of seconds a connection to memcached is held unused in the
# pool before it is closed. (integer value)
#memcache_pool_unused_timeout = 60
# (Optional) Number of seconds that an operation will wait to get a memcached
# client connection from the pool. (integer value)
#memcache_pool_conn_get_timeout = 10
# (Optional) Use the advanced (eventlet safe) memcached client pool. The
# advanced pool will only work under python 2.x. (boolean value)
#memcache_use_advanced_pool = false
# (Optional) Indicate whether to set the X-Service-Catalog header. If False,
# middleware will not ask for service catalog on token validation and will not
# set the X-Service-Catalog header. (boolean value)
#include_service_catalog = true
# Used to control the use and type of token binding. Can be set to: "disabled"
# to not check token binding. "permissive" (default) to validate binding
# information if the bind type is of a form known to the server and ignore it if
# not. "strict" like "permissive" but if the bind type is unknown the token will
# be rejected. "required" any form of token binding is needed to be allowed.
# Finally the name of a binding method that must be present in tokens. (string
# value)
#enforce_token_bind = permissive
# If true, the revocation list will be checked for cached tokens. This requires
# that PKI tokens are configured on the identity server. (boolean value)
#check_revocations_for_cached = false
# Hash algorithms to use for hashing PKI tokens. This may be a single algorithm
# or multiple. The algorithms are those supported by Python standard
# hashlib.new(). The hashes will be tried in the order given, so put the
# preferred one first for performance. The result of the first hash will be
# stored in the cache. This will typically be set to multiple values only while
# migrating from a less secure algorithm to a more secure one. Once all the old
# tokens are expired this option should be set to a single value for better
# performance. (list value)
#hash_algorithms = md5
# Authentication type to load (string value)
# Deprecated group/name - [keystone_authtoken]/auth_plugin
auth_type = password
# Config Section from which to load plugin specific options (string value)
auth_section = keystone_authtoken_password
[keystone_authtoken_password]
#
# From drydock_provisioner
#
# Authentication URL (string value)
auth_url = http://172.20.0.3:5000/
# Domain ID to scope to (string value)
#domain_id = <None>
# Domain name to scope to (string value)
domain_name = ucp
# Project ID to scope to (string value)
# Deprecated group/name - [keystone_authtoken_password]/tenant_id
#project_id = <None>
# Project name to scope to (string value)
# Deprecated group/name - [keystone_authtoken_password]/tenant_name
project_name = service
# Domain ID containing project (string value)
#project_domain_id = <None>
# Domain name containing project (string value)
project_domain_name = ucp
# Trust ID (string value)
#trust_id = <None>
# Optional domain ID to use with v3 and v2 parameters. It will be used for both
# the user and project domain in v3 and ignored in v2 authentication. (string
# value)
#default_domain_id = <None>
# Optional domain name to use with v3 API and v2 parameters. It will be used for
# both the user and project domain in v3 and ignored in v2 authentication.
# (string value)
default_domain_name = ucp
# User id (string value)
#user_id = <None>
# Username (string value)
# Deprecated group/name - [keystone_authtoken_password]/user_name
#username = <None>
user_name = drydock
# User's domain id (string value)
#user_domain_id = <None>
# User's domain name (string value)
user_domain_name = ucp
# User's password (string value)
password = drydock
[logging]
#
# From drydock_provisioner
#
# Global log level for Drydock (string value)
#log_level = INFO
# Logger name for the top-level logger (string value)
#global_logger_name = drydock
# Logger name for OOB driver logging (string value)
#oobdriver_logger_name = ${global_logger_name}.oobdriver
# Logger name for Node driver logging (string value)
#nodedriver_logger_name = ${global_logger_name}.nodedriver
# Logger name for API server logging (string value)
#control_logger_name = ${global_logger_name}.control
[maasdriver]
#
# From drydock_provisioner
#
# The API key for accessing MaaS (string value)
#maas_api_key = <None>
# The URL for accessing MaaS API (string value)
#maas_api_url = <None>
# Polling interval for querying MaaS status in seconds (integer value)
#poll_interval = 10
[oslo_policy]
#
# From oslo.policy
#
# The file that defines policies. (string value)
#policy_file = policy.json
# Default rule. Enforced when a requested rule is not found. (string value)
#policy_default_rule = default
# Directories where policy configuration files are stored. They can be relative
# to any directory in the search path defined by the config_dir option, or
# absolute paths. The file defined by policy_file must exist for these
# directories to be searched. Missing or empty directories are ignored. (multi
# valued)
#policy_dirs = policy.d
[plugins]
#
# From drydock_provisioner
#
# Module path string of a input ingester to enable (multi valued)
#ingester = drydock_provisioner.ingester.plugins.yaml.YamlIngester
# Module path string of a OOB driver to enable (multi valued)
#oob_driver = drydock_provisioner.drivers.oob.pyghmi_driver.PyghmiDriver
# Module path string of the Node driver to enable (string value)
#node_driver = drydock_provisioner.drivers.node.maasdriver.driver.MaasNodeDriver
# Module path string of the Network driver enable (string value)
#network_driver = <None>
[timeouts]
#
# From drydock_provisioner
#
# Fallback timeout when a specific one is not configured (integer value)
#drydock_timeout = 5
# Timeout in minutes for creating site network templates (integer value)
#create_network_template = 2
# Timeout in minutes for creating user credentials (integer value)
#configure_user_credentials = 2
# Timeout in minutes for initial node identification (integer value)
#identify_node = 10
# Timeout in minutes for node commissioning and hardware configuration (integer
# value)
#configure_hardware = 30
# Timeout in minutes for configuring node networking (integer value)
#apply_node_networking = 5
# Timeout in minutes for configuring node platform (integer value)
#apply_node_platform = 5
# Timeout in minutes for deploying a node (integer value)
#deploy_node = 45

13
requirements-direct.txt
View File

@ -1,11 +1,16 @@
PyYAML
pyghmi>=1.0.18
PyYAML===3.12
pyghmi===1.0.18
netaddr
falcon
oslo.versionedobjects>=1.23.0
oslo.versionedobjects===1.23.0
requests
oauthlib
uwsgi>1.4
uwsgi===2.0.15
bson===0.4.7
oslo.config
click===6.7
PasteDeploy==1.5.2
keystonemiddleware===4.9.1
oslo.policy===1.22.1
iso8601===0.1.11
keystoneauth1===2.13.0

69
requirements-lock.txt Normal file
View File

@ -0,0 +1,69 @@
amqp==2.2.1
Babel==2.3.4
bson==0.4.7
cachetools==2.0.0
certifi==2017.7.27.1
chardet==3.0.4
click==6.7
contextlib2==0.5.5
debtcollector==1.17.0
enum-compat==0.0.2
eventlet==0.20.0
falcon==1.2.0
fasteners==0.14.1
futurist==1.3.0
greenlet==0.4.12
idna==2.5
iso8601==0.1.11
Jinja2==2.9.6
keystoneauth1===2.13.0
keystonemiddleware==4.9.1
kombu==4.1.0
MarkupSafe==1.0
monotonic==1.3
msgpack-python==0.4.8
netaddr==0.7.19
netifaces==0.10.6
oauthlib==2.0.2
oslo.concurrency==3.21.0
oslo.config==4.11.0
oslo.context==2.17.0
oslo.i18n==3.17.0
oslo.log==3.30.0
oslo.messaging==5.30.0
oslo.middleware==3.30.0
oslo.policy==1.22.1
oslo.serialization==2.20.0
oslo.service==1.25.0
oslo.utils==3.28.0
oslo.versionedobjects==1.23.0
Paste==2.0.3
PasteDeploy==1.5.2
pbr==3.1.1
pika==0.10.0
pika-pool==0.1.3
positional==1.1.2
prettytable==0.7.2
pycadf==2.6.0
pycrypto==2.6.1
pyghmi==1.0.18
pyinotify==0.9.6
pyparsing==2.2.0
python-dateutil==2.6.1
python-keystoneclient==3.13.0
python-mimeparse==1.6.0
pytz==2017.2
PyYAML==3.12
repoze.lru==0.6
requests==2.18.2
rfc3986==1.1.0
Routes==2.4.1
six==1.10.0
statsd==3.2.1
stevedore==1.25.0
tenacity==4.4.0
urllib3==1.22
uWSGI==2.0.15
vine==1.1.4
WebOb==1.7.3
wrapt==1.10.10

1
requirements-test.txt
View File

@ -4,3 +4,4 @@ responses
mock
tox
oslo.versionedobjects[fixtures]>=1.23.0
oslo.config[fixtures]

13
setup.py
View File

@ -46,20 +46,9 @@ setup(name='drydock_provisioner',
'drydock_provisioner.cli.part',
'drydock_provisioner.cli.task',
'drydock_provisioner.drydock_client'],
install_requires=[
'PyYAML',
'pyghmi>=1.0.18',
'netaddr',
'falcon',
'oslo.versionedobjects>=1.23.0',
'requests',
'oauthlib',
'uwsgi>1.4',
'bson===0.4.7',
'oslo.config',
],
entry_points={
'oslo.config.opts': 'drydock_provisioner = drydock_provisioner.config:list_opts',
'oslo.policy.policies': 'drydock_provisioner = drydock_provisioner.policy:list_policies',
'console_scripts': 'drydock = drydock_provisioner.cli.commands:drydock'
}
)

114
tests/unit/test_api_tasks.py Normal file
View File

@ -0,0 +1,114 @@
# Copyright 2017 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import uuid
import logging
import json
from drydock_provisioner import policy
from drydock_provisioner.orchestrator import Orchestrator
from drydock_provisioner.control.base import DrydockRequestContext, BaseResource
from drydock_provisioner.control.tasks import TaskResource, TasksResource
import pytest
import falcon
logging.basicConfig(level=logging.DEBUG)
class TestTasksApi():
def test_read_tasks(self, mocker):
''' DrydockPolicy.authorized() should correctly use oslo_policy to enforce
RBAC policy based on a DrydockRequestContext instance
'''
mocker.patch('oslo_policy.policy.Enforcer')
state = mocker.MagicMock()
ctx = DrydockRequestContext()
policy_engine = policy.DrydockPolicy()
# Mock policy enforcement
policy_mock_config = {'authorize.return_value': True}
policy_engine.enforcer.configre_mock(**policy_mock_config)
api = TasksResource(state_manager=state)
# Configure context
project_id = str(uuid.uuid4().hex)
ctx.project_id = project_id
user_id = str(uuid.uuid4().hex)
ctx.user_id = user_id
ctx.roles = ['admin']
ctx.set_policy_engine(policy_engine)
# Configure mocked request and response
req = mocker.MagicMock()
resp = mocker.MagicMock()
req.context = ctx
api.on_get(req, resp)
expected_calls = [mocker.call.tasks]
assert state.has_calls(expected_calls)
assert resp.status == falcon.HTTP_200
def test_create_task(self, mocker):
mocker.patch('oslo_policy.policy.Enforcer')
state = mocker.MagicMock()
orch = mocker.MagicMock(spec=Orchestrator, wraps=Orchestrator(state_manager=state))
orch_mock_config = {'execute_task.return_value': True}
orch.configure_mock(**orch_mock_config)
ctx = DrydockRequestContext()
policy_engine = policy.DrydockPolicy()
json_body = json.dumps({
'action': 'verify_site',
'design_id': 'foo',
}).encode('utf-8')
# Mock policy enforcement
policy_mock_config = {'authorize.return_value': True}
policy_engine.enforcer.configure_mock(**policy_mock_config)
api = TasksResource(orchestrator=orch, state_manager=state)
# Configure context
project_id = str(uuid.uuid4().hex)
ctx.project_id = project_id
user_id = str(uuid.uuid4().hex)
ctx.user_id = user_id
ctx.roles = ['admin']
ctx.set_policy_engine(policy_engine)
# Configure mocked request and response
req = mocker.MagicMock(spec=falcon.Request)
req.content_type = 'application/json'
req.stream.read.return_value = json_body
resp = mocker.MagicMock(spec=falcon.Response)
req.context = ctx
api.on_post(req, resp)
assert resp.status == falcon.HTTP_201
assert resp.get_header('Location') is not None
@policy.ApiEnforcer('physical_provisioner:read_task')
def target_function(self, req, resp):
return True

59
tests/unit/test_apienforcer.py Normal file
View File

@ -0,0 +1,59 @@
# Copyright 2017 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import uuid
import logging
from drydock_provisioner import policy
from drydock_provisioner.control.base import DrydockRequestContext
import pytest
logging.basicConfig(level=logging.DEBUG)
class TestEnforcerDecorator():
def test_apienforcer_decorator(self,mocker):
''' DrydockPolicy.authorized() should correctly use oslo_policy to enforce
RBAC policy based on a DrydockRequestContext instance. authorized() is
called via the policy.ApiEnforcer decorator.
'''
mocker.patch('oslo_policy.policy.Enforcer')
ctx = DrydockRequestContext()
policy_engine = policy.DrydockPolicy()
# Configure context
project_id = str(uuid.uuid4())
ctx.project_id = project_id
user_id = str(uuid.uuid4())
ctx.user_id = user_id
ctx.roles = ['admin']
ctx.set_policy_engine(policy_engine)
# Configure mocked request and response
req = mocker.MagicMock()
resp = mocker.MagicMock()
req.context = ctx
self.target_function(req, resp)
expected_calls = [mocker.call.authorize('physical_provisioner:read_task', {'project_id': project_id, 'user_id': user_id},
ctx.to_policy_view())]
policy_engine.enforcer.assert_has_calls(expected_calls)
@policy.ApiEnforcer('physical_provisioner:read_task')
def target_function(self, req, resp):
return True

115
tests/unit/test_auth_middleware.py Normal file
View File

@ -0,0 +1,115 @@
# Copyright 2017 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import uuid
import falcon
import sys
from drydock_provisioner.control.base import DrydockRequest
from drydock_provisioner.control.middleware import AuthMiddleware
import pytest
class TestAuthMiddleware():
# the WSGI env for a request processed by keystone middleware
# with user token
ks_user_env = { 'REQUEST_METHOD': 'GET',
'SCRIPT_NAME': '/foo',
'PATH_INFO': '',
'QUERY_STRING': '',
'CONTENT_TYPE': '',
'CONTENT_LENGTH': 0,
'SERVER_NAME': 'localhost',
'SERVER_PORT': '9000',
'SERVER_PROTOCOL': 'HTTP/1.1',
'HTTP_X_IDENTITY_STATUS': 'Confirmed',
'HTTP_X_PROJECT_ID': '',
'HTTP_X_USER_ID': '',
'HTTP_X_AUTH_TOKEN': '',
'HTTP_X_ROLES': '',
'wsgi.version': (1,0),
'wsgi.url_scheme': 'http',
'wsgi.input': sys.stdin,
'wsgi.errors': sys.stderr,
'wsgi.multithread': False,
'wsgi.multiprocess': False,
'wsgi.run_once': False,
}
# the WSGI env for a request processed by keystone middleware
# with service token
ks_service_env = { 'REQUEST_METHOD': 'GET',
'SCRIPT_NAME': '/foo',
'PATH_INFO': '',
'QUERY_STRING': '',
'CONTENT_TYPE': '',
'CONTENT_LENGTH': 0,
'SERVER_NAME': 'localhost',
'SERVER_PORT': '9000',
'SERVER_PROTOCOL': 'HTTP/1.1',
'HTTP_X_SERVICE_IDENTITY_STATUS': 'Confirmed',
'HTTP_X_SERVICE_PROJECT_ID': '',
'HTTP_X_SERVICE_USER_ID': '',
'HTTP_X_SERVICE_TOKEN': '',
'HTTP_X_ROLES': '',
'wsgi.version': (1,0),
'wsgi.url_scheme': 'http',
'wsgi.input': sys.stdin,
'wsgi.errors': sys.stderr,
'wsgi.multithread': False,
'wsgi.multiprocess': False,
'wsgi.run_once': False,
}
def test_process_request_user(self):
''' AuthMiddleware is expected to correctly identify the headers
added to an authenticated request by keystonemiddleware in a
PasteDeploy configuration
'''
req_env = TestAuthMiddleware.ks_user_env
project_id = str(uuid.uuid4().hex)
req_env['HTTP_X_PROJECT_ID'] = project_id
user_id = str(uuid.uuid4().hex)
req_env['HTTP_X_USER_ID'] = user_id
token = str(uuid.uuid4().hex)
req_env['HTTP_X_AUTH_TOKEN'] = token
middleware = AuthMiddleware()
request = DrydockRequest(req_env)
response = falcon.Response()
middleware.process_request(request, response)
assert request.context.authenticated == True
assert request.context.user_id == user_id
def test_process_request_user_noauth(self):
''' AuthMiddleware is expected to correctly identify the headers
added to an unauthenticated (no token, bad token) request by
keystonemiddleware in a PasteDeploy configuration
'''
req_env = TestAuthMiddleware.ks_user_env
req_env['HTTP_X_IDENTITY_STATUS'] = 'Invalid'
middleware = AuthMiddleware()
request = DrydockRequest(req_env)
response = falcon.Response()
middleware.process_request(request, response)
assert request.context.authenticated == False

10
tests/unit/test_design_inheritance.py
View File

@ -26,9 +26,6 @@ import yaml
class TestClass(object):
def setup_method(self, method):
print("Running test {0}".format(method.__name__))
def test_design_inheritance(self, loaded_design):
orchestrator = Orchestrator(state_manager=loaded_design,
@ -41,13 +38,11 @@ class TestClass(object):
design_data = orchestrator.compute_model_inheritance(design_data)
node = design_data.get_baremetal_node("controller01")
assert node.applied.get('hardware_profile') == 'HPGen9v3'
iface = node.get_applied_interface('bond0')
print(yaml.dump(iface, default_flow_style=False))
assert iface.get_applied_slave_count() == 2
iface = node.get_applied_interface('pxe')
@ -68,7 +63,6 @@ class TestClass(object):
return design_state
@pytest.fixture(scope='module')
def input_files(self, tmpdir_factory, request):
tmpdir = tmpdir_factory.mktemp('data')
@ -80,4 +74,4 @@ class TestClass(object):
dst_file = str(tmpdir) + "/" + f
shutil.copyfile(src_file, dst_file)
return tmpdir
return tmpdir

2
tests/unit/test_ingester.py
View File

@ -78,4 +78,4 @@ class TestClass(object):
dst_file = str(tmpdir) + "/" + f
shutil.copyfile(src_file, dst_file)
return tmpdir
return tmpdir

4
tests/unit/test_ingester_yaml.py
View File

@ -34,7 +34,7 @@ class TestClass(object):
def test_ingest_multidoc(self, input_files):
input_file = input_files.join("multidoc.yaml")
ingester = YamlIngester()
models = ingester.ingest_data(filenames=[str(input_file)])
@ -52,4 +52,4 @@ class TestClass(object):
dst_file = str(tmpdir) + "/" + f
shutil.copyfile(src_file, dst_file)
return tmpdir
return tmpdir

2
tests/unit/test_orch_oob.py
View File

@ -104,4 +104,4 @@ class TestClass(object):
dst_file = str(tmpdir) + "/" + f
shutil.copyfile(src_file, dst_file)
return tmpdir
return tmpdir

63
tests/unit/test_policy_engine.py Normal file
View File

@ -0,0 +1,63 @@
# Copyright 2017 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import uuid
from drydock_provisioner.policy import DrydockPolicy
from drydock_provisioner.control.base import DrydockRequestContext
import pytest
class TestDefaultRules():
def test_register_policy(self, mocker):
''' DrydockPolicy.register_policy() should correctly register all default
policy rules
'''
mocker.patch('oslo_policy.policy.Enforcer')
policy_engine = DrydockPolicy()
policy_engine.register_policy()
expected_calls = [mocker.call.register_defaults(DrydockPolicy.base_rules),
mocker.call.register_defaults(DrydockPolicy.task_rules),
mocker.call.register_defaults(DrydockPolicy.data_rules)]
# Validate the oslo_policy Enforcer was loaded with expected default policy rules
policy_engine.enforcer.assert_has_calls(expected_calls, any_order=True)
def test_authorize_context(self,mocker):
''' DrydockPolicy.authorized() should correctly use oslo_policy to enforce
RBAC policy based on a DrydockRequestContext instance
'''
mocker.patch('oslo_policy.policy.Enforcer')
ctx = DrydockRequestContext()
# Configure context
project_id = str(uuid.uuid4().hex)
ctx.project_id = project_id
user_id = str(uuid.uuid4().hex)
ctx.user_id = user_id
ctx.roles = ['admin']
# Define action
policy_action = 'physical_provisioner:read_task'
policy_engine = DrydockPolicy()
policy_engine.authorize(policy_action, ctx)
expected_calls = [mocker.call.authorize(policy_action, {'project_id': project_id, 'user_id': user_id},
ctx.to_policy_view())]
policy_engine.enforcer.assert_has_calls(expected_calls)

2
tests/yaml_samples/multidoc.yaml
View File

@ -71,4 +71,4 @@ spec:
# Is this link supporting multiple layer 2 networks?
trunking:
mode: '802.1q'
default_network: mgmt
default_network: mgmt

12
tox.ini
View File

@ -9,7 +9,15 @@ setenv=
PYTHONWARNING=all
commands=
py.test \
{posargs}
{posargs}
[testenv:genconfig]
basepython=python3.5
commands = oslo-config-generator --config-file=etc/drydock/drydock-config-generator.conf
[testenv:genpolicy]
basepython=python3.5
commands = oslopolicy-sample-generator --config-file etc/drydock/drydock-policy-generator.conf
[flake8]
ignore=E302,H306