diff --git a/README.rst b/README.rst index f16bbc0..f078f3c 100644 --- a/README.rst +++ b/README.rst @@ -29,7 +29,7 @@ Service specific settings, like the nova service type, are set with the default service type as a prefix. For instance, to set a special service_type for trove set -:: +.. code-block:: bash export OS_DATABASE_SERVICE_TYPE=rax:database @@ -56,7 +56,7 @@ Service specific settings, like the nova service type, are set with the default service type as a prefix. For instance, to set a special service_type for trove (because you're using Rackspace) set: -:: +.. code-block:: yaml database_service_type: 'rax:database' @@ -85,7 +85,7 @@ look in an OS specific config dir An example config file is probably helpful: -:: +.. code-block:: yaml clouds: mordred: @@ -117,7 +117,7 @@ An example config file is probably helpful: - IAD You may note a few things. First, since `auth_url` settings are silly -and embarrasingly ugly, known cloud vendor profile information is included and +and embarrassingly ugly, known cloud vendor profile information is included and may be referenced by name. One of the benefits of that is that `auth_url` isn't the only thing the vendor defaults contain. For instance, since Rackspace lists `rax:database` as the service type for trove, `os-client-config` @@ -145,6 +145,34 @@ as a result of a chosen plugin need to go into the auth dict. For password auth, this includes `auth_url`, `username` and `password` as well as anything related to domains, projects and trusts. +Splitting Secrets +----------------- + +In some scenarios, such as configuration management controlled environments, +it might be easier to have secrets in one file and non-secrets in another. +This is fully supported via an optional file `secure.yaml` which follows all +the same location rules as `clouds.yaml`. It can contain anything you put +in `clouds.yaml` and will take precedence over anything in the `clouds.yaml` +file. + +.. code-block:: yaml + + # clouds.yaml + clouds: + internap: + profile: internap + auth: + username: api-55f9a00fb2619 + project_name: inap-17037 + regions: + - ams01 + - nyj01 + # secure.yaml + clouds: + internap: + auth: + password: XXXXXXXXXXXXXXXXX + SSL Settings ------------ @@ -181,7 +209,7 @@ that the resource should never expire. and presents the cache information so that your various applications that are connecting to OpenStack can share a cache should you desire. -:: +.. code-block:: yaml cache: class: dogpile.cache.pylibmc @@ -214,7 +242,7 @@ caused it to not actually function. In that case, there is a config option you can set to unbreak you `force_ipv4`, or `OS_FORCE_IPV4` boolean environment variable. -:: +.. code-block:: yaml client: force_ipv4: true @@ -237,12 +265,44 @@ environment variable. The above snippet will tell client programs to prefer returning an IPv4 address. +Per-region settings +------------------- + +Sometimes you have a cloud provider that has config that is common to the +cloud, but also with some things you might want to express on a per-region +basis. For instance, Internap provides a public and private network specific +to the user in each region, and putting the values of those networks into +config can make consuming programs more efficient. + +To support this, the region list can actually be a list of dicts, and any +setting that can be set at the cloud level can be overridden for that +region. + +:: + + clouds: + internap: + profile: internap + auth: + password: XXXXXXXXXXXXXXXXX + username: api-55f9a00fb2619 + project_name: inap-17037 + regions: + - name: ams01 + values: + external_network: inap-17037-WAN1654 + internal_network: inap-17037-LAN4820 + - name: nyj01 + values: + external_network: inap-17037-WAN7752 + internal_network: inap-17037-LAN6745 + Usage ----- The simplest and least useful thing you can do is: -:: +.. code-block:: python python -m os_client_config.config @@ -251,7 +311,7 @@ it from python, which is much more likely what you want to do, things like: Get a named cloud. -:: +.. code-block:: python import os_client_config @@ -261,10 +321,71 @@ Get a named cloud. Or, get all of the clouds. -:: +.. code-block:: python import os_client_config cloud_config = os_client_config.OpenStackConfig().get_all_clouds() for cloud in cloud_config: print(cloud.name, cloud.region, cloud.config) + +argparse +-------- + +If you're using os-client-config from a program that wants to process +command line options, there is a registration function to register the +arguments that both os-client-config and keystoneauth know how to deal +with - as well as a consumption argument. + +.. code-block:: python + + import argparse + import sys + + import os_client_config + + cloud_config = os_client_config.OpenStackConfig() + parser = argparse.ArgumentParser() + cloud_config.register_argparse_arguments(parser, sys.argv) + + options = parser.parse_args() + + cloud = cloud_config.get_one_cloud(argparse=options) + +Constructing Legacy Client objects +---------------------------------- + +If all you want to do is get a Client object from a python-\*client library, +and you want it to do all the normal things related to clouds.yaml, `OS_` +environment variables, a helper function is provided. The following +will get you a fully configured `novaclient` instance. + +.. code-block:: python + + import argparse + + import os_client_config + + nova = os_client_config.make_client('compute') + +If you want to do the same thing but also support command line parsing. + +.. code-block:: python + + import argparse + + import os_client_config + + nova = os_client_config.make_client( + 'compute', options=argparse.ArgumentParser()) + +If you want to get fancier than that in your python, then the rest of the +API is available to you. But often times, you just want to do the one thing. + +Source +------ + +* Free software: Apache license +* Documentation: http://docs.openstack.org/developer/os-client-config +* Source: http://git.openstack.org/cgit/openstack/os-client-config +* Bugs: http://bugs.launchpad.net/os-client-config diff --git a/doc/source/conf.py b/doc/source/conf.py index 221de3c..208517c 100755 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -23,7 +23,8 @@ sys.path.insert(0, os.path.abspath('../..')) extensions = [ 'sphinx.ext.autodoc', #'sphinx.ext.intersphinx', - 'oslosphinx' + 'oslosphinx', + 'reno.sphinxext' ] # autodoc generation is a bit aggressive and a nuisance when doing heavy diff --git a/doc/source/index.rst b/doc/source/index.rst index cc5dbf4..bf667b7 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -7,6 +7,7 @@ contributing installation api-reference + releasenotes Indices and tables ================== diff --git a/doc/source/releasenotes.rst b/doc/source/releasenotes.rst new file mode 100644 index 0000000..2a4bceb --- /dev/null +++ b/doc/source/releasenotes.rst @@ -0,0 +1,5 @@ +============= +Release Notes +============= + +.. release-notes:: diff --git a/doc/source/vendor-support.rst b/doc/source/vendor-support.rst index 90fd31f..46c95d8 100644 --- a/doc/source/vendor-support.rst +++ b/doc/source/vendor-support.rst @@ -16,6 +16,7 @@ These are the default behaviors unless a cloud is configured differently. * Identity uses `password` authentication * Identity API Version is 2 * Image API Version is 2 +* Volume API Version is 2 * Images must be in `qcow2` format * Images are uploaded using PUT interface * Public IPv4 is directly routable via DHCP from Neutron @@ -51,6 +52,7 @@ nz_wlg_2 Wellington, NZ * Image API Version is 1 * Images must be in `raw` format +* Volume API Version is 1 citycloud --------- @@ -67,11 +69,12 @@ Kna1 Karlskrona, SE * Identity API Version is 3 * Public IPv4 is provided via NAT with Neutron Floating IP +* Volume API Version is 1 conoha ------ -https://identity.%(region_name)s.conoha.io/v2.0 +https://identity.%(region_name)s.conoha.io ============== ================ Region Name Human Name @@ -86,7 +89,7 @@ sjc1 San Jose, CA datacentred ----------- -https://compute.datacentred.io:5000/v2.0 +https://compute.datacentred.io:5000 ============== ================ Region Name Human Name @@ -137,6 +140,8 @@ it-mil1 Milan, IT de-fra1 Frankfurt, DE ============== ================ +* Volume API Version is 1 + hp -- @@ -151,6 +156,20 @@ region-b.geo-1 US East * DNS Service Type is `hpext:dns` * Image API Version is 1 +* Public IPv4 is provided via NAT with Neutron Floating IP +* Volume API Version is 1 + +ibmcloud +-------- + +https://identity.open.softlayer.com + +============== ================ +Region Name Human Name +============== ================ +london London, UK +============== ================ + * Public IPv4 is provided via NAT with Neutron Floating IP internap @@ -212,6 +231,7 @@ SYD Sydney * Uploaded Images need properties to not use vendor agent:: :vm_mode: hvm :xenapi_use_agent: False +* Volume API Version is 1 runabove -------- @@ -241,6 +261,7 @@ ZH Zurich, CH * Images must be in `raw` format * Images must be uploaded using the Glance Task Interface +* Volume API Version is 1 ultimum ------- @@ -253,6 +274,8 @@ Region Name Human Name RegionOne Region One ============== ================ +* Volume API Version is 1 + unitedstack ----------- @@ -267,14 +290,18 @@ gd1 Guangdong * Identity API Version is 3 * Images must be in `raw` format +* Volume API Version is 1 vexxhost -------- -http://auth.api.thenebulacloud.com:5000/v2.0/ +http://auth.vexxhost.net ============== ================ Region Name Human Name ============== ================ ca-ymq-1 Montreal ============== ================ + +* DNS API Version is 1 +* Identity API Version is 3 diff --git a/openstack-common.conf b/openstack-common.conf deleted file mode 100644 index e8eb2aa..0000000 --- a/openstack-common.conf +++ /dev/null @@ -1,6 +0,0 @@ -[DEFAULT] - -# The list of modules to copy from oslo-incubator.git - -# The base module to hold the copy of openstack.common -base=os_client_config \ No newline at end of file diff --git a/os_client_config/__init__.py b/os_client_config/__init__.py index 00e6ff5..52fcb85 100644 --- a/os_client_config/__init__.py +++ b/os_client_config/__init__.py @@ -12,6 +12,9 @@ # License for the specific language governing permissions and limitations # under the License. +import sys + +from os_client_config import cloud_config from os_client_config.config import OpenStackConfig # noqa @@ -29,4 +32,27 @@ def simple_client(service_key, cloud=None, region_name=None): at OpenStack REST APIs with a properly configured keystone session. """ return OpenStackConfig().get_one_cloud( - cloud=cloud, region_name=region_name).get_session_client('compute') + cloud=cloud, region_name=region_name).get_session_client(service_key) + + +def make_client(service_key, constructor=None, options=None, **kwargs): + """Simple wrapper for getting a client instance from a client lib. + + OpenStack Client Libraries all have a fairly consistent constructor + interface which os-client-config supports. In the simple case, there + is one and only one right way to construct a client object. If as a user + you don't want to do fancy things, just use this. It honors OS_ environment + variables and clouds.yaml - and takes as **kwargs anything you'd expect + to pass in. + """ + if not constructor: + constructor = cloud_config._get_client(service_key) + config = OpenStackConfig() + if options: + config.register_argparse_options(options, sys.argv, service_key) + parsed_options = options.parse_args(sys.argv) + else: + parsed_options = None + + cloud = config.get_one_cloud(options=parsed_options, **kwargs) + return cloud.get_legacy_client(service_key, constructor) diff --git a/os_client_config/cloud_config.py b/os_client_config/cloud_config.py index 18ea4c1..85c6f2a 100644 --- a/os_client_config/cloud_config.py +++ b/os_client_config/cloud_config.py @@ -12,6 +12,7 @@ # License for the specific language governing permissions and limitations # under the License. +import importlib import warnings from keystoneauth1 import adapter @@ -20,9 +21,44 @@ from keystoneauth1 import session import requestsexceptions from os_client_config import _log +from os_client_config import constructors from os_client_config import exceptions +def _get_client(service_key): + class_mapping = constructors.get_constructor_mapping() + if service_key not in class_mapping: + raise exceptions.OpenStackConfigException( + "Service {service_key} is unkown. Please pass in a client" + " constructor or submit a patch to os-client-config".format( + service_key=service_key)) + mod_name, ctr_name = class_mapping[service_key].rsplit('.', 1) + lib_name = mod_name.split('.')[0] + try: + mod = importlib.import_module(mod_name) + except ImportError: + raise exceptions.OpenStackConfigException( + "Client for '{service_key}' was requested, but" + " {mod_name} was unable to be imported. Either import" + " the module yourself and pass the constructor in as an argument," + " or perhaps you do not have python-{lib_name} installed.".format( + service_key=service_key, + mod_name=mod_name, + lib_name=lib_name)) + try: + ctr = getattr(mod, ctr_name) + except AttributeError: + raise exceptions.OpenStackConfigException( + "Client for '{service_key}' was requested, but although" + " {mod_name} imported fine, the constructor at {fullname}" + " as not found. Please check your installation, we have no" + " clue what is wrong with your computer.".format( + service_key=service_key, + mod_name=mod_name, + fullname=class_mapping[service_key])) + return ctr + + def _make_key(key, service_type): if not service_type: return key @@ -128,8 +164,9 @@ class CloudConfig(object): return self.config.get(key, None) def get_endpoint(self, service_type): - key = _make_key('endpoint', service_type) - return self.config.get(key, None) + key = _make_key('endpoint_override', service_type) + old_key = _make_key('endpoint', service_type) + return self.config.get(key, self.config.get(old_key, None)) @property def prefer_ipv6(self): @@ -217,8 +254,8 @@ class CloudConfig(object): return endpoint def get_legacy_client( - self, service_key, client_class, interface_key=None, - pass_version_arg=True, **kwargs): + self, service_key, client_class=None, interface_key=None, + pass_version_arg=True, version=None, **kwargs): """Return a legacy OpenStack client object for the given config. Most of the OpenStack python-*client libraries have the same @@ -250,19 +287,25 @@ class CloudConfig(object): already understand that this is the case for network, so it can be omitted in that case. + :param version: (optional) Version string to override the configured + version string. :param kwargs: (optional) keyword args are passed through to the Client constructor, so this is in case anything additional needs to be passed in. """ + if not client_class: + client_class = _get_client(service_key) + # Because of course swift is different if service_key == 'object-store': return self._get_swift_client(client_class=client_class, **kwargs) interface = self.get_interface(service_key) # trigger exception on lack of service endpoint = self.get_session_endpoint(service_key) + endpoint_override = self.get_endpoint(service_key) if not interface_key: - if service_key == 'image': + if service_key in ('image', 'key-manager'): interface_key = 'interface' else: interface_key = 'endpoint_type' @@ -271,6 +314,7 @@ class CloudConfig(object): session=self.get_session(), service_name=self.get_service_name(service_key), service_type=self.get_service_type(service_key), + endpoint_override=endpoint_override, region_name=self.region) if service_key == 'image': @@ -279,13 +323,21 @@ class CloudConfig(object): # would need to do if they were requesting 'image' - then # they necessarily have glanceclient installed from glanceclient.common import utils as glance_utils - endpoint, version = glance_utils.strip_version(endpoint) - constructor_kwargs['endpoint'] = endpoint + endpoint, detected_version = glance_utils.strip_version(endpoint) + # If the user has passed in a version, that's explicit, use it + if not version: + version = detected_version + # If the user has passed in or configured an override, use it. + # Otherwise, ALWAYS pass in an endpoint_override becuase + # we've already done version stripping, so we don't want version + # reconstruction to happen twice + if not endpoint_override: + constructor_kwargs['endpoint_override'] = endpoint constructor_kwargs.update(kwargs) constructor_kwargs[interface_key] = interface - constructor_args = [] if pass_version_arg: - version = self.get_api_version(service_key) + if not version: + version = self.get_api_version(service_key) # Temporary workaround while we wait for python-openstackclient # to be able to handle 2.0 which is what neutronclient expects if service_key == 'network' and version == '2': @@ -295,9 +347,12 @@ class CloudConfig(object): if 'endpoint' not in constructor_kwargs: endpoint = self.get_session_endpoint('identity') constructor_kwargs['endpoint'] = endpoint - constructor_args.append(version) + if service_key == 'network': + constructor_kwargs['api_version'] = version + else: + constructor_kwargs['version'] = version - return client_class(*constructor_args, **constructor_kwargs) + return client_class(**constructor_kwargs) def _get_swift_client(self, client_class, **kwargs): session = self.get_session() diff --git a/os_client_config/config.py b/os_client_config/config.py index 5f7c402..378cd3b 100644 --- a/os_client_config/config.py +++ b/os_client_config/config.py @@ -13,17 +13,21 @@ # under the License. +# alias because we already had an option named argparse +import argparse as argparse_mod +import collections +import copy import json import os +import sys import warnings import appdirs -try: - from keystoneauth1 import loading -except ImportError: - loading = None +from keystoneauth1 import adapter +from keystoneauth1 import loading import yaml +from os_client_config import _log from os_client_config import cloud_config from os_client_config import defaults from os_client_config import exceptions @@ -51,6 +55,11 @@ CONFIG_FILES = [ for d in CONFIG_SEARCH_PATH for s in YAML_SUFFIXES + JSON_SUFFIXES ] +SECURE_FILES = [ + os.path.join(d, 'secure' + s) + for d in CONFIG_SEARCH_PATH + for s in YAML_SUFFIXES + JSON_SUFFIXES +] VENDOR_FILES = [ os.path.join(d, 'clouds-public' + s) for d in CONFIG_SEARCH_PATH @@ -102,8 +111,23 @@ def _get_os_environ(envvar_prefix=None): return ret -def _auth_update(old_dict, new_dict): +def _merge_clouds(old_dict, new_dict): + """Like dict.update, except handling nested dicts.""" + ret = old_dict.copy() + for (k, v) in new_dict.items(): + if isinstance(v, dict): + if k in ret: + ret[k] = _merge_clouds(ret[k], v) + else: + ret[k] = v.copy() + else: + ret[k] = v + return ret + + +def _auth_update(old_dict, new_dict_source): """Like dict.update, except handling the nested dict called auth.""" + new_dict = copy.deepcopy(new_dict_source) for (k, v) in new_dict.items(): if k == 'auth': if k in old_dict: @@ -115,24 +139,63 @@ def _auth_update(old_dict, new_dict): return old_dict +def _fix_argv(argv): + # Transform any _ characters in arg names to - so that we don't + # have to throw billions of compat argparse arguments around all + # over the place. + processed = collections.defaultdict(list) + for index in range(0, len(argv)): + if argv[index].startswith('--'): + split_args = argv[index].split('=') + orig = split_args[0] + new = orig.replace('_', '-') + if orig != new: + split_args[0] = new + argv[index] = "=".join(split_args) + # Save both for later so we can throw an error about dupes + processed[new].append(orig) + overlap = [] + for new, old in processed.items(): + if len(old) > 1: + overlap.extend(old) + if overlap: + raise exceptions.OpenStackConfigException( + "The following options were given: '{options}' which contain" + " duplicates except that one has _ and one has -. There is" + " no sane way for us to know what you're doing. Remove the" + " duplicate option and try again".format( + options=','.join(overlap))) + + class OpenStackConfig(object): def __init__(self, config_files=None, vendor_files=None, override_defaults=None, force_ipv4=None, - envvar_prefix=None): + envvar_prefix=None, secure_files=None): + self.log = _log.setup_logging(__name__) + self._config_files = config_files or CONFIG_FILES + self._secure_files = secure_files or SECURE_FILES self._vendor_files = vendor_files or VENDOR_FILES config_file_override = os.environ.pop('OS_CLIENT_CONFIG_FILE', None) if config_file_override: self._config_files.insert(0, config_file_override) + secure_file_override = os.environ.pop('OS_CLIENT_SECURE_FILE', None) + if secure_file_override: + self._secure_files.insert(0, secure_file_override) + self.defaults = defaults.get_defaults() if override_defaults: self.defaults.update(override_defaults) # First, use a config file if it exists where expected self.config_filename, self.cloud_config = self._load_config_file() + _, secure_config = self._load_secure_file() + if secure_config: + self.cloud_config = _merge_clouds( + self.cloud_config, secure_config) if not self.cloud_config: self.cloud_config = {'clouds': {}} @@ -180,6 +243,8 @@ class OpenStackConfig(object): envvars = _get_os_environ(envvar_prefix=envvar_prefix) if envvars: self.cloud_config['clouds'][self.envvar_key] = envvars + if not self.default_cloud: + self.default_cloud = self.envvar_key # Finally, fall through and make a cloud that starts with defaults # because we need somewhere to put arguments, and there are neither @@ -187,6 +252,7 @@ class OpenStackConfig(object): if not self.cloud_config['clouds']: self.cloud_config = dict( clouds=dict(defaults=dict(self.defaults))) + self.default_cloud = 'defaults' self._cache_expiration_time = 0 self._cache_path = CACHE_PATH @@ -217,9 +283,28 @@ class OpenStackConfig(object): self._cache_expiration = cache_settings.get( 'expiration', self._cache_expiration) + # Flag location to hold the peeked value of an argparse timeout value + self._argv_timeout = False + + def get_extra_config(self, key, defaults=None): + """Fetch an arbitrary extra chunk of config, laying in defaults. + + :param string key: name of the config section to fetch + :param dict defaults: (optional) default values to merge under the + found config + """ + if not defaults: + defaults = {} + return _merge_clouds( + self._normalize_keys(defaults), + self._normalize_keys(self.cloud_config.get(key, {}))) + def _load_config_file(self): return self._load_yaml_json_file(self._config_files) + def _load_secure_file(self): + return self._load_yaml_json_file(self._secure_files) + def _load_vendor_file(self): return self._load_yaml_json_file(self._vendor_files) @@ -231,7 +316,7 @@ class OpenStackConfig(object): return path, json.load(f) else: return path, yaml.safe_load(f) - return (None, None) + return (None, {}) def _normalize_keys(self, config): new_config = {} @@ -265,17 +350,36 @@ class OpenStackConfig(object): return self._cache_class def get_cache_arguments(self): - return self._cache_arguments.copy() + return copy.deepcopy(self._cache_arguments) def get_cache_expiration(self): - return self._cache_expiration.copy() + return copy.deepcopy(self._cache_expiration) + + def _expand_region_name(self, region_name): + return {'name': region_name, 'values': {}} + + def _expand_regions(self, regions): + ret = [] + for region in regions: + if isinstance(region, dict): + ret.append(copy.deepcopy(region)) + else: + ret.append(self._expand_region_name(region)) + return ret def _get_regions(self, cloud): if cloud not in self.cloud_config['clouds']: - return [''] + return [self._expand_region_name('')] + regions = self._get_known_regions(cloud) + if not regions: + # We don't know of any regions use a workable default. + regions = [self._expand_region_name('')] + return regions + + def _get_known_regions(self, cloud): config = self._normalize_keys(self.cloud_config['clouds'][cloud]) if 'regions' in config: - return config['regions'] + return self._expand_regions(config['regions']) elif 'region_name' in config: regions = config['region_name'].split(',') if len(regions) > 1: @@ -283,22 +387,41 @@ class OpenStackConfig(object): "Comma separated lists in region_name are deprecated." " Please use a yaml list in the regions" " parameter in {0} instead.".format(self.config_filename)) - return regions + return self._expand_regions(regions) else: # crappit. we don't have a region defined. new_cloud = dict() our_cloud = self.cloud_config['clouds'].get(cloud, dict()) self._expand_vendor_profile(cloud, new_cloud, our_cloud) if 'regions' in new_cloud and new_cloud['regions']: - return new_cloud['regions'] + return self._expand_regions(new_cloud['regions']) elif 'region_name' in new_cloud and new_cloud['region_name']: - return [new_cloud['region_name']] - else: - # Wow. We really tried - return [''] + return [self._expand_region_name(new_cloud['region_name'])] - def _get_region(self, cloud=None): - return self._get_regions(cloud)[0] + def _get_region(self, cloud=None, region_name=''): + if region_name is None: + region_name = '' + if not cloud: + return self._expand_region_name(region_name) + + regions = self._get_known_regions(cloud) + if not regions: + return self._expand_region_name(region_name) + + if not region_name: + return regions[0] + + for region in regions: + if region['name'] == region_name: + return region + + raise exceptions.OpenStackConfigException( + 'Region {region_name} is not a valid region name for cloud' + ' {cloud}. Valid choices are {region_list}. Please note that' + ' region names are case sensitive.'.format( + region_name=region_name, + region_list=','.join([r['name'] for r in regions]), + cloud=cloud)) def get_cloud_names(self): return self.cloud_config['clouds'].keys() @@ -325,7 +448,7 @@ class OpenStackConfig(object): if 'cloud' in cloud: del cloud['cloud'] - return self._fix_backwards_madness(cloud) + return cloud def _expand_vendor_profile(self, name, cloud, our_cloud): # Expand a profile if it exists. 'cloud' is an old confusing name @@ -386,6 +509,7 @@ class OpenStackConfig(object): 'project_domain_id': ('project_domain_id', 'project-domain-id'), 'project_domain_name': ( 'project_domain_name', 'project-domain-name'), + 'token': ('auth-token', 'auth_token', 'token'), } for target_key, possible_values in mappings.items(): target = None @@ -420,6 +544,104 @@ class OpenStackConfig(object): cloud['auth_type'] = 'password' return cloud + def register_argparse_arguments(self, parser, argv, service_keys=[]): + """Register all of the common argparse options needed. + + Given an argparse parser, register the keystoneauth Session arguments, + the keystoneauth Auth Plugin Options and os-cloud. Also, peek in the + argv to see if all of the auth plugin options should be registered + or merely the ones already configured. + :param argparse.ArgumentParser: parser to attach argparse options to + :param list argv: the arguments provided to the application + :param string service_keys: Service or list of services this argparse + should be specialized for, if known. + The first item in the list will be used + as the default value for service_type + (optional) + + :raises exceptions.OpenStackConfigException if an invalid auth-type + is requested + """ + + # Fix argv in place - mapping any keys with embedded _ in them to - + _fix_argv(argv) + + local_parser = argparse_mod.ArgumentParser(add_help=False) + + for p in (parser, local_parser): + p.add_argument( + '--os-cloud', + metavar='', + default=os.environ.get('OS_CLOUD', None), + help='Named cloud to connect to') + + # we need to peek to see if timeout was actually passed, since + # the keystoneauth declaration of it has a default, which means + # we have no clue if the value we get is from the ksa default + # for from the user passing it explicitly. We'll stash it for later + local_parser.add_argument('--timeout', metavar='') + + # We need for get_one_cloud to be able to peek at whether a token + # was passed so that we can swap the default from password to + # token if it was. And we need to also peek for --os-auth-token + # for novaclient backwards compat + local_parser.add_argument('--os-token') + local_parser.add_argument('--os-auth-token') + + # Peek into the future and see if we have an auth-type set in + # config AND a cloud set, so that we know which command line + # arguments to register and show to the user (the user may want + # to say something like: + # openstack --os-cloud=foo --os-oidctoken=bar + # although I think that user is the cause of my personal pain + options, _args = local_parser.parse_known_args(argv) + if options.timeout: + self._argv_timeout = True + + # validate = False because we're not _actually_ loading here + # we're only peeking, so it's the wrong time to assert that + # the rest of the arguments given are invalid for the plugin + # chosen (for instance, --help may be requested, so that the + # user can see what options he may want to give + cloud = self.get_one_cloud(argparse=options, validate=False) + default_auth_type = cloud.config['auth_type'] + + try: + loading.register_auth_argparse_arguments( + parser, argv, default=default_auth_type) + except Exception: + # Hidiing the keystoneauth exception because we're not actually + # loading the auth plugin at this point, so the error message + # from it doesn't actually make sense to os-client-config users + options, _args = parser.parse_known_args(argv) + plugin_names = loading.get_available_plugin_names() + raise exceptions.OpenStackConfigException( + "An invalid auth-type was specified: {auth_type}." + " Valid choices are: {plugin_names}.".format( + auth_type=options.os_auth_type, + plugin_names=",".join(plugin_names))) + + if service_keys: + primary_service = service_keys[0] + else: + primary_service = None + loading.register_session_argparse_arguments(parser) + adapter.register_adapter_argparse_arguments( + parser, service_type=primary_service) + for service_key in service_keys: + # legacy clients have un-prefixed api-version options + parser.add_argument( + '--{service_key}-api-version'.format( + service_key=service_key.replace('_', '-'), + help=argparse_mod.SUPPRESS)) + adapter.register_service_adapter_argparse_arguments( + parser, service_type=service_key) + + # Backwards compat options for legacy clients + parser.add_argument('--http-timeout', help=argparse_mod.SUPPRESS) + parser.add_argument('--os-endpoint-type', help=argparse_mod.SUPPRESS) + parser.add_argument('--endpoint-type', help=argparse_mod.SUPPRESS) + def _fix_backwards_interface(self, cloud): new_cloud = {} for key in cloud.keys(): @@ -430,13 +652,39 @@ class OpenStackConfig(object): new_cloud[target_key] = cloud[key] return new_cloud + def _fix_backwards_api_timeout(self, cloud): + new_cloud = {} + # requests can only have one timeout, which means that in a single + # cloud there is no point in different timeout values. However, + # for some reason many of the legacy clients decided to shove their + # service name in to the arg name for reasons surpassin sanity. If + # we find any values that are not api_timeout, overwrite api_timeout + # with the value + service_timeout = None + for key in cloud.keys(): + if key.endswith('timeout') and not ( + key == 'timeout' or key == 'api_timeout'): + service_timeout = cloud[key] + else: + new_cloud[key] = cloud[key] + if service_timeout is not None: + new_cloud['api_timeout'] = service_timeout + # The common argparse arg from keystoneauth is called timeout, but + # os-client-config expects it to be called api_timeout + if self._argv_timeout: + if 'timeout' in new_cloud and new_cloud['timeout']: + new_cloud['api_timeout'] = new_cloud.pop('timeout') + return new_cloud + def get_all_clouds(self): clouds = [] for cloud in self.get_cloud_names(): for region in self._get_regions(cloud): - clouds.append(self.get_one_cloud(cloud, region_name=region)) + if region: + clouds.append(self.get_one_cloud( + cloud, region_name=region['name'])) return clouds def _fix_args(self, args, argparse=None): @@ -607,38 +855,49 @@ class OpenStackConfig(object): on missing required auth parameters """ - if cloud is None and self.default_cloud: - cloud = self.default_cloud - - if cloud is None and self.envvar_key in self.get_cloud_names(): - cloud = self.envvar_key - args = self._fix_args(kwargs, argparse=argparse) - if 'region_name' not in args or args['region_name'] is None: - args['region_name'] = self._get_region(cloud) + if cloud is None: + if 'cloud' in args: + cloud = args['cloud'] + else: + cloud = self.default_cloud config = self._get_base_cloud_config(cloud) + # Get region specific settings + if 'region_name' not in args: + args['region_name'] = '' + region = self._get_region(cloud=cloud, region_name=args['region_name']) + args['region_name'] = region['name'] + region_args = copy.deepcopy(region['values']) + # Regions is a list that we can use to create a list of cloud/region # objects. It does not belong in the single-cloud dict - regions = config.pop('regions', None) - if regions and args['region_name'] not in regions: - raise exceptions.OpenStackConfigException( - 'Region {region_name} is not a valid region name for cloud' - ' {cloud}. Valid choices are {region_list}. Please note that' - ' region names are case sensitive.'.format( - region_name=args['region_name'], - region_list=','.join(regions), - cloud=cloud)) + config.pop('regions', None) # Can't just do update, because None values take over - for (key, val) in iter(args.items()): - if val is not None: - if key == 'auth' and config[key] is not None: - config[key] = _auth_update(config[key], val) - else: - config[key] = val + for arg_list in region_args, args: + for (key, val) in iter(arg_list.items()): + if val is not None: + if key == 'auth' and config[key] is not None: + config[key] = _auth_update(config[key], val) + else: + config[key] = val + + # Infer token plugin if a token was given + if (('auth' in config and 'token' in config['auth']) or + ('auth_token' in config and config['auth_token']) or + ('token' in config and config['token'])): + config.setdefault('token', config.pop('auth_token', None)) + + # These backwards compat values are only set via argparse. If it's + # there, it's because it was passed in explicitly, and should win + config = self._fix_backwards_api_timeout(config) + if 'endpoint_type' in config: + config['interface'] = config.pop('endpoint_type') + + config = self._fix_backwards_madness(config) for key in BOOL_KEYS: if key in config: @@ -657,27 +916,24 @@ class OpenStackConfig(object): # compatible behaviour config = self.auth_config_hook(config) - if loading: - if validate: - try: - loader = self._get_auth_loader(config) - config = self._validate_auth(config, loader) - auth_plugin = loader.load_from_options(**config['auth']) - except Exception as e: - # We WANT the ksa exception normally - # but OSC can't handle it right now, so we try deferring - # to ksc. If that ALSO fails, it means there is likely - # a deeper issue, so we assume the ksa error was correct - auth_plugin = None - try: - config = self._validate_auth_ksc(config) - except Exception: - raise e - else: + if validate: + try: + loader = self._get_auth_loader(config) + config = self._validate_auth(config, loader) + auth_plugin = loader.load_from_options(**config['auth']) + except Exception as e: + # We WANT the ksa exception normally + # but OSC can't handle it right now, so we try deferring + # to ksc. If that ALSO fails, it means there is likely + # a deeper issue, so we assume the ksa error was correct + self.log.debug("Deferring keystone exception: {e}".format(e=e)) auth_plugin = None + try: + config = self._validate_auth_ksc(config) + except Exception: + raise e else: auth_plugin = None - config = self._validate_auth_ksc(config) # If any of the defaults reference other values, we need to expand for (key, value) in config.items(): @@ -735,4 +991,15 @@ class OpenStackConfig(object): if __name__ == '__main__': config = OpenStackConfig().get_all_clouds() for cloud in config: - print(cloud.name, cloud.region, cloud.config) + print_cloud = False + if len(sys.argv) == 1: + print_cloud = True + elif len(sys.argv) == 3 and ( + sys.argv[1] == cloud.name and sys.argv[2] == cloud.region): + print_cloud = True + elif len(sys.argv) == 2 and ( + sys.argv[1] == cloud.name): + print_cloud = True + + if print_cloud: + print(cloud.name, cloud.region, cloud.config) diff --git a/os_client_config/constructors.json b/os_client_config/constructors.json new file mode 100644 index 0000000..89c844c --- /dev/null +++ b/os_client_config/constructors.json @@ -0,0 +1,12 @@ +{ + "compute": "novaclient.client.Client", + "database": "troveclient.client.Client", + "identity": "keystoneclient.client.Client", + "image": "glanceclient.Client", + "key-manager": "barbicanclient.client.Client", + "metering": "ceilometerclient.client.Client", + "network": "neutronclient.neutron.client.Client", + "object-store": "swiftclient.client.Connection", + "orchestration": "heatclient.client.Client", + "volume": "cinderclient.client.Client" +} diff --git a/os_client_config/constructors.py b/os_client_config/constructors.py new file mode 100644 index 0000000..e88ac92 --- /dev/null +++ b/os_client_config/constructors.py @@ -0,0 +1,28 @@ +# Copyright (c) 2014 Hewlett-Packard Development Company, L.P. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +import json +import os + +_json_path = os.path.join( + os.path.dirname(os.path.realpath(__file__)), 'constructors.json') +_class_mapping = None + + +def get_constructor_mapping(): + global _class_mapping + if not _class_mapping: + with open(_json_path, 'r') as json_file: + _class_mapping = json.load(json_file) + return _class_mapping diff --git a/os_client_config/defaults.json b/os_client_config/defaults.json index eb8162e..f501862 100644 --- a/os_client_config/defaults.json +++ b/os_client_config/defaults.json @@ -12,9 +12,11 @@ "image_api_use_tasks": false, "image_api_version": "2", "image_format": "qcow2", + "key_manager_api_version": "v1", + "metering_api_version": "2", "network_api_version": "2", "object_store_api_version": "1", "orchestration_api_version": "1", "secgroup_source": "neutron", - "volume_api_version": "1" + "volume_api_version": "2" } diff --git a/os_client_config/tests/base.py b/os_client_config/tests/base.py index 3d94e25..3f00f6d 100644 --- a/os_client_config/tests/base.py +++ b/os_client_config/tests/base.py @@ -16,6 +16,7 @@ # under the License. +import copy import os import tempfile @@ -64,7 +65,6 @@ USER_CONF = { 'auth': { 'auth_url': 'http://example.com/v2', 'username': 'testuser', - 'password': 'testpass', 'project_name': 'testproject', }, 'region-name': 'test-region', @@ -97,8 +97,18 @@ USER_CONF = { 'auth_url': 'http://example.com/v2', }, 'regions': [ - 'region1', - 'region2', + { + 'name': 'region1', + 'values': { + 'external_network': 'region1-network', + } + }, + { + 'name': 'region2', + 'values': { + 'external_network': 'my-network', + } + } ], }, '_test_cloud_hyphenated': { @@ -109,8 +119,29 @@ USER_CONF = { 'auth_url': 'http://example.com/v2', }, 'region_name': 'test-region', - } + }, + '_test-cloud_no_region': { + 'profile': '_test_cloud_in_our_cloud', + 'auth': { + 'auth_url': 'http://example.com/v2', + 'username': 'testuser', + 'password': 'testpass', + }, + }, }, + 'ansible': { + 'expand-hostvars': False, + 'use_hostnames': True, + }, +} +SECURE_CONF = { + 'clouds': { + '_test_cloud_no_vendor': { + 'auth': { + 'password': 'testpass', + }, + } + } } NO_CONF = { 'cache': {'max_age': 1}, @@ -131,10 +162,11 @@ class TestCase(base.BaseTestCase): super(TestCase, self).setUp() self.useFixture(fixtures.NestedTempfile()) - conf = dict(USER_CONF) + conf = copy.deepcopy(USER_CONF) tdir = self.useFixture(fixtures.TempDir()) conf['cache']['path'] = tdir.path self.cloud_yaml = _write_yaml(conf) + self.secure_yaml = _write_yaml(SECURE_CONF) self.vendor_yaml = _write_yaml(VENDOR_CONF) self.no_yaml = _write_yaml(NO_CONF) @@ -155,6 +187,7 @@ class TestCase(base.BaseTestCase): self.assertIsNone(cc.cloud) self.assertIn('username', cc.auth) self.assertEqual('testuser', cc.auth['username']) + self.assertEqual('testpass', cc.auth['password']) self.assertFalse(cc.config['image_api_use_tasks']) self.assertTrue('project_name' in cc.auth or 'project_id' in cc.auth) if 'project_name' in cc.auth: diff --git a/os_client_config/tests/test_cloud_config.py b/os_client_config/tests/test_cloud_config.py index 9e683d1..a01d0e1 100644 --- a/os_client_config/tests/test_cloud_config.py +++ b/os_client_config/tests/test_cloud_config.py @@ -25,8 +25,9 @@ from os_client_config.tests import base fake_config_dict = {'a': 1, 'os_b': 2, 'c': 3, 'os_c': 4} fake_services_dict = { 'compute_api_version': '2', - 'compute_endpoint': 'http://compute.example.com', + 'compute_endpoint_override': 'http://compute.example.com', 'compute_region_name': 'region-bl', + 'telemetry_endpoint': 'http://telemetry.example.com', 'interface': 'public', 'image_service_type': 'mage', 'identity_interface': 'admin', @@ -47,7 +48,7 @@ class TestCloudConfig(base.TestCase): self.assertEqual(1, cc.a) # Look up prefixed attribute, fail - returns None - self.assertEqual(None, cc.os_b) + self.assertIsNone(cc.os_b) # Look up straight value, then prefixed value self.assertEqual(3, cc.c) @@ -139,7 +140,7 @@ class TestCloudConfig(base.TestCase): self.assertEqual('region-al', cc.get_region_name()) self.assertEqual('region-al', cc.get_region_name('image')) self.assertEqual('region-bl', cc.get_region_name('compute')) - self.assertEqual(None, cc.get_api_version('image')) + self.assertIsNone(cc.get_api_version('image')) self.assertEqual('2', cc.get_api_version('compute')) self.assertEqual('mage', cc.get_service_type('image')) self.assertEqual('compute', cc.get_service_type('compute')) @@ -147,9 +148,8 @@ class TestCloudConfig(base.TestCase): self.assertEqual('volume', cc.get_service_type('volume')) self.assertEqual('http://compute.example.com', cc.get_endpoint('compute')) - self.assertEqual(None, - cc.get_endpoint('image')) - self.assertEqual(None, cc.get_service_name('compute')) + self.assertIsNone(cc.get_endpoint('image')) + self.assertIsNone(cc.get_service_name('compute')) self.assertEqual('locks', cc.get_service_name('identity')) def test_volume_override(self): @@ -189,14 +189,24 @@ class TestCloudConfig(base.TestCase): verify=True, cert=None, timeout=9) @mock.patch.object(ksa_session, 'Session') - def test_override_session_endpoint(self, mock_session): + def test_override_session_endpoint_override(self, mock_session): config_dict = defaults.get_defaults() config_dict.update(fake_services_dict) cc = cloud_config.CloudConfig( "test1", "region-al", config_dict, auth_plugin=mock.Mock()) self.assertEqual( cc.get_session_endpoint('compute'), - fake_services_dict['compute_endpoint']) + fake_services_dict['compute_endpoint_override']) + + @mock.patch.object(ksa_session, 'Session') + def test_override_session_endpoint(self, mock_session): + config_dict = defaults.get_defaults() + config_dict.update(fake_services_dict) + cc = cloud_config.CloudConfig( + "test1", "region-al", config_dict, auth_plugin=mock.Mock()) + self.assertEqual( + cc.get_session_endpoint('telemetry'), + fake_services_dict['telemetry_endpoint']) @mock.patch.object(cloud_config.CloudConfig, 'get_session') def test_session_endpoint_identity(self, mock_get_session): @@ -294,9 +304,96 @@ class TestCloudConfig(base.TestCase): "test1", "region-al", config_dict, auth_plugin=mock.Mock()) cc.get_legacy_client('image', mock_client) mock_client.assert_called_with( - '2', + version=2.0, service_name=None, - endpoint='http://example.com', + endpoint_override='http://example.com', + region_name='region-al', + interface='public', + session=mock.ANY, + # Not a typo - the config dict above overrides this + service_type='mage' + ) + + @mock.patch.object(cloud_config.CloudConfig, 'get_session_endpoint') + def test_legacy_client_image_override(self, mock_get_session_endpoint): + mock_client = mock.Mock() + mock_get_session_endpoint.return_value = 'http://example.com/v2' + config_dict = defaults.get_defaults() + config_dict.update(fake_services_dict) + config_dict['image_endpoint_override'] = 'http://example.com/override' + cc = cloud_config.CloudConfig( + "test1", "region-al", config_dict, auth_plugin=mock.Mock()) + cc.get_legacy_client('image', mock_client) + mock_client.assert_called_with( + version=2.0, + service_name=None, + endpoint_override='http://example.com/override', + region_name='region-al', + interface='public', + session=mock.ANY, + # Not a typo - the config dict above overrides this + service_type='mage' + ) + + @mock.patch.object(cloud_config.CloudConfig, 'get_session_endpoint') + def test_legacy_client_image_versioned(self, mock_get_session_endpoint): + mock_client = mock.Mock() + mock_get_session_endpoint.return_value = 'http://example.com/v2' + config_dict = defaults.get_defaults() + config_dict.update(fake_services_dict) + # v2 endpoint was passed, 1 requested in config, endpoint wins + config_dict['image_api_version'] = '1' + cc = cloud_config.CloudConfig( + "test1", "region-al", config_dict, auth_plugin=mock.Mock()) + cc.get_legacy_client('image', mock_client) + mock_client.assert_called_with( + version=2.0, + service_name=None, + endpoint_override='http://example.com', + region_name='region-al', + interface='public', + session=mock.ANY, + # Not a typo - the config dict above overrides this + service_type='mage' + ) + + @mock.patch.object(cloud_config.CloudConfig, 'get_session_endpoint') + def test_legacy_client_image_unversioned(self, mock_get_session_endpoint): + mock_client = mock.Mock() + mock_get_session_endpoint.return_value = 'http://example.com/' + config_dict = defaults.get_defaults() + config_dict.update(fake_services_dict) + # Versionless endpoint, config wins + config_dict['image_api_version'] = '1' + cc = cloud_config.CloudConfig( + "test1", "region-al", config_dict, auth_plugin=mock.Mock()) + cc.get_legacy_client('image', mock_client) + mock_client.assert_called_with( + version='1', + service_name=None, + endpoint_override='http://example.com', + region_name='region-al', + interface='public', + session=mock.ANY, + # Not a typo - the config dict above overrides this + service_type='mage' + ) + + @mock.patch.object(cloud_config.CloudConfig, 'get_session_endpoint') + def test_legacy_client_image_argument(self, mock_get_session_endpoint): + mock_client = mock.Mock() + mock_get_session_endpoint.return_value = 'http://example.com/v3' + config_dict = defaults.get_defaults() + config_dict.update(fake_services_dict) + # Versionless endpoint, config wins + config_dict['image_api_version'] = '6' + cc = cloud_config.CloudConfig( + "test1", "region-al", config_dict, auth_plugin=mock.Mock()) + cc.get_legacy_client('image', mock_client, version='beef') + mock_client.assert_called_with( + version='beef', + service_name=None, + endpoint_override='http://example.com', region_name='region-al', interface='public', session=mock.ANY, @@ -314,8 +411,9 @@ class TestCloudConfig(base.TestCase): "test1", "region-al", config_dict, auth_plugin=mock.Mock()) cc.get_legacy_client('network', mock_client) mock_client.assert_called_with( - '2.0', + api_version='2.0', endpoint_type='public', + endpoint_override=None, region_name='region-al', service_type='network', session=mock.ANY, @@ -331,8 +429,9 @@ class TestCloudConfig(base.TestCase): "test1", "region-al", config_dict, auth_plugin=mock.Mock()) cc.get_legacy_client('compute', mock_client) mock_client.assert_called_with( - '2', + version='2', endpoint_type='public', + endpoint_override='http://compute.example.com', region_name='region-al', service_type='compute', session=mock.ANY, @@ -348,9 +447,10 @@ class TestCloudConfig(base.TestCase): "test1", "region-al", config_dict, auth_plugin=mock.Mock()) cc.get_legacy_client('identity', mock_client) mock_client.assert_called_with( - '2.0', + version='2.0', endpoint='http://example.com/v2', endpoint_type='admin', + endpoint_override=None, region_name='region-al', service_type='identity', session=mock.ANY, @@ -367,9 +467,10 @@ class TestCloudConfig(base.TestCase): "test1", "region-al", config_dict, auth_plugin=mock.Mock()) cc.get_legacy_client('identity', mock_client) mock_client.assert_called_with( - '3', + version='3', endpoint='http://example.com', endpoint_type='admin', + endpoint_override=None, region_name='region-al', service_type='identity', session=mock.ANY, diff --git a/os_client_config/tests/test_config.py b/os_client_config/tests/test_config.py index d225b7c..1a16bd8 100644 --- a/os_client_config/tests/test_config.py +++ b/os_client_config/tests/test_config.py @@ -17,6 +17,7 @@ import copy import os import fixtures +import testtools import yaml from os_client_config import cloud_config @@ -30,7 +31,8 @@ class TestConfig(base.TestCase): def test_get_all_clouds(self): c = config.OpenStackConfig(config_files=[self.cloud_yaml], - vendor_files=[self.vendor_yaml]) + vendor_files=[self.vendor_yaml], + secure_files=[self.no_yaml]) clouds = c.get_all_clouds() # We add one by hand because the regions cloud is going to exist # twice since it has two regions in it @@ -74,7 +76,8 @@ class TestConfig(base.TestCase): def test_get_one_cloud_with_config_files(self): c = config.OpenStackConfig(config_files=[self.cloud_yaml], - vendor_files=[self.vendor_yaml]) + vendor_files=[self.vendor_yaml], + secure_files=[self.secure_yaml]) self.assertIsInstance(c.cloud_config, dict) self.assertIn('cache', c.cloud_config) self.assertIsInstance(c.cloud_config['cache'], dict) @@ -129,7 +132,8 @@ class TestConfig(base.TestCase): def test_fallthrough(self): c = config.OpenStackConfig(config_files=[self.no_yaml], - vendor_files=[self.no_yaml]) + vendor_files=[self.no_yaml], + secure_files=[self.no_yaml]) for k in os.environ.keys(): if k.startswith('OS_'): self.useFixture(fixtures.EnvironmentVariable(k)) @@ -137,7 +141,8 @@ class TestConfig(base.TestCase): def test_prefer_ipv6_true(self): c = config.OpenStackConfig(config_files=[self.no_yaml], - vendor_files=[self.no_yaml]) + vendor_files=[self.no_yaml], + secure_files=[self.no_yaml]) cc = c.get_one_cloud(cloud='defaults', validate=False) self.assertTrue(cc.prefer_ipv6) @@ -155,7 +160,8 @@ class TestConfig(base.TestCase): def test_force_ipv4_false(self): c = config.OpenStackConfig(config_files=[self.no_yaml], - vendor_files=[self.no_yaml]) + vendor_files=[self.no_yaml], + secure_files=[self.no_yaml]) cc = c.get_one_cloud(cloud='defaults', validate=False) self.assertFalse(cc.force_ipv4) @@ -165,19 +171,29 @@ class TestConfig(base.TestCase): self.assertEqual('user', cc.auth['username']) self.assertEqual('testpass', cc.auth['password']) + def test_only_secure_yaml(self): + c = config.OpenStackConfig(config_files=['nonexistent'], + vendor_files=['nonexistent'], + secure_files=[self.secure_yaml]) + cc = c.get_one_cloud(cloud='_test_cloud_no_vendor') + self.assertEqual('testpass', cc.auth['password']) + def test_get_cloud_names(self): - c = config.OpenStackConfig(config_files=[self.cloud_yaml]) + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + secure_files=[self.no_yaml]) self.assertEqual( ['_test-cloud-domain-id_', '_test-cloud-int-project_', '_test-cloud_', + '_test-cloud_no_region', '_test_cloud_hyphenated', '_test_cloud_no_vendor', '_test_cloud_regions', ], sorted(c.get_cloud_names())) c = config.OpenStackConfig(config_files=[self.no_yaml], - vendor_files=[self.no_yaml]) + vendor_files=[self.no_yaml], + secure_files=[self.no_yaml]) for k in os.environ.keys(): if k.startswith('OS_'): self.useFixture(fixtures.EnvironmentVariable(k)) @@ -218,8 +234,72 @@ class TestConfig(base.TestCase): new_config) with open(self.cloud_yaml) as fh: written_config = yaml.safe_load(fh) + # We write a cache config for testing + written_config['cache'].pop('path', None) self.assertEqual(written_config, resulting_config) + def test_get_region_no_region_default(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml], + secure_files=[self.no_yaml]) + region = c._get_region(cloud='_test-cloud_no_region') + self.assertEqual(region, {'name': '', 'values': {}}) + + def test_get_region_no_region(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml], + secure_files=[self.no_yaml]) + region = c._get_region(cloud='_test-cloud_no_region', + region_name='override-region') + self.assertEqual(region, {'name': 'override-region', 'values': {}}) + + def test_get_region_region_is_none(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml], + secure_files=[self.no_yaml]) + region = c._get_region(cloud='_test-cloud_no_region', region_name=None) + self.assertEqual(region, {'name': '', 'values': {}}) + + def test_get_region_region_set(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml], + secure_files=[self.no_yaml]) + region = c._get_region(cloud='_test-cloud_', region_name='test-region') + self.assertEqual(region, {'name': 'test-region', 'values': {}}) + + def test_get_region_many_regions_default(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml], + secure_files=[self.no_yaml]) + region = c._get_region(cloud='_test_cloud_regions', + region_name='') + self.assertEqual(region, {'name': 'region1', 'values': + {'external_network': 'region1-network'}}) + + def test_get_region_many_regions(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml], + secure_files=[self.no_yaml]) + region = c._get_region(cloud='_test_cloud_regions', + region_name='region2') + self.assertEqual(region, {'name': 'region2', 'values': + {'external_network': 'my-network'}}) + + def test_get_region_invalid_region(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml], + secure_files=[self.no_yaml]) + self.assertRaises( + exceptions.OpenStackConfigException, c._get_region, + cloud='_test_cloud_regions', region_name='invalid-region') + + def test_get_region_no_cloud(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml], + secure_files=[self.no_yaml]) + region = c._get_region(region_name='no-cloud-region') + self.assertEqual(region, {'name': 'no-cloud-region', 'values': {}}) + class TestConfigArgparse(base.TestCase): @@ -231,18 +311,28 @@ class TestConfigArgparse(base.TestCase): username='user', password='password', project_name='project', - region_name='other-test-region', + region_name='region2', snack_type='cookie', + os_auth_token='no-good-things', ) + self.options = argparse.Namespace(**self.args) + def test_get_one_cloud_bad_region_argparse(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + + self.assertRaises( + exceptions.OpenStackConfigException, c.get_one_cloud, + cloud='_test-cloud_', argparse=self.options) + def test_get_one_cloud_argparse(self): c = config.OpenStackConfig(config_files=[self.cloud_yaml], vendor_files=[self.vendor_yaml]) - cc = c.get_one_cloud(cloud='_test-cloud_', argparse=self.options) - self._assert_cloud_details(cc) - self.assertEqual(cc.region_name, 'other-test-region') + cc = c.get_one_cloud( + cloud='_test_cloud_regions', argparse=self.options) + self.assertEqual(cc.region_name, 'region2') self.assertEqual(cc.snack_type, 'cookie') def test_get_one_cloud_just_argparse(self): @@ -251,7 +341,7 @@ class TestConfigArgparse(base.TestCase): cc = c.get_one_cloud(argparse=self.options) self.assertIsNone(cc.cloud) - self.assertEqual(cc.region_name, 'other-test-region') + self.assertEqual(cc.region_name, 'region2') self.assertEqual(cc.snack_type, 'cookie') def test_get_one_cloud_just_kwargs(self): @@ -260,7 +350,7 @@ class TestConfigArgparse(base.TestCase): cc = c.get_one_cloud(**self.args) self.assertIsNone(cc.cloud) - self.assertEqual(cc.region_name, 'other-test-region') + self.assertEqual(cc.region_name, 'region2') self.assertEqual(cc.snack_type, 'cookie') def test_get_one_cloud_dash_kwargs(self): @@ -310,10 +400,10 @@ class TestConfigArgparse(base.TestCase): def test_get_one_cloud_bad_region_no_regions(self): c = config.OpenStackConfig(config_files=[self.cloud_yaml], vendor_files=[self.vendor_yaml]) - - cc = c.get_one_cloud(cloud='_test-cloud_', region_name='bad_region') - self._assert_cloud_details(cc) - self.assertEqual(cc.region_name, 'bad_region') + self.assertRaises( + exceptions.OpenStackConfigException, + c.get_one_cloud, + cloud='_test-cloud_', region_name='bad_region') def test_get_one_cloud_no_argparse_region2(self): c = config.OpenStackConfig(config_files=[self.cloud_yaml], @@ -325,6 +415,26 @@ class TestConfigArgparse(base.TestCase): self.assertEqual(cc.region_name, 'region2') self.assertIsNone(cc.snack_type) + def test_get_one_cloud_network(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + + cc = c.get_one_cloud( + cloud='_test_cloud_regions', region_name='region1', argparse=None) + self._assert_cloud_details(cc) + self.assertEqual(cc.region_name, 'region1') + self.assertEqual('region1-network', cc.config['external_network']) + + def test_get_one_cloud_per_region_network(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + + cc = c.get_one_cloud( + cloud='_test_cloud_regions', region_name='region2', argparse=None) + self._assert_cloud_details(cc) + self.assertEqual(cc.region_name, 'region2') + self.assertEqual('my-network', cc.config['external_network']) + def test_fix_env_args(self): c = config.OpenStackConfig(config_files=[self.cloud_yaml], vendor_files=[self.vendor_yaml]) @@ -334,6 +444,211 @@ class TestConfigArgparse(base.TestCase): self.assertDictEqual({'compute_api_version': 1}, fixed_args) + def test_extra_config(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + + defaults = {'use_hostnames': False, 'other-value': 'something'} + ansible_options = c.get_extra_config('ansible', defaults) + + # This should show that the default for use_hostnames above is + # overridden by the value in the config file defined in base.py + # It should also show that other-value key is normalized and passed + # through even though there is no corresponding value in the config + # file, and that expand-hostvars key is normalized and the value + # from the config comes through even though there is no default. + self.assertDictEqual( + { + 'expand_hostvars': False, + 'use_hostnames': True, + 'other_value': 'something', + }, + ansible_options) + + def test_register_argparse_cloud(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + parser = argparse.ArgumentParser() + c.register_argparse_arguments(parser, []) + opts, _remain = parser.parse_known_args(['--os-cloud', 'foo']) + self.assertEqual(opts.os_cloud, 'foo') + + def test_env_argparse_precedence(self): + self.useFixture(fixtures.EnvironmentVariable( + 'OS_TENANT_NAME', 'tenants-are-bad')) + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + + cc = c.get_one_cloud( + cloud='envvars', argparse=self.options) + self.assertEqual(cc.auth['project_name'], 'project') + + def test_argparse_default_no_token(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + + parser = argparse.ArgumentParser() + c.register_argparse_arguments(parser, []) + # novaclient will add this + parser.add_argument('--os-auth-token') + opts, _remain = parser.parse_known_args() + cc = c.get_one_cloud( + cloud='_test_cloud_regions', argparse=opts) + self.assertEqual(cc.config['auth_type'], 'password') + self.assertNotIn('token', cc.config['auth']) + + def test_argparse_token(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + + parser = argparse.ArgumentParser() + c.register_argparse_arguments(parser, []) + # novaclient will add this + parser.add_argument('--os-auth-token') + opts, _remain = parser.parse_known_args( + ['--os-auth-token', 'very-bad-things', + '--os-auth-type', 'token']) + cc = c.get_one_cloud(argparse=opts) + self.assertEqual(cc.config['auth_type'], 'token') + self.assertEqual(cc.config['auth']['token'], 'very-bad-things') + + def test_argparse_underscores(self): + c = config.OpenStackConfig(config_files=[self.no_yaml], + vendor_files=[self.no_yaml], + secure_files=[self.no_yaml]) + parser = argparse.ArgumentParser() + parser.add_argument('--os_username') + argv = [ + '--os_username', 'user', '--os_password', 'pass', + '--os-auth-url', 'auth-url', '--os-project-name', 'project'] + c.register_argparse_arguments(parser, argv=argv) + opts, _remain = parser.parse_known_args(argv) + cc = c.get_one_cloud(argparse=opts) + self.assertEqual(cc.config['auth']['username'], 'user') + self.assertEqual(cc.config['auth']['password'], 'pass') + self.assertEqual(cc.config['auth']['auth_url'], 'auth-url') + + def test_argparse_underscores_duplicate(self): + c = config.OpenStackConfig(config_files=[self.no_yaml], + vendor_files=[self.no_yaml], + secure_files=[self.no_yaml]) + parser = argparse.ArgumentParser() + parser.add_argument('--os_username') + argv = [ + '--os_username', 'user', '--os_password', 'pass', + '--os-username', 'user1', '--os-password', 'pass1', + '--os-auth-url', 'auth-url', '--os-project-name', 'project'] + self.assertRaises( + exceptions.OpenStackConfigException, + c.register_argparse_arguments, + parser=parser, argv=argv) + + def test_register_argparse_bad_plugin(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + parser = argparse.ArgumentParser() + self.assertRaises( + exceptions.OpenStackConfigException, + c.register_argparse_arguments, + parser, ['--os-auth-type', 'foo']) + + def test_register_argparse_not_password(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + parser = argparse.ArgumentParser() + args = [ + '--os-auth-type', 'v3token', + '--os-token', 'some-secret', + ] + c.register_argparse_arguments(parser, args) + opts, _remain = parser.parse_known_args(args) + self.assertEqual(opts.os_token, 'some-secret') + + def test_register_argparse_password(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + parser = argparse.ArgumentParser() + args = [ + '--os-password', 'some-secret', + ] + c.register_argparse_arguments(parser, args) + opts, _remain = parser.parse_known_args(args) + self.assertEqual(opts.os_password, 'some-secret') + with testtools.ExpectedException(AttributeError): + opts.os_token + + def test_register_argparse_service_type(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + parser = argparse.ArgumentParser() + args = [ + '--os-service-type', 'network', + '--os-endpoint-type', 'admin', + '--http-timeout', '20', + ] + c.register_argparse_arguments(parser, args) + opts, _remain = parser.parse_known_args(args) + self.assertEqual(opts.os_service_type, 'network') + self.assertEqual(opts.os_endpoint_type, 'admin') + self.assertEqual(opts.http_timeout, '20') + with testtools.ExpectedException(AttributeError): + opts.os_network_service_type + cloud = c.get_one_cloud(argparse=opts, verify=False) + self.assertEqual(cloud.config['service_type'], 'network') + self.assertEqual(cloud.config['interface'], 'admin') + self.assertEqual(cloud.config['api_timeout'], '20') + self.assertNotIn('http_timeout', cloud.config) + + def test_register_argparse_network_service_type(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + parser = argparse.ArgumentParser() + args = [ + '--os-endpoint-type', 'admin', + '--network-api-version', '4', + ] + c.register_argparse_arguments(parser, args, ['network']) + opts, _remain = parser.parse_known_args(args) + self.assertEqual(opts.os_service_type, 'network') + self.assertEqual(opts.os_endpoint_type, 'admin') + self.assertEqual(opts.os_network_service_type, None) + self.assertEqual(opts.os_network_api_version, None) + self.assertEqual(opts.network_api_version, '4') + cloud = c.get_one_cloud(argparse=opts, verify=False) + self.assertEqual(cloud.config['service_type'], 'network') + self.assertEqual(cloud.config['interface'], 'admin') + self.assertEqual(cloud.config['network_api_version'], '4') + self.assertNotIn('http_timeout', cloud.config) + + def test_register_argparse_network_service_types(self): + c = config.OpenStackConfig(config_files=[self.cloud_yaml], + vendor_files=[self.vendor_yaml]) + parser = argparse.ArgumentParser() + args = [ + '--os-compute-service-name', 'cloudServers', + '--os-network-service-type', 'badtype', + '--os-endpoint-type', 'admin', + '--network-api-version', '4', + ] + c.register_argparse_arguments( + parser, args, ['compute', 'network', 'volume']) + opts, _remain = parser.parse_known_args(args) + self.assertEqual(opts.os_network_service_type, 'badtype') + self.assertEqual(opts.os_compute_service_type, None) + self.assertEqual(opts.os_volume_service_type, None) + self.assertEqual(opts.os_service_type, 'compute') + self.assertEqual(opts.os_compute_service_name, 'cloudServers') + self.assertEqual(opts.os_endpoint_type, 'admin') + self.assertEqual(opts.os_network_api_version, None) + self.assertEqual(opts.network_api_version, '4') + cloud = c.get_one_cloud(argparse=opts, verify=False) + self.assertEqual(cloud.config['service_type'], 'compute') + self.assertEqual(cloud.config['network_service_type'], 'badtype') + self.assertEqual(cloud.config['interface'], 'admin') + self.assertEqual(cloud.config['network_api_version'], '4') + self.assertNotIn('volume_service_type', cloud.config) + self.assertNotIn('http_timeout', cloud.config) + class TestConfigDefault(base.TestCase): diff --git a/os_client_config/tests/test_environ.py b/os_client_config/tests/test_environ.py index 0ff800f..b75db1c 100644 --- a/os_client_config/tests/test_environ.py +++ b/os_client_config/tests/test_environ.py @@ -29,6 +29,8 @@ class TestEnviron(base.TestCase): fixtures.EnvironmentVariable('OS_AUTH_URL', 'https://example.com')) self.useFixture( fixtures.EnvironmentVariable('OS_USERNAME', 'testuser')) + self.useFixture( + fixtures.EnvironmentVariable('OS_PASSWORD', 'testpass')) self.useFixture( fixtures.EnvironmentVariable('OS_PROJECT_NAME', 'testproject')) self.useFixture( @@ -57,13 +59,15 @@ class TestEnviron(base.TestCase): self.useFixture( fixtures.EnvironmentVariable('OS_PREFER_IPV6', 'false')) c = config.OpenStackConfig(config_files=[self.cloud_yaml], - vendor_files=[self.vendor_yaml]) + vendor_files=[self.vendor_yaml], + secure_files=[self.secure_yaml]) cc = c.get_one_cloud('_test-cloud_') self.assertFalse(cc.prefer_ipv6) def test_environ_exists(self): c = config.OpenStackConfig(config_files=[self.cloud_yaml], - vendor_files=[self.vendor_yaml]) + vendor_files=[self.vendor_yaml], + secure_files=[self.secure_yaml]) cc = c.get_one_cloud('envvars') self._assert_cloud_details(cc) self.assertNotIn('auth_url', cc.config) @@ -78,7 +82,8 @@ class TestEnviron(base.TestCase): def test_environ_prefix(self): c = config.OpenStackConfig(config_files=[self.cloud_yaml], vendor_files=[self.vendor_yaml], - envvar_prefix='NOVA_') + envvar_prefix='NOVA_', + secure_files=[self.secure_yaml]) cc = c.get_one_cloud('envvars') self._assert_cloud_details(cc) self.assertNotIn('auth_url', cc.config) @@ -92,7 +97,8 @@ class TestEnviron(base.TestCase): def test_get_one_cloud_with_config_files(self): c = config.OpenStackConfig(config_files=[self.cloud_yaml], - vendor_files=[self.vendor_yaml]) + vendor_files=[self.vendor_yaml], + secure_files=[self.secure_yaml]) self.assertIsInstance(c.cloud_config, dict) self.assertIn('cache', c.cloud_config) self.assertIsInstance(c.cloud_config['cache'], dict) diff --git a/os_client_config/vendors/auro.json b/os_client_config/vendors/auro.json index 1e59f01..a9e709b 100644 --- a/os_client_config/vendors/auro.json +++ b/os_client_config/vendors/auro.json @@ -4,6 +4,7 @@ "auth": { "auth_url": "https://api.van1.auro.io:5000/v2.0" }, + "identity_api_version": "2", "region_name": "van1" } } diff --git a/os_client_config/vendors/bluebox.json b/os_client_config/vendors/bluebox.json index 2227aac..647c842 100644 --- a/os_client_config/vendors/bluebox.json +++ b/os_client_config/vendors/bluebox.json @@ -1,6 +1,7 @@ { "name": "bluebox", "profile": { + "volume_api_version": "1", "region_name": "RegionOne" } } diff --git a/os_client_config/vendors/catalyst.json b/os_client_config/vendors/catalyst.json index ddde838..3ad7507 100644 --- a/os_client_config/vendors/catalyst.json +++ b/os_client_config/vendors/catalyst.json @@ -9,6 +9,7 @@ "nz_wlg_2" ], "image_api_version": "1", + "volume_api_version": "1", "image_format": "raw" } } diff --git a/os_client_config/vendors/citycloud.json b/os_client_config/vendors/citycloud.json index f6c57c7..64cadce 100644 --- a/os_client_config/vendors/citycloud.json +++ b/os_client_config/vendors/citycloud.json @@ -9,6 +9,7 @@ "Sto2", "Kna1" ], + "volume_api_version": "1", "identity_api_version": "3" } } diff --git a/os_client_config/vendors/conoha.json b/os_client_config/vendors/conoha.json index 8e33ca4..5636f09 100644 --- a/os_client_config/vendors/conoha.json +++ b/os_client_config/vendors/conoha.json @@ -2,12 +2,13 @@ "name": "conoha", "profile": { "auth": { - "auth_url": "https://identity.{region_name}.conoha.io/v2.0" + "auth_url": "https://identity.{region_name}.conoha.io" }, "regions": [ "sin1", "sjc1", "tyo1" - ] + ], + "identity_api_version": "2" } } diff --git a/os_client_config/vendors/datacentred.json b/os_client_config/vendors/datacentred.json index 1fb4dbb..2be4a58 100644 --- a/os_client_config/vendors/datacentred.json +++ b/os_client_config/vendors/datacentred.json @@ -2,9 +2,10 @@ "name": "datacentred", "profile": { "auth": { - "auth_url": "https://compute.datacentred.io:5000/v2.0" + "auth_url": "https://compute.datacentred.io:5000" }, "region-name": "sal01", + "identity_api_version": "2", "image_api_version": "1" } } diff --git a/os_client_config/vendors/dreamhost.json b/os_client_config/vendors/dreamhost.json index 8580826..6fc2ccf 100644 --- a/os_client_config/vendors/dreamhost.json +++ b/os_client_config/vendors/dreamhost.json @@ -2,8 +2,9 @@ "name": "dreamhost", "profile": { "auth": { - "auth_url": "https://keystone.dream.io/v2.0" + "auth_url": "https://keystone.dream.io" }, + "identity_api_version": "3", "region_name": "RegionOne", "image_format": "raw" } diff --git a/os_client_config/vendors/elastx.json b/os_client_config/vendors/elastx.json index cac755e..1e72482 100644 --- a/os_client_config/vendors/elastx.json +++ b/os_client_config/vendors/elastx.json @@ -2,8 +2,9 @@ "name": "elastx", "profile": { "auth": { - "auth_url": "https://ops.elastx.net:5000/v2.0" + "auth_url": "https://ops.elastx.net:5000" }, + "identity_api_version": "3", "region_name": "regionOne" } } diff --git a/os_client_config/vendors/entercloudsuite.json b/os_client_config/vendors/entercloudsuite.json index 826c25f..6d2fc12 100644 --- a/os_client_config/vendors/entercloudsuite.json +++ b/os_client_config/vendors/entercloudsuite.json @@ -2,8 +2,10 @@ "name": "entercloudsuite", "profile": { "auth": { - "auth_url": "https://api.entercloudsuite.com/v2.0" + "auth_url": "https://api.entercloudsuite.com/" }, + "identity_api_version": "3", + "volume_api_version": "1", "regions": [ "it-mil1", "nl-ams1", diff --git a/os_client_config/vendors/hp.json b/os_client_config/vendors/hp.json index 10789a9..b06b90a 100644 --- a/os_client_config/vendors/hp.json +++ b/os_client_config/vendors/hp.json @@ -2,13 +2,15 @@ "name": "hp", "profile": { "auth": { - "auth_url": "https://region-b.geo-1.identity.hpcloudsvc.com:35357/v2.0" + "auth_url": "https://region-b.geo-1.identity.hpcloudsvc.com:35357" }, "regions": [ "region-a.geo-1", "region-b.geo-1" ], + "identity_api_version": "3", "dns_service_type": "hpext:dns", + "volume_api_version": "1", "image_api_version": "1" } } diff --git a/os_client_config/vendors/ibmcloud.json b/os_client_config/vendors/ibmcloud.json new file mode 100644 index 0000000..90962c6 --- /dev/null +++ b/os_client_config/vendors/ibmcloud.json @@ -0,0 +1,13 @@ +{ + "name": "ibmcloud", + "profile": { + "auth": { + "auth_url": "https://identity.open.softlayer.com" + }, + "volume_api_version": "2", + "identity_api_version": "3", + "regions": [ + "london" + ] + } +} diff --git a/os_client_config/vendors/internap.json b/os_client_config/vendors/internap.json index 9b27536..d5ad49f 100644 --- a/os_client_config/vendors/internap.json +++ b/os_client_config/vendors/internap.json @@ -2,13 +2,14 @@ "name": "internap", "profile": { "auth": { - "auth_url": "https://identity.api.cloud.iweb.com/v2.0" + "auth_url": "https://identity.api.cloud.iweb.com" }, "regions": [ "ams01", "da01", "nyj01" ], + "identity_api_version": "3", "image_api_version": "1", "floating_ip_source": "None" } diff --git a/os_client_config/vendors/ovh.json b/os_client_config/vendors/ovh.json index 032741f..664f161 100644 --- a/os_client_config/vendors/ovh.json +++ b/os_client_config/vendors/ovh.json @@ -2,13 +2,14 @@ "name": "ovh", "profile": { "auth": { - "auth_url": "https://auth.cloud.ovh.net/v2.0" + "auth_url": "https://auth.cloud.ovh.net/" }, "regions": [ "BHS1", "GRA1", "SBG1" ], + "identity_api_version": "3", "image_format": "raw", "floating_ip_source": "None" } diff --git a/os_client_config/vendors/rackspace.json b/os_client_config/vendors/rackspace.json index 582e122..3fbbacd 100644 --- a/os_client_config/vendors/rackspace.json +++ b/os_client_config/vendors/rackspace.json @@ -18,6 +18,7 @@ "image_format": "vhd", "floating_ip_source": "None", "secgroup_source": "None", + "volume_api_version": "1", "disable_vendor_agent": { "vm_mode": "hvm", "xenapi_use_agent": false diff --git a/os_client_config/vendors/runabove.json b/os_client_config/vendors/runabove.json index 56dd945..abf1116 100644 --- a/os_client_config/vendors/runabove.json +++ b/os_client_config/vendors/runabove.json @@ -2,12 +2,13 @@ "name": "runabove", "profile": { "auth": { - "auth_url": "https://auth.runabove.io/v2.0" + "auth_url": "https://auth.runabove.io/" }, "regions": [ "BHS-1", "SBG-1" ], + "identity_api_version": "3", "image_format": "qcow2", "floating_ip_source": "None" } diff --git a/os_client_config/vendors/switchengines.json b/os_client_config/vendors/switchengines.json index 8a7c566..46f6325 100644 --- a/os_client_config/vendors/switchengines.json +++ b/os_client_config/vendors/switchengines.json @@ -8,6 +8,7 @@ "LS", "ZH" ], + "volume_api_version": "1", "image_api_use_tasks": true, "image_format": "raw" } diff --git a/os_client_config/vendors/ultimum.json b/os_client_config/vendors/ultimum.json index ada6e3d..4bfd088 100644 --- a/os_client_config/vendors/ultimum.json +++ b/os_client_config/vendors/ultimum.json @@ -2,8 +2,10 @@ "name": "ultimum", "profile": { "auth": { - "auth_url": "https://console.ultimum-cloud.com:5000/v2.0" + "auth_url": "https://console.ultimum-cloud.com:5000/" }, + "identity_api_version": "3", + "volume_api_version": "1", "region-name": "RegionOne" } } diff --git a/os_client_config/vendors/unitedstack.json b/os_client_config/vendors/unitedstack.json index 41f4585..ac8be11 100644 --- a/os_client_config/vendors/unitedstack.json +++ b/os_client_config/vendors/unitedstack.json @@ -8,6 +8,7 @@ "bj1", "gd1" ], + "volume_api_version": "1", "identity_api_version": "3", "image_format": "raw", "floating_ip_source": "None" diff --git a/os_client_config/vendors/vexxhost.json b/os_client_config/vendors/vexxhost.json index 25911ca..dd683be 100644 --- a/os_client_config/vendors/vexxhost.json +++ b/os_client_config/vendors/vexxhost.json @@ -2,9 +2,13 @@ "name": "vexxhost", "profile": { "auth": { - "auth_url": "http://auth.api.thenebulacloud.com:5000/v2.0/" + "auth_url": "http://auth.vexxhost.net" }, - "region_name": "ca-ymq-1", + "regions": [ + "ca-ymq-1" + ], + "dns_api_version": "1", + "identity_api_version": "3", "floating_ip_source": "None" } } diff --git a/releasenotes/notes/started-using-reno-242e2b0cd27f9480.yaml b/releasenotes/notes/started-using-reno-242e2b0cd27f9480.yaml new file mode 100644 index 0000000..d7cfb51 --- /dev/null +++ b/releasenotes/notes/started-using-reno-242e2b0cd27f9480.yaml @@ -0,0 +1,3 @@ +--- +other: +- Started using reno for release notes. diff --git a/requirements.txt b/requirements.txt index 3c32ced..1531be8 100644 --- a/requirements.txt +++ b/requirements.txt @@ -3,5 +3,5 @@ # process, which may cause wedges in the gate later. PyYAML>=3.1.0 appdirs>=1.3.0 -keystoneauth1>=1.0.0 +keystoneauth1>=2.1.0 requestsexceptions>=1.1.1 # Apache-2.0 diff --git a/setup.cfg b/setup.cfg index bc4f128..89df35c 100644 --- a/setup.cfg +++ b/setup.cfg @@ -15,7 +15,6 @@ classifier = Programming Language :: Python Programming Language :: Python :: 2 Programming Language :: Python :: 2.7 - Programming Language :: Python :: 2.6 Programming Language :: Python :: 3 Programming Language :: Python :: 3.3 Programming Language :: Python :: 3.4 diff --git a/test-requirements.txt b/test-requirements.txt index 7053051..a50a202 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -16,6 +16,7 @@ python-subunit>=0.0.18 sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3 oslosphinx>=2.5.0,<2.6.0 # Apache-2.0 oslotest>=1.5.1,<1.6.0 # Apache-2.0 +reno>=0.1.1 # Apache2 testrepository>=0.0.18 testscenarios>=0.4 testtools>=0.9.36,!=1.2.0 diff --git a/tox.ini b/tox.ini index 7a2d3a0..95dff6b 100644 --- a/tox.ini +++ b/tox.ini @@ -21,7 +21,12 @@ commands = {posargs} commands = python setup.py test --coverage --coverage-package-name=os_client_config --testr-args='{posargs}' [testenv:docs] -commands = python setup.py build_sphinx +deps = + {[testenv]deps} + readme +commands = + python setup.py build_sphinx + python setup.py check -r -s [flake8] # H803 skipped on purpose per list discussion.