Add metadata servvice VMwareGuestInfoService
VMwareGuestInfoService is a metadata service which uses VMware's rpctool to extract guest metadata and userdata configured for machines running on VMware hypervisors. The implementation is similar to: https://github.com/vmware/cloud-init-vmware-guestinfo Supported features for the metadata service: * instance id * hostname * admin username * admin password * public SSH keys * userdata Configuration options: ```ini [vmwarequestinfo] vmware_rpctool_path=%ProgramFiles%/VMware/VMware Tools/rpctool.exe ``` The VMware RPC tool used to query the instance metadata and userdata needs to be present at the config option path. Both json and yaml are supported as metadata formats. The metadata / userdata can be encoded in base64, gzip or gzip+base64. Example metadata in yaml format: ```yaml instance-id: cloud-vm local-hostname: cloud-vm admin-username: cloud-username admin-password: Passw0rd public-keys-data: | ssh-key 1 ssh-key 2 ``` This metadata content needs to be sent as string in the guestinfo dictionary, thus needs to be converted to base64 (it is recommended to gzip it too). To convert to gzip+base64 format: ```bash cat metadata.yml | gzip.exe -9 | base64.exe -w0 ``` Co-Authored-By: Rui Lopes <rgl@ruilopes.com> Change-Id: I6a8430e87ee03d2e8fdd2685b05e60c5c0ffb5be
This commit is contained in:
Adrian Vladu
parent
fd2c15bef3
commit
a77477e16e
@ -24,6 +24,7 @@ _OPT_PATHS = (
|
||||
'cloudbaseinit.conf.azure.AzureOptions',
|
||||
'cloudbaseinit.conf.ovf.OvfOptions',
|
||||
'cloudbaseinit.conf.packet.PacketOptions',
|
||||
'cloudbaseinit.conf.vmwareguestinfo.VMwareGuestInfoConfigOptions',
|
||||
)
|
||||
|
||||
|
||||
|
||||
46
cloudbaseinit/conf/vmwareguestinfo.py
Normal file
46
cloudbaseinit/conf/vmwareguestinfo.py
Normal file
@ -0,0 +1,46 @@
|
||||
# Copyright 2020 Cloudbase Solutions Srl
|
||||
# Copyright 2019 ruilopes.com
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
"""Config options available for the VMware metadata service."""
|
||||
|
||||
from oslo_config import cfg
|
||||
|
||||
from cloudbaseinit.conf import base as conf_base
|
||||
|
||||
|
||||
class VMwareGuestInfoConfigOptions(conf_base.Options):
|
||||
|
||||
"""Config options available for the VMware GuestInfo metadata service."""
|
||||
|
||||
def __init__(self, config):
|
||||
super(VMwareGuestInfoConfigOptions, self).__init__(
|
||||
config, group="vmwareguestinfo")
|
||||
self._options = [
|
||||
cfg.StrOpt(
|
||||
'vmware_rpctool_path',
|
||||
default="%ProgramFiles%/VMware/VMware Tools/rpctool.exe",
|
||||
help='The local path where VMware rpctool is found')
|
||||
]
|
||||
|
||||
def register(self):
|
||||
"""Register the current options to the global ConfigOpts object."""
|
||||
group = cfg.OptGroup(self.group_name,
|
||||
title='VMware GuestInfo Options')
|
||||
self._config.register_group(group)
|
||||
self._config.register_opts(self._options, group=group)
|
||||
|
||||
def list(self):
|
||||
"""Return a list which contains all the available options."""
|
||||
return self._options
|
||||
171
cloudbaseinit/metadata/services/vmwareguestinfoservice.py
Normal file
171
cloudbaseinit/metadata/services/vmwareguestinfoservice.py
Normal file
@ -0,0 +1,171 @@
|
||||
# Copyright 2020 Cloudbase Solutions Srl
|
||||
# Copyright 2019 ruilopes.com
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import base64
|
||||
import gzip
|
||||
import io
|
||||
import json
|
||||
import os
|
||||
import yaml
|
||||
|
||||
from oslo_log import log as oslo_logging
|
||||
|
||||
from cloudbaseinit import conf as cloudbaseinit_conf
|
||||
from cloudbaseinit import exception
|
||||
from cloudbaseinit.metadata.services import base
|
||||
from cloudbaseinit.osutils import factory as osutils_factory
|
||||
|
||||
CONF = cloudbaseinit_conf.CONF
|
||||
LOG = oslo_logging.getLogger(__name__)
|
||||
|
||||
|
||||
class YamlParserConfigError(Exception):
|
||||
"""Exception for Yaml parsing failures"""
|
||||
pass
|
||||
|
||||
|
||||
class VMwareGuestInfoService(base.BaseMetadataService):
|
||||
def __init__(self):
|
||||
super(VMwareGuestInfoService, self).__init__()
|
||||
self._rpc_tool_path = None
|
||||
self._osutils = osutils_factory.get_os_utils()
|
||||
self._meta_data = {}
|
||||
self._user_data = None
|
||||
|
||||
@staticmethod
|
||||
def _parse_data(raw_data):
|
||||
"""Parse data as json. Fallback to yaml if json parsing fails"""
|
||||
|
||||
try:
|
||||
return json.loads(raw_data)
|
||||
except (TypeError, ValueError, AttributeError):
|
||||
loader = getattr(yaml, 'CLoader', yaml.Loader)
|
||||
try:
|
||||
return yaml.load(raw_data, Loader=loader)
|
||||
except (TypeError, ValueError, AttributeError):
|
||||
raise YamlParserConfigError("Invalid yaml data provided.")
|
||||
|
||||
@staticmethod
|
||||
def _decode_data(raw_data, is_base64, is_gzip):
|
||||
"""Decode raw_data from base64 / ungzip"""
|
||||
if not raw_data:
|
||||
return
|
||||
|
||||
if is_base64:
|
||||
raw_data = base64.b64decode(raw_data)
|
||||
|
||||
if is_gzip:
|
||||
with gzip.GzipFile(fileobj=io.BytesIO(raw_data), mode='rb') as dt:
|
||||
raw_data = dt.read()
|
||||
|
||||
return raw_data
|
||||
|
||||
def _get_guestinfo_value(self, key):
|
||||
rpc_command = 'info-get guestinfo.%s' % key
|
||||
data, stderr, exit_code = self._osutils.execute_process([
|
||||
self._rpc_tool_path,
|
||||
rpc_command
|
||||
])
|
||||
if exit_code:
|
||||
LOG.debug(
|
||||
'Failed to execute "%(rpctool_path)s \'%(rpc_command)s\'" '
|
||||
'with exit code: %(exit_code)s\nstdout: '
|
||||
'%(stdout)s\nstderr: %(stderr)s' % {
|
||||
'rpctool_path': self._rpc_tool_path,
|
||||
'rpc_command': rpc_command, 'exit_code': exit_code,
|
||||
'stdout': data, 'stderr': stderr})
|
||||
return
|
||||
|
||||
return data
|
||||
|
||||
def _get_guest_data(self, key):
|
||||
is_base64 = False
|
||||
is_gzip = False
|
||||
encoding_plain_text = 'plaintext'
|
||||
raw_data = self._get_guestinfo_value(key)
|
||||
raw_encoding = self._get_guestinfo_value("%s.encoding" % key)
|
||||
|
||||
if not raw_encoding or not raw_encoding.strip():
|
||||
raw_encoding = encoding_plain_text
|
||||
|
||||
encoding = raw_encoding.strip()
|
||||
if isinstance(encoding, bytes):
|
||||
encoding = encoding.decode("utf-8")
|
||||
|
||||
if encoding in ('gzip+base64', 'gz+b64'):
|
||||
is_gzip = True
|
||||
is_base64 = True
|
||||
elif encoding in ('base64', 'b64'):
|
||||
is_base64 = True
|
||||
elif encoding != encoding_plain_text:
|
||||
raise exception.CloudbaseInitException(
|
||||
"Encoding %s not supported" % encoding)
|
||||
|
||||
LOG.debug("Decoding key %s: encoding %s", key, encoding)
|
||||
return self._decode_data(raw_data, is_base64, is_gzip)
|
||||
|
||||
def load(self):
|
||||
super(VMwareGuestInfoService, self).load()
|
||||
|
||||
if not CONF.vmwareguestinfo.vmware_rpctool_path:
|
||||
LOG.info("rpctool_path is empty. "
|
||||
"Please provide a value for VMware rpctool path.")
|
||||
return False
|
||||
|
||||
self._rpc_tool_path = os.path.abspath(
|
||||
os.path.expandvars(CONF.vmwareguestinfo.vmware_rpctool_path))
|
||||
|
||||
if not os.path.exists(self._rpc_tool_path):
|
||||
LOG.info("%s does not exist. "
|
||||
"Please provide a valid value for VMware rpctool path."
|
||||
% self._rpc_tool_path)
|
||||
return False
|
||||
|
||||
self._meta_data = self._parse_data(self._get_guest_data('metadata'))
|
||||
if not isinstance(self._meta_data, dict):
|
||||
LOG.warning("Instance metadata is not a dictionary.")
|
||||
self._meta_data = {}
|
||||
|
||||
self._user_data = self._get_guest_data('userdata')
|
||||
|
||||
if self._meta_data or self._user_data:
|
||||
return True
|
||||
|
||||
def _get_data(self, path):
|
||||
pass
|
||||
|
||||
def get_instance_id(self):
|
||||
return self._meta_data.get('instance-id')
|
||||
|
||||
def get_user_data(self):
|
||||
return self._user_data
|
||||
|
||||
def get_host_name(self):
|
||||
return self._meta_data.get('local-hostname')
|
||||
|
||||
def get_public_keys(self):
|
||||
public_keys = []
|
||||
public_keys_data = self._meta_data.get('public-keys-data')
|
||||
|
||||
if public_keys_data:
|
||||
public_keys = public_keys_data.splitlines()
|
||||
|
||||
return list(set((key.strip() for key in public_keys)))
|
||||
|
||||
def get_admin_username(self):
|
||||
return self._meta_data.get('admin-username')
|
||||
|
||||
def get_admin_password(self):
|
||||
return self._meta_data.get('admin-password')
|
||||
@ -0,0 +1,214 @@
|
||||
# Copyright 2020 Cloudbase Solutions Srl
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
import importlib
|
||||
import unittest
|
||||
|
||||
import ddt
|
||||
import yaml
|
||||
|
||||
try:
|
||||
import unittest.mock as mock
|
||||
except ImportError:
|
||||
import mock
|
||||
|
||||
from cloudbaseinit import conf as cloudbaseinit_conf
|
||||
from cloudbaseinit import exception
|
||||
from cloudbaseinit.tests import testutils
|
||||
|
||||
|
||||
CONF = cloudbaseinit_conf.CONF
|
||||
BASE_MODULE_PATH = 'cloudbaseinit.metadata.services.vmwareguestinfoservice'
|
||||
MODULE_PATH = BASE_MODULE_PATH + '.VMwareGuestInfoService'
|
||||
|
||||
|
||||
class FakeException(Exception):
|
||||
pass
|
||||
|
||||
|
||||
@ddt.ddt
|
||||
class VMwareGuestInfoServiceTest(unittest.TestCase):
|
||||
|
||||
@mock.patch('cloudbaseinit.osutils.factory.get_os_utils')
|
||||
def setUp(self, mock_os_utils):
|
||||
self._module = importlib.import_module(BASE_MODULE_PATH)
|
||||
self._service = (self._module.VMwareGuestInfoService())
|
||||
self.snatcher = testutils.LogSnatcher(BASE_MODULE_PATH)
|
||||
|
||||
@ddt.data((b'', (None, False)),
|
||||
(b'{}', ({}, False)),
|
||||
(b'---', (None, True)),
|
||||
(b'test: test', ({"test": "test"}, True)))
|
||||
@ddt.unpack
|
||||
@mock.patch("json.loads")
|
||||
@mock.patch("yaml.load")
|
||||
def test_parse_data(self, stream, expected_parsed_output,
|
||||
mock_yaml_load, mock_json_loads):
|
||||
if not expected_parsed_output[1]:
|
||||
mock_json_loads.return_value = expected_parsed_output[0]
|
||||
else:
|
||||
mock_json_loads.side_effect = TypeError("Failed to parse json")
|
||||
mock_yaml_load.return_value = expected_parsed_output[0]
|
||||
|
||||
parsed_output = self._service._parse_data(stream)
|
||||
|
||||
mock_json_loads.assert_called_once_with(stream)
|
||||
if expected_parsed_output[1]:
|
||||
loader = getattr(yaml, 'CLoader', yaml.Loader)
|
||||
mock_yaml_load.assert_called_once_with(stream, Loader=loader)
|
||||
|
||||
self.assertEqual(parsed_output, expected_parsed_output[0])
|
||||
|
||||
@ddt.data(((None, False, False), None),
|
||||
(('', False, False), None),
|
||||
(('dGVzdCANCg==', True, False), b'test \r\n'),
|
||||
(('H4sIAAq5MV4CAytR4OUCAGQ5L6gEAAAA', True, True), b't \r\n'),
|
||||
)
|
||||
@ddt.unpack
|
||||
def test_decode_data(self, raw_data, expected_decoded_data):
|
||||
|
||||
decoded_data = self._service._decode_data(raw_data[0], raw_data[1],
|
||||
raw_data[2])
|
||||
|
||||
self.assertEqual(decoded_data, expected_decoded_data)
|
||||
|
||||
@mock.patch('os.path.abspath')
|
||||
@mock.patch('os.path.exists')
|
||||
def _test_load_no_rpc_tool(self, expected_output, rpc_tool_path,
|
||||
mock_path_exists, mock_abs_path):
|
||||
CONF.set_override('vmware_rpctool_path', rpc_tool_path,
|
||||
'vmwareguestinfo')
|
||||
mock_abs_path.return_value = rpc_tool_path
|
||||
mock_path_exists.return_value = False
|
||||
with testutils.LogSnatcher('cloudbaseinit.metadata.services.'
|
||||
'vmwareguestinfoservice') as snatcher:
|
||||
result = self._service.load()
|
||||
self.assertEqual(result, False)
|
||||
self.assertEqual([expected_output], snatcher.output)
|
||||
|
||||
def test_load_rpc_tool_undefined(self):
|
||||
expected_output = ('rpctool_path is empty. '
|
||||
'Please provide a value for VMware rpctool path.')
|
||||
|
||||
self._test_load_no_rpc_tool(expected_output, None)
|
||||
|
||||
def test_load_rpc_tool_not_existent(self):
|
||||
expected_output = ('fake_path does not exist. '
|
||||
'Please provide a valid value '
|
||||
'for VMware rpctool path.')
|
||||
|
||||
self._test_load_no_rpc_tool(expected_output, 'fake_path')
|
||||
|
||||
@mock.patch('os.path.exists')
|
||||
@mock.patch(MODULE_PATH + "._parse_data")
|
||||
@mock.patch(MODULE_PATH + "._get_guest_data")
|
||||
def _test_load_meta_data(self, mock_get_guestinfo, mock_parse,
|
||||
mock_os_path_exists, parse_return=None,
|
||||
get_guest_data_result=None, exception=False,
|
||||
expected_result=None, meta_data_return=False):
|
||||
|
||||
mock_os_path_exists.return_value = True
|
||||
mock_parse.return_value = parse_return
|
||||
|
||||
if not exception:
|
||||
mock_get_guestinfo.return_value = get_guest_data_result
|
||||
result = self._service.load()
|
||||
self.assertEqual(result, expected_result)
|
||||
mock_get_guestinfo.assert_called_with('userdata')
|
||||
mock_parse.assert_called_once_with(get_guest_data_result)
|
||||
self.assertEqual(mock_get_guestinfo.call_count, 2)
|
||||
self.assertEqual(self._service._meta_data, meta_data_return)
|
||||
self.assertEqual(self._service._user_data, get_guest_data_result)
|
||||
else:
|
||||
mock_get_guestinfo.side_effect = FakeException("Fake")
|
||||
self.assertRaises(FakeException, self._service.load)
|
||||
|
||||
def test_load_no_meta_data(self):
|
||||
self._test_load_meta_data(meta_data_return={})
|
||||
|
||||
def test_load_no_user_data(self):
|
||||
parse_return = {"fake": "metadata"}
|
||||
self._test_load_meta_data(parse_return=parse_return,
|
||||
expected_result=True,
|
||||
meta_data_return=parse_return)
|
||||
|
||||
def test_load_fail(self):
|
||||
self._test_load_meta_data(parse_return={"fake": "metadata"},
|
||||
exception=True)
|
||||
|
||||
def test_load(self):
|
||||
parse_return = {"fake": "metadata"}
|
||||
self._test_load_meta_data(parse_return=parse_return,
|
||||
get_guest_data_result="fake userdata",
|
||||
expected_result=True,
|
||||
meta_data_return=parse_return)
|
||||
|
||||
def test_load_no_dict_metadata(self):
|
||||
self._test_load_meta_data(parse_return="not_a_dict",
|
||||
expected_result=None, meta_data_return={})
|
||||
|
||||
@ddt.data((None, []),
|
||||
('', []),
|
||||
(b'', []),
|
||||
(b'ssh1', [b"ssh1"]),
|
||||
('ssh1', ["ssh1"]),
|
||||
('ssh1 ssh2', ["ssh1 ssh2"]),
|
||||
('ssh1 test\nssh2\n', ["ssh1 test", "ssh2"]))
|
||||
@ddt.unpack
|
||||
def test_get_public_keys(self, keys_data, expected_keys):
|
||||
self._service._meta_data = {
|
||||
"public-keys-data": keys_data
|
||||
}
|
||||
public_keys = self._service.get_public_keys()
|
||||
public_keys.sort()
|
||||
expected_keys.sort()
|
||||
self.assertEqual(public_keys, expected_keys)
|
||||
|
||||
@ddt.data((('metadata', ''), (False, False)),
|
||||
(('metadata', 'b64'), (True, False)),
|
||||
(('metadata', 'base64'), (True, False)),
|
||||
(('metadata', 'gzip+base64'), (True, True)),
|
||||
(('metadata', 'gz+b64'), (True, True)))
|
||||
@ddt.unpack
|
||||
@mock.patch(MODULE_PATH + "._decode_data")
|
||||
@mock.patch(MODULE_PATH + "._get_guestinfo_value")
|
||||
def test_get_guest_data(self, test_data, expected_encoding,
|
||||
mock_get_guestinfo_value,
|
||||
mock_decode_data):
|
||||
|
||||
(data_key, encoding_ret) = test_data
|
||||
(is_base64, is_gzip) = expected_encoding
|
||||
data_key_ret = 'fake_data'
|
||||
decoded_data = 'fake_decoded_data'
|
||||
|
||||
def guest_info_side_effect(*args, **kwargs):
|
||||
if args[0] == data_key:
|
||||
return data_key_ret
|
||||
return encoding_ret
|
||||
|
||||
mock_get_guestinfo_value.side_effect = guest_info_side_effect
|
||||
mock_decode_data.return_value = decoded_data
|
||||
|
||||
data = self._service._get_guest_data(data_key)
|
||||
|
||||
self.assertEqual(data, decoded_data)
|
||||
mock_decode_data.assert_called_once_with(data_key_ret,
|
||||
is_base64, is_gzip)
|
||||
|
||||
@mock.patch(MODULE_PATH + "._get_guestinfo_value")
|
||||
def test_get_guest_data_fail(self, mock_get_guestinfo_value):
|
||||
|
||||
mock_get_guestinfo_value.return_value = "no encoding"
|
||||
self.assertRaises(exception.CloudbaseInitException,
|
||||
self._service._get_guest_data, 'fake_key')
|
||||
@ -369,3 +369,67 @@ these plugins can lock or misconfigure the user, leading to unwanted problems.
|
||||
Plugins that set NTP, MTU, extend volumes are idempotent and can be re-executed
|
||||
with no issues. Make sure that if you configure cloudbase-init to run local scripts,
|
||||
those local scripts are idempotent.
|
||||
|
||||
|
||||
VMware GuestInfo Service
|
||||
------------------------
|
||||
|
||||
.. class:: cloudbaseinit.metadata.services.vmwareguestinfoservice.VMwareGuestInfoService
|
||||
|
||||
VMwareGuestInfoService is a metadata service which uses VMware's rpctool to extract guest
|
||||
metadata and userdata configured for machines running on VMware hypervisors.
|
||||
|
||||
The VMware RPC tool used to query the instance metadata and userdata needs to be present at
|
||||
the config option path.
|
||||
|
||||
Both json and yaml are supported as metadata formats.
|
||||
The metadata / userdata can be encoded in base64, gzip or gzip+base64.
|
||||
|
||||
Example metadata in yaml format:
|
||||
|
||||
.. code-block:: yaml
|
||||
|
||||
instance-id: cloud-vm
|
||||
local-hostname: cloud-vm
|
||||
admin-username: cloud-username
|
||||
admin-password: Passw0rd
|
||||
public-keys-data: |
|
||||
ssh-key 1
|
||||
ssh-key 2
|
||||
|
||||
This metadata content needs to be set as string in the guestinfo
|
||||
dictionary, thus needs to be converted to base64 (it is recommended to
|
||||
gzip it too).
|
||||
To convert to gzip+base64 format:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cat metadata.yml | gzip.exe -9 | base64.exe -w0
|
||||
|
||||
The output of the gzip+base64 conversion needs to be set in the instance guestinfo, along with
|
||||
the encoding of the metadata / userdata.
|
||||
|
||||
For more information on how to achieve this, please check https://github.com/vmware/cloud-init-vmware-guestinfo#configuration
|
||||
|
||||
This is an example how to set the information from the instance:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
<rpctool_path> "info-set guestinfo.metadata <gzip+base64-encoded-metadata>"
|
||||
<rpctool_path> "info-set guestinfo.metadata.encoding gzip+base64"
|
||||
<rpctool_path> "info-set guestinfo.userdata <gzip+base64-encoded-userdata>"
|
||||
<rpctool_path> "info-set guestinfo.userdata.encoding gzip+base64"
|
||||
|
||||
|
||||
Capabilities:
|
||||
|
||||
* instance id
|
||||
* hostname
|
||||
* public keys
|
||||
* admin user name
|
||||
* admin user password
|
||||
* user data
|
||||
|
||||
Config options for `vmwareguestinfo` section:
|
||||
|
||||
* vmware_rpctool_path (string: "%ProgramFiles%/VMware/VMware Tools/rpctool.exe")
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user