42559c7f19
The patch brings a couple of new concepts into cloudinit.
We have a general miniframework for defining plugins and discovering them,
with an actual implementation which uses the *pkgutil* builtin module.
Built atop of this framework, we have a new data source loader, found
in cloudinit.bases.DataSourceLoader. The loader operates on three concepts:
- the *module iterator*, which is used to list modules from a specific
location (in our case, from cloudinit.sources.__path__). The data
source loader takes care to use only the modules that exports a
given API.
- the data source discovery API assumes that each data source module exports
a function called `data_sources`, which should return a tuple of data source
classes that the said module exports. The loader filters the modules
that provides this API.
- the data source loader uses a new concept called *search strategy*
for discovering a potential data source. The search strategies
are classes whose purpose is to select one or more data sources
from a data source stream (any iterable).
There are multiple ways to implement a strategy, the search
can be either serial or parallel, there's no additional requirement
as long as they return an iterable.
Also, the strategies can be stacked together, for instance, having
two strategies, one for selecting only the network data sources
and another for selecting the available data sources from a list
of potential data sources.
This patch also adds a new API that uses the DataSourceLoader
with a given module iterator and strategies for selecting one
data source that cloudinit can use.
Change-Id: I30f312191ce40e45445ed9e3fc8a3d4651903280
55 lines
1.5 KiB
Python
55 lines
1.5 KiB
Python
# Copyright 2015 Canonical Ltd.
|
|
# This file is part of cloud-init. See LICENCE file for license information.
|
|
#
|
|
# vi: ts=4 expandtab
|
|
|
|
"""Various base classes and implementations for finding *plugins*."""
|
|
|
|
import abc
|
|
import pkgutil
|
|
|
|
import six
|
|
|
|
from cloudinit import logging
|
|
|
|
|
|
LOG = logging.getLogger(__name__)
|
|
|
|
|
|
@six.add_metaclass(abc.ABCMeta)
|
|
class BaseModuleIterator(object):
|
|
"""Base class for describing a *module iterator*
|
|
|
|
A module iterator is a class that's capable of listing
|
|
modules or packages from a specific location, which are
|
|
already loaded.
|
|
"""
|
|
|
|
def __init__(self, search_paths):
|
|
self._search_paths = search_paths
|
|
|
|
@abc.abstractmethod
|
|
def list_modules(self):
|
|
"""List all the modules that this finder knows about."""
|
|
|
|
|
|
class PkgutilModuleIterator(BaseModuleIterator):
|
|
"""A class based on the *pkgutil* module for discovering modules."""
|
|
|
|
@staticmethod
|
|
def _find_module(finder, module):
|
|
"""Delegate to the *finder* for finding the given module."""
|
|
return finder.find_module(module).load_module(module)
|
|
|
|
def list_modules(self):
|
|
"""List all modules that this class knows about."""
|
|
for finder, name, _ in pkgutil.walk_packages(self._search_paths):
|
|
try:
|
|
module = self._find_module(finder, name)
|
|
except ImportError:
|
|
LOG.debug('Could not import the module %r using the '
|
|
'search path %r', name, finder.path)
|
|
continue
|
|
|
|
yield module
|