Alexander Dobdin
GitHub
commit
bcc324c426
@ -25,16 +25,21 @@ Configuring actions
|
||||
Actions can be configured in a separate yaml file (by default ``rq.yaml`` is used) and / or defind in the main config file or passed via command line options ``-P``, ``-C``, ``-S``, ``-G``.
|
||||
|
||||
The following actions are available for definition:
|
||||
|
||||
* **put** - a list of tuples / 2-element lists: [source, destination]. Passed to ``scp`` like so ``scp source <node-ip>:destination``. Wildcards supported for source.
|
||||
* **cmds** - a list of dicts: {'command-name':'command-string'}. Example: {'command-1': 'uptime'}. Command string is a bash string. Commands are executed in a sorted order of their names.
|
||||
* **scripts** - a list of script filenames located on a local system. If filename does not contain path separator, the script is expected ot be located inside ``rqdir/scripts``. Otherwise the provided path is used to read the script.
|
||||
* **scripts** - a list of elements, each of which can be a string or a dict:
|
||||
* string - represents a script filename located on a local system. If filename does not contain a path separator, the script is expected ot be located inside ``rqdir/scripts``. Otherwise the provided path is used to access the script. Example: ``'./my-test-script.sh'``
|
||||
* dict - use this option if you need to pass variables to your script. Script parameters are not supported, but instead you can use env variables. A dict should only contain one key which is the script filename (read above), and the value is a Bash space-separated variable assignment string. Example: ``'./my-test-script.sh': 'var1=123 var2="HELLO WORLD"'``
|
||||
* **LIMITATION**: if you use a script with the same name more than once for a given node, the collected output will only contain the last execution's result.
|
||||
* **INFO**: Scripts are not copied to the destination system - script code is passed as stdin to `bash -s` executed via ssh or locally. Therefore passing parameters to scripts is not supported (unlike cmds where you can write any Bash string). You can use variables in your scripts instead. Scripts are executed in the following order: all scripts without variables, sorted by their full filename, then all scripts with variables, also sorted by full filename. Therefore if the order matters, it's better to put all scripts into the same folder and name them according to the order in which you want them executed on the same node, and mind that scripts with variables are executed after all scripts without variables. If you need to mix scripts with variables and without and maintain order, just use dict structure for all scripts, and set `null` as the value for those which do not need variables.
|
||||
* **files** - a list of filenames to collect. passed to ``scp``. Supports wildcards.
|
||||
* **filelists** - a list of filelist filenames located on a local system. Filelist is a text file containing files and directories to collect, passed to rsync. Does not support wildcards. If filename does not contain path separator, the filelist is expected to be located inside ``rqdir/filelists``. Otherwise the provided path is used to read the filelist.
|
||||
* **log_files**
|
||||
** **path** - base path to scan for logs
|
||||
** **include** - regexp string to match log files against for inclusion (if not set = include all)
|
||||
** **exclude** - regexp string to match log files against. Excludes matched files from collection.
|
||||
** **start** - date or datetime string to collect only files modified on or after the specified time. Format - ``YYYY-MM-DD`` or ``YYYY-MM-DD HH:MM:SS``
|
||||
* **path** - base path to scan for logs
|
||||
* **include** - regexp string to match log files against for inclusion (if not set = include all)
|
||||
* **exclude** - regexp string to match log files against. Excludes matched files from collection.
|
||||
* **start** - date or datetime string to collect only files modified on or after the specified time. Format - ``YYYY-MM-DD`` or ``YYYY-MM-DD HH:MM:SS``
|
||||
|
||||
===============
|
||||
Filtering nodes
|
||||
@ -77,6 +82,16 @@ It is possible to define special **by_<parameter-name>** dicts in config to (re)
|
||||
|
||||
In this example for any controller node, cmds setting will be reset to the value above. For nodes without controller role, default (none) values will be used.
|
||||
|
||||
It is also possible to define a special **once_by_<parameter-name>** which works similarly, but will only result in attributes being assigned to single (first in the list) matching node. Example:
|
||||
|
||||
::
|
||||
|
||||
once_by_roles:
|
||||
controller:
|
||||
cmds: {'check-uptime': 'uptime'}
|
||||
|
||||
Such configuration will result in `uptime` being executed on only one node with controller role, not on every controller.
|
||||
|
||||
=============
|
||||
rqfile format
|
||||
=============
|
||||
@ -108,13 +123,14 @@ Configuration application order
|
||||
===============================
|
||||
|
||||
Configuration is assembled and applied in a specific order:
|
||||
|
||||
1. default configuration is initialized. See ``timmy/conf.py`` for details.
|
||||
2. command line parameters, if defined, are used to modify the configuration.
|
||||
3. **rqfile**, if defined (default - ``rq.yaml``), is converted and injected into the configuration. At this stage the configuration is in its final form.
|
||||
4. for every node, configuration is applied, except ``once_by_`` directives:
|
||||
4.1 first the top-level attributes are set
|
||||
4.2 then ``by_<attribute-name>`` parameters except ``by_id`` are iterated to override or append(accumulate) the attributes
|
||||
4.3 then ``by_id`` is iterated to override any matching attributes, redefining what was set before
|
||||
1. first the top-level attributes are set
|
||||
2. then ``by_<attribute-name>`` parameters except ``by_id`` are iterated to override or append(accumulate) the attributes
|
||||
3. then ``by_id`` is iterated to override any matching attributes, redefining what was set before
|
||||
5. finally ``once_by_`<attribute-name>`` parameters are applied - only for one matching node for any set of matching values. This is useful for example if you want a specific file or command from only a single node matching a specific role, like running ``nova list`` only on one controller.
|
||||
|
||||
Once you are done with the configuration, you might want to familiarize yourself with :doc:`Usage </usage>`.
|
||||
|
||||
@ -12,28 +12,39 @@ Basic parameters:
|
||||
|
||||
* ``--only-logs`` only collect logs (skip files, filelists, commands and scripts)
|
||||
* ``-l``, ``--logs`` also collect logs (logs are not collected by default due to their size)
|
||||
* ``-C <command>`` enables ``shell mode``\*, Bash command (string) to execute on nodes. Using multiple ``-C`` statements will give the same result as using one with several commands separated by ``;`` (traditional Shell syntax), but for each ``-C`` statement a new SSH connection is established
|
||||
* ``-S <script>`` enables ``shell mode``, name of the Bash script file (you need to put it into ``scripts`` folder inside a path specified by ``rqdir`` config parameter, defaults to ``rq``) to execute on nodes
|
||||
* ``-P <file/path> <dest>`` enables ``shell mode``, upload local data to nodes (wildcards supported). You must specify 2 values for each ``-P`` switch.
|
||||
* ``-G <file/path>`` enables ``shell mode``, download (collect) data from nodes
|
||||
* ``-e``, ``--env`` filter by environment ID
|
||||
* ``-R``, ``--role`` filter by role
|
||||
* ``--config`` use custom configuration file to overwrite defaults. See ``config.yaml`` as an example
|
||||
* ``-c``, ``--config`` use custom configuration file to overwrite defaults. See ``config.yaml`` as an example
|
||||
* ``-j``, ``--nodes-json`` use json file instead of polling Fuel (to generate json file use ``fuel node --json``) - speeds up initialization
|
||||
* ``-o``, ``--dest-file`` the name/path for output archive, default is ``general.tar.gz`` and put into ``/tmp/timmy/archives``.
|
||||
* ``-v``, ``--verbose`` verbose(INFO) logging
|
||||
* ``-d``, ``--debug`` debug(DEBUG) logging
|
||||
|
||||
Shell mode - rqfile (``rq.yaml`` by default) is skipped, Fuel node is skipped, outputs of commands (specified with ``-C`` options) and scripts (specified with ``-S``) are printed on screen.
|
||||
**Shell Mode** - a mode of exectution which does the following changes:
|
||||
|
||||
* rqfile (``rq.yaml`` by default) is skipped
|
||||
* Fuel node is skipped
|
||||
* outputs of commands (specified with ``-C`` options) and scripts (specified with ``-S``) are printed on screen
|
||||
* any actions (cmds, scripts, files, filelists, put, **except** logs) and Parameter Based configuration defined in config are ignored.
|
||||
|
||||
The following parameters ("actions") are available, using any of them enables **Shell Mode**:
|
||||
|
||||
* ``-C <command>`` - Bash command (string) to execute on nodes. Using multiple ``-C`` statements will give the same result as using one with several commands separated by ``;`` (traditional Shell syntax), but for each ``-C`` statement a new SSH connection is established
|
||||
* ``-S <script>`` - name of the Bash script file to execute on nodes (if you do not have a path separator in the filename, you need to put the file into ``scripts`` folder inside a path specified by ``rqdir`` config parameter, defaults to ``rq``. If path separator is present, a given filename will be used directly as provided)
|
||||
* ``-P <file/path> <dest>`` - upload local data to nodes (wildcards supported). You must specify 2 values for each ``-P`` switch.
|
||||
* ``-G <file/path>`` - download (collect) data from nodes
|
||||
|
||||
========
|
||||
Examples
|
||||
========
|
||||
|
||||
* ``timmy`` - run according to the default configuration and default actions. Default actions are defined in ``rq.yaml`` (``/usr/share/timmy/rq.yaml``). Logs are not collected.
|
||||
* ``timmy -l`` - run default actions and also collect logs (default log setup applied - defaults are hardcoded in ``timmy/conf.py``). Such execution is similar to Fuel's "diagnostic snapshot" action, but will finish faster and collect less logs.
|
||||
* ``timmy --only-logs`` - only collect logs, no actions performed (default log setup, as above)
|
||||
* ``timmy -C 'uptime; free -m'`` - check uptime and memory on all nodes
|
||||
* ``timmy -G /etc/nova/nova.conf`` - get nova.conf from all nodes
|
||||
* ``timmy -R controller -P package.deb '' -C 'dpkg -i package.deb' -C 'rm package.deb' -C 'dpkg -l | grep [p]ackage'`` - push a package to all nodes, install it, remove the file and check that it is installed
|
||||
* ``timmy -с myconf.yaml`` - use a custom config file and run according to it
|
||||
* ``timmy -с myconf.yaml`` - use a custom config file and run according to it. Custom config can specify any actions, log setup, and other settings. See configuration doc for more details.
|
||||
|
||||
===============================
|
||||
Using custom configuration file
|
||||
|
||||
18
setup.py
18
setup.py
@ -1,4 +1,20 @@
|
||||
#!/usr/bin/env python
|
||||
#!/usr/bin/env python2
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2015 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
|
||||
from setuptools import setup
|
||||
import os
|
||||
|
||||
17
timmy.py
17
timmy.py
@ -1,4 +1,19 @@
|
||||
#!/usr/bin/env python
|
||||
#!/usr/bin/env python2
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2015 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from timmy.cli import main
|
||||
|
||||
|
||||
18
timmy/cli.py
18
timmy/cli.py
@ -54,7 +54,7 @@ def parse_args():
|
||||
help='Redirect Timmy log to a file.')
|
||||
parser.add_argument('-e', '--env', type=int,
|
||||
help='Env ID. Run only on specific environment.')
|
||||
parser.add_argument('-R', '--role', action='append',
|
||||
parser.add_argument('-r', '--role', action='append',
|
||||
help=('Can be specified multiple times.'
|
||||
' Run only on the specified role.'))
|
||||
parser.add_argument('-G', '--get', action='append',
|
||||
@ -80,6 +80,8 @@ def parse_args():
|
||||
' Each argument must contain two strings -'
|
||||
' source file/path/mask and dest. file/path.'
|
||||
' For help on shell mode, read timmy/conf.py.'))
|
||||
parser.add_argument('--rqfile', help='Path to an rqfile in yaml format,'
|
||||
' overrides default.')
|
||||
parser.add_argument('-l', '--logs',
|
||||
help=('Collect logs from nodes. Logs are not collected'
|
||||
' by default due to their size.'),
|
||||
@ -87,6 +89,9 @@ def parse_args():
|
||||
parser.add_argument('--fuel-ip', help='fuel ip address')
|
||||
parser.add_argument('--fuel-user', help='fuel username')
|
||||
parser.add_argument('--fuel-pass', help='fuel password')
|
||||
parser.add_argument('--fuel-proxy',
|
||||
help='use os system proxy variables for fuelclient',
|
||||
action='store_true')
|
||||
parser.add_argument('--only-logs',
|
||||
action='store_true',
|
||||
help=('Only collect logs, do not run commands or'
|
||||
@ -135,6 +140,9 @@ def parse_args():
|
||||
'selected if more -v are provided it will '
|
||||
'step to INFO and DEBUG unless the option '
|
||||
'-q(--quiet) is specified'))
|
||||
parser.add_argument('--fuel-cli', action='store_true',
|
||||
help=('Use fuel command line client instead of '
|
||||
'fuelclient library'))
|
||||
return parser
|
||||
|
||||
|
||||
@ -145,7 +153,7 @@ def main(argv=None):
|
||||
parser = parse_args()
|
||||
args = parser.parse_args(argv[1:])
|
||||
loglevels = [logging.WARNING, logging.INFO, logging.DEBUG]
|
||||
if args.quiet:
|
||||
if args.quiet and not args.log_file:
|
||||
args.verbose = 0
|
||||
loglevel = loglevels[min(len(loglevels)-1, args.verbose)]
|
||||
FORMAT = ('%(asctime)s %(levelname)s: %(module)s: '
|
||||
@ -161,11 +169,15 @@ def main(argv=None):
|
||||
conf['fuel_user'] = args.fuel_user
|
||||
if args.fuel_pass:
|
||||
conf['fuel_pass'] = args.fuel_pass
|
||||
if args.fuel_proxy:
|
||||
conf['fuel_skip_proxy'] = False
|
||||
if args.put or args.command or args.script or args.get:
|
||||
conf['shell_mode'] = True
|
||||
conf['do_print_results'] = True
|
||||
if args.no_clean:
|
||||
conf['clean'] = False
|
||||
if args.rqfile:
|
||||
conf['rqfile'] = args.rqfile
|
||||
if conf['shell_mode']:
|
||||
filter = conf['hard_filter']
|
||||
# config cleanup for shell mode
|
||||
@ -201,6 +213,8 @@ def main(argv=None):
|
||||
if args.dest_file:
|
||||
conf['archive_dir'] = os.path.split(args.dest_file)[0]
|
||||
conf['archive_name'] = os.path.split(args.dest_file)[1]
|
||||
if args.fuel_cli:
|
||||
conf['fuelclient'] = False
|
||||
logger.info('Using rqdir: %s, rqfile: %s' %
|
||||
(conf['rqdir'], conf['rqfile']))
|
||||
nm = pretty_run(args.quiet, 'Initializing node data',
|
||||
|
||||
@ -1,3 +1,20 @@
|
||||
#!/usr/bin/env python2
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2015 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
from tools import load_yaml_file
|
||||
from tempfile import gettempdir
|
||||
import os
|
||||
@ -15,6 +32,9 @@ def load_conf(filename):
|
||||
conf['fuel_ip'] = '127.0.0.1'
|
||||
conf['fuel_user'] = 'admin'
|
||||
conf['fuel_pass'] = 'admin'
|
||||
conf['fuel_tenant'] = 'admin'
|
||||
conf['fuelclient'] = True # use fuelclient library by default
|
||||
conf['fuel_skip_proxy'] = True
|
||||
conf['timeout'] = 15
|
||||
conf['prefix'] = 'nice -n 19 ionice -c 3'
|
||||
rqdir = 'rq'
|
||||
|
||||
19
timmy/env.py
19
timmy/env.py
@ -1,5 +1,22 @@
|
||||
#!/usr/bin/env python2
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# Copyright 2015 Mirantis, Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
project_name = 'timmy'
|
||||
version = '1.5.1'
|
||||
version = '1.8.1'
|
||||
|
||||
if __name__ == '__main__':
|
||||
exit(0)
|
||||
|
||||
@ -1,4 +1,26 @@
|
||||
#! /usr/bin/env python
|
||||
#! /usr/bin/env python2
|
||||
# -*- coding: utf-8 -*-
|
||||
|
||||
# The MIT License (MIT)
|
||||
# Copyright (c) 2009 Max Polk
|
||||
# Permission is hereby granted, free of charge, to any person
|
||||
# obtaining a copy of this software and associated documentation files
|
||||
# (the "Software"), to deal in the Software without restriction,
|
||||
# including without limitation the rights to use, copy, modify, merge,
|
||||
# publish, distribute, sublicense, and/or sell copies of the Software,
|
||||
# and to permit persons to whom the Software is furnished to do so,
|
||||
# subject to the following conditions:
|
||||
#
|
||||
# The above copyright notice and this permission notice shall be
|
||||
# included in all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
||||
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
||||
# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
||||
# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
||||
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
||||
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
||||
import os
|
||||
import errno
|
||||
|
||||
137
timmy/nodes.py
137
timmy/nodes.py
@ -30,6 +30,17 @@ import tools
|
||||
from tools import w_list, run_with_lock
|
||||
from copy import deepcopy
|
||||
|
||||
try:
|
||||
from fuelclient.client import Client as FuelClient
|
||||
except:
|
||||
FuelClient = None
|
||||
|
||||
try:
|
||||
from fuelclient.client import logger
|
||||
logger.handlers = []
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
class Node(object):
|
||||
ckey = 'cmds'
|
||||
@ -166,11 +177,12 @@ class Node(object):
|
||||
if code != 0:
|
||||
self.logger.warning('node: %s: could not determine'
|
||||
' MOS release' % self.id)
|
||||
release = 'n/a'
|
||||
else:
|
||||
self.release = release.strip('\n "\'')
|
||||
release = release.strip('\n "\'')
|
||||
self.logger.info('node: %s, MOS release: %s' %
|
||||
(self.id, self.release))
|
||||
return self.release
|
||||
(self.id, release))
|
||||
return release
|
||||
|
||||
def exec_cmd(self, fake=False, ok_codes=None):
|
||||
sn = 'node-%s' % self.id
|
||||
@ -205,9 +217,14 @@ class Node(object):
|
||||
dfile)
|
||||
if self.scripts:
|
||||
tools.mdir(ddir)
|
||||
self.scripts = sorted(self.scripts)
|
||||
scripts = sorted(self.scripts)
|
||||
mapscr = {}
|
||||
for scr in self.scripts:
|
||||
for scr in scripts:
|
||||
if type(scr) is dict:
|
||||
env_vars = scr.values()[0]
|
||||
scr = scr.keys()[0]
|
||||
else:
|
||||
env_vars = self.env_vars
|
||||
if os.path.sep in scr:
|
||||
f = scr
|
||||
else:
|
||||
@ -223,7 +240,7 @@ class Node(object):
|
||||
outs, errs, code = tools.ssh_node(ip=self.ip,
|
||||
filename=f,
|
||||
ssh_opts=self.ssh_opts,
|
||||
env_vars=self.env_vars,
|
||||
env_vars=env_vars,
|
||||
timeout=self.timeout,
|
||||
prefix=self.prefix)
|
||||
self.check_code(code, 'exec_cmd', 'script %s' % f, errs,
|
||||
@ -295,6 +312,7 @@ class Node(object):
|
||||
file=f[0],
|
||||
dest=f[1],
|
||||
recursive=True)
|
||||
self.check_code(code, 'put_files', 'tools.put_file_scp', errs)
|
||||
|
||||
def logs_populate(self, timeout=5):
|
||||
|
||||
@ -388,17 +406,42 @@ class NodeManager(object):
|
||||
self.import_rq()
|
||||
self.nodes = {}
|
||||
self.fuel_init()
|
||||
# save os environment variables
|
||||
environ = os.environ
|
||||
if FuelClient and conf['fuelclient']:
|
||||
try:
|
||||
if self.conf['fuel_skip_proxy']:
|
||||
os.environ['HTTPS_PROXY'] = ''
|
||||
os.environ['HTTP_PROXY'] = ''
|
||||
os.environ['https_proxy'] = ''
|
||||
os.environ['http_proxy'] = ''
|
||||
self.logger.info('Setup fuelclient instance')
|
||||
self.fuelclient = FuelClient()
|
||||
self.fuelclient.username = self.conf['fuel_user']
|
||||
self.fuelclient.password = self.conf['fuel_pass']
|
||||
self.fuelclient.tenant_name = self.conf['fuel_tenant']
|
||||
# self.fuelclient.debug_mode(True)
|
||||
except Exception as e:
|
||||
self.logger.info('Failed to setup fuelclient instance:%s' % e,
|
||||
exc_info=True)
|
||||
self.fuelclient = None
|
||||
else:
|
||||
self.logger.info('Skipping setup fuelclient instance')
|
||||
self.fuelclient = None
|
||||
if nodes_json:
|
||||
self.nodes_json = tools.load_json_file(nodes_json)
|
||||
else:
|
||||
self.nodes_json = json.loads(self.get_nodes_json())
|
||||
if (not self.get_nodes_fuelclient() and
|
||||
not self.get_nodes_cli()):
|
||||
sys.exit(4)
|
||||
self.nodes_init()
|
||||
# apply soft-filter on all nodes
|
||||
for node in self.nodes.values():
|
||||
if not self.filter(node, self.conf['soft_filter']):
|
||||
node.filtered_out = True
|
||||
if not conf['shell_mode']:
|
||||
self.nodes_get_release()
|
||||
if not self.get_release_fuel_client():
|
||||
self.get_release_cli()
|
||||
self.nodes_reapply_conf()
|
||||
self.conf_assign_once()
|
||||
if extended:
|
||||
@ -407,6 +450,8 @@ class NodeManager(object):
|
||||
do additional apply_conf(clean=False) with this yaml.
|
||||
Move some stuff from rq.yaml to extended.yaml'''
|
||||
pass
|
||||
# restore os environment variables
|
||||
os.environ = environ
|
||||
|
||||
def __str__(self):
|
||||
def ml_column(matrix, i):
|
||||
@ -505,7 +550,51 @@ class NodeManager(object):
|
||||
fuelnode.filtered_out = True
|
||||
self.nodes[self.conf['fuel_ip']] = fuelnode
|
||||
|
||||
def get_nodes_json(self):
|
||||
def get_nodes_fuelclient(self):
|
||||
if not self.fuelclient:
|
||||
return False
|
||||
try:
|
||||
self.nodes_json = self.fuelclient.get_request('nodes')
|
||||
self.logger.debug(self.nodes_json)
|
||||
return True
|
||||
except Exception as e:
|
||||
self.logger.warning(("NodeManager: can't "
|
||||
"get node list from fuel client:\n%s" % (e)),
|
||||
exc_info=True)
|
||||
return False
|
||||
|
||||
def get_release_fuel_client(self):
|
||||
if not self.fuelclient:
|
||||
return False
|
||||
try:
|
||||
self.logger.info('getting release from fuel')
|
||||
v = self.fuelclient.get_request('version')
|
||||
fuel_version = v['release']
|
||||
self.logger.debug('version response:%s' % v)
|
||||
clusters = self.fuelclient.get_request('clusters')
|
||||
self.logger.debug('clusters response:%s' % clusters)
|
||||
except:
|
||||
self.logger.warning(("Can't get fuel version or "
|
||||
"clusters information"))
|
||||
return False
|
||||
self.nodes[self.conf['fuel_ip']].release = fuel_version
|
||||
cldict = {}
|
||||
for cluster in clusters:
|
||||
cldict[cluster['id']] = cluster
|
||||
if cldict:
|
||||
for node in self.nodes.values():
|
||||
if node.cluster:
|
||||
node.release = cldict[node.cluster]['fuel_version']
|
||||
else:
|
||||
# set to n/a or may be fuel_version
|
||||
if node.id != 0:
|
||||
node.release = 'n/a'
|
||||
self.logger.info('node: %s - release: %s' % (node.id,
|
||||
node.release))
|
||||
return True
|
||||
|
||||
def get_nodes_cli(self):
|
||||
self.logger.info('use CLI for getting node information')
|
||||
fuelnode = self.nodes[self.conf['fuel_ip']]
|
||||
fuel_node_cmd = ('fuel node list --json --user %s --password %s' %
|
||||
(self.conf['fuel_user'],
|
||||
@ -516,10 +605,22 @@ class NodeManager(object):
|
||||
timeout=fuelnode.timeout,
|
||||
prefix=fuelnode.prefix)
|
||||
if code != 0:
|
||||
self.logger.critical(('NodeManager: cannot get '
|
||||
'fuel node list: %s') % err)
|
||||
sys.exit(4)
|
||||
return nodes_json
|
||||
self.logger.warning(('NodeManager: cannot get '
|
||||
'fuel node list from CLI: %s') % err)
|
||||
self.nodes_json = None
|
||||
return False
|
||||
self.nodes_json = json.loads(nodes_json)
|
||||
return True
|
||||
|
||||
def get_release_cli(self):
|
||||
run_items = []
|
||||
for key, node in self.nodes.items():
|
||||
if not node.filtered_out:
|
||||
run_items.append(tools.RunItem(target=node.get_release,
|
||||
key=key))
|
||||
result = tools.run_batch(run_items, 100, dict_result=True)
|
||||
for key in result:
|
||||
self.nodes[key].release = result[key]
|
||||
|
||||
def nodes_init(self):
|
||||
for node_data in self.nodes_json:
|
||||
@ -544,16 +645,6 @@ class NodeManager(object):
|
||||
if self.filter(node, self.conf['hard_filter']):
|
||||
self.nodes[node.ip] = node
|
||||
|
||||
def nodes_get_release(self):
|
||||
run_items = []
|
||||
for key, node in self.nodes.items():
|
||||
if not node.filtered_out:
|
||||
run_items.append(tools.RunItem(target=node.get_release,
|
||||
key=key))
|
||||
result = tools.run_batch(run_items, 100, dict_result=True)
|
||||
for key in result:
|
||||
self.nodes[key].release = result[key]
|
||||
|
||||
def conf_assign_once(self):
|
||||
once = Node.conf_once_prefix
|
||||
p = Node.conf_match_prefix
|
||||
|
||||
@ -31,7 +31,6 @@ from flock import FLock
|
||||
from tempfile import gettempdir
|
||||
from pipes import quote
|
||||
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
slowpipe = '''
|
||||
import sys
|
||||
@ -148,26 +147,6 @@ def run_batch(item_list, maxthreads, dict_result=False):
|
||||
raise KeyboardInterrupt()
|
||||
|
||||
|
||||
def get_dir_structure(rootdir):
|
||||
"""
|
||||
Creates a nested dictionary that represents the folder structure of rootdir
|
||||
"""
|
||||
dir = {}
|
||||
try:
|
||||
rootdir = rootdir.rstrip(os.sep)
|
||||
start = rootdir.rfind(os.sep) + 1
|
||||
for path, dirs, files in os.walk(rootdir):
|
||||
folders = path[start:].split(os.sep)
|
||||
subdir = dict.fromkeys(files)
|
||||
parent = reduce(dict.get, folders[:-1], dir)
|
||||
parent[folders[-1]] = subdir
|
||||
except:
|
||||
logger.critical('failed to create list of the directory: %s' %
|
||||
rootdir)
|
||||
sys.exit(1)
|
||||
return dir
|
||||
|
||||
|
||||
def load_json_file(filename):
|
||||
"""
|
||||
Loads json data from file
|
||||
@ -343,5 +322,6 @@ def free_space(destdir, timeout):
|
||||
def w_list(value):
|
||||
return value if type(value) == list else [value]
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
exit(0)
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user