Alexander Tivelkov 8203a6ac07 Graceful error handling via workflow commands
Workflow rules can now define "failure" blocks to handle the various
errors and exceptions occurred while executing the actions.
These blocks are passed to 'update-cf-stack' and 'send-command'
functions, so their callbacks can call them to handle the errors and
exceptions. The actual error and exception data may be passed via
context in the same way as result is passed to "success" handlers

If 'failure" block is skipped, the global exception is risen, which
interrupts the workflow execution and reports an unhandled error to
the API at the error level.
To gracefully stop the workflow execution without throwing an exception,
a 'failure' block may define a '<stop/>' command, which interrupts the
execution after the end of current loop and without throwing any
exceptions

These changes allow to handle exceptions and unexpected states while
interacting with heat, as well as agent-side exceptions delivered to
conductor from Agent. Also, the same approach works for handling a
timeout while waiting for task result from the Agent.

To support timeouts a 'send-command' function must be passed with a
'timeout' parameter. If it is skipped, the timeout is considered to
be infinite.

The workflows have been update with failure blocks on all the
commands. These blocks contain error-level reporting and <stop/>
command to interrupt the flow.
No timeouts were set in workflows, so timeout feature is currently
inactive for the existing workflows (as the
'inititialization timeout' concept needs to be introduced)

Change-Id: Ia791d4656463240ed197bcd90b9d9eae648270af
2013-08-06 16:56:07 +04:00

89 lines
3.2 KiB
Python

# Copyright (c) 2013 Mirantis Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from muranoconductor.commands.windows_agent import AgentTimeoutException
from muranoconductor.commands.windows_agent import UnhandledAgentException
import xml_code_engine
from openstack.common import log as logging
log = logging.getLogger(__name__)
def send_command(engine, context, body, template, service, unit, mappings=None,
result=None, error=None, timeout=None, **kwargs):
if not mappings:
mappings = {}
command_dispatcher = context['/commandDispatcher']
if timeout:
timeout = int(timeout)
def callback(result_value):
log.info(
'Received result from {2} for {0}: {1}'.format(
template, result_value, unit))
ok = []
errors = []
if isinstance(result_value, AgentTimeoutException):
errors.append({
'type': "timeout",
'messages': [result_value.message],
'timeout': result_value.timeout
})
else:
if result_value['IsException']:
msg = "A general exception has occurred in the Agent: " + \
result_value['Result']
errors.append({
'type': "general",
'messages': [msg],
})
else:
for res in result_value['Result']:
if res['IsException']:
errors.append({
'type': 'inner',
'messages': res['Result']
})
else:
ok.append(res)
if ok:
if result is not None:
context[result] = ok
success_handler = body.find('success')
if success_handler is not None:
engine.evaluate_content(success_handler, context)
if errors:
if error is not None:
context[error] = errors
failure_handler = body.find('failure')
if failure_handler is not None:
engine.evaluate_content(failure_handler, context)
else:
log.error("No failure block found for exception")
if isinstance(result_value, AgentTimeoutException):
raise result_value
else:
raise UnhandledAgentException(errors)
command_dispatcher.execute(
name='agent', template=template, mappings=mappings,
unit=unit, service=service, callback=callback, timeout=timeout)
xml_code_engine.XmlCodeEngine.register_function(send_command, "send-command")