36 Commits

Author SHA1 Message Date
Tin Lam
cbc3044d4d Fix CRLF issues
Some files in the repo has a DOS/Windows CRLF (\r\n) ending instead
of the Unix return.  This patch set fixes the issue.

Change-Id: Ic6601f3ace9507f30c487f495198c2397b5bf805
2017-06-03 11:39:05 -05:00
Tin Lam
2ea460f044 Remove executable flag from files
The current repo has a number of python and conf files with executable flag
set (+x).  This patch set removes that flag from the files.

Change-Id: I00a5f92afae22f14b046e32563cba731cdaa47b2
Signed-off-by: Tin Lam <tin@irrational.io>
2017-05-30 17:32:55 +00:00
Jenkins
7de1e62db5 Merge "Enable PEP8 check and correct all PEP8 issues" 2017-05-30 17:26:28 +00:00
Tin Lam
67f6434bc0 Enable PEP8 check and correct all PEP8 issues
This patch set enables PEP8 standard check for project Valet, and
corrects all outstanding PEP8 issues.

Story: #2001040
Task: #4602

Co-Authored-By: Omar Rivera <gomarivera@gmail.com>
Change-Id: I4b987ff28b02ea8a6da77fb0f29eda1515d212ac
2017-05-28 16:18:17 +00:00
Jenkins
0844b8b5ad Merge "Only admin role can access valet" 2017-05-26 19:27:51 +00:00
Tin Lam
66ef9173a9 Fix requirements.txt and test-requirements.txt
This patch set fixes the requirements, and two workarounds in the
documentation due to the previously broken requirements.

Change-Id: I3e2e7ec5897270f266ba5ea84c9927f7623b4483
2017-05-26 09:51:22 -05:00
Tin Lam
7254f9ec97 Fix initial gate
This patch set fixes the initial gating failures for project Valet.

Change-Id: Id61cfa4171b8f1e7dc4824abe3d7fee7191aef92
2017-05-25 14:43:44 -05:00
sg774j
d3ae421148 Only admin role can access valet
Enforced only admin can create plans in valet. If unauthorized,
appropriate error message will be displayed.

Change-Id: I8166f5675b247bc309db51b9b671a511e1d6abca
2017-05-23 07:12:59 -05:00
Omar Rivera
ea89fbfbbb Updates information for PyPI release
The 'valet' project name is taken in python index.
We are going to publish using "openstack-valet".
2017-05-09 14:02:20 -05:00
Omar Rivera
d2e79829a3 Syncrhonized plugins codebase 2017-05-05 12:07:42 -05:00
sg774j
76ab80040a High Availibilty of Music
Added capability of reading and writing to
one of the other 2 valet Music databases
incase the local database is down. The
sequence of fallback is read from the valet
configuration file

Change-Id: I0de371996c63f3d1fb386e54cf4e239b464d870a
2017-05-05 11:42:53 -05:00
Chris
faee9727ab Changes Entry Points for Valet Engine
Add /usr/bin/valet-engine entry point

Change-Id: Ia9bd7e5962dddb52dd3aecacc266c9c1ffd5f477
2017-05-05 11:34:43 -05:00
Cliff Parsons
dbda429146 Music host/port configuration issue
Engine was not passing the configured host/port combination
into the appropriate classes. This change corrects the problem.
Fix permissions issue with /var/log/valet dir
Set permissions so that valet-engine can create the engine.log file

Change-Id: Ib79c01faf9009031c87a6f59f1b8818afcfc824e
2017-05-05 11:34:16 -05:00
Gueyoung Jung
ce1282b192 Refactor Engine to reduce delay
Change-Id: If6594a034db6e398e26dfdafafbae0c5fbfbf51f
2017-05-05 09:48:25 -05:00
Cliff Parsons
860554760b Fix engine startup issue in resource.py
Fixed a bug in the compute_avail_resources() method
 the new local variables static_ram_standby_ratio,
 static_cpu_standby_ratio, and static_disk_standby_ratio
 were being used before initialization.

Change-Id: Id03cd7a84992bcbf41cd779535226e354e9a6384
2017-05-04 23:12:16 -05:00
Jung, Gueyoung (gj6146)
d65a76e8e9 Debug listener error
Change-Id: I241ec0229df6ab7e2b40b99bf463b17d7eca1374
2017-05-04 23:11:53 -05:00
Jung, Gueyoung (gj6146)
7dcf0f9ea8 Add params to valet conf
Change-Id: I20698f542e05b667b2b59c07f27f8debcb777587
2017-05-04 23:10:52 -05:00
Jung, Gueyoung (gj6146)
1c53afa58e Improve success rate
Change-Id: I59e4a8d8e23575fb0af74924dad027340517f7e7
2017-05-04 23:08:52 -05:00
Omar Rivera
46bd04cb24 Trace back is seen on populate.log
ConfigFileValueError: invalid literal for
int() with base 10: 'rabbit_port'. At the very beginning of Valet
deployment, /usr/etc/valet/valet.conf file is getting read instead
of /etc/valet/valet.conf. Here the value of port is expected to be
an int and not string. Changing the value to int.

Change-Id: I820c5f44a76d5a4a313d896b50242071b37eb6b2
2017-05-04 22:12:15 -05:00
Omar Rivera
000a827f03 Updates pecan_populate.sh
Change-Id: I4461d8becd3eeaa52ade205b6428d7140b01992d
2017-05-04 22:09:22 -05:00
Jung, Gueyoung (gj6146)
0dcfb240b2 Handle locations for ad-hoc requests
Change-Id: I025b9afd6ebb1027019e2b9a35934b8af6ab092a
2017-05-04 22:06:59 -05:00
Joe D'Andrea
1e146b1499 Add locations to POST /v1/plans API
When determining placements for a related set of resources
via OpenStack Heat, there's no a priori knowledge of specific
location candidates. Thus the /v1/plans API never accounted
for them. However, when it comes to ad hoc placement of a
single resource (e.g., via OpenStack Nova), such knowledge
*does* exist. This is one piece of a three-part change (same
issue tracking ID).

Change-Id: I327b87a1a1104019239547d03600cdd849ea6501
2017-05-04 21:43:50 -05:00
Gueyoung Jung
8083445865 Lazy resource status update to db
Client not catching exceptions
Engine Health Check refined

Change-Id: I1d5a5d385471815bfe0c2beb50bcf9ac57d9cae8
2017-05-04 21:29:24 -05:00
Gueyoung Jung
fc0d8fc3fc Resolve affinity group with flavor id
Fix tempest exclusivity test
default_ram_allocation_ratio to float
cpu&disk ratio to float
incorrect types for conf options
valet-engine does not restart after manual kill

Change-Id: I3eae2bb2144a5857dae17b99fb868b57116cef05
2017-05-04 17:32:46 -05:00
Gueyoung Jung
6185beead9 Improve performance by deferring events
Change-Id: Icefc62f7f058fbc0e094c5153cb0567b034109f6
2017-05-04 06:52:10 -05:00
Gueyoung Jung
b3c3bdb6fa Matching flavor with id
Fix parallel running

Change-Id: I015649e7287cf1ebb254639c24a2e355c6d03fd1
2017-05-01 22:47:55 -05:00
Gueyoung Jung
ce6b9b9871 Resolve resource status sync
adding configuration to tempest.conf

Change-Id: I4c76f885db3774e91c927fb5bbe0687080274067
2017-05-01 22:30:48 -05:00
Joe D'Andrea
0ff46ffb36 Improve Music query and ORM performance
From the early days of Valet, there has been a known O(n)
inefficiency concerning Music query filtering. This is further
exacerbated by the instantiation of Music-backed ORM objects,
leading to a O(n*m) inefficiency.

This commit is a mitigation that greatly increases the likelihood
of O(1) filtering performance. valet-engine job status check
efficiency is also improved as a result.

This commit also fixes a condition where valet-api may create
multiple valet-engine placement requests for a stack id that
already has a request in process (e.g., if nova-scheduler calls
valet-api multiple times due to a retry by nova-controller).

IMPORTANT: When applying this commit, perform the following changes.

1: In valet.conf, ensure "tries" and "interval" are set to
100 and 0.1, respectively. (If they're commented out, change
the commented versions. There is no need to uncomment them.)

[music]
tries = 100
interval = 0.1

2: Manually create secondary keys in the valet-api cassandra tables.

For instance, given a keyspace of "valet_aic", execute these commands
in cqlsh on the cassandra server used by valet-api.

CREATE INDEX ON valet_aic.plans (stack_id);
CREATE INDEX ON valet_aic.placements (plan_id);
CREATE INDEX ON valet_aic.placements (orchestration_id);
CREATE INDEX ON valet_aic.placements (resource_id);

Until Music is updated to handle this automatically, these commands
must be executed every time the tables are dropped/re-added.

3: Determine the realistically expected number of simultaneous plan
requests for valet-api. Ensure the server running valet-api is sized
appropriately, set the httpd configuration's thread count to match
this number (perhaps a few extra), and restart the daemon.

For example, to specify 10 threads in Apache2 httpd, edit the
WSGIDaemonProcess directive for valet-api (this is one line):

WSGIDaemonProcess valet user=ubuntu group=ubuntu
    threads=10 python-home=/opt/stack/heat.venv

4: To start with a clean slate, clear out any residual placement
requests and responses in the valet-engine cassandra tables.

For instance, given a keyspace of "valet_aic", execute these commands
in cqlsh on the cassandra server used by valet-engine.

TRUNCATE valet_aic.placement_requests;
TRUNCATE valet_aic.placement_results;

Change-Id: I76528c9b81dc451241ecc547cadc18cc4b1284df
2017-05-01 22:24:29 -05:00
Gueyoung Jung
543ef67ed1 Improve delay with fine-grained locking
Change-Id: I5a8ef020e0c3dda50fa725c6b6509bc876593115
2017-05-01 22:20:47 -05:00
Omar Rivera
125f5ccfab Fix erroneous Engine bootstrap
Use oslo_log
getting music out of api
updating valet.conf
Health Check for hanged Engine
Wrong python-requests version

Change-Id: Ibb2649c81dad94e51b579c0a99cfaf37b626095c
2017-05-01 21:46:30 -05:00
Nickolas Vela
e66e4329cd Use upstream Music link
Updates references to upstream Music repo
2017-05-01 19:25:09 -05:00
Tanvir Talukder
ad2b7fab31 Fix pep8 and docstring violations
Fix issues in py files in the directories
valet/valet/api/common
valet/api
valet/cli
valet/engine/optimizer/
valet/tests/
valet/tests/tempest
valet_plugins
2017-05-01 19:24:57 -05:00
MARTIN
5b9c5cf5b8 Adds license header for .py files 2017-05-01 19:10:48 -05:00
Tin Lam
e07d769270 Clean-up for tox -epy35 errors
Py35 does allow for the loading of modules in the same folder without
specifying the patch.  Updated import to use absolute pathing.

This patch set also remove `# noqa` in EOF import statements.  Those
should be addressed in future patch set that do PEP8 clean up.
2017-05-01 19:10:47 -05:00
Tanvir Talukder
e30684b0c3 Valet Documentation
Added REST API Documentation
Added Valet Plugins Documentation
Added Apache License info
2017-05-01 19:07:42 -05:00
Tanvir Talukder
ae593ca663 Initial commit 2016-12-12 08:50:24 -06:00