249 Commits

Author SHA1 Message Date
Jesse Pretorius
0c4bb0fb1b Use centralised test scripts
This patch consumes the test scripts implemented by
https://review.openstack.org/375061 to ensure that
the tests and test preparation is consistent and
more maintainable.

Change-Id: I6f05ec7046d5613804fa8b4c2fb9323370239da0
2016-09-28 08:56:33 +01:00
Jesse Pretorius
0bbacf63ba Revert dynamic includes for inventory-based conditionals
Forcing dynamic includes for inventory-based conditionals
results in incomplete execution.

Change-Id: I8e8e2b386951ab99ae8fbac6ca348c51dfcda91d
14.0.0.0rc2
2016-09-26 19:52:25 +01:00
Jesse Pretorius
7872b49691 Force Ansible to use dynamic includes
Ansible 2.1.1 introduces a regression in the way conditional
includes are handled which results in every task in the
included file being evaluated even if the condition for the
include is not met. This extends the run time significantly
for a deployment.

This patch forces all conditional includes to be dynamic.

Change-Id: Ia4c26126a7c1fc2728252eacf71870b7bfb5b6a5
Related-Bug: https://github.com/ansible/ansible/issues/17687
2016-09-22 14:08:35 +01:00
Jenkins
4cb21da177 Merge "Remove testing vars present in test repo" 14.0.0.0rc1 2016-09-20 18:56:21 +00:00
avnish
e21be41b08 Update home page link in cfg file
Change-Id: I51293947c3530546eb6f1446c1e08f0dc402dd6c
2016-09-20 14:21:45 +05:30
Travis Truman
8d836da6b8 Remove testing vars present in test repo
No need to override these any longer now that
they are declared in the tests repo

Change-Id: I0f19fb22fb95987956ea4a1468efde99cbe35140
2016-09-19 14:25:18 -04:00
Jenkins
c6ecc19f86 Merge "Address ansible_ssh_* var deprecation" 2016-09-19 16:25:42 +00:00
Travis Truman
a74af47939 Address ansible_ssh_* var deprecation
Replacing usage of ansible_ssh_host, ansible_ssh_user,
ansible_ssh_port with ansible_host, ansible_user and ansible_port

Change-Id: I4adb6484c13523a2527adc62846b736b0c5f228e
2016-09-17 17:50:47 -04:00
Kevin Carter
3614448799 Update testing bits for consistency
Change-Id: I5bbfdb11cd3a07bba1c47987a87743efcc33d15a
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-09-17 11:57:30 -05:00
Andy McCrae
01e1299110 Remove unrequired messaging setup task file
The messaging setup happens as part of the playbook.
This task file never gets called, so can be removed.

Change-Id: I1aba4c5a26ad0e85e0cb001a0863681acd6f6008
2016-09-16 15:26:40 +01:00
Kevin Carter
1bfcd10bc5
Add role linking to tox tests
Change-Id: I36df88a9e02a96e099ac63a3d23c2f79b936b258
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-09-14 10:58:29 -05:00
Travis Truman
05892b5aa9 Ansible 2.1.1 role testing
Change-Id: Icf4bce17e6aa042b60a54499749695917a96dba9
2016-09-12 16:03:28 -04:00
Marc Gariepy
94114142d6 Fix depreciation "Using bare variables"
add full variable syntax

Change-Id: Ic945ffb3c321a93db14776d5b1892d790c146ca4
2016-09-12 11:51:14 -04:00
Jesse Pretorius
07509721e8 Compress test execution logs
The volume of logs we collect in CI jobs are extensive. This
patch ensures that the logs collected are compressed to reduce
the space taken in CI systems.

PYTHONUNBUFFERED is also set to ensure that the console log
from the CI jobs is recorded in the exact order of execution.

Change-Id: Ia37c4fe369dd8ce9d49c21a46c575707b1826d46
Related-Bug: #1620849
2016-09-09 10:38:13 +01:00
Jesse Pretorius
2fd095b68b Update paste, policy and rootwrap configurations 2016-09-08
Change-Id: I181b53c70cc51ef3a46a9af076f740048351c4de
2016-09-08 18:05:20 +01:00
Jenkins
e44230acfd Merge "Add credential_setup for keystone" 2016-09-08 14:55:18 +00:00
Lance Bragstad
f0ed20dbeb Re-activate service catalog caching
Now that the service catalogue caching issues have been addressed
upstream, remove the keystone.conf section which disables caching
for the service catalog.

Change-Id: I7636843fe66d386b093ccf109da679585a565a14
2016-09-08 11:48:25 +00:00
Andy McCrae
fbd9535221 Add credential_setup for keystone
We need to setup the appropriate directory for credential_setup and run
the keystone-manage credential_setup command.

We created the directory and the '[credential]' stanza in the
keystone.conf, which will ensure we can add additional settings using
config_template if any further are required.

We need to setup the autorotation cron job and distribution for
credential keys.

Additionally, we include all tempest tests now that we are
supporting this feature.

Change-Id: Ifd85ed1a64538ed037e4426cc50238d2b16d51e5
2016-09-08 12:06:48 +01:00
Jenkins
88dd8c4e15 Merge "Add tempest to keystone role tests" 2016-09-07 07:21:37 +00:00
Jenkins
d56cbeb348 Merge "Fix nginx SCRIPT_NAME uwsgi_param" 2016-09-06 22:40:47 +00:00
Andy McCrae
a0d71d6fff Add tempest to keystone role tests
The keystone functional tests are currently not using tempest, and are
quite lacking in functionality.

This PR adds tempest testing to the keystone role.
We use the tempest.api.identity tests but exclude the tests for
credentials and ec2 which we don't currently support.

TODO: (andymccr)
We need to add support for these (and remove the exclusion) by
performing a 'keystone-manage credential_setup --keystone-user keystone
--keystone-group keystone'

Change-Id: I9a7207e75040c304c53820795cb66ce9be00c350
2016-09-06 16:24:01 +01:00
Andy McCrae
42cef50e5f Fix nginx SCRIPT_NAME uwsgi_param
The SCRIPT_NAME uwsgi_param is passed to the client resulting in errors
such as:

UnknownConnectionError: Unexpected exception for
http://10.1.1.101:5000keystone-wsgi-public/v3/auth/tokens:
Failed to parse: 10.1.1.101:5000keystone-wsgi-public

We should default this to a blank string to ensure this works
appropriately.

Change-Id: I3da36f8e2281eefdbad903d438ffd93ddd2f5071
2016-09-05 16:26:34 +01:00
Andy McCrae
d0e5097ecc Use the central test repository for Keystone
Move to use the central test repository. Including a few changes to keep
consistency when moving.

* Change network range to fit into recommended infra ranges.
* Include memcached for future keystone testing

Change-Id: I6c79f291a5893d910a6dd743b39a01c7fac11bdb
2016-09-05 10:36:33 +01:00
Jenkins
ec65f10b7e Merge "Fix apache + uwsgi for keystone" 2016-09-01 16:29:56 +00:00
Andy McCrae
38dbd423d6 Fix apache + uwsgi for keystone
The vars port specification in the keystone-httpd.conf.j2 were
incorrectly referencing vars instead of strings, causing failures.

Additionally, the conf created for uwsgi and apache includes all the
WSGI settings - which is not in line with how it should look.

Encasing the WSGI settings in an if statement so that it lines up with
the example conf file from the keystone docs.

We will also need to enable the proxy_http module for uwsgi with apache.

Change-Id: I64eec88452333eee6397ccbad13cd80a5275e607
2016-09-01 14:36:03 +01:00
Jenkins
f687af453a Merge "Fix nginx to work with RedHat/CentOS" 2016-09-01 10:59:42 +00:00
Jenkins
13a6085bca Merge "Allow Uwsgi configuration overrides" 2016-09-01 09:40:34 +00:00
Andy McCrae
15733bbd2f Fix nginx to work with RedHat/CentOS
The CentOS version of nginx uses a conf.d style directory for virtual
hosts instead of a sites-available/sites-enabled approach.

We can add a var to select where the configuration file is placed, and
only perform the link on Debian based systems.

Change-Id: I00b8af093e17a4450b642a1534b8ec647c9d2513
2016-09-01 10:30:22 +01:00
Jesse Pretorius
b6f914a2dc Shorten tox target names
Due to the length limit for the shabeng line [1] the execution of
these tox targets in OpenStack-CI is failing (the full shebang
length is 130 chars).

This patch shortens the names appropriately.

[1] https://github.com/pypa/virtualenv/issues/596

Change-Id: I9011eac714e40d33baff7c1a1fc6eb0fdf47df55
2016-09-01 09:28:28 +01:00
Steve Lewis
3b47fc7ef5 Allow Uwsgi configuration overrides
Release note is updated to describe this functionality as well as
general improvments for clarity.

Change-Id: I41838010fc4b6e892bec08035798f096aff5af8f
Related: blueprint keystone-uwsgi
2016-09-01 07:20:12 +01:00
Jenkins
6b3af85bf2 Merge "Configure Apache to proxy for Uwsgi" 2016-08-31 18:46:22 +00:00
Jenkins
b9161ddabb Merge "Isolate mod_wsgi from Apache install" 2016-08-31 18:43:46 +00:00
Steve Lewis
52b1a71759 Configure Apache to proxy for Uwsgi
A tox target to test Apache with Uwsgi is introduced.

A release note summarizing all Uwsgi and Nginx feature work for this
blueprint is included.

Change-Id: I5c89e4d9925a3077111aabe85aaa6f6eaa944848
Related: blueprint keystone-uwsgi
2016-08-31 18:12:42 +01:00
Steve Lewis
db7248bdc1 Isolate mod_wsgi from Apache install
This allows deploys to elect to have Keystone run off of Apache
without implying the use of mod_wsgi, such as with uwsgi. A
following patch will introduce the Apache configuration needed to
link Apache to uwsgi so that existing Federation support can be
used.

Release notes summarizing all changes for this blueprint will follow
in the final patch in the series.

Change-Id: Idf9e48b0c93174648982cf27cf922d3801565c74
Related: blueprint keystone-uwsgi
2016-08-31 17:09:19 +00:00
Jesse Pretorius
30bd479510 Correct developer mode package var name
In https://review.openstack.org/363077 the var name was incorrectly
set to keystone_developer_distro_packages instead of
keystone_developer_mode_distro_packages as has been the pattern
used across all the other roles. This patch corrects the var name
to ensure that it fits the same pattern as all other roles.

Change-Id: Ia2d3b14b6d3fcf86bdc53a901f1ef2aa6e8128bd
2016-08-31 09:06:29 +01:00
Jesse Pretorius
2b8aa0703c Rename package lists (and related vars) appropriately
In order to make it easier to differentiate between the lists of
python packages, distribution packages, downloaded packages,
package pins and other similar variables the variable names are
being changed to ensure that they have a more explicit suffix
that defines the purpose and makes the naming more consistent.

This is to facilitate a lookup plugin which will be able to look
up all the package lists and present them as a consolidated piece
of data which may be used for artifact preparation.

Change-Id: Ia9a7f3c237cc58d00c351a3393e5a723323b6890
14.0.0.0b3
2016-08-30 20:05:21 +00:00
Steve Lewis
4edb378b1e Install and configure Nginx
Apply configuration to add request time to the access log.
Creates virtual hosts for each Keystone service.
Enables SSL termination within Nginx.

The Uwsgi sockets are updated to match the Keystone developer docs
to improve consistency of experience for operators.

No Shibboleth integration is included.
Not introducing any additional Nginx restarts based on changes in
Federation configuration yet for this reason.

Change-Id: Iec42810be7ff6d05fa38deb23996e66e0c34da8e
Related: blueprint keystone-uwsgi
2016-08-29 17:03:07 -07:00
Jenkins
812ea82d71 Merge "Implement CentOS 7 support in os_keystone" 2016-08-29 17:37:10 +00:00
Kevin Carter
0de819e92a Implement CentOS 7 support in os_keystone
This change implements CentOS 7 support within the os_keystone role.

Depends-on: I333fb1887339e8dc9ebf10ff137dda3cff629dc0
Change-Id: Ib339cd0657f7008fa48bf74f8d6ddd4b8add2ea1
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
2016-08-29 14:20:48 +00:00
Adam Reznechek
61759e7ddd Work around Ansible vcpu fact bug on ppc64le
This changes how keystone_wsgi_processes is set to work around a
bug in Ansible where ansible_processor_vcpus is reported as 0 on
ppc64le today due to issues with /proc/cpuinfo format differences.
An upstream fix has been proposed at [1].

[1] - ansible/ansible#16533

Change-Id: I5cf812a03d7d716cd9beadc46804565c88df1d25
2016-08-26 12:05:04 -05:00
Jenkins
9d0a762371 Merge "Add a test Scenario for uwsgi & nginx" 2016-08-25 12:59:24 +00:00
Andy McCrae
7e5548e39d Add a test Scenario for uwsgi & nginx
New func_uwsgi-nginx test scenario that will test keystone installed
with uwsgi behind nginx.
This will start off as experimental but can be used to test the final
nginx solution as it is finalised.

Change-Id: Id5ec7b1895b51232aacf1c86e564563de6c21d3f
Related: blueprint keystone-uwsgi
2016-08-25 09:18:35 +00:00
Steve Lewis
9082c793cc Install and configure uWSGI
When an Apache + mod_wsgi configuration is not selected, configure
the two Keystone services with uWSGI service profiles.

Two arbitrary ports are selected for uWSGI to listen on, so that it
may be proxied for by a dedicated web server. This is in preparation
for laying down Nginx in a future patch.

Notify events are updated to restart the Keystone uWSGI services
where Keystone's configuration is modified only. Because federation
concerns will be isolated within the dedicated web server, changes
to federation configuration of Shiboleth do not trigger restarts of
uWSGI. Similarly, SSL certificate changes do not trigger restarts.

Change-Id: I99e16a999c496e68fb25fa2630d9b211c9755ea4
Related: blueprint keystone-uwsgi
2016-08-22 11:04:55 -07:00
Steve Lewis
b1c2f9c00f Isolate Apache components
In preparation for moving keystone to uwsgi under nginx the Apache
related components are being isolated so a branched install option
(keystone_apache_mod_wsgi_enabled) can be used to accommodate both
deployment models.

Change-Id: Idd9de25d1906dba526b5761ad1a8f75b672af29d
Related: blueprint keystone-uwsgi
2016-08-22 17:42:54 +00:00
Jenkins
e43e15960e Merge "Force a restart of all the apache nodes during upgrade" 2016-08-22 10:37:44 +00:00
Jesse Pretorius
39faeb0718 Make all linting tests use upper-constraints
As per [1] all linting tests can now use upper-constraints. This patch
removes all instances of the install_command override relating to lint
testing which were needed to negate the use of upper-constraints.

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-August/101474.html

Change-Id: I51a1d62524c2ec88b31a8ca2ed9c9acbe5790f75
2016-08-20 16:06:02 +01:00
Jenkins
0315649648 Merge "Adding Vagrantfile for local testing/dev" 2016-08-20 14:46:11 +00:00
Travis Truman
9fd8ff0579 Adding Vagrantfile for local testing/dev
Running `vagrant up` will launch an Ubuntu Trusty
box and run all of the gate checks

Change-Id: Ie48bb936a1c7f5fbc7b1f8a2968632d11f479564
2016-08-19 15:31:51 -04:00
Qin Wang
3122ff649d Add SNI support via OS packages for os_keystone
Some Linux distributions, such as CentOS 7 and Xenial, have trouble
validating SSL certificates when using get_url with servers
that use Server Name Indication (SNI).

This patch adds those packages to the list of required packages and
uses bindep to install them in developer test environments the same
way that the gate tests install them.

Change-Id: Ifdee90709330e189165f2fade67ae1f9289b6981
2016-08-19 16:55:18 +00:00
Michael Gugino
50730dafe1 Add apt-get update to run_tests
Change-Id: I19b820e5518d6fe1bca9087336a43e8397b2560f
2016-08-18 16:37:56 -04:00