179 Commits

Author SHA1 Message Date
rajesh.kudaka
490dd63c2c Enable probes config for etcd
This commit enables configuration of probes
for etcd pod by manipulating/overriding values in
values.yaml or through manifests.

Change-Id: I69eabd13f8ea8b97a33281ad993ec2e88b9280bc
2019-08-09 09:28:47 +00:00
Zuul
3155a3824d Merge "(haproxy) Add rationality check to config" 2019-07-23 21:59:53 +00:00
rajesh.kudaka
856b6132fa Add deploy promenade gate check
Change-Id: I47b448234f95b157e55bf649310d1ebd71abc7c0
2019-06-11 04:53:20 -05:00
Hussey, Scott (sh8121)
41e21e1a6e (haproxy) Add rationality check to config
- When the anchor provides a new haproxy config file
  to the running haproxy, add a reasonable check that
  the new config is valid:
    - Is it a valid config file per haproxy
    - Does it contain the expected number of frontends

- Update helm version for linting to 2.14.1

Change-Id: I7a49deb372831c44f05c7baa870735c515519cb2
2019-06-10 11:01:13 -05:00
Zuul
aea0c9d1e9 Merge "templates: separate genesis and join sources" 2019-06-06 19:56:26 +00:00
Egorov, Stanislav (se6518)
955deeda41 New source for hyperkube binary definition
Now it's possible to use hyperkube Docker image to extract hyperkube binary.
Use case for this feature is kubelet/kubectl delivery in one binary(hyperkube)
which is built into Docker image. Promenade will extract hyperkube from Docker image,
create symlinks for kubelet/kubectl pointed to hyperkube. To do so promenade container
need to be configured to use Docker on the host where this container will be created.
This is happening only for script generation for genesis node. Later when promenade
will be started as a service pod inside ucp cluster it will generate scripts for joining nodes
by using cached hyperkube from /tmp.

Old way to delivery kubelet from tarball is still supported.

Configuration for the new method.

Need to export environment variables to properly configure Docker in Docker.
Docker socket should be provided as a mounted file inside promenade.
Also need to set temporary permissions for this socket during the build scripts stage.

Example:
DOCKER_SOCK="/var/run/docker.sock"
sudo chmod o+rw $DOCKER_SOCK
export DOCKER_HOST="unix:/${DOCKER_SOCK}"
export PROMENADE_TMP="abs_path_tmp_dir_on_host"
export PROMENADE_TMP_LOCAL="tmp_dir_inside_container"

After genesis scripts generation Docker socket permission should be turned back:
sudo chmod o-rw $DOCKER_SOCK

Change-Id: Ida22ea934fc551fec34df162d8147c8b9e630330
2019-06-06 10:30:29 -07:00
Drew Walters
8748348b96 templates: separate genesis and join sources
Currently, the package, repository, and key lists are used by up.sh for
genesis and join. This is not desirable when using an in-cluster
mirroring service, as the service address may change after it has been
deployed.

This commit separates the sources for genesis and join to circumvent the
aforementioned pain point. A 'common' entry in the
'promenade/HostSystem/v1' document can be used if a common source for
genesis and join is desired.

Co-authored-by: Rick Bartra <rb560u@att.com>
Change-Id: Ieb2513da0cff587297cfcbf5629d908696349621
2019-05-24 17:32:55 -04:00
Sean Eagan
184d114062 Move to tiller 2.14.0
This version fixes manifest validation [0], so a couple invalid
manifests are fixed in this patchset as well.

[0]: 32d7f1a3fc

Change-Id: I0cbdf21cf016271bef2d8a541687ce3ab28081ce
2019-05-20 14:17:46 -05:00
Drew Walters
93a839f588 CI: Add chart build job for latest Helm toolkit
This commit introduces a non-voting job to lint Helm charts against the
latest version of Helm toolkit from OpenStack-Helm Infra. This job
should serve as an indicator of when it's safe to advance the version of
Helm toolkit used by Airship.

Additionally, this commit modifies all Helm chart lint jobs to run on
each commit, regardless of the files modified by a change. This should
not introduce a noticeable difference in CI runtime, as these jobs
execute quicker than the tox jobs.

Change-Id: I0b38ef3388629b88ea6d6f88eabdc2d7f5f69a3b
2019-05-07 20:03:22 +00:00
Zuul
105fa608d7 Merge "Add optional external_ip to promjoin" 2019-04-25 21:25:48 +00:00
Zuul
66caaf74f7 Merge "proxy ENV variables to promenade container" 2019-04-24 18:12:28 +00:00
Zuul
2741ea1f90 Merge "Use apps/v1 k8s controllers and add labels" 2019-04-24 13:08:37 +00:00
Sean Eagan
2e2a30515c Use apps/v1 k8s controllers and add labels
Daemonset update strategy defaults to OnDelete in v1beta1, whereas
it defaults to RollingUpdate in v1, which seems prefereable.

This also adds helm-toolkit based labels at the controller level
to match standard usage such as for example by armada as wait labels.

This change has been tested using the promenade resiliency gate.

Change-Id: I9fd1bc4caedc0a6717b779e5333640ca8dc78b7e
2019-04-23 09:24:53 -05:00
Matt McEuen
c18e3a8d9d Add optional external_ip to promjoin
Adds an optional external_ip parameter to the prom join script API,
and to the Genesis and KubernetesNode schema.
This is used to populate the host's IP address in its /etc/hosts
file if present, according to normal hosts conventions.
If the value is not passed to prom-join or is absent from a
Genesis or KubernetesNode document, then the hosts file defaults
to the current loopback IP for the hostname (business as usual).

Change-Id: I58dc219923b18aaf9c83453b896ce509664d8766
2019-04-23 08:45:34 -05:00
Zuul
9f6bed21ac Merge "Allow certgen independent of bundle creation" 2019-04-15 18:35:19 +00:00
Zuul
690d9e878e Merge "Change image pull policy from Always to IfNotPresent." 2019-04-12 21:18:59 +00:00
Anderson, Craig (ca846m)
f4d991f010 Allow certgen independent of bundle creation
Additionally, fix indentation.

Change-Id: I598167a2b73f1376fae2f5fb813fd1b10647892e
2019-04-11 10:56:57 -07:00
Scott Hussey
66ab47386f (zuul) Fix image publish job
- Fix issue in post pipeline image publish job introduced
  by Ansible update.

Change-Id: I23d8621901e225fb7b134b276156f248720b248f
2019-04-03 14:54:48 -05:00
kranthikirang
be175e2c3a proxy ENV variables to promenade container
Change-Id: I0cb95aac7bdb6cfffd0b188a70c3d86bf764002a
Signed-off-by: kranthikirang <kranthi.guttikonda@att.com>
2019-04-01 14:21:52 -04:00
Drew Walters
c178634b89 tools: Update Helm to v2.13.1
Helm v2.13.1 has been released [0]. This change uplifts Helm to v2.13.1
and restores usage of the latest Helm toolkit to match other Airship
projects.

[0] https://github.com/helm/helm/releases/tag/v2.13.1

Depends-On: https://review.openstack.org/647812
Change-Id: I4299a8736e0cedf7b15950614a4d72fcfd893a2d
2019-03-28 15:22:05 +00:00
Matt McEuen
7c9b2253f1 Updating Docker Gate use of zuul.newrev
- Zuul updated ansible to 2.7, no longer uses missing variables.
- Using an if to try and address.

Based on Aaron Sheffield's PS for Pegleg:
https://review.openstack.org/#/c/645631/

Change-Id: I6cf52a2b9c804c29cc727ad60d45c05a8450c9e9
2019-03-22 12:07:48 -05:00
Sean Eagan
70d7315b25 Pin helm-toolkit to fix resiliency gate
See [0] for why this is needed.

[0]: https://review.openstack.org/#/c/642825/

Change-Id: I9808f22813f1c797d78d43169bf0f41d1c1578ab
2019-03-13 12:16:25 -05:00
Sean Eagan
c3d4a464cb build: support helm-toolkit pinning
This pins the version of helm-toolkit used in the build due to
a breaking change in helm-toolkit.

Once helm 2.13.1 is released with this fix [0] we can update to use
it, and update the helm-toolkit pin to "master".

[0]: https://github.com/helm/helm/pull/5411

Change-Id: I827f1145c6d670cc955b82155455455a4f3ca486
2019-03-12 11:58:24 -05:00
Matt McEuen
e4cab73d0f Update to Kubernetes 1.11.6
This change updates the following components in the Promenade charts,
docs, and example bootstrap configuration:
  Kubernetes 1.10.11 -> 1.11.6
  CoreDNS 1.1.2 -> 1.1.3 (per k8s 1.11 recommendations)
  Etcd 3.2.14 -> 3.2.18 (per k8s 1.11 recommendations)
  Tiller 2.10.0 -> 2.12.1 (per Helm k8s support)

This change has been tested by the Promenade resiliency gate.

Change-Id: Ia70de212dd2d50c6638578b92c750a4d5c791229
2019-02-05 17:29:59 -06:00
leledashenqi
c09ca80bf0 Change image pull policy from Always to IfNotPresent.
No need to repeat pulling image when container recreate.

Change-Id: I6b3b1e3b963093644faa88ba084138684d78cfab
Story:2004662
Task:28629
2018-12-21 15:01:15 +08:00
Mark Burnett
cdd1a6bd28 Update Kubernetes to 1.10.11
Change-Id: If1479f7a5d0a8ea459eed39172a0bc1f89935e36
2018-12-18 11:32:28 -06:00
Zuul
20f27f6281 Merge "Fix: git commit id labels on images" 2018-10-16 11:51:26 +00:00
Kaspars Skels
6b0d8b7d10 Add /opt resources to cleanup.sh
Change-Id: I8795032932167f7b09ad9162fc2b9f8b86a7eb9a
2018-10-05 20:49:56 -05:00
Zuul
dcd847b35b Merge "[gate] Mask AVX2 to prevent 16.04 on 16.04 VMs boot crashes" 2018-09-28 14:26:36 +00:00
Roman Gorshunov
f44b3cc384 Fix: various documentation and URL fixes
1) UCP -> Airship
2) readthedocs.org -> readthedocs.io (there is redirect)
3) http -> https
4) attcomdev -> airshipit (repo on quay.io)
5) att-comdev -> openstack/airship-* (repo on github/openstack git)
6) many URLs have been verified and adjusted to be current
7) no need for 'en/latest/' path in URL of the RTD
8) added more info to some setup.cfg and setup.py files
9) ucp-integration docs are now in airship-in-a-bottle
10) various other minor fixes

Change-Id: I55f546b8ad22bf2e3097ff831d021ed5a35059de
2018-09-25 11:44:57 +02:00
Chris Wedgwood
9b4814ad0c [gate] Mask AVX2 to prevent 16.04 on 16.04 VMs boot crashes
16.04 on 16.04 VMs will crash (illegal instruction in
raid6_avx21_gen_syndrome) on boot if the host has AVX2 and we pass
that through.

The issue seems to be the guest kernel sees presence of AVX2 so raid6
module makes use of it - though it's not enabled (different bits).

Until this is resolved in the host hypervisor (kvm) and/or guest
kernel, mask out AVX2 to prevent crashes in L1/L2 VMs.

Change-Id: I0ca8edb8f62f9f2e96aa5e265bac631c346d0eac
2018-09-21 22:16:56 +00:00
Roman Gorshunov
02c5f2943e Fix: git commit id labels on images
1) Use OCI Image Specs for labels instead of custom 'commit-id=xxxxx'
   or legacy "Label Schema"
2) Fix missing git commit id labels on images (.revision)
3) Add human-readable title (.title) of the image, URL (.url), and
   a few other properties (annotations) according to the latest Specs
4) Unify docker-image-build.yaml playbook with other Airship-*
   components

Change-Id: I89afed3bf6a1f9fa92391d605bb6b3c871e58126
2018-09-21 03:31:13 +02:00
Mark Burnett
8bc8c7c028 Implement encryption for genesis/join scripts
This introduces a new document called `EncryptionPolicy` to configure
this behavior.  It currently only supports using symmetric encryption
with `GPG`, but that should be available on all Ubuntu systems (which is
what we currently support) and should also be fairly reliable.

Change-Id: I06d4faa119b736773df0d8cbf0e7a23fd98edcdf
Depends-On: https://review.openstack.org/#/c/602175/
2018-09-14 11:32:12 -05:00
Mark Burnett
3ac95edc60 Fix: re-add incorrectly removed format test
This was removed in Iccf6228ab9e6d621d3047994b3adc192d67273c9 but should
not have been as it has allowed for code format drift.

This also

* Pins the version of yapf to 0.24.0
* Fixes some drift
* Updates formatting to the version of yapf being used

Change-Id: Ie3d9fd6344a29d8ddb76a36d4a31d001a4c8b7c6
2018-09-12 10:40:45 -05:00
Zuul
6e4ea38955 Merge "Enable gate shell linting" 2018-09-10 17:32:00 +00:00
Mark Burnett
0233c30ffb Update tiller version to 2.10.0
This also makes a corresponding update to the Makefile to address a bug
with which $(HELM) is being used that was exposed during local testing.

Change-Id: I08da45c1f232960c58ab482053befed83da6fdd6
2018-08-30 15:54:07 -05:00
Mark Burnett
ee10b894bf Enable gate shell linting
This enables linting the various supporting shell scripts for the gate
during the unit test job.

Change-Id: I95dc098a602b2ae6807165d3d08c1cb57ba0c702
2018-08-29 08:13:59 -05:00
Mark Burnett
d6b0eeb733 Remove version pinning from examples
These pinned versions are frequently removed from Ubuntu's repositories,
leading to broken tests.  Removing them will expose us to breakage from
unexpected updates, but will avoid issues with removed packages.

Change-Id: I30bf993fa20f0054ee77eb47086a784f6d02d2ac
2018-08-24 11:06:14 -05:00
Gage Hugo
f60f6d7a85 Consolidate pep8/bandit zuul gating
This change adds the global zuul pep8 tox job, which runs both
bandit and pep8 using tox. This also removes the two other airship
specific lint-pep8 and bandit zuul jobs since they are both covered
by the default openstack global one.

Also cleaned up the tox.ini by moving the requirements into the
test-requirements.txt file.

Change-Id: Iccf6228ab9e6d621d3047994b3adc192d67273c9
2018-08-21 12:57:02 -05:00
Zuul
9917eb5bfb Merge "Allow multiple join IPs" 2018-08-13 21:17:31 +00:00
Zuul
290448fe83 Merge "Adding node-labels api" 2018-08-09 21:31:56 +00:00
pallav
ea5de25b1a Adding node-labels api
Blueprint: https://review.openstack.org/#/c/583343

1. Added node-labels api for managing node labels in
   kubernetes cluster
2. Added unit test cases
3. Updated documents
4. Resiliency gate script update

Change-Id: Iebd49706b3fdbb3650f2e46c5a7fbd21d236b906
2018-08-09 23:58:59 +05:30
Zuul
7a06bef72c Merge "Update to Helm 2.9.1" 2018-08-07 18:01:33 +00:00
Matt McEuen
ba3aacf6a9 Update to Helm 2.9.1
Update all Helm and Tiller references to 2.9.1, which has fixes
for template rendering bugs. Some references are already updated,
and this PS updates the rest.

Change-Id: Id617cc866d09f5cb3708fcf329a21bc60e17ce45
2018-08-05 14:58:54 -05:00
Mark Burnett
8d3e44c7d6 Allow multiple join IPs
This provides more robustness in testing and removes a nosec.

Additionally, commit 5a8b1d8 introduced a random failure in the
resiliency gate, due to there being a chance to choose the intentionally
downed node for the join ip.

Change-Id: I77b410b8e51f9d41eca2be4f5f770694140733b4
2018-08-02 15:46:49 -05:00
Zuul
37ebe7ba11 Merge "Use Docker-CE for building images in the resil gate" 2018-08-02 20:02:40 +00:00
Aaron Sheffield
e1c6a62cee Added storagePolicy for HAProxy
- Added storagePolicy in bootstrap-armada-config.yaml for haproxy.

Change-Id: I89c250deea7a5626904b01425f7c59c24fc9e23d
2018-08-01 10:16:11 -05:00
Mark Burnett
3d685501db Use Docker-CE for building images in the resil gate
Change-Id: I707b698608a2711e807c75410eaa3969bfa04683
2018-07-26 10:50:28 -05:00
Zuul
3a9d3b4839 Merge "Update Dockerfile to allow override of FROM variable" 2018-07-26 15:23:09 +00:00
Jerome Brette
5232d17a2a Update Dockerfile to allow override of FROM variable
l is to let user customize the base image of the component
by passing FROM=myimage during the build process. This would let any
project leveraging Airship ensure that the base image is matching the
security requirements for that project and still use the same Dockerfile.
This will also ease the control of the /etc/apt/source.list
and thereby the result of apt-get update/upgrade procedure.
2. The above goal is achievable by using docker-ce feature such as:
ARG FROM="defaultbaseimage:xx"
FROM ${FROM}
For this reason, the installation of docker.io in the Zuul gating is beeing
replaced by docker-ce.
3. Third Goal is to bring consistency with the other compoenents leveraging
Helm such as the openstack-helm and potentially use bindep the same way
the LOCI images are to ensure
4. The new syntax in the Dockerfile is still commented out until the associated
image builder have been updated to use docker-ce as they have been for the LOCI
images.

Change-Id: Ie5ae836221dc3cb9bdafc6e5e6670f914d3d1bb4
2018-07-24 21:11:35 +00:00