airship/promenade
Code Issues Proposed changes

Remove Tiller

Browse Source 
Depends-On: https://review.opendev.org/c/airship/armada/+/812047
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: Ic1eddda3639d629a94bb39f93bf48da54445469f
This commit is contained in:
Sean Eagan 2021-07-26 10:01:37 -05:00 committed by Sean Eagan
parent 1017536b9f
commit 53d0ecb7f9
19 changed files with 2 additions and 317 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

35
doc/source/configuration/genesis.rst
View File

@ -28,10 +28,6 @@ Here is a complete sample document:
metrics:
output_dir: /var/log/armada/metrics
max_attempts: 5
tiller:
listen: 24134
probe_listen: 24135
storage: secret
labels:
static:
- calico-etcd=enabled
@ -45,8 +41,6 @@ Here is a complete sample document:
- ucp-control-plane=enabled
images:
armada: quay.io/airshipit/armada:latest
helm:
tiller: ghcr.io/helm/tiller:v2.17.0
kubernetes:
apiserver: k8s.gcr.io/kube-apiserver-amd64:v1.20.5
controller-manager: k8s.gcr.io/kube-controller-manager-amd64:v1.20.5
@ -86,30 +80,6 @@ Configuration for Armada bootstrap metric collection.
| | | Can be set to 0 to disable metrics collection. |
+-----------------+----------+---------------------------------------------------------------------------------------+
Tiller
------
Configuration options for bootstrapping with Tiller.
+-----------------+----------+---------------------------------------------------------------------------------------+
| keyword | type | action |
+=================+==========+=======================================================================================+
| storage | string | (optional, not passed by default) The tiller `storage`_ arg to use. ` |
+-----------------+----------+---------------------------------------------------------------------------------------+
| listen | integer | (optional, default `24134`) The tiller `listen` arg to use. See `Ports`_. |
+-----------------+----------+---------------------------------------------------------------------------------------+
| probe_listen | integer | (optional, default `24135`) The tiller `probe_listen` arg to use. See `Ports`_. |
+-----------------+----------+---------------------------------------------------------------------------------------+
Ports
^^^^^
By default, promenade uses tiller ports outside of `net.ipv4.ip_local_port_range` to
avoid conflicts with apiserver connections to etcd, see `example`_.
The `listen` and `probe_listen` parameters allow setting these back to the
upstream tiller defaults (or any other value) if desired.
Bootstrapping Images
--------------------
@ -118,13 +88,8 @@ Bootstrapping images are specified in the top level key ``images``:
.. code-block:: yaml
armada: <Armada image for bootstrapping>
helm:
tiller: <Tiller image for bootstrapping>
kubernetes:
apiserver: <API server image for bootstrapping>
controller-manager: <Controller Manager image for bootstrapping>
etcd: <etcd image for bootstrapping>
scheduler: <Scheduler image for bootstrapping>
.. _storage: https://helm.sh/docs/using_helm/#tiller-s-release-information
.. _example: https://helm.sh/docs/developing_charts/#chart-dependencies

1
doc/source/design.rst
View File

@ -31,7 +31,6 @@ documents:
* ``scheduler``
* Etcd_ for use by the Kubernetes_ ``apiserver``
* Helm_'s server process ``tiller``
* CoreDNS_ to be used for Kubernetes_ ``apiserver`` discovery
With these components up, it is possible to leverage Armada_ to deploy Helm_

2
doc/source/troubleshooting/genesis.rst
View File

@ -35,7 +35,7 @@ provisioning of other containers. For example:
.. code-block:: console
CRITICAL armada [-] Unhandled error: armada.exceptions.tiller_exceptions.ReleaseException: Failed to Install release: barbican
CRITICAL armada [-] Unhandled error: armada.exceptions.helm_exceptions.HelmCommandException: Failed to Install release: barbican
Use ``kubectl logs`` on the failed pod to determine the reason for the failure.
E.g.:

4
examples/basic/Genesis.yaml
View File

@ -29,8 +29,6 @@ data:
- --v=3
armada:
target_manifest: cluster-bootstrap
tiller:
storage: secret
etcd:
auxiliary_threshold: 3
labels:
@ -47,8 +45,6 @@ data:
run_as_user: 65534
images:
armada: quay.io/airshipit/armada:master-ubuntu_bionic
helm:
tiller: ghcr.io/helm/tiller:v2.17.0
kubernetes:
apiserver: k8s.gcr.io/kube-apiserver-amd64:v1.20.5
controller-manager: k8s.gcr.io/kube-controller-manager-amd64:v1.20.5

34
examples/basic/armada-resources.yaml
View File

@ -75,7 +75,6 @@ data:
- kubernetes-apiserver
- kubernetes-controller-manager
- kubernetes-scheduler
- tiller
---
schema: armada/ChartGroup/v1
metadata:
@ -1126,39 +1125,6 @@ data:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: tiller
layeringDefinition:
abstract: false
layer: site
storagePolicy: cleartext
data:
chart_name: tiller
release: tiller
namespace: kube-system
install:
no_hooks: false
upgrade:
no_hooks: false
wait:
timeout: 600
values:
images:
tags:
tiller: ghcr.io/helm/tiller:v2.17.0
labels:
node_selector_key: ucp-control-plane
node_selector_value: enabled
source:
type: git
location: https://opendev.org/airship/armada
subpath: charts/tiller
reference: master
dependencies:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: promenade

4
examples/complete/Genesis.yaml
View File

@ -13,8 +13,6 @@ data:
external_ip: 192.168.77.10
armada:
target_manifest: cluster-bootstrap
tiller:
storage: secret
etcd:
auxiliary_threshold: 3
labels:
@ -36,8 +34,6 @@ data:
run_as_user: 65534
images:
armada: quay.io/airshipit/armada:master-ubuntu_bionic
helm:
tiller: ghcr.io/helm/tiller:v2.17.0
kubernetes:
apiserver: k8s.gcr.io/kube-apiserver-amd64:v1.20.5
controller-manager: k8s.gcr.io/kube-controller-manager-amd64:v1.20.5

35
examples/complete/armada-resources.yaml
View File

@ -112,7 +112,6 @@ data:
- ucp-rabbitmq
- ucp-barbican
- ingress
- tiller
---
schema: armada/ChartGroup/v1
metadata:
@ -1678,33 +1677,6 @@ data:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: tiller
layeringDefinition:
abstract: false
layer: site
storagePolicy: cleartext
data:
chart_name: tiller
release: tiller
namespace: kube-system
values:
images:
tags:
tiller: ghcr.io/helm/tiller:v2.17.0
labels:
node_selector_key: ucp-control-plane
node_selector_value: enabled
source:
type: git
location: https://opendev.org/airship/armada
subpath: charts/tiller
reference: master
dependencies:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: deckhand
@ -1855,13 +1827,6 @@ data:
release: armada
namespace: ucp
values:
manifests:
deployment_tiller: false
service_tiller_deploy: false
conf:
armada:
DEFAULT:
tiller_namespace: kube-system
images:
tags:
api: quay.io/airshipit/armada:master-ubuntu_bionic

4
examples/containerd/Genesis.yaml
View File

@ -29,8 +29,6 @@ data:
- --v=3
armada:
target_manifest: cluster-bootstrap
tiller:
storage: secret
etcd:
auxiliary_threshold: 3
labels:
@ -47,8 +45,6 @@ data:
run_as_user: 65534
images:
armada: quay.io/airshipit/armada:master-ubuntu_bionic
helm:
tiller: ghcr.io/helm/tiller:v2.17.0
kubernetes:
apiserver: k8s.gcr.io/kube-apiserver-amd64:v1.20.5
controller-manager: k8s.gcr.io/kube-controller-manager-amd64:v1.20.5

34
examples/containerd/armada-resources.yaml
View File

@ -75,7 +75,6 @@ data:
- kubernetes-apiserver
- kubernetes-controller-manager
- kubernetes-scheduler
- tiller
---
schema: armada/ChartGroup/v1
metadata:
@ -914,39 +913,6 @@ data:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: tiller
layeringDefinition:
abstract: false
layer: site
storagePolicy: cleartext
data:
chart_name: tiller
release: tiller
namespace: kube-system
install:
no_hooks: false
upgrade:
no_hooks: false
wait:
timeout: 600
values:
images:
tags:
tiller: ghcr.io/helm/tiller:v2.17.0
labels:
node_selector_key: ucp-control-plane
node_selector_value: enabled
source:
type: git
location: https://opendev.org/airship/armada.git
subpath: charts/tiller
reference: master
dependencies:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: promenade

4
examples/gate/Genesis.yaml
View File

@ -29,8 +29,6 @@ data:
- --v=3
armada:
target_manifest: cluster-bootstrap
tiller:
storage: secret
etcd:
auxiliary_threshold: 3
labels:
@ -47,8 +45,6 @@ data:
run_as_user: 65534
images:
armada: quay.io/airshipit/armada:master-ubuntu_bionic
helm:
tiller: ghcr.io/helm/tiller:v2.17.0
kubernetes:
apiserver: k8s.gcr.io/kube-apiserver-amd64:v1.20.5
controller-manager: k8s.gcr.io/kube-controller-manager-amd64:v1.20.5

34
examples/gate/armada-resources.yaml
View File

@ -75,7 +75,6 @@ data:
- kubernetes-apiserver
- kubernetes-controller-manager
- kubernetes-scheduler
- tiller
---
schema: armada/ChartGroup/v1
metadata:
@ -920,39 +919,6 @@ data:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: tiller
layeringDefinition:
abstract: false
layer: site
storagePolicy: cleartext
data:
chart_name: tiller
release: tiller
namespace: kube-system
install:
no_hooks: false
upgrade:
no_hooks: false
wait:
timeout: 600
values:
images:
tags:
tiller: ghcr.io/helm/tiller:v2.17.0
labels:
node_selector_key: ucp-control-plane
node_selector_value: enabled
source:
type: git
location: https://opendev.org/airship/armada.git
subpath: charts/tiller
reference: master
dependencies:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: promenade

19
promenade/schemas/Genesis.yaml
View File

@ -112,16 +112,6 @@ data:
auxiliary_threshold:
type: integer
additionalProperties: false
tiller:
type: object
properties:
listen:
type: integer
probe_listen:
type: integer
storage:
type: string
additionalProperties: false
files:
type: array
@ -164,14 +154,6 @@ data:
properties:
armada:
$ref: '#/definitions/image'
helm:
type: object
properties:
tiller:
$ref: '#/definitions/image'
required:
- tiller
additionalProperties: false
kubernetes:
type: object
properties:
@ -191,7 +173,6 @@ data:
additionalProperties: false
required:
- armada
- helm
- kubernetes
additionalProperties: false

63
promenade/templates/roles/genesis/etc/kubernetes/manifests/bootstrap-armada.yaml
View File

@ -1,10 +1,3 @@
{# By default, use tiller ports outside of `net.ipv4.ip_local_port_range` to
avoid conflicts with apiserver connections to etcd, see [0].
[0]: https://github.com/helm/helm/issues/4886
#}
{% set tiller_listen = config['Genesis:tiller.listen'] | default(24134) %}
{% set tiller_probe_listen = config['Genesis:tiller.probe_listen'] | default(24135) %}
{% set tiller_storage = config.get_path('Genesis:tiller.storage') %}
---
apiVersion: v1
kind: Pod
@ -13,63 +6,11 @@ metadata:
namespace: kube-system
labels:
application: promenade
component: genesis-tiller
component: genesis
spec:
dnsPolicy: Default
hostNetwork: true
containers:
- env:
- name: TILLER_NAMESPACE
value: kube-system
- name: KUBECONFIG
value: /etc/kubernetes/admin/config
image: {{ config['Genesis:images.helm.tiller'] }}
command:
- /tiller
{%- if tiller_storage %}
- "--storage={{ tiller_storage }}"
{%- endif %}
- -listen
- ":{{ tiller_listen }}"
- -probe-listen
- ":{{ tiller_probe_listen }}"
- -logtostderr
- -v
- "5"
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /liveness
port: {{ tiller_probe_listen }}
scheme: HTTP
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
name: tiller
ports:
- containerPort: {{ tiller_listen }}
name: tiller
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /readiness
port: {{ tiller_probe_listen }}
scheme: HTTP
initialDelaySeconds: 1
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 1
resources: {}
securityContext:
runAsUser: 0
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- name: auth
mountPath: /etc/kubernetes/admin
- name: armada
image: {{ config['Genesis:images.armada'] }}
securityContext:
@ -87,8 +28,6 @@ spec:
if armada \
apply \
--target-manifest {{ config.get_path('Genesis:armada.target_manifest', 'cluster-bootstrap') }} \
--tiller-host 127.0.0.1 \
--tiller-port {{ tiller_listen }} \
$([[ $attempt -le $ARMADA_METRICS_MAX_ATTEMPTS ]] && echo --metrics-output "${ARMADA_METRICS_OUTPUT_DIR}/armada-bootstrap-${attempt}.prom") \
/etc/genesis/armada/assets/manifest.yaml &>> "${ARMADA_LOGFILE}"; then
break

3
tests/unit/api/test_validatedesign.py
View File

@ -102,9 +102,6 @@ VALID_DOCS = [
'n0',
'images': {
'armada': 'quay.io/airshipit/armada:master-ubuntu_bionic',
'helm': {
'tiller': 'ghcr.io/helm/tiller:v2.17.0'
},
'kubernetes': {
'apiserver':
'k8s.gcr.io/kube-apiserver-amd64:v1.20.5',

4
tests/unit/builder_data/simple/Genesis.yaml
View File

@ -19,8 +19,6 @@ data:
- --endpoint-reconciler-type=lease
armada:
target_manifest: cluster-bootstrap
tiller:
storage: secret
labels:
dynamic:
- calico-etcd=enabled
@ -33,8 +31,6 @@ data:
- ucp-control-plane=enabled
images:
armada: quay.io/airshipit/armada:master-ubuntu_bionic
helm:
tiller: ghcr.io/helm/tiller:v2.17.0
kubernetes:
apiserver: k8s.gcr.io/kube-apiserver-amd64:v1.20.5
controller-manager: k8s.gcr.io/kube-controller-manager-amd64:v1.20.5

35
tests/unit/builder_data/simple/armada-resources.yaml
View File

@ -75,7 +75,6 @@ data:
- kubernetes-apiserver
- kubernetes-controller-manager
- kubernetes-scheduler
- tiller
---
schema: armada/ChartGroup/v1
metadata:
@ -963,40 +962,6 @@ data:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: tiller
layeringDefinition:
abstract: false
layer: site
storagePolicy: cleartext
data:
chart_name: tiller
release: tiller
namespace: kube-system
install:
no_hooks: false
upgrade:
no_hooks: false
timeout: 600
wait:
timeout: 600
values:
images:
tags:
tiller: ghcr.io/helm/tiller:v2.17.0
labels:
node_selector_key: ucp-control-plane
node_selector_value: enabled
source:
type: git
location: https://opendev.org/airship/armada
subpath: charts/tiller
reference: master
dependencies:
- helm-toolkit
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: promenade

2
tools/gate/config-templates/genesis-config.yaml
View File

@ -31,8 +31,6 @@ data:
- ucp-control-plane=enabled
images:
armada: ${IMAGE_ARMADA}
helm:
tiller: ${IMAGE_TILLER}
kubernetes:
apiserver: ${IMAGE_APISERVER}
controller-manager: ${IMAGE_CONTROLLER_MANAGER}

1
tools/gate/default-config-env
View File

@ -13,5 +13,4 @@ IMAGE_CONTROLLER_MANAGER=k8s.gcr.io/kube-controller-manager-amd64:v1.20.5
IMAGE_SCHEDULER=k8s.gcr.io/kube-scheduler-amd64:v1.20.5
IMAGE_PROXY=k8s.gcr.io/kube-proxy-amd64:v1.20.5
IMAGE_ANCHOR=cwedgwood/kubectl:v1.20.5-1
IMAGE_TILLER=ghcr.io/helm/tiller:v2.17.0
KUBELET_URL=https://dl.k8s.io/v1.20.5/kubernetes-node-linux-amd64.tar.gz

1
tools/registry/IMAGES
View File

@ -8,7 +8,6 @@ k8s.gcr.io/kube-proxy-amd64,v1.20.5,proxy
k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64,1.14.4,k8s-dns-dnsmasq-nanny-amd64
k8s.gcr.io/k8s-dns-kube-dns-amd64,1.14.4,k8s-dns-kube-dns-amd64
k8s.gcr.io/k8s-dns-sidecar-amd64,1.14.4,k8s-dns-sidecar-amd64
gcr.io/kubernetes-helm/tiller,v2.14.0,tiller
lachlanevenson/k8s-helm,v3.6.3,helm
quay.io/airshipit/armada,master,armada
quay.io/calico/cni,v1.11.0,calico-cni