As part of the DevOps cycle, our team run test per commit and per Merge Request, to catch any bug early in development. We heavily depend on GitLab’s Review Apps, and the Kubernetes integration, for the DAST tests.

Because we use a microservice architecture on our software, and our deploys are on Kubernetes, we decided to use ambassador as the API gateway. At this point, we found an issue.

Disclaimer: We are NOT using AutoDevOps. We have scripts to prepare and deploy and stop review apps resources, including ambassador Hosts and Mapping.

The problem

If you read the Review App’s docs you can see that GitLab creates a new Kubernetes Namespace where our review app will live. Inside this namespace, it creates a new Service Account with a RoleBinding using admin as roleRef (you can find this in this file).

This Service Account has admin rights inside this new namespace, but only over the default Kubernetes resources. So when our review app needs to create some Host or Mapping resources it crashes with:

1
2
 User "<service-account>" cannot get resource "mappings" in API group "getambassador.io" in the namespace "<namespace>"
 User "<service-account>" cannot get resource "hosts" in API group "getambassador.io" in the namespace "<namespace>"

Let’s be sure that the service account can’t create these resources:

1
2
3
4
$ kubectl auth can-i create host \
  -n gitlab-review-app-namespace \
  --as=system:serviceaccount:gitlab-review-app-namespace:gitlab-review-app-service-account
no

Well, we can’t…

gitlab-review-app-namespace and gitlab-review-app-service-account are just placeholders. Check what values are correct in your use case

The solution

Even if you can edit the admin ClusterRole without errors, it won’t be applied. In this case, you can create a new ClusterRole and indicate that it should be aggregated to admin.

Saving this definition in ambassador_cluster_role.yml:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
 name: aggregate-ambassador-admin
 labels:
   # Add these permissions to the "admin" default roles.
   rbac.authorization.k8s.io/aggregate-to-admin: "true"
rules:
- apiGroups:
 - getambassador.io
 resources:
 - mappings
 - hosts
 verbs:
 - create
 - delete
 - deletecollection
 - patch
 - update
 - get
 - list
 - watch

We can apply on the cluster with kubectl apply -f ambassador_cluster_role.yml

Let’s check if this work:

1
2
3
4
$ kubectl auth can-i create host \
  -n gitlab-review-app-namespace \
  --as=system:serviceaccount:gitlab-review-app-namespace:gitlab-review-app-service-account
yes

Problem solved! And now, our Review Apps can work with ambassador