In default chart configuration, pods fail to start on GKE cluster due to quota violation #1120

What steps did you take and what happened:

• Spin up a GKE cluster with Kubernetes 1.19.7-gke.800 (currently only available on the rapid channel IIUC).
• Run helm upgrade --install --atomic gatekeeper gatekeeper/gatekeeper --namespace kube-system --version 3.3.0

What did you expect to happen:
Expected gatekeeper-related pods to start up. Instead, I got the following events in the gatekeeper-system namespace:

0s          Warning   FailedCreate        replicaset/gatekeeper-controller-manager-64664bc54c   Error creating: insufficient quota to match these scopes: [{PriorityClass In [system-node-critical system-cluster-critical]}]
0s          Warning   FailedCreate        replicaset/gatekeeper-audit-565fdfc544                Error creating: insufficient quota to match these scopes: [{PriorityClass In [system-node-critical system-cluster-critical]}]

Setting controllerManager.priorityClassName and audit.priorityClassName to "" "fixes" the issue, but is sub-optimal because it disables the priority classes.

Anything else you would like to add:
From some digging, it seems that GKE adds a ResourceQuota in the kube-system namespace with the following spec:

spec:
  hard:
    pods: 1G
  scopeSelector:
    matchExpressions:
    - operator: In
      scopeName: PriorityClass
      values:
      - system-node-critical
      - system-cluster-critical

I think a similar ResourceQuota would have to be made in the gatekeeper-system namespace for pods to start correctly.

An easy fix would be to optionally skip namespace creation in the helm chart, with some documentation about adding it separately (along with a ResourceQuota). Another option might be to optionally create the ResourceQuota in the chart.

Environment:

• Gatekeeper version: 3.3.0
• Kubernetes version: (use kubectl version): 1.19.7