gke: add resource quotas for Cilium Namespace - #13878
Conversation
|
test-gke |
|
Excellent, thanks! 🙌 Related #13806 |
pchaigno
left a comment
There was a problem hiding this comment.
We still create the cilium namespace in test/gke/select-cluster.sh. Not sure if we can get rid of that yet.
|
FYI; for context: The addon manager has an additional behavior that is unexpected - it also reads the kubernetes.io/cluster-service label, and if true, it behaves exactly as though the deployment had a label of addonmanager.kuberntes.io/mode=Reconcile To avoid this behavior, there are a couple of label combinations the resource can have -
This behavior of addonmanager has been slated for deprecation and removal, but it has not happened yet - |
In GKE environments, if Cilium is deployed in 'kube-system' namespace with the 'kubernetes.io/cluster-service: "true"' labels, GKE will delete the DaemonSet within 1 minutes [1]. To avoid this problem these labels are no longer installed for new installations, however, if the user tries to install this combination of options, we should fail the installation and warn the user about the correct usage. [1] kubernetes/kubernetes#51376 Signed-off-by: André Martins <andre@cilium.io>
d615c42 to
86addba
Compare
|
test-gke |
86addba to
4885c2b
Compare
|
test-gke |
pchaigno
left a comment
There was a problem hiding this comment.
👍 only for my codeowners (docs-structure) since I don't think I'm qualified to review the rest.
There was a problem hiding this comment.
Just out of curiosity, where can I find more details about this number - 5k nodes from scalability perspective?
There was a problem hiding this comment.
There was a problem hiding this comment.
Surely we should allow users to customize this via Helm?
|
test-me-please |
joestringer
left a comment
There was a problem hiding this comment.
Minor followup item, I don't expect many users asking for this since the limits should suffice for most scenarios but it's always possible.
There was a problem hiding this comment.
Surely we should allow users to customize this via Helm?
When deploying Cilium in its own namespace, it's required to define resource quotas. For now we will create a ResourceQuota for 10k pods that are node-critical and 15 pods that are cluster-critical. Signed-off-by: André Martins <andre@cilium.io>
09b71ee to
f166217
Compare
|
test-gke |
Support for ResourceQuotas (specifically for GKE) was added in #13878. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
Support for ResourceQuotas (specifically for GKE) was added in #13878. Signed-off-by: Sebastian Wicki <sebastian@isovalent.com>
When deploying Cilium in its own namespace, it's required to define
resource quotas. For now we will create a ResourceQuota for 10k pods
that are node-critical and 15 pods that are cluster-critical.
Also, add checkers in helm if users try to install Cilium in
kube-systemnamespace with the deprecated labels.
Fixes #13852