kindnetd: WaitForCacheSync before reconcile loop to avoid 10s NodeReady lag - #4216

Problem

On fresh kind clusters, node NodeReady=true flips ~10 seconds later than necessary.
During those 10 seconds, kubelet's runtime status stays at KubeletNotReady with reason
NetworkPluginNotReady even though CNI configuration and kindnet's control loop are ready
to run.

Root cause

images/kindnetd/cmd/kindnetd/main.go starts informers but never waits for their cache to
sync before entering the reconcile ticker loop:

informersFactory.Start(ctx.Done())
ticker := time.NewTicker(10 * time.Second)
defer ticker.Stop()

for {
    nodes, err := nodeLister.List(labels.Everything())  // empty on 1st iteration
    ...
    reconcileNodes(nodes)  // no-op, nothing to reconcile
    select {
    case <-ticker.C:  // waits a full 10 seconds
    }
}

The informer's initial LIST typically completes within a few hundred milliseconds of
Start, but the first loop iteration runs synchronously right after Start returns —
before the LIST response has landed. nodeLister.List(...) returns an empty slice,
reconcileNodes finds nothing to do, and the loop blocks on the 10-second ticker before
trying again. The first meaningful reconcile — the one that writes
/etc/cni/net.d/10-kindnet.conflist and logs Handling node with IPs — is thus gated
behind the ticker rather than behind the (fast) cache-sync.

Fix

Wait for the node informer cache to sync before entering the reconcile loop.

 import (
     ...
     "k8s.io/client-go/informers"
     "k8s.io/client-go/kubernetes"
     "k8s.io/client-go/rest"
+    "k8s.io/client-go/tools/cache"
     "k8s.io/klog/v2"
 )

     // main control loop
     informersFactory.Start(ctx.Done())
+    // Wait for the node cache to populate before entering the reconcile
+    // loop. Without this, the first iteration lists an empty cache and
+    // blocks on the 10s ticker before doing any real work — delaying
+    // NodeReady by a full tick.
+    if !cache.WaitForCacheSync(ctx.Done(), nodeInformer.Informer().HasSynced) {
+        klog.Fatalf("failed to sync node informer cache")
+    }
     ticker := time.NewTicker(10 * time.Second)
     defer ticker.Stop()

Evidence

Kindnetd container logs on a fresh single-node kind cluster (v1.33.7, arm64). Same host,
same node image, same cluster — only the kindnetd binary differed between runs.

Before:

17:54:23.430  probe TCP address ...:6443
17:54:23.432  connected to apiserver
17:54:23.697  Starting controller: kube-network-policies
17:54:23.697  Waiting for informer caches to sync
17:54:23.927  Caches are synced
17:54:23.928  Synchronized state with the runtime
17:54:33.642  Handling node with IPs: map[172.19.0.2:{}]   <-- 10s later

After (patched binary):

20:13:12.921  connected to apiserver
20:13:13.112  Starting controller: kube-network-policies
20:13:13.112  Waiting for the policy engine to become ready...
20:13:13.413  Handling node with IPs: map[172.19.0.2:{}]   <-- 492ms after connect
20:13:13.616  Policy engine is ready

Reduction in kindnetd's own timeline in the observed environment: ~9.5 seconds. The
theoretical maximum savings on any bring-up is one full reconcile ticker period (10s).

Downstream impact (motivation)

Consumers of kind that fan out CNI-dependent workloads during cluster bring-up
(cert-manager, service meshes, trust-bundle mounts) frequently observe pods stuck in
ContainerCreating with FailedMount events, retrying against kubelet's
nestedpendingoperations exponential backoff (500ms → 1s → 2s → 4s → 8s → 16s → ...). The
10s NodeReady lag pushes these pods into the 8-16s backoff tier before the resources they
depend on become available; removing the lag lets those pods succeed on their first mount
attempt.

Risk

• WaitForCacheSync returns false only on ctx cancellation.
• klog.Fatalf for the sync-fail branch matches the idiom used across
  kubelet/controller-manager and flushes klog buffers before exiting; the existing panic on
  line 286 could be aligned separately if desired.
• No change to steady-state behavior — the 10-second reconcile ticker is untouched. This
  only affects the first iteration after startup.
• The pod, namespace, and network-policy informers are still started concurrently and
  continue to sync in the background. kube-network-policies already does its own
  WaitForCacheSync for what it needs. Happy to extend the sync-wait to all four informers
  if reviewers prefer uniformity.

Testing

• Verified manually by rebuilding kindnetd, overlaying the binary onto
  docker.io/kindest/kindnetd:v20251212-v0.29.0-alpha-105-g20ccfc88, retagging inside the
  kind node's containerd, restarting the DaemonSet pod, and comparing pre-/post-patch log
  timestamps for Handling node with IPs (see logs above).
• The change is 3 lines using a well-established client-go idiom (WaitForCacheSync
  following informersFactory.Start); regression risk is bounded to the first-iteration
  ordering.

This PR was drafted with AI assistance; code, logs, and testing are my own.