Get a model running on Kubernetes in minutes.

Dynamo's production path is Kubernetes-native: you install the platform with Helm, submit Dynamo custom resources, and let the operator reconcile inference graphs into pods, services, routing, model-loading, and scaling resources. The local and container guides remain useful for development, but Kubernetes is the canonical path for shared GPU clusters and multi-node serving.

• Helm (v3.0+) installed

• Kubernetes cluster (v1.30+) with GPU nodes
• kubectl (v1.30+)
• NVIDIA GPU Operator installed on the cluster

• Kubernetes cluster (v1.34+) with XPU nodes and Dynamic Resource Allocation (DRA) API v1 enabled
• kubectl matching your Kubernetes minor version (v1.34+ for these DRA API v1 templates)
• Intel resource drivers for Kubernetes installed with a gpu.intel.com DeviceClass
• vLLM XPU runtime image built from this repository

Create a HuggingFace token secret for model downloads. If you don't have a token, see the HuggingFace token guide.

export HF_TOKEN=<your-hf-token>

kubectl create secret generic hf-token-secret \
  --from-literal=HF_TOKEN="$HF_TOKEN"

If you don't have the GPU Operator yet:

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia --force-update
helm repo update nvidia
helm install gpu-operator nvidia/gpu-operator \
  --namespace gpu-operator --create-namespace \
  --wait --timeout=600s

--set driver.enabled=false --set toolkit.enabled=false

Install the Intel resource drivers for Kubernetes by following the Intel resource drivers for Kubernetes instructions.

Verify that the cluster exposes the DRA resources used by the XPU templates:

kubectl get deviceclasses
kubectl get resourceslices

The vLLM XPU templates expect a gpu.intel.com DeviceClass.

CUDA deployments require the GPU Operator. XPU deployments require the Intel resource drivers for Kubernetes and DRA. For additional features like RDMA, Prometheus, or multinode scheduling with Grove/KAI Scheduler, see the Installation Guide.

Optionally, verify your cluster is ready:

./deploy/pre-deployment/pre-deployment-check.sh

export NAMESPACE=dynamo-system
helm install dynamo-platform \
  oci://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-platform \
  --version "1.2.1" \
  --namespace "$NAMESPACE" \
  --create-namespace

Wait for the platform pods:

kubectl get pods -n $NAMESPACE
# Expected: dynamo-operator-*, etcd-*, nats-* pods all Running

Before applying the first YAML, it helps to know the Kubernetes resources Dynamo uses. These are Dynamo's native control-plane objects; you describe the inference graph, and the operator owns the Kubernetes deployments, services, and component rollout around it:

flowchart LR
    DGDR["DGDR<br/>generator/profiler"] --> DGD["DGD<br/>live deployment"]
    DGD --> DCD["DCDs<br/>component deployments"]
    DCD --> Pods["Pods and Services"]

This quickstart uses DGDR because it avoids hand-writing the first DGD. After DGDR generates and applies the DGD, the DGDR reaches a terminal state, similar to a Kubernetes Job. The DGD persists and serves your model.

DGDR can also carry supported generated-deployment features such as features.planner for Planner configuration and features.mocker for mocker mode. KV-aware routing is not currently exposed as a DGDR feature field; use a direct DGD, a tuned recipe, or overrides.dgd when you need to set router mode or other graph-level details explicitly.

For tuned production-style manifests, start from Dynamo recipes. For the full deployment model, see the Deployment Overview.

Save this DGDR to generate and deploy a DGD for Qwen/Qwen3-0.6B:

# qwen3-quickstart.yaml
apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeploymentRequest
metadata:
  name: qwen3-quickstart
spec:
  model: Qwen/Qwen3-0.6B
  backend: auto
  image: "nvcr.io/nvidia/ai-dynamo/dynamo-planner:1.2.1"  # dynamo-frontend for Dynamo < 1.1.0

The DGDR generates a DGD similar in shape to the following. If you already know the backend and runtime image you want, you can apply this canonical DGD object directly instead of using DGDR:

# qwen3-dgd.yaml
apiVersion: nvidia.com/v1beta1
kind: DynamoGraphDeployment
metadata:
  name: qwen3-direct
spec:
  components:
    - name: Frontend
      type: frontend
      replicas: 1
      podTemplate:
        spec:
          containers:
            - name: main
              image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.2.1
              envFrom:
                - secretRef:
                    name: hf-token-secret
    - name: VllmDecodeWorker
      type: worker
      replicas: 1
      podTemplate:
        spec:
          containers:
            - name: main
              image: nvcr.io/nvidia/ai-dynamo/vllm-runtime:1.2.1
              command:
                - python3
                - -m
                - dynamo.vllm
              args:
                - --model
                - Qwen/Qwen3-0.6B
              envFrom:
                - secretRef:
                    name: hf-token-secret
              resources:
                limits:
                  nvidia.com/gpu: "1"
                requests:
                  ephemeral-storage: 2Gi
              workingDir: /workspace/examples/backends/vllm

Apply exactly one of the manifests.

Option A: generate and apply a DGD with DGDR.

kubectl apply -f qwen3-quickstart.yaml -n $NAMESPACE

Option B: apply the DGD directly.

kubectl apply -f qwen3-dgd.yaml -n $NAMESPACE

If you use DGDR, watch it progress from Pending to Profiling to Deploying to Deployed:

kubectl get dgdr qwen3-quickstart -n $NAMESPACE -w

Build and push a vLLM XPU runtime image:

container/render.py --framework=vllm --device=xpu --target=runtime --output-short-filename
docker build -t <registry>/vllm-runtime-xpu:<tag> \
  -f container/rendered.Dockerfile .
docker push <registry>/vllm-runtime-xpu:<tag>

Use the DRA-based vLLM XPU templates from the repository:

cd <dynamo-source-root>/examples/backends/vllm/deploy/v1beta1/xpu

Update the template image fields to use <registry>/vllm-runtime-xpu:<tag>, then apply one template:

# Aggregated deployment
kubectl apply -f agg_xpu_dra.yaml -n $NAMESPACE

# Or disaggregated deployment
kubectl apply -f disagg_xpu_dra.yaml -n $NAMESPACE

Verify DRA allocation:

kubectl get resourceclaim -n $NAMESPACE
kubectl get resourceslices

In both paths, the DGD is the live serving resource:

kubectl get dynamographdeployment -n $NAMESPACE
kubectl get dynamocomponentdeployment -n $NAMESPACE

Once the DGD is ready, it is serving the model:

# Find and port-forward the frontend
FRONTEND_SVC=$(kubectl get svc -n $NAMESPACE -o name | grep frontend | head -1)
kubectl port-forward "$FRONTEND_SVC" 8000:8000 -n $NAMESPACE &

# Send a request
curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-0.6B",
    "messages": [{"role": "user", "content": "What is NVIDIA Dynamo?"}],
    "max_tokens": 200
  }' | python3 -m json.tool

kubectl delete dgdr qwen3-quickstart -n $NAMESPACE --ignore-not-found
kubectl delete dynamographdeployment qwen3-quickstart qwen3-direct \
  -n $NAMESPACE --ignore-not-found

• Installation Guide — Cloud provider setup, accelerator resource components, and optional components (Grove, RDMA, model caching, Prometheus)
• Deployment Overview — DGD, DCD, DGDR, recipes, strategy selection, and common pitfalls
• DGDR Reference — Spec reference, lifecycle phases, monitoring commands, and generated DGD behavior
• Creating Deployments — Hand-craft a DGD spec for full control