Course outline▾
Week 1 · Bare-Metal Infrastructure & Advanced Containerization
- Day 1The AI Infrastructure Shift — Why the Model is Only 10% of the Production System
- Day 2Advanced Docker for AI — Multi-Stage Builds for Heavy Python Runtimes
- Day 3Hardware Acceleration in Docker — GPU Passthrough and Limits for Constrained Hardware
- Day 4Optimizing Model Weights — Quantization Basics (GGUF, AWQ) for Efficient Inference
- Day 5Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution
- Day 6Containerizing AI Microservices — Building and Packaging APIs with FastAPI
- Day 7Docker Compose — Architecting a Multi-Container Database and Inference Stack Locally
Week 2 · Kubernetes Foundation for MLOps
- Day 8K3d and Lightweight Kubernetes — Running Multi-Node Clusters on 16 GB of RAM
- Day 9Kubernetes Core for Machine Learning — Pods, Deployments and Headless Services
- Day 10Persistent Volumes (PV/PVC) — Managing Heavy Model Weights and Vector Stores
- Day 11StatefulSets for Vector Databases — Deploying Milvus or Qdrant in Kubernetes
- Day 12Auto-Scaling AI Inference: Using Horizontal Pod Autoscalers (HPA) for Spiky API Traffic
- Day 13Helm Charts: Packaging, Versioning, and Updating AI Deployments
- Day 14Week 2 Capstone: Deploying a Highly Available Local AI Application on K3d
Week 3 · High-Performance Networking & Security Mesh
- Day 15AI Network Bottlenecks: Transitioning from iptables to eBPF
- Day 16Cilium & eBPF: Sub-Millisecond Routing for Heavy AI Microservices
- Day 17Service Mesh 101: Installing Istio over Your Local Cluster
- Day 18Envoy Proxy as an AI Gateway: Load Balancing and Terminating LLM Prompts
- Day 19Zero-Trust mTLS: Encrypting Traffic Between Autonomous Agents
- Day 20API Rate Limiting & Egress Control: Preventing Agents from Exfiltrating Data
- Day 21Distributed Tracing: Debugging Agent Workflows with Jaeger and OpenTelemetry
Week 4 · Enterprise Agents & CI/CD Workflows
- Day 22Introduction to MLOps: Bridging AI Development with Continuous Integration
- Day 23GitOps with ArgoCD: Automated, Self-Healing AI Deployments
- Day 24Building Ephemeral AI Sandboxes: Secure Execution Environments for Agent Code
- Day 25Multi-Agent Orchestration Frameworks: Exploring Mastra, LangGraph, and CrewAI
- Day 26Tool-Calling in Production: Connecting Agents Securely to Internal Databases
- Day 27MLOps Guardrails: Evaluating Agents and Preventing Hallucinations in Real-Time
- Day 28Continuous Training Pipelines: Connecting Data Ingestion to Automated Model Retraining
- Day 29Managing Technical Debt and Infrastructure Drift in Enterprise AI
- Day 30The AI-Native Platform: Scaling from a Sandbox Environment to Global Cloud Architecture
Day 11: StatefulSets for Vector Databases — Deploying Milvus or Qdrant in Kubernetes
2026-11-17 · 8 min read
A crashed stateless pod comes back with a random name and a new IP, and nobody minds. A database node that returns with a different identity can break cluster consensus and force expensive re-sharding. A StatefulSet fixes that with stable network identities, ordered operations and automatic per-pod storage.
Identity and storage
Each pod gets a predictable DNS name, such as qdrant-0.qdrant-headless.ai-platform.svc.cluster.local, plus its own PVC. Pods start in order, qdrant-0 then qdrant-1, and the name survives a restart.
Why Deployments fail databases
- Identity instability. Random names and IPs after a restart force the cluster to re-sync large indexes.
- Storage contention. A Deployment shares one claim among its pods. A StatefulSet uses
volumeClaimTemplatesto create a dedicated volume per replica. - Uncontrolled termination. A Deployment can stop many pods at once. A StatefulSet stops them in strict reverse order,
qdrant-2, thenqdrant-1, thenqdrant-0, which helps the cluster keep quorum.
The manifest
This runs Qdrant in clustered mode. Qdrant needs each new peer to know where to bootstrap from, so the first pod starts the cluster and the others join it.
apiVersion: v1
kind: Service
metadata:
name: qdrant-headless
namespace: ai-platform
spec:
clusterIP: None
selector: { app: qdrant }
ports:
- { name: http, port: 6333 }
- { name: internal-p2p, port: 6335 }
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: qdrant
namespace: ai-platform
spec:
serviceName: qdrant-headless
replicas: 3
podManagementPolicy: OrderedReady
selector:
matchLabels: { app: qdrant }
template:
metadata:
labels: { app: qdrant }
spec:
containers:
- name: qdrant
image: qdrant/qdrant:v1.9.0
env:
- { name: QDRANT__CLUSTER__ENABLED, value: "true" }
- { name: QDRANT__CLUSTER__P2P__PORT, value: "6335" }
command: ["/bin/sh", "-c"]
args:
- |
HOST="$(hostname).qdrant-headless"
if [ "$(hostname)" = "qdrant-0" ]; then
./qdrant --uri "http://${HOST}:6335"
else
./qdrant --bootstrap "http://qdrant-0.qdrant-headless:6335" --uri "http://${HOST}:6335"
fi
ports:
- { name: http, containerPort: 6333 }
- { name: p2p, containerPort: 6335 }
resources:
requests: { cpu: "1000m", memory: 2Gi }
limits: { cpu: "2000m", memory: 4Gi }
volumeMounts:
- { name: qdrant-data, mountPath: /qdrant/storage }
volumeClaimTemplates:
- metadata: { name: qdrant-data }
spec:
accessModes: [ReadWriteOnce]
resources:
requests: { storage: 20Gi }
Check the bootstrap flags against the Qdrant version you run, because clustering options have changed between releases.
How many replicas?
Raft-based clusters need a majority of nodes to agree. One and two nodes tolerate no failures, three tolerate one, and five tolerate two. Two replicas are fine for a lab, but for real fault tolerance run three.
Proper StatefulSet design protects your vector indexes through restarts and upgrades, and it gives the rest of the stack a database that behaves predictably.
Run it yourself
This is the exact example from the video. Run against a real K3d cluster. A small nginx image stands in for Qdrant so three pods start quickly.
{apiVersion: v1, kind: Service, metadata: {name: qdrant-headless},
spec: {clusterIP: None, selector: {app: qdrant}}}
---
apiVersion: apps/v1
kind: StatefulSet
metadata: {name: qdrant}
spec:
serviceName: qdrant-headless
replicas: 3
selector: {matchLabels: {app: qdrant}}
template:
metadata: {labels: {app: qdrant}}
spec: {containers: [{name: qdrant, image: "nginx:1.27-alpine"}]}
volumeClaimTemplates:
- metadata: {name: data}
spec: {accessModes: [ReadWriteOnce],
resources: {requests: {storage: 1Gi}}}
Run it:
kubectl apply -f statefulset.yaml
Output I got while recording the video:
qdrant-0 Running
qdrant-1 Running
qdrant-2 Running
data-qdrant-0 Bound
data-qdrant-1 Bound
data-qdrant-2 Bound
headless clusterIP: None
Coming Up Next
Day 12: Auto-scaling AI inference with Horizontal Pod Autoscalers for spiky API traffic.
#StatefulSet #VectorDatabase #Qdrant #Milvus #CloudNative #DataEngineering #SystemDesign