Course outline▾
Week 1 · Bare-Metal Infrastructure & Advanced Containerization
- Day 1The AI Infrastructure Shift — Why the Model is Only 10% of the Production System
- Day 2Advanced Docker for AI — Multi-Stage Builds for Heavy Python Runtimes
- Day 3Hardware Acceleration in Docker — GPU Passthrough and Limits for Constrained Hardware
- Day 4Optimizing Model Weights — Quantization Basics (GGUF, AWQ) for Efficient Inference
- Day 5Local AI Runtimes — Deploying Ollama and vLLM for Bare-Metal Execution
- Day 6Containerizing AI Microservices — Building and Packaging APIs with FastAPI
- Day 7Docker Compose — Architecting a Multi-Container Database and Inference Stack Locally
Week 2 · Kubernetes Foundation for MLOps
- Day 8K3d and Lightweight Kubernetes — Running Multi-Node Clusters on 16 GB of RAM
- Day 9Kubernetes Core for Machine Learning — Pods, Deployments and Headless Services
- Day 10Persistent Volumes (PV/PVC) — Managing Heavy Model Weights and Vector Stores
- Day 11StatefulSets for Vector Databases — Deploying Milvus or Qdrant in Kubernetes
- Day 12Auto-Scaling AI Inference — Using Horizontal Pod Autoscalers for Spiky API Traffic
- Day 13Helm Charts — Packaging, Versioning and Updating AI Deployments
- Day 14Week 2 Capstone — Deploying a Highly Available Local AI Application on K3d
Week 3 · High-Performance Networking & Security Mesh
- Day 15AI Network Bottlenecks: Transitioning from iptables to eBPF
- Day 16Cilium & eBPF: Sub-Millisecond Routing for Heavy AI Microservices
- Day 17Service Mesh 101: Installing Istio over Your Local Cluster
- Day 18Envoy Proxy as an AI Gateway: Load Balancing and Terminating LLM Prompts
- Day 19Zero-Trust mTLS: Encrypting Traffic Between Autonomous Agents
- Day 20API Rate Limiting & Egress Control: Preventing Agents from Exfiltrating Data
- Day 21Distributed Tracing: Debugging Agent Workflows with Jaeger and OpenTelemetry
Week 4 · Enterprise Agents & CI/CD Workflows
- Day 22Introduction to MLOps: Bridging AI Development with Continuous Integration
- Day 23GitOps with ArgoCD: Automated, Self-Healing AI Deployments
- Day 24Building Ephemeral AI Sandboxes: Secure Execution Environments for Agent Code
- Day 25Multi-Agent Orchestration Frameworks: Exploring Mastra, LangGraph, and CrewAI
- Day 26Tool-Calling in Production: Connecting Agents Securely to Internal Databases
- Day 27MLOps Guardrails: Evaluating Agents and Preventing Hallucinations in Real-Time
- Day 28Continuous Training Pipelines: Connecting Data Ingestion to Automated Model Retraining
- Day 29Managing Technical Debt and Infrastructure Drift in Enterprise AI
- Day 30The AI-Native Platform: Scaling from a Sandbox Environment to Global Cloud Architecture
Day 14: Week 2 Capstone — Deploying a Highly Available Local AI Application on K3d
2026-11-20 · 9 min read
This capstone ties Week 2 together. You will run a small but realistic platform on one machine: a multi-node cluster, persistent storage, a clustered vector database, scalable inference and a Helm release, then break it on purpose to see it heal.
The architecture
Traffic enters through the k3d proxy on port 80 and reaches an ingress controller. The ingress sends it to the ai-gateway-service, a ClusterIP in front of a FastAPI orchestrator. The orchestrator calls the Qdrant StatefulSet (with its headless service and a dedicated PVC per pod) for embeddings, and the vLLM Deployment (with a ReadOnlyMany shared-weights volume) for tokens.
One choice to reconcile: this diagram uses an ingress controller, so do not disable Traefik for the capstone. Day 8 disabled it to make room for Envoy later, but here we use K3s's built-in Traefik so the stack works end to end.
The steps
# 1. Create the cluster (keep the built-in Traefik this time)
mkdir -p ~/ai-storage/qdrant ~/ai-storage/models
k3d cluster create ai-capstone --servers 1 --agents 2 \
-p "80:80@loadbalancer" \
--volume ~/ai-storage/models:/opt/ai/models@agent:*
# 2. Install the chart into its own namespace
helm upgrade --install enterprise-ai ./helm-ai-stack \
--set global.namespace=ai-system --create-namespace \
-f ./helm-ai-stack/values-staging.yaml
# 3. Check where everything landed
kubectl get nodes -o wide
kubectl get pods -n ai-system -o wide
kubectl get pvc -n ai-system
A note on permissions. You will often see sudo chmod -R 777 in tutorials to get a volume working. That is a lab shortcut that makes a folder writable by everyone, and you should not carry it into production. Prefer creating the folders as your own user, as above, or set the right owner and a tight mode such as 755.
If your machine has no GPU, replace the vLLM image in the chart with an Ollama-based one. The manifests keep the same shape, and you can still exercise every Kubernetes concept in the lesson.
Break it on purpose
Fault injection is how you learn whether the design holds.
kubectl delete pod -n ai-system -l app=local-ai-llm --now
kubectl get pods -n ai-system -w
The Deployment controller notices the missing pod and creates a replacement at once. Because the weights live on a ReadOnlyMany volume that is already on the nodes, the new pod does not re-download anything. It passes its startup probe as soon as the model has loaded, which is much quicker than a fresh download.
Verify the whole system
- Nodes show
Ready. - Pods are spread across the agents, with Qdrant and vLLM on different nodes.
- PVCs show
Bound. - Qdrant peers can see each other (check the cluster endpoint on port 6333).
- The HPA can read its metric.
- Deleting a pod recovers, and you can time how long that takes.
That completes Week 2. You now have a cluster you can create in seconds, storage that outlives pods, a database that keeps its identity, inference that scales on the right signal and a Helm chart that ships it all. Week 3 turns to the network and security layer: eBPF, Cilium and service meshes.
Run it yourself
This is the exact example from the video. Run against a real K3d cluster. A small nginx image stands in for the model server, and the video deletes one of the two pods and counts what comes back.
apiVersion: apps/v1
kind: Deployment
metadata: {name: llm}
spec:
replicas: 2
selector: {matchLabels: {app: llm}}
template:
metadata: {labels: {app: llm}}
spec:
containers:
- name: llm
image: nginx:1.27-alpine
readinessProbe:
httpGet: {path: /, port: 80}
periodSeconds: 1
Run it:
kubectl apply -f deploy.yaml
kubectl delete pod <one of the two pods>
kubectl get pods
Output I got while recording the video:
before: 2/2 ready
deleted: 1 pod
after: 2/2 ready
new pods created: 1
deleted pod gone: yes
Coming Up Next
Day 15: AI network bottlenecks: moving from iptables to eBPF.
#Kubernetes #MLOps #SystemDesign #CapStone #CloudNative #DevOps #DataScience #AIEngineering