Day 17: Containerizing AI Workloads — Production Dockerfiles for Python APIs

2026-10-17 · 7 min read

Watch the video lesson, or subscribe on YouTube for a new lesson every day.

Moving a model from a notebook to a service means putting it in a container. But AI images are not like a Node.js or Go service. Deep learning frameworks bring heavy CUDA shared libraries, C++ build tools and multi-gigabyte wheel files, and a careless Dockerfile can balloon past 10 GB.

In plain terms: Imagine shipping a finished chair. A naive Dockerfile mails the chair together with the whole workshop: saws, sawdust and offcuts. A multi-stage build ships only the chair.

A naive Dockerfile that installs everything in one image ends up carrying compilers and build caches along with the large CUDA and PyTorch libraries. A multi-stage build ships only the runtime pieces. The bar widths are illustrative.

The anatomy of a bloated image

A naive Dockerfile that runs RUN pip install torch transformers fastapi in a default base image causes three operational problems.

A naive AI image duplicates build caches inside its layers, leaves compilers in production which widens the attack surface, and is so large that new pods start slowly when traffic spikes.

Layer duplication. Build caches and downloaded wheels stay trapped in read-only image layers, which slows registry pushes and pulls.
Security exposure. Leaving gcc, g++ and make in the production image widens the attack surface.
Slow deployments. Huge images saturate the network during autoscaling, so new pods start late exactly when traffic spikes.

Three production patterns

Multi-stage builds. Split the Dockerfile in two. The builder stage has the compilers; it downloads and builds dependencies into a virtual environment at /opt/venv. The runtime stage starts from a clean, minimal base such as python:3.11-slim and copies in only that environment.

In stage one, the builder stage has compilers and installs every dependency into a virtual environment. In stage two, a clean slim runtime image receives only that virtual environment, and everything else from stage one is discarded.

Wheel caching. A BuildKit cache mount (--mount=type=cache,target=/root/.cache/pip) keeps pip's download cache outside the image, so repeated CI builds do not download the heavy packages again.

Non-root security. Never run the process as root. Create a dedicated user and start the ASGI server as that user.

A production-ready AI image uses a multi-stage build, caches pip downloads with a BuildKit cache mount, runs the server as a non-root user and pins its dependency versions, so builds are reproducible and auditable.

Here is a complete example for a FastAPI model server. It assumes a requirements.txt with pinned versions and an app/ folder with a main.py, and it is a template to adapt rather than a tuned production file. A GPU image would use a CUDA-enabled base and PyTorch build instead.

# syntax=docker/dockerfile:1

# ---- Stage 1: builder (has compilers, gets thrown away) ----
FROM python:3.11-slim AS builder
RUN python -m venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
COPY requirements.txt .
RUN --mount=type=cache,target=/root/.cache/pip \
    pip install -r requirements.txt

# ---- Stage 2: runtime (clean and minimal) ----
FROM python:3.11-slim
COPY --from=builder /opt/venv /opt/venv
ENV PATH="/opt/venv/bin:$PATH"
RUN useradd -m -u 1000 appuser
WORKDIR /app
COPY --chown=appuser:appuser app/ ./app/
USER appuser
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Build it with DOCKER_BUILDKIT=1 docker build -t model-api .. Writing lean, deterministic Dockerfiles turns chaotic research dependencies into reproducible, auditable artifacts that a cluster scheduler can start quickly.

Coming Up Next

Day 18: Docker Compose for multi-container AI stacks.

#Docker #Containerization #DevOps #AIEngineering #FastAPI