Docker Quest

Every developer has heard it. Every ops engineer has dreaded it. "It works on my machine" is the universal admission that environments are broken.

The problem is dependency hell: your app depends on Python 3.12, but staging has 3.10. Your Node.js modules install differently on macOS vs Linux. OS-level libraries like libssl or libpq have different versions across machines.

Environment drift between dev, staging, and production means hours spent debugging issues that have nothing to do with your actual code. Every team member has a slightly different setup. New hire onboarding takes days of "getting the environment right."

The cost? Engineers spend hours debugging environment issues instead of building features. Deployments fail for reasons unrelated to code changes. Hot fixes get delayed because "it doesn't reproduce locally."

Containers solve this by packaging your application with its entire environment — same OS, same libraries, same everything — everywhere it runs.

Virtual Machines solve environment isolation by running a full operating system copy per instance. A hypervisor (VMware, VirtualBox, KVM) sits between hardware and VMs, each with its own kernel, init system, and userland.

Containers take a different approach: they share the host's kernel and isolate at the process level. No hypervisor, no guest OS — just your application and its dependencies in an isolated namespace.

The trade-off: VMs provide stronger isolation (separate kernels), while containers are faster and more resource-efficient. For most application workloads, container-level isolation is sufficient.

Docker isn't magic — it's a user-friendly wrapper around Linux kernel features that have existed for years:

Linux Namespaces provide isolation. Each container gets its own view of:

• PID — Process IDs (container sees its own process tree)
• NET — Network stack (own IP, ports, routing)
• MNT — Filesystem mounts
• UTS — Hostname
• IPC — Inter-process communication
• USER — User/group IDs

cgroups (Control Groups) enforce resource limits — CPU, memory, I/O per container. A runaway container can't starve the host.

Union Filesystems (OverlayFS) provide a layered file system. Images are built from read-only layers stacked on top of each other, with a thin writable layer for the running container. This makes image storage and distribution incredibly efficient.

Together: isolated, resource-limited processes sharing the host kernel.

This is the most fundamental concept in Docker:

• Image = blueprint / class = a read-only template containing your application + all its dependencies
• Container = running instance / object = a writable layer on top of an image, executing your code

You can create multiple containers from the same image, each with its own state — just like creating multiple objects from the same class.

# One image, many containers
docker run -d --name web1 nginx
docker run -d --name web2 nginx
docker run -d --name web3 nginx
# Three independent containers from the same nginx image

Images are immutable. When you "change" an image, you're actually creating a new image with new layers. Containers add a thin writable layer on top for runtime changes (logs, temp files), but that layer is ephemeral — it vanishes when the container is removed.

When you type docker run nginx, here's what actually happens:

┌─────────────┐    ┌──────────────┐    ┌────────────┐    ┌────────┐
│ Docker CLI   │───▶│ Docker Daemon │───▶│ containerd  │───▶│  runc   │
│ (docker)     │    │ (dockerd)     │    │             │    │         │
└─────────────┘    └──────────────┘    └────────────┘    └────────┘
                          │
                    ┌─────┴─────┐
                    │ REST API   │
                    └───────────┘

• Docker CLI (docker) — the command-line client you interact with
• Docker Daemon (dockerd) — background service that exposes a REST API, manages images, networking, and volumes
• containerd — container lifecycle manager (start, stop, pause, delete)
• runc — the low-level OCI-compliant runtime that actually creates containers using Linux primitives (namespaces + cgroups)

This layered architecture means you can swap components. Kubernetes uses containerd directly (without dockerd), and runc can be replaced with alternative runtimes.

The Open Container Initiative (OCI) defines two critical standards:

• Image Specification — how container images are structured (manifest, config, layers)
• Runtime Specification — how containers are created and run

This means images built with Docker run on Podman, containerd, CRI-O, and any other OCI-compliant runtime. You're not locked into Docker.

An OCI image consists of:

• Manifest — metadata listing the layers and config
• Config — runtime settings (CMD, ENV, exposed ports)
• Layers — filesystem diffs stacked to form the complete filesystem

This portability is what makes containers a true industry standard — build once, run on any OCI-compliant platform, on any cloud.

Docker Engine runs natively on Linux — containers share the host kernel directly. This is the fastest, most efficient setup.

Docker Desktop is the macOS/Windows experience. Since containers need a Linux kernel, Docker Desktop runs a lightweight Linux VM under the hood (HyperKit on older macOS, Apple Virtualization on newer macOS, WSL2 on Windows).

Licensing: Docker Desktop is free for personal use, education, and small businesses (<250 employees, <$10M revenue). Larger enterprises need a paid subscription.

Alternatives:

• OrbStack — fastest on macOS, lower resource usage
• Colima — CLI-focused, free and open-source
• Rancher Desktop — GUI + built-in Kubernetes
• Podman Desktop — daemonless alternative

# macOS: Install with Homebrew
brew install --cask docker        # Docker Desktop
# OR
brew install colima docker        # Colima + Docker CLI
colima start                      # Start the VM

# Linux: Install Docker Engine directly
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER     # Add user to docker group

Docker is the most popular container platform, but it's part of a larger ecosystem:

• Docker — the most popular container platform, great developer experience
• Podman — daemonless, rootless alternative (drop-in Docker CLI replacement)
• containerd — the runtime under Docker AND Kubernetes
• Kubernetes (K8s) — container orchestration at scale (NOT needed for most projects)
• Docker Compose — multi-container orchestration for local dev and small deployments

When NOT to use Docker:

• Simple one-off scripts that don't need environment isolation
• macOS filesystem-heavy workloads (VM overhead causes I/O slowdown)
• When the complexity cost outweighs the isolation benefit

For most web applications and services, Docker is the right choice. Start with Docker, add Compose when you have multiple services, and reach for Kubernetes only when you truly need multi-machine orchestration.

Images are stored in registries (Docker Hub by default). docker pull downloads an image to your local machine.

# Pull an image from Docker Hub
docker pull nginx              # Pulls nginx:latest (DON'T rely on :latest in production!)
docker pull nginx:1.25-alpine  # Pull specific version + variant
docker pull python:3.12-slim   # Slim variant — smaller than full

# Pull by digest (immutable — guaranteed same image)
docker pull nginx@sha256:abc123...

The :latest pitfall: "latest" is just a tag name — it's NOT necessarily the most recent version. It's mutable: the image it points to can change at any time. Always use specific version tags in production for reproducible builds.

docker run is the most important command — it creates and starts a container from an image.

# Basic run (foreground, blocks terminal)
docker run nginx

# Detached mode (-d) — runs in background
docker run -d nginx

# Name your container
docker run -d --name my-nginx nginx

# Auto-remove on stop (great for one-off tasks)
docker run --rm nginx nginx -v

# Interactive terminal (-it) for debugging
docker run -it ubuntu bash
docker run -it python:3.12 python

# Combine flags
docker run -d --name web --rm nginx

Key flags: -d (detached/background), --name (human-readable name), --rm (auto-cleanup), -it (interactive terminal with TTY).

Containers have their own network namespace — their ports aren't accessible from the host by default. Use -p to map host ports to container ports.

# Map host port 8080 to container port 80
docker run -d -p 8080:80 nginx
# Now visit http://localhost:8080

# Map multiple ports
docker run -d -p 8080:80 -p 8443:443 nginx

# Random host port (Docker picks one)
docker run -d -p 80 nginx
# Check which port: docker ps

# Bind to specific interface
docker run -d -p 127.0.0.1:8080:80 nginx  # Only accessible from localhost

The format is -p HOST_PORT:CONTAINER_PORT. Remember: host first, container second.

See what's running (and what has run) with docker ps.

# Running containers
docker ps

# All containers (including stopped)
docker ps -a

# Custom format
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

# Only container IDs
docker ps -q

docker ps shows: container ID, image, command, created time, status, ports, and name. The -a flag is essential for finding stopped containers that might be using resources.

Containers have a lifecycle: created → running → paused → stopped → removed.

# Stop a running container (sends SIGTERM, then SIGKILL after timeout)
docker stop my-nginx

# Start a stopped container
docker start my-nginx

# Restart a container
docker restart my-nginx

# Stop all running containers
docker stop $(docker ps -q)

docker stop sends SIGTERM first (giving the process time for graceful shutdown), then SIGKILL after a timeout (default 10 seconds). Use docker kill for immediate SIGKILL.

Docker artifacts accumulate fast. Regular cleanup keeps your system healthy.

# Remove a stopped container
docker rm my-nginx

# Force remove a running container
docker rm -f my-nginx

# Remove an image
docker rmi nginx:1.25

# Remove all stopped containers
docker container prune

# Remove all unused images
docker image prune

# Nuclear option — remove everything unused
docker system prune -a --volumes
# WARNING: removes all stopped containers, unused networks,
#          unused images, and optionally volumes

Be careful with docker system prune -a --volumes — it removes ALL unused data including named volumes with your database data. Use targeted prune commands in production.

docker exec runs a command inside an already-running container — essential for debugging.

# Run a command inside a running container
docker exec my-nginx ls /etc/nginx

# Interactive shell
docker exec -it my-nginx /bin/bash
# or for Alpine-based images
docker exec -it my-nginx /bin/sh

# Run as specific user
docker exec -u root my-nginx whoami

The difference from docker run: exec enters an existing container, while run creates a new one. Use exec for debugging running services; use run for one-off tasks.

Logs are your primary debugging tool. Docker captures stdout and stderr from the container's main process.

# View all logs
docker logs my-nginx

# Follow logs in real-time (like tail -f)
docker logs -f my-nginx

# Last 100 lines
docker logs --tail 100 my-nginx

# Logs since a time
docker logs --since 2024-01-01T00:00:00 my-nginx
docker logs --since 30m my-nginx  # Last 30 minutes

Containers should log to stdout/stderr (not to files). This lets Docker, and orchestrators like Kubernetes, handle log collection, rotation, and forwarding.

docker inspect returns detailed JSON metadata about any Docker object.

# Full JSON metadata
docker inspect my-nginx

# Extract specific fields
docker inspect --format '{{.NetworkSettings.IPAddress}}' my-nginx
docker inspect --format '{{.State.Status}}' my-nginx

# Image details
docker image inspect nginx:1.25

The Go template syntax (--format) extracts specific fields from the JSON. Useful for scripting and automation — get IP addresses, mount points, environment variables, and more.

Environment variables are the standard way to configure containers — twelve-factor app style.

# Set individual variables
docker run -d -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=myapp postgres:16

# Use an env file
docker run -d --env-file ./database.env postgres:16

# database.env file format:
POSTGRES_PASSWORD=secret
POSTGRES_DB=myapp
POSTGRES_USER=admin

Most official Docker images are configured entirely through environment variables. Check the image's documentation on Docker Hub for available variables.

Docker can consume significant disk space over time. Monitor and clean up regularly.

# Check disk usage
docker system df

# Detailed disk usage
docker system df -v

# Prune dangling images (untagged)
docker image prune

# Prune everything (careful!)
docker system prune -a --volumes

# Check Docker version and info
docker version
docker info

docker system df is your first stop when disk space runs low. It shows space used by images, containers, volumes, and build cache.

Consistent naming makes your container ecosystem manageable.

# Image naming: registry/repository:tag
docker.io/library/nginx:1.25-alpine    # Full qualified name
nginx:1.25-alpine                       # Short form (Docker Hub official)
ghcr.io/owner/myapp:v2.1.0             # GitHub Container Registry
myregistry.com/team/service:latest      # Private registry

# Tag strategies:
# ✅ Semantic versioning: myapp:1.2.3, myapp:1.2, myapp:1
# ✅ Git SHA: myapp:abc1234
# ✅ Date-based: myapp:2024-01-15
# ❌ Only :latest — unpredictable, non-reproducible

Use semantic versioning for releases and Git SHAs for traceability. Always have a way to know exactly what code is running in any container.

A Dockerfile is a text file with instructions to build an image. Each instruction creates a layer in the image.

# Every Dockerfile starts with FROM — the base image
FROM python:3.12-slim

# Set working directory (creates if doesn't exist)
WORKDIR /app

# Copy files from build context into image
COPY requirements.txt .

# Run commands during build
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY . .

# Document which port the app uses (doesn't actually publish)
EXPOSE 8000

# Default command when container starts
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Key instructions: FROM (base image), WORKDIR (set directory), COPY (add files), RUN (execute commands), EXPOSE (document port), CMD (default command).

When you run docker build, the entire directory (build context) gets sent to the Docker daemon. Use .dockerignore to exclude unnecessary files.

# Build an image (. = build context = current directory)
docker build -t myapp:1.0 .

# Everything in the build context gets sent to the daemon
# Use .dockerignore to exclude files

# .dockerignore
.git
.gitignore
__pycache__
*.pyc
.env
.venv
node_modules
Dockerfile
docker-compose*.yml
*.md
.DS_Store

A good .dockerignore speeds up builds dramatically. Without it, your entire .git history, node_modules, and other large directories get sent to the daemon on every build.

The docker build command reads a Dockerfile and creates an image.

# Build with tag
docker build -t myapp:1.0 .

# Build with custom Dockerfile
docker build -f Dockerfile.prod -t myapp:prod .

# Build with no cache (rebuild everything)
docker build --no-cache -t myapp:1.0 .

# Build with build arguments
docker build --build-arg VERSION=1.0 -t myapp:1.0 .

The -t flag tags the image with a name and optional version. Without it, you'd have to reference the image by its SHA hash.

Docker caches each layer. If a layer's instruction and inputs haven't changed, Docker reuses the cache. Order matters.

# ❌ BAD: Cache busted on every code change
FROM python:3.12-slim
WORKDIR /app
COPY . .                              # Code + requirements copied together
RUN pip install -r requirements.txt   # Reinstalls ALL deps on any change

# ✅ GOOD: Dependencies cached separately
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .               # Copy deps file first
RUN pip install -r requirements.txt   # Cached unless requirements change
COPY . .                              # Code changes don't bust dep cache

Rule: Put things that change less frequently (dependencies) before things that change often (application code). This maximizes cache hits and speeds up builds.

Multi-stage builds let you use one image for building and a different (smaller) image for running. Only the final stage becomes your image.

# Stage 1: Build
FROM node:22-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

# Stage 2: Production (only the built output)
FROM nginx:alpine
COPY --from=builder /app/dist /usr/share/nginx/html
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
# Result: ~25MB instead of ~1GB with node_modules!

The AS builder names a stage. COPY --from=builder copies files from that stage into the current one. Build tools, source code, and node_modules stay in the builder stage — only the compiled output makes it to production.

Choosing the right base image is crucial for security, size, and compatibility.

# Full image — everything included, largest
python:3.12            # ~1GB, includes build tools, man pages
# Good for: development, when you need to compile C extensions

# Slim — no build tools, moderate size
python:3.12-slim       # ~150MB, Debian-based without extras
# Good for: most production apps

# Alpine — musl libc, tiny
python:3.12-alpine     # ~50MB, Alpine Linux
# Good for: Go, Rust (static binaries). Tricky for Python (musl vs glibc)

# Distroless — no shell, no package manager
gcr.io/distroless/python3  # Minimal, highest security
# Good for: hardened production, no debugging tools

# Scratch — literally empty
# FROM scratch           # 0MB base, only for static binaries

Recommendation: Start with -slim for most apps. Use alpine for Go/Rust. Use distroless when security is paramount. Avoid full images in production.

Both copy files into the image, but they're different.

# COPY — simple file/directory copy (USE THIS almost always)
COPY ./src /app/src
COPY requirements.txt .

# ADD — same as COPY but with extras:
# 1. Auto-extracts compressed archives
ADD archive.tar.gz /app/    # Extracts into /app/
# 2. Can download from URLs (but prefer RUN curl/wget)
ADD https://example.com/file /app/file

# Rule: Use COPY unless you specifically need ADD's features

COPY is explicit and predictable. ADD has implicit behavior (auto-extraction) that can surprise you. Docker best practices recommend COPY for almost all cases.

Both define what runs when a container starts, but they work differently.

# CMD — default command (easily overridden)
CMD ["python", "app.py"]
# Override: docker run myapp python other_script.py

# ENTRYPOINT — fixed executable (args appended)
ENTRYPOINT ["python"]
CMD ["app.py"]          # Default argument
# docker run myapp              → python app.py
# docker run myapp other.py     → python other.py

# Exec form vs Shell form:
CMD ["python", "app.py"]      # Exec form (preferred) — PID 1 is python
CMD python app.py              # Shell form — PID 1 is /bin/sh -c
# Exec form receives signals properly (SIGTERM for graceful shutdown)

Always use exec form (JSON array syntax). Shell form wraps your command in /bin/sh -c, which means your process doesn't receive signals directly — breaking graceful shutdown.

ARG and ENV both set variables, but at different scopes.

# ARG — build-time only (not available in running container)
ARG PYTHON_VERSION=3.12
FROM python:${PYTHON_VERSION}-slim

ARG BUILD_DATE
LABEL build-date=$BUILD_DATE

# ENV — runtime (available in running container)
ENV APP_ENV=production
ENV DATABASE_URL=sqlite:///db.sqlite3

# Usage:
docker build --build-arg PYTHON_VERSION=3.11 -t myapp .
docker run -e APP_ENV=development myapp

ARG values disappear after the build — they don't end up in the final image. ENV values persist and are available at runtime. Use ARG for build configuration, ENV for runtime configuration.

A production-ready Dockerfile follows several security and maintenance best practices.

FROM python:3.12-slim

# Labels for metadata
LABEL maintainer="dev@example.com"
LABEL version="1.0"
LABEL description="FastAPI application"

# Always use WORKDIR instead of cd
WORKDIR /app

# Document the port (informational only)
EXPOSE 8000

# Create non-root user for security
RUN addgroup --system appgroup && adduser --system --group appuser

# Copy and install as root, then switch
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .

# Switch to non-root user LAST (before CMD)
USER appuser

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Key practices: Use WORKDIR (not cd), add LABELs for metadata, create and switch to a non-root user (reduces attack surface), and document ports with EXPOSE.

Health checks let Docker monitor if your application is actually healthy, not just running.

# Built-in health monitoring
HEALTHCHECK --interval=30s --timeout=3s --retries=3 --start-period=10s \
  CMD curl -f http://localhost:8000/health || exit 1

# Check health status
# docker ps shows (healthy), (unhealthy), or (health: starting)
docker inspect --format '{{.State.Health.Status}}' container_name

Parameters: --interval (check frequency), --timeout (max check duration), --retries (failures before unhealthy), --start-period (grace period for startup).

Let's put it all together with a production-ready FastAPI Dockerfile using multi-stage builds.

# Dockerfile for a production FastAPI application
FROM python:3.12-slim AS base

# Prevent Python from writing .pyc files and enable unbuffered output
ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

WORKDIR /app

# Install dependencies in a separate stage for caching
FROM base AS deps
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Production stage
FROM base AS production
WORKDIR /app

# Copy installed packages from deps stage
COPY --from=deps /usr/local/lib/python3.12/site-packages /usr/local/lib/python3.12/site-packages
COPY --from=deps /usr/local/bin /usr/local/bin

# Create non-root user
RUN addgroup --system app && adduser --system --group app

# Copy application code
COPY . .

# Switch to non-root user
USER app

EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
  CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

# Build and run
docker build -t fastapi-app:1.0 .
docker run -d -p 8000:8000 --name api fastapi-app:1.0
curl http://localhost:8000/health

Containers are disposable by design. Any data written inside a container's writable layer disappears when the container is removed.

# Prove it:
docker run -it --name test ubuntu bash -c "echo 'hello' > /data.txt && cat /data.txt"
# Output: hello
docker rm test
docker run -it --name test ubuntu bash -c "cat /data.txt"
# Error: No such file or directory — data is gone!
docker rm test

This is a feature, not a bug. Ephemeral containers are easy to replace, scale, and update. But databases, uploads, and configuration need to persist. That's where volumes come in.

Docker provides three mechanisms for persistent and temporary storage:

# 1. VOLUMES — Docker-managed, best for persistent data
docker run -v mydata:/app/data myapp

# 2. BIND MOUNTS — Map host directory, best for development
docker run -v $(pwd)/src:/app/src myapp

# 3. TMPFS — In-memory, best for sensitive/temporary data
docker run --tmpfs /app/cache myapp

┌───────────────────────────────────────────┐
│              Docker Host                   │
│                                           │
│  ┌─────────┐  ┌──────────┐  ┌─────────┐  │
│  │ Volume   │  │Bind Mount│  │  tmpfs   │  │
│  │/var/lib/ │  │ Host Dir │  │ (Memory) │  │
│  │docker/   │  │          │  │          │  │
│  │volumes/  │  │          │  │          │  │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  │
│       │             │             │         │
│  ┌────▼─────────────▼─────────────▼─────┐  │
│  │           Container Filesystem        │  │
│  │     /app/data  /app/src  /app/cache   │  │
│  └──────────────────────────────────────┘  │
└───────────────────────────────────────────┘

Named volumes are Docker-managed storage, stored at /var/lib/docker/volumes/ on the host.

# Create a named volume
docker volume create mydata

# List volumes
docker volume ls

# Inspect volume details
docker volume inspect mydata

# Remove a volume
docker volume rm mydata

# Remove all unused volumes
docker volume prune

Volumes are the recommended way to persist data. They work on all platforms, are managed by Docker, and can be backed up, migrated, and shared between containers.

Named volumes persist data across container lifecycle events — stop, remove, recreate.

# Mount a named volume
docker run -d --name db \
  -v pgdata:/var/lib/postgresql/data \
  -e POSTGRES_PASSWORD=secret \
  postgres:16

# Data persists across container restarts
docker stop db
docker rm db
docker run -d --name db \
  -v pgdata:/var/lib/postgresql/data \
  -e POSTGRES_PASSWORD=secret \
  postgres:16
# Data is still there!

The volume pgdata lives independently of any container. Even after docker rm, the volume and its data remain until explicitly deleted with docker volume rm.

Bind mounts map a host directory directly into the container — changes on either side are instantly visible.

# Mount current directory (development workflow)
docker run -d \
  -v $(pwd)/src:/app/src \
  -v $(pwd)/config:/app/config:ro \
  -p 8000:8000 \
  myapp:dev

# Changes on host instantly appear in container
# Great for development hot-reload

Bind mounts are perfect for development: edit code on your host, see changes immediately in the container. Add :ro for read-only mounts when the container shouldn't modify the files.

Read-only mounts prevent the container from modifying mounted files — a security best practice.

# Add :ro to make mount read-only
docker run -d \
  -v config.json:/app/config.json:ro \
  -v /etc/ssl/certs:/etc/ssl/certs:ro \
  myapp

# Container can read but not modify these files
# Security best practice for config and certificates

Use :ro for configuration files, SSL certificates, and any data the container should consume but never change. This follows the principle of least privilege.

Every database container needs a volume. Without one, you lose all data on container removal.

# PostgreSQL with persistent data
docker run -d --name postgres \
  -v pgdata:/var/lib/postgresql/data \
  -e POSTGRES_PASSWORD=secret \
  -e POSTGRES_DB=myapp \
  -p 5432:5432 \
  postgres:16

# MySQL with persistent data
docker run -d --name mysql \
  -v mysqldata:/var/lib/mysql \
  -e MYSQL_ROOT_PASSWORD=secret \
  -e MYSQL_DATABASE=myapp \
  -p 3306:3306 \
  mysql:8

# Redis with persistent data
docker run -d --name redis \
  -v redisdata:/data \
  redis:7-alpine redis-server --appendonly yes

Know the data directory for your database: PostgreSQL uses /var/lib/postgresql/data, MySQL uses /var/lib/mysql, Redis uses /data.

Volumes need backup strategies, just like any other storage.

# Copy file from container to host
docker cp postgres:/var/lib/postgresql/data/pg_dump.sql ./backup.sql

# Backup a volume using a temporary container
docker run --rm \
  -v pgdata:/source:ro \
  -v $(pwd)/backups:/backup \
  ubuntu tar czf /backup/pgdata-$(date +%Y%m%d).tar.gz -C /source .

# Restore from backup
docker run --rm \
  -v pgdata:/target \
  -v $(pwd)/backups:/backup \
  ubuntu bash -c "cd /target && tar xzf /backup/pgdata-20240115.tar.gz"

The pattern: spin up a temporary container with both the volume and a host directory mounted, then tar/compress. For databases, prefer native dump tools (pg_dump, mysqldump) for consistency.

Volume performance varies significantly by platform.

# macOS: File sharing is through a VM — slower than native Linux
# Options for Docker Desktop on Mac:
# - VirtioFS (default, fastest option on macOS)
# - gRPC FUSE (older, slower)

# For heavy I/O workloads on macOS:
# 1. Use named volumes for node_modules, vendor, etc.
# 2. Use :cached or :delegated consistency flags
docker run -v $(pwd):/app:cached myapp    # Host writes prioritized
docker run -v $(pwd):/app:delegated myapp # Container writes prioritized

# Linux: Native filesystem speed — no overhead
# This is why production containers run on Linux

Pro tip: On macOS, keep node_modules in a named volume instead of bind-mounting from the host. This avoids the VM filesystem overhead for the most I/O-heavy directory.

tmpfs mounts store data in memory — fast, ephemeral, and secure.

# In-memory storage — data disappears when container stops
docker run -d \
  --tmpfs /app/cache:size=100m \
  --tmpfs /tmp \
  myapp

# Use cases:
# - Sensitive data (tokens, keys) that shouldn't persist
# - High-speed caching
# - Temporary build artifacts

tmpfs never writes to the host filesystem — data exists only in RAM. This makes it perfect for secrets (no traces on disk) and high-speed caches (no I/O overhead).

Docker creates three networks by default when installed.

# Docker creates three networks by default
docker network ls
# NETWORK ID   NAME      DRIVER   SCOPE
# abc123       bridge    bridge   local   ← default
# def456       host      host     local
# ghi789       none      null     local

• bridge — Default. Containers get isolated network with NAT to host
• host — Container shares host's network stack (Linux only)
• none — No networking at all

The default bridge network has a critical limitation: no DNS resolution between containers.

# Default bridge — containers can't resolve each other by name
docker run -d --name web1 nginx
docker run -d --name web2 nginx
# web1 cannot reach web2 by name on default bridge!

# Containers on default bridge communicate via IP only
docker inspect web1 --format '{{.NetworkSettings.IPAddress}}'

On the default bridge, you'd need to know the container's IP address to communicate — which changes every time the container restarts. This is why custom networks exist.

Custom bridge networks enable automatic DNS resolution between containers — the #1 reason to use them.

# Create a custom network
docker network create mynet

# Containers on custom networks CAN resolve by name (DNS!)
docker run -d --name api --network mynet nginx
docker run -d --name web --network mynet nginx

# From web, you can reach api by name:
docker exec web curl http://api:80
# Docker's embedded DNS makes this work automatically

Always use custom networks for multi-container applications. The built-in DNS makes service discovery effortless — no need to manage IP addresses.

The full set of network management commands.

# Create a network
docker network create --driver bridge mynet

# List networks
docker network ls

# Inspect a network
docker network inspect mynet

# Connect a running container to a network
docker network connect mynet existing-container

# Disconnect from a network
docker network disconnect mynet existing-container

# Remove a network
docker network rm mynet

# Remove all unused networks
docker network prune

Containers can be connected to multiple networks simultaneously — useful for creating network boundaries (e.g., an API server connected to both frontend and backend networks).

Docker runs an embedded DNS server at 127.0.0.11 on custom networks.

# Docker's embedded DNS server (127.0.0.11) runs on custom networks
# Containers resolve each other by:
# 1. Container name
# 2. Network aliases

docker run -d --name api --network mynet --network-alias backend nginx

# Other containers on mynet can reach it as:
# - http://api:80       (by name)
# - http://backend:80   (by alias)

Network aliases let you give a container multiple DNS names — useful for service migration or when multiple containers should respond to the same name (basic load balancing).

Host networking removes network isolation — the container uses the host's network stack directly.

# Container uses host's network stack directly (Linux only!)
docker run -d --network host nginx
# nginx is now on host port 80 — no port mapping needed

# WARNING: No network isolation! Container sees ALL host network traffic
# NOT available on macOS/Windows (they use a VM)

Use cases: performance-sensitive applications where NAT overhead matters, or when the container needs to bind to specific host interfaces. Generally avoid in production — the isolation trade-off isn't worth it for most apps.

The modern pattern for multi-container communication: custom networks with service names.

# Create network
docker network create app-net

# Start backend
docker run -d --name api --network app-net \
  -e DATABASE_URL=postgres://db:5432/myapp \
  myapi:1.0

# Start database
docker run -d --name db --network app-net \
  -e POSTGRES_PASSWORD=secret \
  postgres:16

# Start frontend (connected to same network + exposed to host)
docker run -d --name web --network app-net \
  -p 3000:3000 \
  myfrontend:1.0

# api can reach db:5432
# web can reach api:8000
# Only web is exposed to the host via -p

Never use --link — it's deprecated. Custom networks are the correct approach.

A common source of confusion: EXPOSE in a Dockerfile does NOT make a port accessible.

# In Dockerfile — EXPOSE is documentation only
EXPOSE 8000
# This does NOT make the port accessible! It's just metadata.

# At runtime — -p actually publishes the port
docker run -p 8000:8000 myapp
# NOW port 8000 is accessible from the host

EXPOSE documents intent — "this container listens on port 8000." The -p flag at runtime creates the actual port mapping. Think of EXPOSE as a comment for humans and tools.

Use separate networks to create security boundaries between services.

# Containers on different networks are isolated
docker network create frontend-net
docker network create backend-net

# Web server on frontend only
docker run -d --name web --network frontend-net -p 80:80 nginx

# API on both networks (bridge between frontend and backend)
docker run -d --name api --network frontend-net myapi
docker network connect backend-net api

# Database on backend only (NOT accessible from frontend)
docker run -d --name db --network backend-net postgres:16

# web → api ✓ (both on frontend-net)
# api → db  ✓ (both on backend-net)
# web → db  ✗ (different networks — isolated!)

This pattern ensures the database is never directly accessible from the public-facing web tier — defense in depth.

Network issues are common in containerized apps. Here's your debugging toolkit.

# Inspect network to see connected containers
docker network inspect mynet

# Test connectivity from inside a container
docker exec api ping db
docker exec api curl http://web:80

# DNS lookup
docker exec api nslookup db

# Use a debug container with networking tools
docker run --rm --network mynet nicolaka/netshoot \
  curl http://api:8000/health

# Check port bindings
docker port my-container

The nicolaka/netshoot image is packed with networking tools (curl, dig, nslookup, tcpdump, etc.) — perfect for debugging Docker networking issues without installing tools in your application containers.

Running multi-container apps manually is tedious and error-prone. Compare:

# Without Compose — manually starting each container:
docker network create app-net
docker volume create pgdata
docker run -d --name db --network app-net -v pgdata:/var/lib/postgresql/data -e POSTGRES_PASSWORD=secret postgres:16
docker run -d --name redis --network app-net redis:7-alpine
docker run -d --name api --network app-net -p 8000:8000 -e DATABASE_URL=postgres://db:5432/myapp myapi:1.0
docker run -d --name web --network app-net -p 3000:3000 myfrontend:1.0

# With Compose — one file, one command:
docker compose up -d
# That's it. All services, networks, and volumes defined in compose.yaml

Docker Compose defines your entire application stack in a single YAML file. Networks, volumes, environment variables, dependencies — everything declarative and version-controlled.

Modern Compose uses compose.yaml (no version key needed).

# compose.yaml (modern filename — no more docker-compose.yml needed)
# Note: the "version:" key is REMOVED — it's no longer needed or used

services:
  api:
    build: ./backend
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgres://postgres:secret@db:5432/myapp
    depends_on:
      - db
      - redis

  web:
    build: ./frontend
    ports:
      - "3000:3000"

  db:
    image: postgres:16
    volumes:
      - pgdata:/var/lib/postgresql/data
    environment:
      - POSTGRES_PASSWORD=secret

  redis:
    image: redis:7-alpine

volumes:
  pgdata:

The services block defines containers. Each service can build from a Dockerfile or use a pre-built image. The volumes top-level key declares named volumes.

Services support a rich set of configuration options.

services:
  api:
    # Build from a Dockerfile
    build:
      context: ./backend
      dockerfile: Dockerfile
      args:
        PYTHON_VERSION: "3.12"

    # Container name (optional — Compose generates one)
    container_name: myapp-api

    # Port mapping
    ports:
      - "8000:8000"

    # Environment variables
    environment:
      APP_ENV: production
      SECRET_KEY: ${SECRET_KEY}  # From .env file or shell

    # Environment from file
    env_file:
      - .env
      - .env.local

    # Volume mounts
    volumes:
      - ./backend/src:/app/src          # Bind mount (dev)
      - static-files:/app/static        # Named volume

    # Resource limits
    deploy:
      resources:
        limits:
          cpus: "1.0"
          memory: 512M

    # Restart policy
    restart: unless-stopped

    # Health check
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 3s
      retries: 3
      start_period: 10s

The two commands you'll use most often.

# Start all services
docker compose up

# Start in detached mode (background)
docker compose up -d

# Rebuild images before starting
docker compose up -d --build

# Start specific services only
docker compose up -d api db

# Stop and remove containers, networks
docker compose down

# Also remove volumes (WARNING: deletes data!)
docker compose down --volumes

# Also remove images
docker compose down --rmi all

up creates networks, volumes, and starts containers. down stops and removes containers and networks. Add --volumes cautiously — it destroys persistent data.

Once services are running, Compose provides management commands.

# List running services
docker compose ps

# View logs
docker compose logs
docker compose logs -f api        # Follow specific service
docker compose logs --tail 50 api # Last 50 lines

# Execute command in running service
docker compose exec api bash
docker compose exec db psql -U postgres

# Run a one-off command
docker compose run --rm api python manage.py migrate

# Restart a specific service
docker compose restart api

# Scale a service (multiple instances)
docker compose up -d --scale web=3

The difference between exec and run: exec enters a running container; run creates a new, temporary container. Use run --rm for one-off tasks like migrations.

Compose can build images alongside running them.

services:
  # Build from local Dockerfile
  api:
    build:
      context: ./backend
      dockerfile: Dockerfile.prod
      target: production  # Multi-stage build target
    image: myapp-api:latest  # Also tag the built image

  # Use pre-built image
  db:
    image: postgres:16

  # Build with caching
  web:
    build:
      context: ./frontend
      cache_from:
        - myregistry.com/web:latest

The target field selects a specific stage from a multi-stage Dockerfile. The image field alongside build tags the locally-built image.

Compose supports multiple ways to inject environment variables.

# .env file (auto-loaded by Compose)
# POSTGRES_PASSWORD=secret
# APP_ENV=development
# API_PORT=8000

# compose.yaml — variable interpolation
services:
  api:
    ports:
      - "${API_PORT:-8000}:8000"  # Default value with :-
    environment:
      APP_ENV: ${APP_ENV}
      DB_PASS: ${POSTGRES_PASSWORD}

  db:
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}

# Use different env files
docker compose --env-file .env.production up -d

The ${VAR:-default} syntax provides fallback values. A .env file in the same directory as compose.yaml is auto-loaded for interpolation.

Compose creates a default network automatically. All services can reach each other by service name.

services:
  web:
    # Can reach api at http://api:8000
    # Can reach db at db:5432
    build: ./frontend
    ports:
      - "3000:3000"
    networks:
      - frontend

  api:
    build: ./backend
    networks:
      - frontend
      - backend

  db:
    image: postgres:16
    networks:
      - backend  # Only accessible from backend network

networks:
  frontend:
  backend:

Custom networks in Compose provide the same isolation as standalone Docker networks. Services on the same network discover each other by name; services on different networks are isolated.

Control startup order with depends_on and health conditions.

services:
  api:
    build: ./backend
    depends_on:
      db:
        condition: service_healthy  # Wait until db is ready!
      redis:
        condition: service_started  # Just wait until started

  db:
    image: postgres:16
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 3s
      retries: 5
      start_period: 10s

  redis:
    image: redis:7-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5

service_healthy waits until the dependency's health check passes. Without this, your app might start before the database is ready to accept connections — a very common bug.

Profiles let you define optional services that only start when explicitly requested.

services:
  api:
    build: ./backend
    # No profile — always starts

  db:
    image: postgres:16
    # No profile — always starts

  adminer:
    image: adminer
    profiles: [debug]  # Only starts with --profile debug
    ports:
      - "8080:8080"

  mailhog:
    image: mailhog/mailhog
    profiles: [debug]
    ports:
      - "1025:1025"
      - "8025:8025"

# Start without debug services
docker compose up -d

# Start with debug services
docker compose --profile debug up -d

Great for development tools (database GUIs, email catchers, debug dashboards) that you don't want running all the time.

Compose automatically merges compose.override.yaml with compose.yaml — perfect for dev-specific settings.

# compose.yaml — base configuration
services:
  api:
    build: ./backend
    ports:
      - "8000:8000"

# compose.override.yaml — auto-loaded for local dev!
services:
  api:
    volumes:
      - ./backend:/app       # Mount source for hot-reload
    environment:
      - APP_ENV=development
    command: uvicorn main:app --reload --host 0.0.0.0

# Local dev (auto-loads override)
docker compose up -d

# Production (skip override, use production file)
docker compose -f compose.yaml -f compose.prod.yaml up -d

A complete development stack: FastAPI + React + PostgreSQL + Redis.

# compose.yaml — FastAPI + React + PostgreSQL + Redis
services:
  api:
    build:
      context: ./backend
      target: development
    ports:
      - "8000:8000"
    volumes:
      - ./backend/app:/app/app
    environment:
      DATABASE_URL: postgresql://postgres:secret@db:5432/myapp
      REDIS_URL: redis://redis:6379
      APP_ENV: development
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_healthy
    command: uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

  web:
    build:
      context: ./frontend
      target: development
    ports:
      - "3000:3000"
    volumes:
      - ./frontend/src:/app/src
    environment:
      VITE_API_URL: http://localhost:8000
    command: npm run dev -- --host 0.0.0.0

  db:
    image: postgres:16-alpine
    volumes:
      - pgdata:/var/lib/postgresql/data
    environment:
      POSTGRES_PASSWORD: secret
      POSTGRES_DB: myapp
    ports:
      - "5432:5432"
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 5s
      timeout: 3s
      retries: 5

  redis:
    image: redis:7-alpine
    volumes:
      - redisdata:/data
    ports:
      - "6379:6379"
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5
    command: redis-server --appendonly yes

volumes:
  pgdata:
  redisdata:

Use multi-stage builds to create separate dev and prod images from one Dockerfile.

# Dockerfile with multi-stage for dev and prod

# Base stage — shared dependencies
FROM python:3.12-slim AS base
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Development stage — includes dev tools
FROM base AS development
RUN pip install debugpy pytest httpx
COPY . .
CMD ["uvicorn", "app.main:app", "--reload", "--host", "0.0.0.0", "--port", "8000"]

# Production stage — optimized and secure
FROM base AS production
RUN addgroup --system app && adduser --system --group app
COPY . .
USER app
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--workers", "4"]

# Build dev image
docker build --target development -t myapp:dev .

# Build prod image
docker build --target production -t myapp:prod .

Keep the development feedback loop fast with bind mounts and watch mode.

# compose.yaml for development with hot-reload
services:
  api:
    build:
      context: ./backend
      target: development
    volumes:
      - ./backend:/app              # Mount source code
      - /app/__pycache__            # Exclude cache (anonymous volume)
    command: uvicorn app.main:app --reload --host 0.0.0.0

  web:
    build:
      context: ./frontend
      target: development
    volumes:
      - ./frontend/src:/app/src     # Mount only source
      - /app/node_modules           # Exclude node_modules
    command: npm run dev -- --host 0.0.0.0

# Compose Watch (modern alternative — built into Compose v2.22+):
services:
  api:
    build: ./backend
    develop:
      watch:
        - path: ./backend/app
          action: sync
          target: /app/app
        - path: ./backend/requirements.txt
          action: rebuild

# Start with watch mode
docker compose watch

Build images that run on both AMD64 (Intel/AMD) and ARM64 (Apple Silicon, AWS Graviton).

# Create a buildx builder
docker buildx create --name multiplatform --driver docker-container --bootstrap
docker buildx use multiplatform

# Build for AMD64 + ARM64 and push to registry
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --tag myregistry.com/myapp:1.0 \
  --push \
  .

# Inspect the manifest
docker buildx imagetools inspect myregistry.com/myapp:1.0

With Apple Silicon Macs widespread and ARM servers gaining popularity (AWS Graviton is cheaper + faster), multi-architecture builds are increasingly important.

Automate image builds and pushes in your CI pipeline.

# .github/workflows/docker.yml
name: Build and Push Docker Image

on:
  push:
    branches: [main]
    tags: ['v*']

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  build:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write

    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-qemu-action@v3
      - uses: docker/setup-buildx-action@v3

      - uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - uses: docker/metadata-action@v5
        id: meta
        with:
          images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
          tags: |
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}
            type=sha

      - uses: docker/build-push-action@v5
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

A systematic approach to debugging container issues.

# Step 1: Check if container is running
docker ps -a | grep my-container

# Step 2: Check logs
docker logs my-container
docker logs --tail 50 -f my-container

# Step 3: Check container details
docker inspect my-container

# Step 4: Get inside the container
docker exec -it my-container /bin/bash
# or /bin/sh for Alpine

# Step 5: Check processes
docker exec my-container ps aux

# Step 6: Check network
docker exec my-container cat /etc/hosts
docker exec my-container ping other-service

# Step 7: Check filesystem
docker exec my-container ls -la /app
docker exec my-container df -h

Always start with logs (docker logs). Most issues are visible there. If not, shell into the container and inspect the environment directly.

Common exit codes and what they mean.

# 1. Check exit code and logs
docker ps -a --format "table {{.Names}}\t{{.Status}}"
docker logs failing-container

# 2. Common exit codes:
# Exit 0   — Command finished (not a long-running process)
#            Fix: Use CMD that keeps running
#
# Exit 1   — Application error
#            Fix: Check logs, usually code/config issue
#
# Exit 126 — Permission denied
#            Fix: chmod +x entrypoint script, check USER
#
# Exit 127 — Command not found
#            Fix: Check CMD/ENTRYPOINT path
#
# Exit 137 — Out of memory (OOM killed)
#            Fix: Increase --memory limit or optimize app
#
# Exit 139 — Segfault
#            Fix: Check native library issues (musl vs glibc)

# 3. Test interactively
docker run -it myapp /bin/sh
# Then manually run the CMD to see errors

# 4. Check port conflicts
docker port my-container
lsof -i :8000  # Is something else using this port?

Prevent runaway containers from starving the host.

# Limit memory
docker run -d --memory=512m --memory-swap=1g myapp

# Limit CPU
docker run -d --cpus=1.5 myapp           # 1.5 CPU cores
docker run -d --cpu-shares=512 myapp      # Relative weight

# Check resource usage
docker stats
docker stats --format "table {{.Name}}\t{{.CPUPerc}}\t{{.MemUsage}}"

docker stats is your real-time monitoring dashboard. Watch CPU and memory usage to right-size your limits. Start generous, then tighten based on observed usage.

Container logs can grow unbounded. Configure limits to prevent disk exhaustion.

# /etc/docker/daemon.json
{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

# Per-container log limits
docker run -d \
  --log-opt max-size=10m \
  --log-opt max-file=3 \
  myapp

# Compose log config
services:
  api:
    image: myapp
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

Handling secrets properly is critical for security.

# ❌ BAD: Secrets in Dockerfile (baked into image layers!)
ENV API_KEY=super-secret-key

# ❌ RISKY: Secrets in compose.yaml (checked into git)
environment:
  - API_KEY=super-secret-key

# ✅ BETTER: .env file (add to .gitignore!)
# .env
# API_KEY=super-secret-key

# ✅ BETTER: env_file in compose.yaml
services:
  api:
    env_file: .env

Best: Use your CI/CD system's secret management (GitHub Secrets, GitLab CI variables) or a dedicated tool (HashiCorp Vault, AWS Secrets Manager). Never commit secrets to version control.

Smaller images are faster to pull, more secure, and cheaper to store.

# Check image size
docker images myapp
docker image inspect myapp:latest --format '{{.Size}}'

# Compare sizes
docker images --format "table {{.Repository}}:{{.Tag}}\t{{.Size}}"

# Optimization checklist:
# 1. Multi-stage builds
# 2. .dockerignore (exclude .git, node_modules, etc.)
# 3. Minimal base images (slim, alpine)
# 4. Combine RUN commands (fewer layers)
# 5. Clean up in the same RUN step

# Security scanning with Docker Scout
docker scout quickview myapp:latest
docker scout cves myapp:latest
docker scout recommendations myapp:latest

# Alternative: Trivy
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \
  aquasec/trivy image myapp:latest

Run security scans in CI. Fix critical and high CVEs before deploying. Use docker scout or trivy to identify vulnerable packages in your images.

Podman is a drop-in Docker replacement with key architectural differences.

# Podman — drop-in Docker replacement
# Key differences:
# - Daemonless: no background service needed
# - Rootless by default: runs without root privileges
# - OCI-compliant: same images work in both
# - CLI compatible: alias docker=podman

# Install on macOS
brew install podman
podman machine init
podman machine start

# Same commands work!
podman pull nginx
podman run -d -p 8080:80 nginx
podman ps

Podman's daemonless architecture means no single point of failure and better security. If Docker Desktop licensing is a concern, Podman is a strong alternative.

containerd is the container runtime used by both Docker and Kubernetes.

┌────────────┐    ┌────────────┐
│   Docker    │    │ Kubernetes │
│   CLI/API   │    │  kubelet   │
└──────┬─────┘    └──────┬─────┘
       │                 │
       ▼                 ▼
┌────────────────────────────┐
│        containerd           │
│  (container runtime)        │
└────────────┬───────────────┘
             │
             ▼
┌────────────────────────────┐
│          runc               │
│  (OCI runtime — creates     │
│   actual Linux containers)  │
└────────────────────────────┘

You rarely interact with containerd directly, but understanding the stack explains why "Kubernetes dropped Docker" was misleading — they dropped the Docker daemon, but still use containerd (which Docker also uses).

Kubernetes is powerful — and often overkill. Know when to reach for it.

# Kubernetes is for:
# ✅ Running hundreds of containers across many machines
# ✅ Auto-scaling based on load
# ✅ Self-healing (auto-restart failed containers)
# ✅ Rolling deployments with zero downtime
# ✅ Service discovery and load balancing at scale

# Kubernetes is NOT for:
# ❌ Small projects (1-10 services)
# ❌ Solo developers or small teams
# ❌ When Docker Compose is sufficient
# ❌ Learning Docker (learn Docker first!)

# The spectrum:
# Single container     → docker run
# Multiple services    → Docker Compose
# Multiple machines    → Kubernetes

Most applications don't need Kubernetes. If you're not running across multiple machines or need auto-scaling, Docker Compose is simpler, faster to set up, and easier to maintain.

Registries store and distribute container images — like GitHub for code, but for containers.

# Docker Hub — default, most images
docker push myuser/myapp:1.0

# GitHub Container Registry (GHCR)
docker login ghcr.io -u USERNAME --password-stdin
docker tag myapp:1.0 ghcr.io/username/myapp:1.0
docker push ghcr.io/username/myapp:1.0

# AWS ECR
aws ecr get-login-password | docker login --username AWS --password-stdin 123456.dkr.ecr.us-east-1.amazonaws.com
docker tag myapp:1.0 123456.dkr.ecr.us-east-1.amazonaws.com/myapp:1.0
docker push 123456.dkr.ecr.us-east-1.amazonaws.com/myapp:1.0

# Self-hosted with Harbor
# harbor.mycompany.com/project/myapp:1.0

Choose based on your ecosystem: GHCR for GitHub-centric workflows, ECR/GCR/ACR for cloud-native, Docker Hub for public images, Harbor for self-hosted enterprise.

Good tagging makes deployments reproducible and rollbacks possible.

# ✅ Semantic versioning (recommended)
docker tag myapp:latest myapp:1.2.3
docker tag myapp:latest myapp:1.2
docker tag myapp:latest myapp:1
# Users can pin to major, minor, or patch

# ✅ Git SHA (for traceability)
docker tag myapp:latest myapp:abc1234
# Know exactly which commit produced this image

# ✅ Date-based
docker tag myapp:latest myapp:2024-01-15

# ❌ Only :latest (anti-pattern)
# - Not reproducible
# - Can't rollback
# - "latest" doesn't mean "newest" — it's just a default tag

Best practice: tag with semver + Git SHA. Semver for human communication, Git SHA for exact traceability. Always push both tags.

You don't need Kubernetes to run containers in production. Managed services handle the heavy lifting.

# Managed container services (no Kubernetes needed!):

# AWS ECS (Elastic Container Service)
# - Managed container orchestration
# - Works with Fargate (serverless) or EC2

# Google Cloud Run
# - Fully managed, scales to zero
# - Pay only when handling requests

# Azure Container Apps
# - Managed Kubernetes underneath
# - Scales to zero, event-driven

# Railway, Fly.io, Render
# - Developer-friendly platforms
# - Push and deploy container images

# Example: Deploy to Cloud Run
gcloud run deploy myapp \
  --image gcr.io/myproject/myapp:1.0 \
  --platform managed \
  --region us-central1 \
  --allow-unauthenticated

Dev Containers standardize development environments using Docker.

// .devcontainer/devcontainer.json
{
  "name": "Python Dev",
  "image": "mcr.microsoft.com/devcontainers/python:3.12",
  "features": {
    "ghcr.io/devcontainers/features/github-cli:1": {},
    "ghcr.io/devcontainers/features/docker-in-docker:2": {}
  },
  "forwardPorts": [8000],
  "postCreateCommand": "pip install -r requirements.txt",
  "customizations": {
    "vscode": {
      "extensions": [
        "ms-python.python",
        "ms-python.vscode-pylance",
        "charliermarsh.ruff"
      ]
    }
  },
  "remoteUser": "vscode"
}

Benefits: Every developer gets an identical environment. New contributors are productive in minutes, not hours. Works with VS Code and GitHub Codespaces.

WebAssembly (Wasm) is emerging as the next evolution of containers.

# Docker now supports WebAssembly (Wasm) runtimes
# Wasm containers are:
# - 10-100x faster startup than Linux containers
# - Much smaller (1-10MB vs 100MB+)
# - Architecture-neutral (runs anywhere)
# - Better sandboxing

# Run a Wasm container
docker run --runtime=io.containerd.wasmedge.v1 \
  --platform=wasi/wasm32 \
  myapp:wasm

# Supported runtimes:
# - WasmEdge (io.containerd.wasmedge.v1)
# - Wasmtime (io.containerd.wasmtime.v1)
# - Spin (io.containerd.spin.v1)

# Still early days, but the future of lightweight containers

Wasm won't replace Linux containers — it complements them. For lightweight, fast-starting workloads (serverless functions, edge computing), Wasm is compelling. For full application stacks, Linux containers remain the standard.