Experiments

4 min read

Docker

Docker Engine

  • aka Docker CE
  • open source: https://github.com/moby/moby
  • only runs on linux
  • contains:
    • server: dockerd
    • client: REST API + CLI
  • internally, dockerd delegates down the chain:
    • dockerd —> containerd —> runc —> Linux namespaces + cgroups
  • Projects in CNCF: https://landscape.cncf.io/ (Check graduated status)

Docker Desktop

  • Since Docker engine can only run on linux
  • Docker Desktop creates a Linux VM to run Docker Engine
    • macOS: uses Virtualization.framework (Apple Silicon) or QEMU (Intel)
    • Windows: uses WSL 2 backend (recommended) or Hyper-V backend
  • used only during development; production uses Docker Engine on a Linux server directly

Docker Daemon

Container runtime

RuntimeLayerNotes
containerdHigh-levelindustry standard; used by Kubernetes, Docker
CRI-OHigh-levelKubernetes-native OCI runtime
runcLow-leveldefault; spawns containers via namespaces/cgroups
crunLow-levelfaster C replacement for runc
gVisor (runsc)Sandboxedintercepts syscalls in userspace (Google)
Kata ContainersVM-basedeach container runs in a microVM

Tools

CLI with Runtimes

  • docker CLI
    • dockerd (moby)
  • nerdctl CLI
    • containerd (contaiNERD)
  • podman CLI
    • runc, crun, runv (or any other OCI compliant runtime)
    • daemonless and rootless by default
    • each container is a direct child process (no central daemon)
    • podman commands are drop-in replacements for docker

Management

  • Docker desktop
  • Rancher desktop
  • Podman desktop

Linting Dockerfile

  • Hadolint

Rootless Containers

  • run daemon/containers as non-root using user namespaces
  • container root maps to an unprivileged host user
    • smaller blast radius if a container is compromised

Linux Kernel Primitives

  • The three kernel features that make containers possible

Namespaces (isolation)

  • Namespace wraps a global kernel resource so each container sees its own isolated copy
  • runc calls clone() / unshare() syscalls to create new namespaces per container
  • without namespaces, ps inside a container would show all host processes
  • Linux has 8 namespace types:
# list all namespaces of a running container (get PID first)
docker inspect --format '{{.State.Pid}}' <container>
lsns -p <pid>
 
# see namespace files kernel exposes per process
ls -la /proc/<pid>/ns/
 
# enter a container's network namespace from the host
nsenter -t <pid> -n ip addr
 
# demo: hostname isolation via uts namespace
unshare --uts bash
hostname isolated-test   # only visible inside this shell
NamespaceIsolates
pidprocess IDs — container PID 1 host PID
netnetwork interfaces, routing tables, ports
mntfilesystem mount points
utshostname and domain name
ipcSystem V IPC, POSIX message queues
userUID/GID mappings (enables rootless containers)
cgroupcgroup root view
timesystem clock offsets (Linux 5.6+)

cgroups (resource limits)

  • Control Groups
  • It is a kernel mechanism to limit, account for, and isolate resource usage of process groups
  • Two versions in use:
    • cgroups v1: per-resource hierarchy (separate trees for cpu, memory, etc.)
    • cgroups v2: unified hierarchy — one tree for all resources; preferred since Linux 4.5
  • resources controlled:
    • CPU shares / quota
    • memory limit + OOM kill threshold
    • block I/O weight
    • device access
  • Docker maps --memory, --cpus, --cpu-shares flags directly to cgroup entries under /sys/fs/cgroup/
# run a container with resource limits
docker run --memory=256m --cpus=0.5 nginx
 
# inspect the cgroup limit Docker wrote (cgroups v2)
cat /sys/fs/cgroup/system.slice/docker-<full-id>.scope/memory.max
cat /sys/fs/cgroup/system.slice/docker-<full-id>.scope/cpu.max
 
# live resource usage
docker stats <container>
 
# demo: spawn a process in a new cgroup manually (cgroups v2)
mkdir /sys/fs/cgroup/demo
echo 52428800 > /sys/fs/cgroup/demo/memory.max   # 50 MB
echo $$ > /sys/fs/cgroup/demo/cgroup.procs        # add current shell

OverlayFS (layered images)

  • It is union filesystem that merges multiple directory trees into one view
  • Docker uses it as the default storage driver (overlay2)
  • Two layers per container:
    • lower dir: read-only image layers stacked on top of each other
    • upper dir: read-write container layer (copy-on-write)
    • merged view: what the container actually sees
  • How copy-on-write works:
    • reading a file: served directly from the lower layer
    • writing a file: file is first copied up to the upper layer, then modified
    • deleting a file: a whiteout file is created in the upper layer to mask it
  • Benefits:
    • image layers are shared across containers (no duplication on disk)
    • fast container start - no full filesystem copy needed
  • storage driver alternatives: btrfs, zfs
# confirm storage driver in use
> docker info | grep 'Storage Driver'
Storage Driver: overlay2
 
# inspect OverlayFS mounts for a running container
> docker inspect --format '{{json .GraphDriver}}' <container> | jq
{
  "Data": {
    "LowerDir": "/var/lib/docker/overlay2/6051c4...9af0/diff",
    "MergedDir": "/var/lib/docker/overlay2/8c35512...f469/merged",
    "UpperDir": "/var/lib/docker/overlay2/8c35512...f469/diff",
    "WorkDir": "/var/lib/docker/overlay2/8c3551...46cf469/work"
  },
  "Name": "overlay2"
}
 
# see the actual merged filesystem on the host
ls /var/lib/docker/overlay2/<layer-id>/merged/
 
# list image layers (each line = one OverlayFS lower dir)
docker history <image>
 
# demo: manual overlayfs mount
mkdir lower upper work merged
echo 'from image' > lower/file.txt
mount -t overlay overlay -o lowerdir=lower,upperdir=upper,workdir=work merged
echo 'modified' > merged/file.txt   # copy-on-write: written to upper/, lower/ unchanged

Backlinks