TopolearnIt explains layer sharing, slow first writes to big files and why data belongs in volumes.
A container's root filesystem is a union mount: several directories stacked so they appear as one. On Linux, Docker uses the kernel's overlayfs, either through the overlay2 storage driver or, with the containerd image store, containerd's overlayfs snapshotter. docker info shows which.
An overlay mount has:
- lowerdir: one or more read-only directories, the image layers, stacked top to bottom.
- upperdir: the container's writable layer.
- workdir: scratch space the kernel needs for atomic operations.
- merged: the unified view that becomes the container's
/.
Reads fall through the stack: the topmost layer that has a file wins. Because lower layers are read-only and identified by content, a hundred containers from the same image share one copy on disk and in the page cache.
Writes follow copy-on-write:
- Modifying a file from a lower layer first triggers a copy-up: the whole file is copied into upperdir, then modified. The first write to a 2 GB file copies 2 GB. That's one reason databases and other write-heavy data belong on volumes, which bypass the overlay entirely.
- Deleting a lower file creates a whiteout in upperdir (a 0/0 character device) that hides it; the data still exists below.
- Replacing a directory uses an opaque marker so lower contents are hidden.
Image layers work the same way at build time: each layer tarball stores additions plus whiteout entries (.wh.<name> in the tar format). That's why deleting files in a later Dockerfile step never shrinks an image.
Inspect a container's overlay with docker inspect -f '{{json .GraphDriver.Data}}' (overlay2 driver) or mount | grep overlay.
Example
docker info --format '{{.Driver}}' # overlay2 (or overlayfs via containerd)
docker run -d --name t alpine sleep 1d
docker inspect t --format '{{json .GraphDriver.Data}}' | tr ',' '\n'
# "LowerDir":"/var/lib/docker/overlay2/<id>-init/diff:/var/lib/docker/overlay2/<layer>/diff"
# "MergedDir":".../merged" "UpperDir":".../diff" "WorkDir":".../work"
docker exec t sh -c 'echo x >> /etc/hosts.allow; rm /etc/motd'
UPPER=$(docker inspect t --format '{{.GraphDriver.Data.UpperDir}}')
sudo ls -l "$UPPER/etc"
# -rw-r--r-- ... hosts.allow <- copied up, then modified
# c--------- 0, 0 ... motd <- whiteout hiding the lower file
docker diff t # C /etc A /etc/hosts.allow D /etc/motdRemember
- lowerdir = read-only image layers, upperdir = container layer, merged = /
- First write to a lower file copies the whole file up
- Deletes are whiteouts; the data stays in the lower layer
- Write-heavy data belongs on volumes, outside the overlay
Watch out for
- Storing a large, frequently written file (like a SQLite DB) in the image and paying copy-up plus overlay overhead in every container.