GeekCoding101 make your way to geek

Docker Notes: Multi-Stage Builds, Save/Load & Registry

Docker notes: multi-stage Dockerfile, docker commit, save/load vs export/import, a private registry with registry:2, and where images are stored.

8 min read
Docker Notes
On this pageBuild a Docker image

This is my Docker reference: how to build an image (with a Dockerfile or docker commit), move it to another machine (docker save / docker load), host it on your own private registry, and find where Docker keeps images on disk. The short version: build with a multi-stage Dockerfile, ship images with docker save | docker load or a registry, and run docker info to see where they live (usually /var/lib/docker on Linux).

Hi there! This is yet another note from me ^^ I’ve been working with container technologies for years, and it’s a good habit to dump my notes here. These were first written in March 2018 and have been refreshed for current Docker. I hope you find them useful as well.

Build a Docker image

Method 1: docker build with a Dockerfile

A Dockerfile is the standard, reproducible way to build an image. In it you define:

  • the base image to start from,
  • the files to copy in,
  • the configuration steps to run,
  • the process to start.

You know I like using Django for projects. Here is a multi-stage Dockerfile based on a Cookiecutter Django local setup. The first stage builds Python wheels. The second stage installs those wheels, so the build tools never land in the final image.

# define an alias for the specific python version used in this file.
FROM python:3.11.6-slim-bullseye AS python
# Python build stage
FROM python AS python-build-stage
ARG BUILD_ENVIRONMENT=local
# Install apt packages
RUN apt-get update && apt-get install --no-install-recommends -y \
# dependencies for building Python packages
build-essential \
# psycopg2 dependencies
libpq-dev
# Requirements are installed here to ensure they will be cached.
COPY ./requirements .
# Create Python Dependency and Sub-Dependency Wheels.
RUN pip wheel --wheel-dir /usr/src/app/wheels \
-r ${BUILD_ENVIRONMENT}.txt
# Python 'run' stage
FROM python AS python-run-stage
ARG BUILD_ENVIRONMENT=local
ARG APP_HOME=/app
ENV PYTHONUNBUFFERED=1
ENV PYTHONDONTWRITEBYTECODE=1
ENV BUILD_ENV=${BUILD_ENVIRONMENT}
WORKDIR ${APP_HOME}
# devcontainer dependencies and utils
RUN apt-get update && apt-get install --no-install-recommends -y \
sudo git bash-completion ssh vim
RUN echo "alias ls='ls --color=auto'" >> ~/.bashrc
RUN echo "alias ll='ls -lh --color=auto'" >> ~/.bashrc
# Create devcontainer user and add it to sudoers
RUN groupadd --gid 1000 dev-user \
&& useradd --uid 1000 --gid dev-user --shell /bin/bash --create-home dev-user \
&& echo dev-user ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/dev-user \
&& chmod 0440 /etc/sudoers.d/dev-user
# Install required system dependencies
RUN apt-get update && apt-get install --no-install-recommends -y \
# psycopg2 dependencies
libpq-dev \
# Translations dependencies
gettext \
# cleaning up unused files
&& apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \
&& rm -rf /var/lib/apt/lists/*
# All absolute dir copies ignore workdir instruction. All relative dir copies are wrt to the workdir instruction
# copy python dependency wheels from python-build-stage
COPY --from=python-build-stage /usr/src/app/wheels /wheels/
# use wheels to install python dependencies
RUN pip install --no-cache-dir --no-index --find-links=/wheels/ /wheels/* \
&& rm -rf /wheels/
COPY ./compose/production/django/entrypoint /entrypoint
RUN sed -i 's/\r$//g' /entrypoint
RUN chmod +x /entrypoint
COPY ./compose/local/django/start /start
RUN sed -i 's/\r$//g' /start
RUN chmod +x /start
COPY ./compose/local/django/celery/worker/start /start-celeryworker
RUN sed -i 's/\r$//g' /start-celeryworker
RUN chmod +x /start-celeryworker
COPY ./compose/local/django/celery/beat/start /start-celerybeat
RUN sed -i 's/\r$//g' /start-celerybeat
RUN chmod +x /start-celerybeat
COPY ./compose/local/django/celery/flower/start /start-flower
RUN sed -i 's/\r$//g' /start-flower
RUN chmod +x /start-flower
# copy application code to WORKDIR
COPY . ${APP_HOME}
ENTRYPOINT ["/entrypoint"]

Build and tag it from the project root:

Terminal window
docker build -t myproject/django:dev .

The sed -i 's/\r$//g' lines strip Windows line endings from the start scripts. Without them, a script saved on Windows fails inside the Linux container.

Method 2: docker commit

The other way is to turn a running (or stopped) container into an image:

Terminal window
docker commit <container_id> <new_image_name>:<tag>

This takes the container’s current filesystem changes, on top of the image it was started from, and saves them as a new image in your local Docker storage. It’s handy for quick experiments. It isn’t reproducible, though: nobody can see how the image was made. For anything you’ll keep, write a Dockerfile.

Move images between machines: save/load vs export/import

Once you have an image, you’ll often want to share it or copy it to a machine without registry access. Docker has two pairs of commands for this, and they are not the same.

docker save / docker load works on images and keeps layers, tags and history:

Terminal window
docker save <image_name:version> > exported_file.tar
docker load < exported_file.tar

Compress it, or stream it straight to another host over SSH:

Terminal window
docker save <image_name:version> | gzip > exported_file.tar.gz
docker load < exported_file.tar.gz
docker save <image_name:version> | ssh user@other-host docker load

docker export / docker import works on a container’s filesystem and produces a single flattened image:

Terminal window
docker export <container_id> > container_fs.tar
docker import container_fs.tar <new_image_name>:<tag>
docker save / docker loaddocker export / docker import
Works onImagesContainers
Layers and historyKeptFlattened into one layer
TagsKeptYou set a new name on import
Image config (CMD, ENTRYPOINT, ENV)KeptLost (can be re-added with docker import -c)
Typical useCopying an image to an offline machineSnapshotting a container’s filesystem

Run a private Docker registry

My original 2018 notes installed the old docker-registry package from Docker’s yum repo on CentOS 7.2. That package, that repo and the v1 registry API are all retired. Today the simple way is to run the official registry image as a container.

Start the registry

Terminal window
docker run -d \
-p 5000:5000 \
--restart=always \
--name registry \
-v /data/docker/registry:/var/lib/registry \
registry:2

The -v mount plays the role of the old storage_path setting. Pushed images are stored under /data/docker/registry on the host, so they survive a container restart or upgrade.

Verify the registry

Terminal window
curl http://localhost:5000/v2/_catalog

On a fresh registry you should get:

{"repositories":[]}

You can also check the container with docker ps --filter name=registry and docker logs registry.

Set up clients to use the registry

Docker treats localhost:5000 as trusted out of the box. Other machines reaching the registry over plain HTTP must list it as an insecure registry. Add it to /etc/docker/daemon.json, merging with any settings already there:

{
"insecure-registries": ["your_ip_or_hostname:5000"]
}

Then restart Docker:

Terminal window
sudo systemctl restart docker

An insecure registry is only suitable for a lab or a trusted internal network. For anything else, put TLS and authentication in front of it.

Push to the registry

First get an image to push, for example from Docker Hub:

Terminal window
docker pull alpine

If you push a name that doesn’t exist locally yet, you get an error:

# docker push your_ip_or_hostname:5000/ci
The push refers to a repository [your_ip_or_hostname:5000/ci]
An image does not exist locally with the tag: your_ip_or_hostname:5000/ci

The image name has to include the registry address. So tag the image with your registry’s address first, then push:

Terminal window
docker tag alpine your_ip_or_hostname:5000/alpine
docker push your_ip_or_hostname:5000/alpine

Check that it arrived, then pull it from any configured client:

Terminal window
curl http://your_ip_or_hostname:5000/v2/_catalog
docker pull your_ip_or_hostname:5000/alpine

Docker storage

Where does Docker store images?

On Linux it’s usually /var/lib/docker/. The exact layout inside depends on the storage driver Docker uses. Ask Docker directly instead of guessing:

Terminal window
docker info --format '{{.DockerRootDir}}' # root directory, e.g. /var/lib/docker
docker info --format '{{.Driver}}' # storage driver, e.g. overlay2
docker system df # space used by images, containers, volumes
SetupWhere images live
Docker Engine on Linux/var/lib/docker/ by default. With the overlay2 driver, layer contents are under /var/lib/docker/overlay2/
Docker Desktop (macOS, Windows)Inside the Linux VM that Docker Desktop runs, stored as a disk image file, not directly on your host filesystem

To move Docker’s data to another disk, set data-root in /etc/docker/daemon.json and restart Docker. To pick a storage driver, use the storage-driver key in the same file, or the --storage-driver daemon option.

{
"data-root": "/data/docker",
"storage-driver": "overlay2"
}

Older Docker versions used drivers like aufs, with layer contents in /var/lib/docker/aufs/diff/<id> and metadata in /var/lib/docker/graph/<id>. That layout is legacy. You’ll only see it on old hosts.

Don’t edit files under /var/lib/docker by hand. Use docker rmi, docker image prune and docker system prune to clean up.

Cheat sheet

CommandWhat it does
docker versionClient and server versions
docker infoDaemon details: storage driver, root dir, registries
docker imagesList local images
docker ps -aList containers, including stopped ones
docker rmi <image>Remove an image
docker run -it ubuntu bashStart an interactive container
docker run -d ubuntu /bin/sh -c "while true; do echo hello world; sleep 1; done"Start a detached container
docker exec -it <container> bashOpen a shell in a running container
docker stop <container>Stop a container
docker inspect <container or image>Show full JSON details
docker tag <source_image> <target_image>Add a new name/tag to an image
docker logs -f <container>Follow a container’s logs
docker system dfDisk usage summary
docker image pruneRemove dangling images

FAQ

What’s the difference between docker save and docker export? docker save exports an image with all its layers, tags and config, and you restore it with docker load. docker export dumps a container’s filesystem as one flat tarball, and docker import turns that into a new single-layer image without the original history or config.

Should I use docker commit or a Dockerfile? Use a Dockerfile for anything you’ll rebuild, share or deploy, because it records exactly how the image is built. docker commit is fine for a quick snapshot while experimenting.

Why does docker push say “An image does not exist locally with the tag”? The image name must include the registry address. Run docker tag <image> your_ip_or_hostname:5000/<name> first, then push that exact name.

How do I see which images are in my private registry? Query the registry API: curl http://your_ip_or_hostname:5000/v2/_catalog lists repositories, and curl http://your_ip_or_hostname:5000/v2/<name>/tags/list lists the tags for one repository.

Where are Docker images stored on a Mac? With Docker Desktop, images live inside the Linux VM’s disk image, not in a normal folder on macOS. Use docker system df to see how much space they take and docker image prune to free it.

Okay, that’s all from me. Thanks for reading! Questions or corrections are welcome via the contact page.

Thanks for reading my blog post! Feel free to check out my other posts or contact me via the social links in the footer.

Keep reading

All posts