Preloading System Containers with Inner Containers

November 11, 2019

Introduction

As described in this blog post, using Nestybox system containers as Docker sandboxed environments can be very useful.

The software that makes this possible is Nestybox’s Sysbox container runtime, which integrates with Docker and enables it to deploy system containers.

One of the novel features of Sysbox is that it allows users to use Docker to build system container images that come pre-loaded with inner container images. And it’s easily done via a Dockerfile and docker build command.

Once you’ve build a system container that includes inner container images, deployment of that system container voids the need for the Docker instance running within the system container to pull those images from the network.

This article shows an example of how do to this.

See it work!

Contents

• About Sysbox
• Docker Engine Configuration
• Pre-loading a System Container with Inner Container Images
• Using Docker Commit to Snapshot System Containers
• Conclusion
• Try it for free!

About Sysbox

If you want to try the examples that follow, you must first install the Nestybox system container runtime Sysbox in your machine.

Sysbox integrates with Docker, enabling it to build and deploy system containers just as you would any other Docker container. The difference is that within the system container, you can now run system level software that does not normally run on a Docker container, without resorting to the unsecure privileged mode or complex configurations.

You can get Sysbox for free at the Nestybox website. Once you install it, you simply deploy system containers with Docker as shown in the examples below.

Docker Engine Configuration

In order to build a system container image that comes pre-loaded with inner container images, we must first reconfigure the host’s Docker daemon to use the sysbox-runc runtime as it’s default runtime.

This is needed because the Docker build process must call into the Sysbox runtime during the build steps, as some of those steps require deploying a system container and running Docker inside of it to pull the inner images.

Unfortunately the docker build command does not currently take a --runtime option (unlike the docker run command which does). As a result, the Docker engine’s default runtime must be set to sysbox-runc, if only temporarily while the build takes place. Once the build has completed, we can revert the default runtime to its original setting.

Note: re-configuring the Docker engine is only needed when building a system container that comes preloaded with inner Docker container images. If you are building a system container that does not include inner container images, no reconfiguration of the Docker engine is required, and you would build the system container just like any other Docker container.

To reconfigure the Docker daemon, edit its /etc/docker/daemon.json file. Here is how the /etc/docker/daemon.json file should look like. Notice the line at the end of the file:

{
    "runtimes": {
        "sysbox-runc": {
            "path": "/usr/local/sbin/sysbox-runc"
        }
    },
    "default-runtime": "sysbox-runc"
}

Then restart the Docker daemon service:

$ systemctl restart docker.service

We are now ready to perform the build of the system container that includes inner images. The next section explains how this is done.

Pre-loading a System Container with Inner Container Images

First, we need a Dockerfile to build the system container image. At a high level, the Dockerfile needs to do the following:

1) Install Docker inside the system container (i.e., the inner Docker).

2) Request the inner Docker to pull the desired inner container images.

It’s pretty simple really. Here is an sample Dockerfile:

FROM alpine:latest

RUN apk update && apk add docker
COPY docker-pull.sh /usr/bin
RUN chmod +x /usr/bin/docker-pull.sh && docker-pull.sh && rm /usr/bin/docker-pull.sh

The key instruction in the Dockerfile shown above are the COPY and subsequent RUN instructions. Notice that they are copying a script called docker-pull.sh into the system container, executing it, and removing it.

The docker-pull.sh script is shown below:

#!/bin/sh

# dockerd start
dockerd > /var/log/dockerd.log 2>&1 &
sleep 3

# pull inner images
docker pull busybox:latest
docker pull alpine:latest

# dockerd cleanup (remove the .pid file as otherwise it prevents
# dockerd from launching correctly inside sys container)
kill $(cat /var/run/docker.pid)
kill $(cat /run/docker/containerd/containerd.pid)
rm -f /var/run/docker.pid
rm -f /run/docker/containerd/containerd.pid

As shown, the script simply runs Docker inside the system container, pulls the inner container images (in this case the busybox and alpine images), and does some cleanup. Pretty simple.

The reason we need this script in the first place is because it’s hard to put all of these commands into a single Dockerfile RUN instruction. It’s simpler to put them in a separate script and call it from the RUN instruction.

Let’s see what happens when we execute docker build on this Dockerfile to build the system container image:

$ docker build -t nestybox/syscont-with-inner-containers:latest .

Sending build context to Docker daemon  3.072kB
Step 1/4 : FROM alpine:latest
 ---> 965ea09ff2eb
Step 2/4 : RUN apk update && apk add docker
 ---> Running in 145c0cd1df84
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
v3.10.3-19-g7f993019c4 [http://dl-cdn.alpinelinux.org/alpine/v3.10/main]
v3.10.3-13-g8068beb776 [http://dl-cdn.alpinelinux.org/alpine/v3.10/community]
OK: 10338 distinct packages available
(1/12) Installing ca-certificates (20190108-r0)
(2/12) Installing libseccomp (2.4.1-r0)
(3/12) Installing runc (1.0.0_rc8-r0)
(4/12) Installing containerd (1.2.9-r0)
(5/12) Installing libmnl (1.0.4-r0)
(6/12) Installing libnftnl-libs (1.1.3-r0)
(7/12) Installing iptables (1.8.3-r0)
(8/12) Installing tini-static (0.18.0-r0)
(9/12) Installing device-mapper-libs (2.02.184-r0)
(10/12) Installing docker-engine (18.09.8-r0)
(11/12) Installing docker-cli (18.09.8-r0)
(12/12) Installing docker (18.09.8-r0)
Executing docker-18.09.8-r0.pre-install
Executing busybox-1.30.1-r2.trigger
Executing ca-certificates-20190108-r0.trigger
OK: 278 MiB in 26 packages
Removing intermediate container 145c0cd1df84
 ---> 78995e9b92ae
Step 3/4 : COPY docker-pull.sh /usr/bin
 ---> d73a303d280d
Step 4/4 : RUN chmod +x /usr/bin/docker-pull.sh && docker-pull.sh && rm /usr/bin/docker-pull.sh
 ---> Running in 94802f58cbfd
latest: Pulling from library/busybox
0f8c40e1270f: Pulling fs layer
0f8c40e1270f: Verifying Checksum
0f8c40e1270f: Download complete
0f8c40e1270f: Pull complete
Digest: sha256:1303dbf110c57f3edf68d9f5a16c082ec06c4cf7604831669faf2c712260b5a0
Status: Downloaded newer image for busybox:latest
latest: Pulling from library/alpine
89d9c30c1d48: Pulling fs layer
89d9c30c1d48: Verifying Checksum
89d9c30c1d48: Download complete
89d9c30c1d48: Pull complete
Digest: sha256:c19173c5ada610a5989151111163d28a67368362762534d8a8121ce95cf2bd5a
Status: Downloaded newer image for alpine:latest
Removing intermediate container 94802f58cbfd
 ---> 136945f31870
Successfully built 136945f31870
Successfully tagged nestybox/syscont-with-inner-containers:latest

We can see from above that the Docker build process installed Docker inside the system container, and then used that inner Docker to pull the busybox and alpine container images.

The result is a system container image that has Docker, as well as busybox and alpine container images stored within it. Cool!

Once the build is complete, we can optionally revert the default-runtime config in the /etc/docker/daemon.json file we did earlier (it’s only needed for the Docker build, but not for running the system container).

Before proceeding, it’s a good idea to prune any dangling images created during the Docker build process to save storage.

$ docker image prune

Now to the fun part; let’s run the newly created system container image:

$ docker run --runtime=sysbox-runc -it --rm --hostname=syscont nestybox/syscont-with-inner-containers:latest
/ #

And let’s start Docker inside the system container:

/ # dockerd > /var/log/dockerd.log 2>&1 &

And let’s verify the inner container images are indeed pre-loaded within the system container:

/ # docker image ls
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
busybox             latest              020584afccce        6 days ago          1.22MB
alpine              latest              965ea09ff2eb        2 weeks ago         5.55MB

There they are, great!

Now let’s deploy one of those:

/ # docker run -it busybox
/ #

As expected, the busybox container runs without the need for Docker to pull its image from the network.

This is cool because it allows you to build a pre-configured Docker sandbox environment that comes with the inner images that you desire within it, voiding the need to download those from the network when the system container is running. All captured by a single, portable, and easy-to-deploy Nestybox system container.

Using Docker Commit to Snapshot System Containers

In the example above we used docker build to build a system container image that comes pre-loaded with inner images.

An alternative approach made possible by Sysbox is to use docker commit on a running system container image that includes inner container.

This Nestybox blog post has info on how to do this.

Conclusion

As shown above, using Docker + Nestybox it’s easy to build a system container image that comes pre-loaded with inner container images, with a simple Dockerfile.

This is cool because it allows you to build a pre-configured Docker sandbox environment that comes with the inner images that you desire, all captured by a single, portable, and easy-to-deploy Nestybox system container.

Try it for free!

You can try Sysbox for free! Check our website for info on how to get it.

We are looking for early adopters and your feedback would be much appreciated!