In a Dockerfile, is there a way to avoid copying files to make them accessible to the RUN command?

Question

I need to build a Docker image containing a pre-populate a database. For now, I am using the following commands in our Dockerfile:

COPY db-dump.gz /tmp
RUN zcat /tmp/db-dump.gz | mysql

But is there a way to achieve the same result without copying first the (potentially large) dump into the container?

At shell-level, I could pipe the data into the container using something along the lines of (untested):

zcat db-dump.gz | docker exec -i $CID mysql

But I don't know if there is any similar option available in a Dockerfile.

score 6 · Accepted Answer · answered Jan 24 '19 at 13:43

You can do this with BuildKit and the experimental frontend. As of 18.09, BuildKit can be enabled with either:

export DOCKER_BUILDKIT=1

for a single shell, or to change the default for the host, you can add to /etc/docker/daemon.json:

{
  "features": {"buildkit": true}
}

You'll need to reload the docker engine after changing the above.

With BuildKit enabled, your Dockerfile would look like:

# syntax=docker/dockerfile:experimental

RUN --mount=type=bind,source=db-dump.gz,target=/tmp/db-dump.gz \
    zcat /tmp/db-dump.gz | mysql

You then build with a docker build . command as before, but with BuildKit enabled.

To see more about BuildKit's experimental features, see https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/experimental.md

score 1 · Answer 2 · answered Jan 24 '19 at 08:26

But is there a way to achieve the same result without copying first the (potentially large) dump into the container?

The only way that ables you to do that (which I know), is :

RUN gunzip < $(curl -SL http://docker-localhost/db-dump.gz) \
        | mysql

For this you need to have your dump available on a website, which isn't ideal (probably not as fast as what you are doing) If you look at the good Dockerfiles (public ones), they mostly get archive files from the web, then remove the archives.

Your commands in Dockerfile seem very good. You could just add a && rm /tmp/db-dump.gz after mysql.

score 1 · Answer 3 · answered Jan 24 '19 at 13:32

Another idea would be to mount a volume which would give the container access to a directory on the local filesystem. This would allow you to access the zip without copying the archive into the container; however, you wouldn't be able to ship the container with the data.

In a Dockerfile, is there a way to avoid copying files to make them accessible to the RUN command?

3 Answers3