How can I make docker-compose pull images using a socks5 proxy?

Question

I'm trying to pull images through a socks5 proxy. I've started the socks5 proxy on port 8888 like this:

ssh -NC -D 8888 parham@***.***.**

However, when I run docker-compose like this, it is evident that it's not using the environment variable:

$ ALL_PROXY=socks5://127.0.0.1:8888 docker-compose up

How can I make Docker pull images through this proxy?

Answer 1

Looks like this feature has been recently added to docker, particularly in version 1.11.0. Though there isn't enough documentation about its usage, setting the ALL_PROXY environment variable should work, according to this comment.

$ export ALL_PROXY=socks5://localhost:port 
$ docker pull image

I'm not sure about docker-compose at the moment. If your primary aim is to speed up the download process using socks5, you might be able to achieve the purpose by manually pulling images using docker pull before finally running docker-compose up.

Answer 2

Method 1

Error:

docker-compose up 
Pulling web (jitsi/web:stable-6433)...
ERROR: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

then setting HTTP_PROXY and HTTPS_PROXY this way:

export HTTP_PROXY=socks5://localhost:9040/
export HTTPS_PROXY=socks5://localhost:9040/

result:

docker-compose up 
Pulling web (jitsi/web:stable-6433)...
stable-6433: Pulling from jitsi/web
b380bbd43752: Downloading [===================>                               ]  10.36MB/27.14MB
0312863d422f: Download complete
503063a74a58: Downloading [============================>                      ]  7.998MB/14.15MB
392a862abfe3: Download complete
41a073b705ed: Download complete
a8f4844f6941: Download complete
f500f1f2f4ae: Downloading [============>                                      ]  11.69MB/47.56MB

A shot of before HTTP_PROXY and after setting them up:

Method 2

If method 1 did not work (simple one) you can go with this one:

first

Find systemd location for configuration:

systemctl status docker | grep docker.service\;
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)

which here for my on Debian 9 is /lib/systemd/system/docker.service and its directory is /lib/systemd/system/
So cd into /lib/systemd/system/.

second

mkdir for docker.service.d or:

mkdir docker.service.d && cd !$

third

In docker.service.d directory create a .conf file like: http-proxy.conf

forth

Add these in http-proxy.conf file:

[Service]
Environment="HTTP_PROXY=socks5://localhost:9040"
Environment="HTTPS_PROXY=socks5://localhost:9040"

NOTE that localhost:9040 is mine, update it if you have a different IP or PORT

fifth

Because we update the configuration run:

systemctl daemon-reload

and then restart docker service

systemctl restart docker

Now it is up and running:

Answer 3

@nikhilweee's answer above is missing a crucial point.

The proxy used for image-pull is taken from the environment of dockerd. There is presently no way to change it without restarting dockerd.

I have filed moby issue #37335 about it, because I wanted to use a specific proxy for a single pull command, but not in general.

Answer 4

In Docker 17.07 and higher, you can also use json file for proxy configuration:

       $ vim ~/.docker/config.json
      {
        "proxies":
          {
            "default":
             {
              "httpProxy": "http://127.0.0.1:8118",
              "httpsProxy": "http://127.0.0.1:8118"
              }
          }
      }

    $  docker pull ubuntu:18.04
    $  docker-compose build

More info in official docs.

And in addition to use Socks, You can forward Socks to HTTP by using privoxy:

    $ sudo apt install -y privoxy
    $ sudo vim /etc/privoxy/config
          forward-socks5 / 127.0.0.1:9090  .
          forward-socks4a / 127.0.0.1:9090 .
          forward-socks5t             /     127.0.0.1:9090 .
          forward         192.168.*.*/     .
          forward            10.*.*.*/     .
          forward           127.*.*.*/     .
          forward           localhost/     .

$ sudo service privoxy restart

More info in official docs.

Answer 5

Took me ages to figure this out. The answer here are mixing things. To pull an image, as of version 27.1.1, you need to configure the proxy in the daemon, not the CLI.

case explanation: From one machine I could not access a registry, as the IP was blocked and all packets where silently dropped, so I needed a proxy/tunnel to a machine that had no issues.

Step by step explanation for one time image download

1. Set a simple SOCKS proxy with ssh in the background: ssh -NfD 8888 user@remote_machine
2. Export the env variable: export ALL_PROXY=socks5://localhost:8888
3. Stop the daemon (the containers will still be running, you just stop the "container management" daemon): e.g. systemctl stop docker.socket
4. Start the daemon manually in this terminal, where the env var got exported (leave this open, to view the logs in case something is different).
5. Open a second terminal and pull the image and/or login as necessary. (furthermore you can check all containers are still running and where never stopped).
6. When you are finished, just stop the dockerd (ctrl-c in the terminal) and restart the service: systemctl start docker.socket).
7. you can now stop the ssh socks server

If you need this to be a permanent solution, configure containerd as mentioned in the docs, take care that according to the proxy daemon doc the proxy keys uses snake case, instead of camel case, as in the CLI config. (so it would be all-proxy, and not allProxy). Not sure if both work...

If you need this now and then, put it in a script (e.g. proxy.sh)

#!/bin/bash
remote=remote_host
port=8888
echo "Will run $@ via $remote:$port"
echo "Setting up SOCKS proxy:"
ssh -ND $port $remote &
ssh_pid=$!
export ALL_PROXY="socks5://localhost:$port"
echo "Restarting the daemon"
systemctl stop docker.socket
dockerd &
dockerd_pid=$!
echo "Waiting dockerd to come up"
sleep 5
echo "(issuing) $@"
$@
echo "Stopping dockerd"
kill $dockerd_pid
sleep 1
echo "Restarting the docker socket"
systemctl start docker.socket
echo "Stopping the socks proxy"
kill $ssh_pid

(don't forget to chmod +x proxy.sh)

And you'll use it like:

proxy.sh docker pull the.unreachable.registry/image:tag