120

I am using nginx/0.7.68, running on CentOS, with the following configuration:

server {
    listen       80;
    server_name ***;
    index index.html index.htm index.php default.html default.htm default.php;

    location / {
            root   /***;
            proxy_pass   http://***:8888;
            index  index.html index.htm;
    }
    # where *** is my variables

The proxy_pass is to a DNS record whose IP changes frequently. Nginx caches the outdated IP address, resulting in a request to the wrong IP address.

How can I stop nginx from caching the IP address, when it is outdated?

Jordan Stewart
  • 241
xiamx
  • 1,392

7 Answers7

171

Accepted answer didn't work for me on nginx/1.4.2.

Using a variable in proxy_pass forces re-resolution of the DNS names because NGINX treats variables differently to static configuration. From the NGINX proxy_pass documentation:

Parameter value can contain variables. In this case, if an address is specified as a domain name, the name is searched among the described server groups, and, if not found, is determined using a resolver.

For example:

server {
    ...
    resolver 127.0.0.1;
    set $backend "http://dynamic.example.com:80";
    proxy_pass $backend;
    ...
}

Note: A resolver (i.e. the name server to use) MUST be available and configured for this to work (and entries inside a /etc/hosts file won't be used in a lookup).

By default, version 1.1.9 or later versions of NGINX cache answers using the TTL value of a response and an optional valid parameter allows the cache time to be overridden:

resolver 127.0.0.1 [::1]:5353 valid=30s;

Before version 1.1.9, tuning of caching time was not possible, and nginx always cached answers for the duration of 5 minutes..

ohaal
  • 2,462
17

There is valuable information in gansbrest comment and ohaal answer.

But I think it's important to mention this official nginx article, posted in 2016, it clearly explains nginx behaviour on this matter and the possible solutions: https://www.nginx.com/blog/dns-service-discovery-nginx-plus/

We indeed have to "Set the Domain Name in a Variable" and use the resolver directive.

however, using a variable changes the rewrite behaviour. You may have to use the rewrite directive, it depends on your location and proxy_pass setup.

PS: would have post a comment but not enough points yet...

Jack B.
  • 171
  • 1
  • 3
13

ohaal's answer takes most of us there, but there is a case where the DNS resolver does not live at 127.0.0.1 (eg when you're in a special containerized environment)

In that case, you may want to change the nginx conf to resolver ${DNS_SERVER};. Then, before you start nginx, run

export DNS_SERVER=$(cat /etc/resolv.conf |grep -i '^nameserver'|head -n1|cut -d ' ' -f2)
envsubst '${DNS_SERVER}' < your_nginx.conf.template > your_nginx.conf

Note, that you need the gettext package installed, as that provides the envsubst command.

Moritur
  • 103
wonton
  • 231
12

It's an intriguing question and AFAIK that's not going to work well. You can try to use the upstream module and use the directives for failover to see if it works as a hack.

2018 edit: a lot of things changed. Check the answer by @ohaal to get real information about this.

Anon
  • 1,293
coredump
  • 12,921
4

As of nginx version 1.27.3 non-enterprise versions can also use resolve parameter on server inside upstream block to make domains re-resolve.

resolver 8.8.8.8

upstream http_backend {
    zone upstream_dynamic 64k;
    server custom.domain.tld:443 resolve;


keepalive 16;



}


server {
    ...


location /http/ {
    proxy_pass https://http_backend;
    proxy_http_version 1.1;
    proxy_set_header Connection &quot;&quot;;
    ...
}



}

In order for this parameter to work, the resolver directive must be specified in the http block or in the corresponding upstream block.

Paku
  • 141
2

I've hacked together a script to watch a conf.d folder upstreams for dns changes and reload nginx upon detection. It's a first pass, and surely can be improved (next pass, I'll use nginx -T to parse upstreams specifically. Same idea could be used for proxy_pass directives):

#!/bin/bash

get_upstreams() {
  local files=$@
  grep -hEo '(server\s+)[^:;]+' $files | cut -d' ' -f 2
}

resolve_hosts() {
  local hosts=$@
  for h in $hosts; do dig +short $h; done | sort -u
}

watch_dir=$1

[ -d $watch_dir ] || exit 2

upstreams=$(get_upstreams $watch_dir/*)
ips=$(resolve_hosts $upstreams)
if [ ! "$ips" ]; then
  echo "Found no resolvable hosts in $watch_dir files."
fi

host_hash=$(echo $ips | /usr/bin/sha512sum)

echo $host_hash
echo $ips

while [ -d $watch_dir ]; do
  sleep 30
  upstreams=$(get_upstreams $watch_dir/*)
  ips=$(resolve_hosts $upstreams)
  new_hash=$(echo $ips | /usr/bin/sha512sum)
  if [ "$host_hash" != "$new_hash" ]; then
    echo Detected an upstream address change.  $ips
    echo Reloading nginx
    echo $new_hash
    echo $ips
    /sbin/service nginx reload
    host_hash=$new_hash
  fi
done
mushuweasel
  • 121
1

The most voted answer from @ohaal didn't work for me. I had to use a variant of the proposed solution:

resolver 127.0.0.11 ipv6=off valid=10s;

location ~ ^/SomePath/(.*)$ {
    proxy_pass http://dynamic.example.com:80/SomePath/$1$is_args$args;
}


location ~ ^/Other/Path/(.*)$ {
    proxy_pass http://dynamic.example.com:80/Other/Path/$1$is_args$args;
}

Unlike the other solution, this works also of for urls like:

http://proxied:80/SomePath/bla
http://proxied:80/SomePath/bla/bla?whatever=123
http://proxied:80/Other/Path/bla
http://proxied:80/Other/Path/bla/bla

See https://stackoverflow.com/a/8130872/948938

Juan Ignacio Barisich
  • 111
  • 4