32

I'm looking into rate-limiting using nginx's HttpLimitReqModule. However, requests are all coming from the same IP (a loadbalancer), with the real IP address in the headers.

Is there a way to have nginx rate-limit based on the ip in the X-Forwarded-For header instead of the ip of the source?

John Brodie
  • 423

4 Answers4

36

Yes, typical rate-limiting configuration definition string looks like:

 limit_req_zone  $binary_remote_addr zone=zone:16m rate=1r/s;

where $binary_remote_addr is the unique key for limiter. You should try changing it to $http_x_forwarded_for variable which gets the value of X-Forwarded-For header. Although this will increase memory consumption because $binary_remote_addr is using compressed binary format for storing IP addresses and $http_x_forwarded_for is not.

 limit_req_zone  $http_x_forwarded_for zone=zone:16m rate=1r/s;
markdsievers
  • 103
Andrei Mikhaltsov
  • 3,097
6

The limit_req_zone directive defines the variable to be used as key for request grouping.
Usually, the $binary_remote_addr is used rather than $remote_addr because it is smaller and saves space.

Maybe you alternatively want to use the ngx_http_realip_module.
This will rewrite the remote address variables to the address provided in a custom header and will also make logging and other variable usage easier.

aland
  • 182
Lukas
  • 1,024
  • 6
  • 14
2

It's best to use the realip module in this case:

limit_req_zone $binary_remote_addr zone=one:10m rate=12r/m;

server {
    real_ip_header     X-Forwarded-For;
    # docker (except for docker swarm supposedly)
    set_real_ip_from   172.16.0.0/12;
    set_real_ip_from   192.168.0.0/16;
    # cloudflare (https://www.cloudflare.com/ips-v4)
    set_real_ip_from   173.245.48.0/20;
    set_real_ip_from   103.21.244.0/22;
    set_real_ip_from   103.22.200.0/22;
    set_real_ip_from   103.31.4.0/22;
    set_real_ip_from   141.101.64.0/18;
    set_real_ip_from   108.162.192.0/18;
    set_real_ip_from   190.93.240.0/20;
    set_real_ip_from   188.114.96.0/20;
    set_real_ip_from   197.234.240.0/22;
    set_real_ip_from   198.41.128.0/17;
    set_real_ip_from   162.158.0.0/15;
    set_real_ip_from   104.16.0.0/13;
    set_real_ip_from   104.24.0.0/14;
    set_real_ip_from   172.64.0.0/13;
    set_real_ip_from   131.0.72.0/22;
    real_ip_recursive  on;


location / {
    limit_req zone=one burst=5;
    proxy_pass http://backend;
    # proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;  # nginx < 1.9.7, it will add the overriden $remote_addr to X-Forwarded-For
    proxy_set_header  X-Forwarded-For  "$http_x_forwarded_for, $realip_remote_addr";  # nginx >= 1.9.7
}



}

You can find a proof-of-concept here.

The $proxy_add_x_forwarded_for issue. The difference between real_ip_recursive = on and real_ip_recursive = off.

x-yuri
  • 2,526
0

Just as an example and demonstration of setting up real_ip using the header.

  • We precise a header to take the ip from

  • The client ip will be then overridden with that value

  • Then we can keep using $binary_remote_addr

    • not always x-forwarded-for is used.

  • That method can help too with logs (default format will work as $remote_addr now it's set to the overridden value)

  • Nginx Module to use: ngx_http_realip_module Example of setting that up with Cloudflare (more than one server source, or ip range)

# Ipv4
set_real_ip_from 173.245.48.0/20;
set_real_ip_from 103.21.244.0/22;
set_real_ip_from 103.22.200.0/22;
set_real_ip_from 103.31.4.0/22;
set_real_ip_from 141.101.64.0/18;
set_real_ip_from 108.162.192.0/18;
set_real_ip_from 190.93.240.0/20;
set_real_ip_from 188.114.96.0/20;
set_real_ip_from 197.234.240.0/22;
set_real_ip_from 198.41.128.0/17;
set_real_ip_from 162.158.0.0/15;
set_real_ip_from 104.16.0.0/13;
set_real_ip_from 104.24.0.0/14;
set_real_ip_from 172.64.0.0/13;
set_real_ip_from 131.0.72.0/22;

Ipv6


set_real_ip_from 2400:cb00::/32;
set_real_ip_from 2606:4700::/32;
set_real_ip_from 2803:f800::/32;
set_real_ip_from 2405:b500::/32;
set_real_ip_from 2405:8100::/32;
set_real_ip_from 2a06:98c0::/29;
set_real_ip_from 2c0f:f248::/32;


real_ip_header CF-Connecting-IP;
  • you can have that in a file. ex: cloudflare_ip.conf and add it to conf.d

You can see the usage of

  • set_real_ip_from

    • only the requests coming from those ip's, the overriding will be applied to

  • real_ip_header

  • The header containing the client ip connecting through Cloudflare is CF-Connecting-IP

  • Cloudflare guide

And for the rate limit

limit_req_zone $binary_remote_addr zone=limit_ip_10rs:10m rate=10r/s;
  • We keep using $binary_remote_addr
Mohamed Allal
  • 111