How can I use HAproxy with SSL and get X-Forwarded-For headers AND tell PHP that SSL is in use?

Question

I have the following setup:

(internet) ---> [  pfSense Box  ]    /-> [ Apache / PHP server ]
                [running HAproxy] --+--> [ Apache / PHP server ]
                                    +--> [ Apache / PHP server ]
                                     \-> [ Apache / PHP server ]

For HTTP requests this works great, requests are distributed to my Apache servers just fine. For SSL requests, I had HAproxy distributing the requests using TCP load balancing, and it worked however since HAproxy didn't act as a proxy, it didn't add the X-Forwarded-For HTTP header, and the Apache / PHP servers didn't know the client's real IP address.

So, I added stunnel in front of HAproxy, reading that stunnel could add the X-Forwarded-For HTTP header. However, the package which I could install into pfSense does not add this header... also, this apparently kills my ability to use KeepAlive requests, which I would really like to keep. But the biggest issue which killed that idea was that stunnel converted the HTTPS requests into plain HTTP requests, so PHP didn't know that SSL was enabled and tried to redirect to the SSL site.

How can I use HAproxy to load balance across a number of SSL servers, allowing those servers to both know the client's IP address and know that SSL is in use? And if possible, how can I do it on my pfSense server?

Or should I drop all this and just use nginx?

score 40 · Answer 1 · edited Sep 30 '14 at 16:17

Just for the record, as this thread is often referred to concerning HAProxy + SSL, HAProxy does support native SSL on both sides since 1.5-dev12. So having X-Forwarded-For, HTTP keep-alive as well as a header telling the server that the connection was made over SSL is as simple as the following :

listen front
    bind :80
    bind :443 ssl crt /etc/haproxy/haproxy.pem
    mode http
    option http-server-close
    option forwardfor
    http-request set-header X-Forwarded-Proto https if { ssl_fc }
    server srv1 1.1.1.1:80 check ...
    ...

I'm sure that by the time you came up with something different, but at least new visitors will get the easy solution now :-)

score 15 · Accepted Answer · answered Aug 17 '11 at 21:38

You dont need to drop it all, you could just use nginx in front of haproxy for SSL support, keeping all your load balancing config. You dont even need to use nginx for HTTP if you don't want to. Nginx can pass both X-Forwarded-For and a custom header indicating SSL is in use (and client cert information if you want). Nginx config snippet that sends required information:

proxy_set_header SCHEME $scheme;      # http/https
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header CLIENT_CERT $ssl_client_raw_cert;

Josh · Answer 3 · 2012-11-02T12:48:30.370

For anyone else who finds this question, I followed Ochoto's advice and used nginx. Here's the specific steps I used to make this work on my pfSense router:

Using the pfsense web interface, I installed the pfsense PfJailctl package and the "jail_template" package under System > Packages so I could create a FreeBSD jail under which to compile and install nginx on the pfsense system.
I configured a jail for my nginx server under Services > Jails, giving the new jail the same hostname and IP address of the virtual IP alias which I had HAproxy running on. I bound the jail to the WAN interface. I used the default jail template and enabled unionfs rather than nullfs.
Once the jail had been started, I SSHed in to the pfsense box and ran jls to find the jail's number. I then ran jexec 1 sh to get a shell inside the jail. From there I set up BSD ports and installed nginx using:
```
portsnap extract
portsnap fetch update
cd /usr/ports/www/nginx
make install clean
```

I then configured nginx to listen on port 443, and pass all requests to HAproxy on port 80, including the real IP and the SSL status inside HTTP headers. My usr/local/etc/nginx/nginx.conf looks like:

worker_processes  1;

events {
    worker_connections  2048;
}

http {
    upstream haproxy {
        server 209.59.186.35:80;
    }

    server {
        listen       443;
        server_name  my.host.name default_server;
        ssl                  on;
        ssl_certificate      my.crt;
        ssl_certificate_key  my.key;
        ssl_session_timeout  5m;

        ssl_protocols  SSLv3 TLSv1;
        ssl_ciphers  HIGH:!aNULL:!MD5;
        ssl_prefer_server_ciphers   on;

        location / {
            proxy_pass http://haproxy;

            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

            proxy_set_header X-Forwarded-Proto https;
        }
    }

}

I then modified my PHP application to detect the X-Forwarded-Proto HTTP Header:

function usingSSL()
{
    return (
       (isset($_SERVER['HTTPS']) && strtolower($_SERVER['HTTPS']) == 'on' )
        || (isset($_SERVER['HTTP_X_FORWARDED_PROTO'])
               && strtolower($_SERVER['HTTP_X_FORWARDED_PROTO']) == 'https' ));
}

So the final setup is:

(internet) ---> [ -> nginx -> haproxy -]--> (pool of apache servers)
                [    (pfSense server)  ]

greg0ire · Answer 4 · 2013-01-15T11:04:19.787

My configuration for a 1.5-dev-17 version of haproxy:

global
        log 127.0.0.1   local0
        log 127.0.0.1   local1 notice
        #log loghost    local0 info
        maxconn 4096
        #chroot /usr/share/haproxy
        user haproxy
        group haproxy
        daemon
        #debug
        #quiet

defaults
        log     global
        mode    http
        option  httplog
        option  dontlognull
        option  http-server-close
        retries 3
        option redispatch
        fullconn 1000        
        maxconn 1000
        timeout queue 600s
        timeout connect 5s
        timeout client 600s
        timeout server 600s

frontend http-in
        bind *:80
        bind *:443 ssl crt /usr/local/etc/ssl/certs
        reqadd X-Forwarded-Proto:\ https if { ssl_fc }
        default_backend varnish-ha
        option forwardfor
backend varnish-ha
  server hafront1 10.1.69.1:6081  minconn 100 maxqueue 10000

It uses the ssl_fc ACL. Please note that the option http-server-close part is very important.

score 5 · Answer 5 · answered Aug 17 '11 at 21:48

HAProxy can't hit an SSL backend without using raw TCP mode, losing X-Forwarded-For, but, you could potentially re-encrypt the traffic with a listening stunnel for the backend transit. Ugly, though.

I like Ochoto's approach better, with a caveat: nginx is a perfectly capable load balancer; if you're using it, I'd say use it for everything. Proxy your incoming HTTPS to load balanced HTTPS backends - and that way, there's no need for custom headers for SSL information (unless you do need the client certificate).

score 2 · Answer 6 · answered Apr 29 '13 at 13:15

I implemented a solution last year to integrate HAProxy with pfSense in a way that it harnesses all features of HAProxy and maintains a good isolation with pfSense. So that it is a viable option for production environments. SSL is terminated on HAProxy. I installed HAProxy inside a jail in pfSense using ezjail and Ports Collection. That way it is very easy to maintain both components independently. And you may install whatever version you want. I started with 1.5-dev13. And since then it is working perfectly for me. I have documented the whole thing here.

Installing HAProxy on pfSense

BTW Willy, thanks a lot for such an excellent product.

How can I use HAproxy with SSL and get X-Forwarded-For headers AND tell PHP that SSL is in use?

6 Answers6

Linked