1

I have a Debian box running firewalld set up a as gateway NAT/router. This device has two NICS; wan --- public interface, assigned to the external firewalld zone, dynamically assigned IP address using DHCP ---, and lan1 --- private interface, assigned to the trusted firewalld zone, statically assigned IP address of 192.168.0.1/16. The system is also configured to do NAT and port forwarding using firewalld:

(router) $> sudo firewall-cmd --info-zone=external
external (active)
  target: DROP
  icmp-block-inversion: yes
  interfaces: wan
  sources:
  services: http https ssh
  ports: 8443/tcp
  protocols:
  forward: yes
  masquerade: yes
  forward-ports:
        port=8443:proto=tcp:toport=8443:toaddr=192.168.0.2
        port=443:proto=tcp:toport=443:toaddr=192.168.0.2
        port=80:proto=tcp:toport=80:toaddr=192.168.0.2
  source-ports:
  icmp-blocks: echo-reply echo-request fragmentation-needed neighbour-advertisement neighbour-solicitation packet-too-big port-unreachable router-advertisement router-solicitation time-exceeded
  rich rules:

(router) $> sudo firewall-cmd --info-zone=trusted trusted (active) target: ACCEPT icmp-block-inversion: no interfaces: lan1 lo sources: services: ssh ports: protocols: forward: yes masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:

(router) $> sudo sysctl -n net.ipv4.ip_forward 1

So far, so good. Devices within the internal 192.168.0.0/16 network can access the internet without any issues, and incoming connections on the wan interface to ports 80, 443, and 8443 are correctly forwarded to 192.168.0.2 where a reverse proxy instance is listening.

The problem arises when I want to use the public-facing IP address of the router from an internal device to access the reverse proxy on 192.168.0.2. To illustrate this, I have two public DNS records associated with the public address of wan; hq.mydomain.tld, which is proxied through Cloudflare, and gateway.mydomain.tld, which instead points directly to the public IP address of wan. If I hit hq.mydomain.tld from an internal device, the request in properly routed to 192.168.0.2, as expected, as the request is first going to Cloudflare and then proxied back to my gateway:

$> curl -sS -D - https://hq.mydomain.tld -o /dev/null -L
HTTP/2 200
date: Sat, 16 Dec 2023 16:43:09 GMT
content-type: text/html; charset=utf-8
cache-control: public, max-age=600
...

However, if I instead hit gateway.mydomain.tld from an internal device, it seems as if no rules are applied and the request instead directly hits the router. Nothing is listening on the forwarded ports on the router, and so the request immediately fails.

$> curl -sS -D - https://gateway.mydomain.tld -o /dev/null -L
curl: (7) Failed to connect to gateway.mydomain.tld port 443 after 713 ms: Couldn't connect to server

The same thing happens if instead of the public hostname I use the public IP of the router.

$> curl -sS -D - https://a.b.c.d -o /dev/null -L
curl: (7) Failed to connect to a.b.c.d port 443 after 18 ms: Couldn't connect to server

This also happens if I curl from the router itself, either using it's own public IP address or localhost:

(router) $> curl -sS -D - https://a.b.c.d. -o /dev/null -L
curl: (7) Failed to connect to a.b.c.d port 443 after 0 ms: Couldn't connect to server

(router) $> curl -sS -D - https://localhost -o /dev/null -L curl: (7) Failed to connect to localhost port 443 after 0 ms: Couldn't connect to server

My question is then: how do I configure firewalld (or its backend, nftables) to apply the public port forwarding rules to traffic originating from the internal network or from the router itself? Note that I can't use rules which directly use the public IP of wan, as this is a DHCP-assigned IP address and could change at any moment, rendering the rules useless.

I have tried a bunch of different things without success:

  • Setting up identical port forwarding rules on the trusted zone.

  • Using direct firewalld port forwarding rules on the loopback interface.

  • Finally, I tried creating a firewalld policy using trusted (the internal zone) as a the ingress zone and HOST as the egress zone:

    (router) $> sudo firewall-cmd --info-policy=internal_fwd
    internal_fwd (active)
      priority: -1
      target: CONTINUE
      ingress-zones: trusted
      egress-zones: HOST
      services:
      ports:
      protocols:
      masquerade: no
      forward-ports:
      source-ports:
      icmp-blocks:
      rich rules:
    

    I really thought this would work, as such a policy applies to any traffic coming from the internal network that terminates in the router. However, it seems you can't use a policy with an egress-zones: HOST to forward ports to another device:

    (router) $> sudo firewall-cmd --permanent --policy=internal_fwd --add-forward-port=port=8443:proto=tcp:toport=8443:toaddr=192.168.0.2
    Error: INVALID_FORWARD: Policy 'internal_fwd': A 'forward-port' with 'to-addr' is invalid for egress zone 'HOST'
    
3dg3
  • 11
  • 2

2 Answers2

0

The most common practice is to NAT from external to internal. First enable enable masquerading for that zone (I use public in this example)

firewall-cmd --zone=public --add-masquerade

firewall-cmd --permanent --zone=public --add-forward-port=port=80:proto=tcp:toaddr=192.168.x.x:toport=80

Owh yeah you also need to enable net.ipv4.ip_forward = 1

Turdie
  • 2,945
0

This is a case of NAT hairpinning: here the final target is in the same LAN as the original source, so it will attempt to reply directly to it, bypassing the NAT inverse transformations that has to be done for replies. As the source doesn't know of the final target address (it doesn't know about 192.168.0.2 and so TCP replies from this source will be discarded with a TCP RST).

To handle NAT hairpinning, the replies also have to go through the NAT router so they can undergo the inverse NAT transformation. One easy way to do this is to also perform NAT on the source address in addition to the destination address. Any address that would get the reverse proxy server to reply through the NAT router is acceptable: its LAN address, its public address or any other address, including a fictional address, that would have the NAT router as gateway for the reverse proxy server. Here, for simplicity, the LAN address will be used (through the selective use of masquerade).

With pure nftables this could have been handled with a rule such as below in nat/postrouting:

iif lan1 oif lan1 ct status dnat masquerade

But this can't be done within firewalld. Instead using source address along output interface in a rich rule can achieve the same. The rich rule will perform masquerade on the trusted zone when the source is within 192.168.0.0/16. This happens only when the packet has been routed back from 192.168.0.0/16 to 192.168.0.0/16 (ie to lan1, ie to trusted), meaning NAT was performed or this wouldn't have happened (except for the corner case of a misconfigured system in trusted zone that would use the gateway to reach other 192.168.0.0/16 addresses instead of directly):

firewall-cmd --zone=trusted --add-rich-rule='rule family="ipv4" source address="192.168.0.0/16" masquerade'

This results in this diff for the nftables backend ruleset:

--- /tmp/ruleset1   2023-12-20 10:22:31.937292389 +0000
+++ /tmp/ruleset2   2023-12-20 10:26:31.155214737 +0000
@@ -519,6 +519,7 @@
    }
chain nat_POST_trusted_allow {
  •   ip saddr 192.168.0.0/16 oifname != "lo" masquerade
    

    }

    chain nat_POST_trusted_post {

which is exactly what is intended.

Alas that's not enough. Because the port forwarding rules were added to the external zone, no DNAT actually happened, because the only zone having been traversed is the trusted zone (tied to the lan1 interface), but DNAT rules are performed only when the packet traverses the external zone (tied to the wan interface), and firewalld doesn't appear to have fallbacks (the relevant parts in the nftables backend use goto statements rather than jump statements, precluding the traversal of further chains). Port forwarding rules have to be duplicated in the trusted zone so they are also performed for this case:

firewall-cmd --zone=trusted --add-forward-port=port=8443:proto=tcp:toport=8443:toaddr=192.168.0.2
firewall-cmd --zone=trusted --add-forward-port=port=443:proto=tcp:toport=443:toaddr=192.168.0.2
firewall-cmd --zone=trusted --add-forward-port=port=80:proto=tcp:toport=80:toaddr=192.168.0.2

Use --permanent on all previous commands once correct behavior was verified.

Note: the reverse proxy's logs will all show the NAT router's 192.168.0.1 as source for all clients behind lan1. This appears unavoidable within the constraints of the rich rules language. Using directly nftables, one could have performed a prefix change in nat/postrouting like:

snat ip prefix to ip saddr map { 192.168.0.0/16 : 10.168.0.0/16 }

to still be able to distinguish individual clients from the trusted zone in proxy proxy's logs with the inverse mapping (eg: 10.168.1.2 means the client was actually 192.168.1.2). I don't know how to do this within firewalld.


UPDATE: the router NAT case from itself to itself can be handled using firewalld policies, but even here there are workarounds needed.

Active Policies

Policies only become active if all of the following are true.

  • The ingress zones list contain at least one regular zone or a single symbolic zone.

  • The egress zones list contain at least one regular zone or a single symbolic zone.

  • For non symbolic zones, the zone must be active. That is, it must have interfaces or sources assigned to it.

If the policy is not active then the policy has no effect.

Symbolic zones:

HOST

This symbolic zone is for traffic flowing to or from the host running firewalld. This corresponds to netfilter (iptables/nftables) chains INPUT and OUTPUT.

  • If used in the egress zones list it will apply to traffic on the INPUT chain.

  • If used in the ingress zones list it will apply to traffic on the OUTPUT chain.

Workarounds are needed: both ingress and egress are needed, HOST can be used only once and has to be in ingress to have rules added to nat/output, and port-forwarding rules require that the egress zone has no interface added to it, else an error such as:

Error: INVALID_ZONE: Policy 'nat-hairpin-host': 'forward-port' cannot be used because egress zone 'trusted' has assigned interfaces

will happen.

So use HOST in ingress and recycle an unused zone: internal to be used as egress zone with instead of the lo interface bound to it, the host's address on the wan interface, to meet all prerequisites to have the policy activated (and then only when reaching the WAN address). One could use an other unused zone or even define an additional custom zone by adding an XML file (eg: by following this) instead of internal. There won't be much interaction with the extra zone chosen, especially considering that filter/input and output rules always explicitly grant access to the lo interface.

The documentation about using policies doesn't provide much examples from CLI and describes an XML file. Still RHEL's documentation provides examples (that are unrelated to this problem). Policies can only be used along --permanent (and thus requiring at the end --reload).

The instructions below will consider 192.0.2.2/24 is the address set on wan to access services. So in the end:

firewall-cmd --permanent --new-policy nat-hairpin-host
firewall-cmd --permanent --policy=nat-hairpin-host --add-ingress-zone=HOST
firewall-cmd --permanent --policy=nat-hairpin-host --add-egress-zone=internal
firewall-cmd --permanent --policy=nat-hairpin-host --add-forward-port=port=8443:proto=tcp:toport=8443:toaddr=192.168.0.2
firewall-cmd --permanent --policy=nat-hairpin-host --add-forward-port=port=443:proto=tcp:toport=443:toaddr=192.168.0.2
firewall-cmd --permanent --policy=nat-hairpin-host --add-forward-port=port=80:proto=tcp:toport=80:toaddr=192.168.0.2

firewall-cmd --permanent --zone=internal --add-source=192.0.2.2

firewall-cmd --reload

Which will in the end create along plumbing rules and equivalent filter rules, something similar to this:

[...]
    chain nat_OUTPUT_POLICIES_pre {
            ip daddr 192.0.2.2 jump nat_OUT_policy_nat-hairpin-host
    }

[...]

    chain nat_OUT_policy_nat-hairpin-host_allow {
            meta nfproto ipv4 tcp dport 443 dnat ip to 192.168.0.2:443
            meta nfproto ipv4 tcp dport 80 dnat ip to 192.168.0.2:80
            meta nfproto ipv4 tcp dport 8443 dnat ip to 192.168.0.2:8443
    }

The reverse proxy will see the source being the WAN address (the primary address for the WAN IP LAN if multiple addresses are set) since this source wasn't (and didn't have to be) changed, and that's the source the NAT router selected when reaching itself before DNAT rerouted it elsewhere.

# ip route get 192.0.2.2
local 192.0.2.2 dev lo src 192.0.2.2 uid 0 
    cache <local> 
A.B
  • 13,968