4

We're moving a bunch of services, say from 1.2.3.4 to 5.6.7.8.

In order to test that the new services are correctly configured, we'd like to redirect (to the new host) all traffic destined for the original host that originates from our testing workstations.

Of course, such redirection could be implemented on routers within the network itself—but for stability reasons we've decided to implement it on each workstation directly (which are all OS X 10.10 Yosemite, so use pre-v4.7 OpenBSD pf).

I've added to /etc/pf.anchors/com.apple:

rdr-anchor "910.TestServiceMove/*"
anchor "910.TestServiceMove/*"
load anchor "910.TestServiceMove" from "/etc/pf.anchors/910.TestServiceMove"

And created /etc/pf.anchors/910.TestServiceMove:

rdr pass log on lo0 from any to 1.2.3.4 -> 5.6.7.8
pass out log route-to lo0 from any to 1.2.3.4 keep state

When the rules are loaded, both appear to work correctly:

$ sudo tcpdump -v -n -e -ttt -i pflog0
tcpdump: WARNING: pflog0: no IPv4 address assigned
tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 65535 bytes
00:00:00.000000 rule 0.910.TestServiceMove.0/0(match): pass out on en1: (tos 0x0, ttl 64, id 40691, offset 0, flags [DF], proto TCP (6), length 64)
    9.9.9.9.58029 > 1.2.3.4.22: Flags [S], cksum 0x291a (correct), seq 3399416413, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS val 2063366865 ecr 0,sackOK,eol], length 0
00:00:00.000047 rule 0/0(match): rdr in on lo0: (tos 0x0, ttl 64, id 40691, offset 0, flags [DF], proto TCP (6), length 64, bad cksum 896a (->b4da)!)
    9.9.9.9.58029 > 5.6.7.8.22: Flags [S], cksum 0xb284 (correct), seq 3399416413, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS val 2063366865 ecr 0,sackOK,eol], length 0

But the TCP handshake does not complete (SYN-ACKs are ignored, and SYN is repeatedly sent until the connection times out):

$ sudo tcpdump -v -n -e -ttt host 5.6.7.8
tcpdump: data link type PKTAP
tcpdump: listening on pktap, link-type PKTAP (Packet Tap), capture size 65535 bytes
00:00:00.000000 e8:80:2e:e7:67:bc > 84:80:2d:35:e5:43, ethertype IPv4 (0x0800), length 78: (tos 0x0, ttl 63, id 40691, offset 0, flags [DF], proto TCP (6), length 64)
    9.9.9.9.58029 > 5.6.7.8.22: Flags [S], cksum 0xb284 (correct), seq 3399416413, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS val 2063366865 ecr 0,sackOK,eol], length 0
00:00:00.015524 84:80:2d:35:e5:43 > e8:80:2e:e7:67:bc, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 52, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    5.6.7.8.22 > 9.9.9.9.58029: Flags [S.], cksum 0x7ce4 (correct), seq 1901846890, ack 3399416414, win 14480, options [mss 1460,sackOK,TS val 523934721 ecr 2063366865,nop,wscale 7], length 0
00:00:00.986946 e8:80:2e:e7:67:bc > 84:80:2d:35:e5:43, ethertype IPv4 (0x0800), length 78: (tos 0x0, ttl 63, id 25319, offset 0, flags [DF], proto TCP (6), length 64)
    9.9.9.9.58029 > 5.6.7.8.22: Flags [S], cksum 0xae9c (correct), seq 3399416413, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS val 2063367865 ecr 0,sackOK,eol], length 0
00:00:00.014938 84:80:2d:35:e5:43 > e8:80:2e:e7:67:bc, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 52, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    5.6.7.8.22 > 9.9.9.9.58029: Flags [S.], cksum 0x78fa (correct), seq 1901846890, ack 3399416414, win 14480, options [mss 1460,sackOK,TS val 523935723 ecr 2063366865,nop,wscale 7], length 0
00:00:00.397794 84:80:2d:35:e5:43 > e8:80:2e:e7:67:bc, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 52, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    5.6.7.8.22 > 9.9.9.9.58029: Flags [S.], cksum 0x776c (correct), seq 1901846890, ack 3399416414, win 14480, options [mss 1460,sackOK,TS val 523936121 ecr 2063366865,nop,wscale 7], length 0
00:00:00.588237 e8:80:2e:e7:67:bc > 84:80:2d:35:e5:43, ethertype IPv4 (0x0800), length 78: (tos 0x0, ttl 63, id 50201, offset 0, flags [DF], proto TCP (6), length 64)
    9.9.9.9.58029 > 5.6.7.8.22: Flags [S], cksum 0xaab4 (correct), seq 3399416413, win 65535, options [mss 1460,nop,wscale 5,nop,nop,TS val 2063368865 ecr 0,sackOK,eol], length 0

I guess that the TCP stack is discarding SYN-ACKs that have originated from a host other than that to which the SYN was sent. But shouldn't redirection rules rewrite traffic in both directions—indeed, shouldn't keep state be ensuring that the connection is tracked for this purpose?

eggyal
  • 422
  • 5
  • 17

3 Answers3

1

If the services you are accessing are all accessed by name then I'd just create a host entry with in the hosts file on your dev machines.

For example, if your machine was called myserver and resolved to 1.2.3.4 Then you can now create a host file entry /etc/hosts with

myserver 5.6.7.8

Anything that now tries to reach myserver will end up hitting the new machine.

Alternatively, just create a new isolated network with the new server at the old address and plug your dev machine into it to test.

The problem with a straight redirection or route rule is that the new server doesn't answer under the name of the old address. So you'd have to use NAT.

hookenz
  • 14,848
0

Wouldn't it just be easier to throw the machines behind a simple loadbalancer or NAT? That's generally more of an industry-standard practice, anyway, and would allow you to further scale the application over time.

RVT
  • 407
-1

Can you just offer the "old" address from the "new" server ala sub-interface, or is the new address in a totally different subnet?

If you can't attach the new IP to the new server, then you haven't really moved the address. To route traffic across the old address to the new server, you need to NAT the address in the old subnet. Traffic would land on the "old" address and then be source NAT'd to a new address on the same server/appliance and sent to the "new" IP. Return traffic would then pass back to the SNAT address, which would un-NAT the traffic and return it to the original requester. This is really not a good long term solution, especially if the "new" address is across a WAN link. You've put off an inevitable failure and added latency to the mix as well.

A better long term solution would be to consider: 1. use DNS instead of IP 2. virtualize either the IP address (NAT'ing device like a firewall or ipmasq box), 3. or the server itself (load-balancer)

Rick Buford
  • 166
  • 5