1

I have an apache server that works perfect until I connect to VPN and then all connections to server time out.

Now to my understanding the issue is tun0 becomes the default output interface hence apache gets confused as how to send packets out, so I tried to fix it using control groups by marking packets going out from apache and redirecting them through eth0 as described in this SU answer, but it doesn't work anymore after I upgraded my Ubuntu OS to version 16.04. This is my network diagram:

enter image description here

And here's my network details:

me@mypc:~$ ip route list
0.0.0.0/1 via 10.132.1.5 dev tun0 
default via 192.168.0.1 dev eth0  proto static  metric 100 
10.132.1.1 via 10.132.1.5 dev tun0 
10.132.1.5 dev tun0  proto kernel  scope link  src 10.132.1.6 
123.4.5.6 via 192.168.0.1 dev eth0 
234.5.6.7 via 192.168.0.1 dev eth0 
128.0.0.0/1 via 10.132.1.5 dev tun0 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.6  metric 100 

me@mypc:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:cc:a9:b3:c9:41  
          inet addr:192.168.0.6  Bcast:192.168.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:864897 errors:0 dropped:0 overruns:0 frame:0
          TX packets:467142 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1088053099 (1.0 GB)  TX bytes:220201868 (220.2 MB)
          Interrupt:17 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          ...

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.132.1.6  P-t-P:10.132.1.5  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:46622 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14950 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:60587170 (60.5 MB)  TX bytes:1396546 (1.3 MB)

I've done further testing and discovered that if I add this routing rule:

sudo route add -host 123.4.5.6 gw 192.168.0.1

I become able to connect to the server from devices connected to my router using the router ip 123.4.5.6 but not from any other ip address.

And after setting up the control groups on apache and trying the following command:

sudo cgexec -g net_cls:novpn wget http://www.whatsmyip.org/

and checking the ip in the downloaded webpage it would be my router ip 123.4.5.6 and not my vpn ip 10.132.1.6.

So I guess the control groups solution works somehow but not with apache and the incoming packets are being received successfully by apache but nothing is going out.

How can I configure apache to use eth0 to output packets instead of tun0?

razzak
  • 33
  • 2
  • 9

3 Answers3

3

Managing 2 default routes and getting some traffic to go via one and some to go via the other is a big pain and I would recommend you avoid it if possible. If you don't want to access the internet via the VPN then your setup can be greatly simplified which would solve the issues. Do you really need to access the internet via the VPN? also whats the significance of 10.132.1.5 I don't get why your using it to route to other hosts in 10.132.1.x ? you already have an IP in that subnet 10.132.1.x and should be directly connected.

so IF you don't want to access the internet via the VPN and there is no need to route via 10.32.1.5 you could simplify your routing table down to: 

default via 192.168.0.1 dev eth0  proto static  metric 100 
10.132.1.0/24 dev tun0  proto kernel  scope link  src 10.132.1.6 
169.254.0.0/16 dev eth0  scope link  metric 1000 
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.6     metric 100

Which will also solve the apache issue. Why only those routes? firstly:

10.132.1.0/24 dev tun0  proto kernel  scope link  src 10.132.1.6

should be the only route you need to get to all the hosts on the other side of the vpn. As this route matches 10.232.1.0 - 10.232.1.254 if there are other 10.x.x.x address on the vpn network you could broaden the route to 10.0.0.0/8 If you don't understand what the /8 /24 bits mean I highly suggest you read What is the "slash" after the IP? or google for "CIDR notation". will make the /32 /1 bit make more sense.

these old routes put together define your default route via the VPN

0.0.0.0/1 via 10.132.1.5 dev tun0 
128.0.0.0/1 via 10.132.1.5 dev tun0

Breaking it up into 2 routes (one for the first half of the internet and the 2nd covering the rest) means that they have higher priority because they more specific than your default route:

default via 192.168.0.1 dev eth0  proto static  metric 100

The problem then is that you now need to explicitly add routes that are even more specific, to force traffic via your local router(192.16.8.0.1). 

123.4.5.6 via 192.168.0.1 dev eth0 
234.5.6.7 via 192.168.0.1 dev eth0

Without the default route via the VPN those routes aren't needed. If you do need to route traffic via 10.232.1.5 and need to keep to sets of default routes I'd recommend using route rules instead of overriding the default route. Route rules are more flexible but generally how you do it is to match source IP to default gateway, so all traffic sourced from your VPN IP(10.1.232.6) goes vis the VPN default route and all traffic sourced from your local IP(192.168.0.6) uses your ISP as the default route. see https://unix.stackexchange.com/questions/22770/two-interfaces-two-addresses-two-gateways for a guide on how to do that.

Community
  • 1
Nath
  • 1,362
1

So you need both default gateways; then the way to do this then is with route rules:

1) Add a secondary IP to eth0 - ie 192.168.1.7 and restart apache (sounds like your config is listen 0.0.0.0:80 so you'll just need to restart apache to get it to listen on the new IP.

2) Change the Nat rules on your router to foward traffic to this IP:

3) Create a new default route in a secondary route table lets name the table 'apache':

echo "1 apache" >> /etc/iproute2/rt_tables

4) Add a default route to this table via your local router.

ip route add default via 192.168.0.1 dev eth0 table apache

5) Finally you need a rule to define which traffic should use the apache route table.

ip rule add from 192.168.0.7 table apache

192.168.0.7 is a secondary IP and Apache is the only process using it this rule should only match the traffic leaving apache in response to web requests. This will ensure that only that specific traffic will be use the new route table and it won't mess with your VPN traffic or its current routing behaviour.

please note that the ip commands won't persist after reboot. To make them persistent add them to your interface up scripts to make run every time your laptop reboots.

P.S Leaving my old answer and did this as a new answer as this is very different solution.

Nath
  • 1,362
0

Just tell apache to listen on multiple IP addresses.

I think you'll want to have these lines in your configuration.

Listen 192.168.0.6:80
Listen 10.132.1.6:80

This tells apache to 'bind' to port 80 on each of those IP addresses. If you're using virtual hosts, you'll also have to add these lines to their configurations as well.

Sidenote

If you're not using one, I recommend using a static IP on eth0 as well as on the VPN server.

If you're not able to configure a static IP on the VPN, there are workarounds you could do. One could be to generate the relevant Listen entries after you connect to the VPN. Or, maybe there's a clean solution that uses hostnames (if you have a DNS server).

Official Documentation:

BobTuckerman
  • 448