0

I'm rather new to this, so please be patient.

My setup is essentially thre devices using mqtt, connecting through two brokers both running emqx. Both emqx brokers are also supposed to be set up as haproxy load balancers, with keepalived in case one goes down.

broker 1's IP is 192.168.1.201

broker 2's IP address is 192.168.1.202

keepalived's virtual IP is 192.168.1.200

haproxy.cfg is 

frontend emqx_tcp
    bind *:1883
    option tcplog
    mode tcp
    default_backend emqx_tcp_back

backend emqx_tcp_back
    balance roundrobin
    server emqx_node_1 192.168.1.201:1883 check
    server emqx_node_2 192.168.1.202:1883 check

keepalived.conf is 

global_defs {
        lvs_id haproxy01
}

vrrp_sync_group SyncGroup01 {
        group {
                VI_1
        }
}

vrrp_script chkhaproxy {
        script "/usr/bin/killall -0 haproxy"
        script "/usr/sbin/service haproxy start"
        interval 9
        timeout 3
        weight 20
        rise 2
        fall 4
}

vrrp_instance VI_1 {
        interface eth0                # interface to monitor
        state MASTER
        virtual_router_id 51          # Assign one ID for this route
        priority 101                  # 101 on MASTER, 100 on BACKUP
        advert_int 5
        authentication {
                auth_type PASS
                auth_pass password
        }
        virtual_ipaddress {
                192.168.1.200         # the virtual IP
        }
        track_script {
                chkhaproxy
        }
}

Problem is, I can't seem to get either haproxy or keepalived to work as intended. When I have broker 1 only running haproxy (keepalived is not running), and connect directly to broker 1's IP address from the client, it seems to only forward connections to broker 2. It doesn't seem to be able to connect to broker 1 (and yes, I checked that they were both up. If only broker 2 is up, the client just doesn't connect). After some digging, I thought that it might be because haproxy and the broker use the same IP. So I tried using a virtual IP from keepalived. But after trying that, I found that I couldn't even connect to the virtual IP. The client, upon trying to connect to it, would give me the error "OSError: [Errno 113] No route to host".

what am I missing?

Edit: upon tailing the logs with tailf /var/log/syslog this is the result

Feb 7 14:56:19 pi01 Keepalived_healthcheckers[10453]: Opening file '/etc/keepalived/keepalived.conf'.

Feb 7 14:56:19 pi01 Keepalived_healthcheckers[10453]: Unknown keyword 'lvs_id'

Feb 7 14:56:19 pi01 Keepalived_vrrp[10454]: Default interface eth0 does not exist and no interface specified. Skipping static address 192.168.1.200.

Feb 7 14:56:19 pi01 Keepalived_vrrp[10454]: Unable to load ipset library - libipset.so.3: cannot open shared object file: No such file or directory

Feb 7 14:56:19 pi01 Keepalived_vrrp[10454]: VRRP_Instance(VI_1) Unknown interface ! Feb 7 14:56:19 pi01 Keepalived_healthcheckers[10453]: Using LinkWatch kernel netlink reflector...

Feb 7 14:56:20 pi01 Keepalived_vrrp[10454]: Stopped

Feb 7 14:56:20 pi01 Keepalived[10450]: Keepalived_vrrp exited with permanent error CONFIG. Terminating

Feb 7 14:56:20 pi01 Keepalived[10450]: Stopping

Feb 7 14:56:20 pi01 Keepalived_healthcheckers[10453]: Stopped

Feb 7 14:56:25 pi01 Keepalived[10450]: Stopped Keepalived v1.3.2 (12/25,2016)

It seems to consistently fail upon the line Using "LinkWatch kernel netlink reflector"

Brandon
  • 1

1 Answers1

0

First problem: HAProxy is configured to listen on all interfaces on the same port used by emqx. This should cause you problems.

Either set up a specific NIC for the keepalived/haproxy listener if you want them to use the same port, or make HAProxy and emqx listen on different ports.

Once you have that working, it’s time to look at Keepalived. My best tip there is to tail your logs. Make sure the router ID is identical between the master and backup, and that the backup gets a lower prioriy than the master. I frankly don’t remember from the top of my head if you must allow mac spoofing or similar in the switch for VRRP (the protocol used by keepalived) to work.

Mikael H
  • 5,179