Juniper LACP with linux server not increase speed

Question

And:

I'm using R630 with Debian 11 and Ex2200 with version 12.3R12.4

The network: each server using 4 link to sw

server 1  ==== switch ==== server 2

Here is server config ( both server )

auto lo
iface lo inet loopback
iface eno1 inet manual 
iface eno2 inet manual 
iface eno3 inet manual 
iface eno4 inet manual
auto bond0
iface bond0 inet manual
bond-mode 4
bond-miimon 100
bond-lacp-rate 1
bond-slaves eno1 eno2 eno3 eno4


auto vmbr0
iface vmbr0 inet static
bridge_fd 0
bridge_stp off
bridge_ports bond0

address xxxxxxxxxxxxxxxxxx/xx
gateway xxxxxxxxxxxxxxxxxxxxx

Here is switch config

system {
    root-authentication {
        encrypted-password "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"; ## SECRET-DATA
    }
    name-server {
        xxxxxxxxxxxxxxxx;
    }
    services {
        ssh {
            root-login allow;
            protocol-version v2;
        }
        telnet;
        web-management {
            http {
                port 80;
                interface me0.0;
            }
        }
    }
    syslog {
        user * {
            any emergency;
        }
        file messages {
            any notice;
            authorization info;
        }
        file interactive-commands {
            interactive-commands any;
        }
    }
}
chassis {
    aggregated-devices {
        ethernet {
            device-count 8;
        }
    }
}
interfaces {
    ge-0/0/0 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ge-0/0/1 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ge-0/0/2 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ge-0/0/3 {
        unit 0 {                        
            family ethernet-switching;
        }
    }
    ge-0/0/4 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ge-0/0/5 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ge-0/0/6 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ge-0/0/7 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ge-0/0/8 {
        ether-options {
            802.3ad ae0;
        }
    }
    ge-0/0/9 {
        ether-options {
            802.3ad ae0;
        }
    }
    ge-0/0/10 {
        ether-options {
            802.3ad ae0;
        }
    }
    ge-0/0/11 {
        ether-options {
            802.3ad ae0;
        }
    }
    ge-0/0/12 {
        ether-options {
            802.3ad ae1;
        }
    }
    ge-0/0/13 {
        ether-options {
            802.3ad ae1;
        }
    }
    ge-0/0/14 {
        ether-options {
            802.3ad ae1;
        }
    }
    ge-0/0/15 {                         
        ether-options {
            802.3ad ae1;
        }
    }
    ge-0/0/16 {
        ether-options {
            802.3ad ae2;
        }
    }
    ge-0/0/17 {
        ether-options {
            802.3ad ae2;
        }
    }
    ge-0/0/18 {
        ether-options {
            802.3ad ae2;
        }
    }
    ge-0/0/19 {
        ether-options {
            802.3ad ae2;
        }
    }
    ge-0/0/20 {
        ether-options {
            802.3ad ae3;
        }
    }
    ge-0/0/21 {
        ether-options {
            802.3ad ae3;
        }
    }
    ge-0/0/22 {
        ether-options {
            802.3ad ae3;
        }
    }
    ge-0/0/23 {
        ether-options {
            802.3ad ae3;
        }
    }
    ae0 {
        aggregated-ether-options {
            lacp {
                active;
            }
        }
        unit 0 {
            family ethernet-switching;
        }
    }
    ae1 {
        aggregated-ether-options {
            lacp {
                active;
            }                           
        }
        unit 0 {
            family ethernet-switching;
        }
    }
    ae2 {
        aggregated-ether-options {
            lacp {
                active;
            }
        }
        unit 0 {
            family ethernet-switching;
        }
    }
    ae3 {
        aggregated-ether-options {
            lacp {
                active;
            }
        }
        unit 0 {
            family ethernet-switching;
        }
    }
    ae4 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ae5 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ae6 {
        unit 0 {
            family ethernet-switching;
        }
    }
    ae7 {
        unit 0 {
            family ethernet-switching;
        }
    }
    me0 {
        unit 0 {
            family inet {
                address xxxxxxxxxxxxxxxxx;
            }
        }
    }
}
routing-options {
    static {
        route 0.0.0.0/0 next-hop xxxxxxxxx;
    }
}
protocols {                             
    igmp-snooping {
        vlan all;
    }
    stp;
    lldp {
        interface all;
    }
    lldp-med {
        interface all;
    }
}
ethernet-switching-options {
    storm-control {
        interface all;
    }
}
vlans {
    default {
        vlan-id 1;
        interface {
            ae0.0;
            ae1.0;
            ae2.0;
            ae3.0;
            ae4.0;
            ae5.0;
            ae6.0;
            ae7.0;
            ge-0/0/0.0;
            ge-0/0/1.0;
            ge-0/0/2.0;
            ge-0/0/3.0;
            ge-0/0/4.0;
            ge-0/0/5.0;
            ge-0/0/6.0;
            ge-0/0/7.0;
        }
    }
}

Then I use iperf3 check speed result in 1Gbps both tcp and udp.

But the link status is saying I'm 4Gbps

All port showing same MAC address

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
    link/ether xxxxxxxxxxxxxxxxx brd ff:ff:ff:ff:ff:ff
    altname enp1s0f0
3: eno2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
    link/ether xxxxxxxxxxxxxxxxx brd ff:ff:ff:ff:ff:ff permaddr xxxxxxxxxxxxxxxxx
    altname enp1s0f1
4: eno3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
    link/ether xxxxxxxxxxxxxxxxx brd ff:ff:ff:ff:ff:ff permaddr xxxxxxxxxxxxxxxxx
    altname enp2s0f0
5: eno4: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000
    link/ether xxxxxxxxxxxxxxxxx brd ff:ff:ff:ff:ff:ff permaddr xxxxxxxxxxxxxxxxx
    altname enp2s0f1
6: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master vmbr0 state UP mode DEFAULT group default qlen 1000
    link/ether xxxxxxxxxxxxxxxxx brd ff:ff:ff:ff:ff:ff
15: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether xxxxxxxxxxxxxxxxx brd ff:ff:ff:ff:ff:ff
root@master:~# cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v5.10.0-16-amd64
Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Peer Notification Delay (ms): 0
802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: xxxxxxxxxxxxxxxxxxxx
Active Aggregator Info:
    Aggregator ID: 29
    Number of ports: 4
    Actor Key: 9
    Partner Key: 1
    Partner Mac Address: xxxxxxxxxxxxxxxxxxxxxxxxxx

Switch show link speed is 4Gbps

root@switch> show interfaces ae0 
Physical interface: ae0, Enabled, Physical link is Up
  Interface index: 157, SNMP ifIndex: 550
  Link-level type: Ethernet, MTU: 1514, Speed: 4Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 0
  Device flags   : Present Running
  Interface flags: SNMP-Traps Internal: 0x4000
  Current address: 3c:8a:b0:14:d5:83, Hardware address: 3c:8a:b0:14:d5:83
  Last flapped   : 2016-01-22 03:14:17 UTC (00:51:49 ago)
  Input rate     : 4096 bps (4 pps)
  Output rate    : 6240 bps (5 pps)
Logical interface ae0.0 (Index 65) (SNMP ifIndex 552)
    Flags: Up SNMP-Traps 0x4000 Encapsulation: ENET2
    Statistics        Packets        pps         Bytes          bps
    Bundle:
        Input :            55          0          3820            0
        Output:         79804          0       8458752            0
    Adaptive Statistics:
        Adaptive Adjusts:          0
        Adaptive Scans  :          0
        Adaptive Updates:          0
    Protocol eth-switch
      Flags: None
root@switch> show interfaces ae1

Physical interface: ae1, Enabled, Physical link is Up
  Interface index: 158, SNMP ifIndex: 553
  Link-level type: Ethernet, MTU: 1514, Speed: 4Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 0
  Device flags   : Present Running
  Interface flags: SNMP-Traps Internal: 0x4000
  Current address: 3c:8a:b0:14:d5:84, Hardware address: 3c:8a:b0:14:d5:84
  Last flapped   : 2016-01-22 03:14:29 UTC (00:51:56 ago)
  Input rate     : 5088 bps (1 pps)
  Output rate    : 4576 bps (1 pps)
Logical interface ae1.0 (Index 66) (SNMP ifIndex 588)
    Flags: Up SNMP-Traps 0x4000 Encapsulation: ENET2
    Statistics        Packets        pps         Bytes          bps
    Bundle:
        Input :            58          0          4010            0
        Output:         80338          0       8515483            0
    Adaptive Statistics:
        Adaptive Adjusts:          0
        Adaptive Scans  :          0
        Adaptive Updates:          0
    Protocol eth-switch
      Flags: None

As far as I know, Last time when I using Huawei switch there is an option load-balance src-dst-mac is the key, To make speed superposition.

But I didn't find any relatived config in JunOS.

Question: How to config LACP or hash or whatever mechanism affect LB in JunOS ?

score 0 · Accepted Answer · answered Apr 15 '23 at 03:29

I am going to steal the answer for your question from another post I read elsewhere:

A quick and dirty explanation is that a single line of communication using LACP will not split packets over multiple interfaces. For example, if you have a single TCP connection streaming packets from HostA to HostB it will not span interfaces to send those packets. I've been looking at LACP a lot here lately for a solution we are working on and this is a common misconception that 'bonding' or 'trunking' multiple network interfaces with LACP gives you a "throughput" of the combined interfaces. Some vendors have made proprietary drivers that will route over multiple interfaces but the LACP standard does not from what I've read.

Link aggregation (LACP/802.3ad) max throughput

Juniper LACP with linux server not increase speed

1 Answers1