7

I have DL380 server with QLogic Gigabit Ethernet installed. Simply trying to create a bond but can't seem to get throughout more than 1 Gig link. All 3 cables from 2 servers are connected to S40 switch where I created the LACP (Lag) the link comes up and lag shows active, but I just can't get more than 1 Gig throughput. I am testing with iperf3. Tried all different bonding modes, rr, 802.3d everything but can't just go more than 900 Mbps or so. I am missing something but can't figure out. 

Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer3+4 (1)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 9c:8e:99:0b:78:70
Active Aggregator Info:
    Aggregator ID: 4
    Number of ports: 3
    Actor Key: 9
    Partner Key: 418
    Partner Mac Address: 00:01:e8:d5:f4:f3

Slave Interface: enp3s0f1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 9c:8e:99:0b:78:70
Slave queue ID: 0
Aggregator ID: 4
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 9c:8e:99:0b:78:70
    port key: 9
    port priority: 255
    port number: 1
    port state: 63
details partner lacp pdu:
    system priority: 32768
    system mac address: 00:01:e8:d5:f4:f3
    oper key: 418
    port priority: 128
    port number: 12
    port state: 63

Slave Interface: enp4s0f0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 9c:8e:99:0b:78:72
Slave queue ID: 0
Aggregator ID: 4
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 9c:8e:99:0b:78:70
    port key: 9
    port priority: 255
    port number: 2
    port state: 63
details partner lacp pdu:
    system priority: 32768
    system mac address: 00:01:e8:d5:f4:f3
    oper key: 418
    port priority: 128
    port number: 7
    port state: 63

Slave Interface: enp4s0f1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 9c:8e:99:0b:78:74
Slave queue ID: 0
Aggregator ID: 4
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 9c:8e:99:0b:78:70
    port key: 9
    port priority: 255
    port number: 3
    port state: 63
details partner lacp pdu:
    system priority: 32768
    system mac address: 00:01:e8:d5:f4:f3
    oper key: 418
    port priority: 128
    port number: 5
    port state: 63

Tried all sort of google stuff but can't seem to get it working. And I am out of any ideas. Appreciate if someone can drive me to the correct direction.

Thanks.

NBhatti
  • 173

2 Answers2

11

@ewwhite right. I'll just explain some thing. When you test your link between two machines, you use only one NIC, LACP will not split packets across multiple interfaces for a single stream/thread. For example a single TCP stream will always send/receive packets on the same NIC. So you could see higher speed only when test with more then one destination. There is good answer, where it described.

Alexander Tolkachev
  • 4,666
7

It seems like bonding and LACP are one of the worst understood concepts in networking.

But the short explanation is that you'll never achieve more than one connection's throughout with a single source-destination pair. If you need more bandwidth on a single connection, you'll have to move to 10GbE.

ewwhite
  • 201,205