Server Fault
Server Fault

Questions tagged [rdma]

13 questions
4
votes
2 answers

Set up Windows 10 SRP initiator

How do can I setup an SRP Windows 10 initiator? I've seen several mentions of an initiator being included in WinOF but I haven't found any instructions on how to do it. The initiator will be Windows 10 Pro with a Connect-X2 DDR interface. I have…
chew socks
  • 222
3
votes
3 answers

Can SMB Direct be used together with a SMB network share on one NAS?

I'm currently working on upgrading our server storage and I'm looking into SMB direct in combination with an RDMA supported NIC. SMB Direct is interesting as it has very low latency compared to competitive protocols. I would however, like the server…
aardbol
  • 1,493
3
votes
0 answers

Why isn't SMB Direct working over RDMA?

In a Windows lab environment, I'm trying to test performance of SMB Direct using Mellanox RDMA adapters with global pause. I wanted to test first with Global Pause for simplicity before switching to PFC. I think I've set up everything correctly,…
Elliott B
  • 200
2
votes
1 answer

Beegfs-client does not work with mellanox OFED driver

I am trying to configure and test beegfs with RDMA as explained in: https://community.mellanox.com/s/article/howto-configure-and-test-beegfs-with-rdma?t=1570613300675 My test configuration: OS: ubuntu 16.04 on both servers (Kernel version…
Ramon
  • 21
2
votes
2 answers

Dual-port RDMA Throughput Issue

I have two nodes connected with dual-port Mellanox Connext-X3 VPI HCAs via an IB switch. The nodes are two socket machines with Hasswell CPUs and 2 16GB DIMMs per each socket (totaling 64GB). Everything seems to work perfectly, except for the…
Mohammad Hedayati
  • 729
2
votes
1 answer

How does RDMA transfer between two servers without the CPU?

I've been reading about RoCE and it talks about transferring data zero copy between the memory in one server directly to the memory of an application in another server. Most articles make a point to say this is done without involvement from the CPU.…
Grant Curell
  • 1,188
1
vote
1 answer

Infiniband configured and connected but indicator lights unlit on HBAs and no functionality in OS

I have installed two dual-port FDR Infiniband VPI HBAs, one in each of two servers running CentOS 6.9, server1>lspci 03:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3] server2>lspci 81:00.0 Network controller: Mellanox…
rg6
  • 195
1
vote
1 answer

Infiniband OpenSM N-to-N port routing configuration

I have 10 servers with two CPUs each and one Mellanox 100G Infiniband NIC per CPU. Each NIC is connected to a single Mellanox 36 port 100G IB switch. My RDMA application runs as one process per NUMA node and binds to the local NIC to avoid cross CPU…
Hugo Maxwell
  • 121
1
vote
1 answer

Does DPDK/RDMA between 2 machines gives lower latency than local host ping

I know loopback will go through the kernel network stack until reach IP layer, including syscall overhead and some memory copy overhead. DPDK and RDMA use different technology to avoid these. So let's say I have two machine connected by dpdk/rdma,…
Liang Mury
  • 13
0
votes
1 answer

Cisco 800 Series RPing's to ISP are Intermittant

According to my ISP, our occasional internet outages are do to a router configuration problem. They say the router must RPing the ISP regularly. Failure to do so in 4 hours results in outgoing and incoming internet loss. According to their logs the…
Dorothy
  • 179
  • 1
  • 8
0
votes
0 answers

Is it possible to dump memory from one Ubuntu VM to another?

I'm trying to use Azure VMs to host a game server for me and some friends since spot instances are cheap, but the downside is that they get deallocated/deleted pretty quickly in practice (somewhere like every 30 mins when I was testing). So I'm…
ROODAY
  • 235
0
votes
1 answer

Role of Local Routing Header (LRH) in RoCEv2

In the RoCEv2 specification (Annex 17) https://www.infinibandta.org/ibta-specifications-download/ the header BTH+ includes the Local Routing Header (LRH). The routing though is done with IP. Is there any use for the LRH ? Are the source and…
Jim
  • 101
-1
votes
1 answer

Could I enable ROCE v2 on both MCX623106AN-CDAT 100GbE NIC cards and connect each other without any switch?

Since I do not have ROCE v2 supported switch, and I am planning to connect two MCX623106AN-CDAT 100GbE NIC cards each other from two different server systems, the OS I plan to use is Windows Server 2022, so, I’d like to understand if it is possible…
Jacky Lee
  • 85