18
Question

Is there a general rule for whether inserting PCIe-connected hardware into (not removing from) a running motherboard (or merely powered, if relevant) can be dangerous to the hardware, especially the motherboard? I've seen a myriad question about removal, but not addition.

If there's no general rule, am I looking for a specific certification or named capability on the to-be-attached hardware and/or motherboard?

Rationale

I'm evaluating a bug with a PCIe NIC - a TP-LINK AXE5400 causing Windows 11 to BSoD if connected during OS installation from USB - which involves a significant amount of connection and disconnection. I tend to have UEFI Fast Boot disabled for good measure, and don't want to have to re-enable it and manually shutdown and reboot every time I want to evaluate it (dis/re-)connected.

(Lack of) Duplicates

  1. This is rather the opposite of Can I pull a card from the PCI-x bus without powering down?

    I accidentally forgot to unplug a server's power when pulling a card even though the system was powered down at the OS level.

    Unfortunately doing so seems to have corrupted the system's BIOS and the IPMI/BMC BIOS. It also seems to have fried the on-board video.

    This has occurred on a Supermicro motherboard which is why I tried re-flashing the main BIOS and IPMI BIOS. Reflashing the main BIOS went without problems and no errors are shown in any BIOS logs.

    I thought hotplug technology should have meant that I could physically pull a card while it is running. Searches of the internet seem to imply that.

    ...because I can imagine that the device tree would be significantly less problematically affected if an addition occurs, since it's not like removing a GPU during operation (which would cause all kinds of foreseeable problems).

  2. https://serverfault.com/search?q=insert+pcie+running+machine returned 0 results (at 2024-08-05).

RokeJulianLockhart
  • 316

2 Answers2

18

It’s a definite maybe

PCI express theoretically supports hotplug (the common term for what you want to do).

For it to work correctly though, all of the following needs to be true:

  • The baseboard’s PCIe controllers must support it. Many, many systems do actually have hardware support for it, but this is often hidden by a lack of firmware support.
  • The system firmware must support it. Almost never the case on client systems targeted at consumer usage. May or may not be the case for actual server hardware, though it’s usually something you can confirm pretty easily with the manufacturer.
  • The OS, if running, must support it. Modern Windows Server versions have full support AFAICT. Linux usually supports it (support in the kernel is optional at build time but enabled by most distributions, no userspace components are required). FreeBSD has full support.
  • The card itself must support hotplug. This is usually advertised for most expansion cards that support it, and is generally very common in server-grade network or storage controllers (those are the big ones that often get hotplugged).

Assuming all those criteria are met, then yes, it’s safe as long as you’re taking care to avoid the usual hazards associated with mucking around inside the chassis of a computer while it’s running.

Do note however that negotiation of link parameters (link width, MPS, etc) for a hot-plugged card may not result in the exact same parameters you would see if you booted the system with the card installed, and peer to peer transfers to/from the card may not work until you reboot (this is often a side-effect of the link parameters).

Austin Hemmelgarn
  • 3,021
10

The term you want to search for is PCIe hot-add or hotplug. https://electronics.stackexchange.com/questions/208767/does-pcie-hotplug-actually-work-in-practice

Your specific question seems to be if it will damage the hardware, and I think the general answer is "no" because it's designed to allow this. Double check that your chipset supports it.

mfinni
  • 36,892