Development

Oh, the States You'll Go To: The AMD GPU Reset Bug and Everything It Isn't

Sparrow 15 min read
Linux Vfio Virtualization Amd Hardware Gaming

An exploded view of a reference Radeon RX 5700 XT

Most people do not have a reset bug at all

they have one of several different problems that look identical from the onset. Black screens, Windows code 43, libvirt coughing up Unknown PCI header type '127' or lspci saying your card is revision ff. All of which can be explained with:

Reset gap (a precondition): the device holds state that nothing in the PCIe reset path can see or clear, and the resets available to whatever driver owns the card next don’t touch it either. Every consumer AMD card from Polaris on has this, the RX 9000 included.

Reset bug (a failure): the gap, plus the vendor’s shipped driver being unable to recover the card through the normal ownership flow, no standard path brings the card back, leaving custom code, driver-cycling rituals, or a full power cycle as the only ways out.

The TLDR version: When your card loads the driver loads firmware into the card. When your card gets unbinded nothing clears that memory at all, the card just keeps running whatever the last driver left on it. Linux and other drivers try to execute resets based on what Linux exposes, which is not the actual reset AMD uses on the card to clear the memory. If the only reset on offer can’t reach the firmware you end up with a mismatch in your GPU, i.e. that which was never cleared from before and the new firmware the guest’s driver loads. When this happens your card gets confused. This is the reset gap. Most cards at this point can rebind to the official amdgpu driver and have the card reset natively without issue. Cards that need their power rails poked, or custom out of band code run to get them back to a usable state, are the ones that suffer from the reset bug. Most of the errors people bump into are indeed fixable if you do not conflate the two.

If you want to know whether your card has an actual bug, check vendor-reset. If you don’t see your card in the list you most likely don’t have a reset bug and have something else. HardPass has a built-in state manager that keeps the card out of the bad states people mistake for this one. If you want to know what your card is actually holding onto between boots, and how GPU memory works, keep going.

The whole post revolves around the gap between the two dashed boxes and how do we get to them. Everything the host can reach is on the left, everything that holds state is on the right
The whole post revolves around the gap between the two dashed boxes and how do we get to them. Everything the host can reach is on the left, everything that holds state is on the right

GPU Firmware, Not the VRAM You’re Looking For

As a baseline when people talk about this bug and say memory, they don’t mean VRAM. They mean firmware, which on an AMD card lives on a flash chip near the edge of the board, and what it stores is the ROM aka VBIOS.

A VBIOS Walks Into a BAR

If you were to dump this chip’s memory, aka VBIOS dumping, what comes off isn’t just one binary, but a collection of files which is different from other flash ROM if you’re used to Phone ROM or Game ROMs. Out of all these files that are dumped only one is used by the host’s memory map.

That part is called the VGA ROM which is an executable. Your system firmware runs it at boot, you can think of it as the init system for the hardware as this is how you get a picture on screen before any driver has loaded. What it runs are the ATOM tables, a description of this exact board, memory timings for the modules actually fitted, clock and voltage ceilings, the fan curve, which connectors exist and where they go. Although AMD standardizes the format, the values are filled in and maintained by your card manufacturer, i.e. AIB partners (Asus, MSI, Gigabyte, etc.), which is why two 5700 XTs with the same chip underneath can behave differently based on manufacturer. Running those tables is what people refer to as “posting” the card. Video BIOS, VBIOS, VGA ROM, and option ROM are all alternative names for the same thing.

So the host just executes the VGA ROM and the card comes up, right? Yes, but it has to read the thing off the card before it can run a single byte of it, and it can’t read that or anything else on the card directly, i.e. we don’t just do DMA (Direct Memory Access). Reading from the card has to be done through a “Pointer”, which is an address the card agrees to answer at (it’s not a real pointer but as an analogy it’s close enough). If you’ve come from embedded, it’s the same idea as a memory-mapped peripheral like GPIOs, UARTs, timers. To really drill this home, here’s a plain lspci -vvv of an RX 5700 XT, our RDNA test subject, taken from a dump posted to the amd-gfx mailing list in 2019 when Navi was new and nothing worked. The arrangement is standard across RDNA, and the older Polaris and Vega cards have the same one with a different reset mechanism.

Region 0: Memory at 2fb0000000 (64-bit, prefetchable) [size=256M]
Region 2: Memory at 2fc0000000 (64-bit, prefetchable) [size=2M]
Region 4: I/O ports at d000 [size=256]
Region 5: Memory at c0100000 (32-bit, non-prefetchable) [size=512K]
Expansion ROM at c01a0000 [disabled] [size=128K]

Every one of those lines is one of those “Pointers”, and the proper term is a BAR, a Base Address Register. The critical piece is that a BAR isn’t memory (which is why it’s not a pointer in the traditional sense). It’s a slot in the card’s config space holding an address. lspci calls them Regions. Region 0 is the window onto VRAM, 256 MB of it here, which is the host looking at 8 GB of GDDR6 through a letterbox because Resizable BAR is off. Region 2 is the doorbells, how a driver tells the card there’s work waiting for it. Region 4 is a small I/O port range that amdgpu dropped in Linux 5.13 and no longer maps at all. And Region 5 is 512 KB of registers, which is the one we’ll focus on.

The Expansion ROM here is the ROM BAR which is a BAR that points at the VGA ROM instead of at VRAM or registers. It sits at offset 0x30 in config space rather than with the other six at 0x10 through 0x24, and it carries an enable bit, which is what [disabled] is referencing here. It’s switched off most of the time, so the ROM isn’t in your address space until something turns it on, and that’s why reading/dumping the ROM out of sysfs needs an echo 1 before the file gives you anything.

The one other way to read the ROM region is the method done by the official drivers, which can read that flash chip directly through a pair of registers in Region 5 without going near the BAR. That will make a lot more sense later, because what the ROM BAR hands you and what is actually on the chip are not always the same.

Region 5, Home of the PSP and SMU

AMD’s block diagram of an amdgpu device, showing GC, SMU, VCN, SDMA, PSP, DCN and PCIe NBIO inside one die with UMC memory controllers around the edge AMD’s own block diagram of an amdgpu device, shipped in the Linux kernel documentation. Yellow dashed lines are PSP interaction, orange dotted arrows are resets. Follow the orange: every reset starts at the SMU, and not one of them starts at PCIe (NBIO) down in the corner.

The PSP is the Platform Security Processor, which is another small processor on the same die with its own firmware and its own operating system. AMD’s kernel documentation has it handling security policy and validating the firmware for every other block on the chip. That firmware is called navi10_sos.bin (for the RX 5000 series we’ll be using for this article but this applies to all RDNA consumer cards), pulled off your host’s disk out of /lib/firmware/amdgpu/ when the driver binds.

The sos in that filename signifies the PSP’s own operating system, the sOS. A bootloader inside the PSP runs first, it loads the sOS, and from then on the sOS is what answers requests to the PSP. The kernel uses two different names for it, sOS in its comments and TOS in the macro that waits for it to be ready.

From bind time onward the PSP relies on the sOS, which as we mentioned is pulled off disk. To keep that safe the PSP fences off a slice of the card’s own GDDR6 and parks the verified copies in there. That’s called the TMR, the Trusted Memory Region, which the driver sets aside in psp_tmr_init(). So the PSP actually has two firmwares at two different moments. On an orderly unbind the driver does tear the TMR back down, so those bytes don’t outlive it. What does outlive it is the sOS, i.e. the original firmware that was loaded.

The SMU is the System Management Unit, the power microcontroller for the whole chip and the block that actually hands resets out to everything else on the die. In AMD’s code for the RX 5000 the PSP is MP0 and the SMU is MP1, both at version 11.0.0 on Navi 10, which you can read off the kernel’s own discrete GPU table.

The only way to reach SMU or PSP is Region 5, specifically rows of mailbox registers sat inside that window, MP0_SMN_C2PMSG_* for the PSP and MP1_SMN_C2PMSG_* for the SMU. So a driver bringing one of these cards up pulls signed blobs off your disk, hands them through a mailbox to a security processor, which checks them and parks them in the TMR, then brings the SMU up.

One last thing. If running the ATOM tables is what posts a card, something has to record that it happened, and it’s one bit in a register. amdgpu_atombios_scratch_need_asic_init() reads BIOS scratch register 7, looks for ATOM_S7_ASIC_INIT_COMPLETE_MASK, and if the bit is set the driver skips posting entirely.

A New Challenger Approaches: vfio-pci

Say you hand the card to amdgpu at boot. It initialises everything above, your desktop comes up, the OS is running. Perfect. Now say you want to hand that same card to a Windows guest instead, so libvirt unbinds amdgpu and binds vfio-pci, which is a passthrough stub that deliberately knows nothing about graphics hardware, it just maps the device into a VM.

Whose Card Is It Anyway? What the VM Inherits

The VM can see the memory through the BARs but it cannot validate any of it. It doesn’t know whether the sOS still running in there is one it would have trusted, whether the SMU is running somebody else’s power policy, whether the posted flag it can read was set for this boot or the last one, or whether the thing on the other end of that mailbox is mid-conversation with a driver that no longer exists. From where the guest is standing, a card that’s already running looks exactly like a card that’s broken. As such we can see the amdgpu driver always checks on start if the card is dirty.

drivers/gpu/drm/amd/amdgpu/nv.c L511-L526 at Linux 97d73c8

static bool nv_need_reset_on_init(struct amdgpu_device *adev)
{
	u32 sol_reg;

	if (adev->flags & AMD_IS_APU)
		return false;

	/* Check sOS sign of life register to confirm sys driver and sOS
	 * are already been loaded.
	 */
	sol_reg = RREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_81);
	if (sol_reg)
		return true;

	return false;
}

The Seven Deadly PCIe Resets

The fix to this dirty state is called a reset. A reset is the kernel telling the card to drop whatever it was doing and come back up like it had only just been plugged in, so the next driver along can assume nothing and start clean. This might sound like a switch, but there are actually multiple types of resets.

drivers/pci/pci.c L5190-L5199 at Linux 97d73c8

const struct pci_reset_fn_method pci_reset_fn_methods[] = {
	{ },
	{ pci_dev_specific_reset, .name = "device_specific" },
	{ pci_dev_acpi_reset, .name = "acpi" },
	{ pcie_reset_flr, .name = "flr" },
	{ pci_af_flr, .name = "af_flr" },
	{ pci_pm_reset, .name = "pm" },
	{ pci_reset_bus_function, .name = "bus" },
	{ cxl_reset_bus_function, .name = "cxl_bus" },
};

When a device turns up Linux probes each of those in order and keeps an inventory of the ones that respond, as not all cards have all the reset mechanisms implemented. Some even have reset mechanisms that are not exposed through these methods, which is true of every AMD card in this post. You can see what your device has available under the reset_method file in sysfs, which is the list non-official sources walk such as vfio-pci. So let’s walk the list ourselves with our RX 5700 XT and see what is available for us.

RungWhat it isOn an RX 5000
device_specificA per-device routine, matched on IDsEmpty, no AMD GPU has ever been in it
acpiAn _RST method from board firmwareUsually absent
flrPCIe’s reset-one-function commandRefused, reports FLReset-
af_flrThe same via the older Advanced Features capRefused, no AF capability
pmD3hot and back through config spaceRefused twice over, see below
busSecondary Bus Reset from the bridge aboveThe only one that fires
cxl_busReset through a CXL port registerNot a CXL device
Diagram of the Linux pci_reset_fn_methods ladder showing device_specific highlighted as the slot vendor-reset fills, bus as the only method that fires, and acpi, flr, af_flr, pm and cxl_bus all drawn as unavailable
Six of the seven are unavailable on this card. The only one left is SBR.

We also note the card reports NoSoftRst+, which is just saying shoving it into D3hot and back doesn’t reset anything. The pm has two locks, because some older Radeon once claimed a D3hot round trip reset it when in reality it didn’t touch the framebuffer and monitor sync at all, which is a very funny way to lie to an operating system.

drivers/pci/quirks.c L3863-L3872 at Linux 97d73c8

/*
 * Some AMD/ATI GPUS (HD8570 - Oland) report that a D3hot->D0 transition
 * causes a reset (i.e., they advertise NoSoftRst-).  This transition seems
 * to have no effect on the device: it retains the framebuffer contents and
 * monitor sync.  Advertising this support makes other layers, like VFIO,
 * assume pci_reset_function() is viable for this device.  Mark it as
 * unavailable to skip it when testing reset methods.
 */
DECLARE_PCI_FIXUP_CLASS_HEADER(PCI_VENDOR_ID_ATI, PCI_ANY_ID,
			       PCI_CLASS_DISPLAY_VGA, 8, quirk_no_pm_reset);

Why a Secondary Bus Reset Isn’t Enough

So the only reset that actually fires on these cards is a Secondary Bus Reset. The bridge above drops the link, config space in front of the card gets cleared, the card’s PCIe front end comes back up and reports that it worked. It does not go near Region 5.

The RX 5000 block diagram with the upstream port, downstream port, PCIe NBIO function and HDMI audio function highlighted as the only things a Secondary Bus Reset touches, while the SMU, PSP, GC, DCN, VCN, SDMA and VRAM behind NBIO stay dimmed
What a Secondary Bus Reset actually touches, in purple. Everything in grey is the state your next guest inherits.

So Why Hasn’t AMD Fixed It?

So if SBR clearly doesn’t wipe the PSP and SMU and causes the card to be left in a very confused state, then what is AMD doing about this, why don’t they solve anything and keep saying they’re looking into it?

Lisa vs Reset: Mode1 and BACO

They technically are fully compliant with everything they’re supposed to do. If you’ve done any compliance work you’re quite aware of what we mean, i.e. spirit of the law vs word of the law. The spec defines the letter of a reset, by defining what it has to do and AMD’s PCIe front end does all of it, drops the link, clears config space, reports back that it worked. What the spec never explicitly writes down is the two processors sat behind that front end still holding the last owner’s firmware. Which is violating the spirit of the law, being able to reuse your card from a blank state when the reset is complete.

Which leaves two ways out, FLR is optional but they left the bit clear or the kernel’s own quirk table is where Intel and Samsung and Qualcomm put devices that need special handling. There’s a working reset sat right there on the card, it’s just not one anybody else can call.

The card has its own resets that happen exclusively within the GPU. Mode1 is a single command left in a mailbox, for the firmware to read and trigger the internal sequence from. BACO, Bus Active Chip Off, involves walking the chip through powering its own core down, which takes dozens of register writes in a fixed order. Doing the second one badly is how you brick a card. There are a couple more in amdgpu’s list, but these don’t matter for Navi desktop cards.

amdgpu decides which of these to use when it’s trying to reset itself by first picking BACO if the SMU says the board supports it and mode1 as a fallback. At bind time the SMU’s hardware init hasn’t run yet, so the BACO check comes back false and mode1 wins. Mode1 can be driven by the SMU or the PSP, and the RX 5000 SMU code has no mode1 routine at all, so the job lands on the PSP.

drivers/gpu/drm/amd/amdgpu/psp_v11_0.c L410-L432 at Linux 97d73c8

static int psp_v11_0_mode1_reset(struct psp_context *psp)
{
	int ret;
	uint32_t offset;
	struct amdgpu_device *adev = psp->adev;

	offset = SOC15_REG_OFFSET(MP0, 0, mmMP0_SMN_C2PMSG_64);

	ret = psp_wait_for(psp, offset, MBOX_TOS_READY_FLAG,
			   MBOX_TOS_READY_MASK, 0);

	if (ret) {
		drm_info(adev_to_drm(adev), "psp is not working correctly before mode1 reset!\n");
		return -EINVAL;
	}

	/*send the mode 1 reset command*/
	WREG32(offset, GFX_CTRL_CMD_ID_MODE1_RST);

	msleep(500);

	return 0;
}
The RX 5000 block diagram with the PSP, SMU and every block they reach highlighted, showing how AMD's mode1 reset reaches the firmware behind NBIO while the PCIe switch ports are dimmed
The reset that works, in purple. Compare it with the last diagram: this one gets past NBIO because it goes through a mailbox, and only a driver that has mapped Region 5 can open that door.

If PSP and SMU resets are what we actually need, we should just expose those and call it a day right? Well not quite, in the case of a gap you can just rebind amdgpu and call it a day. For the RX 5000 though it’s not just a gap, look at what that function does before it sends anything. It waits for the mailbox to say it’s ready, and if that times out it logs psp is not working correctly before mode1 reset! and returns -EINVAL. It has no retry or fallback mechanism. That failure travels all the way back up to the bind, which prints asic reset on init failed and gives up, so the driver never attaches. A card whose PSP has stopped answering never gets past that check. Neither does a card that has stopped answering at all, because it reads back as all ones, and all ones doesn’t match the ready flag either.

How vendor-reset Brings the RX 5700 XT Back

So how does vendor-reset help us here, and how did gnif solve this problem? We know he triggers the resets on the card directly as we mentioned before:

FamilyCardsWhat it does
Polaris 10/11/12RX 470 to 590, 460/560, 540/550BACO if the card can, else a config space sequence
Vega 10Vega 56, 64, FrontierPSP mode1, BACO if that fails
Vega 20Radeon VII, plus the MI100 which is really ArcturusPSP mode1, BACO if that fails
Navi 10/12/14RX 5300 to 5700 XT, plus Pro W5700 and Pro 5600MPSP mode1

He does a PSP mode1 for RDNA1 by hooking into pci_dev_specific_reset() with ftrace, so the routine runs when that gets called. The purpose is to register it as a device_specific, which can be called from Linux’s own paths because it now appears in reset_method. The routine also offers a safety layer. An SBR on one of these makes the card fall off the PCIe bus entirely, and gnif blocks it before anything else gets a chance to call it:

src/amd/common.c L31-L36 at vendor-reset cbb4dd5

int amd_common_probe(const struct vendor_reset_cfg *cfg, struct pci_dev *dev)
{
  /* disable bus reset for the card, seems to be an issue with all of them */
  dev->dev_flags |= PCI_DEV_FLAGS_NO_BUS_RESET;
  return 0;
}

With that flag set the bus reset refuses. So the routine checks the sign of life register, the SMU’s interrupt state and the PSP bootloader flag, leaving the card alone if all three say it’s already clean. Then it retraces amdgpu’s steps, wipes BIOS scratch registers 6 and 7 so the next driver posts the card from scratch, settles the SMU with DisallowGfxOff and PrepareMp1ForReset, and sends the exact same mode1 command amdgpu sends. So far this is the same reset as amdgpu with guard rails around the path it triggers, but there is one big difference.

src/amd/navi10.c L176-L194 at vendor-reset cbb4dd5

  /* check validity of PSP before reset */
  offset = SOC15_REG_OFFSET(MP0, 0, mmMP0_SMN_C2PMSG_64);
  tmp = psp_wait_for(adev, offset, 0x80000000, 0x8000FFFF, false);
  if (tmp)
    vr_warn(dev, "timed out waiting for PSP to reach valid state, but continuing anyway\n");

  /* reset command */
  WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_64, GFX_CTRL_CMD_ID_MODE1_RST);
  msleep(500);

The difference being amdgpu won’t send the command unless the mailbox says it’s ready, then writes it, sleeps half a second and reports success without reading anything back, leaving the checking to its caller. vendor-reset presses on through a silent mailbox and then demands an acknowledgement on C2PMSG_33, failing without one. Which boils down to covering the case of the PSP going quiet partway through. On an RX 5000 because the SMU has no mode1 of its own you have to fix the PSP route. Every card since got another way in, which is exactly why we don’t need this for newer GPUs.

Now the caveats. This is a module from outside the tree, so loading it taints your kernel, and it has to load early, because the kernel’s default reset breaks the card in a way the module can’t undo. People on newer kernels have reported it would not build, which is the risk with anything out of tree. Kernel changes upstream can break the driver.

Ctrl+F section

What is the AMD reset bug on the RX 5000 series? It’s the card failing to come back after one owner hands it to another. The card has no Function Level Reset and its power management reset is disabled, so Linux and QEMU fall back to a Secondary Bus Reset, which resets the PCIe link and never speaks to the PSP or SMU that hold the previous owner’s state. That part is the reset gap. What makes it a bug is that we cannot just bind it back to amdgpu and reset the card, because by then the PSP has gone silent.

Which cards are affected? The Navi 10 cards, meaning the RX 5600, 5600 XT, 5700 and 5700 XT, and the Navi 14 cards, meaning the RX 5300, 5500 and 5500 XT. They share the same PSP 11 family and the same reset code in amdgpu. vendor-reset’s device table also lists Navi 12.

Why do some cards reset fine and others don’t? This is nuanced. The SMU runs on a parameter set that came out of your VBIOS, and your VBIOS came from whoever built the board, not AMD. This covers VR phase mapping, current telemetry calibration, the fan curve, power limits. The amdgpu driver reads that out of the ATOM tables and writes it into the SMU at init, so a dirty SMU still holds the last owner’s version of all of it, and how much trouble that causes depends on the information that got pulled from the ATOM tables, since the official reset mechanisms never wiped the PSP/SMU.

Why does my RX 5700 XT work the first boot but not the second? The first start gets a card nobody has touched since power on, so the guest driver loads the PSP’s sOS and the SMU firmware cleanly. When the machine stops, the only reset available is a bus reset that doesn’t reach that firmware, so the second start inherits everything the first guest left running.

What does “Unknown PCI header type ‘127’” mean? The card answered a config space read with all ones. The header type byte reads 0xff, mask off the multi-function bit and you’ve got 0x7f, which is 127. Revision ff in lspci is that same all-ones read. It means the card has stopped responding on the bus.

Does Linux 7.0 fix the RX 5000 reset bug? No. What makes this a bug rather than a gap is the PSP going silent and amdgpu refusing to send the reset without it, and that’s inherent to the card rather than to any kernel version.

Is the new d3cold reset method in the kernel, and would it help? No to both. It was proposed in May 2026 as an eighth method and the general form was dropped from the series by version 9 after review. It was only ever offered to single-function devices, and the RX 5000 GPU shares its bus with its HDMI audio function.

Why do I have to write device_specific to reset_method? Because Linux worked out which reset methods your card supports when it discovered the card, which happened before vendor-reset loaded. Writing the name back makes the kernel probe that method again. Linux 5.15 made reset_method writable, which is why the shipped udev rule targets 5.15 and newer.

How do I check which reset methods my RX 5000 supports? Read the file for the GPU function, cat /sys/bus/pci/devices/<GPU ADDRESS>/reset_method, using your card’s address out of lspci. With vendor-reset working it should say device_specific or if your upstream kernel added it. Writing a method the card doesn’t support fails with “Unsupported reset method” in the kernel log.

Does the HDMI audio function have to go to the VM too? In practice yes. vfio-pci’s fallback bus reset only works when every device on that bus is bound to vfio-pci, and vendor-reset wakes the audio function to save and restore its state around every reset anyway.

Does Smart Access Memory or Resizable BAR cause the reset bug? No. SAM only changes how much VRAM the host can see through the card’s Region 0 window, it doesn’t change what the PSP and SMU keep running inside. Some passthrough tooling still asks you to turn Resizable BAR off on an RX 5000, which is a separate compatibility matter.

Do I need to dump my VBIOS and pass it with romfile=? Not for this bug. rombar= and romfile= only change how the guest gets at the VGA ROM through the ROM BAR, which doesn’t touch the PSP or SMU state the reset bug is about. The advice comes from gnif’s RX 7000 writeup, where a full flash dump works as a romfile by being invalid enough that the guest BIOS refuses to post the card twice. On an RX 5000 the VBIOS sits at the front of the flash rather than partway in, so that reasoning doesn’t carry over.

Can I use my RX 5000 as the host display and still pass it through? Yes, not easily though, because amdgpu doesn’t reset a discrete card when it unbinds, so the VM inherits the host’s running sOS. With vendor-reset in place vfio-pci resets it when QEMU opens the card, and amdgpu should post it fresh when you hand it back.

Can I use vendor-reset on an RX 6000 or RX 7000? No, and gnif has been very clear regarding the deliberate cutoff. Every generation since has moved the reset command further from the main problem:

CardMode1 goes toChecked before sending
RX 5000the PSP mailboxPSP ready to take a command
RX 6000the SMU mailboxPM firmware 58.26, PSP alive
RX 7000the SMU debug mailboxnothing
RX 9000the SMU debug mailboxnothing

navi10_ppt.c has no mode1 routine in it at all, so the PSP is the only way in and amdgpu won’t use it unless the PSP says it’s ready so a normal unbind and bind of amdgpu doesn’t free this. By RDNA4 the support check is a // TODO and a return true, and the send is three register writes with no handshake at all. On top of that, pointing routines written for Vega and Navi 10 at silicon they were never written for is extremely dangerous. gnif’s answer in June 2026 was that for post 5000 cards those routines “are not only not needed, but can be dangerous as they poke at the GPU in a way that is unsupported and can result in physical damage to the device”. His workaround for later generations is to bind the card back to amdgpu and unbind it again before starting the VM, “which takes advantage of the updated and maintained reset routines built into the amdgpu module these days”. That works because binding is what triggers the reset. Yes there is a race condition here, but for most cards this is better than the alternative.

Do I still need vendor-reset if I use HardPass? It is recommended for the RX 5000 series. Nothing in userspace can replace it, because this reset has to happen inside the kernel’s PCI path.

And Will You Succeed? Yes, You Will Indeed

So the reset bug was never a defect in the silicon. AMD follows the law of the PCIe specification without following the spirit. A reset either reaches the PSP and the SMU or it doesn’t, and the ones Linux exposes for AMD don’t, and the on card one that works was never wired to anything Linux can call. On the modern cards the reset happens when amdgpu loads, not when it unloads, so a card you hand back and forth through amdgpu gets cleaned up on every way in. Which is why the amdgpu tends to rebind fine.

Working out which bug you’re dealing with when you’ve ruled out a reset bug costs you an afternoon and a prayer. HardPass checks the card before you ever start a VM, tells you whether this is the fault you actually have, and handles the non-reset-bug cases.

Related Posts