Second GPU/ Asus Proart b550 creator

[Dancrafter]

Second GPU
According to the instructions, packaging and website, it is possible to install a graphics card in the 2nd slot of the Asus Proart b550-creator mainboard. This is with reduced bandwidth (x8 instead of x16) and with reduced bandwidth on slot 1, but that would be fine.
Unfortunately, the graphics card in slot 2 is not recognized.

In Slot 1 and 3 both a single and 2 GPU's work, but that wont work for me because teh iommu Groups of the third slot are linked to the networkcard.

Unfortunately there is no function in the bios to change the lanes of the slots manually/ I have not been able to find it yet.

My CPU 9500x has enough lanes to handel x8/x8.

 

The problem also persists with:
different GPUs (970,3070ti, Arc A750)
with and without SSD (should be connected directly to the Cpu anyway)
Bios updated
Bios reset
another card (network) works in the slot

 

because the Warranty expired 1 Month ago the darn Asus support leaves me hanging for at least two hours in their phone line. (stopped trying to reach them after 3h) :)

[Tetras]

21 minutes ago, Dancrafter said:

Unfortunately there is no function in the bios to change the lanes of the slots manually/ I have not been able to find it yet.

I think it would be under Advanced > Onboard Devices Configuration > PCIEX16_1 Bandwidth and PCIEX16_2 Switch.

[igormp]

I have that exact same motherboard with a 5950x, and I'm running 2x3090s on it without any issues.

 

Is your issue that you have not been able to passthrough it to a VM, or does the host OS not see the 2nd GPU at all?

[Dancrafter]

On 4/25/2025 at 4:22 PM, Tetras said:

I think it would be under Advanced > Onboard Devices Configuration > PCIEX16_1 Bandwidth and PCIEX16_2 Switch.

Hi, there i can only turn on x8\x4\x4 Bifurcation, witch to my knowledge is for support of specific pcie to M.2 expansion cards.

[Dancrafter]

On 4/25/2025 at 4:53 PM, igormp said:

I have that exact same motherboard with a 5950x, and I'm running 2x3090s on it without any issues.

 

Is your issue that you have not been able to passthrough it to a VM, or does the host OS not see the 2nd GPU at all?

Yes this would be the plan. But currently if the GPU is inserted in the second slot the Host system doesn't register the card at all.

 

[igormp]

57 minutes ago, Dancrafter said:

Hi, there i can only turn on x8\x4\x4 Bifurcation, witch to my knowledge is for support of specific pcie to M.2 expansion cards.

This setting is only relevant if you want to use one of those PCIe splitters to plug multiple devices within a single slot.

58 minutes ago, Dancrafter said:

Yes this would be the plan. But currently if the GPU is inserted in the second slot the Host system doesn't register the card at all.

This seems to be mostly a software issue, given that a network card in that slot along with a GPU works.

What's your host system? How are you verifying if it's working or not?

 

I'm assuming you're using a linux host since you're looking into passthrough stuff, does it not show at all on a lspci? Is there nothing relevant within your dmesg logs?

[Dancrafter]

52 minutes ago, igormp said:

This setting is only relevant if you want to use one of those PCIe splitters to plug multiple devices within a single slot.

This seems to be mostly a software issue, given that a network card in that slot along with a GPU works.

What's your host system? How are you verifying if it's working or not?

 

I'm assuming you're using a linux host since you're looking into passthrough stuff, does it not show at all on a lspci? Is there nothing relevant within your dmesg logs?

Hostsystem is Proxmox, lspci and dmesg don't recognize it at all.

[igormp]

1 hour ago, Dancrafter said:

Hostsystem is Proxmox, lspci and dmesg don't recognize it at all.

With both GPUs installed, what's the link width of your top GPU?

You should be able to check this with an lspci -vv under the LnkSta part. Here's mine for example:

3c:00.0 VGA compatible controller: NVIDIA Corporation GA102 [GeForce RTX 3090] (rev a1) (prog-if 00 [VGA controller])
[...]
		LnkCap:	Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <512ns, L1 <4us
			ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes, LnkDisable- CommClk+
			ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 2.5GT/s (downgraded), Width x8 (downgraded)
			TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

 

[Dancrafter]

Had to fix the Networking before i got both Gpus running at the same time with any kind of System access again.

The Nvidia GPU seems to be stuck in x16 bandwidth mode.

 

3b:00.0 VGA compatible controller: NVIDIA Corporation GA104 [GeForce RTX 3070 Ti] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: Gigabyte Technology Co., Ltd GA104 [GeForce RTX 3070 Ti]
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 170
        IOMMU group: 16
        Region 0: Memory at ed000000 (32-bit, non-prefetchable) [size=16M]
        Region 1: Memory at 7fe0000000 (64-bit, prefetchable) [size=256M]
        Region 3: Memory at 7ff0000000 (64-bit, prefetchable) [size=32M]
        Region 5: I/O ports at f000 [size=128]
        Expansion ROM at ee000000 [disabled] [size=512K]
        Capabilities: [60] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
                Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
                Address: 00000000fee00000  Data: 0000
        Capabilities: [78] Express (v2) Legacy Endpoint, MSI 00
                DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
                        ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
                DevCtl: CorrErr+ NonFatalErr+ FatalErr+ UnsupReq+
                        RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
                        MaxPayload 256 bytes, MaxReadReq 512 bytes
                DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
                LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <512ns, L1 <4us
                        ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
                LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
                        ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
                LnkSta: Speed 2.5GT/s (downgraded), Width x16
                        TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
                DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR-
                         10BitTagComp+ 10BitTagReq+ OBFF Via message, ExtFmt- EETLPPrefix-
                         EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
                         FRS-
                         AtomicOpsCap: 32bit- 64bit- 128bitCAS-
                DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq+ OBFF Disabled,
                         AtomicOpsCtl: ReqEn-
                LnkCap2: Supported Link Speeds: 2.5-16GT/s, Crosslink- Retimer+ 2Retimers+ DRS-
                LnkCtl2: Target Link Speed: 16GT/s, EnterCompliance- SpeedDis-
                         Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
                         Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
                LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
                         EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
                         Retimer- 2Retimers- CrosslinkRes: unsupported
        Capabilities: [b4] Vendor Specific Information: Len=14 <?>
        Capabilities: [100 v1] Virtual Channel
                Caps:   LPEVC=0 RefClk=100ns PATEntryBits=1
                Arb:    Fixed- WRR32- WRR64- WRR128-
                Ctrl:   ArbSelect=Fixed
                Status: InProgress-
                VC0:    Caps:   PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
                        Arb:    Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
                        Ctrl:   Enable+ ID=0 ArbSelect=Fixed TC/VC=01
                        Status: NegoPending- InProgress-
        Capabilities: [258 v1] L1 PM Substates
                L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+
                          PortCommonModeRestoreTime=255us PortTPowerOnTime=10us
                L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1-
                           T_CommonMode=0us LTR1.2_Threshold=32768ns
                L1SubCtl2: T_PwrOn=10us
        Capabilities: [128 v1] Power Budgeting <?>
        Capabilities: [420 v2] Advanced Error Reporting
                UESta:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UEMsk:  DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
                UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
                CESta:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
                CEMsk:  RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
                AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
                        MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
                HeaderLog: 00000000 00000000 00000000 00000000
        Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
        Capabilities: [900 v1] Secondary PCI Express
                LnkCtl3: LnkEquIntrruptEn- PerformEqu-
                LaneErrStat: 0
        Capabilities: [bb0 v1] Physical Resizable BAR
                BAR 0: current size: 16MB, supported: 16MB
                BAR 1: current size: 256MB, supported: 64MB 128MB 256MB 512MB 1GB 2GB 4GB 8GB
                BAR 3: current size: 32MB, supported: 32MB
        Capabilities: [c1c v1] Physical Layer 16.0 GT/s <?>
        Capabilities: [d00 v1] Lane Margining at the Receiver <?>
        Capabilities: [e00 v1] Data Link Feature <?>
        Kernel driver in use: vfio-pci
        Kernel modules: nvidiafb, nouveau

 

[igormp]

On 5/1/2025 at 8:18 AM, Dancrafter said:

The Nvidia GPU seems to be stuck in x16 bandwidth mode.

Dang, it's like the 2nd GPU really doesn't exist at all.

I unfortunately can't reboot my desktop at this point in time, so I can't look into the bios to try to find anything useful in there.

A bit grasping at straws at this point, but:

1- Is your BIOS up to date? If so, could you try to actually downgrade to a previous version just for sanity checking?

2- The 2nd GPU does get powered properly and has its fans spinning, right?

3- Try to downgrade the PCIe by forcing it to 3.0 in both slots, just as a sanity check

4- Kind of a desperate attempt, but you could try to cover half of the pins on the GPU that's on the first slot so it forces it to only use x8 lanes. This way the other 8 lanes would be electrically "free", and hopefully your 2nd GPU could be picked up. Something like so should work:

(source)

[Dancrafter]

I currently have the latest Bios installed have gone down to a version from 2024 i had lying around.

Power doesn't seem to be the issue the GPU works fine in the 3d slot, and in the second slot also spins up and gets warm.

I tried with both cards and just the second forced to pcie gen3, no difference.

 

Taping the Gpu is an interesting idea, altough i think i would have to tape it down to a x4 width :).

Maybe gone try that sometime.

 

[igormp]

30 minutes ago, Dancrafter said:

altough i think i would have to tape it down to a x4 width :).

Don't think so, you only have x16 lanes available for both GPUs, the first one is eating all of those. My idea was that, with some taping halfway through, the first GPU would only be "allowed" to use x8 lanes, leaving the other x8 lanes available, which the 2nd GPU should hopefully pick up.

[Dancrafter]

Going through the Manual again, i have found one interesting thing.

According to the manual i should have the option to switch bifurcation to x4/x4/x4/x4 since i have a 5000 Series CPU, instead i have x8/x4/x4 as an option so the one that corresponds to the 4000 Series CPUs.