Please Educate me

[iMint]

Can someone please educate me on something.  So I watched the $100k P2 vid the other day and Linus had a splitter running off of 1 16x PCIe lane and that got me to thinking wouldn't that be a issue with the amount bandwidth running off of 1 lane? But when I saw it he had like 2 or 3 cards plugged into the splitter, but that splitter was running off of 1 lane.  Not 100% on how that would work so thats why I'm asking.

 

Registered for this forum a long time ago and just now remembered it was here lol.  typed "L" in my address bar in this came up so I figured I'd ask the question that has been bugging me for a day or so.

[Spotty]

30 minutes ago, iMint said:

So I watched the $100k P2 vid the other day and Linus had a splitter running off of 1 16x PCIe lane and that got me to thinking wouldn't that be a issue with the amount bandwidth running off of 1 lane? But when I saw it he had like 2 or 3 cards plugged into the splitter, but that splitter was running off of 1 lane.  Not 100% on how that would work so thats why I'm asking.

The slot can run at x16, but that doesn't mean the device plugged in to it requires the full bandwidth provided by the x16 PCIe lanes.
For example, even a 1080ti can run just as well with PCIe x8 lanes as it can with PCIe x16 lanes.

Spoiler

From a quick look, there is a little below a 1% performance difference in PCI-e 3.0 x16 and PCI-e 3.0 x8 slots. The difference is not even close to perceptible and should be ignored as inconsequential to users fretting over potential slot or lane limitations.
https://www.gamersnexus.net/guides/2488-pci-e-3-x8-vs-x16-performance-impact-on-gpus

 

The Optane PCIe SSDs that he was using requires PCIe x4 lanes, so you could in theory split PCIe x16 lane up to support 4 of those devices without running in to bandwidth limitations. It's also possible that the Optane SSDs actually require less bandwidth than that in reality. Likewise the USB hubs they are going to use will most likely not require a huge amount of bandwidth over the PCIe lanes, requiring probably PCIe x1 lane.

Spoiler

Interface PCIe NVMe 3.0 x4
https://ark.intel.com/products/123626/Intel-Optane-SSD-900P-Series-480GB-12-Height-PCIe-x4-20nm-3D-XPoint

It's also worth considering that the workstation CPUs and motherboards being used in the build support a lot more PCIe lanes than a standard consumer CPU/board does. So it's possible that most, if not all, of the PCIe x16 slots on the motherboard can support full x16 speeds, whereas in a consumer board it's typically just the top slot support x16 bandwidth, with the 2nd slot supporting x8 and the third supporting x4, and also where the first slot is reduced to x8 if the other slots are populated - or some similar configuration depending on the exact motherboard/CPU.

Spoiler

https://www.asus.com/au/Motherboards/ROG-STRIX-Z370-E-GAMING/specifications/

 

I'm not sure which exact workstation CPUs and motherboards they are using, but if they say the model number in the video you will be able to search it and find out how many PCIe lanes it supports across all of the motherboards PCIe slots (if they don't mention it in the videos).

We will just have to wait and see in Part 3 what solution they end up going with for having all the graphics cards, storage, and USB add in cards installed in to the system. From the video it appeared that they are going to struggle to get UnRaid to detect the attached devices individually through the PCIe splitter. (AFAIK Unraid needs to detect them as individual devices so you can passthrough control of them to individual virtual machines)

[iMint]

16 hours ago, VegetableStu said:

depends on the use case. (that's a x4 lane PCIe bridge by the way)

Linus is using it for plugging USB 3.1 cards onto it, and those rarely saturates a PCIe 3.0 x4 connection all at the same time (and even if it did, it's just USB storage sticks or even SD cards, so the only thing that would slow down are transfer speeds, not system hardware processing)

 

(his idea of multiplexing both a GPU and a storage device on the same x16 slot though... problem is the SSD will still have to round-trip through the CPU back to the GPU for texture loading or video data streaming (hence AMD's Radeon SSG)(although Nvidia vaguely has a similar API, but that's on the program side and not exposed to the driver side if I read right (i.e. the card does not do this automatically (e.g. for (say) VLC player))), but other than that I'm guessing there might be very little overlap)

Ohhhh.  Then we saw it wrong because I thought he was using the graphics cards on the splitter which was why I thought there would be a bandwidth issue.

 

Thanks