Help with PCIe lanes and PCH lanes please on x470 MB

[Gear-K]

Hello, I am putting together a new PC, not bleeding edge but decent fast for me, on the eve' of the new Navi releases.  My new PC build is as follows.

 

ASUS Prime X470-Pro AM4 AMD X470

4 - G.Skill Trident F4-3200C14D-16GTZR memory modules (I know only 2 are QVL but it's C14 so I'm going to give 4 sticks a whirl)

ASUS ROG Strix1080ti (bitcoin miners tears are my gain here for a used card)

AMD RYZEN 7 2700X 8-Core 3.7 GHz

Western Digital WD Blue SN500 NVMe M.2 2280 500GB PCI-Express 3.0 x2 for the 2nd M.2 slot

SAMSUNG 970 EVO PLUS M.2 2280 1TB PCIe Gen 3.0 x4, for the main M.2 slot

(few more things, Noctua cooler, evga 750 pw supply, bunch of RGB fans, and a couple of WD Black HDD SATA drives)

 

Now here is where I'm left wondering.  I can not find any clear documentation about high speed IO lanes or 'chipset lanes' for the x470 boards.  I know the Graphics card is going to try to slurp up all the x16 PCIe lanes direct to the CPU, but what will the M.2 NVMe slots use?  The primary slot wants to consume x4 and the secondary wants to use x2, are there high speed IO lanes for them to use from the PCH or are they going to have to steal from the CPU lanes and drop my video card to x8 lanes??  

 

Anyone know the esoteric ways of the PCIe and PCH lanes for the AMD chipsets out there? 

 

[Stormseeker9]

What exact psu will you be using (model) ?

 

550w would be plenty here. Get a Corsair RM550x or similar and save some dollars.

[Gear-K]

Planning on this Power Supply, I like to have a bit of 'room' with the Power supply as I tend to keep them at least one MB/cpu upgrade iteration.  

EVGA 750 GQ 210-GQ-0750-V1 80+ GOLD 750W Semi Modular EVGA

 

[Gear-K]

I have searched for this and this is the closest answer I found but it's a clear answer for the x370 chipset (and the B350/320 as well), but this isn't the same as the x470.  

  Following this I'd be using all 16 for the GPU,

then I'd be using "2nd row in the diagram" the 4 'chipset' lanes for the M.2 NVMe for the first NVMe,

and then finally the "4th row in the diagram" option A for 2 Sata and 1x2 NVMe,

this would be perfect!  I get a slow down on the first or second NVMe transfers if I used some of the USB 3.1 ports (possibly, trial and error might show which ports are sharing the lanes), but this isn't my chipset and I can't find any similar data on the x470 over the past 2 days of searching. 

[mariushm]

The processor has 24 PCI-E lanes, which are arranged like this:

 

16 go to video card ( x370 and x470 chipsets allow to be split into 2 x8 slots for video cards, for SLI... because SLI requires minimum x8 while crossfire works even with x4)

4 go to the first M.2 connector

4 go to the chipset

 

Note some CPUs with integrated graphics only output 8 pci-e lanes instead of 16 lanes for video.

 

So the m.2 connector closest to the CPU socket is connected directly to CPU using 4 pci-e 3.0 lanes.

 

The chipsets create additional pci-e lanes, which are pci-e 2.0 (x5xx series will create pci-e 4.0 lanes)

If I remember correctly, x370 and x470 chipsets create 8 pci-e 2.0 lanes which are usually arranged as x4 + 2 x1 + maybe some onboard stuff. Some motherboards route 2 or 4 of these pci-e lanes to a 2nd m.2 connector, or the pci-e x4 slot gets disabled if you install a nvme drive to 2nd m.2 connector (because the 4 pci-e lanes from the x4 slot are re-routed to m.2 connector)

 

The x350 and x450 create only 6 pci-e 2.0 lanes, so typically you have less pci-e slots, and no 2nd m.2 nvme slot, the 2nd slot is often SATA only.

 

[nick name]

In the manual:

[Gear-K]

Thanks @Nick and @Mariushm, your input led me to a little more info, at first I had seen what you both posted and it seemed in conflict (the manual stating all the first M.2 was PCIe 3.0 x4 lane and second M.2 was x2 PCI 3.0 x2 lanes (which steals lanes from PCIe x1_1 and x1_3), but then the chipset I think it still only creating PCI 2.0 x8 lanes for use through the PCH.  So I was confused. 

 

I think I found the answer but there is a LOT of conflicting information out there, so this article (AnandTech) has details on the X470 chipset and it echos what GN mentioned that the x470 chipset is almost identical to the x370 chipset.  So I think it's clear both have 8 PCI 2.0 lanes from the PCH.  Then in this article (Tom's hardware) even though they seem to misstate the Ryzen 7 has 28 PCIe lanes, though the majority of posts out there state it has 24 PCIe 3.0 lanes (I see some stating as few as 20 PCIe lanes), this article connected a few dots for me.  I was then able to search further based on these 2 new pieces of information. 

 

So I think what's the throughput status is this, 24 lanes from the CPU, 4 are dedicated to the first M.2 NVMe slot, and then 4 are dedicated to the PCH to create those 8 PCI 2.0 lanes??  Which would mean that if I put 2nd M.2 drive in the PCIe x2 slot (stealing the stated throughput for PCIe 1_1 and 1_3) then I think there are only x2 PCI 3.0 lanes left over for the chipset to push the data from USB devices through which might be OK, or there are still the x4 lanes available to the PCH and I will just see contention or slowdowns when I use USB 3.0.  What seems a bit annoying is that from the "tom's" article above, the M.2 NVMe slot that has the dedicated x4 PCI 3.0 slots, shares those pipes with the SATA connectors??  If that's the case I wonder how the chipset manages traffic for those lanes and does anything get preference (hopefully the M.2 devices in every case, IMHO)? 

 

Now while this all makes some sense now, I still HOPE this is how the M.2 slots work on the ASUS boards, as from that manual page above it doesn't state anything about the sharing of the PCIe lanes for the M.2 device or where the second M.2 device gets it's x2 PCI 3.0 lanes, the above is my current best guess. 

 

[mariushm]

33 minutes ago, Gear-K said:

So I think what's the throughput status is this, 24 lanes from the CPU, 4 are dedicated to the first M.2 NVMe slot, and then 4 are dedicated to the PCH to create those 8 PCI 2.0 lanes??  Which would mean that if I put 2nd M.2 drive in the PCIe x2 slot (stealing the stated throughput for PCIe 1_1 and 1_3) then I think there are only x2 PCI 3.0 lanes left over for the chipset to push the data from USB devices through which might be OK, or there are still the x4 lanes available to the PCH and I will just see contention or slowdowns when I use USB 3.0.  What seems a bit annoying is that from the "tom's" article above, the M.2 NVMe slot that has the dedicated x4 PCI 3.0 slots, shares those pipes with the SATA connectors??  If that's the case I wonder how the chipset manages traffic for those lanes and does anything get preference (hopefully the M.2 devices in every case, IMHO)? 

 

 

No, it's not like this.

 

Think of the x470 chip as a network switch or a router, which has a bunch of network ports. It can take signal from a bunch of network cards / computers and mixes the data packets together and sends them through a single cable to the outside world (to the router, or to the ISP and so on).

 

The CPU connects to the x470 chip using pci-e 3.0 x4 connection. This means there's approximately 4 GB/s theoretical speed in both directions. (1 pci-e 3.0 lane is ~970 MB/s in both directions)

 

The x470 chip creates 8 pci-e 2.0 lanes, where each lane has a maximum speed of 500 MB/s in both directions.

But these are only the pci-e lanes that go outside the chip and which can be routed to M.2 connectors, or to PCI-E slots or to onboard chips (sound cards, extra usb controllers, extra sata controllers, various more exotic things like let's say PCI to PCI-e bridge chips to add PCI slots for backwards compatibility ).

The x470 also creates SATA connectors and USB 2 ports and USB 3.0 ports (5gbps) and USB 3.1 gen 2 ports (10 gbps)

 

Imagine that inside the x470 chip there's several pci-e 2.0 lanes which are directly connected to these internal controllers:

 

4 pci-e lanes go to the SATA controller which handles the 6 SATA ports OR 3 pci-e lanes are used to connect 3 separate SATA controllers, each controlling 2 SATA ports (so you have 6 SATA ports in total)

1 pci-e lane inside the chip goes to a USB 2 controller which creates up to 7 USB 2.0 ports

2 or more pci-e lane go to the USB 3 controller (5gbps) which creates 4 usb 3.0 ports - so you have multiples of 500 MB/s creating 4 USB 3 ports

1 or several pci-e lanes go to the USB 3.1 gen 2 controller (10gbps) which creates 1 or 2 usb 3.1 gen 2 ports

 

So basically the x470 takes in 4 pci-e 3.0 lanes (4 GB/s) and may very well create 16 or 24 pci-e 2.0 lanes but only 8 are exposed to the outside world, for the pci-e slots or m.2 connectors... the other are hidden inside the chip, connecting everything else the chip offers.

Or, the chip could create 8 pci-e 3.0 lanes internally, and then have a tiny pci-e controller which takes in 2 pci-e 3.0 lanes and creates 8 pci-e 2.0 lanes and the other 6 pci-e 3.0 lanes are used internally inside the chip to connect the other controllers.

So internally you could have in total 8-10 GB/s of bandwidth (16-24 x 500 MB/s) or 8 GB/s (if we go with the 8 x pci-e 3.0 lanes version), but because you're not using ALL of these lanes near their maximum throughput, the data coming and going into those lanes can be multiplexed together and squeezed into those 4 GB/s that are between CPU and processor.

 

Also, keep in mind that for some things, you don't have to move data through that channel that goes to the CPU. For example, if you copy a file from one hard drive to another, and both hard drives are connected to SATA ports, then the data could very well simply be routed between SATA ports, it doesn't have to be sent all the way to the CPU only to be sent back to the chipset, to go in the other SATA port.

 

---

 

The first m.2 receives its pci-e lanes directly from CPU, so you know for sure they're pci-e 3.0 x4 , which means you have a maximum theoretical speed of 4 x ~970 MB/s in both directions.

The second m.2 connector may not have pci-e lanes at all, or may have 2 or 4 pci-e lanes, and considering the x470 chip creates only pci-e 2.0 lanes, you know you'll have 2 or 4 pci-e 2.0 lanes. So, a maximum speed of 2-4 x 500 MB/s

 

nick name above has attached a screenshot of the manual which tells you... the second m.2 connector only has 2 pci-e lanes routed to the connector.  To the best of my knowledge, x470 chipset only creates pci-e 2.0 lanes, so I'd say the manual is wrong when it says there pci-e 3.0 x2, i doubt it's 3.0

 

Also, you can see it says that if you insert something in the pci-e x1_1 or pci-e x1_3  slots (two pci-e x1 slots), then the bios gives priority to those slots, so it disables the pci-e lanes to the second m.2 connector, as there's not enough pci-e lanes to connect both those two pci-e x1 slots and the second m.2 connector to the chip.

 

 

 

[Gear-K]

@mariushm I think we are on the same page, just saying the first part differently.  I agree with you also that the second M.2 connector is going to be 'interesting' to test out, if it has x2 PCI 2.0 lanes that will be easy to benchmark vs x2 PCI 3.0 lanes.  IF it has x2 PCI 3.0 lanes I will come back here and we can both guess at how the PCH is doing what it's doing and still giving x2 PCI 3.0 lanes to that M.2 port.  It would make a lot more sense if it was x2 PCI 2.0 lanes!    Thanks!