Can I use one of these cheap adapter cards to run 4 NVME SSDs from 1 PCIe 16x slot?

[mrleaw]

Hi everyone,

I'm planning to get 4 NVME SSDs (for NAS Server purposes) and was wondering if my mainboard (ASUS ROG Strix Z370 H Gaming) can handle to drive all of them through a single PCIe 16x slot. I also attached a screenshot from the motherboard manual, which makes it seem like it's not a supported configuration. Or should I just use two of the mainboard M.2 slots and get a PCIe to 2x M.2 NVME adapter card for the other 2 SSDs?

Thanks for your help

[Radium_Angel]

Do you want the 4 SSDs to be bootable or in RAID mode?

[emosun]

are you doing a server task that requires the drive speed to exceed the networks speed

[porina]

See if you can understand this page.  https://www.asus.com/us/support/FAQ/1037507/

It is for Asus' own version of such a card but I'd guess similar applies to others. Your board is on the list but I don't get exactly what configuration is supported. It doesn't look like 4 whatever it is.

 

Edit: I think I get it now. You can use 3 in one slot, and 0 in another. OR you can use 1 in 1st slot and 2 in other. Either way, that's still 3 total. I think this is because the x16 slots can only split to x8/x4/x4.

 

Quote

Or should I just use two of the mainboard M.2 slots and get a PCIe to 2x M.2 NVME adapter card for the other 2 SSDs?

This should be possible, but you need to double check where the mobo gets the lanes from for the on board M.2 so it doesn't conflict with the PCIe slots.

[Morgan MLGman]

1 hour ago, mrleaw said:

I'm planning to get 4 NVME SSDs (for NAS Server purposes) and was wondering if my mainboard (ASUS ROG Strix Z370 H Gaming) can handle to drive all of them through a single PCIe 16x slot.

It is possible to use one slot to drive all of those drives, however as long as you have a GPU in the primary slot, there is going to be a bandwidth limitation as inserting this card into the second slot will probably drive the speed of both slots to x8/x8. This already reduces the available bandwidth for your GPU as well as for this card, as what would be possible would only be 4 x M.2 PCIe 3.0 x2. Still, even that would require a motherboard that supports PCIe bifurcation.

There is a certain limit of bandwidth that both the CPU and the Chipset provide to the motherboard ports, it is not advised to shoot too close to that limit and it's definitely not the case that all of the ports can run at their maximum bandwidth at once.

Another thing is that if you use such a "cheap" card, it doesn't have a hardware RAID controller of any kind, so all RAID is purely software in this case. This has both upsides and downsides.
You could run two drives from the onboard M.2 slots and the other two from the add-on 2 x m.2 card, although you'd need to check your motherboard manual if anything gets disabled if the secondary onboard M.2 slot is populated, as well as what kind of bandwidth that slot supports. Mixing all of that into a RAID is not an optimal solution, although a software RAID should still work.
For such configurations people usually turned to HEDT platforms due to those having a lot more available PCIe lanes for devices like M.2's and were better suited for RAID.

I guess you'd need to specify your needs better and evaluate from there what solution you actually require. Maybe a better solution would be to buy a dedicated NAS like a small Synology or QNAP?

[NorKris]

18 minutes ago, emosun said:

are you doing a server task that requires the drive speed to exceed the networks speed

he might have 10gig  

[mrleaw]

13 minutes ago, emosun said:

are you doing a server task that requires the drive speed to exceed the networks speed

I'm running 5x4TB HDDs (RAIDZ2 configuration) in my TrueNAS server right now. And am planning on using two of them as L2ARC, and the other two as a second pool with the two SSDs mirrored for my TrueNAS Apps and VMs

[PDifolco]

33 minutes ago, mrleaw said:

Hi everyone,

I'm planning to get 4 NVME SSDs (for NAS Server purposes) and was wondering if my mainboard (ASUS ROG Strix Z370 H Gaming) can handle to drive all of them through a single PCIe 16x slot. I also attached a screenshot from the motherboard manual, which makes it seem like it's not a supported configuration. Or should I just use two of the mainboard M.2 slots and get a PCIe to 2x M.2 NVME adapter card for the other 2 SSDs?

Thanks for your help

Dont think it can work as your board only support x8/x4+x4 and not x4x4x4x4 (no retail board does afaik) 

rgds

[NorKris]

2 minutes ago, PDifolco said:

 not x4x4x4x4 (no retail board does afaik) 

 

that is what i had in my system before  

[mrleaw]

8 minutes ago, Morgan MLGman said:

It is possible to use one slot to drive all of those drives, however as long as you have a GPU in the primary slot, there is going to be a bandwidth limitation as inserting this card into the second slot will probably drive the speed of both slots to x8/x8. This already reduces the available bandwidth for your GPU as well as for this card, as what would be possible would only be 4 x M.2 PCIe 3.0 x2.

There is a certain limit of bandwidth that both the CPU and the Chipset provide to the motherboard ports, it is not advised to shoot too close to that limit and it's definitely not the case that all of the ports can run at their maximum bandwidth at once.

Another thing is that if you use such a "cheap" card, it doesn't have a hardware RAID controller of any kind, so all RAID is purely software in this case. You could run two drives from the onboard M.2 slots and the other two from the add-on 2 x m.2 card, although you'd need to check your motherboard manual if anything gets disabled if the secondary onboard M.2 slot is populated, as well as what kind of bandwidth that slot supports. Still mixing all of that into a RAID is not an optimal solution, although a software RAID should still work.
For such configurations people usually turned to HEDT platforms due to those having a lot more available PCIe lanes for devices like M.2's and are better suited for RAID usually.

I guess you'd need to specify your needs better and evaluate from there what solution you actually require. Maybe a better solution would be to just buy a dedicated NAS like a small Synology or QNAP?

So I guess I'll go with two ssds in the mainboard directly as these m.2 ports can be used and don't disable any other slots, and then use a 2x ssd card in the secondary PCIe slot. I guess 8x for the GPU should be fine as I'm just using my old GTX 1080 for it's NVENC features, just need to encode, decode, and transcode probably one stream at the same time.

Of course HEDT would be the better choice, but I have almost all of the components lying around, so might as well try to use them.

Thanks for your help

[OhYou_]

check if your motherboard supports pci-e bifurcation 

[mrleaw]

Just now, OhYou_ said:

check if your motherboard supports pci-e bifurcation 

it didn't mention it, so I guess it doesn't 

[PDifolco]

8 minutes ago, NorKris said:

that is what i had in my system before  

Which board did you have ? Server/Threadripper one ?

[NorKris]

Just now, PDifolco said:

Which board did you have ? Server/Threadripper one ?

x299  

 

but all u need is a 16slot with bifurcation and ur GPU in another slot (if u need a GPU that is..) 

[Electronics Wizardy]

21 minutes ago, mrleaw said:

I'm running 5x4TB HDDs (RAIDZ2 configuration) in my TrueNAS server right now. And am planning on using two of them as L2ARC, and the other two as a second pool with the two SSDs mirrored for my TrueNAS Apps and VMs

What are you using the HDD pool for? I'd generally skip the l2arc. From my uses, the Hit ratio is very low and provides almost no benfit for home server uses.

[Morgan MLGman]

42 minutes ago, mrleaw said:

it didn't mention it, so I guess it doesn't 

Without PCIe bifurcation support there's no way for such card to work then, for instance my MSI X570 Prestige Creation comes with an MSI Xpander card which adds two M.2 PCIe 4.0 x 4 slots and it requires setting the slot into a x4/x4 mode instead of x8 to use it. The card you linked would require a full X16 bandwidth split into x4/x4/x4/x4.

This feature as I mentioned was usually available on HEDT motherboards, not mainstream ones like yours.

The only way for it to work would be to use both M.2 slots on your motherboard and a dual M.2 add-on card put in the second slot and setting that slot into x4/x4 mode, provided your motherboard supports that.

[porina]

53 minutes ago, Morgan MLGman said:

Without PCIe bifurcation support there's no way for such card to work then

The manual excerpt in OP shows two x16 slots break down to 8/4/4, and the link in my earlier post to Asus shows the mobo could support 3 drives in that configuration. So it partially works. The 4th drive can't be used since the mobo doesn't break some of the lanes below x8.

[Morgan MLGman]

20 minutes ago, porina said:

The manual excerpt in OP shows two x16 slots break down to 8/4/4, and the link in my earlier post to Asus shows the mobo could support 3 drives in that configuration. So it partially works. The 4th drive can't be used since the mobo doesn't break some of the lanes below x8.

I think it means that (assuming there is something present in the primary x16) once the second slot is occupied with anything, the first slot defaults to x8 and the second slot is another x8 which may be further divided into x4/x4 for those dual M.2 cards like the ASUS Hyper M.2 or MSI M.2 Xpander

Although this seems to say that three SSDs are possible, provided you use the primary PCIe slot? Not really sure what "3/1" or "0/2" mean exactly:

[porina]

4 minutes ago, Morgan MLGman said:

I think it means that (assuming there is something present in the primary x16) once the second slot is occupied with anything, the first slot defaults to x8 and the second slot is another x8 which may be further divided to x4/x4 for those dual M.2 cards like the ASUS Hyper M.2 or MSI M.2 Xpander

Looking at it again, you're right, but I was partially right. It looks like the two x16 slots split the CPU lanes to x8+x8 if both are in use.

 

From the link I posted before:

 

It looks like if you put a 4 M.2 card in the main (normally GPU) slot with 2nd empty, that can run 3 M.2 drives. If you use the 1st slot as x8 (potentially one x4 M.2) then the 2nd slot can break its 8 lanes into x4+x4 supporting two more M.2 drives. It looks like the first slot can only break the lower 8 lanes as one group.

[kokosnh]

you could also use a PCIe switch card, but that will cost you, the PCIe 3.0 ones cost 200-300$ for the cheaper ones...