PROBLEMS WITH HOME NAS EXPANDABILITY

[sendel]

Hello everyone!

 

I have a couple of servers running TrueNas Scale that i wanted to merge in the main one, leaving me one free to tinker with Kubernetes and TalOS. This main server is built in a Fractal Design 7 XL that can hold up to 20 disks in the storage configuration.

The thing is that i built this server during the pandemic, and the hardware choices weren't the best (mainly due to availability in my area, Spain). So i have an AMD 3400G with 32 GB of RAM on a B450 MoBo. I have a total of 6 disks installed, plus an ssd with a USB adapter to boot the OS from.
I was looking into an HBA in order to migrate the 4 diks from the other server, but I realized that my mobo just have an operational x16 PCI slot that is x8 because of the iGPU, and it is now populated with a P200 that i use for transcoding. So, what would be  a good path for upgrading?

I am considering the following options:

• Built the server around other platform with more PCI lanes and slots
• Upgrade if possible the mobo to one that will allow me adding HBA (i couldn't find one)

I would love to retain as much hardware as possible (RAM for example) and keep TDP low (i am in 65W for CPU right now)

 

I'd really appreciate your advice in this one! Thanks

[Bersella AI]

Replacement to the motherboard would be more cost-efficient, since all you need is an additional PCIe x16 slot. Grab a B550 board such as:

• ASUS TUF B550M-PLUS
• Gigabyte B550M-DS3H
• ASRock B550M-Pro4

[ki8aras]

22 minutes ago, sendel said:

am considering the following options:

• Built the server around other platform with more PCI lanes and slots
• Upgrade if possible the mobo to one that will allow me adding HBA (i couldn't find one)

I would love to retain as much hardware as possible (RAM for example) and keep TDP low (i am in 65W for CPU right now)

 

I'd really appreciate your advice in this one! Thanks

you can look into pcie bifurcation cards, although you need to make sure your motherboard supports it

here's an example:


 

[sendel]

Hi! Thanks for the quick responses!

 

18 minutes ago, ki8aras said:

you can look into pcie bifurcation cards, although you need to make sure your motherboard supports it

here's an example:


 

I think this would not be an option, because my PCI x16 is already limited to x8 by de iGPU, and i need x8 for the HBA and x8 for the GPU (the p200)

 

21 minutes ago, Bersella AI said:

Replacement to the motherboard would be more cost-efficient, since all you need is an additional PCIe x16 slot. Grab a B550 board such as:

• ASUS TUF B550M-PLUS
• Gigabyte B550M-DS3H
• ASRock B550M-Pro4

I've been looking into these mobos and i think the Asrock would be a good fit:

[sendel]

Well, now that i look closely. Wouldn't i hit the same wall? The second x16 only would work on x4 mode, right? @Bersella AI

[Bersella AI]

1 minute ago, sendel said:

Well, now that i look closely. Wouldn't i hit the same wall? The second x16 only would work on x4 mode, right? @Bersella AI

I assume that the P100 GPU occupies two slots, so that it would not interfere with the HBA card. The latter would also work in x4 mode since it does not require a lot of bandwidth.

[sendel]

The HBA as far as i can tell needs at least x8 lanes, but maybe i'm wrong. This is the one i was looking into:

https://es.aliexpress.com/item/1005005030563910.html

 

the info that it need 8 lanes i took it from here:

[Bersella AI]

3 minutes ago, sendel said:

The HBA as far as i can tell needs at least x8 lanes, but maybe i'm wrong. This is the one i was looking into:

https://es.aliexpress.com/item/1005005030563910.html

 

the info that it need 8 lanes i took it from here:

This would not be a problem, since all PCIe cards support connections with fewer lanes. For example, you might have noticed reviews on TechPowerup which explored the impact of various PCIe configurations on cards like RTX 4090 and RX 6600XT, indicating that these cards are workable with fewer PCIe lanes.

[sendel]

Well, i'll give it a try then. Thanks for your patience and wisdom

[Needfuldoer]

37 minutes ago, sendel said:

The HBA as far as i can tell needs at least x8 lanes, but maybe i'm wrong. This is the one i was looking into:

https://es.aliexpress.com/item/1005005030563910.html

 

the info that it need 8 lanes i took it from here:

That figure is referring to SAS lanes, not PCIe lanes.

 

SAS is a little bit like a network, where multiple drives can share SAS lanes through a port multiplier (analogous to a network switch). If you connect a bunch of port multipliers through that HBA, they would all share those eight lanes' worth of bandwidth to the controller, which then has a PCIe Gen3 x 8 connection to the rest of your PC. Even if that card is plugged into a PCIe Gen3 x4 slot, that's 31 gigabits per second between the processor and the drives. You're not going to saturate that, especially not if you're only running a regular Gigabit Ethernet network.

[digitalscream]

Question: are you actually using SAS drives or SATA drives? If SATA, then it's pointless going for a SAS 12G card. You could just get a SAS 6G card and have exactly the same capacity and bandwidth, and that'll both be cheaper and only require PCIE 4x.

 

On top of that...can you boot the server using a SATA SSD instead of NVMe (or do you have a spare M.2 slot)?

(just read the OP more closely)

 

You can get an adapter that turns your M.2 slot into a PCIE 4x slot (you can get them which are physically 16x if necessary, or just an open-ended 4x). I'd advise getting the ones that use an Oculink interconnect, they're much easier to deal with. I've done this a few times, works a treat.

[sendel]

2 hours ago, Needfuldoer said:

That figure is referring to SAS lanes, not PCIe lanes.

 

SAS is a little bit like a network, where multiple drives can share SAS lanes through a port multiplier (analogous to a network switch). If you connect a bunch of port multipliers through that HBA, they would all share those eight lanes' worth of bandwidth to the controller, which then has a PCIe Gen3 x 8 connection to the rest of your PC. Even if that card is plugged into a PCIe Gen3 x4 slot, that's 31 gigabits per second between the processor and the drives. You're not going to saturate that, especially not if you're only running a regular Gigabit Ethernet network.

Oh nice to know! I'll take it into account then
 

 

1 hour ago, digitalscream said:

Question: are you actually using SAS drives or SATA drives? If SATA, then it's pointless going for a SAS 12G card. You could just get a SAS 6G card and have exactly the same capacity and bandwidth, and that'll both be cheaper and only require PCIE 4x.

 

On top of that...can you boot the server using a SATA SSD instead of NVMe (or do you have a spare M.2 slot)?

(just read the OP more closely)

 

You can get an adapter that turns your M.2 slot into a PCIE 4x slot (you can get them which are physically 16x if necessary, or just an open-ended 4x). I'd advise getting the ones that use an Oculink interconnect, they're much easier to deal with. I've done this a few times, works a treat.

I am using sata drives, but the m2 slot is deactivated if I use all of the SATA ports onboard