All SSD Home NAS Hardware Recommendation

[sageofredondo]

 

I am thinking of a home all SSD server with one (or two) of those ASUS 4x nvme pcie cards with consumer ssds.

A few reasons for this:

* I have sensory issues and limited space. The only noise I want to hear is some Noctua fans.

* Limited space means an mATX (2x4 nvme) or an ITX (1x4 nvme) case and no need for a NAS card.
* I may want a developer's VM on the NAS box and we devs love speedy systems.
* 10 gig networking cards or maybe 25 gig fiber down the line.
* all ssd nas also means more free ram for containers or a vm and less power demand.
 

Quote

One common complain I hear is that all the ssds could fail at the same time since they have write maximums. That did not happen in this case in the video; I am inferring from the video that you ran the server for months until you replaced it with the new server so they did not fail at the same time. Is there any truth to it in practice?

@Nystemy gave a good answer for this here:

I plan to use TrueNAS Scale (Linux version, I want Docker/Podman). I am aware of running data consistency checks (I watched LTT not setup a CentOS ZFS machine properly).

I wanted to know if anyone has done something like this or they have any build recommendations. AVX2 is the bare minimum I would consider for CPU compatibility. I have an i7-10700k with 64 gigs of non-ecc ram I am considering using as this NAS system and upgrading my gaming system to a Ryzen 7800X3D.

[TimedPing]

I'm running 6x Samsung PM981a 1TB SSD (OEM version of 970 evo) for my VM boot drive pool in RAIDZ1, no complain so far and I can get it for cheap and have pretty good TBW in my opinion. It's running on old Haswell-EP E5-2695v3 with 4x 32GB RDIMM RAM and TrueNAS Scale for OS. If you can get this SSD for cheap, it's a must try. Another option will be the newer Samsung PM991a. Just make sure your motherboard support PCIe bifurcation in x4x4x4x4 mode before you touch anything else, if not then you need to either motherboard that support it or buy more moderately expensive PLX NVMe card.

[sageofredondo]

5 hours ago, TimedPing said:

I'm running 6x Samsung PM981a 1TB SSD (OEM version of 970 evo) for my VM boot drive pool in RAIDZ1, no complain so far and I can get it for cheap and have pretty good TBW in my opinion. It's running on old Haswell-EP E5-2695v3 with 4x 32GB RDIMM RAM and TrueNAS Scale for OS. If you can get this SSD for cheap, it's a must try. Another option will be the newer Samsung PM991a. Just make sure your motherboard support PCIe bifurcation in x4x4x4x4 mode before you touch anything else, if not then you need to either motherboard that support it or buy more moderately expensive PLX NVMe card.

What are you using for the PCIe card? I do not see it either in the above or in your system list in your signature.

What would be a 'more moderately expensive PLX NVMe card'?

[TimedPing]

4 minutes ago, sageofredondo said:

What are you using for the PCIe card? I do not see it either in the above or in your system list in your signature.

What would be a 'more moderately expensive PLX NVMe card'?

I mentioned it in my signature under homelab AIO server, I'm using generic Chinese PCIe x16 to 4 nvme passive bifurcation card for $30 USD because my motherboard support PCIe bifurcation

[sageofredondo]

8 minutes ago, TimedPing said:

I mentioned it in my signature under homelab AIO server, I'm using generic Chinese PCIe x16 to 4 nvme passive bifurcation card for $30 USD because my motherboard support PCIe bifurcation

Thank you, just to verify if I want to use a consumer motherboard. Is this a PLX board? https://www.amazon.com/dp/B084HMHGSP
 

If not, what would you recommend?

[TimedPing]

6 minutes ago, sageofredondo said:

Is this a PLX board? https://www.amazon.com/dp/B084HMHGSP

No, this one is the more expensive option of my $30 Chinese card

7 minutes ago, sageofredondo said:

Thank you, just to verify if I want to use a consumer motherboard.

What motherboard are you using? Can you dig your motherboard BIOS for PCIe splitting/Bifurcation options? It's usually under advanced settings/PCIe settings/PCH settings

[Needfuldoer]

If you're using a regular desktop platform (not a Xeon, Threadripper, or EPYC), I'd recommend running SATA SSDs so you don't use up your PCIe x16 slot. That's better spent on an HBA, a GPU for transcoding video, or high speed networking, and you won't notice a difference between NVMe and SATA SSDs over the network anyway.

[Nystemy]

9 hours ago, sageofredondo said:

 

 

I am thinking of a home all SSD server with one (or two) of those ASUS 4x nvme pcie cards with consumer ssds.

A few reasons for this:

* I have sensory issues and limited space. The only noise I want to hear is some Noctua fans.

* Limited space means an mATX (2x4 nvme) or an ITX (1x4 nvme) case and no need for a NAS card.
* I may want a developer's VM on the NAS box and we devs love speedy systems.
* 10 gig networking cards or maybe 25 gig fiber down the line.
* all ssd nas also means more free ram for containers or a vm and less power demand.
 

@Nystemy gave a good answer for this here:

I plan to use TrueNAS Scale (Linux version, I want Docker/Podman). I am aware of running data consistency checks (I watched LTT not setup a CentOS ZFS machine properly).

I wanted to know if anyone has done something like this or they have any build recommendations. AVX2 is the bare minimum I would consider for CPU compatibility. I have an i7-10700k with 64 gigs of non-ecc ram I am considering using as this NAS system and upgrading my gaming system to a Ryzen 7800X3D.

As others have stated, for a home server I would recommend SATA drives. They are more than fast enough for the server to bottleneck on your local network connections. Even 10 Gb/s networking is "slow" compared to a handful of SATA SSDs.

A better explanation of flash data retention can also be found here:

Write endurance and data retention time isn't purely about manufacturing quality vs the amount of data written.
Temperature plays a key role in the whole thing as well, both in regards to how quickly the drive forgets, but also how much it wears with each write.

In the end. Flash is a complex beast. Unlike magnetic storage that generally prefers a constant temperature of around 25-30 C at all times until it most often dies due to corrupted firmware or mechanical failure.

[sageofredondo]

On 6/20/2023 at 5:10 AM, TimedPing said:

No, this one is the more expensive option of my $30 Chinese card

What motherboard are you using? Can you dig your motherboard BIOS for PCIe splitting/Bifurcation options? It's usually under advanced settings/PCIe settings/PCH settings

Asus PRIME Z490M-PLUS Micro ATX LGA1200 Motherboard

Didn't have to dig. Asus lists it as handling 3 m.2s from a quick glance. https://www.asus.com/support/FAQ/1037507/

[sageofredondo]

On 6/20/2023 at 8:21 AM, Nystemy said:

As others have stated, for a home server I would recommend SATA drives. They are more than fast enough for the server to bottleneck on your local network connections. Even 10 Gb/s networking is "slow" compared to a handful of SATA SSDs.

Thank you for the info.

I guess the important part for me is if they would all fail at once which would take out any RAID. Which does not seem to be the case.

To me the last part is size. Having to get both a HBA card and find a place for 4-8 2.5 ssds. Not sure what kind of case I would get or even one that would be on the small size. That is why I prefer a m.2 solution as I could use a single x16 card for them (if i can find a reasonable one). Do you have any recommendations?

[sageofredondo]

On 6/20/2023 at 5:46 AM, Needfuldoer said:

If you're using a regular desktop platform (not a Xeon, Threadripper, or EPYC), I'd recommend running SATA SSDs so you don't use up your PCIe x16 slot. That's better spent on an HBA, a GPU for transcoding video, or high speed networking, and you won't notice a difference between NVMe and SATA SSDs over the network anyway.

 

The board has 5 SATA ports. I seen videos from Wendell on how the onboard SATA ports are less reliable for RAID. That is primary reason for using NVMe-more reliable. Is that your experience?

Also, the cpu is intel with a working igpu so quicksync for transcoding video.

 

[Nystemy]

6 hours ago, sageofredondo said:

Thank you for the info.

I guess the important part for me is if they would all fail at once which would take out any RAID. Which does not seem to be the case.

To me the last part is size. Having to get both a HBA card and find a place for 4-8 2.5 ssds. Not sure what kind of case I would get or even one that would be on the small size. That is why I prefer a m.2 solution as I could use a single x16 card for them (if i can find a reasonable one). Do you have any recommendations?

I myself use a 12x 3.5 inch bay 2U DIY rack case for my own home server. (it is far from small though...)

Then use an HBA providing me with 2x SFF8087 ports that provide 4 SAS/SATA connections each. However, the backplane in the rack case would need a third SFF8087 port on my HBA to be able to use all 12 bays, but as a start 8 bays is more than sufficient for me at the moment considering that I only use half of them... _{(did for fun try to use an SFF8087 to 4x SATA cable to see if I could just hook the last backplane to the motherboard's many unused SATA ports, but something in the SAS/SATA standard made this not work...)}

However, my home server is more than just a NAS, but also the home to my various VMs. _{(and to be fair, a larger case would have been nice so that I could add in a GPU or two for accelerating some 3d rendering projects.)}

[Needfuldoer]

38 minutes ago, Nystemy said:

However, my home server is more than just a NAS, but also the home to my various VMs. _{(and to be fair, a larger case would have been nice so that I could add in a GPU or two for accelerating some 3d rendering projects.)}

That's part of the reason I migrated from a PowerEdge R730XD to an X99 motherboard in a Rosewill 4U case without a SATA backplane. (Noise and power consumption were the others.) I have a dozen hard drives, so my options are limited.

 

A 3U, 16-bay SuperMicro case was also an option, but that would've been even more expensive.

[TimedPing]

10 hours ago, sageofredondo said:

Asus PRIME Z490M-PLUS Micro ATX LGA1200 Motherboard

Didn't have to dig. Asus lists it as handling 3 m.2s from a quick glance. https://www.asus.com/support/FAQ/1037507/

Well you jsut got back, I thought you're gone forever. It's not that part, I saying about PCIe lane splitting not the actual M.2 slot

[Electronics Wizardy]

11 hours ago, sageofredondo said:

 

The board has 5 SATA ports. I seen videos from Wendell on how the onboard SATA ports are less reliable for RAID. That is primary reason for using NVMe-more reliable. Is that your experience?

Also, the cpu is intel with a working igpu so quicksync for transcoding video.

 

I'd don't see any issues with the onboard sata. I'd use tht here. I'd say sata and nvme have about the same failure rate. And also if the sata controller fails, it typically will just stop and you can use the drives on a different system.

 

Failure rate on ssds is pretty low, so I wouldn't worry bout that here, but make sure you have backups.