LTT: 320 terabytes in a normal PC case!! The DIY 4k editing NAS

[NickHeavy]

 

Fractal's new Define 7 XL has an enormous capacity for hard drives, so it only makes sense to build and deploy a 4K editing NAS to compliment our recently built 4K editing workstation.

For about $1000(not including the drives) you can build up this system and add up to 20 drives, like Segate's Iron Wolf Pro line. Using Unraid or ProxMox you'll be able to manage your data effectively.

Buy Define 7XL

On Amazon (PAID LINK): https://geni.us/ZW6CMlG

On Newegg (PAID LINK): https://geni.us/OlDwOs

Buy Ryzen 2600

On Amazon (PAID LINK): https://geni.us/u0va6y1

On Newegg (PAID LINK): https://geni.us/K9Q2zfV

Buy Corsair Vengeance LPX 16GB

On Amazon (PAID LINK): https://geni.us/c4TbFdq

On Newegg (PAID LINK): https://geni.us/GjeD

Buy Asus ROG STRIX B450-F

On Amazon (PAID LINK): https://geni.us/cNGeUBX

On Newegg (PAID LINK): https://geni.us/8ptR

Buy Corsair HX750

On Amazon (PAID LINK): https://geni.us/vXmUAO

On Newegg (PAID LINK): https://geni.us/BY6404

Buy Adata SU655

On Amazon (PAID LINK): https://geni.us/nBJHqE

On Newegg (PAID LINK): https://geni.us/qtXBCMa

Buy Intel X540-T1 10G NIC

On Amazon (PAID LINK): https://geni.us/n9AG

On Newegg (PAID LINK): https://geni.us/Gd03

Buy Lsi Logic 9201-16i 16port 6gb/S

On Ebay: https://lmg.gg/KlY3B

On Amazon (PAID LINK): https://geni.us/VnBQ

On Newegg (PAID LINK): https://geni.us/B0VTIE

Buy SFF-8087 to 4x SATA

On Amazon (PAID LINK): https://geni.us/dzQC

On Newegg (PAID LINK): https://geni.us/SDKW

Buy Seagate IronWolf Pro 16TB

On Amazon (PAID LINK): https://geni.us/9fQM

On Newegg (PAID LINK): https://geni.us/QdRuZk

[kelvinhall05]

Yessss, we need more cases like this!

[seon123]

Nice for storing all of your homework

[Lurick]

Now I can finally install Call of Duty Super HUGE edition

[nick name]

17 minutes ago, seon123 said:

Nice for storing all of your homework

I keep other people's homework.  

[mariushm]

a bit off topic : no idea why the geni.us links point me to amazon.it ... i'd rather have amazon.co.uk  or .de  ... almost never use the .it version

 

going more to the topic ... Fractal Design Define 7 XL Nero is 250 dollars... no thanks

Lots of cases can handle 20 drives, especially if converting 5.25" bays to 3.5" ... and you can have drives mounted o nthe back of the case and above the psu shroud...

 

It may be time to go 2.5" ... there's right now a big sale on 1U and 2U servers ... Quanta for days... 300+ v1-v4 1U/2U servers for sale! | Web Hosting Talk

 

Quote

2U
50x D51B-2U (v3/v4) w/ 9271-8I, 2x 10G on mezz, Dual Heatsink, 24x SFF Trays, Dual PSU, Rails - $250ea
50x S210-X22RQ (v1/v2) w/ 9271-8I, 2x 10G on mezz, Dual Heatsinks, 24x SFF Trays, Dual PSU, Rails - $150ea

Storage
150x SD1Q-1ULH 12x3.5” SATA/SAS + 6x2.5” (SSD only) + 1x PCIe 2280 M.2 w/ D-1541 CPU Onboard, 2x 10G on mezz, Trays, Dual PSU, Rails - $450ea

 

[Windows7ge]

I miss the LIAN LI PC-Q26

[Godvater]

I am glad to see this video/project. I have a very similar DIY Unraid NAS: Fractal Design Define R5, R5 3600, 32GB DDR4, 3x8TB(will increase with discounts)

 

One thing you guys completely didn't mention in the video is accessing the server from other machines. I seem to have a lot of problems with this and even after trying most things mentioned in unraid forums, my connection to the NAS is not reliable as I keep getting "permission errors". How do you setup your Samba shares both on your windows and on Unraid machines that it works reliably enough to edit video off of it? @NickHeavy

[Embeh]

Hello, someone gave me some older-ish hardware for basically free. It includes this Controller from supermicro (website link). It says raid card, which Linus says not to get. But it has an HBA mode. Can I use that fine and which raid do you recommend for just NAS - like file storage?

 

Thanks!

 

Martin

[Optimoos]

Seems like the perfect time to drop this info - I've been doing a bunch of evaluation on home storage as I prepared to rebuild my home NAS. This included going way overboard and doing a bunch of diskspd testing against FreeNAS, Unraid, and Windows. I was going to do a long write-up on the whole process but when the results started coming back figured my nerd cred would get obliterated so I dropped it. This video is begging for the details to be shared though.

 

Essentially I wanted to test what an "amateur hobbyist" could expect to get performance wise out of these three platforms. This meant mostly next-next-nexting through install and configuration, finding a document that described the setup I was going for, following it, and testing the results. The goal was to use the SSDs to provide enhanced performance in whatever way made sense to the platform. For FreeNAS, this meant a pair of them acting as cache and a pair acting as journal drives; in Unraid they were assigned as cache disks; in Windows they were configured as tiered storage in a storage space (mirrored SSD tier, parity HDD tier). Tests were run multiple times with various options to diskspd, these options are all recorded in the results spreadsheet. In all cases I was testing against a CIFS/SMB share. Here's the hardware details:

 

Server system:

AMD 3400G on ASUS Prime X570 Pro

32GB DDR4-3200

WDC WDS250G1B0C (250g boot drive)

4 x Samsung 860 EVO 500GB SSD (using motherboard SATA)

8 x WDC WD2003FYYS 2TB HDD (using LSI 9305-16i HBA)

Intel X520 10GB NIC

 

Testing system:

i7-4790K @ 4GHz on Z97X-UD3H

16GB DDR3-1600

Windows 10 Pro

Intel X520 10GB NIC

 

Network switch: Cisco 3750X w/C3KX-NM-10G

 

Without further ado, here are the results: https://docs.google.com/spreadsheets/d/17S0K3_VM2vIDYFKhjz4QJS18S_VaMHRjhbQ75CBTaHI/edit?usp=sharing

 

Multi and Single denote whether diskspd was using a single target file for read/writes or multiple (4) files. If people are interested I can share the powershell script I wrote for the test but it's basically just a bunch of loops iterating over the config options as listed in the spreadsheet. As for the results... Yes, that's right, at the very least for my testing setup, Storage Spaces kind of wiped the floor with everything else.

[_bolek_]

@Embeh You can basically go for any LSI/3wave/Avage/Broadcom SATA/SAS controller or raid controller with IT mode ora IT Mode compatible.

There many card like IBM Megaraid, DELL Perc, Supermicro or other but like i mention first check if card have LSI/Broadcom chip.

Here you can find all old chip and card that support HBA mode that can be bayed on ebay and can be crossflash to support newest LSI firmware

https://forums.servethehome.com/index.php?threads/lsi-raid-controller-and-hba-complete-listing-plus-oem-models.599/

 

But i don't like this episode: 

- They mentioned costs but go unRIDE.

- They mentioned stability and still go unRIDE

- They mentioned VM and still go for unRIDE

 

Why you didn't finish episode on FreenNAS from last year? and in this episode didn't show FreeNAS (even when you mention it).

I think It's better then unRIDE and its FREE.

 

O and i forget mention that your suggestion to configure zvol and vdev is totally wrong and misleading.

At last FreeNAS creator cover for this:

https://www.ixsystems.com/blog/zfs-pool-performance-1/

https://www.ixsystems.com/blog/zfs-pool-performance-2/

 

Yes i know what im saying. I build many NAS home and for corp (trying many software and ended using FreeNAS for all of them).

Next is that no matter with platform you use (Linux of *BSD), because of limited participation form AMD in creating drivers for Linux/BSD performance suck.

 

PS: for NAS you can use old hardware like Xeon X series with support ECC and it cheep  like DELL T620 with 12 x 3,5 hotplug + Perc H310

[Luscious]

No way are my storage needs at this level, but I have been having a pretty good time using StableBit DrivePool on a set of four 8TB SMR drives. In the last 5 years or so I have yet to have any issues with the software and have not lost any data either.

 

My use case is purely as a media server to store and stream MKV and VOB content. It does that job well and I'm happy!

 

RAID cards can run up several grand, but the bigger danger is the RAID hardware failing rendering your perfectly intact data on your drives useless. That's not the case with software solutions, as a simple reboot can get the thing up and running again.

 

$1000 isn't bad for something that can accommodate 16 drives, although the need to fix it yourself if something goes south doesn't inspire confidence. Also, having a giant tower case isn't the best use of space or cooling - I'd be looking into rackmount for anything over 8-12 drives.

 

Thinkmate has some very neat NAS solutions that can go up to 36 drives. Sure, they're pricey, but you get a 5 year NBD on site warranty, tested and compatible hardware, work stand-alone (unlike a JBOD enclosure that requires a separate server) and also can be configured with real time volume mirror and failover clustering when using two or more of them. They even have a 24 drive U.2 variant. If you're mad enough to rock a dual Xeon or Epyc workstation in your home, or have a handful in a small production office, investing in a NAS like this makes perfect sense.

[Egg-Roll]

I don't think this is necessarily a good deal for what you could get. If all you are doing is file hosting even with live read/writes during editing esp photos and nothing else then a cheap old server rack board like the one I have for $100 all included is likely a far better option. The case is nice and is well within a acceptable range for the drive bays considering I got a 15 bay 4U for $140 (CAD). 

 

5 hours ago, mariushm said:

It may be time to go 2.5" ... there's right now a big sale on 1U and 2U servers ...

I hope that includes hardware else it is a complete rip off possibly, Not to mention shipping a whole U rack of any size would be murder upon itself outside of the USA...

 

22 minutes ago, Luscious said:

If you're mad enough to rock a dual Xeon or Epyc workstation in your home, or have a handful in a small production office, investing in a NAS like this makes perfect sense.

Never crossed your mined that they couldn't justify buying one because it's cheaper to just buy the said equipment for every user you just called them mad for owning because they likely needed to anyways, now did it? Server racks like that should be left for companies that can push out at least the servers initial costs in additional income a month by using said server(in other words the server would bring in it's costs in additional sales every month, not profit but sales), which aren't the people you are calling mad. The $6000 additional costs just to get to 24 cores on the 24 bay server variant would easily buy 1 potentially 2 Epyc computers, which brings it down to what's more useful a worker or 2 that have the power right now as needed w/o waiting or bottlenecks and a server that you could do the way Linus did for far less or your choice? If I was one of those "mad" people I would chose Linus's route, because it's cheaper.

[ichihaifu]

This video made me realize something fairly obvious, but critical about Unraid: You will be limited to a single drive performance bottleneck when accessing data that is offloaded from cache, given the way it accomplishes storage pools.

I suppose that is fair when you don't care about performance, or if your use scenario is one where you almost never interact with "old data".

 

EDIT: I don't remember SSD cache being installed on this system, so no wonder the performance was choppy (and given very little screen time). In this use case it would've been always choppy unless there was striped raid SSD cache in front of spinners that accommodated the entire active data set.

[skitzor]

Recently built something similar for a media server, using the Fractal Arc XL and taking advantage of a hard drive cage of my other Fractal case. Have 10x3.5", 2x2.5", and still room for 2x3.5", with all 4x5.25" left (which is another 6x3.5" at least). Mainly a plex server so it has a basic i3-8100, which is fairly competent at transcoding thanks to the hd630 graphics.

 

One question I had about this build was the inclusion of ECC RAM. Is it really necessary? What happens if it isn't used and there are memory errors, and how likely are they?

[Spotty]

13 hours ago, mariushm said:

a bit off topic : no idea why the geni.us links point me to amazon.it ... i'd rather have amazon.co.uk  or .de  ... almost never use the .it version

It's supposed to redirect to the most local version of Amazon depending on your location. When I click the link it takes me to the .au Australian version of Amazon. As far as I know Amazon does not have a presence in Romania so it must be redirecting to the nearest match, which for some reason seems to be Amazon Italy for you. I'm not sure if there is any way to change where it redirects to. If you change the URL to .co.uk or .de it should work, assuming they stock the items.

[GodAtum]

15 hours ago, kelvinhall05 said:

Yessss, we need more cases like this!

No this is just idiotic. No hotswap or drive status light, so if 1 fails you have no idea which one. Do it properly and get a $50 4U server case.

[GodAtum]

9 hours ago, Luscious said:

No way are my storage needs at this level, but I have been having a pretty good time using StableBit DrivePool on a set of four 8TB SMR drives. In the last 5 years or so I have yet to have any issues with the software and have not lost any data either.

 

My use case is purely as a media server to store and stream MKV and VOB content. It does that job well and I'm happy!

 

RAID cards can run up several grand, but the bigger danger is the RAID hardware failing rendering your perfectly intact data on your drives useless. That's not the case with software solutions, as a simple reboot can get the thing up and running again.

 

$1000 isn't bad for something that can accommodate 16 drives, although the need to fix it yourself if something goes south doesn't inspire confidence. Also, having a giant tower case isn't the best use of space or cooling - I'd be looking into rackmount for anything over 8-12 drives.

 

Thinkmate has some very neat NAS solutions that can go up to 36 drives. Sure, they're pricey, but you get a 5 year NBD on site warranty, tested and compatible hardware, work stand-alone (unlike a JBOD enclosure that requires a separate server) and also can be configured with real time volume mirror and failover clustering when using two or more of them. They even have a 24 drive U.2 variant. If you're mad enough to rock a dual Xeon or Epyc workstation in your home, or have a handful in a small production office, investing in a NAS like this makes perfect sense.

I would never recommend software RAID for professional use. Using an HBA card with Freenas or Xponology.

[gopala]

hi help me i saw youtube i need more sata storage due motherboard sata port not enough

 

you see screenshot both card is different model ???

 

 

Lsi Logic 9201-16i  and small size card different model i think

 

so what model you using help

 

[mariushm]

The bottom card is LSI Logic 9201-16i ... it has 4 connectors in which you can insert a cable that splits into 4 sata connectors, so you get 4x4 = 16 sata connectors.

The other card ... is just some random 4 port sas/sata connector, there's lots of them and there's nothing interesting about them.

 

You can buy cards on eBay from around 15$ ... search for "hba card"

Here's some examples

 

10$ (but SATA2  aka max 300MB/s per connector) : https://www.ebay.com/itm/Gigabyte-GC-RLE086-RH-LSI-1068E-HBA-card-8-port-SAS-SATA-PCI-E-array-card-3081e/192653365174?

 

19$ +7$ shipping 8 port HBA card : https://www.ebay.com/itm/LSI-SAS-9210-8i-8-port-6Gb-s-PCIe-HBA-RAID-SATA-Controller-card-M1015-9211-8I/171972993154

 

40$ (us seller) : https://www.ebay.com/itm/LSI-9210-8i-6Gbps-SAS-SATA-8-Ports-HBA-PCI-E-RAID-Controller-Card-US-seller/132628492132

 

 

Cables are cheap, 5$ each : https://www.ebay.com/itm/Mini-SAS-SFF-8087-36Pin-to-4-SATA-7-PIN-HD-Splitter-Breakout-Cable-50cm-Blue/192035934122

 

 

 

[gopala]

1 hour ago, mariushm said:

The bottom card is LSI Logic 9201-16i ... it has 4 connectors in which you can insert a cable that splits into 4 sata connectors, so you get 4x4 = 16 sata connectors.

The other card ... is just some random 4 port sas/sata connector, there's lots of them and there's nothing interesting about them.

 

You can buy cards on eBay from around 15$ ... search for "hba card"

Here's some examples

 

10$ (but SATA2  aka max 300MB/s per connector) : https://www.ebay.com/itm/Gigabyte-GC-RLE086-RH-LSI-1068E-HBA-card-8-port-SAS-SATA-PCI-E-array-card-3081e/192653365174?

 

19$ +7$ shipping 8 port HBA card : https://www.ebay.com/itm/LSI-SAS-9210-8i-8-port-6Gb-s-PCIe-HBA-RAID-SATA-Controller-card-M1015-9211-8I/171972993154

 

40$ (us seller) : https://www.ebay.com/itm/LSI-9210-8i-6Gbps-SAS-SATA-8-Ports-HBA-PCI-E-RAID-Controller-Card-US-seller/132628492132

 

 

Cables are cheap, 5$ each : https://www.ebay.com/itm/Mini-SAS-SFF-8087-36Pin-to-4-SATA-7-PIN-HD-Splitter-Breakout-Cable-50cm-Blue/192035934122

 

look like cheap but don't want bad quality

 

i need good quality like youtube he using i see he run 24h7d never had problem that i think he use high quality suitable long usage running

 

[snoopy82481]

6 hours ago, GodAtum said:

No this is just idiotic. No hotswap or drive status light, so if 1 fails you have no idea which one. Do it properly and get a $50 4U server case.

Which server case are you talking about?  I have been looking for one and have not found any for that price.  I have a Dell 7010 I want to transplant for a file/media server.

[mariushm]

11 minutes ago, gopala said:

look like cheap but don't want bad quality

 

i need hood quality like he using youtube i see he run 24h7d never had problem that i think he use high quality suitable long usage running

 

The cards above are controllers used in dedicated servers, all over the world, they were designed to run 24/7 ...

They're spare parts for servers, new old stock etc

The Gigabyte cards were bundled with Opteron (socket G34) servers and motherboards, the servers were decomissioned because everyone upgraded to more power efficient servers and now they're sold cheap because the controller chip can only do sata2 (3 gbps or ~250 MB/s ) not sata3 (6gbps or ~550MB/s ) ... so that's why those cards are 10$, because most people are willing to pay double (20-30$) for a sata 3 (6gbps) card.

 

The LSI card I linked to is plain boring LSI controller, very common, well known, datacenter card ... good for 24/7 use and all that.

[RoseLuck462]

$210 for such a no-frills case? I don't know if I would do that...

[Thanatopsis]

On 3/7/2020 at 4:13 PM, Godvater said:

I am glad to see this video/project. I have a very similar DIY Unraid NAS: Fractal Design Define R5, R5 3600, 32GB DDR4, 3x8TB(will increase with discounts)

 

One thing you guys completely didn't mention in the video is accessing the server from other machines. I seem to have a lot of problems with this and even after trying most things mentioned in unraid forums, my connection to the NAS is not reliable as I keep getting "permission errors". How do you setup your Samba shares both on your windows and on Unraid machines that it works reliably enough to edit video off of it? @NickHeavy

I would have to look at your setup to be sure, but by default windows will try to use the user account of the person loged on to your pc and its login info as the SAMBA info.  This can create issues if your permissions on your NAS do not use the same user name.  The simple solution is to make sure you are connecting to the NAS as Guest or a generic user that does have permissions on the NAS and you will need to use the NAS's password for that generic account.