ultimate home network Theoretically the best nas/networking setup without going overkil

[BDunkz]

Hey,

been a while since ive been on here so I’m probably out of the loop.

heres my theoretical plan; set up these systems with the best parts (not going overkill) and have a really clean production/files transfer system, 1 workstation/productivity pc (rackmount), 1 Mac (used by other family member), 1 gaming pc, servallence system, servalence encoding server, “valt nas”,

“temp nas”.

i know i haven’t really explained but what are your thoughts

all thanks in advance,

 

BDunkz

[Electronics Wizardy]

How much you wanna spend? 

 

Id personally go with a used r720 and give it a disk shelf or two. Lots of cpu power, lots of ram, run vms for everything, and you can daisy chain the sas disk enclosure to add hundreds of drives if you want.

[BDunkz]

1 minute ago, Electronics Wizardy said:

How much you wanna spend? 

 

Id personally go with a used r720 and give it a disk shelf or two. Lots of cpu power, lots of ram, run vms for everything, and you can daisy chain the sas disk enclosure to add hundreds of drives if you want.

Sorry as i said, havnt done pc stuff for a while, couple questions i forgot to mention: could i run the encoder from the “temp nas” and have it automacicaly transfer the the ‘valt’. I want the valt only doing file transfers not vm’s or anyhting to minimise expense there. Are 10 gbp/s cards still a thing?

[BDunkz]

7 minutes ago, Electronics Wizardy said:

How much you wanna spend? 

 

Id personally go with a used r720 and give it a disk shelf or two. Lots of cpu power, lots of ram, run vms for everything, and you can daisy chain the sas disk enclosure to add hundreds of drives if you want.

Also is ECC memory worth the price?

 

$1500 aud MAX for the server/nas side without drives

[Electronics Wizardy]

12 minutes ago, BDunkz said:

Sorry as i said, havnt done pc stuff for a while, couple questions i forgot to mention: could i run the encoder from the “temp nas” and have it automacicaly transfer the the ‘valt’. I want the valt only doing file transfers not vm’s or anyhting to minimise expense there. Are 10 gbp/s cards still a thing?

Do you want a seprate temp nas? Id just make multiple shares on the main one.

 

10gbe(and faster) are still alive and strong.

 

Id get this server https://www.ebay.com/itm/Dell-R720-8-Bay-SFF-Server-x2-2-20GHz-E5-2660-16-Cores-32GB-H310-SPS-4-Trays/192527810383?hash=item2cd38d3f4f:g:PqoAAOSwbEZa6NeO

 

Then these drive enclosures https://www.ebay.com/itm/Lenovo-ThinkServer-SA120-DAS-Array-70F10000UX-3-Year-Warranty/332337644546?hash=item4d60de0402:g:-5AAAOSwT5tWGx5V

 

then a hba like this https://www.ebay.com/itm/LSI-SAS9207-8E-Host-Bus-Adapter-H3-25427-02H/152906725627?epid=1608086694&hash=item2399f378fb:g:tJMAAOSwhfdadNG3

13 minutes ago, BDunkz said:

Also is ECC memory worth the price?

 

$1500 aud MAX for the server/nas side without drives

I don't do aud prices, so look for this stuff in your area.

 

For these servers ecc is cheaper, so might as well.

[BDunkz]

7 minutes ago, Electronics Wizardy said:

Do you want a seprate temp nas? Id just make multiple shares on the main one.

 

10gbe(and faster) are still alive and strong.

 

Id get this server https://www.ebay.com/itm/Dell-R720-8-Bay-SFF-Server-x2-2-20GHz-E5-2660-16-Cores-32GB-H310-SPS-4-Trays/192527810383?hash=item2cd38d3f4f:g:PqoAAOSwbEZa6NeO

 

Then these drive enclosures https://www.ebay.com/itm/Lenovo-ThinkServer-SA120-DAS-Array-70F10000UX-3-Year-Warranty/332337644546?hash=item4d60de0402:g:-5AAAOSwT5tWGx5V

 

then a hba like this https://www.ebay.com/itm/LSI-SAS9207-8E-Host-Bus-Adapter-H3-25427-02H/152906725627?epid=1608086694&hash=item2399f378fb:g:tJMAAOSwhfdadNG3

I don't do aud prices, so look for this stuff in your area.

 

For these servers ecc is cheaper, so might as well.

Wouldn’t it be better/ faster to have a temp nas to dump everything on to then transfer everything to the main nas say every day, instead of waiting ages for hdds to transfer everything?

 

thanks for all the links, is diy not worth it anymore for servers?

[Electronics Wizardy]

2 minutes ago, BDunkz said:

Wouldn’t it be better/ faster to have a temp nas to dump everything on to then transfer everything to the main nas say every day, instead of waiting ages for hdds to transfer everything?

 

thanks for all the links, is diy not worth it anymore for servers?

You can have a ssd cache, and a large hdd array will be very fast, a temp nas really won't help here.

 

This setup will probably hit 800mB/s easily.

[jde3]

3 minutes ago, BDunkz said:

Wouldn’t it be better/ faster to have a temp nas to dump everything on to then transfer everything to the main nas say every day, instead of waiting ages for hdds to transfer everything?

 

thanks for all the links, is diy not worth it anymore for servers?

That depends what you want to use them for.. If you want to work on the files from the NAS then you may want two. One for live copy and one for archive. (this is a great situation for ZFS send and receive btw) .. even your vault NAS will require offsite backup for anything you can never ever lose, because you can have a house fire or some such.

[BDunkz]

3 minutes ago, Electronics Wizardy said:

You can have a ssd cache, and a large hdd array will be very fast, a temp nas really won't help here.

 

This setup will probably hit 800mB/s easily.

So it would be better just to run everything from one server with different shares?

Edited May 8, 2018 by BDunkz

[Electronics Wizardy]

1 minute ago, BDunkz said:

So it would be better just to run everything from one server with different shares?

Thats what id do. 

 

What are you using the storage for? You can have a smaller ssd array for vms or something that needs high iops if you want.

[BDunkz]

1 minute ago, jde3 said:

That depends what you want to use them for.. If you want to work on the files from the NAS then you may want two. One for live copy and one for archive. (this is a great situation for ZFS send and receive btw) .. even your vault NAS will require offsite backup for anything you can never ever lose, because you can have a house fire or some such.

ZFS please explain 

[BDunkz]

3 minutes ago, Electronics Wizardy said:

Thats what id do. 

 

What are you using the storage for? You can have a smaller ssd array for vms or something that needs high iops if you want.

A lot of different things actually, family photos, system snapshots/backups, school and work files, videos ect..

 

vms perhaps a media server like plex, idk whats new and also a couple gaming servers thats why i was thinking of 2 systems,

[Electronics Wizardy]

Just now, BDunkz said:

ZFS please explain 

ZFS ia filesystem that is normally used for NAS and storage arrays. 

 

Its very good at data protection and have many advanced features like built in raid support, checksumming, snapshots, COW(copy on write, a better way of writing data to disk for many uses)

 

Id use ZFS here personally. Id add sa120 storage box, and setup a raid z2 or raid z3. Then if you need more storage, you can add another storage array later and easily expand the file system.

[Electronics Wizardy]

1 minute ago, BDunkz said:

A lot of different things actually, family photos, system snapshots/backups, school and work files, videos ect..

 

vms perhaps a media server like plex, idk whats new and also a couple gaming servers

So nothing that needs very high iops. 

Id just make a main hdd array, it will probably fill up a 10gbe connection with 12 disks, and a ssd or two on for the host os. 

 

Id personally run proxmox here. It has good support for ZFS and vms.

[BDunkz]

1 minute ago, Electronics Wizardy said:

ZFS ia filesystem that is normally used for NAS and storage arrays. 

 

Its very good at data protection and have many advanced features like built in raid support, checksumming, snapshots, COW(copy on write, a better way of writing data to disk for many uses)

 

Id use ZFS here personally. Id add sa120 storage box, and setup a raid z2 or raid z3. Then if you need more storage, you can add another storage array later and easily expand the file system.

So the actual sever hardware is separate from the drives? Different chassis?

[Electronics Wizardy]

Just now, BDunkz said:

So the actual sever hardware is separate from the drives? Different chassis?

You can do it either way. 

 

Either find a server with lots of drive bays or get a server and add an external drive bay to it.

[BDunkz]

1 minute ago, Electronics Wizardy said:

You can do it either way. 

 

Either find a server with lots of drive bays or get a server and add an external drive bay to it.

I know it depends on expandibility but how many U do you think the server/s would take up if i had say, 20 drives?

[Electronics Wizardy]

Just now, BDunkz said:

I know it depends on expandibility but how many U do you think the server/s would take up if i had say, 20 drives?

Are you willing to spend extra for a server with less U?

 

For the setup above, your thinking 6u. You can get a 4u supermicro case aswell, but I like the dell servers(normally more intergrated and often cheaper).

[jde3]

22 minutes ago, BDunkz said:

ZFS please explain 

ZFS is a file system + a volume manager in one package. It's a redesign of how file systems work or what is called a "modern advanced filesystem" It provides you pooled storage and separates those into volumes it called Datasets. So a group of drives is a pool, and a dataset is a separate grouping on those drives.. sorta like a mount point. or a folder but a folder that might have different options, different snaptshot schedule etc.
 

Spoiler

 

Sun did a talk about ZFS design and development here if you want to really know how it works and why.

 

 

 

ZFS has a feature called ZFS Send and Recieve.. and that allows you to send a snapshot of a dataset to another machine basically making a offsite clone of the dataset. It is very efficient because being the filesystem itself it already knows where the blocks are so unlike rsync it dosen't need to calculate checksums for the files and can only send what has changed.

 

So.. in your setup if you go the two NAS route.. System 1 will do snapshots.. every hour maybe.. and then rotate them into a nightly.. then you can send that snapshot to the archive system with ZFS send.

[BDunkz]

31 minutes ago, Electronics Wizardy said:

Are you willing to spend extra for a server with less U?

 

For the setup above, your thinking 6u. You can get a 4u supermicro case aswell, but I like the dell servers(normally more intergrated and often cheaper).

Idrc about how many U just wondering, space isnt really an issue

[BDunkz]

19 minutes ago, jde3 said:

ZFS is a file system + a volume manager in one package. It's a redesign of how file systems work or what is called a "modern advanced filesystem" It provides you pooled storage and separates those into volumes it called Datasets. So a group of drives is a pool, and a dataset is a separate grouping on those drives.. sorta like a mount point. or a folder but a folder that might have different options, different snaptshot schedule etc.
 

  Reveal hidden contents

 

Sun did a talk about ZFS design and development here if you want to really know how it works and why.

 

 

 

ZFS has a feature called ZFS Send and Recieve.. and that allows you to send a snapshot of a dataset to another machine basically making a offsite clone of the dataset. It is very efficient because being the filesystem itself it already knows where the blocks are so unlike rsync it dosen't need to calculate checksums for the files and can only send what has changed.

 

So.. in your setup if you go the two NAS route.. System 1 will do snapshots.. every hour maybe.. and then rotate them into a nightly.. then you can send that snapshot to the archive system with ZFS send.

Thanks, very good explanation 

[BDunkz]

@jde3 @Electronics Wizardy is freenas still teh way to go? Or is there some new, no brainer os?

[Electronics Wizardy]

1 hour ago, BDunkz said:

@jde3 @Electronics Wizardy is freenas still teh way to go? Or is there some new, no brainer os?

Id go proxmox. You get much better vm support and the same ZFS under the hood.

 

 

[jde3]

1 hour ago, BDunkz said:

@jde3 @Electronics Wizardy is freenas still teh way to go? Or is there some new, no brainer os?

That is somewhat of a flavor thing. For me, I'd just use FreeBSD base and manage it all from cron. (old school, I know.. I'm old school)

 

A lot of people like FreeNAS and it's basically a web front end UI for FreeBSD, I'm pretty sure that would have the most integration for zfs send and receive in it's UI. TrueOS (made by the same company) uses ZFS send and receive for it's life preserver software and that integrates with FreeNAS. There is also NAS4Free that is fairly sold no frills distro. I wasn't aware proxmox had zfs support. It'd be using ZFS on Linux (or ZoL) and ZoL is fine at this point.

 

Not sure how you can get more non-brainer than FreeNAS.. from my perspective at least. What your dealing with has a level of complexity that isn't easily hidden without gimping a lot of it's power.. you don't want to mask that through too much GUI gobbledygook.

[2bitmarksman]

Dell R510 with a Perc H200, flashed to IT mode with the LSI 9211-8i firmware and a 10g NIC for a FreeNAS box, and a Dell R610 (or 2) or Dell R710 (or 2) for your servers, also with 10g NIC's for a very cheap peer to peer storage and server solution. If you want full 10g networking that uses a switch, I'd suggest the Quanta LB6M. It's a relatively cheap 10g switch if all you want is 10g layer 2 switching. Depending on configurations, you can get all that for between 1200 to 1800 USD (1500 USD without the switch)