Software for NAS/Media server

[Donci]

What software stack do you suggest for this server?

Usage: -NAS: should also be accessable from outside the network, photo sharing within and outside of the network even from smartphones

-Media server: 1080p video streaming

Hardware: - SAS HDDs: starting with 2 (something like RAID 1) dynamically expandable up to 8 (something like RAID 6)

- 2 x SSD Caching (something like RAID 1) 

- 16GB ECC ram

- 8threads at around 3Ghz (e5 v4 so pretty new xeon)

 

I would preferably like a GUI and as little command line interfaces as possible.

 

[Skiiwee29]

FREEnas sounds like it would work. 

[Donci]

Just now, legacy99 said:

FREEnas sounds like it would work. 

The thing is thats just the OS: 

-What file system? (HwRAID doesnt seem that compelling)

-How do I connect from outside the network?

-Caching?

-Media streaming and photo sharing platform?

[Skiiwee29]

12 minutes ago, Donci said:

The thing is thats just the OS: 

-What file system? (HwRAID doesnt seem that compelling)

-How do I connect from outside the network?

-Caching?

-Media streaming and photo sharing platform?

it supports full raid arrays. I have not used it personally, but it has been recommended to me for when I build a home NAS. 

[Oshino Shinobu]

10 minutes ago, Donci said:

The thing is thats just the OS: 

-What file system? (HwRAID doesnt seem that compelling)

-How do I connect from outside the network?

-Caching?

-Media streaming and photo sharing platform?

1) ZFS file system is really your only choice with FreeNAS. While it does support UFS, there's really no point in using it. For pretty much all Linux (especially FreeBSD), software RAID is the better option to use. As for migrating from RAID 1 to RAID 5/6, I'm not sure how well that works or if it's possible in the first place. You can expand an existing RAID 5 or 6 array though. 

 

2) You have a few options. There are some plugins that allow for things like OwnCloud to be set up if you have a membership. If not, you'd probably want to setup SFTP (not FTPS). There are guides around for how to do this, and pretty much everything else. One of the best parts of FreeNAS is the community support, development and guides that are around. The process for SFTP can be quite involved, depending on the approach you take. 

 

3) Caching is supported. Again there are guides around for this. It's probably in the official documentation. 

 

4) Probably the best solution for this is the Plex plugin, especially for Video streaming. Setting up Plex to allow access from outside the network is fairly easy too. 

[Donci]

25 minutes ago, Oshino Shinobu said:

1) ZFS file system is really your only choice with FreeNAS. While it does support UFS, there's really no point in using it. For pretty much all Linux (especially FreeBSD), software RAID is the better option to use. As for migrating from RAID 1 to RAID 5/6, I'm not sure how well that works or if it's possible in the first place. You can expand an existing RAID 5 or 6 array though. 

 

2) You have a few options. There are some plugins that allow for things like OwnCloud to be set up if you have a membership. If not, you'd probably want to setup SFTP (not FTPS). There are guides around for how to do this, and pretty much everything else. One of the best parts of FreeNAS is the community support, development and guides that are around. The process for SFTP can be quite involved, depending on the approach you take. 

 

3) Caching is supported. Again there are guides around for this. It's probably in the official documentation. 

 

4) Probably the best solution for this is the Plex plugin, especially for Video streaming. Setting up Plex to allow access from outside the network is fairly easy too. 

Thanks! 

For 1) : I have heard BTRFS can do switch between RAIDs

[Pixel5]

the hardware is already given or will be bought for this build?

 

because for the mentioned applications you wanna use this for the hardware is way overkill and you could do this literally with a raspberrypi beside the limitations of the LAN port on it.

 

you need almost no CPU power and you could easily get away with less then 1GB of ram for what you want to do with it.

[Oshino Shinobu]

2 minutes ago, Donci said:

Thanks! 

For 1) : I have heard BTRFS can do switch between RAIDs

I know next to nothing about BTRFS, so I can't really comment on that. I do know that FreeNAS doesn't support BTRFS though, so you'd need to look into another OS if it's an important feature to you. 

 

As you're going to have two drives initially, you could consider spending a little more to start with 3 drives for RAID 5 operation so it can be expanded later on. In general, it's not the best situation (ideally, you'd want to start with more drives in RAID 5), but it does mean you won't have to offload all of the data when it comes to adding more drives and starting a RAID 5 array. After a little more research, it looks like to transition between RAIDZ (RAID 5) and RAIDZ2 (RAID 6), you need to recreate the storage pool from scratch, so you'd need to offload all data first, create the pool, then move it back. 

 

Either way, I would strongly recommend a backup for all of the data on any array (unless the data isn't important, but then you wouldn't be going for so much redundancy anyway). RAID 5 still has the big data vulnerabilities when it's rebuilding. 

[Donci]

55 minutes ago, Oshino Shinobu said:

I know next to nothing about BTRFS, so I can't really comment on that. I do know that FreeNAS doesn't support BTRFS though, so you'd need to look into another OS if it's an important feature to you. 

 

As you're going to have two drives initially, you could consider spending a little more to start with 3 drives for RAID 5 operation so it can be expanded later on. In general, it's not the best situation (ideally, you'd want to start with more drives in RAID 5), but it does mean you won't have to offload all of the data when it comes to adding more drives and starting a RAID 5 array. After a little more research, it looks like to transition between RAIDZ (RAID 5) and RAIDZ2 (RAID 6), you need to recreate the storage pool from scratch, so you'd need to offload all data first, create the pool, then move it back. 

 

Either way, I would strongly recommend a backup for all of the data on any array (unless the data isn't important, but then you wouldn't be going for so much redundancy anyway). RAID 5 still has the big data vulnerabilities when it's rebuilding. 

There will definitely be a backup, although it will probably be backblaze, so getting the data back would be to say the least complicated. I think BTRFS is really the only option for me because I definitely dont have 8TB of free storage. And I thing RAID 5 ( atleast in btrfs) can be done with just 2 drives, although it would be somewhat pointless because BTRFS can just change between raids.

[Donci]

1 hour ago, Pixel5 said:

the hardware is already given or will be bought for this build?

 

because for the mentioned applications you wanna use this for the hardware is way overkill and you could do this literally with a raspberrypi beside the limitations of the LAN port on it.

 

you need almost no CPU power and you could easily get away with less then 1GB of ram for what you want to do with it.

Yes, the problem is, that I know of no modern xeon (DDR4 ECC Reg supported) that is lower powered than the e5-1620v4. I mean the e5-2603v4 is like 100$ less expensive so that could be a thing, but it runs at like 1.7Ghz. You are right about the ram tho, but I dont know how you would connect a SAS Jbod controller to a pi  

[Oshino Shinobu]

12 minutes ago, Donci said:

There will definitely be a backup, although it will probably be backblaze, so getting the data back would be to say the least complicated. I think BTRFS is really the only option for me because I definitely dont have 8TB of free storage. And I thing RAID 5 ( atleast in btrfs) can be done with just 2 drives, although it would be somewhat pointless because BTRFS can just change between raids.

Due to how RAID 5 works, it cannot run on 2 drives. RAID 5 on two drives would basically be a weird combo of RAID 1 and RAID 0 (but not RAID 10). 

 

 

[Bakerking31]

9 hours ago, Donci said:

 

Hardware: - SAS HDDs: starting with 2 (something like RAID 1) dynamically expandable up to 8 (something like RAID 6)

 

 

This is not possible in FreeNas.  You can add more pools but you cannot expand the current pools by adding drives.

 

UnRaid does allow you to add drives to dynamically expand storage and you set how many parity drives you want.  If you have 4 drives and 1 is parity and you want to add a 5th to be a parity drive, you can designate it as parity.  The downside is that its not a true raid so the performance is a bit slower.  But with SSD caches there isn't any real performance difference.  UnRaid will also do all of your other requirements.

[itsnippets]

I've run OSX + Plex.

Debian + Plex (setup as command line only).

and Windows + Plex.

 

But FreeNAS is the main choice, it has a plex plugin.  As you can see there is one common thread regardless of the core OS I use! 

[Donci]

8 hours ago, Bakerking31 said:

This is not possible in FreeNas.  You can add more pools but you cannot expand the current pools by adding drives.

 

UnRaid does allow you to add drives to dynamically expand storage and you set how many parity drives you want.  If you have 4 drives and 1 is parity and you want to add a 5th to be a parity drive, you can designate it as parity.  The downside is that its not a true raid so the performance is a bit slower.  But with SSD caches there isn't any real performance difference.  UnRaid will also do all of your other requirements.

If unraids built in RAID allows for that thats good, but does it do checksums to protect against bitrot? If it does i dont even need btrfs

[Bakerking31]

8 hours ago, Donci said:

If unraids built in RAID allows for that thats good, but does it do checksums to protect against bitrot? If it does i dont even need btrfs

It does not.

 

Each OS has plusses and minuses so its just what is most important to you.  Everything on my media server that is important to me (the photos) is already backed up in multiple locations.  If I lose data due to bitrot its going to be TV shows and movies which are easily replaceable. 

[Lurick]

Plex is great for sharing media locally and outside the network.

 

To access outside the network you'll need to port forward but that's pretty easy, depending on the router you've got.