nas Home NAS Help

[ChalkChalkson]

1 minute ago, SCHISCHKA said:

I have tested single drive, two drive, and three drive raid0. single drive is ok but it slows down with small files. two drives is better, it gives you better random access and comes close to saturating gigabit. three drives really maxes out the network and does not have much more to offer over two drives. RAID0 also speeds up backups.

 

That is not a proper backup/data-loss plan. two drives on snapshot rotation is better protection than raid.

 

if you had a proper backup solution you should be protected from:

1 lightning

2 power surge

3 physical theft

4 ransomware

5 accidental deletion

6 fire

7 drive failure

8 software bugs

 

RAID protects you from:

1 drive failure.

 

8:1 reasons to not waste your money implementing enterprise technology into the home.

 

Yes, a proper backup plan ( I am a second server at my parents place) is better, but parity is A: a good start and B: always timely, depending on you plan a backup is often about a week behind, and takes hours to restore from... 

The random performance in RAID 0 is a plus tough, but I don't think there are many situations where this matters a lot.

 

Despite, as I said a copy on write parity like unraid is way easier to expand and unless you have a very specific usecase you won't miss the performance of a RAID. 

And parity really goes a long way, but I guess that discussion could be its own thread... I might open that later today  

[SCHISCHKA]

3 minutes ago, Jarsky said:

Windows Server doesnt have to be expensive and theres of course Windows 10 - both with Storage Spaces and ReFS which are excellent. 

FYI: purchasing keys off reddit is piracy. I dont care if you buy off there I'm just OCD about misleading people when it comes to money. Everything I have seen on reddit is not a proper license. It is all stuff that breaks a shit ton of MS rules about how you can purchase a license.

4 minutes ago, Jarsky said:

Linux Mint isn't a bad idea for a NAS, its still a linux 2.6 kernel

what do you mean its 2.6 kernel?

6 minutes ago, Jarsky said:

Black drives are quite expensive in my country (theyre about 40% more than Red drives) - I see theyre about 30% more in the US as well - and being high performance 7200rpm drives they do suck a lot more power than the Red, Green or Purple drives which would be completely suitable for home use. My WD Red RAID5 saturates 10Gbit

they cost more coz they last longer. They have more spare space for bad sectors. power difference isnt much for just one or two drives. there are two different red drives. one 5400rpm with 3 year warranty. red pro is between 5400 and 7200rpm and has five year warranty. red pro is up there in price with black.

 

 

[Jarsky]

2 minutes ago, SCHISCHKA said:

what do you mean its 2.6 kernel?

Meant to say 2.6+ which is when mdadm and ext4 were implemented. any kernel from there upwards will support md features. 

[SCHISCHKA]

4 minutes ago, ChalkChalkson said:

Yes, a proper backup plan ( I am a second server at my parents place) is better, but parity is A: a good start and B: always timely, depending on you plan a backup is often about a week behind, and takes hours to restore from... 

The random performance in RAID 0 is a plus tough, but I don't think there are many situations where this matters a lot.

 

Despite, as I said a copy on write parity like unraid is way easier to expand and unless you have a very specific usecase you won't miss the performance of a RAID. 

And parity really goes a long way, but I guess that discussion could be its own thread... I might open that later today  

I have 5 minute backup built into my computer for work with snapshots in case i need to rollback. This really does require RAID0. Anything else would be too slow. Turning it into RAID5 or 10 or 6 would be a waste of money, and this is my argument; those extra drives are better off in an external USB case.

I have a cheap computer as a fileserver that has a mix of arrays both RAID1 and 0 made up of old drives. The RAID0 makes a huge difference. When you do rsync on RAID1 vs RAID0 you can notice the difference.

[SCHISCHKA]

3 minutes ago, Jarsky said:

Meant to say 2.6+ which is when mdadm and ext4 were implemented. any kernel from there upwards will support md features. 

ok that is super old like an ISP modem. BTRFS i think requires 4.4 and i believe they sorted out BTRFS raid5 in kernel 4.9 or 4.10

[ChalkChalkson]

1 minute ago, SCHISCHKA said:

I have 5 minute backup built into my computer for work with snapshots in case i need to rollback. This really does require RAID0. Anything else would be too slow. Turning it into RAID5 or 10 or 6 would be a waste of money, and this is my argument; those extra drives are better off in an external USB case.

I have a cheap computer as a fileserver that has a mix of arrays both RAID1 and 0 made up of old drives. The RAID0 makes a huge difference. When you do rsync on RAID1 vs RAID0 you can notice the difference.

when I don't have a ton of data to sync the longest time of the rsyncs is spent by drawcalls for the filelist in the console   

[Jarsky]

2 minutes ago, SCHISCHKA said:

ok that is super old like an ISP modem. BTRFS i think requires 4.4 and i believe they sorted out BTRFS raid5 in kernel 4.9 or 4.10

I still wouldn't trust BTRFS RAID with my data yet (on my production server) - BTRFS with MD is a more tried and tested solution currently. 

[SCHISCHKA]

6 minutes ago, ChalkChalkson said:

when I don't have a ton of data to sync the longest time of the rsyncs is spent by drawcalls for the filelist in the console   

when doing a snapshot reading the drive takes the longest time. I get around this by having different settings for different folders. games dont need to be backed up every day. Only my current project needs to be backed up at a high frequency. My multimedia gets backed up manually.

9 minutes ago, Jarsky said:

I still wouldn't trust BTRFS RAID with my data yet (on my production server) - BTRFS with MD is a more tried and tested solution currently. 

Im using it for my /home. no problems so far. The reason I did this is because I can add a drive and extend it live. I use mdadm on my fileserver coz it emails me when a drive fails. I have not read the man page for BTRFS so i dont know if it has that feature.

[ChalkChalkson]

2 minutes ago, SCHISCHKA said:

when doing a snapshot reading the drive takes the longest time. I get around this by having different settings for different folders. games dont need to be backed up every day. Only my current project needs to be backed up at a high frequency. My multimedia gets backed up manually.

I don't do snapshots, I literally just rsync the files over, for work stuff and documents I create a new folder each time, giving me timed fall backs, multimedia is just rsynced without anything fancy, meaning I only copy new files over, making the total size of a sync less than a gig, in most cases less than 10 meg, for that you don't need RAID 0  

[SCHISCHKA]

35 minutes ago, ChalkChalkson said:

I don't do snapshots, I literally just rsync the files over, for work stuff and documents I create a new folder each time, giving me timed fall backs, multimedia is just rsynced without anything fancy, meaning I only copy new files over, making the total size of a sync less than a gig, in most cases less than 10 meg, for that you don't need RAID 0  

rsync can do snapshots. if youre using linux have a look at backintime. its a nice gui for rsync.

Git is quite good for documents you dont have to mess around with folders. I dont know what work you do, im a programmer so its compulsory for me to use git.

There is a multimedia component to what I do so if i did what you do i would probably run out of space.

[BDunkz]

20 hours ago, SCHISCHKA said:

then you do not need redundant drives or special NAS drives. Use this saving for backup drives.

Redundant drives in a home NAS do nothing except cost you money. Redundant drives are intended to keep your server running while you rebuild your array after a failed drive. For non 24/7 home use you can expect a drive to fail in approx 10 years with good quality drives.

But I'm storing family photos and important documents so dont I need redundancy?

[SCHISCHKA]

27 minutes ago, BDunkz said:

But I'm storing family photos and important documents so dont I need redundancy?

you need offline backup. how often do you need to store your photos?

photos can be done once a week using USB drives stored offline & rotated offsite.

for daily documents use your NAS, cloud storage, and USB drives as above.

I feel like a broken record. Redundancy keeps your webserver or database running non-stop when a drive fails. it does not protect your photos/documents and is not a replacement for a proper backup plan.

[SCHISCHKA]

lightning

fire

theft

ransomeware

accidental file deletion

 

RAID protects you from non of these.

[BDunkz]

2 minutes ago, SCHISCHKA said:

you need offline backup. how often do you need to store your photos?

photos can be done once a week using USB drives stored offline & rotated offsite.

for daily documents use your NAS, cloud storage, and USB drives as above.

I feel like a broken record. Redundancy keeps your webserver or database running non-stop when a drive fails. it does not protect your photos/documents and is not a replacement for a proper backup plan.

so if a drive fails will I lose the stuff on it?

 

[BDunkz]

1 minute ago, SCHISCHKA said:

lightning

fire

theft

ransomeware

accidental file deletion

 

RAID protects you from non of these.

that's fine I just don't want the photos lost due to coruption

[SCHISCHKA]

Just now, BDunkz said:

so if a drive fails will I lose the stuff on it?

 

of course but that is what backups are for.

OK here are your options:

two drives in RAID1: give you protection from drive failure

OR

one drive with one USB external making copies that gets rotated with another USB stored offsite: gives you protection from drive failure and everything else that can cause data loss.

 

[BDunkz]

Just now, SCHISCHKA said:

of course but that is what backups are for.

OK here are your options:

two drives in RAID1: give you protection from drive failure

OR

one drive with one USB external making copies that gets rotated with another USB stored offsite: gives you protection from drive failure and everything else that can cause data loss.

 

ok so with raid 1 I wont lose photos if a drive fails but the server will be down till I get a new drive?

 

[BDunkz]

what if I use something like unraid?

[SCHISCHKA]

22 minutes ago, BDunkz said:

what if I use something like unraid?

unraid is an OS. unraid just makes things user friendly with a web interface.

[Jarsky]

49 minutes ago, BDunkz said:

ok so with raid 1 I wont lose photos if a drive fails but the server will be down till I get a new drive?

 

RAID0 stripes file data across all disks. a single drive failure would be total data loss. Requires at least 2 disks.

RAID1 has your complete file on a disk, which is then mirrored to a secondary disk. if a drive fails, your data will be available, but unprotected until you replace the drive and the data is re-mirrored. Requires an even number of disks.

RAID5 (RAIDZ1 in ZFS) like raid0 stripes file data across all disks but implements parity. If a drive fails your data will be available, but unprotected until you replace the drive and the parity is recalculated across the array. This requires at least 3 disks - with a maximum recommendation of 6-8 off memory.

RAID6 (RAIDZ2 in ZFS) would be 2 drive tolerance- same as RAID5, but this uses 2 parity drives - when 1 disk goes down your data is still protected.Requires at least 4 drives, and highly expandable.

RAIDZ3 in ZFS - same as RAIDZ2 but 3 drive tolerance.

 

UnRAID (XFS & BTRFS) stores its data and parity in a different way to traditional RAID & ZFS - it writes the entire file and parity to a single disk. The advantage to this is that in the event you cannot recalculate parity (e.g a second drive is lost) then you only use those drives data, not the entire array. additionally it means you can mix drive capacities - so you can add all your drives to the array regardless if theyre 500GB, 1TB, 2TB or 4TB - you could use all 4 of these drives in a single unRAID array. The downside is that because you arent striping data you're only utilising that single drives speed - so it is slower for read & write, however unRAID implements a CoW caching system so you can use SSD's to boost the write speed to the array (it writes to the ssd cache, and then copies it to the protected data array). The speed isnt generally an issue to most home users - but for business use you're better to use a traditional raid or zfs.

[ChalkChalkson]

11 hours ago, Jarsky said:

...

I thank you for laying it out for OP, I feel we talked over him a lot.

But what would you recommend for OPs home NAS?

[ChalkChalkson]

11 hours ago, SCHISCHKA said:

unraid is an OS. unraid just makes things user friendly with a web interface.

 

12 hours ago, BDunkz said:

what if I use something like unraid?

Unraid isn't really just an OS, as @Jarsky helpfully stated.

It has its own way of building the same redundancy as a RAID 5 without striping, so even if you lose the "array"  (meaning the redundancy and a data drive) you won't lose all data, just the data on the compromised data drive (data drives if you have multiple failures). It also has the giant benefit @Jarsky described of allowing drives of different sizes, and it allows flexible adding of drives, you don't need to add them in specific chunks, or even start from scratch each time.

If you use a single parity drive in unraid, and a single drive fails everything is fine, you can even access your data while you are waiting for a new drive, no problem. The performance of unraid however is (more of less) the same as if you just used a single massive drive

[ChalkChalkson]

14 hours ago, Jarsky said:

But given the experience of OP, it probably wouldnt be suitable though you can use something like Webmin to create basic software raids like with any NAS OS GUI but its much less feature rich - most commands should be done via CLI. 

Given OPs experience I'd go with unraid, great documentation, and you get by without knowing a single thing about Linux. RAID, BTRFS, XFS etc. but still get redundancy and many nice server features, with the possibility of just treating it like any other linux server in the future.

[BDunkz]

19 hours ago, ChalkChalkson said:

RAID isn't that great if you want to have your system be easily expandable... Unraid has a great solution for this, it uses copy on write parity that is equivalent to RAID 5 or 6, but doesn't stripe the data, and drives can added at any time.

OS: I'd suggest unraid, or, if paying for a piece of software is not an option for some reason, FreeNAS is a great option, though I can't provide nearly as much help with FreeNAS, as I could with unraid  

How much is unsaid?

[ChalkChalkson]

Just now, BDunkz said:

How much is unsaid?

about 50$ i think

unrelated funfact: you can chain unraid free trials... they really ought to fix that, I am sure they are losing quite a bit of money with that