Jump to content

Server Storage and Redundancy Techniques and Software

I am curious what hard drives and SSDs are people using on their home servers/NAS/video surveillance storage and generally for large home data storage. I am also wondering what redundancy techniques and software are people using on those servers/NAS/etc and on their personal devices that they are backing up to their home server.

 

I know this topic is quite wide-ranging, but I felt it would not have made a lot of sense if I broke it in two or three different posts.

 

Server Storage Devices

 

Personally, I've started with some Seagate IronWolf 6TB hard drives and then jumped on Seagate Exos 10TB hard drives.

 

I also have some SATA SSDs (Samsung 860) in the server that I have set in RAID 0 (through software) which I am using them to store and run my virtual machines from them.

 

Server Redundancy Technique (RAID, FileSync-ing, Backups etc)

 

Most of the data I have on the server is personal data, backups and media. I do not have surveillance footage on the server at this time, but I might have it in the future and I would to hear about how people who have that are dealing with it.

 

For personal data (including projects, work, study etc), I keep it on my main laptop, on OneDrive (some 1TB available there and extremely cheap), on the server (syncs from OneDrive) and as backups (also on the same server, but on another drive than the OneDrive folder).

 

For the backups themselves, I don't use any kind of protection (if they're gone, then they're gone, I'll just make new ones) but I do run additional backup plans that store backups locally on my other computers and laptops. So for a system backup, I would have a comprehensive backup plan that backs up the system multiple times per week and then keeps several chains of incremental backups that go back a couple of months and stores those backups on the server. At the same time, I would have a secondary backup plan for the same data that runs only once per week and only keeps one chain of incremental backups going back 4 weeks and stores those backups on the individual device (on a drive other than the system drive being backed up, naturally).

 

For my media library, I wanted initially to go with a RAID 1 approach or some other hardware or software real-time disk mirroring approach, but I ended up using a file syncronisation/folder mirroring software called Bvckup 2. I use it to sync my media library and its backup (located on separate hard drives) every 24 hours (though the software can be used for real-time synchronisation as well). What I love about this software, besides the fact that it does delta copying and that it is very user friendly, is that I was able to set it up so that each time a file gets deleted from the main library, rather than being simply deleted from the backup library as well, the file gets marked for deletion and is moved into an archive where it automatically gets deleted 2 days later (it can be set for any period) unless restored. So new files get backed up no later than 24 hours (or immediately if I were to use real-time sync) and deleted files remain accessible for up to 3 days (2 days minimum plus the period between deleting the file and the daily sync task).

 

Software for Backup of Server and Personal Devices

 

In regards to back-up software, I am using Acronis True Image for my computers, Acronis Backup for my server and virtual machines (I know it's a bad choice, I wanted to use Veeam for my VMs, but it does not support VMWare Workstation, only Esxi) and, for my Android phone I am using a combination of OneDrive for my photos, OneSync for other small files (e.g. configuration files etc) I want on OneDrive (the OneDrive app does not allow automatic upload of files from my phone, except for photos and videos, so that's why I need OneSync) and Resilio Sync (basically Bvckup 2 but peer-to-peer based rather than local/LAN limited) for other backups (though mostly I use Resilio Sync for sync-ing movies between my Server and my phone because I don't want to pay for a Plex subscription for the "priviledge" of downloading files from my own server).

 

What advice would others offer me on how to improve this set-up and what storage hardware and redundancy techniques and software are you using for your servers/NAS?

Link to comment
Share on other sites

Link to post
Share on other sites

My NAS has 5 WD Red 3TB HDDs + 1 SanDisk SSD Plus 120GB as a Cache.

I use unRAID single parity for redundancy and I don't really have a backup solution for the machine as I have nowhere to put those 12TB of backed up files.

WHIPLASH

CPU: Intel Core i7 6700k @4.7GHz

RAM: Corsair Vengeance RGB DDR4 2x8GB @3000MHz

MOBO: Asus ROG Maximus VIII Ranger

GPU: EVGA GTX 1080 FTW2

PSU: EVGA Supernova 650GS

CASE: Fractal Design Define S

Link to comment
Share on other sites

Link to post
Share on other sites

I'm using RAID 10 for redundancy + improved IO Times (it fully saturates my Gbe), on a synology DS416j.

I back up all more important stuff (not much, less than 1TB) on my OneDrive that comes with my office subscription.

I use WD Red Pros, as I heard the regular Reds were good, and decided to one-up that recommendation with the red pros (I have 4 of the 6TB 7200RPM 256MB cache WD Red pros, they might be helium-filled).

 

If I ever upgrade my NAS, I'd go with something like a 1U or 2U rack mount, w/ a mITX embedded x86 cpu and 10 Gbe.

Do the server-grade drives come with warranty, b/c they might be classified as OEM.

Link to comment
Share on other sites

Link to post
Share on other sites

Also, apparently something that can speed up your NAS is to buy some used server SSDs (MLC ones will last basically forever), and use those as a cache, but network is usually the bottleneck.

Link to comment
Share on other sites

Link to post
Share on other sites

14 minutes ago, Firewrath9 said:

I'm using RAID 10 for redundancy + improved IO Times (it fully saturates my Gbe), on a synology DS416j.

I back up all more important stuff (not much, less than 1TB) on my OneDrive that comes with my office subscription.

I use WD Red Pros, as I heard the regular Reds were good, and decided to one-up that recommendation with the red pros (I have 4 of the 6TB 7200RPM 256MB cache WD Red pros, they might be helium-filled).

 

If I ever upgrade my NAS, I'd go with something like a 1U or 2U rack mount, w/ a mITX embedded x86 cpu and 10 Gbe.

Do the server-grade drives come with warranty, b/c they might be classified as OEM.

They do come with warranty. IronWolf is 2 years, IronWolf Pro is 5 years and Exos is 5 years. Exos is basically the best and it is at the same price as IronWolf whilst the IronWolf Pro is usually bout $60 more expensive. Also, some Exos drives come with hard drive encryption (they have something simillar to a TPM module in them). You can also get Exos SAS, but I don't have SAS support so I went to the usual SATA ones (however, there is virtually no price difference between them). 

Link to comment
Share on other sites

Link to post
Share on other sites

18 minutes ago, Firewrath9 said:

Also, apparently something that can speed up your NAS is to buy some used server SSDs (MLC ones will last basically forever), and use those as a cache, but network is usually the bottleneck.

Yes, I considered tiered storage, but I'm not sure that would help with media files and backups (those are the largest files on my server). My personal files (under 200GB) are simply being kept on NVMe SSDs (on my PC) and SATA SSDs (on my server) so they are not an issue in term of speed.

Link to comment
Share on other sites

Link to post
Share on other sites

13 minutes ago, ArchGabriel said:

They do come with warranty. IronWolf is 2 years, IronWolf Pro is 5 years and Exos is 5 years. Exos is basically the best and it is at the same price as IronWolf whilst the IronWolf Pro is usually bout $60 more expensive. Also, some Exos drives come with hard drive encryption (they have something simillar to a TPM module in them). You can also get Exos SAS, but I don't have SAS support so I went to the usual SATA ones (however, there is virtually no price difference between them). 

Did you register, because I've heard of ppl buying Enterprise/Sever level HDDs (WD Gold/HGST (rip) He10) incl. the Exos where they were considered OEM as the seller just bought them oem w/o warranty.

Link to comment
Share on other sites

Link to post
Share on other sites

42 minutes ago, Firewrath9 said:

Did you register, because I've heard of ppl buying Enterprise/Sever level HDDs (WD Gold/HGST (rip) He10) incl. the Exos where they were considered OEM as the seller just bought them oem w/o warranty.

 
 

Yes, I've registered them on the Seagate Website.  Shows 5 years and the seller lists 5 years as well. This is the one I've got: https://www.ebuyer.com/743943-seagate-enterprise-capacity-10tb-3-5-hard-drive-512e-sata-at-ebuyer-com-st10000nm0016

Link to comment
Share on other sites

Link to post
Share on other sites

Also, here is proof from the Seagate registration portal.

gf.png

Link to comment
Share on other sites

Link to post
Share on other sites

All my drives, boot, data, and backup, are now SSDs. I gave away most of my spinners (I still have the original 5400 rpm 2.5" 500GB Seagate HDDs from my notebooks and a couple of 7200 rpm 2.5" 500GB WD Blacks but they are mothballed).

 

My "redundancy" is a close cousin to sneakernet. Drive failure is not the only cause of data loss (malware, theft, disaster, user error, etc. are also causes for data loss) so my emphasis is on a solid backup scheme. On my desktop machine (deceased with a new one under construction), I have a set of four backup drives for each data drive in the machine (all SSDs): two of each set are kept onsite in a drawer and the other two are kept offsite in my safe deposit box at my credit union. I swap the onsite and offsite backups no less than once a month to ensurre the offsite backups are as up to date as practical.

 

Since any drive, including backup drives, can irrecoverably fail at anytime without warning, I have the extra backup drives (my so called redundancy). It saved my bacon data once when a corrupted data drive corrupted a backup drive (I also tweaked my backup procedure a bit after that).

 

I switched to SSDs because the HDDs I was lugging back and forth between home and my credit union were killing my shoulders and back. They also ate up a lot of space and I was running out of room to store them. To justify the expense, I decided to keep my then nine year old pickup for another eight years (it was a year old when I bought it). That actually saved me half money I would have spent otherwise.

 

I got a Pelican case to transport the backup drives in.

 

IMG_0005.thumb.JPG.f22768af9343be46caf2fdcfb1028570.JPG

 

I made a pair of antistatic foam "egg crates" (pigeon holes?) to keep the bare drives in. They are a slip fit inside the Pelican case. The blue ribbons are for lifting the egg crate out of the Pelican case and a drawer.

 

IMG_0006.thumb.JPG.0cb8cfe331ffc331e63073899b52584a.JPG

 

There are way more slots in the egg crate than I will ever need (there is no way I'll ever have 11 data drives) but that's how the sizing worked out. The ribbons are blue because that was what I had on hand (besides, I like blue ? ).

 

I keep one egg crate with half of the backup drives in my safe deposit box and the other one with the other half of the backup drives in a drawer.

 

IMG_0005.thumb.JPG.01a110a559ff54879ecef8895912c422.JPG

 

I've moved the egg crate to a smaller drawer since this picture was taken. The fact it fits in the smaller drawer perfectly was pure dumb luck. I've also added a fifth data drive (with four more backup drives; two per egg crate) since these pictures were taken).

 

Updating backups is simple. In my old desktop machine, I had three 2.5" hotswap bays (the new one will have four; I won't need more than that often enough to bother with putting in more than four...for now). I would plug in up to three backup drives, then use FreeFileSync, a folder/file syncing program, to update the drives simultaneously (each backup drive being plugged in had to be for different data drives). It took just a few seconds to plug in the backup drives,  no more than a minute or two for the computer to find and index the drives, and just a few seconds to start the backups. Then, I would amuse myself (use your imagination but keep it clean) until all three backups were done, which usually took only a few minutes, shut down all the instances of FreeFileSync, remove the backup drives, then rinse and repeat until done. I could even keep using the computer while the backups were updating. Easy peasy.

 

Storing the backups powered down and disconnected from the computer prevents them from getting infected should the computer get infected. Even though I have full time AV and antimalware protection running, I run scans prior to updating my backups just to be on the safe side (yeah, I'm that paranoid). Having the offsite backup protects my data should my home burn down. Updating the backups at least once a month also ensures the SSDs won't lose data due to the charge that denotes the data leaking down (not likely but why take chances).

 

Besides backing up data, I also make an image of the boot drive so, if the OS gets borked, I can easily restore it. Same is true if the boot drive croaks (I had it happen once; It took me only 45 minutes to get the computer back up and running with most of the time spent being on replacing the old drive).

 

Since my desktop machine died and has been parted out and the new machine hasn't been finished yet (health issues and life happening), I keep the data and backup SSDs refreshed by plugging each data SSD and a backup SSD into my notebook via SATA to USB cables. It's a bit tedious but still doesn't take too long. I need to do it only once a month just before and after I swap out the onsite and offsite backups.

 

My notebooks are handled a bit differently. The one I'm currently using has only data that is also contained on one of the desktop's data drives so I just keep a couple of 2TB SSDs in USMs (Universal Storage Modules) that have a clone of the notebook's drive in it. Usually, I just use FreeFileSync to update the data partition only of the clones. I also use the USM as a sneakernet to do a two way sync with the folders it shares with the desktop data drive. Since the desktop data drive also gets backed up (to the teeth), I feel the notebook is more than adequately protected this way.

 

Every once in a blue moon (genrally after adding or updating programs), I'll reclone the notebook backups. I also make images of the OS and System Reserved partitions. If the drive in my one drive wonder of a notebook should completely die, I can remove a drive from one of the USMs and swap it into the notebook in just a few minutes no matter where I am.

Jeannie

 

As long as anyone is oppressed, no one will be safe and free.

One has to be proactive, not reactive, to ensure the safety of one's data so backup your data! And RAID is NOT a backup!

 

Link to comment
Share on other sites

Link to post
Share on other sites

11 hours ago, ArchGabriel said:

They do come with warranty. IronWolf is 2 years, IronWolf Pro is 5 years and Exos is 5 years. Exos is basically the best and it is at the same price as IronWolf whilst the IronWolf Pro is usually bout $60 more expensive. Also, some Exos drives come with hard drive encryption (they have something simillar to a TPM module in them). You can also get Exos SAS, but I don't have SAS support so I went to the usual SATA ones (however, there is virtually no price difference between them). 

That's strange indeed... my ironwolf 4TB drives have 3 years warranty, or 5 can't remember now. [edit] - it's 3 years, just checked.

 

so, on my main server, I have 3x Ironwolf 4TB drives, and 1 WD red 4TB. On my backup server I have 3x 3TB WD reds, the main server backs up media and important files to the backup server... in the case of main server fails or needs rebuilding, backup server gets promoted to live server for the duration. In addition to the servers, I have a 2TB external drive at my brothers house, that I VPN into to backup my important files onto for off-site backup... usually once a week, but can do it anytime I have a major change or whatever too. I also keep some files locally on my main PC, but just important files... and last but not least I have several USB drives with important files that can be carried on my person (and encrypted).... and my newest addition of a 240GB M.2 drive in a USB C caddy for fast transfers for portable convenience too (Transfers @ approx 320MB/s) for those times when I need it quicker.

Please quote my post, or put @paddy-stone if you want me to respond to you.

Spoiler
  • PCs:- 
  • Main PC build  https://uk.pcpartpicker.com/list/2K6Q7X
  • ASUS x53e  - i7 2670QM / Sony BD writer x8 / Win 10, Elemetary OS, Ubuntu/ Samsung 830 SSD
  • Lenovo G50 - 8Gb RAM - Samsung 860 Evo 250GB SSD - DVD writer
  •  
  • Displays:-
  • Philips 55 OLED 754 model
  • Panasonic 55" 4k TV
  • LG 29" Ultrawide
  • Philips 24" 1080p monitor as backup
  •  
  • Storage/NAS/Servers:-
  • ESXI/test build  https://uk.pcpartpicker.com/list/4wyR9G
  • Main Server https://uk.pcpartpicker.com/list/3Qftyk
  • Backup server - HP Proliant Gen 8 4 bay NAS running FreeNAS ZFS striped 3x3TiB WD reds
  • HP ProLiant G6 Server SE316M1 Twin Hex Core Intel Xeon E5645 2.40GHz 48GB RAM
  •  
  • Gaming/Tablets etc:-
  • Xbox One S 500GB + 2TB HDD
  • PS4
  • Nvidia Shield TV
  • Xiaomi/Pocafone F2 pro 8GB/256GB
  • Xiaomi Redmi Note 4

 

  • Unused Hardware currently :-
  • 4670K MSI mobo 16GB ram
  • i7 6700K  b250 mobo
  • Zotac GTX 1060 6GB Amp! edition
  • Zotac GTX 1050 mini

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

My DS218 NAS has 2 4TB IronWolf drives. My back-up solution is a weekly differential back-up to a USB HDD (also 4TB). Works well enough.

 

My HP ProLiant G10 server only has a single 500GB 860 Evo SSD but that only runs a basic Windows Server 2016 AD server and RDSGW and a handful of Linux machines for specific tasks. File level backup is done to the NAS via Veeam backup and replication free as all I care about is the data. Config is all basic for a reason.

 

Then I have two more servers with SAS disks (one has 4x2TB the other 8x146GB), neither of them are backed up as they only serve as testlab machines or dumping big temporary files.

 

13 minutes ago, paddy-stone said:

That's strange indeed... my ironwolf 4TB drives have 3 years warranty, or 5 can't remember now. [edit] - it's 3 years, just checked.

Yep, mine have 3 years as well. Weird.

PC Specs - AMD Ryzen 7 5800X3D MSI B550M Mortar - 32GB Corsair Vengeance RGB DDR4-3600 @ CL16 - ASRock RX7800XT 660p 1TBGB & Crucial P5 1TB Fractal Define Mini C CM V750v2 - Windows 11 Pro

 

Link to comment
Share on other sites

Link to post
Share on other sites

FreeNAS with 3 arrays. 

5x 4TB RaidZ1 w/ 250GB intel SLC SSD cache: Main Storage (Personal data, movies, some VMs)

4x 3TB RaidZ1: Backups from the main array and personal computers

3x 500GB SSD Raid0: For volatile VMs that need performance, I think I do have 1 VM there I use FreeNAS to induce a snapshot on ESXi, create a snapshot on the dataset, then copy that over to my backup array.

 

~100GB used on amazon glacier for personal pictures / documents, encrypted then transmitted (rclone).

 

Hourly snapshots for personal data, kept for a week. Weekly snapshots kept for a month. Everything older than a month simply "expires" 

Data is real time replicated from my main array to my backup array. 

 

Currently at risk to full system failure (power surge / fire / flood) that would killoff all arrays. Surge is mitigated through a beefy battery backup w/ surge protection. Fire and Flood mitigated through prayer lol? I have a friend I'm working with on sharing storage with, however my upload sucks so I need to stick my initial copy on a hard drive to transfer over at his house. 

 

Edit* Wanted to add the datahoarder subreddit is worth a gander: https://www.reddit.com/r/DataHoarder/

Link to comment
Share on other sites

Link to post
Share on other sites

6 hours ago, paddy-stone said:

That's strange indeed... my ironwolf 4TB drives have 3 years warranty, or 5 can't remember now. [edit] - it's 3 years, just checked.

 

so, on my main server, I have 3x Ironwolf 4TB drives, and 1 WD red 4TB. On my backup server I have 3x 3TB WD reds, the main server backs up media and important files to the backup server... in the case of main server fails or needs rebuilding, backup server gets promoted to live server for the duration. In addition to the servers, I have a 2TB external drive at my brothers house, that I VPN into to backup my important files onto for off-site backup... usually once a week, but can do it anytime I have a major change or whatever too. I also keep some files locally on my main PC, but just important files... and last but not least I have several USB drives with important files that can be carried on my person (and encrypted).... and my newest addition of a 240GB M.2 drive in a USB C caddy for fast transfers for portable convenience too (Transfers @ approx 320MB/s) for those times when I need it quicker.

8

 

Oh, yes, that is 3 years, not 2 on the IronWolf. My mistake.

So you are saying that your main server backups up to your backup server. What about your other devices? Do they back up to the main server and then to the backup one or just to the backup one?

 

6 hours ago, NelizMastr said:

My DS218 NAS has 2 4TB IronWolf drives. My back-up solution is a weekly differential back-up to a USB HDD (also 4TB). Works well enough.

 

My HP ProLiant G10 server only has a single 500GB 860 Evo SSD but that only runs a basic Windows Server 2016 AD server and RDSGW and a handful of Linux machines for specific tasks. File level backup is done to the NAS via Veeam backup and replication free as all I care about is the data. Config is all basic for a reason.

 

Then I have two more servers with SAS disks (one has 4x2TB the other 8x146GB), neither of them are backed up as they only serve as testlab machines or dumping big temporary files.

 

Yep, mine have 3 years as well. Weird.

4

 

That is interesting. I was always worried about losing my VMs. But I can see your path is better by just keeping the VMs simple and then you can reconfigure easily. 

 

5 hours ago, Mikensan said:

FreeNAS with 3 arrays. 

5x 4TB RaidZ1 w/ 250GB intel SLC SSD cache: Main Storage (Personal data, movies, some VMs)

4x 3TB RaidZ1: Backups from the main array and personal computers

3x 500GB SSD Raid0: For volatile VMs that need performance, I think I do have 1 VM there I use FreeNAS to induce a snapshot on ESXi, create a snapshot on the dataset, then copy that over to my backup array.

 

~100GB used on amazon glacier for personal pictures / documents, encrypted then transmitted (rclone).

 

Hourly snapshots for personal data, kept for a week. Weekly snapshots kept for a month. Everything older than a month simply "expires" 

Data is real time replicated from my main array to my backup array. 

 

Currently at risk to full system failure (power surge / fire / flood) that would killoff all arrays. Surge is mitigated through a beefy battery backup w/ surge protection. Fire and Flood mitigated through prayer lol? I have a friend I'm working with on sharing storage with, however my upload sucks so I need to stick my initial copy on a hard drive to transfer over at his house. 

 

Edit* Wanted to add the datahoarder subreddit is worth a gander: https://www.reddit.com/r/DataHoarder/

 

Interesting approach on the backups. It pretty much sounds like you are doing a GFS rotation backup scheme. What software are you using for that? For your 100GB of personal data, how much space do you end up using with your hourly+weekly backups? 

For your full system failure, you could just use a fireproof box/safe or something similar. Alternatively, you could look to store the drives at a nearby storage space (self storage/storage rooms, whatever they are called). Here you could rent an entire square meter (10 sq feet) for about $20 per month. It is expensive, but less headache maybe than driving to your friend's house.

Link to comment
Share on other sites

Link to post
Share on other sites

2 hours ago, ArchGabriel said:

 

Oh, yes, that is 3 years, not 2 on the IronWolf. My mistake.

So you are saying that your main server backups up to your backup server. What about your other devices? Do they back up to the main server and then to the backup one or just to the backup one?

 

Yes, all backups of VMs, disk images and such are backed up to the main server, and then onto the backup server, along with media etc too. Basically most of my stuff gets stored on the servers, that way the backup server can get switched in if a problem with the main server comes up... I just find it easier to cope with this way. Then even if I am out of the house and cannot physically switch my backup server on, I can still tell one of my family to just press the power switch for me, lol. Everything else that I consider "priority or important" files/media and whatever, also gets backed up to the 2 servers, but also to a second location off-site, and to some USB drives. So not much chance of me losing my important files this way with around 5-7 copies on any given day approx.

I mostly use freefilesync for doing my backups, as it's super easy to work with and pretty configurable, including live sync too.

Please quote my post, or put @paddy-stone if you want me to respond to you.

Spoiler
  • PCs:- 
  • Main PC build  https://uk.pcpartpicker.com/list/2K6Q7X
  • ASUS x53e  - i7 2670QM / Sony BD writer x8 / Win 10, Elemetary OS, Ubuntu/ Samsung 830 SSD
  • Lenovo G50 - 8Gb RAM - Samsung 860 Evo 250GB SSD - DVD writer
  •  
  • Displays:-
  • Philips 55 OLED 754 model
  • Panasonic 55" 4k TV
  • LG 29" Ultrawide
  • Philips 24" 1080p monitor as backup
  •  
  • Storage/NAS/Servers:-
  • ESXI/test build  https://uk.pcpartpicker.com/list/4wyR9G
  • Main Server https://uk.pcpartpicker.com/list/3Qftyk
  • Backup server - HP Proliant Gen 8 4 bay NAS running FreeNAS ZFS striped 3x3TiB WD reds
  • HP ProLiant G6 Server SE316M1 Twin Hex Core Intel Xeon E5645 2.40GHz 48GB RAM
  •  
  • Gaming/Tablets etc:-
  • Xbox One S 500GB + 2TB HDD
  • PS4
  • Nvidia Shield TV
  • Xiaomi/Pocafone F2 pro 8GB/256GB
  • Xiaomi Redmi Note 4

 

  • Unused Hardware currently :-
  • 4670K MSI mobo 16GB ram
  • i7 6700K  b250 mobo
  • Zotac GTX 1060 6GB Amp! edition
  • Zotac GTX 1050 mini

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

2 hours ago, paddy-stone said:

Yes, all backups of VMs, disk images and such are backed up to the main server, and then onto the backup server, along with media etc too. Basically most of my stuff gets stored on the servers, that way the backup server can get switched in if a problem with the main server comes up... I just find it easier to cope with this way. Then even if I am out of the house and cannot physically switch my backup server on, I can still tell one of my family to just press the power switch for me, lol. Everything else that I consider "priority or important" files/media and whatever, also gets backed up to the 2 servers, but also to a second location off-site, and to some USB drives. So not much chance of me losing my important files this way with around 5-7 copies on any given day approx.

I mostly use freefilesync for doing my backups, as it's super easy to work with and pretty configurable, including live sync too.

I see. That is really a good strategy. For turning the backup on, you could use WoL or even IPMI if your motherboard supports it.

Link to comment
Share on other sites

Link to post
Share on other sites

10 hours ago, ArchGabriel said:

I see. That is really a good strategy. For turning the backup on, you could use WoL or even IPMI if your motherboard supports it.

Yeah, I have iLo, but haven't taken the time to get to know it yet... and it's a no with the WOL, last time I tried that it screwed me by keep sending the signal and waking my PC all the time. Would rather someone just physically goes the 20 steps to get to my server, LOL - it's unlikely to happen often if at all anyway, and there's pretty much always someone here at home.

Please quote my post, or put @paddy-stone if you want me to respond to you.

Spoiler
  • PCs:- 
  • Main PC build  https://uk.pcpartpicker.com/list/2K6Q7X
  • ASUS x53e  - i7 2670QM / Sony BD writer x8 / Win 10, Elemetary OS, Ubuntu/ Samsung 830 SSD
  • Lenovo G50 - 8Gb RAM - Samsung 860 Evo 250GB SSD - DVD writer
  •  
  • Displays:-
  • Philips 55 OLED 754 model
  • Panasonic 55" 4k TV
  • LG 29" Ultrawide
  • Philips 24" 1080p monitor as backup
  •  
  • Storage/NAS/Servers:-
  • ESXI/test build  https://uk.pcpartpicker.com/list/4wyR9G
  • Main Server https://uk.pcpartpicker.com/list/3Qftyk
  • Backup server - HP Proliant Gen 8 4 bay NAS running FreeNAS ZFS striped 3x3TiB WD reds
  • HP ProLiant G6 Server SE316M1 Twin Hex Core Intel Xeon E5645 2.40GHz 48GB RAM
  •  
  • Gaming/Tablets etc:-
  • Xbox One S 500GB + 2TB HDD
  • PS4
  • Nvidia Shield TV
  • Xiaomi/Pocafone F2 pro 8GB/256GB
  • Xiaomi Redmi Note 4

 

  • Unused Hardware currently :-
  • 4670K MSI mobo 16GB ram
  • i7 6700K  b250 mobo
  • Zotac GTX 1060 6GB Amp! edition
  • Zotac GTX 1050 mini

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

17 hours ago, ArchGabriel said:

nteresting approach on the backups. It pretty much sounds like you are doing a GFS rotation backup scheme. What software are you using for that? For your 100GB of personal data, how much space do you end up using with your hourly+weekly backups? 

For your full system failure, you could just use a fireproof box/safe or something similar. Alternatively, you could look to store the drives at a nearby storage space (self storage/storage rooms, whatever they are called). Here you could rent an entire square meter (10 sq feet) for about $20 per month. It is expensive, but less headache maybe than driving to your friend's house.

I'm just using ZFS snapshots for the hourly/weekly for accidental deletion, no software required. My actual backups are "live" (replicated) and 1:1, so my backups do not take up any more space than my current data (I do have a more agressive compression algorithm enabled, but it isn't doing much). Regarding the 100GB, since I'm using ZFS snapshots only the deltas take up space, so it is very little. The 100GB deltas are not pushed to amazon, only the data as it is during the push (weekly).

 

It's just a one time thing to drive to his house and copy the data, thereafter I will push deltas over our VPN. Issue is just spending the time copying ~10TB of data to disk, just lazy.

 

As @NelizMastr is doing, I too will start using Veeam for my virtual machines. They not too long ago expanded their free edition (now dubbed community edition) which will cover all 3 of my ESXi hosts. I do currently use Veeam windows agent for my desktop, which works really well.

Link to comment
Share on other sites

Link to post
Share on other sites

3 hours ago, Mikensan said:

As @NelizMastr is doing, I too will start using Veeam for my virtual machines. They not too long ago expanded their free edition (now dubbed community edition) which will cover all 3 of my ESXi hosts. I do currently use Veeam windows agent for my desktop, which works really well.

 
 
 

 

Are you sure? I read that it now covers 10 VMs. This is from their website:

 

"In addition to protecting 10 VMs with the level of capabilities provided in Veeam Backup & Replication Standard edition, Veeam Backup & Replication Community Edition also provides FREE, UNLIMITED ad-hoc VM backups and migrations for any additional VMs you may want to protect. With no agents to deploy, as well as powerful recovery options and VeeamZIP™, you get flexibility in options and a reliable, FREE VM backup solution for your daily workload management.  "

 

I'm not really sure what the difference is between the 10 VMs and the other "unlimited" backups.

Link to comment
Share on other sites

Link to post
Share on other sites

1 hour ago, ArchGabriel said:

 

Are you sure? I read that it now covers 10 VMs. This is from their website:

 

"In addition to protecting 10 VMs with the level of capabilities provided in Veeam Backup & Replication Standard edition, Veeam Backup & Replication Community Edition also provides FREE, UNLIMITED ad-hoc VM backups and migrations for any additional VMs you may want to protect. With no agents to deploy, as well as powerful recovery options and VeeamZIP™, you get flexibility in options and a reliable, FREE VM backup solution for your daily workload management.  "

 

I'm not really sure what the difference is between the 10 VMs and the other "unlimited" backups.

10 is fine for me, there are only a handful of VMs that are "critical". Namely my domain controller, SQL server, lab firewall, vcenter appliance, and my docker server. Everything else I could recreate pretty fast. The issue was previously any of those virtual machines could be on a different host as DRS balances the load, Veeam would say "oh no we only allow backups from 1 ESXi host" and I'd be screwed.

 

So I removed Veeam and started using FreeNAS to initiate a snapshot on the VM > take a ZFS Snapshot > delete the VM snapshot > replicate the data to my backup array. Convoluted but it works.

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

On 3/29/2019 at 1:37 AM, Mikensan said:

As @NelizMastr is doing, I too will start using Veeam for my virtual machines. They not too long ago expanded their free edition (now dubbed community edition) which will cover all 3 of my ESXi hosts. I do currently use Veeam windows agent for my desktop, which works really well.

You can also get a Veeam NFR license which is all things everything, similar to VMUG EVALexperience but Veeam NFR is free.

Link to comment
Share on other sites

Link to post
Share on other sites

On 3/30/2019 at 7:32 AM, leadeater said:

You can also get a Veeam NFR license which is all things everything, similar to VMUG EVALexperience but Veeam NFR is free.

I had the NFR, issue was it would only allow me to backup machines from a single hypervisor host :-( So if my DRS moved a VM to balance resources it wouldn't get backed up.

Link to comment
Share on other sites

Link to post
Share on other sites

On 3/27/2019 at 1:31 AM, ArchGabriel said:

I am curious what hard drives and SSDs are people using on their home servers/NAS/video surveillance storage and generally for large home data storage.

I just use whatever I can get on the cheap. I do check reviews to see that the drives I'm buying aren't terrible crap, but other than that, I don't see any reason to spend more time or money on that.

Hand, n. A singular instrument worn at the end of the human arm and commonly thrust into somebody’s pocket.

Link to comment
Share on other sites

Link to post
Share on other sites

My main active storage is 14 x 3TB Red's in a RAID6 with 2 x 500GB Samsung 860 Evo's

I then have 5 x 8TB Seagate's in a RAID5 which I use for long term storage

This is on an LSI 9271-8i RAID controller with a Intel RES2SV240 SAS Expander

 

I do have a new X299 system which is to be a new FreeNAS server, but I had to move a few months ago which put that project on hold due to the ridiculous cost of 10-12TB disks here in New Zealand and all my moving costs. 

 

Apart from a 1TB backup folder that syncs from my PC, its all just media so I don't need backups of the majority of it

Spoiler

Desktop: Ryzen9 5950X | ASUS ROG Crosshair VIII Hero (Wifi) | EVGA RTX 3080Ti FTW3 | 32GB (2x16GB) Corsair Dominator Platinum RGB Pro 3600Mhz | EKWB EK-AIO 360D-RGB | EKWB EK-Vardar RGB Fans | 1TB Samsung 980 Pro, 4TB Samsung 980 Pro | Corsair 5000D Airflow | Corsair HX850 Platinum PSU | Asus ROG 42" OLED PG42UQ + LG 32" 32GK850G Monitor | Roccat Vulcan TKL Pro Keyboard | Logitech G Pro X Superlight  | MicroLab Solo 7C Speakers | Audio-Technica ATH-M50xBT2 LE Headphones | TC-Helicon GoXLR | Audio-Technica AT2035 | LTT Desk Mat | XBOX-X Controller | Windows 11 Pro

 

Spoiler

Server: Fractal Design Define R6 | Ryzen 3950x | ASRock X570 Taichi | EVGA GTX1070 FTW | 64GB (4x16GB) Corsair Vengeance LPX 3000Mhz | Corsair RM850v2 PSU | Fractal S36 Triple AIO | 12 x 8TB HGST Ultrastar He10 (WD Whitelabel) | 500GB Aorus Gen4 NVMe | 2 x 2TB Samsung 970 Evo Plus NVMe | LSI 9211-8i HBA

 

Link to comment
Share on other sites

Link to post
Share on other sites

4 hours ago, Jarsky said:

My main active storage is 14 x 3TB Red's in a RAID6 with 2 x 500GB Samsung 860 Evo's

I then have 5 x 8TB Seagate's in a RAID5 which I use for long term storage

This is on an LSI 9271-8i RAID controller with a Intel RES2SV240 SAS Expander

 

I do have a new X299 system which is to be a new FreeNAS server, but I had to move a few months ago which put that project on hold due to the ridiculous cost of 10-12TB disks here in New Zealand and all my moving costs. 

 

Apart from a 1TB backup folder that syncs from my PC, its all just media so I don't need backups of the majority of it

May I ask what is your reasoning for having 14 small drives and 5 average sized drives (average in the context in which Seagate now does 16TB)? I would imagine fewer larger drives would have been easier to manage, use less space, use less energy and you would not have needed all those expansion cards. I have seen quite a few people using your strategy, I just don't get the reason behind it.

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×