Backing Up Issue - Need Some Help

[mealto]

ok, here is the situation. Have a Windows server running Stablebit Drivepool. Let's make the numbers easy to describe. 4x2TB Drives. So the Drivepool is actually 8GB in size. When I backup a PC and image it using either Macrium Reflect Free or Veeam Endpoint Backup Free, they both complain of not enough storage space. I am guessing that because this is an image backup, the actual file size should be around 2.5 TB for that one single file. I guess the 2.5 TB cannot physcially fit on to 2TB drives no matter how many I have. I was hoping Stabelbit Drivepool was smart enough to handle this since it does show 8TB of free space. Am I right in thinking that 2.5TB cannot fit into any drivepools?

 

I have not tested out Windows 10 Storage Spaces yet, I wonder if it can deal with this issue. Trying really hard NOT to use Storage Spaces actually. 

 

Coming from WHS 2011, I have never had to deal with this issue even for VERY large backups. It must handle things more efficiently or differently.

 

Anyone come across this issue? I might just have to bite the bullet and pick up a dedicated 4TB drive just for this one PC's backup!

[Electronics Wizardy]

If your running 4 equal drives just run a raid 5. 

 

Also there may be a limit on file size on one drive.

 

How big is your systems hdd your backing up. Images can use lots of space.

[Elliott Game]

2 minutes ago, Electronics Wizardy said:

If your running 4 equal drives just run a raid 5. 

 

Also there may be a limit on file size on one drive.

 

How big is your systems hdd your backing up. Images can use lots of space.

why not RAID 6?

[Electronics Wizardy]

1 minute ago, Elliott Game said:

why not RAID 6?

Your running 4 drives. Your wasting lots of space, esp for a server holding backups. Raid 6 will run fine.

[Elliott Game]

3 minutes ago, Electronics Wizardy said:

Your running 4 drives. Your wasting lots of space, esp for a server holding backups. Raid 6 will run fine.

yeah, i guess you mostly would wanna use RAID 6 on a large RAID array...

[mealto]

I don't have any experience running RAID and since I know WIndows + Drivepool, I want to stick to this. But since we are on this topic, will a RAID array let me save one large 3TB file to 4x2TB drives in that array?

[Elliott Game]

1 minute ago, mealto said:

I don't have any experience running RAID and since I know WIndows + Drivepool, I want to stick to this. But since we are on this topic, will a RAID array let me save one large 3TB file to 4x2TB drives in that array?

of coarse!

also, whats ya backup program?

[Elliott Game]

2 minutes ago, mealto said:

I don't have any experience running RAID and since I know WIndows + Drivepool, I want to stick to this. But since we are on this topic, will a RAID array let me save one large 3TB file to 4x2TB drives in that array?

and also, I think RAID is safer, because if ya RAID card dies, RAID arrays can be imported back into a new RAID controller. But I think its harder to do with a drive pool. (Correct me if im wrong)

[mealto]

I am teting out Macrium Reflect Free and Veeam Endpoint Backup Free. Both are complaining that my Stablebit Drivepool has not space. I checked on the drives and sure enough, the one that Drivepool is backing to is full. I was hoping Stabelbit Drivepool would be smart and really pool those drives into one large disk that can allow even 4x2TB drives to copy over a file that is 3TB is size.

 

I may need to rethink my entire setup. 

 

So I am using this to backup PC's and store files (in duplication). It's meant for archiving all data from the entire household if that makes sense. 

 

What suggestions do you guys have. 

[Electronics Wizardy]

2 minutes ago, Elliott Game said:

and also, I think RAID is safer, because if ya RAID card dies, RAID arrays can be imported back into a new RAID controller. But I think its harder to do with a drive pool. (Correct me if im wrong)

A drive pool is normally easier to recover as you can read the files from one drive

[mealto]

2 minutes ago, Elliott Game said:

and also, I think RAID is safer, because if ya RAID card dies, RAID arrays can be imported back into a new RAID controller. But I think its harder to do with a drive pool. (Correct me if im wrong)

For sure it's the other way around. Drivepool is much easier. No RIAD controller, no hardware. Just plug that drive from the Drivepool to another Windows machine and it's readbale. In fact, if you plug that drive into another computer that is running Drivepool, it will add it to that pool automatically.

 

Drivepool allows some folders to be duplicated across drives like important data. Then on folders that say takes a snapshot of your current PC, I don't duplicate those since the chances of both the server and the PC goes down at the same time is less likely.

[Electronics Wizardy]

2 minutes ago, mealto said:

I am teting out Macrium Reflect Free and Veeam Endpoint Backup Free. Both are complaining that my Stablebit Drivepool has not space. I checked on the drives and sure enough, the one that Drivepool is backing to is full. I was hoping Stabelbit Drivepool would be smart and really pool those drives into one large disk that can allow even 4x2TB drives to copy over a file that is 3TB is size.

 

I may need to rethink my entire setup. 

 

So I am using this to backup PC's and store files (in duplication). It's meant for archiving all data from the entire household if that makes sense. 

 

What suggestions do you guys have. 

Id probably use windows backup. It allows making images.

 

Id also use rockstor/freenas and run a zfs/btrfs raid 5.

[mealto]

7 minutes ago, Electronics Wizardy said:

Id probably use windows backup. It allows making images.

 

Id also use rockstor/freenas and run a zfs/btrfs raid 5.

I tested out Windows Image, it's not bad at all. The only downside I found was that over a network, it only keeps 1 instance so you can roll back to 1 state. With other image backups, I am roll back 1 week or 1 month.

 

Keep the suggestions coming as to my next build TIA!

[Elliott Game]

28 minutes ago, Electronics Wizardy said:

A drive pool is normally easier to recover as you can read the files from one drive

soz, had a brainfart when I made the comment...

[leadeater]

3 hours ago, mealto said:

I tested out Windows Image, it's not bad at all. The only downside I found was that over a network, it only keeps 1 instance so you can roll back to 1 state. With other image backups, I am roll back 1 week or 1 month.

 

Keep the suggestions coming as to my next build TIA!

Before Veeam Endpoint Backup existed I got round that issue by using iscsi mounted storage so you could get multiple revisions/days using Windows Backup from a network NAS. I'd stick with your current options you are looking at though.

[dalekphalm]

9 hours ago, mealto said:

ok, here is the situation. Have a Windows server running Stablebit Drivepool. Let's make the numbers easy to describe. 4x2TB Drives. So the Drivepool is actually 8GB in size. When I backup a PC and image it using either Macrium Reflect Free or Veeam Endpoint Backup Free, they both complain of not enough storage space. I am guessing that because this is an image backup, the actual file size should be around 2.5 TB for that one single file. I guess the 2.5 TB cannot physcially fit on to 2TB drives no matter how many I have. I was hoping Stabelbit Drivepool was smart enough to handle this since it does show 8TB of free space. Am I right in thinking that 2.5TB cannot fit into any drivepools?

 

I have not tested out Windows 10 Storage Spaces yet, I wonder if it can deal with this issue. Trying really hard NOT to use Storage Spaces actually. 

 

Coming from WHS 2011, I have never had to deal with this issue even for VERY large backups. It must handle things more efficiently or differently.

 

Anyone come across this issue? I might just have to bite the bullet and pick up a dedicated 4TB drive just for this one PC's backup!

Drivepool is a folder level virtual disk pooling software. With these types of setups (FlexRAID is another similar example), a file cannot be larger than any one individual HDD.

 

This is because Drivepool will intelligently store files across your 4x2TB HDD's, making Windows THINK it's one giant 8TB volume. Despite the fact that Windows thinks this, it's still just a bunch of HDD's strung together. 4x 1.5TB files would fit on your pool, but a single 2.5TB file would not.

 

So to get around this issue, you can do one of a few different things:

1. Create a hardware or software Disk Level RAID 5 array - this will take all 4 drives, and span data across them evenly. You'll lose 2TB of storage though, as RAID 5 needs a single drives worth of space for Parity calculations.

2. Create a hardware or software Disk Level RAID 0 array - you lose any redundancy, and a single lost HDD will mean the entire array is lost, but it would give you the space of all 8TB's.

3. Use different image backup software that allows you to split the image up into many small files. Quite a few backup software have this feature.

 

I have to question what exactly you are backing up, that takes 2.5TB though? You should backup your programs and the OS, sure, but stuff like regular documents, movies, music, why not store this directly on the server anyway?

[Dark]

12 hours ago, Elliott Game said:

yeah, i guess you mostly would wanna use RAID 6 on a large RAID array...

The only time you would want RAID 6 is if you need double parity.

The disadvantage to RAID6 in a low count drive array is the higher percentage loss of usable storage in comparison to raw storage.

[Dark]

12 hours ago, Elliott Game said:

and also, I think RAID is safer, because if ya RAID card dies, RAID arrays can be imported back into a new RAID controller. But I think its harder to do with a drive pool. (Correct me if im wrong)

That's a crap shoot and heavily dependent on the controller.  Not to say the controller shouldn't be able to accept the foreign array but they don't always play nice; more so if you're going from one brand to another.

[mealto]

2 hours ago, dalekphalm said:

Drivepool is a folder level virtual disk pooling software. With these types of setups (FlexRAID is another similar example), a file cannot be larger than any one individual HDD.

 

This is because Drivepool will intelligently store files across your 4x2TB HDD's, making Windows THINK it's one giant 8TB volume. Despite the fact that Windows thinks this, it's still just a bunch of HDD's strung together. 4x 1.5TB files would fit on your pool, but a single 2.5TB file would not.

 

So to get around this issue, you can do one of a few different things:

1. Create a hardware or software Disk Level RAID 5 array - this will take all 4 drives, and span data across them evenly. You'll lose 2TB of storage though, as RAID 5 needs a single drives worth of space for Parity calculations.

2. Create a hardware or software Disk Level RAID 0 array - you lose any redundancy, and a single lost HDD will mean the entire array is lost, but it would give you the space of all 8TB's.

3. Use different image backup software that allows you to split the image up into many small files. Quite a few backup software have this feature.

 

I have to question what exactly you are backing up, that takes 2.5TB though? You should backup your programs and the OS, sure, but stuff like regular documents, movies, music, why not store this directly on the server anyway?

Serious thanks for all the info @dalekphalm. It's exactly what I found out after some investigation, the limit of the largest disk is the size limit of a backup image. At this point, I plan to use WIn 10 Pro as the backup server. I now have to upkeep server and my desktop. I just had a thought this morning. I am now seriously considering building a more powerful machine and using it as my desktop and running the "server" off it too. Why shouldn't I do this actually! I can still run Drivepool on some of those drives and file server to the rest of the houslehold and run backups to my desktop. This saves me from having to maintain the server and my desktop, both of which are maxed out in drive bays right now. Any thoughts?

 

And the 2.5 TB is all photos and videos of the family. It's my entire digital collection for all the years since we have made digital content. Right now, it's all sitting in a 3TB drive so file level backups take a little longer. A full image backup is quicker. Having used WHS 2011, the smart de-duplication backup was fantastic and nothing on the market is able to replace it less than $900 or going to Windows Essentials 2012, both of which I am not doing.

 

Any thoughts would be appreciated.

[mealto]

3 hours ago, dalekphalm said:

3. Use different image backup software that allows you to split the image up into many small files. Quite a few backup software have this feature

Hey @dalekphalm, which backup solution would do this? I don't believe image backups for Macrium Reflect Free would do this. I do see Veeam Endpoint Backup has file and folders backup but if I choose this, I cannot choose any other instances (ie. cannot run another backup instance for imaging). So silly of Veeam to do this.

[dalekphalm]

21 minutes ago, mealto said:

Hey @dalekphalm, which backup solution would do this? I don't believe image backups for Macrium Reflect Free would do this. I do see Veeam Endpoint Backup has file and folders backup but if I choose this, I cannot choose any other instances (ie. cannot run another backup instance for imaging). So silly of Veeam to do this.

A couple of options:

http://www.todo-backup.com/products/features/backup-image-splitting.htm
http://www.boot-disk.com/programs_activediskimage.htm

 

I haven't used either of those specifically, but I have used some of their other products.

 

here's another that LifeHacker recommends:

http://lifehacker.com/326086/hot-image-your-pcs-hard-drive-with-driveimage-xml