Is there reliable home-user software for mirroring a partition with a physical drive.

[GuruMeditationError]

I want to backup an 8TB storage drive using 1 x 4TB HDD and 2 x 2TB HDDS.
 

I want to create a 1 x 4TB partition and 2 x 2TB partitions on the 8TB HDD and mirror those partitions to 1 x 4TB HDD and 2 x 2TB HDDs

 

I need them mirrored because I need to organise and update the data on the drives on an ongoing basis without having to keep re-writing the entire disc contents to the backups whenever changes are made. 

 

I should probably add that I can't span the lower-capacity drives because if there's a drive failure I'd loose the files that cross the span between the physical drives (the 4TB drive is showing bad sectors and needs to be backed up).

[Mira Yurizaki]

I use FreeFileSync to manage what's changed between my current copy and my back copy. But if you want versioning, you'll need something else, like using Windows' Shadow Copy.

[manikyath]

i do something similar (although the other way around) for my server's external backups.

 

my server's internal drives are 3TB, and the external is 4TB (that one extra TB is for scratch disk purposes). what i did was partition the 3TB disk, and then used some linux stuff to copy the partition with its exact size to the 4TB disk, and then partition off the remaining bit.

 

for my purpose its not really necessary to make them exactly as big because i use a script to make overnight mirrors, but its more so a nice peace of mind

 

as for the software side of things.. linux does that for you with an afternoon of scripting stuff.

[kb5zue]

I use some software produced by a company called AOMEI.  The software that I use is their partition assistant and their backup manager.  Best of all, they are both FREE.

 

http://www.aomeitech.com/aomei-partition-assistant.html

http://www.aomeitech.com/ab/standard.html

 

 

 

[GuruMeditationError]

6 minutes ago, M.Yurizaki said:

I use FreeFileSync to manage what's changed between my current copy and my back copy. But if you want versioning, you'll need something else, like using Windows' Shadow Copy.

I don't really understand what versioning is in this context?

Essentially I just need those three physical drives to mirror three partitions on the 8GB drive.

 

@M.Yurizaki Also, at a glance, it looks like Windows Storage Spaces can only mirror physical drives, so I'm not sure Shadow copy would be an option? Having said that, if I could do it all under windows it would probably be my preferred option.
 

4 minutes ago, manikyath said:

i do something similar (although the other way around) for my server's external backups.

 

my server's internal drives are 3TB, and the external is 4TB (that one extra TB is for scratch disk purposes). what i did was partition the 3TB disk, and then used some linux stuff to copy the partition with its exact size to the 4TB disk, and then partition off the remaining bit.

 

for my purpose its not really necessary to make them exactly as big because i use a script to make overnight mirrors, but its more so a nice peace of mind

 

as for the software side of things.. linux does that for you with an afternoon of scripting stuff.

Thanks, I've just never used Linux so it would probably take me a week or so to get it installed and get acclimatised enough to it before I could even identify what I'd need to know to frame the questions I need to ask to get the job done.  

4 minutes ago, kb5zue said:

I use some software produced by a company called AOMEI.  The software that I use is their partition assistant and their backup manager.  Best of all, they are both FREE.

 

http://www.aomeitech.com/aomei-partition-assistant.html

http://www.aomeitech.com/ab/standard.html

 

 

 

Do they actually support mirroring of partitions?  I'll take a look at them.

[manikyath]

1 minute ago, GuruMeditationError said:

Thanks, I've just never used Linux so it would probably take me a week or so to get it installed and get acclimatised enough to it before I could even identify what I'd need to know to frame the questions I need to ask to get the job done.  

i'm well aware that it's all voodoo, see that as a last resort

 

i only bothered with the whole mess because i'm a cheapskate, and had very limited physical space in my server for hardware options, as well as me being a tinfoil hat when it comes to relying on other people's voodoo for my storage.

[Mira Yurizaki]

5 minutes ago, GuruMeditationError said:

I don't really understand what versioning is in this context?

Essentially I just need those three physical drives to mirror three partitions on the 8GB drive.

 

@M.Yurizaki Also, at a glance, it looks like Windows Storage Spaces can only mirror physical drives, so I'm not sure Shadow copy would be an option? Having said that, if I could do it all under windows it would probably be my preferred option.

Versioning means you can roll back a file to a previous version if you want. The system only stores the changes though so it's not like storing a 12MB file that grows to 13MB means you now have 25MB taken up, it's only going to take up 13MB.

 

But if all you need to do is mirroring, then you don't have to worry about what I mentioned after the application.

 

Windows' backup is fine and only backs up what changed when you update, but I'm not sure if it lets you access the files like a regular file system or if you need to use the backup service to access it.

[GuruMeditationError]

5 minutes ago, manikyath said:

i'm well aware that it's all voodoo, see that as a last resort

 

i only bothered with the whole mess because i'm a cheapskate, and had very limited physical space in my server for hardware options, as well as me being a tinfoil hat when it comes to relying on other people's voodoo for my storage.

I know, it's something I'd probably do if I had the energy for it. I just find myself getting into downward-spiralling, vicious cycles with a lot of this tech stuff so it's something I try and avoid going to deep on, for fear of spending a fortnight on something that empties my wallet and yields no results.  

[GuruMeditationError]

Just now, M.Yurizaki said:

Versioning means you can roll back a file to a previous version if you want. The system only stores the changes though so it's not like storing a 12MB file that grows to 13MB means you now have 25MB taken up, it's only going to take up 13MB.

 

But if all you need to do is mirroring, then you don't have to worry about what I mentioned after the application.

Absolutely...just mirroring. It's just for storage drives so not something that's going to be making changes to my OS, so I probably don't really need to version it. I'll take a look at the software you linked to.

[Doramius]

I use Macrium Reflect.  It does support mirroring of Partitions.  You just have to make sure that anything that is in a RAID, is identified as a single drive, first.  It's free, too.

[GuruMeditationError]

6 minutes ago, Doramius said:

I use Macrium Reflect.  It does support mirroring of Partitions.  You just have to make sure that anything that is in a RAID, is identified as a single drive, first.  It's free, too.

Thanks, it looks like that's the kind of thing I'm going to need. 

When you say, make sure it's identified as a single drive, is there a trick to creating a partition so that RAID will see it that way?  When you put it like that it sounds like it might be a potential obstacle?

[Doramius]

It's not really an obstacle.  If you have a RAID, check disk manager and just make sure the RAID lists correctly as a single drive.  If it doesn't, you need to make sure it shows as a single drive.  That's more of an issue regarding the OS or RAID software, and not so much of Macrium Reflect.  Macrium just needs to be able to identify the original drive, and the new drive.  It will copy partition, to partition.  It's really a very simple tool to use, very powerful, and i like the "free" part of it.

[GuruMeditationError]

3 minutes ago, Doramius said:

It's not really an obstacle.  If you have a RAID, check disk manager and just make sure the RAID lists correctly as a single drive.  If it doesn't, you need to make sure it shows as a single drive.  That's more of an issue regarding the OS or RAID software, and not so much of Macrium Reflect.  Macrium just needs to be able to identify the original drive, and the new drive.  It will copy partition, to partition.  It's really a very simple tool to use, very powerful, and i like the "free" part of it.

Okay, thanks but I was hoping to avoid using a single span in favour of partitions on the larger drive, mirrored to each of the smaller physical drives.

I've used very basic home-user RAID software in the past but I don't remember if RAID solutions generally count partitions as separate drives.

[Doramius]

You can go the other direction, too.  Macrium just identifies the data and transfers it.  So if your RAID has the data, and you want to copy it to a non-RAID disk, you can do that too.  Note that the cloned drive, must be the same or larger in size than the partitions being transferred.  This is also not a major issue, as if Microsoft identifies the partition and you haven't used all the space, you can choose to shrink the partition on the origin drive, and then resize the partition larger on the clone drive to utilize the extra unused space. 

overall, my explanation sounds WAY more difficult than it actually is.  the hardest part is waiting for the data to transfer.  But hey, I can be impatient at times.  That's why I like my software solutions to be quick, simple, and get the job done without spending any more time than I have to.

[GuruMeditationError]

9 minutes ago, Doramius said:

You can go the other direction, too.  Macrium just identifies the data and transfers it.  So if your RAID has the data, and you want to copy it to a non-RAID disk, you can do that too.  Note that the cloned drive, must be the same or larger in size than the partitions being transferred.  This is also not a major issue, as if Microsoft identifies the partition and you haven't used all the space, you can choose to shrink the partition on the origin drive, and then resize the partition larger on the clone drive to utilize the extra unused space. 

overall, my explanation sounds WAY more difficult than it actually is.  the hardest part is waiting for the data to transfer.  But hey, I can be impatient at times.  That's why I like my software solutions to be quick, simple, and get the job done without spending any more time than I have to.

But you need an existing RAID? It can't handle the process itself?

If it comes to setting up RAID I'd only be working with Marvel 88SExxxx and I'm not sure it'll support Macrium Reflect?

I'm in the process of installing Macrium Reflect.

I've got an old, failing, 160GB server drive that I can partition and see if it'll show up as separate drives in RAID?

If I can get the partitions to show up in Windows (using Windows to create the partitions) do you know if that would be enough for Macrium Reflect to work effectively? 

[Doramius]

I think you'll be fine.  The wording in your first post may also be throwing me off as to what you're trying to do.  It sounds like you're trying to move 8TB data to 1x4TB and 2x2TB drives.  Or is it the other way around?  Or you just haven't fully filled up the 8TB and you're moving data to the other drives.

 

FYI - Macrium will also just straight copy a drive.  It could be an Apple, a Linux drive.  It doesn't have to be a RAID drive.  Or you can grab a single drive from a RAID that you want to copy, and replace.  Again, it's pretty powerful.

[GuruMeditationError]

5 minutes ago, Doramius said:

I think you'll be fine.  The wording in your first post may also be throwing me off as to what you're trying to do.  It sounds like you're trying to move 8TB data to 1x4TB and 2x2TB drives.  Or is it the other way around?  Or you just haven't fully filled up the 8TB and you're moving data to the other drives.

 

FYI - Macrium will also just straight copy a drive.  It could be an Apple, a Linux drive.  It doesn't have to be a RAID drive.  Or you can grab a single drive from a RAID that you want to copy, and replace.  Again, it's pretty powerful.

I know, I'm not sure it'll do exactly what I'm looking for?

 

I was hoping to mirror the smaller drive's to the partitions as you'd have with a RAID setup, so that both drives are updated simultaneously in real-time. I'm not sure this is going to do it for me, I think I'll need to look at other potential solutions, but thanks.

[GuruMeditationError]

1 hour ago, M.Yurizaki said:

I use FreeFileSync to manage what's changed between my current copy and my back copy. But if you want versioning, you'll need something else, like using Windows' Shadow Copy.

That actually looks pretty good...I think it might be just what I need.

[GuruMeditationError]

1 hour ago, M.Yurizaki said:

I use FreeFileSync to manage what's changed between my current copy and my back copy. But if you want versioning, you'll need something else, like using Windows' Shadow Copy.

lol

 

Upon closer examination I've deselected this as the solution as it has to be the most confusing thing I've ever seen. lol

 

I'm just going to manage this stuff manually.

[GuruMeditationError]

9 hours ago, Ingrid_W said:

If you're talking about creating backup image for the hard drive that may contain bad sectors, it is worth to try AOMEI Backupper. It provides intelligent backup, to some extent, will skip the bad sectors. The same theory also goes to clone hard drive with bad sectors. 

Regards. 

Thanks. I'll probably go that route as it looks like I'm going to be managing this stuff manually.

 

I hadn't considered that the PC would still use the bad sectors  

At this point I'm probably going to just buy a 4TB drive and move the files over to that and have the old 4TB drive as the backup but if that program will let me avoid the bad sectors I think it's definitely worth looking at. Thanks. 

[dalekphalm]

@GuruMeditationError if you want something fairly simple, you may want to look at using RoboCopy and the /mir flag (mirror).

 

You use this either directly in Command Prompt, or you use it in a batch file.

 

For example, create a batch file that has this:

robocopy "E:\" "F:\" /MIR

What this command does, is mirror everything in drive E:\ to drive F:\. Mirroring will skip files that are identical, but will overwrite older versions of the same file, and, crucially important detail, will delete files that no longer exist in the source folder/directory.

 

You can automate this by using Task Scheduler in windows, and set it on a time schedule (Eg: every day at 2 AM). How frequently you set the schedule is really up to you and how often the data changes. If there are big changes every day, I'd only do it once a day. But if you have lots of little changes every day, you may want it to run every hour, for example.

 

I actually use this to backup our WIFI logs at work off of the server that receives the logs, onto a network drive for later use.

 

EDIT: furthermore, you can either create a separate batch file and corresponding task for each drive (and this gives you an opportunity to stagger the mirroring, if desired), or you can include separate robocopy commands for each drive all in one bigger batch file, for simplicity - both are valid choices, just depends on how much granularity you want.