Best backup software/method

[Keudn]

Hi all,

 

I am looking for some recommendations for backup software and methods to my NAS. I need to backup both my photography drive and my C: drive, which is 3.5TB in total. I have a Drobo NAS, and I would like for it to act identically to a cloud backup solution like onedrive where I would do an initial full backup and then only update changed or new/deleted files. This is because most of my data is going to be ingested in batches and then will sit there unaltered after I am done with my work on that project, so it doesn't make sense for me to overwrite that data every x days with another full backup. Some research has led me to incremental backups, but that isn't exactly what I want as those seem to be typically configured to run in-between occasional scheduled full backups. Ideally I would never run a full backup after the first initial one. I don't keep my PC running overnight, and I don't want performance hindered by huge full backups every week/month that takes hours and hours. Any suggestions?

 

[Electronics Wizardy]

How about veeam? They have a free version for workstations and it does those baackups very well.  It does the incremant backups well too.

[Keudn]

I actually just uninstalled Veeam as it was taking 24+ hours to do a backup and would constantly say my NAS wasn't connected

[Electronics Wizardy]

3 minutes ago, Keudn said:

I actually just uninstalled Veeam as it was taking 24+ hours to do a backup and would constantly say my NAS wasn't connected

What was it doing?  Veeam should be able to do this well. Id give veeam anouther shot here.

 

Do you want a image backups? Should it be a bootable backup?

 

You can use something like robocopy + snapshots on the nas if you don't care about the image.

 

 

[Keudn]

Just now, Electronics Wizardy said:

What was it doing?  Veeam should be able to do this well. Id give veeam anouther shot here.

 

Do you want a image backups? Should it be a bootable backup?

 

You can use something like robocopy + snapshots on the nas if you don't care about the image.

 

 

I don't remember the exact message but it was completing the actual file transferring and then spending 10s of hours verifying/processing. For the photo drive I don't really care how its backed up, I'll be doing the whole drive but it doesn't have to be an image. For the C: drive I want an exact image of the whole drive that I can drop onto a new SSD and be back up and running again.

[tikker]

34 minutes ago, Keudn said:

Ideally I would never run a full backup after the first initial one. I don't keep my PC running overnight, and I don't want performance hindered by huge full backups every week/month that takes hours and hours. Any suggestions?

Avoiding that is the point of incremental backups. Only the differences will be stored. That does need a full backup as a reference point, but as long as that reference exists you can do incremental. The reason why you'd want to do a fresh full backup every now and then, I'd say, is to avoid the incremental "chain" becoming too long. The longer it gets, the more vulnerable you become to losing part of that chain.

 

How fast is the connection to your NAS? At e.g ~100 MB/s write speed for a HDD, 3.5 TB would take about 10-12 hours to copy. If there's substantial overhead of many and smaller files where you never truly reach that speed it'll easily take longer. It's simply not at trivial amount to transfer over home stuff.

 

At similar read speeds verifying would take a similar time as it needs to read everything back and check it against the original.

[Electronics Wizardy]

1 minute ago, Keudn said:

I don't remember the exact message but it was completing the actual file transferring and then spending 10s of hours verifying/processing. For the photo drive I don't really care how its backed up, I'll be doing the whole drive but it doesn't have to be an image. For the C: drive I want an exact image of the whole drive that I can drop onto a new SSD and be back up and running again.

You can change how often it verifys, or turn that off in veam, but its probably a good idea to let that run so it so there isn't any backup corruption.

 

Id just keep using veeam here. 

 

For the network connections, try using the full unc path instead of the drive letter.

[Keudn]

3 minutes ago, tikker said:

Avoiding that is the point of incremental backups. Only the differences will be stored. That does need a full backup as a reference point, but as long as that reference exists you can do incremental. The reason why you'd want to do a fresh full backup every now and then, I'd say, is to avoid the incremental "chain" becoming too long. The longer it gets, the more vulnerable you become to losing part of that chain.

 

How fast is the connection to your NAS? At e.g ~100 MB/s write speed for a HDD, 3.5 TB would take about 10-12 hours to copy. If there's substantial overhead of many and smaller files where you never truly reach that speed it'll easily take longer. It's simply not at trivial amount to transfer over home stuff.

 

At similar read speeds verifying would take a similar time as it needs to read everything back and check it against the original.

Yeah that was the problem, every daily incremental backup would takes HOURS verifying even though it only changed a few 10s of GB of files. Both NAS and my PC are connected directly to my router so I get the full gigabit connection, I'm limited by my drive speeds in the NAS.

 

1 minute ago, Electronics Wizardy said:

You can change how often it verifys, or turn that off in veam, but its probably a good idea to let that run so it so there isn't any backup corruption.

 

Id just keep using veeam here. 

 

For the network connections, try using the full unc path instead of the drive letter.

Veeam as it was setup was unusable, I don't leave my machine running for 12 hours a day and even if I do it causes significant hitches in games, I have to completely close out of the application from task manager, even if its not actively backing up or verifying. Because of that I almost never finished an incremental backup even ones of only a few 10s of GB which isn't acceptable imo.

[Electronics Wizardy]

1 minute ago, Keudn said:

Veeam as it was setup was unusable, I don't leave my machine running for 12 hours a day and even if I do it causes significant hitches in games, I have to completely close out of the application from task manager, even if its not actively backing up or verifying. Because of that I almost never finished an incremental backup even ones of only a few 10s of GB which isn't acceptable imo.

That seems like a weird issue, Id try reinstalling. Never seen that issue before.

 

I normally do 10-50GB incrementals in like 10 minuts with no performance loss. 

 

Can you show your settings? 

 

What was cpu and disk usage like?

[tikker]

4 minutes ago, Keudn said:

Yeah that was the problem, every daily incremental backup would takes HOURS verifying even though it only changed a few 10s of GB of files. Both NAS and my PC are connected directly to my router so I get the full gigabit connection, I'm limited by my drive speeds in the NAS.

Yeah I just chimed in to say those timescales sound normal to me. I have never used Veeam, but it sounds like you can simply change how often it verifies and make it so it doesn't after every single backup. I'll leave it to the expert from here

[Keudn]

1 minute ago, Electronics Wizardy said:

That seems like a weird issue, Id try reinstalling. Never seen that issue before.

 

I normally do 10-50GB incrementals in like 10 minuts with no performance loss. 

 

Can you show your settings? 

 

What was cpu and disk usage like?

I can redownload and reconfigure it, will it pick up the current backup or do I need to wipe my NAS and do a full one again

[Electronics Wizardy]

Just now, Keudn said:

I can redownload and reconfigure it, will it pick up the current backup or do I need to wipe my NAS and do a full one again

Id setup a new backup chain, then you can change settings.

[Keudn]

Alright, just installed Veeam Agent and it remembered everything. I checked a recently canceled backup, it backed up 500MB. After that it read 504GB from my C: drive at 24MB/s and took 6 hours, and then spent 8.5 hours "finalizing" before I eventually canceled it.

[Electronics Wizardy]

20 minutes ago, Keudn said:

Alright, just installed Veeam Agent and it remembered everything. I checked a recently canceled backup, it backed up 500MB. After that it read 504GB from my C: drive at 24MB/s and took 6 hours, and then spent 8.5 hours "finalizing" before I eventually canceled it.

What compression settings were you using? What cpu usage did you see?

 

What happens if you let it run?

[Keudn]

1 hour ago, Electronics Wizardy said:

What compression settings were you using? What cpu usage did you see?

 

What happens if you let it run?

Optimal compression and the CPU usage isn't very high, my info is outdated I have a 3800x now. If I let it run it eventually finishes, it just takes an obscenely long time to "finalize"