Homebrew Multi-site Cloud Storage | Any Alternatives to Resilio Sync?

[Aachor]

Hey all, this may be a bit esoteric, and possibly in the wrong forum, but I wanted to see if any of you all have input anyways...

So, I help run a small video-production company. We have small offices in two cities about 350 miles apart. In each office, we have an identical 100TB storage server (running in RAID6) currently storing about 40TB+. Both are running Ubuntu Server. Both our teams share data, so we keep the servers synchronized one with the other using Resilio Sync. This allows (sometimes with a day of lag time) both teams to have access to the same data. It also gives us a live off-site backup.

Does anyone know of any alternative to Resilio Sync that could be practical for our situation? We really rely on Sync's ability to not over-write newer versions of files with older versions, so that we can update files without having to replace them. But there are a number of things about its implementation that feel clumsy to me (e.g. overwriting file ownership and permissions). I'd love to test an alternative if anyone has a suggestion.

[Electronics Wizardy]

Do you have a VPN between the locations? 

 

What is the network speed?

 

Id probalby just run rsync as a cron job.

[brwainer]

SyncThing is essentially the open source version of Resilio Sync.

[Aachor]

12 hours ago, brwainer said:

SyncThing is essentially the open source version of Resilio Sync.

Thank you! I was not aware this existed. I'll check it out.

[Aachor]

17 hours ago, Electronics Wizardy said:

Do you have a VPN between the locations? 

 

What is the network speed?

 

Id probalby just run rsync as a cron job.

Sadly, the network speed is capped to about 40mbps up because we're cheap. This means that large transfers can take days, which itself has not been a problem. We had initially considered rsync, but our issue is that it's likely that with whatever schedule we'd set for it, we'd run into situations where cron would be initiating a new instance before a previous had completed its transfer.

[Electronics Wizardy]

3 hours ago, Aachor said:

Sadly, the network speed is capped to about 40mbps up because we're cheap. This means that large transfers can take days, which itself has not been a problem. We had initially considered rsync, but our issue is that it's likely that with whatever schedule we'd set for it, we'd run into situations where cron would be initiating a new instance before a previous had completed its transfer.

You can pretty easily make the sync script wait till the rsync task is done before starting a new one. Id just run the script at boot, then run rsync over and over again.