Hitting OneDrive Limitations

[JimMcBubbles]

Im trying to sync unity projects, image libraries, documents, etc across multiple devices and make them available online for sharing etc, but I think I need a better solution. My cache file is over 2GB just from syncing once. It's not even trying anymore. OneDrive has said it's been processing 144 changes for the last 3+ hours. Ive tried paring down on file count, but I'm nowhere near the recommended < 300000. Are there alternatives to OneDrive for syncing large libraries with 2M+ objects and preferably more than 1TB?

[Radium_Angel]

2 minutes ago, JimMcBubbles said:

preferably more than 1TB

Not for free as far as I know...

[JimMcBubbles]

22 minutes ago, Radium_Angel said:

Not for free as far as I know...

I dont mind paying for it.

[Dedayog]

1 minute ago, JimMcBubbles said:

I dont mind paying for it.

Sync.com is a great one, depending on plan you'll find something for your needs...  I hate OneDrive personally.

 

https://www.sync.com/pricing/

[JimMcBubbles]

52 minutes ago, Dedayog said:

Sync.com is a great one, depending on plan you'll find something for your needs...  I hate OneDrive personally.

 

https://www.sync.com/pricing/

looks like the client has the same limitation

https://www.sync.com/help/general-limits-when-using-sync/

The Sync desktop app and web panel: Sync performance can get slower with more than 300,000 files in Sync. This is not a hard limit, and depends on the hardware specifications of the computer running the app (or web browser), as well as network speed, nested folder structure, and frequency of syncing activity. Many customers successfully sync more files without issue.

[JimMcBubbles]

53 minutes ago, Dedayog said:

Sync.com is a great one, depending on plan you'll find something for your needs...  I hate OneDrive personally.

 

https://www.sync.com/pricing/

how many objects are you syncing?

[Dedayog]

33 minutes ago, JimMcBubbles said:

how many objects are you syncing?

 

 

While it may slow down after 300K, that's when trying to sync the upload.  Sure, that takes time to move that.  But that's with any of them.

 

At almost a million files, we've been running smooth for a year now.  We used OneDrive previously and seriously thought about just adding a server to our office, cuz we hated it so much with sync delays and folders never even showing up after creation for others.

[JimMcBubbles]

5 hours ago, Dedayog said:

 

 

While it may slow down after 300K, that's when trying to sync the upload.  Sure, that takes time to move that.  But that's with any of them.

 

At almost a million files, we've been running smooth for a year now.  We used OneDrive previously and seriously thought about just adding a server to our office, cuz we hated it so much with sync delays and folders never even showing up after creation for others.

Thank you for taking the time to check! I might just have to make the switch! thanks!

[JimMcBubbles]

On 2/3/2023 at 1:00 PM, JimMcBubbles said:

Im trying to sync unity projects, image libraries, documents, etc across multiple devices and make them available online for sharing etc, but I think I need a better solution. My cache file is over 2GB just from syncing once. It's not even trying anymore. OneDrive has said it's been processing 144 changes for the last 3+ hours. Ive tried paring down on file count, but I'm nowhere near the recommended < 300000. Are there alternatives to OneDrive for syncing large libraries with 2M+ objects and preferably more than 1TB?

Think ive found a workaround. I've disabled Files On-Demand and OneDrive immediately started syncing again... 30-40Mb/s transfers. Got thru 200GB in like 6 hours

[ewitte]

I was looking at more than 1TB for OneDrive.  They offer "unlimited" (or is it 5TB?) for $10/mo extra BUT you have to have more than 5 accounts, or you get the same limit.  So why pay the extra $10?  I'm only using 139GB but looking for additional options since my internet is now unlimited.

[SansVarnic]

Cloud storage is fine, but I always recommend a NAS. You pay once for the hardware and just maintain it yourself. There are several options of course but I use a WD4100EX and I have had it for roughly 10 years. Time I get new drives, at the moment I have 2x 4TB drives in JBOD mode but I am considering changing that to 4 8TB in JBOD. I considered do RAID but JBOD works pretty well for what I have. 

 

I do use Onedrive since it come with my 365 account, but I don't trust it with much, I tend only to use for files I use that I need available for quick access with my phone otherwise I tend to use my NAS for everything else, backups major files and things I need to stay secure like personal taxes, files and such I can afford to lose for any reason.