Jump to content

Hitting OneDrive Limitations

JimMcBubbles
 Share
Posted

Im trying to sync unity projects, image libraries, documents, etc across multiple devices and make them available online for sharing etc, but I think I need a better solution. My cache file is over 2GB just from syncing once. It's not even trying anymore. OneDrive has said it's been processing 144 changes for the last 3+ hours. Ive tried paring down on file count, but I'm nowhere near the recommended < 300000. Are there alternatives to OneDrive for syncing large libraries with 2M+ objects and preferably more than 1TB?
image.png.420043fc7fa206c169b5af936c3ccb92.png

Link to comment
https://linustechtips.com/topic/1485872-hitting-onedrive-limitations/
Share on other sites
Link to post
Share on other sites
Posted
1 minute ago, JimMcBubbles said:

I dont mind paying for it.

Sync.com is a great one, depending on plan you'll find something for your needs...  I hate OneDrive personally.

 

https://www.sync.com/pricing/

"Do what makes the experience better" - in regards to PCs and Life itself.

 

Onyx: Ryzen 7 7800X3D / Gigabyte B650 AORUS Pro AX / ASRock Taichi 7900xtx OC / G. Skill Flare X5 6000CL36 64GB (4x16GB) / Samsung 980 1TB x3 / Super Flower Leadex V Plat Pro 1000 / EK-AIO 360 Basic w/ Silent Wings fans / Fractal Design North XL (black mesh) / LG - UltraGear 45" OLED QHD 240Hz / Mackie CR5BT / SteelSeries Arctis Nova Pro / Cherry MX Board 3.0 / Logitech G502 - https://valid.x86.fr/my9nnr

 

7800X3D - PBO +200, CO -30 all cores, 4.90GHz all core, 5.05GHz single core, Cinebench 23: 18401 multi, 1779 single

 

Khaleesi: Ryzen 5 5600X3D (+200, -30) - ASRock B550M Pro4 - G. Skill Ripjaws V 16GB 3200CL16 - Asus Prime 9060XT 16GB - Samsung 980 1TB + Crucial MX500 1TB - Cudy AX3000 PCIe Wifi 6 - EVGA SuperNOVA 650 P2 - Thermalright Frozen Notte RGB 360 White V2 - NZXT H6 Flow RGB White - LG 34" 3440x1440

 

NAS/Plex/Game Server  Ryzen 9 5900XT 16c/32t - Gigabyte B550M AORUS Elite AX - TeamGroup T-Force Vulcan 64GB 3200CL16 - MSI 1050Ti 4GB - Crucial P3 Plus 500GB + TeamGroup MP44L 2TB (Game) + WD Red Plus 4TBx2 (Plex) - TP-Link AC1200 PCIe Wifi - EVGA SuperNOVA 650 P2 - Thermalright Peerless Assassin 120SE - ASUS Prime AP201 - Currently Hosting: Enshrouded x2, Hytale, Icarus, Windrose. Project Zomboid, Dune Awakening.

 

Sage: Ryzen 7 7800X3D (+200, -30) - Gigabyte B650 Gaming X V2 - ASRock Steel Legend 7900GRE - G. Skill Flare X5 32GB 6000CL32 - TeamGroup MP44L 2TB - Super Flower Leadex Platinum SE 1000w - NZXT H5 Elite

 

Emma: i9 9900K @5.2Ghz - Gigabyte Z370 AORUS Gaming 5 - MSI 6900XT Gaming X Trio - G. Skill Ripjaws V 32GB 3200CL16 - 750 EVO 512GB + 2x 860 EVO 1TB (RAID0) - Super Flower Combat FG 850w - Thermaltake Water 3.0 Ultimate 360 - Fractal Design Define R6 - TP-Link AC1900 PCIe Wifi

 

GF Rig: Steam Deck 512GB OLED, Vizio 43" 4K TV

 

Extra parts: ASUS 6650XT - Gigabyte 1080Ti - Cooler Master Q300L - Gigabyte 450w PSU - Super Flower Leadex V Plat Pro 850w

 

OnePlus Ecosystem: 

OnePlus 11 5G - 16GB RAM, 256GB NAND, Eternal Green. OnePlus Watch 2 - Radiant Steel, OnePlus Buds Pro 2 - Eternal Green

3D Printing: 

Bambu Lab X1 Carbon, AMS, AMS2 Pro (thank you MicroCenter!)

Other Interesting Tech:

- 2021 Volvo S60 Recharge T8 PHEV Polestar Engineered - 415hp/495tq 2.0L 4cyl. turbocharged, supercharged and electrified.

Link to post
Share on other sites
Posted
  • Author
52 minutes ago, Dedayog said:

Sync.com is a great one, depending on plan you'll find something for your needs...  I hate OneDrive personally.

 

https://www.sync.com/pricing/

looks like the client has the same limitation

https://www.sync.com/help/general-limits-when-using-sync/

The Sync desktop app and web panel: Sync performance can get slower with more than 300,000 files in Sync. This is not a hard limit, and depends on the hardware specifications of the computer running the app (or web browser), as well as network speed, nested folder structure, and frequency of syncing activity. Many customers successfully sync more files without issue.

Link to post
Share on other sites
Posted
33 minutes ago, JimMcBubbles said:

how many objects are you syncing?

image.png.33a89be724725d2171352b7b15538526.png

 

 

While it may slow down after 300K, that's when trying to sync the upload.  Sure, that takes time to move that.  But that's with any of them.

 

At almost a million files, we've been running smooth for a year now.  We used OneDrive previously and seriously thought about just adding a server to our office, cuz we hated it so much with sync delays and folders never even showing up after creation for others.

"Do what makes the experience better" - in regards to PCs and Life itself.

 

Onyx: Ryzen 7 7800X3D / Gigabyte B650 AORUS Pro AX / ASRock Taichi 7900xtx OC / G. Skill Flare X5 6000CL36 64GB (4x16GB) / Samsung 980 1TB x3 / Super Flower Leadex V Plat Pro 1000 / EK-AIO 360 Basic w/ Silent Wings fans / Fractal Design North XL (black mesh) / LG - UltraGear 45" OLED QHD 240Hz / Mackie CR5BT / SteelSeries Arctis Nova Pro / Cherry MX Board 3.0 / Logitech G502 - https://valid.x86.fr/my9nnr

 

7800X3D - PBO +200, CO -30 all cores, 4.90GHz all core, 5.05GHz single core, Cinebench 23: 18401 multi, 1779 single

 

Khaleesi: Ryzen 5 5600X3D (+200, -30) - ASRock B550M Pro4 - G. Skill Ripjaws V 16GB 3200CL16 - Asus Prime 9060XT 16GB - Samsung 980 1TB + Crucial MX500 1TB - Cudy AX3000 PCIe Wifi 6 - EVGA SuperNOVA 650 P2 - Thermalright Frozen Notte RGB 360 White V2 - NZXT H6 Flow RGB White - LG 34" 3440x1440

 

NAS/Plex/Game Server  Ryzen 9 5900XT 16c/32t - Gigabyte B550M AORUS Elite AX - TeamGroup T-Force Vulcan 64GB 3200CL16 - MSI 1050Ti 4GB - Crucial P3 Plus 500GB + TeamGroup MP44L 2TB (Game) + WD Red Plus 4TBx2 (Plex) - TP-Link AC1200 PCIe Wifi - EVGA SuperNOVA 650 P2 - Thermalright Peerless Assassin 120SE - ASUS Prime AP201 - Currently Hosting: Enshrouded x2, Hytale, Icarus, Windrose. Project Zomboid, Dune Awakening.

 

Sage: Ryzen 7 7800X3D (+200, -30) - Gigabyte B650 Gaming X V2 - ASRock Steel Legend 7900GRE - G. Skill Flare X5 32GB 6000CL32 - TeamGroup MP44L 2TB - Super Flower Leadex Platinum SE 1000w - NZXT H5 Elite

 

Emma: i9 9900K @5.2Ghz - Gigabyte Z370 AORUS Gaming 5 - MSI 6900XT Gaming X Trio - G. Skill Ripjaws V 32GB 3200CL16 - 750 EVO 512GB + 2x 860 EVO 1TB (RAID0) - Super Flower Combat FG 850w - Thermaltake Water 3.0 Ultimate 360 - Fractal Design Define R6 - TP-Link AC1900 PCIe Wifi

 

GF Rig: Steam Deck 512GB OLED, Vizio 43" 4K TV

 

Extra parts: ASUS 6650XT - Gigabyte 1080Ti - Cooler Master Q300L - Gigabyte 450w PSU - Super Flower Leadex V Plat Pro 850w

 

OnePlus Ecosystem: 

OnePlus 11 5G - 16GB RAM, 256GB NAND, Eternal Green. OnePlus Watch 2 - Radiant Steel, OnePlus Buds Pro 2 - Eternal Green

3D Printing: 

Bambu Lab X1 Carbon, AMS, AMS2 Pro (thank you MicroCenter!)

Other Interesting Tech:

- 2021 Volvo S60 Recharge T8 PHEV Polestar Engineered - 415hp/495tq 2.0L 4cyl. turbocharged, supercharged and electrified.

Link to post
Share on other sites
Posted
  • Author
5 hours ago, Dedayog said:

image.png.33a89be724725d2171352b7b15538526.png

 

 

While it may slow down after 300K, that's when trying to sync the upload.  Sure, that takes time to move that.  But that's with any of them.

 

At almost a million files, we've been running smooth for a year now.  We used OneDrive previously and seriously thought about just adding a server to our office, cuz we hated it so much with sync delays and folders never even showing up after creation for others.

Thank you for taking the time to check! I might just have to make the switch! thanks!

Link to post
Share on other sites
Posted
  • Author
On 2/3/2023 at 1:00 PM, JimMcBubbles said:

Im trying to sync unity projects, image libraries, documents, etc across multiple devices and make them available online for sharing etc, but I think I need a better solution. My cache file is over 2GB just from syncing once. It's not even trying anymore. OneDrive has said it's been processing 144 changes for the last 3+ hours. Ive tried paring down on file count, but I'm nowhere near the recommended < 300000. Are there alternatives to OneDrive for syncing large libraries with 2M+ objects and preferably more than 1TB?
image.png.420043fc7fa206c169b5af936c3ccb92.png

Think ive found a workaround. I've disabled Files On-Demand and OneDrive immediately started syncing again... 30-40Mb/s transfers. Got thru 200GB in like 6 hours

Link to post
Share on other sites
Posted

I was looking at more than 1TB for OneDrive.  They offer "unlimited" (or is it 5TB?) for $10/mo extra BUT you have to have more than 5 accounts, or you get the same limit.  So why pay the extra $10?  I'm only using 139GB but looking for additional options since my internet is now unlimited.

AMD 7950x3D / Gigabyte Aurous Master X670E/ 64GB @ 6000c30 / 3 x 4TB Samsung 990 Pro / 44TB Synology 1522+ / MSI Gaming Trio 4090 / EVGA G6 1000w /Thermaltake View71 / LG C1 48in OLED + MSI 321URX - Moved back to air cooling Phantom Spirit 120 SE.  Server (PLEX) - 155H NUC 64GB  and 60GB Optane drive/ Server (AI) 64GB M4 Max Mac Studio

Link to post
Share on other sites
Posted

Cloud storage is fine, but I always recommend a NAS. You pay once for the hardware and just maintain it yourself. There are several options of course but I use a WD4100EX and I have had it for roughly 10 years. Time I get new drives, at the moment I have 2x 4TB drives in JBOD mode but I am considering changing that to 4 8TB in JBOD. I considered do RAID but JBOD works pretty well for what I have. 

 

I do use Onedrive since it come with my 365 account, but I don't trust it with much, I tend only to use for files I use that I need available for quick access with my phone otherwise I tend to use my NAS for everything else, backups major files and things I need to stay secure like personal taxes, files and such I can afford to lose for any reason.

COMMUNITY STANDARDS   |   TECH NEWS POSTING GUIDELINES   |   FORUM STAFF

LTT Folding Users Tips, Tricks and FAQ   |   F@H & BOINC Badge Request   |   F@H Contribution   |   My Rig   |   Project Steamroller

I am a Moderator, but I am fallible. Discuss or debate with me as you will but please do not argue with me as that will get us nowhere.

 

Spoiler

Character is like a Tree and Reputation like its Shadow. The Shadow is what we think of it; The Tree is the Real thing.  ~ Abraham Lincoln

You have enemies? Good. That means you've stood up for something, sometime in your life.  ~ Winston Churchill

Reputation is a Lifetime to create but takes only seconds to destroy.

Docendo discimus - "to teach is to learn"

 

† † †

† CHRISTIAN MEMBER †

 
 
 
 
 
 

 

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing Programs, Apps and Websites

×