Jump to content

How do you backup petabyte project?

That Linux guy
By That Linux guy
in General Discussion
 Share
Posted
9 minutes ago, That Linux guy said:

How do you backup petabyte project just wondering.

Not sure if this is what they still use today, but
I Hope Google Doesn’t Ban Us... - Abusing Unlimited Google Drive - YouTube
They might also have two servers with one serving as a backup, I'm unsure about that. 

I am NOT a professional and a lot of the time what I'm saying is based on limited knowledge and experience. I'm going to be incorrect at times. 

Motherboard Tier List                   How many watts do I need?
Best B550 Motherboards             Best Intel Z490 Motherboards

PC Troubleshooting                      You don't need a big PSU

PSU Tier List                                Common pc building mistakes 
PC BUILD Guide! (POV)              How to Overclock your CPU 

 

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

Simple, by having 2 other petabyte projects. One on site and one off site for backup redundancy. 

CPU: AMD Ryzen 3700x / GPU: Asus Radeon RX 6750XT OC 12GB / RAM: Corsair Vengeance LPX 2x8GB DDR4-3200
MOBO: MSI B450m Gaming Plus / NVME: Corsair MP510 240GB / Case: TT Core v21 / PSU: Seasonic 750W / OS: Win 10 Pro

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
5 minutes ago, Brok3n But who cares? said:

Not sure if this is what they still use today, but
I Hope Google Doesn’t Ban Us... - Abusing Unlimited Google Drive - YouTube

Last I heard Google caught on and bottle-necked their upload to where it stopped being worth it.

 

I believe they moved to tape storage. They haven't gone into much detail on it but they demonstrated a tape reel and thunderbolt reader to write the project files on.

 

Not the best solution but by far the cheapest given the amount of data.

Guides & Tutorials:

PROXMOX - Rebuilding ZFS RAID rpool After Disk Failure

Mass Deploying Customized Windows 10/11 Installs

Building a GNU/Linux Based Windows Deployment Server

GNU/Linux Installer Server: Installation & Configuration

How to: Use (i)PXE to Install Windows from a Network

Why Memorize IP's When You Can Self-Host DNS Instead?

Ventoy - The USB Multi-Boot Utility!

Introduction to PXE/iPXE Network Boot Featuring FreeBSD & Ubuntu Server

 

Don't see what you need? Check the Full List or *PM me, if I haven't made it I'll add it to the list.

*NOTE: I'll only add it to the list if the request is something I know I can do.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

You get another petabyte storage (or two or three), store it in different location / town.

So incase of outage or disaster you still have a backup.

Ryzen 5700g @ 4.4ghz all cores | Asrock B550M Steel Legend | 3060 | 2x 16gb Micron E 2666 @ 4200mhz cl16 | 500gb WD SN750 | 12 TB HDD | Deepcool Gammax 400 w/ 2 delta 4000rpm push pull | Antec Neo Eco Zen 500w

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
20 minutes ago, Windows7ge said:

Last I heard Google caught on and bottle-necked their upload to where it stopped being worth it.

 

I believe they moved to tape storage. They haven't gone into much detail on it but they demonstrated a tape reel and thunderbolt reader to write the project files on.

 

Not the best solution but by far the cheapest given the amount of data.

Kind of, there’s a limit of 750GB uploaded to it per day.

 

There is talk about unlimited storage moving to an “enterprise” account. That’s what I am using atm and it’s £15ish.

 

I’m currently using around 50TB with another 50ish to upload.

If you look on Reddit Datahoarders there’s lots of people with 500TB+ on their Googledrive 

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
33 minutes ago, That Linux guy said:

How do you backup petabyte project just wondering.

tape for long term

Good luck, Have fun, Build PC, and have a last gen console for use once a year. I should answer most of the time between 9 to 3 PST

NightHawk 3.0: R7 5700x @, B550A vision D, H105, 2x32gb Oloy 3600, Sapphire RX 6700XT  Nitro+, Corsair RM750X, 500 gb 850 evo, 2tb rocket and 5tb Toshiba x300, 2x 6TB WD Black W10 all in a 750D airflow.
GF PC: (nighthawk 2.0): R7 2700x, B450m vision D, 4x8gb Geli 2933, Strix GTX970, CX650M RGB, Obsidian 350D

Skunkworks: R5 3500U, 16gb, 500gb Adata XPG 6000 lite, Vega 8. HP probook G455R G6 Ubuntu 20. LTS

Condor (MC server): 6600K, z170m plus, 16gb corsair vengeance LPX, samsung 750 evo, EVGA BR 450.

Spirt  (NAS) ASUS Z9PR-D12, 2x E5 2620V2, 8x4gb, 24 3tb HDD. F80 800gb cache, trueNAS, 2x12disk raid Z3 stripped

PSU Tier List      Motherboard Tier List     SSD Tier List     How to get PC parts cheap    HP probook 445R G6 review

 

"Stupidity is like trying to find a limit of a constant. You are never truly smart in something, just less stupid."

Camera Gear: X-S10, 16-80 F4, 60D, 24-105 F4, 50mm F1.4, Helios44-m, 2 Cos-11D lavs

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
35 minutes ago, Brok3n But who cares? said:

Not sure if this is what they still use today, but
I Hope Google Doesn’t Ban Us... - Abusing Unlimited Google Drive - YouTube
They might also have two servers with one serving as a backup, I'm unsure about that. 

 

30 minutes ago, Windows7ge said:

Last I heard Google caught on and bottle-necked their upload to where it stopped being worth it.

 

I believe they moved to tape storage. They haven't gone into much detail on it but they demonstrated a tape reel and thunderbolt reader to write the project files on.

 

Not the best solution but by far the cheapest given the amount of data.

 

7 minutes ago, oJIMoBOBo said:

Kind of, there’s a limit of 750GB uploaded to it per day.

 

There is talk about unlimited storage moving to an “enterprise” account. That’s what I am using atm and it’s £15ish.

 

I’m currently using around 50TB with another 50ish to upload.

If you look on Reddit Datahoarders there’s lots of people with 500TB+ on their Googledrive 

 

Yeah google has stopped allowing people to have unlimited storage on their business accounts. Now you can only have a maximum of 5TB on the highest level of the storage you can get, to get more storage you have contact them.

 

2120868735_Screenshotfrom2021-02-2113-01-11.thumb.png.0320c1c59a795dce4009ba7d5318f632.png

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

They use tapes

 

If you want my attention, quote meh! D: or just stick an @samcool55 in your post :3

Spying on everyone to fight against terrorism is like shooting a mosquito with a cannon

Link to comment
Share on other sites
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing General Discussion

×