Jump to content

Why all the storage?

mitchpman13

Just got done watching the new $5,500 tape drive video and it hit me. Why does Linus (and other YouTubers) want to store every video they've ever made? I can understand keeping them for maybe a few years or so, but why all of it and spend the cost of the storage servers? To me, it feels like its a tad bit on the hoarding side, but maybe there something I'm not understanding here. Even if I had a large collection of videos I've made and put on Youtube, I personally couldn't see keeping every single one of them stored.

Link to comment
Share on other sites

Link to post
Share on other sites

If Youtube shuts down and they have to start somewhere else?

CPU: i7-2600K 4751MHz 1.44V (software) --> 1.47V at the back of the socket Motherboard: Asrock Z77 Extreme4 (BCLK: 103.3MHz) CPU Cooler: Noctua NH-D15 RAM: Adata XPG 2x8GB DDR3 (XMP: 2133MHz 10-11-11-30 CR2, custom: 2203MHz 10-11-10-26 CR1 tRFC:230 tREFI:14000) GPU: Asus GTX 1070 Dual (Super Jetstream vbios, +70(2025-2088MHz)/+400(8.8Gbps)) SSD: Samsung 840 Pro 256GB (main boot drive), Transcend SSD370 128GB PSU: Seasonic X-660 80+ Gold Case: Antec P110 Silent, 5 intakes 1 exhaust Monitor: AOC G2460PF 1080p 144Hz (150Hz max w/ DP, 121Hz max w/ HDMI) TN panel Keyboard: Logitech G610 Orion (Cherry MX Blue) with SteelSeries Apex M260 keycaps Mouse: BenQ Zowie FK1

 

Model: HP Omen 17 17-an110ca CPU: i7-8750H (0.125V core & cache, 50mV SA undervolt) GPU: GTX 1060 6GB Mobile (+80/+450, 1650MHz~1750MHz 0.78V~0.85V) RAM: 8+8GB DDR4-2400 18-17-17-39 2T Storage: HP EX920 1TB PCIe x4 M.2 SSD + Crucial MX500 1TB 2.5" SATA SSD, 128GB Toshiba PCIe x2 M.2 SSD (KBG30ZMV128G) gone cooking externally, 1TB Seagate 7200RPM 2.5" HDD (ST1000LM049-2GH172) left outside Monitor: 1080p 126Hz IPS G-sync

 

Desktop benching:

Cinebench R15 Single thread:168 Multi-thread: 833 

SuperPi (v1.5 from Techpowerup, PI value output) 16K: 0.100s 1M: 8.255s 32M: 7m 45.93s

Link to comment
Share on other sites

Link to post
Share on other sites

Just now, KyleandThings said:

would they really want all their unboxing videos tho?

They do in fact. I havent seen a content creator that dont havr backups of their own, they at most compress videos to lower resolutions before storing them.

CPU: i7-2600K 4751MHz 1.44V (software) --> 1.47V at the back of the socket Motherboard: Asrock Z77 Extreme4 (BCLK: 103.3MHz) CPU Cooler: Noctua NH-D15 RAM: Adata XPG 2x8GB DDR3 (XMP: 2133MHz 10-11-11-30 CR2, custom: 2203MHz 10-11-10-26 CR1 tRFC:230 tREFI:14000) GPU: Asus GTX 1070 Dual (Super Jetstream vbios, +70(2025-2088MHz)/+400(8.8Gbps)) SSD: Samsung 840 Pro 256GB (main boot drive), Transcend SSD370 128GB PSU: Seasonic X-660 80+ Gold Case: Antec P110 Silent, 5 intakes 1 exhaust Monitor: AOC G2460PF 1080p 144Hz (150Hz max w/ DP, 121Hz max w/ HDMI) TN panel Keyboard: Logitech G610 Orion (Cherry MX Blue) with SteelSeries Apex M260 keycaps Mouse: BenQ Zowie FK1

 

Model: HP Omen 17 17-an110ca CPU: i7-8750H (0.125V core & cache, 50mV SA undervolt) GPU: GTX 1060 6GB Mobile (+80/+450, 1650MHz~1750MHz 0.78V~0.85V) RAM: 8+8GB DDR4-2400 18-17-17-39 2T Storage: HP EX920 1TB PCIe x4 M.2 SSD + Crucial MX500 1TB 2.5" SATA SSD, 128GB Toshiba PCIe x2 M.2 SSD (KBG30ZMV128G) gone cooking externally, 1TB Seagate 7200RPM 2.5" HDD (ST1000LM049-2GH172) left outside Monitor: 1080p 126Hz IPS G-sync

 

Desktop benching:

Cinebench R15 Single thread:168 Multi-thread: 833 

SuperPi (v1.5 from Techpowerup, PI value output) 16K: 0.100s 1M: 8.255s 32M: 7m 45.93s

Link to comment
Share on other sites

Link to post
Share on other sites

Say you own an accounting firm

 

Every client you have, you will want to make sure you keep a copy or reference of the work you have done for each client.

 

I would believe the same idea applies here. Every product or service you release you want to keep track of it. Can come in handy if there are legal or related problems in the future (Obviously theres many more reasons, but I believe this is the biggest contender)

 

Then as @Jurrunio said, when youtube eventually fails, having a backup of those files will help immensely when moving to a different platform 

Community Standards || Tech News Posting Guidelines

---======================================================================---

CPU: R5 3600 || GPU: RTX 3070|| Memory: 32GB @ 3200 || Cooler: Scythe Big Shuriken || PSU: 650W EVGA GM || Case: NR200P

Link to comment
Share on other sites

Link to post
Share on other sites

well go find all your family photos from decades past and throw them away then.

Link to comment
Share on other sites

Link to post
Share on other sites

Well even when they started Vessel, they didn't upload any old videos, just new ones. If youtube were to fail, a lot of videos wouldn't even really be relevant, who would watch benchmarks on a GTX 680?

17 minutes ago, Redstoneary said:

To them I guess they could be memories, like how you might store all your pictures on an external hard drive.

I can understand working hard on videos and them having some sentimental value, but most of them you'd be very rarely likely to go back and watch.

 

11 minutes ago, Slottr said:

Every product or service you release you want to keep track of it. Can come in handy if there are legal or related problems in the future

I could see this being one, I don't fully understand most copyright laws, but it seems too large of a hassle if someone copyrighted a video of mine from say 8 years ago on a video I might hardly even remember.

Link to comment
Share on other sites

Link to post
Share on other sites

you know while were at it lets throw away all the backups we made of the original reels to starwars seeing as how i can't foresee a situation where that will ever be popular again.

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, emosun said:

well go find all your family photos from decades past and throw them away then.

Photos are something you can skim through very easy, 1000's of hours of videos on products that eventually people will hardly remember?

Link to comment
Share on other sites

Link to post
Share on other sites

1 minute ago, mitchpman13 said:

Photos are something you can skim through very easy

unlike a list of files on a computer , they should make some sort of browser for that , for files.

1 minute ago, mitchpman13 said:

1000's of hours of videos on products that eventually people will hardly remember?

and yet i go to a channel like motorweek and their most popular videos are of reviews of products that are from 20-30 years ago.

Link to comment
Share on other sites

Link to post
Share on other sites

5 minutes ago, mitchpman13 said:

-snip-

So if you made something, say a logo, and then 5 years later you see someone else using it, someone you absolutely know you didnt license it to- you wouldnt care at all?

Community Standards || Tech News Posting Guidelines

---======================================================================---

CPU: R5 3600 || GPU: RTX 3070|| Memory: 32GB @ 3200 || Cooler: Scythe Big Shuriken || PSU: 650W EVGA GM || Case: NR200P

Link to comment
Share on other sites

Link to post
Share on other sites

3 minutes ago, Slottr said:

So if you made something, say a logo, and then 5 years later you see someone else using it, someone you absolutely know you didnt license it to- you wouldnt care at all?

A logo would be a little different as it can be an iconic figure, unlike a video. If it were to be my current one, then I would most likely care, but say an old one I hadn't been using for a while and probably wouldn't go back to, probably not.

 

Link to comment
Share on other sites

Link to post
Share on other sites

6 minutes ago, mitchpman13 said:

A logo would be a little different as it can be an iconic figure, unlike a video. If it were to be my current one, then I would most likely care, but say an old one I hadn't been using for a while and probably wouldn't go back to, probably not.

 

You're cool with someone making money off your hard work?

Community Standards || Tech News Posting Guidelines

---======================================================================---

CPU: R5 3600 || GPU: RTX 3070|| Memory: 32GB @ 3200 || Cooler: Scythe Big Shuriken || PSU: 650W EVGA GM || Case: NR200P

Link to comment
Share on other sites

Link to post
Share on other sites

13 minutes ago, mitchpman13 said:

Well even when they started Vessel, they didn't upload any old videos, just new ones. If youtube were to fail, a lot of videos wouldn't even really be relevant, who would watch benchmarks on a GTX 680?

I can understand working hard on videos and them having some sentimental value, but most of them you'd be very rarely likely to go back and watch.

It's not just about rewatching videos. They commonly use clips from old videos in their new videos, so having all the footage is very necessary

Insanity is not the absence of sanity, but the willingness to ignore it for a purpose. Chaos is the result of this choice. I relish in both.

Link to comment
Share on other sites

Link to post
Share on other sites

They don't just store a single version of each video they've uploaded, I think they also store the original quality files which are really, really large. I think it's unnecessary.

Link to comment
Share on other sites

Link to post
Share on other sites

31 minutes ago, mitchpman13 said:

Photos are something you can skim through very easy, 1000's of hours of videos on products that eventually people will hardly remember?

If it was so easy can I hire you to sort out my photo collection? :) At my worst wildlife photography days, I could generate thousands a session. Even playing tourist using a phone, it is trivially easy to hit hundreds a day. I might only post the best of the best, but I want the others too. They may have a use later on. Not to say I never delete anything, but only the worst like misfocused, motion blurred, or subject is badly framed. After that cull there's still a lot left.

 

I can't imagine NOT storing all of this, at original quality. Since LTT moved to the Reds, I hate to imagine the data they're throwing around.

Gaming system: R7 7800X3D, Asus ROG Strix B650E-F Gaming Wifi, Thermalright Phantom Spirit 120 SE ARGB, Corsair Vengeance 2x 32GB 6000C30, RTX 4070, MSI MPG A850G, Fractal Design North, Samsung 990 Pro 2TB, Acer Predator XB241YU 24" 1440p 144Hz G-Sync + HP LP2475w 24" 1200p 60Hz wide gamut
Productivity system: i9-7980XE, Asus X299 TUF mark 2, Noctua D15, 64GB ram (mixed), RTX 3070, NZXT E850, GameMax Abyss, Samsung 980 Pro 2TB, random 1080p + 720p displays.
Gaming laptop: Lenovo Legion 5, 5800H, RTX 3070, Kingston DDR4 3200C22 2x16GB 2Rx8, Kingston Fury Renegade 1TB + Crucial P1 1TB SSD, 165 Hz IPS 1080p G-Sync Compatible

Link to comment
Share on other sites

Link to post
Share on other sites

5 hours ago, emosun said:

you know while were at it lets throw away all the backups we made of the original reels to starwars seeing as how i can't foresee a situation where that will ever be popular again.

They did.  Sadly.  Yay despecialized!

 

I would think they'd need to keep them, with a the embedded links if they ever lose a sponsor or affiliate it would be nice to be able to republish if needed with different information.   Plus, if there were copyright infringement they would need the originals to prove their case.

Link to comment
Share on other sites

Link to post
Share on other sites

Let me tell you all from the first hand regarding the content organization. First of all, what Linus is doing is an super outdated concept where they do everything manually.

Nowadays people use something called media asset management (MAM). So there you basically have all media categorized with corresponding metadata (which can be basically anything). For each media you have HD material (this is not resolution related name, it can be 1080, 4k etc) and proxy (basically lower quality copy of the HD material). Usually the MAM users make rule that eg. after 3 months HD media is automatically moved to tape storage (or cloud based archive like Amazon Glacier)  and removed from the local storage while proxy is still there. Having proxy available all the time means that you can always preview your content from the MAM so if you really need HD material, it can be retrieved from the backup. 

Not to mention that everything described can be automated.

 

So in the case of LMG if they had a MAM, (oversimplified) workflow would be something like this:

1. Editors write the episode and enter it into MAM (who will be the host, script, enter some keywords, fill the rest of metadata etc.)

2. Episode is recorded and raw materials are ingested into system

3. Editors receive a notification that the material for episode is ready

4. Editor(s) responsible for the current episode open Premiere and since all major systems have integration with Premiere, MAM plugin enables editor to edit the content and make an episode

5. When episode editing is done, editor uses Premiere MAM plugin to upload the episode into system

6. (optional) edited episode is automatically checked for errors in video with video QC software

7. The person in charge receives an notification that the episode is ready so they can review it. If episode is ok it goes to next step, otherwise it is returned back to the  editor to fix it.

8. If episode is ok, responsible person clicks that the episode is ok to air so in the background HD material is transcoded into Youtube friendly format and it is uploaded automatically (there are come options like send tweet that new episode is up etc)

9. After x ammount of time HD material is moved from the storage to archive (cloud or tape) and everything else stays in the system

 

Not to mention if you need really specific video that you can search via metadata fields (assuming that you have meaningfully organized metadata) which can speed up process significantly. For example it would take few seconds to find a video eg. where Linus was talking about AMD R290X with Xeon CPU (not sure if that video even exists, but you get my point)

Link to comment
Share on other sites

Link to post
Share on other sites

I mean I keep all videos I've ever made or recorded. I don't keep originals if I've made edited version. Plus I've re-encoded them from .avi and other bad codecs to .mp4 since I don't have money to get 6tb storage drives.

^^^^ That's my post ^^^^
<-- This is me --- That's your scrollbar -->
vvvv Who's there? vvvv

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×