Petabyte Project is FULL! Time to upgrade...

[Zando_]

Just now, flibberdipper said:

And you can't even handle a business trying to support themselves with their own merch for a little bit each video, which is just incredible. God forbid a company tries to pay the bills and the employees with something of theirs.

I totally don't lowkey want a swacket now

[flibberdipper]

Just now, Zando Bob said:

I totally don't lowkey want a swacket now

No lie a handful of the t-shirts get my attention. I like graphic tees, and theirs are kinda nerdy and look generally neat while not being up in your face about you being some GaYmEr NeRd.

[Zando_]

1 minute ago, flibberdipper said:

No lie a handful of the t-shirts get my attention. I like graphic tees, and theirs are kinda nerdy and look generally neat while not being up in your face about you being some GaYmEr NeRd.

Yee, from what I've seen they're a nice clean techie look, which I like a lot. 

[dalekphalm]

FYI those Seagate IronWolf Pro 12TB drives are in the $550-$570 CAD ballpark - buying all 15 would cost around $8500 CAD.

 

Awesome that Seagate donated the drives, but for infrastructure like that, $8500 is a drop in the bucket for expanding after 2 years.

[DildorTheDecent]

The dig at Samsung lmao

[Lathlaer]

1 hour ago, flibberdipper said:

So basically you block people because they counter your opinion with their own? What kind of assbackwards logic is that?

Well if you notice his post history he has this little crusade against LTT's merch ads to the point where you can find his posts under most recent video topics. It's quite amusing actually, I've come to expect his posts just as I expect LTT pushing their merch

 

I guess he got second wind since he was done in late August

 

On 8/27/2019 at 1:10 AM, Daniel644 said:

but again with the merch ad in the middle of the video so @James @CPotter @nicklmg and all the other LMG staff who's tags I don't feel like figuring out you have lost a viewer over your overkill merch shilling, I'm done, can't do it anymore. the ADS (NOT sponsor spots no matter what you call them they are ads) at the beginning and end where bad enough, now you are just trolling your viewer base with these merch ads in the middle of the video and those of us that have been around longer then literally HALF the current LMG staff are tired of this. #Unsubscribed #CantDoItAnymore

 

[Guest]

1 hour ago, dalekphalm said:FYI those Seagate IronWolf Pro 12TB drives are in the $550-$570 CAD ballpark - buying all 15 would cost around $8500 CAD.

 

Awesome that Seagate donated the drives, but for infrastructure like that, $8500 is a drop in the bucket for expanding after 2 years.

Understatement. The cost of for instance a suitable Fujitsu Eternus DX, well two systems in separate sites for redundancy would bring a tear to the eye of any small businesses accountant. Just the licences for a couple of brocade switches would cost more than that.

[Guest]

3 hours ago, SenKa said:

They explored the idea of tape drives in a video. I believe Linus resolved that it was a good idea but would take forever due to the sheer volume of the data.

While it may sound like that, if you used them in raw mode that is 12tb per tape. They theoretically work at 2.7tb per hour but In practice more like 1.6tb from experience. So given their current load that is about 30days to backup everything, less if they use two drives in a small library. Not too bad if you ask me.

[BasicallyAMod]

4 hours ago, Phill104 said:

Your lack of backup must be a little worrying. You should honestly consider an LTO-8 tape library. While tapes sound like a retro step, with LTO-8 now at 30TB it works out quite cheap. The bonus is you can take backups and store them off-site covering you in event of fire, flood or theft.

a while ago they were working on finding a backup. they had a video on how to cheat google drive storage, and i think they mentioned looking into tapes

 

[Video Beagle]

I thought they HAD gone with that tape drive..maybe not.

 

More and more I wish there was some new storage system for mass data amounts...babylon 5's data crystals would be good. 

 

It wasn't bad in this one...or a previous Anthony vid, because he sorta gives a reaction to Linus's shilling about how subtle it is (in the previous one, he literally sarcastly says "subtle"), but the merch shilling HAS gotten a bit...intrusive lately.  Maybe it was always that way and I just notice it more because it was joked about a lot in the roast, or maybe it's more heavy now.

[Snadzies]

Don't archive 4-8k footage.

Save that for a few months then make compressed 1080p video for archive.

[dalekphalm]

47 minutes ago, Snadzies said:

Don't archive 4-8k footage.

Save that for a few months then make compressed 1080p video for archive.

I think the intention with archiving full quality is that they can go back afterwards and re-use footage for new projects - not that LMG frequently does that.

 

It's the same with film production companies. They are supposed to archive every single reel of film (or byte of data) used, so that - for example - an HD/4K remaster can be done 20 years after the movie was released.

 

I would definitely agree with compressing the 8K footage down to 4K at least, though, after some amount of time.

 

The problem is, how do they automate that without it becoming someone's full time job just to compress and archive footage? I'm sure it can be done, but it's another pain in the ass.

 

My money would be on a tape library system. The biggest problem would be getting caught up from the initial backlog of backing up everything to tape. Once they're caught up, it shouldn't be a problem, and can be largely automated.

[mariushm]

With tape drives, i'd be concerned about advances in technology.

Every few years there's some new models out there, with tapes that are no longer compatible and so on... or you may no longer be able to get the tapes for the proprietary tape drive you want.

If they go with tape backup, let's say you don't go the latest generation tape drives, the highest end ones, simply because they're too expensive... a few years from now, you may not be able to buy tapes for your generation's drive. Or, you may try to recover some data from some backup tapes and the tape drive fails ... so you start looking for replacement drive and they're no longer sold, they're discontinued... 

So you may have to go have hot spares around or keep on tape drive as backup sealed in box etc in controlled environment and so on...

 

We have advanced in compression technology as well... he had mpeg2, h264 , hevc/h265, vp9, now they're working on av1 and hevc replacement codecs....

You could compress the 8K footage that's older than let's say 1 year to some high bitrate "nearly lossless" hevc/h264 (i'm sure they already do it, doubt footage stays in cineform or whatever it's used) and maybe recompress to newer standards for better compression (while using the same "near lossless" compression profile so it's not a "generation loss" / save jpg to jpg issue).

 

 

[dalekphalm]

6 minutes ago, mariushm said:

With tape drives, i'd be concerned about advances in technology.

Every few years there's some new models out there, with tapes that are no longer compatible and so on... or you may no longer be able to get the tapes for the proprietary tape drive you want.

How about don't use proprietary tapes? Just use standard LTO tapes. You can still buy LTO-3 tapes easily that came out in 2005 - and frankly you can probably find older tapes if you look at specialized suppliers.

 

Furthermore, the Drives themselves can read tapes from the previous two generations. So if LMG goes out and buys an LTO-8 set, they can read LTO-6 and LTO-7 tapes. And if they upgrade to LTO-10, they can still read their older LTO-8 tapes.

6 minutes ago, mariushm said:

If they go with tape backup, let's say you don't go the latest generation tape drives, the highest end ones, simply because they're too expensive... a few years from now, you may not be able to buy tapes for your generation's drive. Or, you may try to recover some data from some backup tapes and the tape drive fails ... so you start looking for replacement drive and they're no longer sold, they're discontinued... 

That's not a huge concern, as I've listed above. I can find, in stock (as in, ships today) LTO-3 tapes.

 

And while finding older drives eventually might be a problem, you can find tons of working used ones on eBay (though you may pay through the nose, depending on the demand at any given moment).

6 minutes ago, mariushm said:

So you may have to go have hot spares around or keep on tape drive as backup sealed in box etc in controlled environment and so on...

Keeping a spare drive is a good idea - though there's essentially zero need to keep extra tapes (aside from your standard keep one or two cold spares in case of unexpected failures).

6 minutes ago, mariushm said:

We have advanced in compression technology as well... he had mpeg2, h264 , hevc/h265, vp9, now they're working on av1 and hevc replacement codecs....

You could compress the 8K footage that's older than let's say 1 year to some high bitrate "nearly lossless" hevc/h264 (i'm sure they already do it, doubt footage stays in cineform or whatever it's used) and maybe recompress to newer standards for better compression (while using the same "near lossless" compression profile so it's not a "generation loss" / save jpg to jpg issue).

That is a good - but potentially intensive, suggestion. They'd have to figure out a way to automate it (or near automate it).

[Lathlaer]

1 hour ago, dalekphalm said:

They are supposed to archive every single reel of film (or byte of data) used, so that - for example - an HD/4K remaster can be done 20 years after the movie was released.

Incoming:

 

"Scrapyard Wars Remastered: Enhanced Edition" 8K + never seen before bonus scenes!

[dalekphalm]

Just now, Lathlaer said:

Incoming:

 

"Scrapyard Wars Remastered: Enhanced Edition" 8K + never seen before bonus scenes!

You say that as a joke, but I guarantee you there are LMG fans that would eat that shit up and love it.

 

Outtakes, deleted interviews, etc.

[Lathlaer]

2 minutes ago, dalekphalm said:

You say that as a joke, but I guarantee you there are LMG fans that would eat that shit up and love it.

 

Outtakes, deleted interviews, etc.

Well I wouldn't mind seeing some deleted things as well but the quality upgrade not so much.

 

And they probably haven't been running with RED on the streets anyway (definitely not the first seasons before they had the cameras but I don't think they used RED even when they had it).

[dalekphalm]

Just now, Lathlaer said:

Well I wouldn't mind seeing some deleted things as well but the quality upgrade not so much.

 

And they probably haven't been running with RED on the streets anyway (definitely not the first seasons before they had the cameras but I don't think they used RED even when they had it).

For sure, unlike in Cinema, where if you filmed on 35mm 40 years ago, you can still come back and do an 8K or 16K scan of the reel, they're not going to magically get a massive quality boost from old digital footage (At least, nothing crazy - at best they could release higher bitrate footage... which they already do on Floatplane - YouTube's shit bitrates are likely a huge limiting factor).

[fulminemizzega]

17 hours ago, mynameisjuan said:

Drive isnt working, *hacker scene*, fixed.

 

So just going to blue ball us like that and not tell us why 117 was not working.

 

It's as bad as people who create a post asking for help then responding with, nvm fixed it

Come on! What was the fix? Even if it was something stupid. It can not be just don't know, typed buch of stuff, somehow solved it, and everything is fine. Some of the audience is brain dead (btw why don't you just remove the front door, turn off the AC so you don't need the swacket and backup to AWS no even better GDrive and buy a jellything that just works by magic), most is not (I hope, you tell me...).

Did I forget tapes?

 

----

Besides, when did you upgrade the water cooled server? Is the video already up?

[dalekphalm]

2 minutes ago, fulminemizzega said:

Come on! What was the fix? Even if it was something stupid. It can not be just don't know, typed buch of stuff, somehow solved it, and everything is fine. Some of the audience is brain dead (btw why don't you just remove the front door, turn off the AC so you don't need the swacket and backup to AWS no even better GDrive and buy a jellything that just works by magic), most is not (I hope, you tell me...).

Did I forget tapes?

 

----

Besides, when did you upgrade the water cooled server? Is the video already up?

This is one of the reasons those of us in actual IT roles don't put too much weight into their server videos. We love them for their entertainment value, but it's like watching kids try and figure out calculus after just being taught that the red square doesn't fit into the blue circle cut-out.

 

That, and they - for whatever reason - aren't willing to put much detail into their server videos. They skimp a lot of details, or just edit past when they get stuck, etc. Most of that is likely because they're figuring it out as they go a lot of the time, so they just want to skip all the research and trial and error stuff.

 

But it would be nice if they started to do actual good quality, highly researched regular server content.

[Emily Young]

1 hour ago, fulminemizzega said:

Come on! What was the fix? Even if it was something stupid. It can not be just don't know, typed buch of stuff, somehow solved it, and everything is fine.

It definitely was something stupid: We didn't realize the drive wasn't fresh... The problem was just that it was already formatted. It wasn't talking about physical sector size, it was talking about the formatted cluster size.

 

WHY it wouldn't just nuke the drive when adding it to a zpool or otherwise say "hey, there's data on this drive, you sure about this fam" is beyond me.

[dalekphalm]

4 minutes ago, GabenJr said:

It definitely was something stupid: We didn't realize the drive wasn't fresh... The problem was just that it was already formatted. It wasn't talking about physical sector size, it was talking about the formatted cluster size.

 

WHY it wouldn't just nuke the drive when adding it to a zpool or otherwise say "hey, there's data on this drive, you sure about this fam" is beyond me.

Things like that drive me nuts - I've occasionally had issues with the Windows 10 Media Creator Tool, where when I prep a USB for Windows 10 installation, it fails because it has the wrong formatting and partitioning on it - but I know the utility formats the drive, so why doesn't it just clear everything and reformat?

 

The solution is to go into Diskpark and clean the drive, then reformat with standard NTFS partition and it works fine afterwards - why is that not built-into the MCT?

[Emily Young]

1 minute ago, dalekphalm said:

Things like that drive me nuts - I've occasionally had issues with the Windows 10 Media Creator Tool, where when I prep a USB for Windows 10 installation, it fails because it has the wrong formatting and partitioning on it - but I know the utility formats the drive, so why doesn't it just clear everything and reformat?

 

The solution is to go into Diskpark and clean the drive, then reformat with standard NTFS partition and it works fine afterwards - why is that not built-into the MCT?

Oh boy MCT. Heaven forbid you insert an ExFAT formatted drive...

[dalekphalm]

1 minute ago, GabenJr said:

Oh boy MCT. Heaven forbid you insert an ExFAT formatted drive...

I've actually never tried that...

 

How I miss the old Windows 7 USB DVD Download Tool (Which is a stupid name, considering it doesn't download anything - you need the ISO) - I found it was so much more reliable at creating an installer USB than the MCT.

[fulminemizzega]

43 minutes ago, GabenJr said:

It definitely was something stupid: We didn't realize the drive wasn't fresh... The problem was just that it was already formatted. It wasn't talking about physical sector size, it was talking about the formatted cluster size.

 

WHY it wouldn't just nuke the drive when adding it to a zpool or otherwise say "hey, there's data on this drive, you sure about this fam" is beyond me.

Thanks for the answer. This actually makes sense, I had the impression that 4KiB sector size was needed to get bigger storage capacity, and when did some research a few years ago about sector size for ZFS, I remember reading that no new drives with more than.. say 2tb of storage.. could have had 512 byte sector size. I thought it was related to addressing issues, on wikipedia it looks like it is related to density actually.  see next reply.