Upgrade for the Petabyte Project

[LiamReturn0]

I am sure you have all seen the recent Intel SSD news, 32TB Ruler SSD's. That right there got me thinking about Petabyte Project, as with these ruler SSD's you can have a 32 slot machine to easily make a petabyte system (not including the RAID configuration, purely RAW), and would be much quicker for the type of workflow that LTT has shown off over the years. Oh, did I not mention that these 32 slot machines are 1U? Yeah, think about the storage per litre of that bad boy. Though it might be costly (honestly, corporate accounting from Linus Media Group must still be kicking themselves from the RED situation), future proofing would be easily done. My current system for a NAS is 72 Samsung SSD 860 PRO 4TB's in Synology hardware but I am genuinely thinking of upgrading to a SuperStorage SSG-136R-NR32JBF or something similar from Supermicro and just slapping 32 x 32TB once they actually come out. Guys, please leave your comments and thoughts on this, also if you are reading this Linus, #linusdroptips.

[Cole5]

That would be cool...but at what cost

[LiamReturn0]

That is a VERY good question. Looking at the current Intel P4500 SSD's pricing for an 8TB capacity unit right now (US$5,985.79) and if you follow the laws of relativity, that would make each unit of 32TB roughly US$23,943.16 (but knowing Intel's pricing and bulk deals, it will most certainly be lower than that, maybe $18.5K-20K) leading up to a huge (and disappointing) US$766,181.12 (again, not taking into fact the bulk/Intel rubbish) just for drives. If I have a look at the actual cost of building the old Petabyte Project right now (thinking 45Drives, Seagate 10TB), that would most likely be around $75,000. Correct me if wrong.  This is a huge cost but I am pretty sure none of us has access to LMG's financials, but if we go off of statistical data about YouTube alone we could say LMG makes $240.1K - $3.8M yearly and we can safely assume they are in the higher end due to their content not being toxic like 3/4 of the larger YouTubers.

[LiamReturn0]

If you were to try and do 1PB RAW with Samsung SSD 860 PRO's, you would be paying US$407,168 but honestly do you want 256 units, an entirely custom solution would need to be designed in terms of hardware and potentially software (manual mapping is a bitch, trust me.)

[LiamReturn0]

You would need a Supermicro  2026T-6RF+, 2026T-6RFT+ or 2026T-URF4+, or rather, you would need 16 of those. 32U of space for SSD storage, no thanks, you would also have the cost of not only extra servers (16:1) and also the solution (what I believe you would do is have a similar thing to the 100K PC Part 2 solution which is also used by Synology, to have SAS ports out the back of all of them, and have them link all together through SAS cords, but you would be bottlenecking heavily). Oh, one thing I forgot to mention. Power. You would need that 16,000 CAD UPS to be multiplied for you to have the same level of efficiency as the current Petabyte Project (and other items in the rack.)


Also, pretty sure this thing would cut down power in general with SSD's being KNOWN for having less power consumption than mechanicals.

[Jtalk4456]

@LinusTech

@nicklmg

@colonel_mortis

@AlexTheGreatish

Any thoughts on doing something like this?

[LiamReturn0]

Isn't tagging the guys like a cardinal sin  

[manikyath]

pretty sure linus has held one of those 32TB modules on the show floor somewhere.

 

that said, petabyte project is not the goal for this sort of implementation, not only would the cost be astronomical compared to the already hefty cost of spinning rust, it would be far beyond the speed even linus and his crew can demand from a storage server.

 

EDIT: also, petabyte project is in this specific topic, essentially archive storage.

[Fullmental]

Since the Petabyte Project is for archive storage, I really don't see LMG doing this to hold data that will be accessed so infrequently that the drive speeds are almost irrelevant? Not that it wouldn't be a cool video, but I would certainly hope that they used it for a more speed-sensitive application, like the fast editing server (I forget the name of it), and even then you certainly wouldn't need a full petabyte.

[Firewrath9]

700k is alot of money, why not just buy 10PB of storage and do crazy raids on them like RAID 1100, quadruple the speed, quadruple the safety or even more?

cheaper probably:

https://pcpartpicker.com/product/JmM323/western-digital-gold-10tb-35-7200rpm-internal-hard-drive-wd101kryz

379 for 10TB, 1000 of them is 379k

10PB of WD golds is 379k which is still cheaper than your 766k

16PB for RAID 11110000 (Raid 0 4 times and Raid 1 4 times) (effective 4PB) will still be cheaper than your SSDs.

Add 10 1TB 970 pros for cache and still cheaper

[manikyath]

1 minute ago, Firewrath9 said:

RAID 11110000

thats not how raid works...

 

but other than that, point well made

(PS: also, you did forget to count the price of raid cards)

[Firewrath9]

Just now, manikyath said:

thats not how raid works...

 

but other than that, point well made

(PS: also, you did forget to count the price of raid cards)

still cheaper. you raid the WD golds' 4 times with RAID 1, the another 4 time on RAID 0, Is that not how it works?

Raid cards a drop in the bucket.

https://www.amazon.com/High-Point-RocketRAID-2760A-PCI-Express/dp/B004HIN8NW/ref=sr_1_5?s=electronics&ie=UTF8&qid=1534534628&sr=1-5&keywords=sata+raid+controller+24+port

24 ports for the price of 2 WD golds.

[LiamReturn0]

I understand that the current Petabyte Project acts as a form of archive, I don't mean for this to be for archival purposes. If we want to get into archiving, let's get some tape drives, slap those boys in a vault room. Screw AES-256 encryption, Kappa. But really though, I mean you could do it much smaller and set up an automatic flow system for archiving but also as temporary speedy storage, and also upgrade from 10Gbps  weak crap now hahaha

SSD's have better life expectancy, performance, failure rate, power loss protection, power consumption heat dissipation and hell if we wanna go there, noise.

[manikyath]

Just now, Firewrath9 said:

still cheaper. you raid the WD golds' 4 times with RAID 1, the another 4 time on RAID 0, Is that not how it works?

Raid cards a drop in the bucket.

https://www.amazon.com/High-Point-RocketRAID-2760A-PCI-Express/dp/B004HIN8NW/ref=sr_1_5?s=electronics&ie=UTF8&qid=1534534628&sr=1-5&keywords=sata+raid+controller+24+port

24 ports for the price of 2 WD golds.

you dont add numbers for each "count" of mirrors you make, all that would still result in a raid 10 array.

which, to be honest is also kind of inefficient compared to raid6 at this amount of storage, which in itself is also rather dated compared to the stuff petabyte project is actually using.

[LiamReturn0]

For anyone that doesn't get the 10Gbps joke, I can leave you with the fact that link aggregation and new standards exist if you think I am being genuine about speed.

[manikyath]

1 minute ago, LiamReturn0 said:

I understand that the current Petabyte Project acts as a form of archive, I don't mean for this to be for archival purposes. If we want to get into archiving, let's get some tape drives, slap those boys in a vault room. Screw AES-256 encryption, Kappa. But really though, I mean you could do it much smaller and set up an automatic flow system for archiving but also as temporary speedy storage, and also upgrade from 10Gbps  weak crap now hahaha

SSD's have better life expectancy, performance, failure rate, power loss protection, power consumption heat dissipation and hell if we wanna go there, noise.

thats the point, they dont need a petabyte of active storage, even with the stupendous filesizes they work with their current (already fairly bonkers) SSD server is fine.

[LiamReturn0]

Mani, you are the only person with a valid point.  Who knows, episode whatever of Early Access Hardware 

[manikyath]

1 minute ago, LiamReturn0 said:

Mani, you are the only person with a valid point.  Who knows, episode whatever of Early Access Hardware 

as i said, if i recall linus has already held one of these modules on a show floor.

[DataEmo]

What about tape library with an LTO8 drive and one of Taran's robots as an autoloader? Maybe covered in a video series like the whole room watercooling project some time ago.

[Guest]

I’d be surprised if intel sent over more than 3 of that kind of SSD. 

Let alone 2.  

[orbitalbuzzsaw]

I'd say probably pointless for Petabyte Project, but for the $100,000 PC it might be cool to have one

[Pixel5]

they already have a SSD based server to edit off and the petabyte project is just to long time storage so having it completely SSD based would be pointless.