LTT should buy this SuperComputer

[RexFuzzle]

This used super computer is up for auction

https://gsaauctions.gov/auctions/preview/282996

145k cores, 300TB ram

[Thaldor]

While cool and all but it's getting to the end of it's time (the watercooling is dying, the quick connects are leaking as said in the description) and what would Linus do with it?
 

It would be probably the longest running tech series on the YouTube to get it running:

- Getting all to fit into a building (probably the Labs building from which it still would take a lot of space).

- Upgrading the building to house it (the cooling loops, environmental controls, power and probably fire suppression).

- Fixing the watercooling that has caused enough expenses and downtime within 6 months to make it better for the NCAR to eat their losses and sell it off than keep it.

- Fixing the damages that the failing watercooling has caused (only the CPUs and RAM DIMMs are common parts, pretty much everything else is proprietary to that machine).

- Finding the 1% of the nodes that are currently suffering from ECC errors, fixing them and then probably replacing 10% of the RAM to not have nodes with ECC errors within few months.

- Cabling all of it (all internal cables will be given with the machine but they will be removed, that's a lot of cabling).

- And then finally figuring out what Linus could do with it (no, it won't even run Cinebench, even less Crysis).

 

Oh yeah, and it's also apparently sold without the storage. The HPE SGI ICE XA includes 40 PB of DDN storage with IBMs SpectrumScale which aren't included in the auction (or at least they aren't listed in the auction). Remember how long it took Linus to finish the first Petabyte project?

[dilpickle]

3 hours ago, Thaldor said:

While cool and all but it's getting to the end of it's time (the watercooling is dying, the quick connects are leaking as said in the description) and what would Linus do with it?
 

It would be probably the longest running tech series on the YouTube to get it running:

- Getting all to fit into a building (probably the Labs building from which it still would take a lot of space).

- Upgrading the building to house it (the cooling loops, environmental controls, power and probably fire suppression).

- Fixing the watercooling that has caused enough expenses and downtime within 6 months to make it better for the NCAR to eat their losses and sell it off than keep it.

- Fixing the damages that the failing watercooling has caused (only the CPUs and RAM DIMMs are common parts, pretty much everything else is proprietary to that machine).

- Finding the 1% of the nodes that are currently suffering from ECC errors, fixing them and then probably replacing 10% of the RAM to not have nodes with ECC errors within few months.

- Cabling all of it (all internal cables will be given with the machine but they will be removed, that's a lot of cabling).

- And then finally figuring out what Linus could do with it (no, it won't even run Cinebench, even less Crysis).

 

Oh yeah, and it's also apparently sold without the storage. The HPE SGI ICE XA includes 40 PB of DDN storage with IBMs SpectrumScale which aren't included in the auction (or at least they aren't listed in the auction). Remember how long it took Linus to finish the first Petabyte project?

Honestly a youtuber is the best buyer I can think of for this. Who else would want a broken supercomputer.

[Brian McKee]

1 hour ago, dilpickle said:

Honestly a youtuber is the best buyer I can think of for this. Who else would want a broken supercomputer.

ME

[Thaldor]

6 hours ago, dilpickle said:

Honestly a youtuber is the best buyer I can think of for this. Who else would want a broken supercomputer.

Probably yeah. Most likely if it will sell (the reserve is probably hard to meet) it will sell to someone who will dismantle it and try to sell the parts for some profit or, with a lot less chance, it will sell to someone who already has some supercomputer and is wanting an upgrade and finds this worth it (as in they have the facilities for it and calculate that fixing it and buying the storage is more profitable than buying a completely new one).

 

Used, breaking supercomputers aren't really the hottest selling stuff on the Earth.

They could probably get rid off it easier by selling individual cells, racks and even nodes but that probably would take them more time and effort than just throwing it away and as a government body they probably are just seeking the most cost effective way to getting rid off it as they already have a new one running.

[RollinLower]

14 minutes ago, Thaldor said:

Probably yeah. Most likely if it will sell (the reserve is probably hard to meet) it will sell to someone who will dismantle it and try to sell the parts for some profit or, with a lot less chance, it will sell to someone who already has some supercomputer and is wanting an upgrade and finds this worth it (as in they have the facilities for it and calculate that fixing it and buying the storage is more profitable than buying a completely new one).

 

Used, breaking supercomputers aren't really the hottest selling stuff on the Earth.

They could probably get rid off it easier by selling individual cells, racks and even nodes but that probably would take them more time and effort than just throwing it away and as a government body they probably are just seeking the most cost effective way to getting rid off it as they already have a new one running.

honestly having one of those racks for your homelab would be pretty sick. i'm not even talking about the aging hardware inside, just the personalised racks with the lettering on them

[Needfuldoer]

1 hour ago, RollinLower said:

honestly having one of those racks for your homelab would be pretty sick. i'm not even talking about the aging hardware inside, just the personalised racks with the lettering on them

Broadwell still has legs in a lab environment. (This thing is a cluster with 8,064 Xeon E5-2697v4s.) That's PowerEdge Rx30 / HP Gen9 / Supermicro x10 generation stuff. I wouldn't necessarily run the chassis inside those racks, but it's commodity hardware that can be broken up to populate some standard barebone servers.

 

It's unfortunate the storage isn't included. If it was, they'd probably organize a group buy over on /r/homelab.

[RollinLower]

1 hour ago, Needfuldoer said:

Broadwell still has legs in a lab environment. (This thing is a cluster with 8,064 Xeon E5-2697v4s.) That's PowerEdge Rx30 / HP Gen9 / Supermicro x10 generation stuff. I wouldn't necessarily run the chassis inside those racks, but it's commodity hardware that can be broken up to populate some standard barebone servers.

 

It's unfortunate the storage isn't included. If it was, they'd probably organize a group buy over on /r/homelab.

Gotta store those linux isos somewhere!

[Ben Andersen]

I saw this auction and thought it would be fun to see Linus do videos with it.  I came to the forums to create an account to post the suggestion and was happy to find someone had already posted the auction!

[Ender144]

Created an account just to make this same suggestion. I saw this auction on HackaDay website and come here to suggest Linus make a video series on "We are Building a Super Computer." I bet it would be a long running series! I would watch.  @LinusTech

[GarlicDeliverySystem]

On 4/30/2024 at 3:05 AM, dilpickle said:

Honestly a youtuber is the best buyer I can think of for this. Who else would want a broken supercomputer.

ok, you could probably content farm this, but this thing is huge, literally.

 

Like, they have 8064 processors in there, according to some documentation with 4 dual socket systems per blade that means 1008 blades.

As I understand from a quick glance, the blade enclosures for the compute elements have 9(!) blades only, plus some extra cooling and IO racks. At the minimum one "E-rack" is required for two compute racks.

 

1008/9 = 112 compute racks at minimum

112/2 = 56 extra E-racks for cooling

plus some extra stuff

 

So we are speaking of 168 racks at the very least.

 

All of that would need to be transported, stored, tested, cooled, etc. Forget about the costs of the hardware, finding a place for that and the power bill will be a larger concern.

 

Edit:

[FlyingPotato_is_taken]

Breaking up this LOT into parts is feasible.

 

Actually running this cluster? Wouldn't touch it with a 10 foot pole.

1. wiring dissembled AND no documentation. This will be the hardest to solve issue.

2. damaged nodes.

3. leaks already happened and probably not cleaned up so good luck replacing all fittings and cleaning up past spills.

 That thing won't run anytime soon again.

[LinusTech]

1.7MW

[tkitch]

4 hours ago, LinusTech said:

1.7MW

you gonna let something like the power budget of a small city get in the way of owning a supercomputer?

[Kisai]

14 hours ago, LinusTech said:

1.7MW

You gonna buy a Wind Turbine next?  You know we have these in BC right?

 

As much as the super computer would be cool to have, I think it's best the winner of the auction just part it out to the blade level and recycle the rest. If it was possible to have bought the entire thing (including storage) intact, someone might have a use for it, but without the storage it may as well be a paper weight.

 

 

[GarlicDeliverySystem]

Next challenge: undervolting 8000 CPUs

 

 

might get us to 1.5MW, who knows lol.

[Tim Norris]

Did anyone notice it sold for 480085.00? 

Let me separate that a bit 4,80085 on a calculator that is "for Boobs" and it has to be a meme price. I call shenanigans. 

[Bitter]

On 5/1/2024 at 3:54 PM, LinusTech said:

1.7MW

Two words. Nuclear Reactor. I'm sure the Russians are selling some at fire sale prices these days on the grey market. You'd only need a small one anyway. It's green energy even. You're just not dreaming big enough.

[Kisai]

On 5/5/2024 at 7:10 PM, Bitter said:

Two words. Nuclear Reactor. I'm sure the Russians are selling some at fire sale prices these days on the grey market. You'd only need a small one anyway. It's green energy even. You're just not dreaming big enough.

It's illegal to mine Uranium or build/operate a nuclear reactor in BC. So that won't be happening.

 

That said, a fun, but very silly and expensive project would be to "off-grid" build a wind turbine. Probably not for LTT though. "What can we run on 2MW of power? BOINC? Crypto? SuperComputer?" 

 

I mean they are relatively cheap when you consider other options, just wind isn't 100% reliable.

 

[BrandonTech.05]

How fast could this thing theoretically transcode wan show