Unboxing Canada's BIGGEST Supercomputer

[1988fido]

awesome video , plz more content like this

[SuicideRabbit]

If anyone has questions, I'm on the team that runs this system.

I'd be happy to answer.

[thehardwarehacker]

27 minutes ago, SuicideRabbit said:

If anyone has questions, I'm on the team that runs this system.

I'd be happy to answer.

ill take a shot. im vaguely familiar with island networking topography, is there a resource that explaines that better, at least as far as used with the supercomputer?  

[yathis]

Where is SFU?

 

Its not an unboxing because there is no box.

 

Thirdly, its not that hard to get your facts straight, 5000 vs 9000 sf

Big Diff

[SuicideRabbit]

1 hour ago, thehardwarehacker said:

ill take a shot. im vaguely familiar with island networking topography, is there a resource that explaines that better, at least as far as used with the supercomputer?  

Not a resource I can point you to directly, that I'm aware of. If you are familiar with island topology, we chose this design b/c it was the only way we could get our blocking factor as low as possible. Within an island, every node is 1:1. Across the cluster, b/t islands, the blocking factor is 2:1.

[SuicideRabbit]

1 hour ago, yathis said:

Where is SFU?

Burnaby, British Columbia. About 30 minutes away from downtown Vancouver.

[Enderman]

UBC gotta up their supercomputer game...

[yathis]

1 hour ago, SuicideRabbit said:

Burnaby, British Columbia. About 30 minutes away from downtown Vancouver.

Whats it stand for cuz I thought it was San Fransisco University.

[SuicideRabbit]

1 minute ago, yathis said:

Whats it stand for cuz I thought it was San Fransisco University.

http://www.sfu.ca

[yathis]

Just now, SuicideRabbit said:

http://www.sfu.ca

Ah Simon Fraser University, makes sense. Now I should wikipedia who exactly Simon is.

[Ezio Auditore]

9 hours ago, SuicideRabbit said:

If anyone has questions, I'm on the team that runs this system.

I'd be happy to answer.

How do those giant pipes on the ceiling cool? Like do they shoot out cool air cause saw some hose like structures.

[SuicideRabbit]

35 minutes ago, Ezio Auditore said:

How do those giant pipes on the ceiling cool? Like do they shoot out cool air cause saw some hose like structures.

They carry the water to/from the rather large evaporative chillers outside of the building, which were not shown in the video.

[Arika]

[removed]

[SpaceGhostC2C]

19 hours ago, spwath said:

hmm, only about 7gb ram per cpu

7GB per core  

 

[PCGuy_5960]

Cinebench Score?

[Guest]

Just now, PCGuy_5960 said:

Cinebench Score?

what if it runs on 32 bit os and has to run old Cinebench instead

[ggbaker]

39 minutes ago, PCGuy_5960 said:

Cinebench Score?

I'm also from the SFU crowd...

 

I really tried to get one of the GPU nodes for Linus to run cinebench. Sadly, the tech staff said they were "using them". And also there's the small wrinkle that they don't run Windows.

[themctipers]

21 hours ago, Vespertine said:

Scientific research or not, who actually needs 22,000 Xeon CPUs? I'm confused.

me

 

i need to crack a zip with a password length of 27+, but theres only 9 digits and 4 letters in the whole password.

[ggbaker]

21 hours ago, Vespertine said:

Scientific research or not, who actually needs 22,000 Xeon CPUs? I'm confused.

The facility is available to any Canadian researcher who needs high-performance computing. In general, it wouldn't be used as single monolithic supercomputer: many jobs would be running scattered across the cluster. The job that needs 3TB of RAM on one box can be running there, while the highly-parallel job is running on 1000 cores on the next rack.

 

Apparently, they expect the facility to be utilized at 100%. If there are any unallocated cycles, the particle physicists will snap them up: some part of the LHC project uses the Compute Canada facilities and they always need more compute power.

[laminutederire]

@LinusTech @nickmlg nice video,  but there was an inaccuracy. The ideal PUE isn't 1. You can, at least in recent theories, go beyond that and have sub 1 PUE. Ideally you'd want to get back the energy you consume, or even more if you implement nice methods. This is a very important paradigm shift new minds have to be educated about, at least in my opinion, so since you guys have a huge audience, that'd be a nice touch.

[BlueChinchillaEatingDorito]

On ‎2017‎-‎07‎-‎02 at 11:38 AM, nicklmg said:

Thanks to SFU and Compute Canada for allowing us to visit this INCREDIBLE facility. Learn more about Cedar at http://geni.us/5N3l

 

 

No thank you for letting me know what they were building after all this time.  Now I'm really proud of being a Computing Science student at SFU knowing what's behind all the recent Big Data talk.

On ‎2017‎-‎07‎-‎03 at 2:26 AM, SuicideRabbit said:

They carry the water to/from the rather large evaporative chillers outside of the building, which were not shown in the video.

You're a student there as well?

[SuicideRabbit]

7 hours ago, BlueChinchillaEatingDorito said:

No thank you for letting me know what they were building after all this time.  Now I'm really proud of being a Computing Science student at SFU knowing what's behind all the recent Big Data talk.

You're a student there as well?

Nope. I'm Lead Sys Admin for the system.

[AnonymousGuy]

On 7/5/2017 at 0:40 AM, SuicideRabbit said:

Nope. I'm Lead Sys Admin for the system.

What fluid is circulating through the cooling system?

[SuicideRabbit]

1 hour ago, AnonymousGuy said:

What fluid is circulating through the cooling system?

It's a small bit of a cocktail, but the majority is no different than in your car's radiator: glycol and water.

[AnonymousGuy]

1 hour ago, SuicideRabbit said:

It's a small bit of a cocktail, but the majority is no different than in your car's radiator: glycol and water.

Is it UV sterilized and particulate filtered or is there some sort of biocide in it?  Is it straight glycol+RO water or did you add a scale inhibitor as well?

 

I ask for details because I have a 25 gallon loop in my house with a lot of similarities to how datacenters do cooling, and I want to know the "proper" implementation.