TSMC Exploring On-Chip, Semiconductor-Integrated Watercooling

[Lightwreather]

Summary

TSMC, at the VLSI symposium, recently presented its investigations into on-chip watercooling as a way to battle issues with heat dissipation. And it involves integrating water channels straight into the chip's design. 

 

Quotes

Quote

As transistors get increasingly compressed together due to denser manufacturing technologies and added vertical 3D chip stacking, temperature becomes an increasingly critical problem to address. TSMC's researchers think the solution is allowing water to flow in-between sandwiched circuits. It's an incredibly simple theoretical solution, but is an extremely difficult engineering feat to pull off safely --for the electronics, that is.

 

The mathematics are simple: Current cooling solutions typically work via direct contact with a given chip's heatspreader, direct die contact technologies, or full submersion in a non-conducting fluid. Of these, the first two solutions can only efficiently cool the layers they're directly in contact with, which brings huge problems for vertical chip stacking. The lower layers will have much more trouble dissipating their heat, with of damage or having to throttle, either of which would be bad for performance. Not only that, but the top layer of the chip will have added strain from having to essentially carry the entire package's heat through to the dissipation layer. And liquid submersion, while efficient and likely better for stacked dies, is expensive and difficult to deploy in professional scenarios that are already geared for air or traditional water cooling.

 

TSMC carried out testing on a dummy semicondutor - a Thermal Test Vehicle (TTV), which is essentially a heating element made out of copper - in controlled lab conditions. The company tested three types of silicon water channel integration in controlled conditions: It used a pillar-based channel, where water could flow around active semiconductor pillars to cool them down (think of water around an island); a design which featured a trench design (think of a river, controlled by its shores); and a simple, flat water channel on top of the rest of the silicon chip. The water was passed through an external cooling mechanism that chilled the water down to 25 ºC from its journey through the silicon chips. The company further tested three types of water cooling designs: one with only direct water cooling (DWC), where water has its own circulating channels etched directly into the chip's silicon as part of the manufacturing process; another design with water channels being etched into their own silicon layer on top of the chip proper, with a Thermal Interface Material (TIM) layer of OX (Silicon Oxide Fusion) that carried heat from the chip to the watercooling layer; and lastly a design which swapped the OX layer for a simpler, cheaper liquid metal solution.

 

TSMC reported that the best solution was by far the direct water cooling method, which could dissipate up to 2.6 kW of heat and offered a temperature delta of 63 ºC. The second-best design was naturally the OX TIM-based one, which could still dissipate up to 2.3 kW of heat and offered a temperature delta of 83 ºC. The liquid metal solution came in last, still managing to dissipate up to 1.8 kW (temperature delta of 75 º C). Of all the water flow designs, the pillar-based one was the best by far.

 

My thoughts

Well, I think we all know that this was soon gonna happen (/s). but yea, On-die watercooling...... This is huge especially if it can dissapate 2.3 kW (2300 Watts, ie.). Tho, at the moment it isn'doesn't really make much sense, but in the near future with 3-D stacking of die layers (eg. intel's Forveros method), one can see how this might be useful. Of course, it will take years for such exotic cooling solutions to truly be adopted in the mainstream. But this is definitely one of the ways forward to enable continued increases in transistor density, continued improvement in the all-important performance-per-area metric, and for future 3D semiconductors (at the cost of some electricity, hopefully this doesn't mean that companies stop focusing on the effieciency of their chips). 

Sources

Tom's hardware

Hardwareluxx.de

[RejZoR]

Problem is also surface. Some chips are so small these days despite massive number of transistors they concentrate all the heat in a super tiny surface area and that's a problem even when IHS is suppose to spread the heat out.

 

I think as some sort of middle stop they could use dummy silicon to extend the size of chips in some cases where space allows. Silicon is incredibly thermally conductive at almost 150 W/m K and if you could make surrounding material larger it should at least in theory spread out heat faster and help with standard cooling we have today. It however brings yield issue since it would take up waffer space and higher use of raw material. Shouldn't affect yield itself per chip since it would just be dead silicon attached to functional silicon.

[Kisai]

1 hour ago, J-from-Nucleon said:

 

My thoughts

Well, I think we all know that this was soon gonna happen (/s). but yea, On-die watercooling...... This is huge especially if it can dissapate 2.3 kW (2300 Watts, ie.)

Unfortunately, no "water cooling" is safe to use in any of these manners except precisely controlled ones, eg a data-center.

 

Home computers will likely never see it, and no manufacturer is going to want to offer a 10 year warranty on something water-cooled. It's pulling teeth just to get a 3 year warranty on anything.

 

Mobile devices certainly aren't going to see it.

 

Heck, what is certainly to happen is that transporting anything using liquid cooling in this matter will likely destroy the cooling system due to improper storage during shipping (think thermal expansion/freezing), where something get's left outside or in a truck overnight and it ends up below freezing. 

 

Like, water-cooling is cool, but if you start designing a chip to require it, it's like with helium in hard drives, these are not future-proof at the minimum, and self-destructing at the worst.

 

[leadeater]

23 minutes ago, Kisai said:

Unfortunately, no "water cooling" is safe to use in any of these manners except precisely controlled ones, eg a data-center.

 

Home computers will likely never see it, and no manufacturer is going to want to offer a 10 year warranty on something water-cooled. It's pulling teeth just to get a 3 year warranty on anything.

 

Mobile devices certainly aren't going to see it.

 

Heck, what is certainly to happen is that transporting anything using liquid cooling in this matter will likely destroy the cooling system due to improper storage during shipping (think thermal expansion/freezing), where something get's left outside or in a truck overnight and it ends up below freezing. 

 

Like, water-cooling is cool, but if you start designing a chip to require it, it's like with helium in hard drives, these are not future-proof at the minimum, and self-destructing at the worst.

Ship them unfilled with fluid containers, doesn't actually have to be "water", and easy way to fill.

 

Also how are He HDD not future proof? Self destructing? What on earth are you on about? He is just the inside chamber filled with He which is a more stable gas with better heat conduction, it is by far better than using air. Also no the chamber cannot be a vacuum as the read/write heads require a gas to float above the platter, without a gas the heads would have to be further away to avoid hitting the platter which would reduce the effectiveness of the heads and reduce the possible aerial density. There are zero drawbacks to He drives, anyone who told you otherwise is talking hot air. He does not leak out of the disk chamber, unless you are hitting your HDDs with hammers, don't do that.

 

P.S. WD proved He does not leak by immersing He HDDs in fluid, recorded over very long time. Also you can run the disks just with air for short time but they will overheat, useful for data recovery.

[TVwazhere]

Like most things this will likely be implemented only for servers for a long time before it makes anything mainstream. But good to see the concept....... holds water :^) 

[StDragon]

32 minutes ago, Kisai said:

Unfortunately, no "water cooling" is safe to use in any of these manners except precisely controlled ones, eg a data-center.

 

Home computers will likely never see it, and no manufacturer is going to want to offer a 10 year warranty on something water-cooled. It's pulling teeth just to get a 3 year warranty on anything.

 

Mobile devices certainly aren't going to see it.

I agree.

 

The fluid would have to be pure enough to not clog the micro channels. It's bad enough when you see custom water loops with fouling because of mineralization (hard water) or some other contamination. One option is to segment the die fluid with a heat exchanger, but then your back to water cooling a traditional way anyways, and the water block can still foul with debris.

I'm thinking a better approach would be integrated phase-change fluid in micro channels. The flashpoint in the phase would draw heat away from the die, then you can carry off the heat in more traditional methods via the heat-spreader. 

[Bombastinator]

49 minutes ago, leadeater said:

Ship them unfilled with fluid containers, doesn't actually have to be "water", and easy way to fill.

 

Also how are He HDD not future proof? Self destructing? What on earth are you on about? He is just the inside chamber filled with He which is a more stable gas with better heat conduction, it is by far better than using air. Also no the chamber cannot be a vacuum as the read/write heads require a gas to float above the platter, without a gas the heads would have to be further away to avoid hitting the platter which would reduce the effectiveness of the heads and reduce the possible aerial density. There are zero drawbacks to He drives, anyone who told you otherwise is talking hot air. He does not leak out of the disk chamber, unless you are hitting your HDDs with hammers, don't do that.

 

P.S. WD proved He does not leak by immersing He HDDs in fluid, recorded over very long time. Also you can run the disks just with air for short time but they will overheat, useful for data recovery.

I could see how it could be an issue in that helium is a small molecule and keeping it contained can be a problem.  Not as bad as hydrogen.  Nitrogen is a larger molecule and pretty easy to keep contained. Argon is also large.  Helium also has issues in that it is a finite and extremely rare resource. All the helium humanity has comes from decaying radioactives.  I’m just guessing though.  I don’t know the methodology of that one either.

[Kisai]

4 hours ago, leadeater said:

 

P.S. WD proved He does not leak by immersing He HDDs in fluid, recorded over very long time. Also you can run the disks just with air for short time but they will overheat, useful for data recovery.

Helium leaks out of everything. Period. It may not be significant, but hard drives are not a solid material, it's a sandwich. All it takes is for the physical drive to change air pressure, temperature, or even the air composition, and whatever is holding the drive together will deteriorate. At least with air models, they will not die from a change in air pressure because they have that one tiny "do not cover" hole in them for that.

 

Look it's not like I'm saying He drives last 1 year and they are toast, it's that is not a viable long term use case, and WD can say whatever they want about it, the drives haven't been in existence for 20 years.

 

https://www.backblaze.com/blog/backblaze-hard-drive-stats-q1-2021/

 

Backblaze indicated in another article which drives were helium, which are the HGST models beginning with HUH. The Seagate models however are not specific, though Seagate indicates the Exos X16 (drives ending with G) and X18 (drives ending with J) are Helium drives.

 

https://www.backblaze.com/blog/helium-filled-hard-drive-failure-rates/

 

Water-cooling, adds all sorts of variables that you can not account for, so like Helium, unless the entire thing is a self-contained unit, you're going to end up with situations where the long term storage, or changes in air pressure (eg altitude) or outside temperature change how it performs or eventually degrades how it's sealed.

[Bombastinator]

I’ve occasionally seen hard drives last for 20 years but it’s because they were not in frequent use.  Also they were all at least 20 years old so they often used technology not even in use anymore.  They did tend to be quite loud. 

[cj09beira]

i really wonder if water is ever going to be the liquid used here, i can more easily see them having a sealed system using a similar liquid to those on heatpipes, as a way to increase the effectiveness of a IHS, 

[Bombastinator]

Just now, cj09beira said:

i really wonder if water is ever going to be the liquid used here, i can more easily see them having a sealed system using a similar liquid to those on heatpipes, as a way to increase the effectiveness of a IHS, 

A common working fluid for heat pipes is low pressure water.  Mostly because it’s cheap and easy.  They fill the open pipe up to the brim, boil the water (thus expanding it),seal it, and let it cool. Water in a hard vacuum will boil instantly. Water in a partial vacuum does weird and interesting things.

[Forbidden Wafer]

13 minutes ago, cj09beira said:

i really wonder if water is ever going to be the liquid used here, i can more easily see them having a sealed system using a similar liquid to those on heatpipes, as a way to increase the effectiveness of a IHS, 

What about the pressure? Just reminding that silicon is basically glass and microscopic fractures tend to grow. If you have a sealed system with liquid turning into gas, you will have an increase in pressure, which can speed up the fracturing.

[cj09beira]

39 minutes ago, Forbidden Wafer said:

What about the pressure? Just reminding that silicon is basically glass and microscopic fractures tend to grow. If you have a sealed system with liquid turning into gas, you will have an increase in pressure, which can speed up the fracturing.

dont forget that the ihs is also made to protect the die from too much pressure, it really shouldn't be a problem glass materials can take a lot of pressure they only struggle with stress points which the ihs protects from

[AnonymousGuy]

Intel has been looking at TSV cooling channels also for stacked die.  Basically using vias as a heatsink. 

[StDragon]

2 hours ago, AnonymousGuy said:

Intel has been looking at TSV cooling channels also for stacked die.  Basically using vias as a heatsink. 

I found this from back in 2010, but it's behind a paywall.

 

https://ieeexplore.ieee.org/document/5429151

"Abstract:

In this paper, an integrated liquid cooling system for 3-D stacked modules with high dissipation level is proposed. The fluidic interconnects in this system are elaborated and the sealing technique for different fluid interfaces is discussed. Meanwhile, the pressure drop for each part of the system is analyzed. The optimized fluidic interconnects minimizing the pressure drop have been designed and fabricated, and the compact system is integrated. In line with the fluidic interconnect design and analysis, an experimental process for hydraulic characterization of the integrated cooling system is established. The pressure drops for different fluidic interconnects in this system are measured and compared with the analyzed results."

 

[leadeater]

6 hours ago, Kisai said:

Helium leaks out of everything. Period.

No it doesn't, Gas Tight is a thing. WD/HGST/Seagate have all proved He leaking is not an issue with He disks. He can leak, He is hard to contain compared to many other gases but it's not impossible. Difficult != impossible.

 

But essentially don't make dire scare mongering statements about HDDs that are not a real issue at all. Failure rates have a lot more to do with He vs Air as well anyway, even though there is no statistically significant difference anyway He drives are more physically dense (more and thinner platters) with more actuator arms and read/write heads the tolerances required are even greater.

 

The amount of He leaking out, even though it doesn't would be at a rate slower than that the bearings in the motors would wear out. HDD will be dead before the gas composition in the chamber would matter.

[Bombastinator]

5 hours ago, cj09beira said:

dont forget that the ihs is also made to protect the die from too much pressure, it really shouldn't be a problem glass materials can take a lot of pressure they only struggle with stress points which the ihs protects from

Am reminded of vacuum bottles.  Negative pressure and positive pressure are in some ways the same. Glass is commonly used for vacuum bottles but everything is rounded

[Bombastinator]

32 minutes ago, leadeater said:

No it doesn't, Gas Tight is a thing. WD/HGST/Seagate have all proved He leaking is not an issue with He disks. He can leak, He is hard to contain compared to many other gases but it's not impossible. Difficult != impossible.

 

But essentially don't make dire scare mongering statements about HDDs that are not a real issue at all. Failure rates have a lot more to do with He vs Air as well anyway, even though there is no statistically significant difference anyway He drives are more physically dense (more and thinner platters) with more actuator arms and read/write heads the tolerances required are even greater.

 

The amount of He leaking out, even though it doesn't would be at a rate slower than that the bearings in the motors would wear out. HDD will be dead before the gas composition in the chamber would matter.

Hydrogen sort of does.  It slows after a while as the material becomes saturated but hydrogen can go through solid steel. Hydrogen tanker trucks are double walled with a layer of mirrored Mylar and a layer of netting in the middle.  The netting is to keep the Mylar from sticking to the steel so it can actually reflect some atoms back into the tank.  There are always losses in transportation.  Tiny tiny little molecules. I don’t know enough about helium.  It’s bigger than hydrogen but still small. Would move slower. 

[leadeater]

3 minutes ago, Bombastinator said:

Hydrogen sort of does.  It slows after a while as the material becomes saturated but hydrogen can go through solid steel. Hydrogen tanker trucks are double walled with a layer of mirrored Mylar and a layer of netting in the middle.  The netting is to keep the Mylar from sticking to the steel so it can actually reflect some atoms back into the tank.  There are always losses in transportation.  Tiny tiny little molecules. I don’t know enough about helium.  It’s bigger than hydrogen but still small. Would move slower. 

Yea but it's not like He HDDs are not internally coated to prevent premutation either and what you'd implement for short term transportation is different to long term storage, especially when talking about low atmospheric pressure (He HDD) vs high atmospheric pressure.

 

None of the HDD manufactures are going to implement a technology that has even a single ounce of risk of being non useful after even as little as 10 years, you can actually warranty HDDs in enterprise storage systems for that length of time if you really want to and if He HDD were guaranteed to be non operable after 10 years because of permutation then they simply would not exist. Since they do exist, actual scientific testing and proof models were done by these companies to know if/when it would be a problem the fact they exist and are being sold in massive qualities is all the evidence anyone should need to know that it isn't an issue at all.

 

Knowledge and information used badly is a real problem, like I said just because something is difficult doesn't make it impossible or some huge problem anyone is actually realistically going to encounter. Synthetic rubbers are used for this very reason, to contain gases like He, does anyone really think these are not being used in He disks?

[Bombastinator]

7 minutes ago, leadeater said:

Yea but it's not like He HDDs are not internally coated to prevent premutation either and what you'd implement for short term transportation is different to long term storage, especially when talking about low atmospheric pressure (He HDD) vs high atmospheric pressure.

 

None of the HDD manufactures are going to implement a technology that has even a single ounce of risk of being non useful after even as little as 10 years, you can actually warranty HDDs in enterprise storage systems for that length of time if you really want to and if He HDD were guaranteed to be non operable after 10 years because of permutation then they simply would not exist. Since they do exist, actual scientific testing and proof models were done by these companies to know if/when it would be a problem the fact they exist and are being sold in massive qualities is all the evidence anyone should need to know that it isn't an issue at all.

 

Knowledge and information used badly is a real problem, like I said just because something is difficult doesn't make it impossible or some huge problem anyone is actually realistically going to encounter. Synthetic rubbers are used for this very reason, to contain gases like He, does anyone really think these are not being used in He disks?

This sounds like a time frame thing. 
 

Might depend on the particular synthetic a lot. there are synthetic rubbers such as the types used in automotive tires that can’t even contain oxygen.  Racers use nitrogen filled tires partially because the pressure is more controllable than air filled. You can fill a tire with nitrogen and expect it to still be full later. Air is of course mostly nitrogen so the things don’t deflate very far. Nitrogen percentage content is a lot higher inside an old tire than a new one though

Edited July 14, 2021 by Bombastinator

[Quackers101]

*turns the CPU into a giant connected heat pipe*

[Bombastinator]

8 minutes ago, Quackers101 said:

*turns the CPU into a giant connected heat pipe*

I vaguely recall a description of a paper from a guy in Switzerland who sort of did that.  Something about channels in the silicon or something.

Edited July 14, 2021 by Bombastinator

[leadeater]

11 minutes ago, Bombastinator said:

Might depend on the particular synthetic a lot. there are synthetic rubbers such as the types used in automotive tires that can’t even contain oxygen.

Nitrile Rubber and EPDM are used for He containment, they have the lowest permutation of anything.

[da na]

I wonder if this will ever become a consumer thing. I could see it being valuable for efficiency in datacenters and servers but honestly consumer CPUs will never output so much heat they need that level of cooling (imo).

[Bombastinator]

9 minutes ago, leadeater said:

Nitrile Rubber and EPDM are used for He containment, they have the lowest permutation of anything.

A nitrile party balloon would kind of suck I suspect.  Nitrile isn’t very stretchy.