Microsoft finds underwater datacenters are reliable, practical and use energy sustainably

[AndreiArgeanu]

Summary

In 2018 Microsoft deployed data centres that were sunk underwater at the sea floor in the Scottish sea. 2 years later it finally comes up looking to be far more reliable than a standard data centre.

 

Quotes

Quote

 On land, data centers run into issues like corrosion from oxygen and humidity and controlling shifts in temperatures. But in a water-tight environment with tight temperature control, far fewer issues crop up. The idea is that these kinds of servers can be easily deployed in sizes big and small near the coasts of areas that need them, giving better local access to cloud-based resources in more places.

The benefits are big. Microsoft says the underwater data center had just one-eighth the failure rate of a land-based data center, a dramatic improvement. That lower failure rate is important, given that it’s much harder to service a busted server when it’s in an airtight container at the bottom of the ocean.

 

My thoughts

From what I've seen so far this looks promising. If everything goes well I wouldn't be surprised to see in 1-2 decades, data centres being sunk at sea rather than taking loads of space in buildings. Not to mention the environmental benefit of not having to run thousands of fans, and other devices to cool down a data centre and keep the humidity and temperature in check.

 

Sources

Verge Microsoft

[Oshino Shinobu]

When you need a diving license to replace a hard drive. 

[Kisai]

10 minutes ago, AndreiArgeanu said:

 

My thoughts

From what I've seen so far this looks promising. If everything goes well I wouldn't be surprised to see in 1-2 decades, data centres being sunk at sea rather than taking loads of space in buildings. Not to mention the environmental benefit of not having to run thousands of fans, and other devices to cool down a data centre and keep the humidity and temperature in check.

 

Sources

Verge Microsoft

 

Well I can think of some long-term problems.

 

1 - Fire. If something fails and explodes, and takes out the entire unit. Can probably be solved by not having the unit filled with O2 but something else that doesn't burn. 

2 - Damage from shipping (eg anchors), or communications lines being severed. You can't just send someone down there to fix it, you have to retrieve the entire whole thing.

3 - Theft, nothing really stopping someone from just hauling it up and running off with it.

4 - Must be maintained at large expense , just how long do you think one of those containers will last before it corrodes straight through?

 

Advantages

1 - More secure from law enforcement and criminals wanting to sabotage individual machines.

2 - Likely lower cost of operating if it can be tied to tidal power.

3 - Lower failure rate of hardware if appropriate hardware is selected from the beginning. 5 years is not really a long time in terms of data center life. 15 years is more of a typical time, as hardware is typically never retired until no longer bootable, and the thing that fails with servers the most is the hard drive, and SSD's are nowhere near as reliable as HDD's given the same length of time. Presumably this can be solved by having two spare drives in each server for each SSD in case there are premature failures.

 

[jagdtigger]

First we should see if the waste heat dumped by these into the water wont have a negative effect on ocean life forms....

[AldiPrayogi]

Ayee whole room water cooling

[Scheer]

2 hours ago, Kisai said:

 

3 - Lower failure rate of hardware if appropriate hardware is selected from the beginning. 5 years is not really a long time in terms of data center life. 15 years is more of a typical time, as hardware is typically never retired until no longer bootable, and the thing that fails with servers the most is the hard drive, and SSD's are nowhere near as reliable as HDD's given the same length of time. Presumably this can be solved by having two spare drives in each server for each SSD in case there are premature failures.

 

Could be wrong here as I haven't looked for a few years, but almost all of my server stuff was 2-3 years old as data center pulls. From the reading I did back then, they rarely keep hardware for very long as its much cheaper to spend $5k on a new server that saves X amount of power as newer, more efficient electronics very quickly pay for themselves. Also helps with performance per unit, and the density gains pay for the cost of new servers as well.

 

I really doubt there is any half knowledgeable person out there running a 15 year old server in a DC, unless its for compatibility of an outdated system.

 

I also don't think SSD reliability has been an issue in the DC for about a decade now, and same with capacity/performance per density, no one keeps the same SSD for more than a few years just take a look at all of the DC pull drives on eBay. Higher speed and capacity will come around to easily justify the upgrade, it would be silly to run a 500mbps 256gb SSD from 2010 in a modern server now.

[Bombastinator]

I can think of one potential problem that would take longer than 2 years to become an issue: shellfish. 
 

the coast guard has a big issue with this on their booeys.   Shellfish will glom onto anything including other shellfish and the result is they get bigger.  In the case of booeys this can actually make them sink and they have to pull them and cut off the shellfish periodically. As long as they can do that I suppose things are fine though.  

[williamcll]

So that's why they call it Microsoft Azure.

[Taf the Ghost]

For 2-5 year cycle deployments, this makes sense. They're nitrogen filled, so you get less oxygen exposure issues, along with the biggest cost saver: world's largest water cooler outside the system. I could see the use for these in a lot of places.

 

The main thing is that Major Cities are always new large bodies of water. In the case of ocean-side cities, it's basically a "Free Real Estate" play because you can just put them right off the coast. I could see MS even doing this in the Bay near Seattle as a next feasibility testing. I suspect these could be very popular in a decade in, say, Tokyo Bay or Singapore. 

[Bombastinator]

21 minutes ago, Taf the Ghost said:

For 2-5 year cycle deployments, this makes sense. They're nitrogen filled, so you get less oxygen exposure issues, along with the biggest cost saver: world's largest water cooler outside the system. I could see the use for these in a lot of places.

 

The main thing is that Major Cities are always new large bodies of water. In the case of ocean-side cities, it's basically a "Free Real Estate" play because you can just put them right off the coast. I could see MS even doing this in the Bay near Seattle as a next feasibility testing. I suspect these could be very popular in a decade in, say, Tokyo Bay or Singapore. 

I don’t know what the cleaning cycle for coast guard bouys is.  Most sessile and semi sessile creatures in the sea have a free swimming larval stage where they look for something handy to stick to.  Water temperature and current play a big part in what decides to stick where.  Lots of things are slow growing though.  Corals and sponges for example.  2 years doesn’t mean there won’t be a problem in 5.  Cleaning works though so as long as they’re willing to haul em off and scrape em off every couple years it’s probably good.

[Jet_ski]

4 hours ago, AndreiArgeanu said:

My thoughts

From what I've seen so far this looks promising. If everything goes well I wouldn't be surprised to see in 1-2 decades, data centres being sunk at sea rather than taking loads of space in buildings. Not to mention the environmental benefit of not having to run thousands of fans, and other devices to cool down a data centre and keep the humidity and temperature in check.

This is an environmental disaster in the making. Besides increasing the water temp, which is going to effect the ecosystem, Microsoft and other companies will abandon these things underwater as soon as they are no longer financially viable to operate.
 

They should instead make their data centers and OS more efficient and output less heat. Purposely heating up the oceans is a terrible idea.

[Kisai]

2 hours ago, Scheer said:

Could be wrong here as I haven't looked for a few years, but almost all of my server stuff was 2-3 years old as data center pulls. From the reading I did back then, they rarely keep hardware for very long as its much cheaper to spend $5k on a new server that saves X amount of power as newer, more efficient electronics very quickly pay for themselves. Also helps with performance per unit, and the density gains pay for the cost of new servers as well.

 

I really doubt there is any half knowledgeable person out there running a 15 year old server in a DC, unless its for compatibility of an outdated system.

 

 

I have Dell PowerEdge 1950's at the office and remotely.  The ones in Cali had drives rotated out because they had literately been running well past the warranty period and the original ones from the off-lease purchase blew up. The ones at the office are... still running.

 

Anyway that wasn't the point, the point was that if you're going to put something underwater, you want to put something that will N-E-V-E-R need to be touched for as long as possible, because bringing it back up just isn't going to happen until there's a cost-effective reason for it. It's the same reason why satellites aren't repaired, they're just abandoned. Just think of these as satellite's except under water and slightly more retrievable than a rocket launch.

 

I'm, actually surprised that nobody has proposed just launching data center "space stations" or lunar stations. Given the substantial energy cost is burned in the initial launch.

[Arika]

7 hours ago, AndreiArgeanu said:

giving better local access to cloud-based resources in more places.

being underwater is about as far away from clouds as your can get though.

[Bombastinator]

8 hours ago, Jet_ski said:

This is an environmental disaster in the making. Besides increasing the water temp, which is going to effect the ecosystem, Microsoft and other companies will abandon these things underwater as soon as they are no longer financially viable to operate.
 

They should instead make their data centers and OS more efficient and output less heat. Purposely heating up the oceans is a terrible idea.

Nuclear power plants dump a lot of hot water as well so there have been fresh water studies. It seems that what happens is different ecosystems spring up because of water temperature but if the power plant shuts down the ecosystems die.  As to the heat thing it would be heating something.  I’m not sure heating water is less dangerous than heating air.  The abandonment thing is an issue. That could be solved with “pick up yer trash” regulation. 

[nick name]

New admins when confronted with hardware failures.

 

 

[HerrKaLeu]

8 hours ago, Jet_ski said:

This is an environmental disaster in the making. Besides increasing the water temp, which is going to effect the ecosystem, Microsoft and other companies will abandon these things underwater as soon as they are no longer financially viable to operate.
 

They should instead make their data centers and OS more efficient and output less heat. Purposely heating up the oceans is a terrible idea.

The heat output is insignifikant compared to power plants or other. A 1000 Mw power plant maybe has 1200MW waste heat depending on efficiency. a DC maybe a few hundred kW depending on size. Also, boat motors get cooled with sea water. I bet a cruise ship has several times the power output of a DC. i assume they select a location with some current and not a dead pool to get fresh water. 

 

How come I live in a 500,000 capita metropolis and only get DSL with 25Mbit, but somewhere in the middle of the ocean people have enough bandwidth to run a DC? 

[WereCatf]

12 hours ago, Kisai said:

1 - Fire. If something fails and explodes, and takes out the entire unit. Can probably be solved by not having the unit filled with O2 but something else that doesn't burn. 

2 - Damage from shipping (eg anchors), or communications lines being severed. You can't just send someone down there to fix it, you have to retrieve the entire whole thing.

3 - Theft, nothing really stopping someone from just hauling it up and running off with it.

4 - Must be maintained at large expense , just how long do you think one of those containers will last before it corrodes straight through?

1. At least in Microsoft's case, that thing was filled with nitrogen.
2. The Ars Technica - article addresses that as well. You'd typically plop these things down in areas where ship-routes aren't allowed anyways.
3. WEIGHT. You don't "just haul it"; you need a full team of people, lots of heavy-duty equipment, a really calm weather and it still takes the whole day to get it up.
4. Humans have a lot of installations underwater that don't "just corrode straight through." It can be there for decades before any physical maintenance has to be done.

[WereCatf]

8 hours ago, Jet_ski said:

This is an environmental disaster in the making. Besides increasing the water temp, which is going to effect the ecosystem

Um, and where do you think the thermal energy from data-centres on land go? It ALL ends up in the environment, one way or another. Thermal energy doesn't magically disappear, that's not how energy works.

[Kisai]

1 minute ago, WereCatf said:

Humans have a lot of installations underwater that don't "just corrode straight through." It can be there for decades before any physical maintenance has to be done.

Yeah, but those are made of concrete. Pretty much everything metal is typically painted with materials that are toxic to resist biological damage, never mind things that are intended to be below the water line and undisturbed.

[Fatih19]

The year is 2100, Microsoft has ordered a deployment of 2 million new datacenters in Neptune, "the whole planet is basically a gigantic fucking iceball" quoted from Stebe Vallmer, CEO of Microsoft.

[Bombastinator]

12 minutes ago, Kisai said:

Yeah, but those are made of concrete. Pretty much everything metal is typically painted with materials that are toxic to resist biological damage, never mind things that are intended to be below the water line and undisturbed.

The traditional material in such paint is copper. In the 18th century there was copper bottoming of ships for this same purpose.  Lately the material has been capsaicin.  They mix red pepper in the paint.  Metal also has other and much more severe problems in salt water.  It’s offset with sacrificial anodes, but those have hard limits on how long they last. There is also the option of PVC.  The question becomes what are the shells made out of? 

[Bombastinator]

33 minutes ago, HerrKaLeu said:

The heat output is insignifikant compared to power plants or other. A 1000 Mw power plant maybe has 1200MW waste heat depending on efficiency. a DC maybe a few hundred kW depending on size. Also, boat motors get cooled with sea water. I bet a cruise ship has several times the power output of a DC. i assume they select a location with some current and not a dead pool to get fresh water. 

 

How come I live in a 500,000 capita metropolis and only get DSL with 25Mbit, but somewhere in the middle of the ocean people have enough bandwidth to run a DC? 

They run their own cables.  You can too if you’re willing to spend the hundreds of thousands of dollars to do it.  There was a particularly clever municipal that ran cables through the sewer pipes of a city. 
 

I personally have been hoping for kazinet. Every home has a sewer connection that goes to the roof of the building through the stack pipe. Run the cable through the sewer, up the stack pipe, and down to the internet connection.  No holes, no drilling, no right-of-way issues. Just little robots scurrying through pipes carrying cable. 

[spartaman64]

whats stopping them from building those containment units on land?

[soldier_ph]

14 hours ago, Kisai said:

Theft, nothing really stopping someone from just hauling it up and running off with it.

Florida man would like to know your location.

[Lord Vile]

14 hours ago, Kisai said:

 

Well I can think of some long-term problems.

 

1 - Fire. If something fails and explodes, and takes out the entire unit. Can probably be solved by not having the unit filled with O2 but something else that doesn't burn. 

Just fill it with Nitrogen

Quote

2 - Damage from shipping (eg anchors), or communications lines being severed. You can't just send someone down there to fix it, you have to retrieve the entire whole thing.

Depends what depths you drop it to, Anchors don't go that far down they're only meant for shallower waters of 100m or less in depth

Quote

3 - Theft, nothing really stopping someone from just hauling it up and running off with it.

Yeah because you're gonna spend hundreds of thousands in equipment to dig up encrypted drives that are worth nothing aside from the data they store, you're also gonna do that quickly enough so that whoever owns the server can't notice it's offline and doesn't it to the police. 

Quote

4 - Must be maintained at large expense , just how long do you think one of those containers will last before it corrodes straight through?

Depends, plenty of materials do fine in sea water.