128-Bit Storage

Hunts_man · February 13, 2018

I came across a blog post from 2004 where a guest author spoke about the reason for developing a 128-bit storage solution, rather than the 64-bit traditional storage solution for ZFS.

They figured that, based on Moores law, Datasets would double every 9-12 months. Turns out... this is a real problem....

"Some customers already have datasets on the order of a petabyte, or 250 bytes. Thus the 64-bit capacity limit of 264 bytes is only 14 doublings away. Moore's Law
for storage predicts that capacity will continue to double every
9-12 months, which means we'll start to hit the 64-bit limit in
about a decade. Storage systems tend to live for several decades,
so it would be foolish to create a new one without anticipating the
needs that will surely arise within its projected lifetime."

It gets more interesting... Or should I say Moore? As it turns out Moores Law is unlikely to continue at the same rate. There are basic quantum mechanics that impose limits to the maximum computational and data storage rates. This brings other issues with it.

"In particular,
it has been shown that 1 kilogram of matter confined to 1 liter
of space can perform at most 10⁵¹ operations per second
on at most 10³¹ bits of information [see Seth Lloyd,
"Ultimate physical limits to computation." Nature 406, 1047-1054
(2000)]. A fully-populated 128-bit storage pool would contain
2¹²⁸ blocks = 2¹³⁷ bytes = 2¹⁴⁰
bits; therefore the minimum mass required to hold the bits would be
(2¹⁴⁰ bits) / (10³¹ bits/kg) = 136 billion
kg."

Yea.... 136 Billion Kilograms of gear. That's rough even for a seasoned data engineer. It's a big number to swallow.

Whats even more ridiculous is the amount of energy it might need to operate a 128-bit storage pool at capacity.

"To operate at the 10³¹ bits/kg limit, however, the
entire mass of the computer must be in the form of pure energy.
By E=mc², the rest energy of 136 billion kg is
1.2x10²⁸ J. The mass of the oceans is about 1.4x10²¹ kg. It takes about 4,000 J to raise the
temperature of 1 kg of water by 1 degree Celcius, and thus about
400,000 J to heat 1 kg of water from freezing to boiling.
The latent heat of vaporization adds another 2 million J/kg.
Thus the energy required to boil the oceans is about
2.4x10⁶ J/kg \* 1.4x10²¹ kg =
3.4x10²⁷ J. Thus, fully populating a 128-bit
storage pool would, literally, require more energy than boiling
the oceans."

Yea... I guess by now you see the implications... Its a rabbit hole of issues and questions that until now I had never considered.

But a lot has changed (Still doing some digging into this).

Here is the original article: https://blogs.oracle.com/bonwick/128-bit-storage:-are-you-high

I also recommend that you check out: https://news.ycombinator.com/

8uhbbhu8 · February 13, 2018

This is not news or a review. Should be in another section.

Interesting concept as it is.

WkdPaul · February 13, 2018

Not Tech News, moved to the Storage section.

bowrilla · February 14, 2018

Moore's law doesn't imply that data sets double in size but the number of transistors on an integrated circuit (correct me if I'm wrong). Since 2012, that development slowed down and is now more on a three year cycle. Correlations between the number of transistors on a chip and the amount of data that is being processed by it are coincidental. Moore's law isn't even a law but only an observation. Predicting mid to far future developments based on Moore's law is pointless.

Maybe the author was talking about what was one year later called Kryder's law (Scientific American, August 2005) – which is also just a vague prediction based on observations of the past and present developments. Kryder never gave a fixed rate for increases in areal platter densities. In 2005 he postulated that if development kept their pace we'd have 40TB 2.5" HDDs by 2020 for ~$40. We're obviously miles away from that.

There's a more relevant article on Moore's law, developments in memory, CPU and storage technologies and Kryder's law at Western Digital: Does storage break Moore's law? Even they mix it all up a bit and jump from speed/bandwith to capacity from HDD to SSD to CPUs.

SSDs do indeed advance faster than HDDs – but they have similar issues just like CPUs do: limits of physics just like it's mentioned by the artice's author. It is not possible to infinitely advance and shrink circuits as quantum mechanics will at some point mess everything up. The limit for silicon based CPUs will be around 5nm afaik – that'll most likely be the end of silicon. Maybe a different material will replace it and offer one or two more steps but that's about it. That's an issue for CPUs as well as memory and storage technologies. At some point you can't go smaller, therefore density increases will slow down and stop at some point.

If we compare the developments in storage density between HDDs in the 90s and early 00s and SSDs around now it's not unlikely to expect a similar slow down as well. Now that article was from 2004. A time at which NTFS was already 11 years old. A 64bit file system that's still relevant today, about 25 years after it's introduction. NTFS's conceptual max storage limit is 16EiB (granted, at the moment implementations max out at 256TiB but same goes for max RAM, the limit of Win10 is 512GiB while it could be as well 16EiB). Neither NTFS nor ZFS (that's the file system the author of that original article worked on) are being used for massive datacenters. That's more likely Apache Hadoop Filesystem. In 2012 Facebook's Hadoop Cluster had 100PiB capacity. They use Hadoop not because 64bit is at it's end but because they need so many clusters for scalability. If there was a HDD that pushed 64bit file systems to the max (so 16EiB capacity) in 2012 Facebook was at about 0.63% of that limit. Thanks to storage networks that limit shouldn't even bother you so … basically that article was pretty pointless. Datacenters already went for parallelization.

There's no real point in going 128bit – neither for storage nor for CPUs. Ever wondered how much mass there is in the observable universe? About 10⁵³kg. That means there are about 10⁸⁰ atoms in the observable universe. A 128bit system could address about 3.4 x 10³⁸. The amount of data is actually pretty insane. So no, nobody needs 128bit file systems.

Besides: before you copy and paste, make sure that exponents are copied as well! Your quotes massively distort the meaning by converting 2⁵⁰to 250 and 2⁶⁴to 264.