Jump to content

Unboxing the PETABYTE

nicklmg
By nicklmg
in LTT Releases
 Share
Posted

I'm just wondering... Why not use Windows Storage Spaces?

I would love some more details on why this specefic setup was chosen :)

 

(Yes, I'm planning to build something like this, and has actually chosen Storage Spaces for the software-raid)

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

I'm also wondering, why gluster fs instead of lustre or ceph? I understand not going for GPFS as it's very expensive... but gluster performance and features compared to the other two is kind of behind...

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
39 minutes ago, Sauron said:

@LinusTech how are you going to backup 800TB?

They're not. This IS the backup. That why the 1000TB became 800TB. I guess they will be using some kind of parity setup (you might haáve heard of RTAID5 or 6). So that if/when some of the drives dies, they will be able to just slot in new drives and restore the data

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
16 minutes ago, Squazzer said:

I'm just wondering... Why not use Windows Storage Spaces?

I would love some more details on why this specefic setup was chosen :)

 

(Yes, I'm planning to build something like this, and has actually chosen Storage Spaces for the software-raid)

I've never seen a productionized windows based distributed fs. Does windows spaces let you create a distributed fs accros a number of server?

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
1 minute ago, Tamarit said:

I've never seen a productionized windows based distributed fs. Does windows spaces let you create a distributed fs accros a number of server?

I actually don't know if it will work with multiple servers.

But you could set up multiple Windows SS's and the but an iSCSI layer in front of it if you wanted multiple servers

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

As mentioned in the video, desperately needs to move to tape backup or some sort of inexpensive cold storage.  You don't "need" 100 drives of hot storage and that many drives * the MTBF of each drive = you're going to be replacing drives every day.  One portion of the engineering compute of my employer archives large files after 6 months to tape.  These files are 4GB+ apiece and there's hundreds generated every day.

 

And the obligatory "seagate can go fuck themselves" for their shit 3TB drives.  Will never use them under any circumstance.  And their RMA-refurb replacement drives are garbage.  Terrible terrible brand.

Workstation:  13700k @ 5.5Ghz || Gigabyte Z790 Ultra || MSI Gaming Trio 4090 Shunt || TeamGroup DDR5-7800 @ 7000 || Corsair AX1500i@240V || whole-house loop.

LANRig/GuestGamingBox: 9900nonK || Gigabyte Z390 Master || ASUS TUF 3090 650W shunt || Corsair SF600 || CPU+GPU watercooled 280 rad pull only || whole-house loop.

Server Router (Untangle): 13600k @ Stock || ASRock Z690 ITX || All 10Gbe || 2x8GB 3200 || PicoPSU 150W 24pin + AX1200i on CPU|| whole-house loop

Server Compute/Storage: 10850K @ 5.1Ghz || Gigabyte Z490 Ultra || EVGA FTW3 3090 1000W || LSI 9280i-24 port || 4TB Samsung 860 Evo, 5x10TB Seagate Enterprise Raid 6, 4x8TB Seagate Archive Backup ||  whole-house loop.

Laptop: HP Elitebook 840 G8 (Intel 1185G7) + 3080Ti Thunderbolt Dock, Razer Blade Stealth 13" 2017 (Intel 8550U)

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

I thought the 100*10=800 joke was binary... because that would work too...
0100=4
0010=2

I make Rainmeter things and other art :D

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
3 hours ago, nicklmg said:

None... we've never had issues with Seagate drives and we've been using them in all of our servers for a couple years now. Zero complaints from us so far.

Now you have jinxed it.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
6 minutes ago, DrMikeNZ said:

Now you have jinxed it.

Nah I knocked on wood right after. Doing so again now.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
1 hour ago, Squazzer said:

I'm just wondering... Why not use Windows Storage Spaces?

I would love some more details on why this specefic setup was chosen :)

 

(Yes, I'm planning to build something like this, and has actually chosen Storage Spaces for the software-raid)

Storage spaces has extremely poor performance.

37 minutes ago, Wyzzy Moon said:

I thought the 100*10=800 joke was binary... because that would work too...
0100=4
0010=2

At this scale the PB vs PiB difference is quite large. There certainly is a component to binary involved as to what will be displayed within the OS.

It really is overdue for operating systems to stop reporting storage and file sizes with incorrect units.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

Why are you not using freeNAS? It is much better for storage purposes

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

Should have setup the archive storage as deduplicated... 160TB would so crazy amounts further... at the cost of performance. 

Can Anybody Link A Virtual Machine while I go download some RAM?

 

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
5 hours ago, wrathoftheturkey said:

Home sweet home?

download (8).jpg

What is that?

 

Since I am to lazy to put something interesting here, I will put everything, but slightly abbreviated. Here is everything:

 

42

 

also, some questions to make you wonder about life:

 

What is I and who is me? Who is you? Which armrest in the movie theatre is yours?

 

also,

 

Welcome to the internet, I will be your guide. Or something.

 

 

My build:

CPU: Intel Core i5-7400 3.0GHz Quad-Core Processor,

 Motherboard: ASRock B250M Pro4 Micro ATX LGA1151 Motherboard, 

Memory: Corsair 8GB (1 x 8GB) DDR4-2133 Memory,

Storage: Seagate Barracuda 1TB 3.5" 7200RPM Internal Hard Drive, 

Video Card: MSI Radeon RX 480 4GB ARMOR OC Video Card, 

Case: Corsair 100R ATX Mid Tower Case , 

Power Supply: Corsair CXM 450W 80+ Bronze Certified Semi-Modular ATX Power Supply, 

Operating System: Microsoft Windows 10 Home Full, 

Wireless Network Adapter: TP-Link TL-WN725N USB 2.0 802.11b/g/n Wi-Fi Adapter, Case Fan: Corsair Air Series White 2 pack 52.2 CFM  120mm Fan

 

ou do not ask why, you ask why not -me

 

Remeber kinds, the only differ between screwing around and scince is writing it down. -Adam Savage.

 

Only two things are infinite: the universe and human stupidity, and I'm not even sure of the former. - Albert Einstein.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
6 hours ago, nicklmg said:

None... we've never had issues with Seagate drives and we've been using them in all of our servers for a couple years now. Zero complaints from us so far.

 

Which series you are using? I saw backblaze listing 3/4 TB as failing super easily while 6/8 were more reliable than WD Red pros.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

Also, what is the price of the 60-slot enclosures? I am trying to project cost of a petabyte storage enclosure for my own team happily to generate half a terabyte of data a week and the cost get pretty steep pretty fast.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
  • Author
46 minutes ago, Andrei Chiffa said:

Which series you are using? I saw backblaze listing 3/4 TB as failing super easily while 6/8 were more reliable than WD Red pros.

We use all ECAP drives from Seagate. I think we have some 6, some 8, and now a bunch of 10 TB.

 

Backblaze data is... well, everyone has their own opinion on it, but I'ave always heard that their testing methodology is questionable at best.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
4 hours ago, wrathoftheturkey said:

freeNAS's main advantage is how lightweight it is -- I don't think they care with their 22 core Xeons

It is also not appropriate for handling a ZFS raid either, then again CentOS and Ubuntu don't fit the build. You would want to look at something such as Nexenta or Illumos (Open Solaris).

 

The best choice would be Solaris itself, because the Open Solaris variants are missing the key components that are proprietary to the enterprise version. There have been attempts to recreate these components in the open source versions but, they do not have the same level of performance and suffer from latency issues.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted

Does anyone here even have a Petabit storage machine?

FYI: A Petabit is 125 TB , no need to use a calc.(I used it)

Please quote me so that I know that you have replied unless it is my own topic.

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
11 hours ago, Squazzer said:

They're not. This IS the backup. That why the 1000TB became 800TB. I guess they will be using some kind of parity setup (you might haáve heard of RTAID5 or 6). So that if/when some of the drives dies, they will be able to just slot in new drives and restore the data

RAID is not backup. You need an off site copy to be safe.

Don't ask to ask, just ask... please 🤨

sudo chmod -R 000 /*

Link to comment
Share on other sites
Link to post
Share on other sites
Posted
26 minutes ago, Sauron said:

RAID is not backup. You need an off site copy to be safe.

Yes, One harddrive to rule them all, one harddrive to find them, One harddrive to bring them all and in the darkness bind them.

Thus, from the war of nature, from famine and death, the most exalted object which we are capable of conceiving, namely, the production of the higher animals, directly follows. There is grandeur in this view of life, with its several powers, having been originally breathed into a few forms or into one; and that, whilst this planet has gone cycling on according to the fixed law of gravity, from so simple a beginning endless forms most beautiful and most wonderful have been, and are being, evolved.

Link to comment
Share on other sites
Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing LTT Releases

×