The $1,000,000 Unboxing. (SPONSORED)

[Nystemy]

5 minutes ago, daice said:

Also NO, depends what mainframes we are talking about some are about pure compute power.

It depends on the application.

Some stuff one would just use a more traditional computing cluster at a lot lower price for the same overall performance. Mainframes are after all rather tightly integrated systems and that adds a fair bit of cost, so for applications where inter process communication isn't as important, then it is a bit of a waste to have such high ratio of interconnect performance compared to computing performance.

 

Mainframes are fairly expensive from a CPU performance standpoint, but yes some workloads do benefit from them. Effectively speaking, it is a regular computer on steroids doing more or less everything it can to have as little NUMA related issues that is typically seen in normal computer clusters. _{(Back in the 80's and 90's mainframes were though "common" as an office workhorse, mainly since the desktop computers of the time were often lacking in performance for more computing intense tasks, even the best desktop computers of the time weren't that impressive compared to the low end options at the time. These days the higher end workstation options are a lot more powerful than what is considered a decent office computer, this has taken a fair portion of mainframe tasks. And a lot of workloads have gotten more scalable so smaller server clusters has taken a huge chunk of what mainframes used to do. In short, if we want to sort a database of a million customers, then we do not need to log into the mainframe these days. A lot of other workloads haven't gotten particularly bigger either, but the needed computing has moved from requiring a mainframe to even store the dataset all the way to now needing a low end laptop to do it in a few seconds or minutes. These developments have made what once were seen as an obvious need become almost irrelevant for the majority of medium to large businesses. Since the mainframe were effectively just an overgrown workstation with more RAM and compute able to run "regular" software.)}

 

Though, to be fair. I would be surprised if pure computing even is 20% of the overall mainframe market these days, compared to transaction handling for the financial sector, or other similar database handling tasks like selling concert/airline tickets, running a stock market, or a slew of other tasks.

 

But mainframes also typically has 1 more card up their sleeve, and that is redundancy. Quite often even on a compute level where if a CPU dies, the workload continues as if nothing happened. _{(or rather "should continue as if nothing happened.")} Though, memory, CPU, and so forth is usually all hot swappable in these systems, after all 99.99% uptime is considered lousy in some mainframe markets. _{(usually the financial market)}

[derr12]

1 hour ago, Required said:

WE TE EF!!!
No Mask!
No Distance!
No Hand Sanitiser!
No Gloves!
No wonder that Covid spread AT LOT!

Rules don't require it in Canada anymore. The rule of thumb now is "you do you" round these parts. 

[derr12]

This hardware unboxing made me throw my pants away in a fit of passion.  

[Technoturnovers]

Meanwhile, I am trying to cram an upgrade of my 5 year old minecraft server into a budget of $250, and I am probably only gonna be able to afford to upgrade the RAM to 64gigs.

Also, I don't even have enough proper drive cages or SATA cables, my SSD is just lying free and two of my hard drives are connected via USB adapter. Well, at least I can live vicariously by watching these videos showing the type of setup I wish I could have, lol

[jordantrizz]

This is an amazing video. But the way they're handling things like they're in a scrap yard makes me die inside.

[Luscious]

Kioxia perhaps short changed you on the drives. That model you have is their datacenter grade drive rated for only 1xDWPD. A better model would have been the 30TB CM6-R giving you more write endurance and higher capacity. Their CM6-V can go even higher at 38TB writes per day on their 12TB model. Given that this has been in the planning for 12 months though it's possible these were what were available at the time.

 

Those servers aren't balls to the wall either - a 7773X and 75F3 will both outperform the 7543 on compute. Your particular workload though probably doesn't benefit from the extra horsepower those CPU's would have. Also, I would seriously consider replacing the sata DOM with a proper m.2 boot drive - those things really are only meant for systems that run 24/7 forever and only need to boot once in a blue moon.

 

There are also more economical solutions if all you want is just a PB of SSD. JBOD is one and there are plenty of options to cram from 32 drives up to 72 SSD's into a single 4U expansion chassis. The big caveat is NVMe - if you need that, along with the combined throughput NVMe provides, your options pretty much come down to doing it the way Kioxia have here. JBOD NVMe does exist but you're capped on that interconnect.

 

It's also quite possible now to fit over a PB of RAID60 HDD storage in a single 4U server with 24TB of NVMe and 3TB of optane - handy if you need a massive read/write cache and a high endurance scratch disk.

 

 

[kumicota]

I just hope that linus stops using samba for it's network sharing and began to use NFS, CIFS or any protocol that isn't bad as SMB

[Results45]

On 3/28/2022 at 3:14 PM, sub68 said:

I can't wait to watch the rest.

I have messed around with old cisco gear at my school and I even think that stuff is cool but current stuff is gonna be intreasting.

 

Hey LMG, you guys gonna gonna try running Swiftshader Crysis again on those dual Zen3 EPYCs and a FULLY pathtraced game on those A100s at 100 samples per pixel?

[Nafis]

What's the purpose of those GPUs in a storage only server ?

[GuiltySpark_]

2 minutes ago, Nafis said:

What's the purpose of those GPUs in a storage only server ?

This segment of the WAN show should explain everything in more detail but if you watched to the video they mention AI benchmarks. 

 

 

[MadAnt250]

4 hours ago, Required said:

WE TE EF!!!
No Mask!
No Distance!
No Hand Sanitiser!
No Gloves!
No wonder that Covid spread AT LOT!

I don't know if you are joking or just being a Karen.

[chasewf]

so when we getting the rest of the videos? I am supper excited I hope these come out soon.

[dogwitch]

3 hours ago, Technoturnovers said:

Meanwhile, I am trying to cram an upgrade of my 5 year old minecraft server into a budget of $250, and I am probably only gonna be able to afford to upgrade the RAM to 64gigs.

Also, I don't even have enough proper drive cages or SATA cables, my SSD is just lying free and two of my hard drives are connected via USB adapter. Well, at least I can live vicariously by watching these videos showing the type of setup I wish I could have, lol

check out used server stuff on ebay or craiglist etc.

on samba thing person mention. windows needs to add more on file transfer protocol.

[Middcore]

6 hours ago, Required said:

WE TE EF!!!
No Mask!
No Distance!
No Hand Sanitiser!
No Gloves!
No wonder that Covid spread AT LOT!

It's spring 2022, basically everybody who had the option and who's not a moron got vaccinated a long time ago, we knew sanitizing stuff didn't do any good by like summer 2020 anyway, and this is no different from how they've been doing all the videos for quite a while. 

[igormp]

8 hours ago, JonoT said:

Thanks to Kioxia for sponsoring this video! Check out their CD6 NVME SSD at https://lmg.gg/rkDFz

$1,000,000 is how much this entire setup costs, not including all the man-hours since it took over a YEAR to make this happen! Kioxia, AMD, NVIDIA, Supermicro, and Wekka combined their efforts, and they have blown our minds.

Please, this time try to do a proper research and come with a proper methodology for ML benchmarks. Avoid at all costs to data-starve those GPUs, and puny benchmarks such as resnet50 won't cut it, even a single A100 can easily overwhelm that task (and I did the benchmarks for that, see bellow).

 

Try to train or fine tune some large-scale transformers or LLM such as BERT on a really large corpus with a really large batch size or some other vision transformer with tons of images. Also make sure that there's a proper training pipeline with data being always fed to the GPU, you really want to avoid data starvating here and I can't emphasize it enough, even though you have tons of fast flash and networking, such hardware that needs to work with many small pieces of data will end up suffering a lot from latency.

 

On 3/23/2022 at 12:39 AM, igormp said:

Got an A100 to try out because I was bored, got some nice numbers and we can clearly see that this workload is simply just too simple for this GPU.


+-------------------+---------------+----------------+----------------+----------------+----------------+---------------+----------------+----------------+----------------+----------------+-----------------+
|    GPU-Imgs/s     | FP32 Batch 64 | FP32 Batch 128 | FP32 Batch 256 | FP32 Batch 384 | FP32 Batch 512 | FP16 Batch 64 | FP16 Batch 128 | FP16 Batch 256 | FP16 Batch 384 | FP16 Batch 512 | FP16 Batch 1024 |
+-------------------+---------------+----------------+----------------+----------------+----------------+---------------+----------------+----------------+----------------+----------------+-----------------+
| 2060 Super        | 172           | NA             |            NA  |            NA  |            NA  | 405           |            444 |            NA  |            NA  |            NA  |            NA   |
| 3060              | 220           | NA             |            NA  |            NA  |            NA  | 475           |            500 |            NA  |            NA  |            NA  |            NA   |
| 3080              | 396           | NA             |            NA  |            NA  |            NA  | 900           |            947 |            NA  |            NA  |            NA  |            NA   |
| V100              | 369           | 394            |            NA  |            NA  |            NA  | 975           |           1117 |            NA  |            NA  |            NA  |            NA   |
| A100              | 766           | 837            |           873  |           865  |           OOM  | 1892          |           2148 |          2379  |          2324  |          2492  |          2362   |
| Radeon VII (ROCm) | 288           | 304            |            NA  |            NA  |            NA  | 393           |            426 |            NA  |            NA  |            NA  |            NA   |
| 6800XT (DirectML) | NA            | 63             |            NA  |            NA  |            NA  | NA            |             52 |            NA  |            NA  |            NA  |            NA   |
+-------------------+---------------+----------------+----------------+----------------+----------------+---------------+----------------+----------------+----------------+----------------+-----------------+

 

Also did an AI-benchmark run:

 

Device Inference Score: 21692
Device Training Score: 23542
Device AI Score: 45234

 

5 hours ago, daice said:

I think renting just the Nvidia V100 was like 80-100 bucks USD an hour, minimum renting hour is an hour.

So would except even a "cell" of a IBM mainframe to be ...very... pricey.

 

Also NO, depends what mainframes we are talking about some are about pure compute power.

More like less than $3/hour (that's a decimal dot)

 

Z systems have no GPU available for them AFAIK, and a similar Z config as the above one without the GPU goes for really cheap:

 

I'm not sure if IBM Cloud has spot/preemptible instances, but that should lower the costs for another 50~80%.

Those systems are usually billed by the second or minute, it'd be weird for IBM to bill by the hour unlike their competitors.

 

 

1 hour ago, GuiltySpark_ said:

This segment of the WAN show should explain everything in more detail but if you watched to the video they mention AI benchmarks. 

 

 

On that video they also mention just "ai benchmarks", so it seems that they have no idea what to do with the GPUs (up to this point).

I would really love to see them partnering up with someone like Yannic Kilcher in order to make proper use of that stuff, or at least get help from an nvidia engineer, they should make proper use out of those GPUs without much issues.

[GuiltySpark_]

2 minutes ago, igormp said:

 

On that video they also mention just "ai benchmarks", so it seems that they have no idea what to do with the GPUs (up to this point).

I would really love to see them partnering up with someone like Yannic Kilcher in order to make proper use of that stuff, or at least get help from an nvidia engineer, they should make proper use out of those GPUs without much issues.

As has been mentioned at least once in this thread already, LTT's video's are 1.) for entertainment and wow factor and 2.) need to appeal to the largest possible audience. While what someone like Yannic Kilcher does is interesting to a subset of people, its not LTT's audience. 

 

...but you knew that already.

[igormp]

20 minutes ago, GuiltySpark_ said:

As has been mentioned at least once in this thread already, LTT's video's are 1.) for entertainment and wow factor and 2.) need to appeal to the largest possible audience. While what someone like Yannic Kilcher does is interesting to a subset of people, its not LTT's audience. 

 

...but you knew that already.

Yes, but running the same benchmark as they did in their A100 video would be pointless and disappointing even for their casual watchers and make no use whatsoever of all that available data throughput, so even their sponsors would be disappointed.

 

I mentioned Yannic because he's an youtuber and would likely be able to setup something interesting enough for a casual video while still making proper use of such hardware, but there's no need to actually have him on the entire video, just the setup part.

 

Having something like a MuJoCo or other RL tasks that can be easily visualized would also be a nice a nice idea instead of just big numbers that no one can properly compare.

 

Anyhow, my comments are mostly because they seem to just be freestyling it without a proper idea on next steps. Seems like they didn't expect such class of hardware to work with and are now overwhelmed by it. My hope is that they already figured it out since the video should have been recorded like 1 or 2 weeks ago.

[Required]

6 hours ago, Middcore said:

It's spring 2022

6 hours ago, Middcore said:

who's not a moron got vaccinated a long time ago

6 hours ago, Middcore said:

sanitizing stuff didn't do any good

Yes it is thats why I am so mad that I still not permit to visit a Restaurant and must wear Mask inside a Building, Public Transport, Car,..! And the glorified vaccine did nothing beside make some People rich and the other a Virus spreader who not even know it.

Sure thats why before enter a Supermarket its recommend and the Staff look after the Customer to use the Hand sanitizer...

[Middcore]

5 hours ago, Required said:

Yes it is thats why I am so mad that I still not permit to visit a Restaurant and must wear Mask inside a Building, Public Transport, Car,..! And the glorified vaccine did nothing beside make some People rich and the other a Virus spreader who not even know it.

Sure thats why before enter a Supermarket its recommend and the Staff look after the Customer to use the Hand sanitizer...

 

This response is nutty in a whole different way than I expected. 

[Required]

1 hour ago, Middcore said:

This response is nutty

Sorry I dont understand.

[Middcore]

1 minute ago, Required said:

Sorry I dont understand.

Doesn't matter. There's a dedicated thread for COVID-19 talk, go rant there. 

[dip]

Would like to see some AI deep learning content.
I would love to collaborate with you on some project using those A100, Let me know if you are interested.

[alpha754293]

On 3/28/2022 at 6:50 PM, kumicota said:

I just hope that linus stops using samba for it's network sharing and began to use NFS, CIFS or any protocol that isn't bad as SMB

So....interestingly enough -- if you actually google/YouTube "NFS vs. iSCSI vs. SMB", there are actually videos (which is supported by evidence) that SMB/CIFS is actually THE more widely supported data transport protocol than NFS. (Sidenote: CIFS is actually a quote "dialect" of SMB. (Source: https://en.wikipedia.org/wiki/Server_Message_Block#CIFS)

 

More importantly, if you also YouTube NFS (on Windows), you will also find and note that NFS can actually perform WORSE on Windows clients than SMB (because NFS isn't native to Windows, so it was added on as an "afterthought" of sorts) whereas SMB, being a Microsoft product, is in Windows "natively". (Why do you think that *nix servers always have to install Samba ex post facto? (Although to be fair, you also often have to install NFS after the fact as well if the default OS install image doesn't include nfs-common and/or nfs-kernel-server packages with said install image.))

The backend can be whatever you want it to be in order to be able to facilitate server-to-server transfers. (e.g. I run NFSoRDMA for backend transfers on systems and clients that support NFSoRDMA over 100 Gbps Infiniband at home).

The front end can also be whatever it is that you want it to be. For my Windows clients at home that either can't or aren't able to support NFSoRDMA on 100 Gbps Infiniband, I use SMB.

Unless you're going to be deploying 200 GbE ConnectX-6 cards in every single workstation, it would be irrelevant whether you're using SMB or not. (e.g. if said Windows clients/desktop editing workstations are ONLY using 10GbE - SMB vs. NFS - would won't even see/notice a difference here).

It'd be an entirely different story if they were going to be using SMB Multi-channel and/or SMB Direct.

But I think that they've already switched over to TrueNAS Core at least once before; so it is not unreasonable to surmise that they are looking to at LEAST go with TrueNAS Core, or perhaps, the potentially better option for them would be to actually move to TrueNAS Scale, once Scale is more stable for production environment usage/deployments. (Which, on that note, they would probably be wise to stick with the more stable TrueNAS Core for production deployment).

And for the record, as someone who runs NFSoRDMA on 100 Gbps Infiniband (granted, I'm using spininng rust SAS 12 Gbps drives instead of NVMe 4.0 x4 SSDs), it really doesn't matter whether you're using NFS or SMB. The speeds are about the same.

You're never going to be able to hit the theoretical peak bandwidth speeds anyways (which are all of the advertised speeds).

[alpha754293]

On 3/28/2022 at 12:58 PM, JonoT said:

Thanks to Kioxia for sponsoring this video! Check out their CD6 NVME SSD at https://lmg.gg/rkDFz

$1,000,000 is how much this entire setup costs, not including all the man-hours since it took over a YEAR to make this happen! Kioxia, AMD, NVIDIA, Supermicro, and Wekka combined their efforts, and they have blown our minds.

I would urge caution against using GlusterFS for each of the storage nodes to be a Gluster volume brick in order to tie all of the storage nodes back to the cluster headnode.

 

The process of removing a Gluster volume brick is not-trivial, or at the very least, you can't detach a gluster brick from a gluster volume quickly (following the command to shrink the gluster volume so that you can detach the brick from the volume).

 

Also, if you are planning on using TrueNAS, please note that as of TrueNAS Core 12.0-U1.1, NFS over RDMA is NOT supported nor operational.

(See my thread here, more specifically, "edit #2" for specific deployment details: https://www.truenas.com/community/threads/truenas-12-0-u1-1-infiniband-support.90996/)

Without SOME form of RDMA, you're going to run into issues trying to hit/maximise your available storage bandwidth capacity.

[alpha754293]

23 hours ago, CosmoKramer said:

For those that might be interested, Patrick from ServeTheHome actually reviewed an Inspur system about 8 months ago, so if you want to learn a little bit more about the Nvidia 8xA100 system, you can get an in-depth taste for it here: