Dual Xeon 24-Core, 48 Thread Workstation Beast

[George Li]

Skip ahead if you don't want to hear the background!

So after recently upgrading my workhorse commercial photography cameras to the Canon 5ds earlier last year and Hasselblad H6d-100 late last year, I've quickly come to the conclusion that my existing workstation just was not cutting it.

When I moved recently to NYC, I decided to leave my old Broadwell- 12 core workstation back home in New Zealand, and just 'make do' with my surface and a cheap iMac ( and more recently a cheap pc) to handle my work. 

Now after an insanely busy few months of travelling and shooting, I've given in and said enough is enough. My files are now in the ranges of 70-150mb raw, 300mb as a single layer tiff, and 800mb-1.5gb by the time the images were finished. Now multiply by hundreds of deliverables every week and you see why I was tearing my hair out!

 

So now, short of buying a mac pro, I needed something that would work for the next 5+ years and just power through everything. GPU isn't important for my work except Capture One does gain a fair boost in speed with at least a single half decent card. But multithreading for heaps of tabs, windows and scrolling around high-resolution images is the key. This build is still in its infancy as one of the key considerations was minimum downtime while migrating to the new system so I took some shortcuts with coolers, fans and other non-essentials to ensure I had a working machine ASAP . And happy to say the build took 5 hours and we're back up to full speed!

 

This build came to a total of $1300 USD, about the same price as a single cpu with the same performance as this machine. I've done some quick benchmarks while doing other stuff (read, not entirely accurate but ballpark) but the cpu benchmarks look like my goal was achieved. An entire build with High end cpu performance but less than the cost of the cpu alone.

Will update as I do more.

Cinebench r20:7105

PassMark CPU: 26099.5

 

Below I will list my parts and possible future upgrades as this year progresses.

My parts are as follows:

Mobo: AsRock Rack EP2C612 WS

CPU: 2x Xeon E5-2680 V3, 12-core 2.5ghz | Future upgrade to 18, 20 or 22 core Xeon V4 Generation (not really necessary but would be fun when they get cheaper lol)

Ram: 8x 4gb DDR4 RDIMM ECC 2133mHz | Future upgrade to 8gb or 16gb sticks. 32 is normally ok if I don't have too many photoshop tabs open, but more would be nice since there is practically no limit to how much RAM I can install with this build)

Cooler: 2x Noctua Dx14 for Narrow ILM | Future upgrade to dual 320mm rads, but might be moving soon so didn't want the hassle.

GPU: 1x Nividia GTX 980 | Future plan in the coming weeks and months upgrade to quad-sli 980. Not necessary, but with them so cheap, just wanted to play!

PSU: EVGA G2 Gold 1000W | Probably need to bump up to 1600w for quad-sli...

Case: Thermaltake View 71 Snow

Storage: Just a little 240gb Sandisk SSD and the 1Tb HDD I pulled out of the old PC. Everything else is External and on NAS.

 

Some notes during the build:

Excuse the python of an atx cable going in a beeline from PSU to motherboard. Unfortunately this big case and monster SSI EEB sized motherboard didn't allow me to route it out the back.

I have an longer cable coming in the mail tomorrow!

 

Motherboard shorted on first start up due to some standoffs for a standard ATX board I left in.

 

This case is an absolute delight to build in. It is so roomy as you can see by the massive gaps on the right, even with a EEB sized motherboard covering all but the bottom gromet. This is also why some of my cable routing is a bit ugly since there were no gromets to simply hide cables through. Some cable-tie cable combs were made on the back side to tidy things up, will add a photo of this tomorrow.

 

Some ram didn't immediately register but after reseating, all is well.

 

The bottom PCI-E slot won't hold a GPU because the headers are in the way, however this case has a vertical mount bracket which I plan on using later for sli.

 

 

See below some images from the build! Feel free to ask any questions and suggest some improvements I could add.

First image is my workstation, minus my 4k Eizo that normally sits in the middle but is currently packed in a Pelican to go on set tomorrow. Excuse the mess! Haven't completely cleaned up or cable managed after the new build.

So this machine typically is driving 4 monitors. 1 at 4k, the wacom at 2.5k and then the 27"iMac and the cinema display monitors 3 and 4. (iMac is in target display mode)

 

 

[jaslion]

It's pretty neat but this is kinda a lot less than ideal situation for dual xeon. Capture one still is heavily single threaded so a cpu with much better single threaded performance would have been a better choice. Also dual cpu systems means that a lot of applications simply cannot use more than one cpu at a time so if you ever start maxing out one cpu you might just see the other not doing anything as the programs just don't understand how to use it.

Quad sli is quite frankly throwing away money as well nothing ever has really supported it but you do you.

 

I would have just gone with a ryzen system for the same money and whilst theoretical performance would have been the same or maybe a tad lower real world performance for your usecase would have been a lot better.

 

Otherwise neat build.

[George Li]

9 hours ago, jaslion said:

It's pretty neat but this is kinda a lot less than ideal situation for dual xeon. Capture one still is heavily single threaded so a cpu with much better single threaded performance would have been a better choice. Also dual cpu systems means that a lot of applications simply cannot use more than one cpu at a time so if you ever start maxing out one cpu you might just see the other not doing anything as the programs just don't understand how to use it.

Quad sli is quite frankly throwing away money as well nothing ever has really supported it but you do you.

 

I would have just gone with a ryzen system for the same money and whilst theoretical performance would have been the same or maybe a tad lower real world performance for your usecase would have been a lot better.

 

Otherwise neat build.

I agree with most although I've found that throwing cores at Photoshop speed ups the rendering - especially heavy filters to 1-2 seconds as opposed to 20-30 seconds before. And capture one usage isn't really affected like you said except when I'm batch processing a couple hundred photos. What used to be a 2-3 hour background task is now 20 mins  honestly for $200 for this matched pair of cpus I can't really complain. I did look at doing a standard build but couldn't find any cpus for a similar price and similar performance.

 

And yeah, quad-sli is impractical in every sense, but at $100 per gtx 980, its purely for the inner nerd than anything practical lol. To be honest a fair portion of this build was purely for the sake of building a dual socket system for dirt cheap!

 

[GDRRiley]

with your PSU max i'd go is 3 way SLI.

if you really want SLI just for for a 2 way config.

they are cheap because they get destroyed by a 1070 which has a lot more VRAM, which helps at above 1080P.

[George Li]

1 hour ago, GDRRiley said:

with your PSU max i'd go is 3 way SLI.

if you really want SLI just for for a 2 way config.

they are cheap because they get destroyed by a 1070 which has a lot more VRAM, which helps at above 1080P.

yea i will test will a wall power meter as well to see how much overhead I have as I add more cards. I don't really care about VRam since I don't game - just want it for hardware acceleration

[GDRRiley]

Just now, George Li said:

yea i will test will a wall power meter as well to see how much overhead I have as I add more cards. I don't really care about VRam since I don't game - just want it for hardware acceleration

that won't really tell you as you have to factor in the efficiency,

many programs use VRAM to hold the data for acceleration I don't know how light room and photo shop do it. .

[George Li]

3 hours ago, GDRRiley said:

that won't really tell you as you have to factor in the efficiency,

many programs use VRAM to hold the data for acceleration I don't know how light room and photo shop do it. .

yeah Photoshop only uses one gpu anyway so no added value there. again, not really a practical addition - more for fun than anything. And don't know about lightroom, I don't use it.