64-Core Budget Build

[TurbulentWinds]

I can't believe Linus did this. I've been planning one of these for about a year now. I can't go forwards now. It's too mainstream 

[themctipers]

just buy some 18 core atoms and strap them together.

[TurbulentWinds]

Just now, themctipers said:

just buy some 18 core atoms and strap them together.

I stuck 4 Raspberry Pi's together. Close enough . . .

[themctipers]

Just now, TurbulentWinds said:

I stuck 4 Raspberry Pi's together. Close enough . . .

i hot glued  and electrical taped my lightning cable back to life.,

[OfficialLament]

Didn't really need to use two PSUs just to power the four CPUs, they could've used a molex to EPS adapter.

They're extremely difficult to find and in my socket 1366 dual-CPU gaming rig build last year it took me a good few days to locate one on Amazon UK (DeLOCK EPS - 2 x 4 pin - power cables (Male/Female, EPS (8-pin), 2 x Molex (4-pin), Straight, Straight, Copper)).

[power666]

I was once considering building one of these years for a cheap 32 core box (four 8 core chips) until I realized that I'd be spending way too much on software licensing to actually use it with Windows 7.  I did however settle for an Asus KGPE-D16 ( https://www.asus.com/us/Commercial-Servers-Workstations/KGPED16/ )  since the professional version of Windows 7 worked fine across that board's two sockets.

 

First off, there is going to be some sever NUMA trade off with these generation of Opterons.  Like Threadripper of today, that generation of Opterons had two dies per socket.  Two sockets equates to four NUMA nodes and the four socket board shown would have 8 nodes.  To get maximum performance, your applications have to be NUMA aware.  If not, playing around with processor affinities to lock an application to the specific numa node where application memory resides can boost performance.  Similarly you need to tweak the BIOS for the memory mode that bests suits your applications.  Performance can swing wildly depending on these settings.  (Example:  https://www.anandtech.com/show/4486/server-rendering-hpc-benchmark-session/6 ).

 

The other thing is that Windows needs some tuning once you get 32 logical processors in Windows.  The old Windows scheduler would only permit a maximum of 32 concurrently running threads per process.  The maximum thread count per process under this schedule was generally limited by memory but only 32 of them could actually be doing work before context switching to the next group.  The way to see a performance gain was to run two instances and each got 32 logical processors each on a 64 core box.  Now there is a different scheduler but it too has various quirks.  This time the limit is 64 concurrent threads per application before it has to be re-written to support more (see:  https://msdn.microsoft.com/en-us/library/windows/desktop/dd405503(v=vs.85).aspx ). 

 

This also ignores individual application settings that affect performance.  Around 32 threads, Amdahl's Law applies here so that some fast single threaded systems were completing a work unit quicker than the application was able to dispatch all the workloads.  To put it another way, the core at the front of the dispatch line was able to finish and return to the line before the last core was handed work to do.  The result in this scenario is that not all the cores were actually being loaded even though they technically scale to that thread count.  Tuning work unit/tile size etc. to reduce dispatch overhead can help performance on high core count/threaded systems.

[airdeano]

would have been a better place to notify us in your status update.

<-LOCKED->