The $32,000 Mac Pro Killer

[GDRRiley]

35 minutes ago, Commodus said:

You'd think so, but I've seen multiple examples of apps gobbling up a lot of that memory.  A many-track audio edit can chew that up, for example, and that's not uncommon in studios.

yes some audio workflows can but there always is the step up to epyc systems with frequency optimized parts.

I've yet to see any video or photo editing need that much.

[hishnash]

1 hour ago, Commodus said:

You'd think so, but I've seen multiple examples of apps gobbling up a lot of that memory.  A many-track audio edit can chew that up, for example, and that's not uncommon in studios.

There are good reasons that Audio workstations are much more memory focused, users have all sorts of extra input devices that input devices that need to be converted in realtime to any one of millions of possible sound tracks samples, the last thing you want is when the user starts the play something on their keyboard that the system needs to go and read the 10GB sample for that instrument from disk (not talking qwerty here talking piano style keyboard).  You need the sound to come out with as little latency as possible so you load everything into ram. 

as an example a single Cello instrument https://www.vsl.co.at/en/Solo_Strings_Bundle/Solo_Cello_2 can be over 9.8GB then consider that you have a composition of 100 or more instruments and you have a few different types of each instrument so that it does not all sound to `copy past` repeated. You used to do this by having many seperate machines networked together with each machine handling only a few instruments. 

[mr moose]

46 minutes ago, hishnash said:

There are good reasons that Audio workstations are much more memory focused, users have all sorts of extra input devices that input devices that need to be converted in realtime to any one of millions of possible sound tracks samples, the last thing you want is when the user starts the play something on their keyboard that the system needs to go and read the 10GB sample for that instrument from disk (not talking qwerty here talking piano style keyboard).  You need the sound to come out with as little latency as possible so you load everything into ram. 

as an example a single Cello instrument https://www.vsl.co.at/en/Solo_Strings_Bundle/Solo_Cello_2 can be over 9.8GB then consider that you have a composition of 100 or more instruments and you have a few different types of each instrument so that it does not all sound to `copy past` repeated. You used to do this by having many seperate machines networked together with each machine handling only a few instruments. 

You don't use the entire sound set from those bundles, That one is so big because it is 2 complete sets of raw wave files. Each set has multiple recordings for every note in every octave in most conditions, each note likely has 20-40 recordings of every different type of playing (e.g staccato,tremolo, vibrato, trills etc).   The reason it is so big is because it contains enough to eliminate the need for any computer modeling or effects, which reduces audio artifacts and consequently reduces the size of the ram the computer needs,  thus  the vast majority of audio workstations only have 8-16G ram.     

 

 

 

 

 

 

[Luscious]

17 hours ago, GDRRiley said:

you need the VRAM for lots of workloads. Davinci reslove at 6K wants 16Gb of VRAM and at 8k wants 20-24gb. hell MI50 with 32gb almost make sense there

Very true, but look at the price of any recent top end Quadro card, it's insane unless you can write it off as a business expense, at which point it doesn't matter if you get a Quadro 8000 in multiples or go last gen V100S with HBM2 on board. A pair of RTX Titans with nvlink will still get you a decent 48GB of combined VRAM which you can do a lot with and is ideal for running your monitor(s). The big caveat is when you try to throw together cards from different micro-architectures - it may work but it most likely won't be officially supported or you will run into driver conflicts.

 

14 hours ago, leadeater said:

If you're buying a system with 32 cores for single core or low thread count performance you're doing it wrong. Also did you price check that CPU? Xeon 6250 and 4TB ram would be half the cost and still more expensive because 4 sockets.

 

I know why Intel makes these SKUs and it's actually not so much for frequency reasons it for those with memory intensive applications that don't benefit from core/thread count.

 

Spending 8 times as much for not that much more performance gain really is not a no brainer.

I never said quad socket systems were cheap, but where do you want to place your minimum? If it's 8 cores or 16 cores then you don't need quad socket or have no need for Windows 10 Workstation OS. My point was that you can get a system with 32 cores that are frequency optimized with high memory support that don't require a server OS/drivers and can also handle 4+ GPU's. Server grade hardware is what it is and Windows 10 Pro for workstation opens up the options for those who need a consumer OS to run on more than just 2 sockets and 8 sticks of RAM, or who have very specific requirements.

 

You don't buy an unlocked and overclocked W3175X for single threaded workloads either, yet you cannot avoid them, and a quad socket configuration would run circles around this one likewise. Bragging rights are a completely different can of worms.

[GDRRiley]

5 minutes ago, Luscious said:

Very true, but look at the price of any recent top end Quadro card, it's insane unless you can write it off as a business expense, at which point it doesn't matter if you get a Quadro 8000 in multiples or go last gen V100S with HBM2 on board. A pair of RTX Titans with nvlink will still get you a decent 48GB of combined VRAM which you can do a lot with and is ideal for running your monitor(s). The big caveat is when you try to throw together cards from different micro-architectures - it may work but it most likely won't be officially supported or you will run into driver conflicts.

 

I never said quad socket systems were cheap, but where do you want to place your minimum? If it's 8 cores or 16 cores then you don't need quad socket or have no need for Windows 10 Workstation OS. My point was that you can get a system with 32 cores that are frequency optimized with high memory support that don't require a server OS/drivers and can also handle 4+ GPU's. Server grade hardware is what it is and Windows 10 Pro for workstation opens up the options for those who need a consumer OS to run on more than just 2 sockets and 8 sticks of RAM, or who have very specific requirements.

 

You don't buy an unlocked and overclocked W3175X for single threaded workloads either, yet you cannot avoid them, and a quad socket configuration would run circles around this one likewise. Bragging rights are a completely different can of worms.

not everything plays nicely with NVlink. if you aren't vram limited dual 2080ti or if you don't use cuda radeon VII pro

 

our point was for a 32core system you can do better. you only need windows 10 pro to run dual epyc frequency optimized parts which are half to third the cost per CPU

 

you quad socket wouldn't vs dual 7F52 (16) or 7F72 (24) they just have a much better memory layout and less overhead to move between cores. I wouldn't be surprised if you quad xeon gold got beat by an OCed W3175X

 

[leadeater]

2 hours ago, Luscious said:

My point was that you can get a system with 32 cores that are frequency optimized with high memory support that don't require a server OS/drivers and can also handle 4+ GPU's.

Didn't sound like it since you were talking about quad socket systems which only come in server board configurations currently and have terrible I/O options, far worse than what was used in the video.

 

The difference between 28 cores and 32 core is minimal so just go with a single socket 28 core frequency optimized CPU and save yourself a huge amount of money and open yourself up to options that actually work for a workstation build.

 

2 hours ago, Luscious said:

You don't buy an unlocked and overclocked W3175X for single threaded workloads either, yet you cannot avoid them, and a quad socket configuration would run circles around this one likewise. Bragging rights are a completely different can of worms.

Then why not buy a dual socket 128 core EPYC system, that'll also run circles around the quad socket system. Yet you can avoid single threaded workloads here because these systems are for applications that don't have any of that. The Mac Pro would be more likely to encounter that but if you are in the professional or research workspace using these kinds of systems single thread is a non issue, you almost always buy these for a single workload single application anyway.

 

We have hundreds of these kinds of workstations across the university, zero are quad socket and very few are dual socket.

 

Crossing NUMA domains is a huge problem for a lot of workloads so that alone makes your quad socket idea a bit pointless but I don't really expect everyone to know that so it doesn't really matter.

[Luscious]

On 6/15/2020 at 5:05 PM, leadeater said:

Then why not buy a dual socket 128 core EPYC system

I already have a fantastic proctologist. I also have absolutely no need for software vendors lining up around the block for me to take it up the rear with per-core licensing fees.

 

On 6/15/2020 at 5:05 PM, leadeater said:

Crossing NUMA domains is a huge problem for a lot of workloads so that alone makes your quad socket idea a bit pointless but I don't really expect everyone to know that so it doesn't really matter.

Which software are you referring to? Are you multitasking that software? Benchmarks such as Linux kernel compile, z-ray, 7zip, openSSL etc. show performance scaling 3x-4x between single socket versus quad socket. A lot depends on HOW you use it, not just WHAT you throw at it.

[leadeater]

6 hours ago, Luscious said:

Which software are you referring to? Are you multitasking that software? Benchmarks such as Linux kernel compile, z-ray, 7zip, openSSL etc. show performance scaling 3x-4x between single socket versus quad socket. A lot depends on HOW you use it, not just WHAT you throw at it.

Yes but in the context of Windows like what was being talked about multi socket support at the application layers isn't that good generally and Windows can't compensate for that.

 

Thing is when it comes to data and research scientists the issue is largely to do with PCIe devices connected to different CPUs and signaling traffic going across NUMA domains and the UPI links between CPUs which are both latency constrained and bandwidth, quad socket is even worse since there is a third of the bandwidth between CPUs compared to dual socket or on lower end Xeons with only 2 UPI links half compared to dual socket. It's those types of reasons why the workstations purchased stick to single socket as introducing a GPU on another socket can and has slowed down the performance to less than a single GPU. This is one of the reasons NVLink exists as it helps remove those type of performance bottlenecks but depending on what you are doing not everything can go across NVLink or you may not be GPU accelerating at all.

 

Even on the large compute clusters with multiple sockets and many GPUs jobs are more commonly partitioned in to sizes that fit on a single socket, unless the person knows and has tested that it can span. Then you just run multiple jobs per compute node.

 

These are people writing their own CUDA code or other code using standard industry frameworks that everyone else uses; for things like AI, Image Analysis, Geological data, Molecular dynamics etc.

 

There's even software we have encountered that only run on Windows and must have a monitor connected and you cannot use remote connectivity or the application freaks out. We found this one out when were given a server with 8 GPUs on loan to try it out as replacement for some of these workstations, because of this the server was returned and we did not buy any.

 

Sometime software sucks or not everyone is highly experienced code writers and optimizers and neither is their primary field computer sciences either, some of that is just nature of change and so much science now days is computer reliant. 7zip and openSSL aren't used here and don't really represent these kinds of workloads, nothing wrong with those benchmarks and still useful just depends on what you are doing.

 

Really all I was saying is going with a quad socket setup isn't so much a no brainer as you mentioned, the cost is far higher so you could purchase more systems instead and then you have to be aware of the types of issues I've pointed out on top of the lack of quad socket motherboards designed for workstation build-outs. Just giving some insight from my experience supporting these kinds of systems at my university which offers Creative Arts and Media Studies courses, Sciences, Engineering, Economics etc etc as I am part of the IT Infrastructure Team so work with all of these departments understanding and providing the resources they need. We do have quad socket systems but these are servers in our datacenter.

[Luscious]

On 6/17/2020 at 9:46 PM, leadeater said:

Yes but in the context of Windows like what was being talked about multi socket support at the application layers isn't that good generally and Windows can't compensate for that.

 

Thing is when it comes to data and research scientists the issue is largely to do with PCIe devices connected to different CPUs and signaling traffic going across NUMA domains and the UPI links between CPUs which are both latency constrained and bandwidth, quad socket is even worse since there is a third of the bandwidth between CPUs compared to dual socket or on lower end Xeons with only 2 UPI links half compared to dual socket. It's those types of reasons why the workstations purchased stick to single socket as introducing a GPU on another socket can and has slowed down the performance to less than a single GPU. This is one of the reasons NVLink exists as it helps remove those type of performance bottlenecks but depending on what you are doing not everything can go across NVLink or you may not be GPU accelerating at all.

 

Even on the large compute clusters with multiple sockets and many GPUs jobs are more commonly partitioned in to sizes that fit on a single socket, unless the person knows and has tested that it can span. Then you just run multiple jobs per compute node.

 

These are people writing their own CUDA code or other code using standard industry frameworks that everyone else uses; for things like AI, Image Analysis, Geological data, Molecular dynamics etc.

 

There's even software we have encountered that only run on Windows and must have a monitor connected and you cannot use remote connectivity or the application freaks out. We found this one out when were given a server with 8 GPUs on loan to try it out as replacement for some of these workstations, because of this the server was returned and we did not buy any.

 

Sometime software sucks or not everyone is highly experienced code writers and optimizers and neither is their primary field computer sciences either, some of that is just nature of change and so much science now days is computer reliant. 7zip and openSSL aren't used here and don't really represent these kinds of workloads, nothing wrong with those benchmarks and still useful just depends on what you are doing.

 

Really all I was saying is going with a quad socket setup isn't so much a no brainer as you mentioned, the cost is far higher so you could purchase more systems instead and then you have to be aware of the types of issues I've pointed out on top of the lack of quad socket motherboards designed for workstation build-outs. Just giving some insight from my experience supporting these kinds of systems at my university which offers Creative Arts and Media Studies courses, Sciences, Engineering, Economics etc etc as I am part of the IT Infrastructure Team so work with all of these departments understanding and providing the resources they need. We do have quad socket systems but these are servers in our datacenter.

Thanks for the explanation. I know the issues with UPI and the upcoming Cooper Lake Xeons will have 6x links to address those among some other tweaks. I agree that a lot depends on what software you are running and how you have your PCIe devices connected. Most dual and quad socket boards are fairly "idiot proof" in that they won't push all your add in cards through a single socket, rather split them, but understanding the board you are buying as well as what cards you are adding will generally get around those hurdles.

 

Windows wouldn't be Windows if it didn't have it's quirks, but that's where I think MS has a lot to offer in the future with a dedicated workstation variant of their OS. If anything it's a way to keep software that only runs on consumer Windows and add in cards that don't have server specific drivers (high end sound cards are a great example) still available as tools for those professionals who need them. Remember also that it was only very recently that Epyc received Windows 10 OS support from vendors - 2P Xeon WAS your only choice.

 

In an ideal world we would have no need for Windows 10 Pro for Workstation or even want to use quad socket outside the server realm. But it would be a worthwhile look to see what can be done with a quad socket build that doesn't require a server operating system, especially jobs that can take advantage of 4+TB RAM as a scratch disk, massively parallel workloads split across sockets/GPU's and multitasking multiple large jobs - tasks that would otherwise bog down an existing 2P workstation. There's ALWAYS a market for high performance, more so among those who can throw $75K and even $250K at a PC and sleep well at night.

[benny_r_t_2]

Hi,

 

The links are reversed for the mobo.  The amz link point to newegg for example

"

Buy Tyan Motherboard

On Amazon (PAID LINK): https://geni.us/yIPi2O

On Newegg (PAID LINK): https://geni.us/rbtmB3W

"

 

 

never mind, I see someone already mentioned it, although still not corrected.

 

 

 

[CZTROLLOLCZ]

On 6/15/2020 at 3:25 AM, Commodus said:

External peripherals, too... you know, Thunderbolt drives, external music interfaces, things creatives use often.

 

The Mac Pro is useful beyond the OS.  It's very quiet even under most loads (useful for previewing media), it's easy to access and maintain, and it's still damn fast at tasks that don't absolutely depend on core count.  And yes, people can use 1TB of RAM right now.  Intense 3D renders, simulations, machine learning... and virtual machines, definitely virtual machines.  Besides, remember that this system will likely be in use for several years -- you really don't want to be forced to replace a system just because you cheaped out on the memory options.

 

Also, you do realize that everyone who depends on a high-end workstation for a living will laugh you out of the building for suggesting a Hackintosh, don't you?  You cannot afford unnecessary downtime if you're in that segment, and a Frankenmac with potential compatibility issues, glitches and a total lack of official support could easily produce unnecessary downtime.  If you need a Mac, get a Mac.  It's not worth being a cheapskate when your profession is on the line.

I have a cousin who has a career as a video editor/camera man. He has a mouse, keyboard, headphone amp/dac and a usb 3 sd card reader. Honestly unless youre using multiple macro keyboards you dont really need that many ports.

 

Honestly in my experience most people use high end headphones. There reason being you can have things like the AC running or people talking in the background. You can upgrade the PC very easily. Hell my cousin has been rocking the core setup for years and only really changing the GPU or the CPU and MOBO when something really better came out.

 

I dont think you understand that not everyone is doing creative stuff as an only career. Yes your IT department wont deploy Hackintoshes, but you at your own home can, not everyone has a strict one day release schedule and can afford the 1 hour per month of crashes (hell thats what premiere is already doing). Not everyone has an unlimited budget to throw at apple.

 

On 6/15/2020 at 5:35 AM, DrMacintosh said:

That's where it actually sells. Easy deployment. Got a staff of editors that need new machines? Mac Pro is a single purchase that requires minimal setup from IT. 

*if youre working with apple software

[Commodus]

51 minutes ago, CZTROLLOLCZ said:

I have a cousin who has a career as a video editor/camera man. He has a mouse, keyboard, headphone amp/dac and a usb 3 sd card reader. Honestly unless youre using multiple macro keyboards you dont really need that many ports.

 

Honestly in my experience most people use high end headphones. There reason being you can have things like the AC running or people talking in the background. You can upgrade the PC very easily. Hell my cousin has been rocking the core setup for years and only really changing the GPU or the CPU and MOBO when something really better came out.

 

I dont think you understand that not everyone is doing creative stuff as an only career. Yes your IT department wont deploy Hackintoshes, but you at your own home can, not everyone has a strict one day release schedule and can afford the 1 hour per month of crashes (hell thats what premiere is already doing). Not everyone has an unlimited budget to throw at apple.

I understand that there are some people who won't need those ports, but you really, really can't claim that anecdotal experiences make that true for every media editor.  Remember, the Mac Pro has to cover everyone from a (well-heeled) freelancer to studios producing blockbuster movies.  With a pro machine, it's better to have too many ports than to need a hub just to plug in another display or a media interface.

 

And you really don't get it: if you depend on your computer for a living, even if it's only part of your living, using a hacked-together machine is a fundamentally bad idea.  The money you save with cheaper parts or a slightly faster rendering time could be wiped out in an instant (and then some) if your system is knocked out of commission.  It's one thing to have a system fail due to a bad component or a glitch in your pro software; it's another to have it fail because you took an entirely unnecessary risk.

[ykv_1107]

Great