planning Virtualisation Threadripper 4k Build

[./Sync]

I need to build a new Work slave, my old Rig is dying and will not be able to keep up with the future loads I need to be handled.

I am an avid Linux user that needs microshit to get things done, so I want to go with virtualization and direct throughput of my second GPU. I will use my 1070 that I already own for this.

 

My budget is around 4.5k, but I am willing to spend more if there is a reason to.

Work that will be done on this Rig will include:

Linux (hardened-arch - i3gaps)

•     Organisation (emacs, taskwarrior, etc.)
•     Web development & smaller python projects
•     Browsing and Office
•     Cybersecurity related Research using multiple VM's

Windows (7)

•     3D (Cinema 4D, Sculptor and similar)
•     Adobe Suite (After Effects, Premiere & Photoshop)
•     Plugin heavy 4k rendering & Motion tracking
•     Gaming, for recording & Streaming I will be using the Atomos Ninja

Now I am aware of the NTP bug with Threadripper and KVM, so, for the time being, I will go with ESIX instead. Are there any Downsides to that?

I am also a bit torn between Threadripper and Intel, factoring in Meltdown and Spectre and Adobes infertility to support new Tech.

After Effects scrubbing is a lot worse with a high amount of cores then with less, links to reference Videos:

Computer #1 has a 4,4GhZ Overclocked 6 core Intel CPU:
https://www.dropbox.com/s/mqqcm9ff9blfmck/AfterEffects_6core4.4Ghz.mp4?dl=0

Computer #2 has an Overclocked 1950X Threadripper 3.8GhZ CPU:
https://www.dropbox.com/s/lalh1gm0z0emxwe/AfterEffects_threadripper.mp4?dl=0

I am planning to the following setup of disk space/Devices:

Windows

•    M.2 for OS/Software/Games 1TB
•    M.2 for ProjectFiles
•    M.2 for Dedicated Cache

Linux

•    Single M.2 for everything

General

• 4TB data & 4TB backup / Archive
• 10TB Video Backup (Have a 100TB NAS 65% filled acting as Archive for this Drive)
• I also have a bunch of 500GB - 4TB drives flying around.

Monitors:

My primary requirements are IPS and 100% of the RGB space without light bleed.

Secondary but not optional would be at least 120hz for Gaming, preferably 144hz at least 2k - sadly AW is delayed with its 4k 144hz monitor which I will get if it holds what they promise.

Optionally I would love to give G-sync a shot

I want to have a 4 Monitor setup with a big TV, so I need 5 Outputs, one or two for Windos and three or four for Linux.

For the Hardware itself:

As already touched on the CPU... I wager to think that it is of more value to have those 32 threads to be able to make use of with virtualization? I hope to be able to OC to 4GHz.

4x16gb RAM @ 3200 to futureproof and maybe upgrade when prices normalize again

A GTX 1080 Ti and 1070 that I own already

I want a big case that can comfortably fit a custom loop. Oh yeah, did I mention custom water cooling?

*EDIT1:

Missed to enter my PcPartPicker link to what I would choose: https://pcpartpicker.com/list/kRdxLD

 

I am very grateful for any input, I will update this thread as we finalize the Plan and there will be a Buildlog with proper Cinematographic videos

Edited January 12, 2018 by ./Sync
Usecases Updated

[LtStaffel]

32 threads is overkill for these two machines. You could probably do with 16. Furthermore, for the tasks you have marked on Linux and considering how Linux is so gentle on hardware as opposed to Windows, I don't think it needs 32Gb of RAM. I think you could get away with 32Gb overall, allocating 24 to Windows and 8 to Linux.

[./Sync]

2 minutes ago, LtStaffel said:

32 threads is overkill for these two machines. You could probably do with 16.

Indeed, it probably is, but it will speed up rendering, and I research a lot of Cybersec related things that often require me to spin up a few VM's at a Time -> Updating the main post to reflect that.

[LtStaffel]

3 minutes ago, ./Sync said:

Indeed, it probably is, but it will speed up rendering, and I research a lot of Cybersec related things that often require me to spin up a few VM's at a Time -> Updating the main post to reflect that.

I know a professional cybersecurity guy and he runs over 5 VMs on a dual core pentium and 16Gb of RAM, you're overestimating how much power you need.

 

https://pcpartpicker.com/list/2J6md6

 

That build leaves you 4 extra RAM slots for an upgrade from 32 to 64 if you find you need it. You can always spend more later, but you might find that 32 is enough. I also added a second 1070 (both marked as purchased) just to take up the space for your 1080ti to ensure compatibility. Since 16 threads will be enough for what you are doing, I toned the Threadripper back down to that. I also put in a cheaper MOBO to save money because I saw no need for the one you had selected. You also had a PSU that was well over the required power, so I saved you a good bit on that while still leaving a nice overhead.

 

Overall, I cut $1500 off the pricetag.

[LtStaffel]

41 minutes ago, ./Sync said:

Now I am aware of the NTP bug with Threadripper and KVM, so, for the time being, I will go with ESIX instead. Are there any Downsides to that?

Horrible GPU passthrough. HyperV is good for putting a GPU through to a VM, it can even divide up the GPU among VMs instead of having to pass the whole thing through. Spend some of the $1500 I saved you in the last post and get a Windows 10 key, run HyperV, and pass the 1070 through to Linux that way.

[./Sync]

1 minute ago, LtStaffel said:

I know a professional cybersecurity guy and he runs over 5 VMs on a dual core pentium and 16Gb of RAM, you're overestimating how much power you need.

 

https://pcpartpicker.com/list/2J6md6

 

That build leaves you 4 extra RAM slots for an upgrade from 32 to 64 if you find you need it. You can always spend more later, but you might find that 32 is enough. I also added a second 1070 (both marked as purchased) just to take up the space for your 1080ti to ensure compatibility. Since 16 threads will be enough for what you are doing, I toned the Threadripper back down to that. I also put in a cheaper MOBO to save money because I saw no need for the one you had selected. You also had a PSU that was well over the required power, so I saved you a good bit on that while still leaving a nice overhead.

That opens a new perspective on things, much appreciated. I have 32gigs in my Rig atm, and I find having problems when rendering 4k, and I would like to render and do other things during that time. Also what leads you to the decision to use 2400 Mhz memory sticks?

[LtStaffel]

4 minutes ago, ./Sync said:

That opens a new perspective on things, much appreciated. I have 32gigs in my Rig atm, and I find having problems when rendering 4k, and I would like to render and do other things during that time. Also what leads you to the decision to use 2400 Mhz memory sticks?

Okay, then bump it back up to 64 but shop the right way. You have 8 slots on that MOBO so you can support as low as 8Gb per stick to get 64 (8*8=64). So go into the filters on PCPP and check everything that has 8Gb sticks, then sort by Price/Gb cheapest first. If an 8Gb stick pops up, add 8 of them. If a 16Gb kit of 2x8Gb pops up, add 4, etc. That way you don't pay an extra $100 just for having it be an all-in-one 64Gb kit.

 

Why not 2400? I saw Ryzen benefitted by a FEW FPS by having higher speeds, but does TR? I don't know. I doubt it. You're already at DDR4 and 2400 is miles faster than DDR3 so I really don't think it matters. Of course someone may come in and prove me wrong, I'm open to that. But I don't think you need it as far as what I know is concerned.

[./Sync]

8 minutes ago, LtStaffel said:

Horrible GPU passthrough. HyperV is good for putting a GPU through to a VM, it can even divide up the GPU among VMs instead of having to pass the whole thing through. Spend some of the $1500 I saved you in the last post and get a Windows 10 key, run HyperV, and pass the 1070 through to Linux that way.

What would performance loss look like with that option? I have the opportunity to start a well funded online content creation that requires a lot of work, for my benefit I would love to get it done seamlessly without waiting on some process hogging all of my resources. In a dream world scenario, I'd love to render a 4k vid, upload another and simultaneously record a 3rd.

4 minutes ago, LtStaffel said:

Okay, then bump it back up to 64 but shop the right way. You have 8 slots on that MOBO so you can support as low as 8Gb per stick to get 64 (8*8=64). So go into the filters on PCPP and check everything that has 8Gb sticks, then sort by Price/Gb cheapest first. If an 8Gb stick pops up, add 8 of them. If a 16Gb kit of 2x8Gb pops up, add 4, etc. That way you don't pay an extra $100 just for having it be an all-in-one 64Gb kit.

 

Why not 2400? I saw Ryzen benefitted by a FEW FPS by having higher speeds, but does TR? I don't know. I doubt it. You're already at DDR4 and 2400 is miles faster than DDR3 so I really don't think it matters. Of course someone may come in and prove me wrong, I'm open to that. But I don't think you need it as far as what I know is concerned.

I do not know exactly either, but I just thought that I could upgrade to 128 later (futureproofing) and if not I can always throw them on another rig?
I see your point, if no one objects to this I will go with the "slower" option.

[LtStaffel]

Just now, ./Sync said:

What would performance loss look like with that option? I have the opportunity to start a well funded online content creation that requires a lot of work, for my benefit I would love to get it done seamlessly without waiting on some process hogging all of my resources. In a dream world scenario, I'd love to render a 4k vid, upload another and simultaneously record a 3rd.

Not performance loss, we're talking ease of setup. I have a friend who has a server running 9 VMs all using one GT710 via HyperV hosted on Windows 10. I can ask him if you need details.

[LtStaffel]

2 minutes ago, ./Sync said:

What would performance loss look like with that option? I have the opportunity to start a well funded online content creation that requires a lot of work, for my benefit I would love to get it done seamlessly without waiting on some process hogging all of my resources. In a dream world scenario, I'd love to render a 4k vid, upload another and simultaneously record a 3rd.

Also, what do you need a GPU on the Linux VM for? I see no reason for it. Sell your 1070 and put the money back into this beast where it counts.

[./Sync]

6 minutes ago, LtStaffel said:

Also, what do you need a GPU on the Linux VM for? I see no reason for it. Sell your 1070 and put the money back into this beast where it counts.

GPU speeds for hashing simpler things, without having to shut down the Win VM and maybe have to reboot to fix the D3 sleep problem caused by the NTP bug.

 

 

As for the Threadripper, wouldn't it them be better to go with a Ryzen 7 @ 8 cores?

Also check my last response, edit happened!

[./Sync]

I just checked on HyperV, running Linux for privacy purposes on a Microsoft dom0 seems sketchy to me. I also want to clarify that I don't want to save money, but instead have a reliable Rig that never lets me down for the next 5-7 years, as I spend around 8h a day using it.

[LtStaffel]

52 minutes ago, ./Sync said:

I do not know exactly either, but I just thought that I could upgrade to 128 later (futureproofing) and if not I can always throw them on another rig?

I see your point, if no one objects to this I will go with the "slower" option.

You barely need 64 now. By the time you need 128 you'll be looking at a new build again.

36 minutes ago, ./Sync said:

GPU speeds for hashing simpler things, without having to shut down the Win VM and maybe have to reboot to fix the D3 sleep problem caused by the NTP bug.

 

 

As for the Threadripper, wouldn't it them be better to go with a Ryzen 7 @ 8 cores?

Also check my last response, edit happened!

Don't forward one GPU to Linux just to crack, run both on Windows and crack with double the power. Hashchat is multiplatform.

You could probably change to Ryzen if you wanted to.

29 minutes ago, ./Sync said:

I just checked on HyperV, running Linux for privacy purposes on a Microsoft dom0 seems sketchy to me. I also want to clarify that I don't want to save money, but instead have a reliable Rig that never lets me down for the next 5-7 years, as I spend around 8h a day using it.

You misread, I said to run Windows 10 as a host and Linux on a HyperV machine. No hypervisor. If you're actually worried about privacy, you're going about this whole thing wrong and you should be running something like Qubes or Tails on a burner laptop. Privacy is all or nothing and I don't want to type a giant wall of text explaining it when it is tangential here. Don't worry about privacy here, it won't work with this setup.

[./Sync]

31 minutes ago, LtStaffel said:

You barely need 64 now. By the time you need 128 you'll be looking at a new build again.

Don't forward one GPU to Linux just to crack, run both on Windows and crack with double the power. Hashchat is multiplatform.

You could probably change to Ryzen if you wanted to.

You misread, I said to run Windows 10 as a host and Linux on a HyperV machine. No hypervisor. If you're actually worried about privacy, you're going about this whole thing wrong and you should be running something like Qubes or Tails on a burner laptop. Privacy is all or nothing and I don't want to type a giant wall of text explaining it when it is tangential here. Don't worry about privacy here, it won't work with this setup.

Running Microsoft as Host is for me unacceptable, sorry. It is the force of habit that taints the moral Compass. I am sure you can understand that

Ofc, for E-banking etc. I have a minimal hardened Gentoo on a Laptop, still I don't want to feed any Corporations with meta that they don't deserve.

 

All of my workflow is based on Linux, and I am trying to keep the use of Windows as minimal as possible.

I am going to rest now, looking forward to more comments as I am still not confident about what I should order. Thankyou again for your Feedback LtStaffel, much appreciated

[LtStaffel]

18 hours ago, ./Sync said:

Running Microsoft as Host is for me unacceptable, sorry. It is the force of habit that taints the moral Compass. I am sure you can understand that

Ofc, for E-banking etc. I have a minimal hardened Gentoo on a Laptop, still I don't want to feed any Corporations with meta that they don't deserve.

 

All of my workflow is based on Linux, and I am trying to keep the use of Windows as minimal as possible.

I am going to rest now, looking forward to more comments as I am still not confident about what I should order. Thankyou again for your Feedback LtStaffel, much appreciated

BLUF (Bottom Line Up Front):
Using Windows as a host provides no more "meta" of worth than a VM in a hypervisor would.

 

Explanation:

I understand liking Linux better and wanting to be as free from Micosoft/Windows as possible, but your philosophy is inconsistent. Having a Windows host running while you primarily use a Linux VM for your workflow is no worse than having a hypervisor with Windows off. The only "meta" a Windows host would provide is "I'm powered on right now", which really shouldn't matter to you. If it does matter, then you're a target (journalist, state power, etc) and you should know all of this or have someone help you. If you use a Windows VM on a hypervisor then you will provide the same meta you'd provide by using Windows as a host and playing games/creating content or whatever you said to do.

Does that all make sense? I feel like I haven't presented my point clearly enough. I'm trying to say that the statement "Running Microsoft as [sic] Host is for me unacceptable" is unfounded and would not be an effective use of time to try to follow. You're discriminating between different proprietary softwares (Adobe's vs Microsoft's). If you actually were against it all, then you wouldn't bother having any form of Windows for any reason.

 

By the way, in case you think I'm not as hardcore as you in that belief, you're talking to someone who had Qubes installed as a daily driver on a laptop and who dual booted several obscure distros at different times on a main PC. I do believe in Linux