Jump to content

VMs server with GPU passed throught

SMaiz
By SMaiz
in New Builds and Planning
 Share
Posted

Hello everyone !

I'm planning to build a new setup, but I need your help !

First, I need to mention that I live in France and I have a budget around 2000€ and I don't need any peripherals.

The idea is to build a server from which I can run VMs. I plan to use Proxmox as an hypervisor, because it's free and I'm used to it.

On this server, I'll have a VM running windows 10 which will run Solidworks with a passed through GPU.

The others VMs are mainly GNU/Linux running webservices or some sort of simulations.

I need to mention that this server will not run any critical work, I can handle some down time.

 

So, I'll need a lot of cores and RAM and here is my first problem. Should I go for a Xeon, two Xeon, a Threadripper (an EPYC ?) ? I am a little bit lost. (By the way, I think ECC memory is almost required for this kind of builds ?)

I think for now something between 64 and 128 GB of RAM should be enough, but I think it's better if I can add some more later.

About the case, it doesn't really matter, but it might be cool to have it in a rackable format. But that's my last priority

Concerning the storage, I'm thinking about using a ZFS array ans a (relatively) "small" SSD for OSes. But I'll be able to expand this in the future if needed.

One last thing about the noise: I don't really care. I mean, yes, I don't want an airplane, but if the build makes a little noise, it is not a real problem if it can save me some money to have better hardware elsewhere.

 

That's all I can think of ;)

 

Thanks in advance for your suggestions !

Link to comment
https://linustechtips.com/topic/1080661-vms-server-with-gpu-passed-throught/
Share on other sites
Link to post
Share on other sites
Posted
8 minutes ago, SMaiz said:

rackable format

NOt with GPU passthrough you won't, unless you like full-sized rackmount cases....

And my preference is ecc and Xeons all the way

NOTE: I no longer frequent this site. If you really need help, PM/DM me and my e.mail will alert me. 

Link to post
Share on other sites
Posted

Unless you plan multiple systems, I'd stick with a conventional case. Much better cooling and easier to work in.

 

I don't believe more than 128GB of main memory is presently available on consumer grade hardware.

80+ ratings certify electrical efficiency. Not quality.

 

Link to post
Share on other sites
Posted
  • Author
On 7/10/2019 at 3:39 PM, brob said:

Unless you plan multiple systems, I'd stick with a conventional case. Much better cooling and easier to work in.

 

I don't believe more than 128GB of main memory is presently available on consumer grade hardware.

Yeah, I think a conventional case is better.

I don't know if I'll need more than 128 GB, it seems enough for my use case.

 

On 7/10/2019 at 2:47 PM, Radium_Angel said:

And my preference is ecc and Xeons all the way

Yeah, I thought about it, it seems quite a standard. But it doesn't look very cheap (?)

Link to post
Share on other sites
Posted
3 hours ago, SMaiz said:

But it doesn't look very cheap (?)

It sort of depends...

DDR3 EEC is *dirt* cheap (I picked up 64GB DDR3 ECC RAM for under 40$ US) but DDR4 is another story...

Ditto the Xeons. Current ones are eyewateringly expensive.

 

NOTE: I no longer frequent this site. If you really need help, PM/DM me and my e.mail will alert me. 

Link to post
Share on other sites
Posted
  • Author

Hey !
So, I thought about it a little a bit, and I've came with two builds, one with a Xeon, one with a threadripper.
I wanted to know what you're thinking about them:

Xeon build:

PCPartPicker Part List
Type Item Price
CPU Intel - Xeon E5-2680 V3 2.5 GHz 12-Core OEM/Tray Processor €700.00 @ Amazon France
CPU Cooler Cooler Master - Hyper H412R 34.1 CFM CPU Cooler €17.81 @ Amazon France
Motherboard ASRock - Fatal1ty X99X Killer ATX LGA2011-3 Motherboard €266.46 @ Amazon France
Memory Crucial - Ballistix Sport AT 32 GB (2 x 16 GB) DDR4-3200 Memory €190.79 @ Amazon France
Memory Crucial - Ballistix Sport AT 32 GB (2 x 16 GB) DDR4-3200 Memory €190.79 @ Amazon France
Storage Crucial - P1 500 GB M.2-2280 Solid State Drive €67.41 @ Amazon France
Storage Seagate - Surveillance HDD 2 TB 3.5" 5400RPM Internal Hard Drive €50.00 @ Amazon France
Video Card AMD - Radeon Pro WX 5100 8 GB Video Card €520.90 @ LDLC
Case Cooler Master - N300 ATX Mid Tower Case €37.69 @ Amazon France
Power Supply Thermaltake - Smart RGB 230V 500 W 80+ Certified ATX Power Supply €44.99 @ Amazon France
  Prices include shipping, taxes, rebates, and discounts  
  Total €2086.84
  Generated by PCPartPicker 2019-07-16 09:18 CEST+0200  

 

Threadripper build:

PCPartPicker Part List
Type Item Price
CPU AMD - Threadripper 2920X 3.5 GHz 12-Core Processor €428.90 @ LDLC
CPU Cooler ARCTIC - Freezer 33 TR (Black/White) CPU Cooler €49.90 @ Amazon France
Thermal Compound Arctic Silver - 5 High-Density Polysynthetic Silver 3.5 g Thermal Paste €7.90 @ Amazon France
Motherboard MSI - X399 SLI PLUS ATX TR4 Motherboard €309.98 @ Cdiscount
Memory Crucial - Ballistix Sport AT 32 GB (2 x 16 GB) DDR4-3200 Memory €190.79 @ Amazon France
Memory Crucial - Ballistix Sport AT 32 GB (2 x 16 GB) DDR4-3200 Memory €190.79 @ Amazon France
Storage Crucial - P1 500 GB M.2-2280 Solid State Drive €67.41 @ Amazon France
Storage Seagate - Surveillance HDD 2 TB 3.5" 5400RPM Internal Hard Drive €50.00 @ Amazon France
Video Card AMD - Radeon Pro WX 5100 8 GB Video Card €520.90 @ LDLC
Case Cooler Master - N300 ATX Mid Tower Case €37.69 @ Amazon France
Power Supply LEPA - MaxBron 800 W 80+ Bronze Certified Semi-modular ATX Power Supply €88.09 @ Amazon France
  Prices include shipping, taxes, rebates, and discounts  
  Total €1942.35
  Generated by PCPartPicker 2019-07-16 09:19 CEST+0200  

 

There is only 64 GB on non-ECC memory on both systems, but boths CPUs and motherboards are ECC compatible so I can update this later.

Link to post
Share on other sites
Posted

Using separate dual channel memory kits runs a small risk of not achieving quad channel memory operation.

 

XMS profiles are in part based on the number of modules in the kit.

 

I like to invest more in a quality psu, 

80+ ratings certify electrical efficiency. Not quality.

 

Link to post
Share on other sites
Posted
  • Author

I came to this:

PCPartPicker Part List
Type Item Price
CPU AMD - Threadripper 2920X 3.5 GHz 12-Core Processor €428.90 @ LDLC
CPU Cooler ARCTIC - Freezer 33 TR (Black/White) CPU Cooler €49.90 @ Amazon France
Thermal Compound Arctic Silver - 5 High-Density Polysynthetic Silver 3.5 g Thermal Paste €7.90 @ Amazon France
Motherboard MSI - X399 SLI PLUS ATX TR4 Motherboard €309.98 @ Cdiscount
Memory Crucial - Ballistix Sport AT 64 GB (4 x 16 GB) DDR4-3200 Memory €381.59 @ Amazon France
Storage Crucial - P1 500 GB M.2-2280 Solid State Drive €67.41 @ Amazon France
Storage Seagate - Surveillance HDD 2 TB 3.5" 5400RPM Internal Hard Drive €50.00 @ Amazon France
Video Card AMD - Radeon Pro WX 5100 8 GB Video Card €520.90 @ LDLC
Case Cooler Master - N300 ATX Mid Tower Case €37.69 @ Amazon France
Power Supply SeaSonic - FOCUS Plus Gold 750 W 80+ Gold Certified Fully Modular ATX Power Supply €100.05 @ Amazon France
  Prices include shipping, taxes, rebates, and discounts  
  Total €1954.32
  Generated by PCPartPicker 2019-07-16 17:27 CEST+0200  

I kept the threadripper because it seems better at a lower price, what do you think ?

Link to post
Share on other sites
Posted
  • Author
12 minutes ago, brob said:

The cpu cooler comes with a small amount of MX-4 thermal compound. 

 

Build looks decent. Have you researched Threadripper gpu passthrough issues and solutions?

Didn't saw this, thanks!
About the passthrought, I already looked at some topics and found different issues and solutions, but I guess I won't be sure until I try it with the real hardware. But since you mention it, I think it could be great to add another small GPU to have it as debug video output and to ensure the BIOS will leave my WX while booting

Link to post
Share on other sites
  • 2 weeks later...
Posted
  • Author

So, it's been a while.
I have received all the parts listed above (except the HDD, but it is not essential), and I have also assembled the entire PC.

All went relatively well, apart from the fact that my system did at first detect the 4 16GB RAM modules but show me only 32GB available (?). After reinstalling the RAM, all was good.
The Proxmox installation and VM setup was very easy and now all is working properly!

So thank you guys for your help!

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share
Go to topic listing New Builds and Planning

×