intel (Intel E5-2650s - two of them) 32 threads

[Bendy_BSD]

Q: What would you guys do with 32 threads of processing cores, VT-X + VT-D enabled, and 32GB of RAM to use?

 

I ask because I initially planned to do the following but I wasn't sure if I'm underutilizing it or overutilizing it:

 

Windows 10 Pro 64 bit + PCI-Passthrough: NVIDIA Tesla vGPU, Marvell On-board RAID Controller, USB 3.0 card (first controller)

FreeNAS 64 bit + PCI-Passthrough: SAS controller in IT mode

Windows 10 Home 64 bit + PCI-Passthrough: GTX 1660 (NVIDIA), USB 3.0 card (second controller)

ReactOS vms (3) + PCI-Passthrough: NVIDIA Tesla vGPU
Host OS - Proxmox VE 6.0+

PfSense + PCI-Passthrough: onboard network controllers (intel quad port), dual port 10Gb SFP+ NIC.

 

(four cores per vm)

 

I'm honestly just curious because I'm kind of rethinking this entire thing, I'm not sure if I'm putting the hardware to good use.

[Electronics Wizardy]

What hypervisor? What tesla?

 

Reactos probably wont' work with vgpu, but have't tested it.

 

Id over allocate cores, so give each vm more if needed, they probably wont' run at full load all the time.

 

What board are you using?

 

Id probably get a newer platform if it was my. I have a dual 2011 system with dual e5 2680 v2's Im using now, but If Id do it again id just go threadripper.

 

[Bendy_BSD]

7 minutes ago, Electronics Wizardy said:

What hypervisor? What tesla?

 

Reactos probably wont' work with vgpu, but have't tested it.

 

Id over allocate cores, so give each vm more if needed, they probably wont' run at full load all the time.

 

What board are you using?

 

Id probably get a newer platform if it was my. I have a dual 2011 system with dual e5 2680 v2's Im using now, but If Id do it again id just go threadripper.

 

Hypervisor: Proxmox VE 6.0+ (64bit)

Tesla --->  NVIDIA Tesla k80 24GB VRAM

Motherboard: Asrock EP2C602-4L/D16

Reason for using the E5-2650s rather than later models within LGA2011 sockets: Lower power consumption; power effecient, less to pay for electrical bill. XP

 

But I can see where you're coming from with that.

 

ReactOS is an open source alternative to windows. And it will run windows drivers including executables.

[Windows7ge]

Whatever the VM's don't use you could throw into BOINC. That's what I have my PROXMOX server doing (dual 2670v1's).

[Bendy_BSD]

1 minute ago, Windows7ge said:

Whatever the VM's don't use you could throw into BOINC. That's what I have my PROXMOX server doing (dual 2670v1's).

~#: whatis BOINC ?

(what is BOINC?)

[Windows7ge]

1 minute ago, Bendy_BSD said:

~#: whatis BOINC ?

(what is BOINC?)

BOINC - Pick a science you'd like to support and crunch away. Containers are great for this. Hope you like CLI.

[Bendy_BSD]

1 minute ago, Windows7ge said:

BOINC - Pick a science you'd like to support and crunch away. Containers are great for this. Hope you like CLI.

Ahhhh I see.  Like protein folding simulations?

[Windows7ge]

Just now, Bendy_BSD said:

Ahhhh I see.  Like protein folding simulations?

Pretty much. There's World Community Grid, Einstein@home, YoYo@home, and a 1000 others. Pick what tickles your fancy the most. Some are just CPU compute some support GPU and vise versa.

[Bendy_BSD]

Just now, Windows7ge said:

Pretty much. There's World Community Grid, Einstein@home, YoYo@home, and a 1000 others. Pick what tickles your fancy the most. Some are just CPU compute some support GPU and vise versa.

Gotcha.   Thank you man. ^^

[Windows7ge]

Just now, Bendy_BSD said:

Gotcha.   Thank you man. ^^

If you decide to contribute let me know I'll help you set it up. I don't know where your skill level is with CLI as you'll want to use containers for this.

 

I also have your motherboard and plan to attempt GPU pass-though so maybe we can help each other here.

[Bendy_BSD]

7 minutes ago, Windows7ge said:

If you decide to contribute let me know I'll help you set it up. I don't know where your skill level is with CLI as you'll want to use containers for this.

 

I also have your motherboard and plan to attempt GPU pass-though so maybe we can help each other here.

I'll think on that.
 

Really?  Nice.    I was curious if you had by any chance tinkered around with the IPMI management?  I tried to get the IPMI set up to where it is supposed to use the dedicated Realtek port rather than the failover port on the intel network controller.  Segmentation of network traffic I mean.  It would be handy to have IPMI but not when it's on the same network as anything else, if my entire network was compromised and that IPMI function was exposed to the LAN as everything else is, one second for a deviant to rootkit the BMC and I'm totally screwed. y'know what I mean?   So out of security concerns I completely disabled the BMC management and disabled the BMC network interfaces.

[Windows7ge]

1 hour ago, Bendy_BSD said:

I'll think on that.
 

Really?  Nice.    I was curious if you had by any chance tinkered around with the IPMI management?  I tried to get the IPMI set up to where it is supposed to use the dedicated Realtek port rather than the failover port on the intel network controller.  Segmentation of network traffic I mean.  It would be handy to have IPMI but not when it's on the same network as anything else, if my entire network was compromised and that IPMI function was exposed to the LAN as everything else is, one second for a deviant to rootkit the BMC and I'm totally screwed. y'know what I mean?   So out of security concerns I completely disabled the BMC management and disabled the BMC network interfaces.

I did set it to use the dedicated Ethernet port. I'll just have to re-look-up the setting. It's inside the IPMI.

[Bendy_BSD]

3 minutes ago, Windows7ge said:

I did set it to use the dedicated Ethernet port. I'll just have to re-look-up the setting. It's inside the IPMI.

is that before or after applying any bios/BMC updates?

 

I'm still using the firmware that was flashed straight from the factory (both stock bmc and bios).

[Windows7ge]

41 minutes ago, Bendy_BSD said:

is that before or after applying any bios/BMC updates?

 

I'm still using the firmware that was flashed straight from the factory (both stock bmc and bios).

I updated my BIOS. Being a VM server I expose to the Internet and use for testing I wanted to patch it for Spectre & Meltdown. I may have updated the BMC. Maybe not. Can't remember. The menu options haven't changed much if at all.

 

I can get you exact versions later tonight or tomorrow.

[Bendy_BSD]

1 minute ago, Windows7ge said:

I updated my BIOS. Being a VM server I expose to the Internet and use for testing I wanted to patch it for Spectre & Meltdown. I may have updated the BMC. Maybe not. Can't remember. The menu options haven't changed much if at all.

 

I can get you exact versions later tonight or tomorrow.

Gotcha.  Thank you man. ^^

[Windows7ge]

15 hours ago, Bendy_BSD said:

Gotcha.  Thank you man. ^^

Let's See:

BMC: V0.19.00

BIOS: V1.90

So I'm on the latest of both.

 

As for setting the dedicated LAN:

Dashboard -> Network Information (Edit) -> LAN Interface [Share LAN | Dedicate LAN] -> Dedicate LAN -> Save

 

This should do it. I wanted to verify it before telling you but everything was being a little buggy so I can't say 100% that this is the answer but it should be.

[Bendy_BSD]

53 minutes ago, Windows7ge said:

Let's See:

BMC: V0.19.00

BIOS: V1.90

So I'm on the latest of both.

 

As for setting the dedicated LAN:

Dashboard -> Network Information (Edit) -> LAN Interface [Share LAN | Dedicate LAN] -> Dedicate LAN -> Save

 

This should do it. I wanted to verify it before telling you but everything was being a little buggy so I can't say 100% that this is the answer but it should be.

gotcha.    Thank you so much man.  And the BMC & BIOS firmware can be found on the Asrock Website right? (Well, the Rack site not the regular consumer site.)

[SenKa]

20 hours ago, Bendy_BSD said:

 

ReactOS is an open source alternative to windows. And it will run windows drivers including executables.

In theory. It is alpha software and should be treated as such

 

Are you supporting the project, or just having fun with it?

[Bendy_BSD]

2 minutes ago, SenKa said:

In theory. It is alpha software and should be treated as such

 

Are you supporting the project, or just having fun with it?

both.

[SenKa]

2 minutes ago, Bendy_BSD said:

both.

Nice!

 

Anyways, for your system I think it is a very good fit for what you're looking to do other than the teslas. Is there an explicit reason for them in your usecase over Quadro or even RTX/GTX?

[Bendy_BSD]

18 minutes ago, SenKa said:

Nice!

 

Anyways, for your system I think it is a very good fit for what you're looking to do other than the teslas. Is there an explicit reason for them in your usecase over Quadro or even RTX/GTX?

Well, the reason for the tesla is because I do plan to have reactos as the vms to utilize the hardware so that it has dedicated video acceleration and it is powerful enough to where gaming is an option.  I have a few friends.  And the three friends that I have I know them in real life and they love to game like I do.    They have their own gaming machines but they're a little heavy and take up space for our LAN parties so what I decided to do is just build one physical machine with vms that have dedicated hardware allocated to them so all they have to do is remote-in and log into the reactos vms via steam-in home streaming and they can use their laptops (cheapo or decent) with a hardwired connection.   It saves space and physical effort.  I can't blame them, hell, my server is a little heaver than their gaming rigs but if I only have to move one heavy machine than all 4-7 (some of them have dedicated stream boxes that transcode their game footage on the fly but no need in doing that when there is software that can relay data to twitch and youtube concurrently right? :)) you know what I mean?

Plus, in the event where their gaming rigs are down for whatever catastrophic reason, I can give them a vpn profile to where they can remotely login and still game without worries.   Redundancy and convenience reasons (the tl;dr of this explanation.) 

 

that and reactos is easy to get than having to spill out more cash to get license keys for windows home editions... they rack up quick if I get three.  >.>

 

And reactos (even though it's alpha software) is lightweight and won't be resource heavy while active on the hypervisor.

 

As for the quadros... well, not enough pci-e slots available.  I only have 6 slots to work with:

x4 - usb 3.0 card

x16 dual port sfp+ nic (10gb)

x16 dual port mini SAS controller.

x16 amd firepro r5000 2gb

x16 tesla k80 24gb

PCI - none.

[Windows7ge]

28 minutes ago, Bendy_BSD said:

gotcha.    Thank you so much man.  And the BMC & BIOS firmware can be found on the Asrock Website right? (Well, the Rack site not the regular consumer site.)

Yep.

[SenKa]

1 hour ago, Bendy_BSD said:

-snip-

More than a valid enough reason then!

 

Do us a favor and post a build thread up so we can all enjoy some hardware porn, eh?

[Bendy_BSD]

11 minutes ago, SenKa said:

More than a valid enough reason then!

 

Do us a favor and post a build thread up so we can all enjoy some hardware porn, eh?

Alright. ^^   just give me a few minutes and I will post it on pcpartpicker.

 

EDIT: https://pcpartpicker.com/user/Bendy_BSD/saved/wm4YTW

[Bendy_BSD]

12 minutes ago, SenKa said:

More than a valid enough reason then!

 

Do us a favor and post a build thread up so we can all enjoy some hardware porn, eh?

Alright!  I added the list of the parts (just look at the last reply and you'll see where it says: "EDIT")