Good motherboard for AMD EPYC 7551 VM/NAS server?

[ImServs]

I am looking to upgrade my current VM/NAS server with more cores, which is leading me to the AMD EPYC platform. I have some Xeon knowledge when it comes to motherboards but none with AMD EPYC. Im looking for SSI-EEB, SSI-CEB, EATX, and ATX motherboards, used is fine, that supports the AMD EPYC 7551, 128gb ram, and at least x2 PCIe 3.0 16x lanes? Budget would be as low as possible but max $400, USD, USA.

 

Does anyone know any good options?

 

[jaslion]

You sure you want a zen 1 epyc?

 

Those were not really good and depending on usecase entirely unfit.

 

What will the server do besides being a nas? What will the vms run? First gen epyc struggled SEVERLY with single core tasks and correctly scheduling between its ccx's and hyperthreading making it pretty poor at single core tasks or multi core tasks that spilled over to another ccx as it could cause waiting times.

 

As for boards honestly any sp3 socket one will do. Just pick the cheapest supermicro you can find on ebay that is atx like.

 

Do keep in mind you'll need a decent modular psu as they do need all cpu power connectors plugged and normally psu's dont come with enough cables for that.

 

Cooler wise noctua made some sp3 coolers and those are about the only decent options for a desktop

 

Just an fyi but a 3950x beats it and doesnt have any of the weird gen 1 issues. Depending on the needed stuff to connect its pcie 4.0 lanes whilst only 24 can be enough?

[Needfuldoer]

5 minutes ago, jaslion said:

As for boards honestly any sp3 socket one will do. Just pick the cheapest supermicro you can find on ebay that is atx like.

One caveat: some first-gen Epyc motherboards don't have BIOS updates to handle Rome or Milan. If you want to upgrade to a better processor later as their prices come down, make sure they're supported.

 

7 minutes ago, jaslion said:

First gen epyc struggled SEVERLY with single core tasks and correctly scheduling between its ccx's and hyperthreading making it pretty poor at single core tasks or multi core tasks that spilled over to another ccx as it could cause waiting times.

Yup. That's why I'm sticking with my dual Broadwell setup for now; Naples would be a lateral move for me at best. Maybe in a couple years the PowerEdge R7525 will come down in price and I can consolidate my two R730s into one.

[jaslion]

5 minutes ago, Needfuldoer said:

One caveat: some first-gen Epyc motherboards don't have BIOS updates to handle Rome or Milan. If you want to upgrade to a better processor later as their prices come down, make sure they're supported

Right! Good catch forgot about that. Just because they are all sp3 doesnt mean they all can do it.

 

5 minutes ago, Needfuldoer said:

Yup. That's why I'm sticking with my dual Broadwell setup for now; Naples would be a lateral move for me at best. Maybe in a couple years the PowerEdge R7525 will come down in price and I can consolidate my two R730s into one.

Yup was only really truely fixed by zen 2 and the zen+ ones had a bandaid solution at best.

 

also the reason they are so cheap. These got replaced relatively soon due to these issues

 

Probably also why the xeon gold stuff is holding on longer than expected in value as the epyc counterparts from that time have these problems.

[Blue4130]

33 minutes ago, ImServs said:

I am looking to upgrade my current VM/NAS server with more cores, which is leading me to the AMD EPYC platform. I have some Xeon knowledge when it comes to motherboards but none with AMD EPYC. Im looking for SSI-EEB, SSI-CEB, EATX, and ATX motherboards, used is fine, that supports the AMD EPYC 7551, 128gb ram, and at least x2 PCIe 3.0 16x lanes? Budget would be as low as possible but max $400, USD, USA.

 

Does anyone know any good options?

 

Instead of looking based on motherboard size, look based on features. Personally, I'd go with a supermicro h11ssl.

 

Realistically, 99% of Epyc boards will support at least two slots of pcie3 x16 and 128gb of ram. It's much harder finding boards that don't.

[ImServs]

4 hours ago, jaslion said:

You sure you want a zen 1 epyc?

 

Those were not really good and depending on usecase entirely unfit.

 

What will the server do besides being a nas? What will the vms run? First gen epyc struggled SEVERLY with single core tasks and correctly scheduling between its ccx's and hyperthreading making it pretty poor at single core tasks or multi core tasks that spilled over to another ccx as it could cause waiting times.

 

As for boards honestly any sp3 socket one will do. Just pick the cheapest supermicro you can find on ebay that is atx like.

 

Do keep in mind you'll need a decent modular psu as they do need all cpu power connectors plugged and normally psu's dont come with enough cables for that.

 

Cooler wise noctua made some sp3 coolers and those are about the only decent options for a desktop

 

Just an fyi but a 3950x beats it and doesnt have any of the weird gen 1 issues. Depending on the needed stuff to connect its pcie 4.0 lanes whilst only 24 can be enough?

Im currently using the 5950x but have ran out of cores. The VM's are being used for game development testing servers and also general game hosting (a large variety of games at once). The goal was to upgrade to 32 cores and Threadripper is insanely priced compared to what it used to be at. 

[ImServs]

4 hours ago, jaslion said:

Right! Good catch forgot about that. Just because they are all sp3 doesnt mean they all can do it.

 

Yup was only really truely fixed by zen 2 and the zen+ ones had a bandaid solution at best.

 

also the reason they are so cheap. These got replaced relatively soon due to these issues

 

Probably also why the xeon gold stuff is holding on longer than expected in value as the epyc counterparts from that time have these problems.

would the AMD EPYC 7502 be a safer bet then? The price more than doubles, but its still cheaper than a 32 core used threadripper

[Blue4130]

15 minutes ago, ImServs said:

Im currently using the 5950x but have ran out of cores. The VM's are being used for game development testing servers and also general game hosting (a large variety of games at once). The goal was to upgrade to 32 cores and Threadripper is insanely priced compared to what it used to be at. 

How s the CPU utilization? Just because a core is assigned to one VM, doesn't mean it can't also be assigned to another as well. Overprovisioning is a wonderful thing.

[ImServs]

4 minutes ago, Blue4130 said:

How s the CPU utilization? Just because a core is assigned to one VM, doesn't mean it can't also be assigned to another as well. Overprovisioning is a wonderful thing.

I mean, with overprovisioning I could likely get away with 24 cores, but at the moment (not added all the servers), its at roughly 65% at peak times.

Edit: its really more about having the headroom if its needed then it is about utilizing the entire cpu immediately. 

[jaslion]

2 hours ago, ImServs said:

Im currently using the 5950x but have ran out of cores. The VM's are being used for game development testing servers and also general game hosting (a large variety of games at once). The goal was to upgrade to 32 cores and Threadripper is insanely priced compared to what it used to be at. 

Epyc and threadripper 1 is pretty terrible at game server hosting due to the major single core and ccx issues.

 

2 hours ago, ImServs said:

would the AMD EPYC 7502 be a safer bet then? The price more than doubles, but its still cheaper than a 32 core used threadripper

A 7503 is the first safe bet. The 7502 is a bandaid fix and whilst better still shows the behavior