EPYC Server Build

[RockerBug17]

Budget (including currency): $2000 USD, or less

Country: USA

Games, programs or workloads that it will be used for: File server, Security system, Plex, Immich, VMs for gaming/rendering, probably more

Other details I won't bother you with anything but this: I am looking for a platform that can grow with me. The EYPC 7001/2/3 series of CPUs, from the lowly 7261 to the mighty 7773x gives me a ton of growing room. I know, it's probably overkill, and it's server grade equipment which means there's likely issues I'll run into (vendor locked, as example). Speaking of issues, that's mostly what I'm concerned with. I already mentioned vendor locked CPUs, but is there anything else I should contemplate before buying hardware? Not specifically with CPUs, but in general with running the whole platform (CPU, RAM, MOBO). I like to tinker and troubleshoot some, but only when I want to. I expect all other times for the system to just work. Please let me know if there's anything else you'd need to know to advise, and thank you.

 

EDIT: I thought I might clean up my use case scenarios for anyone that may be interested. Really, beyond a basic file server this is really a wishlist of things I may never actually do, but it sure would be cool to be able to.

• File Server
• Plex w/ 10 to 20 users, transcoding
• Immich w/ up to 10 users
• Local security system, looking at Scrypted
• VMs 
  • Mostly decommissioned machines I've collected over the years; typically powered off
  • At least one VM for cloud gaming and a dedicated GPU
  • A single VM for rendering (photo/video/3D)
  • Networking (pfsense, Pi-Hole)

I'll update this with anything else I think of. I'm certainly not trying to do all this from the onset, but again, having a clear upgrade path would make all of these things much more manageable for me.

[manikyath]

afaik there's nothing with vender locking epyc? i know it's been a thing for some TR pro chips, but that would be news to me with regards to epyc.

 

on that note.. your problems with epyc come in two parts:

- hardware availability is incredibly spotty, and the motherboard layouts are... weird? basicly, you'll need to plan your choice of motherboard around your plans for the platform, because there's very few motherboards that actually bring out all the capabilities of the platform.

- you're looking at 4 year old chips.. if these are on the second hand market at all, you're still likely gonna blow right trough your budget before you even started... especially with current RAM pricing.

 

i have a server built around an epyc 7313P, i did buy everything new so it's not entirely a fair comparison if you're hoping to score something second hand.. but i spent probably upwards of 4K on this build, basicly to the point that the aluminium 2020 extrusion for the custom chassis i built was peanuts on the total bill.

[Needfuldoer]

I ended up going Cascade Lake (Xeon Scalable 2nd gen) because performance is roughly equal to EPYC but the parts were cheaper and more abundant secondhand. 

 

Also, not all Naples platforms will also support Rome and Milan. The short explanation is that the BIOSes basically don't have enough flash space for the microcode to support them all. I considered a PowerEdge R7415 as my home server (2U, 12 LFF bays), but the performance per core wasn't any better than the Broadwell server I was using at the time and that model doesn't support Rome. (The nearly identical R7515 does, but it's much more expensive.)

 

1 hour ago, manikyath said:

afaik there's nothing with vender locking epyc? i know it's been a thing for some TR pro chips, but that would be news to me with regards to epyc.

Dell, HP, and Lenovo all vendor-lock EPYC. That can actually work out in your favor if you're running one of those platforms, because in general an unlocked CPU will cost more than one that's locked.

 

I don't believe SuperMicro boards lock CPUs, but they also won't work with CPUs locked to another vendor. (But I don't like their motherboards anyway.) 

[Dutch_Master]

1 hour ago, manikyath said:

afaik there's nothing with vender locking epyc?

Vendor locked EPYC is most certainly a thing, sadly

[RockerBug17]

2 hours ago, Needfuldoer said:

Also, not all Naples platforms will also support Rome and Milan.

Thanks for pointing that out. I assumed they were all at least forward compatible solely because they share a socket. In general it looks like there are boards that support Naples/Milan (7001/7002), or boards that support Milan/Rome (7002/7003) via BIOS update. Assuming that's more correct than not that narrows down my choices to Milan/Rome. Ideally I'd like to get a low end Milan with a motherboard that supports Rome.

 

I don't suppose you have any advice on single vs dual socket boards? My understanding is that dual socket boards require two CPUs to even post, and I don't know if each CPU needs memory or if it's shared. There's also the matter of if the software I use can even utilize multiple CPUs. I know I certainly (well, mostly certain) don't need a dual socket server, but since my goal is an upgrade path I might as well look into it.

[Needfuldoer]

1 hour ago, RockerBug17 said:

I don't suppose you have any advice on single vs dual socket boards? My understanding is that dual socket boards require two CPUs to even post, and I don't know if each CPU needs memory or if it's shared.

Every dual-socket motherboard I've every laid hands on will run just fine on a single CPU. You'll lose access to the PCIe and DIMM slots attached to that socket, but everything else works.

 

If you fill both sockets, then both CPUs have to be the same model. You can't mix different SKUs and have a low core count, high clock CPU in one socket and a core-dense, low clock CPU in the other socket. (At least that's how it's always been on Intel.)

 

If you populate both sockets, then both CPUs need memory. You get the best performance when you fill all the slots on a channel and spread the memory evenly across both sockets. (That's why you'll see used high-end servers with relatively low capacity DIMMs, like eight 8 GB DIMMs instead of two 32 GB DIMMs.)

 

The manual for the motherboard or server you get will tell you the optimum order to populate the slots. You can fill them in any order you want, but they won't run their best (and if you put all the RAM on one CPU and don't put any on the other, your machine will fail to POST.)

 

1 hour ago, RockerBug17 said:

There's also the matter of if the software I use can even utilize multiple CPUs.

In general, if your software supports multiple threads, it supports multiple CPUs. That's completely transparent to the software. There's just a little extra latency for memory reads if a process on CPU 1 needs information from the RAM connected to CPU 2.