Server configuration tips for highly parallel workloads

[mldonkey]

Hi all,

 

My academic research requires a lot of computing, and is highly parallel. I currently use a server that has 128 cores, 1.5TB RAM, and 100TB storage. However I would like to move to a model where I have more control over the hardware as I do not own the server we use. I was wondering if it would be possible to configure a useful build within my constrained budget by sourcing used or refurbished parts.

 

Budget: My budget for this project is max $2500.

Location: I'm located in the US PNW. Any advice on sourcing parts or choosing vendors that offer the best value in this region would be greatly appreciated.

Server Use: The primary use of this server will be to run big parallel CPU-intensive workloads. An added bonus, would be if we can add a GPU down the line for GPU loads.

Peripherals: I plan to manage the server remotely, so I won't need any peripherals like a monitor, keyboard, or mouse.

Operating System: A flavor of linux.

Hardware bounds: > 512GB RAM; > 10TB *fast* SSD; > 10TB HDD. Max number of cores my budget will allow for, ideally > 64;

 

Is this at all feasible? I don't need the latest and greatest. And if not, how much would be the minimum $$$ I'd have to spend?

[jaslion]

Not possible with new hardware in budget.

 

Even used this is FAR out of budget. Its simply asking too much for too little money.

 

Like 32 cores with the single core performance of a i5 4690 or something with 256gb of ram and a couple terrabyte of ssd storage + 10tb hdd that is doable. But this will be OLD stuff and 32 cores is the limit. Also it will need a basic gpu because well onboard gpu sucks a lot.

 

Id just keep using the server for now sounds like its at least a xeon gold device which is way better than anything you can get for a mere 2500.

 

To simply have a chance you'd need 5k at least AND some luck finding used parts.

 

Like an epyc 7742 (zen 2 chip) is about 2400$ with board and that would be acceptable performance and then well the rest of the system is another easy 2000$

[mariushm]

There's a seller on eBay with good reputation that sells Epyc processors and motherboards pulled from working servers or new old stock, his store was also linked at ServeTheHome forums and replied to questions there as well

 

He also has cpu+mobo deals and ram for these servers.. here's the store : https://www.ebay.com/sch/i.html?_ssn=tugm4470&store_name=tugm4470&_oac=1&_sop=16

 

For example $5900 for  Supermicro H13SSL-N+AMD Genoa-X EPYC 9V33X（9684X OEM）96C/192T 1152MB 400W  : https://www.ebay.com/itm/176064038015

 

But anyway, more to your budget ... motherboards separately are $250-500, of course some can go into thousands

 

examples

 

$388  Gigabyte mz32-ar0 motherboard Rev 1.0 for supports the 7001 7002 7003 seriescpu | eBay

$388 ASUS KRPA-U16 motherboard supports AMD EPYC7002 series PCIE4.0 servers | eBay

 

$425 DUAL SOCKET Supermicro H11DSI dual-socket motherboard AMD EPYC server motherboard REV2.0, | eBay

 

$435 DUAL SOCKET Supermicro h11dsi-nt 10 Gigabit Ethernet port supports dualAMD epyc 7001/7002cpu | eBay

 

RAM is around 1.5$ to 2$ per GB, for example  $160 for 128 GB (2133 DDR4 ECC)  Applicable to amd epyc series 8 * 16gb 2rx4 pc4-2133p ECC ddr4-ECC REG | eBay   or  $215 for 128 GB 2666 Mhz :  Applicable to amd epyc series 8 * 16gb 2rx4 pc4-2666 ECC ddr4-ECC REG | eBay       or     460$ for 256 GB 2666 Mhz :  Applicable to amd epyc series 8 * 32gb 2rx4 pc4-2666 ECC ddr4-ECC REG | eBay

 

You could start with at least 2 or 4 sticks per CPU (depends on generation, some really need at least 4 sticks).

 

For CPU you'll want to be careful to see what generation the cpu is from, as some motherboards will support only 7001 and 7002, others support 7003 as well ... the number of the CPU won't tell you what generation cpu is. 

The CPUs with P in the name are single socket, "performance" versions ... if you go with dual socket board you want a cpu that can work in dual cpu configurations

 

You can get 64 cores / 128 threads but very low frequency per core, or you can get less cores but higher frequencies... up to you... if you get a dual socket mobo, you could start with a higher performance 32 core / 64 threads cpu and add one later when you have more money, along with ram for it.

 

For example, the $165  Epyc 7451 is 24 core / 48 threads but runs at 2.3 ghz base / turbo to 3.2 ghz, and can run dual socket  : AMD EPYC 7451 PS7451BDVHCAF 24-Core 48-Thread 2.3GHz Socket SP3 CPU Processor | eBay

 

The $200 Epyc 7601 is 32 core / 64 threads with 2.2 ghz base / 3.2 ghz turbo, also dual socket :

AMD EPYC 7601 CPU server processor 32 core 64 thread 2.2ghz | eBay
AMD epyc 7601 retail version 32 cores 64 threads 2.2g CPU server processor | eBay

Epyc 7402 is the newer Rome generation and for $350 you get 24 cores / 48 threads, 2.8ghz base, 3.35ghz turbo , dual socket support : AMD EPYC 7402 cpu processor 24 cores 48 threads 2.8GHZ up to 3.35GHZ 180w | eBay
 - but motherboard must support Rome generation processors.
$700 gets you  48 cores / 96 threads Rome generation  Epyc 7552  2.2 ghz base / 3.2 ghz turbo : AMD EPYC Embedded 7552 CPU processor 2.2GHz 48 core 96 threads 200w | eBay

Coolers are around 60-100$  ... you'll need a good power supply with 2 EPS (cpu 8 pin connectors) at the very least, you get that in 850w and higher power supplies but if you plan to go dual socket best to go for 1000-1200w .. that's around 300 dollars...

[unijab]

3 hours ago, jaslion said:

Even used this is FAR out of budget.

I think Jeff of Craft computing could do it.

 

Could likely get close to what you want in core count on an older Z440 workstation. Would just have to figure out what to do for storage and RAM

[Blue4130]

19 minutes ago, unijab said:

I think Jeff of Craft computing could do it.

 

Could likely get close to what you want in core count on an older Z440 workstation. Would just have to figure out what to do for storage and RAM

How could he do it any different than other people? Unless he calls in favors, used prices are what they are. 512 gb of ram is going to eat up near $1000 of the budget alone.

[jaslion]

2 hours ago, unijab said:

I think Jeff of Craft computing could do it.

 

Could likely get close to what you want in core count on an older Z440 workstation. Would just have to figure out what to do for storage and RAM

You need a z640 or z840 for dual cpu as a z440 is just a single socket.

 

Then xeon v4's have a max of 22 cores at very slow speeds once all cores are goin. Like basically a 3rd gen i3 due to having so many cores. Sure they are only 200$ a pop but they are that for a reason. They are quite slow.

 

Then ddr4 512gb is AT LEAST 1000$. 10tb of ssd depending on the co fig is 600-1000$+, 10tb hdd is like 300.

 

A dual cpu z640 is still 800$ for a basically blank one with no hardware.

 

No matter what you do it goes wildly over budget and you end up with a rig that a 7950x will easily beat in terms of cpu power for example.

 

Its just far too much to ask for too small a budget.

 

I dont know who this Jeff person is but if they are a content creator or something they'll most.likely have some contacts, will be doing this over a very long timeframe since this thing is fot content and doesnt need to be great or simply lie like most do.

[Needfuldoer]

53 minutes ago, jaslion said:

I dont know who this Jeff person is but if they are a content creator or something they'll most.likely have some contacts, will be doing this over a very long timeframe since this thing is fot content and doesnt need to be great or simply lie like most do.

He runs Craft Computing, a homelab channel on YouTube (and Floatplane).

[jaslion]

2 hours ago, Needfuldoer said:

He runs Craft Computing, a homelab channel on YouTube (and Floatplane).

Aha I've seen a couple of their vids and from skimming the latest vid I see where op is coming from. The epyc 7601 is A LOT cheaper since 32 cores and such are VERY common epycs where the high core counts are simply not hence the FAR higher cost. But there are a lot of hidden costs behind the cheap route

[mldonkey]

14 hours ago, mariushm said:

There's a seller on eBay with good reputation that sells Epyc processors and motherboards pulled from working servers or new old stock, his store was also linked at ServeTheHome forums and replied to questions there as well

 

He also has cpu+mobo deals and ram for these servers.. here's the store : https://www.ebay.com/sch/i.html?_ssn=tugm4470&store_name=tugm4470&_oac=1&_sop=16

 

For example $5900 for  Supermicro H13SSL-N+AMD Genoa-X EPYC 9V33X（9684X OEM）96C/192T 1152MB 400W  : https://www.ebay.com/itm/176064038015

 

But anyway, more to your budget ... motherboards separately are $250-500, of course some can go into thousands

 

examples

 

$388  Gigabyte mz32-ar0 motherboard Rev 1.0 for supports the 7001 7002 7003 seriescpu | eBay

$388 ASUS KRPA-U16 motherboard supports AMD EPYC7002 series PCIE4.0 servers | eBay

 

$425 DUAL SOCKET Supermicro H11DSI dual-socket motherboard AMD EPYC server motherboard REV2.0, | eBay

 

$435 DUAL SOCKET Supermicro h11dsi-nt 10 Gigabit Ethernet port supports dualAMD epyc 7001/7002cpu | eBay

 

RAM is around 1.5$ to 2$ per GB, for example  $160 for 128 GB (2133 DDR4 ECC)  Applicable to amd epyc series 8 * 16gb 2rx4 pc4-2133p ECC ddr4-ECC REG | eBay   or  $215 for 128 GB 2666 Mhz :  Applicable to amd epyc series 8 * 16gb 2rx4 pc4-2666 ECC ddr4-ECC REG | eBay       or     460$ for 256 GB 2666 Mhz :  Applicable to amd epyc series 8 * 32gb 2rx4 pc4-2666 ECC ddr4-ECC REG | eBay

 

You could start with at least 2 or 4 sticks per CPU (depends on generation, some really need at least 4 sticks).

 

For CPU you'll want to be careful to see what generation the cpu is from, as some motherboards will support only 7001 and 7002, others support 7003 as well ... the number of the CPU won't tell you what generation cpu is. 

The CPUs with P in the name are single socket, "performance" versions ... if you go with dual socket board you want a cpu that can work in dual cpu configurations

 

You can get 64 cores / 128 threads but very low frequency per core, or you can get less cores but higher frequencies... up to you... if you get a dual socket mobo, you could start with a higher performance 32 core / 64 threads cpu and add one later when you have more money, along with ram for it.

 

For example, the $165  Epyc 7451 is 24 core / 48 threads but runs at 2.3 ghz base / turbo to 3.2 ghz, and can run dual socket  : AMD EPYC 7451 PS7451BDVHCAF 24-Core 48-Thread 2.3GHz Socket SP3 CPU Processor | eBay

 

The $200 Epyc 7601 is 32 core / 64 threads with 2.2 ghz base / 3.2 ghz turbo, also dual socket :

AMD EPYC 7601 CPU server processor 32 core 64 thread 2.2ghz | eBay
AMD epyc 7601 retail version 32 cores 64 threads 2.2g CPU server processor | eBay

Epyc 7402 is the newer Rome generation and for $350 you get 24 cores / 48 threads, 2.8ghz base, 3.35ghz turbo , dual socket support : AMD EPYC 7402 cpu processor 24 cores 48 threads 2.8GHZ up to 3.35GHZ 180w | eBay
 - but motherboard must support Rome generation processors.
$700 gets you  48 cores / 96 threads Rome generation  Epyc 7552  2.2 ghz base / 3.2 ghz turbo : AMD EPYC Embedded 7552 CPU processor 2.2GHz 48 core 96 threads 200w | eBay

Coolers are around 60-100$  ... you'll need a good power supply with 2 EPS (cpu 8 pin connectors) at the very least, you get that in 850w and higher power supplies but if you plan to go dual socket best to go for 1000-1200w .. that's around 300 dollars...

Thank you this was incredibly useful as an entry point to the available options and prices. I think you're right, the best plan might be start small with a clear affordable upgrade path. I would love to hear your thoughts and recommendations (especially on compatibility) on the following build:

Base:

Dual AMD EPYC 7451 and MOBO $709

2x Coolers $72 (Any recommendations for smaller/better/cheaper coolers?)
RAM for $420 I can't tell if this is 2x 64GB (so 128GB) or 2x 32GB. If the former then that seems like a very good deal, and I'd get 1 per CPU for starting setup of 256GB RAM. The upgrade path here would be to 512GB RAM, which while short of the goal I think will be fine.

2x 4TB NVME SSD $576

4x 12TB HDD $320

Total: $2097 

 

I might switch the 7451's for 7601 for an extra $80.

 

The plan here would be to run tier caching on the SSDs and HDDs which reduces my overall SSD storage need actually. With RAID 5 on the HDDs. To compensate for the low ram I can run a 512GB swap file on the SSDs perhaps.

That leaves me with $400 for a PSU and enclosure, which seems doable especially if I stretch the budget by ~$100. Recommendations for efficient PSUs and/or discrete/low profile enclosures welcome (doesn't have to be a rack). 

Upgrade path would be mainly through the RAM later, a modest RTX GPU (that was never in the initial budget), and perhaps more storage? It seems like to get the core count I want I'd need a second dual socket motherboard so that would be significantly down the line. Unless this motherboard supports Rome, which could lead to just a CPU upgrade when prices drop on something like the 7H12.

Edited March 4 by mldonkey

[mldonkey]

14 hours ago, jaslion said:

Not possible with new hardware in budget.

 

Even used this is FAR out of budget. Its simply asking too much for too little money.

 

Like 32 cores with the single core performance of a i5 4690 or something with 256gb of ram and a couple terrabyte of ssd storage + 10tb hdd that is doable. But this will be OLD stuff and 32 cores is the limit. Also it will need a basic gpu because well onboard gpu sucks a lot.

 

Id just keep using the server for now sounds like its at least a xeon gold device which is way better than anything you can get for a mere 2500.

 

To simply have a chance you'd need 5k at least AND some luck finding used parts.

 

Like an epyc 7742 (zen 2 chip) is about 2400$ with board and that would be acceptable performance and then well the rest of the system is another easy 2000$

You are correct it is an Intel Xeon Gold device at base clock of 2.2GHz. I have definitely not committed to switching away from that server if it doesn't make sense. I'd be curious to hear your thoughts on the compromise build posted which is not unlike what you suggest.

[jaslion]

30 minutes ago, mldonkey said:

You are correct it is an Intel Xeon Gold device at base clock of 2.2GHz. I have definitely not committed to switching away from that server if it doesn't make sense. I'd be curious to hear your thoughts on the compromise build posted which is not unlike what you suggest.

 

33 minutes ago, mldonkey said:

Base:

Dual AMD EPYC 7451 and MOBO $709

2x Coolers $72 (Any recommendations for smaller/better/cheaper coolers?)
RAM for $420 I can't tell if this is 2x 64GB (so 128GB) or 2x 32GB. If the former then that seems like a very good deal, and I'd get 1 per CPU for starting setup of 256GB RAM. The upgrade path here would be to 512GB RAM, which while short of the goal I think will be fine.

2x 4TB NVME SSD $576

4x 12TB HDD $320

You WANT a tested kit. Mixing ddr4 ecc stuff at such capacities and potentially dimms will get flaky with functioning well. The listing is for 64gb it seems and 210$ is poor value there.

 

It is also Zen 1 which is a whole step back from zen 2 and it is noticeable as well as showing clear first generation issues with things like core scheduling. Hence why it is so cheap. Expect better single core than the xeon gold WHEN IT WORKS PROPERLY. It being zen 1 this is never a given. It is why threadripper 1000 and zen 1 epycs are avoided as they have poor scheduling hardware wise, performance inconsistencies and show that they are a first gen product.

 

On that note too zen 1 had a BAD memory controller so it is picky. Expect 2133mhz at best basically.

 

Will this be used in a "desktop" system? If so you can just get some noctua tr4 coolers or be quiets or cooler master ml's or ... plenty good not expensive things and dirt cheap used due to being niche. The 1u's are not an option if this is to be used in the same room as you due to EXTREME NOISE that will cause HEARING DAMAGE.

 

I strongly advice against getting used hdds unless all data is of ZERO IMPORTANTCE AT ALL AND CAN BE FULLY LOST WITHOUT ISSUE!!!!

 

for u.2 you need a controller I think this board has it. Make sure to have a backplane adapter to u.2 + sata power connector else you cant use those drives.

 

Btw you are still missing a case and psu here. A case to fit that board will be tricky not yoo cheap so add another 150$

 

Psu a corsair rm850w + a type 4 extra cpu power cable is like 120$

 

To get the core count you want on the sp3 socket you have in the temp build you just need some zen 2 chips. 64 cores are available no problem. However I'd really postpone this and wait till you can get a 64 core zen 2 on a dual socket board. Zen 2 epyc is noticably faster than intels competitive options of the time (one of the xeon golds you have access too right now which is probably a quad socket server you have a resource pool in).

 

Keep in mind sp3 has been replaced  y sp5 (where the GOOOOOOD zen 3 stuff is on) and now sp6 with zen 4 (partly).

 

Sp3 is coming down in value hard rn as the stuff has reached end of life in the server rooms and sp6 is 2x faster per core basically. Sp5 socket chips were also easily 50% faster since thats when amd figured everything out properly and it REALLY shows

 

It is very unlikly you can get a sp5 socket systen in budget but it would be what you wanted.

 

Just keep in mind sp3 is zen, zen + and zen 2. Zen 2 chips are what you want and zen 1 is ehat you wish to avoid. Zen 2 stuff can go up to 64 core a chip btw. Just make sure your board accepts them. Zen + is just a better memory controller mostly which is nice but all the weird performance inconsistencies stayed mostly.

[mldonkey]

5 hours ago, jaslion said:

 

You WANT a tested kit. Mixing ddr4 ecc stuff at such capacities and potentially dimms will get flaky with functioning well. The listing is for 64gb it seems and 210$ is poor value there.

 

It is also Zen 1 which is a whole step back from zen 2 and it is noticeable as well as showing clear first generation issues with things like core scheduling. Hence why it is so cheap. Expect better single core than the xeon gold WHEN IT WORKS PROPERLY. It being zen 1 this is never a given. It is why threadripper 1000 and zen 1 epycs are avoided as they have poor scheduling hardware wise, performance inconsistencies and show that they are a first gen product.

 

On that note too zen 1 had a BAD memory controller so it is picky. Expect 2133mhz at best basically.

 

Will this be used in a "desktop" system? If so you can just get some noctua tr4 coolers or be quiets or cooler master ml's or ... plenty good not expensive things and dirt cheap used due to being niche. The 1u's are not an option if this is to be used in the same room as you due to EXTREME NOISE that will cause HEARING DAMAGE.

 

I strongly advice against getting used hdds unless all data is of ZERO IMPORTANTCE AT ALL AND CAN BE FULLY LOST WITHOUT ISSUE!!!!

 

for u.2 you need a controller I think this board has it. Make sure to have a backplane adapter to u.2 + sata power connector else you cant use those drives.

 

Btw you are still missing a case and psu here. A case to fit that board will be tricky not yoo cheap so add another 150$

 

Psu a corsair rm850w + a type 4 extra cpu power cable is like 120$

 

To get the core count you want on the sp3 socket you have in the temp build you just need some zen 2 chips. 64 cores are available no problem. However I'd really postpone this and wait till you can get a 64 core zen 2 on a dual socket board. Zen 2 epyc is noticably faster than intels competitive options of the time (one of the xeon golds you have access too right now which is probably a quad socket server you have a resource pool in).

 

Keep in mind sp3 has been replaced  y sp5 (where the GOOOOOOD zen 3 stuff is on) and now sp6 with zen 4 (partly).

 

Sp3 is coming down in value hard rn as the stuff has reached end of life in the server rooms and sp6 is 2x faster per core basically. Sp5 socket chips were also easily 50% faster since thats when amd figured everything out properly and it REALLY shows

 

It is very unlikly you can get a sp5 socket systen in budget but it would be what you wanted.

 

Just keep in mind sp3 is zen, zen + and zen 2. Zen 2 chips are what you want and zen 1 is ehat you wish to avoid. Zen 2 stuff can go up to 64 core a chip btw. Just make sure your board accepts them. Zen + is just a better memory controller mostly which is nice but all the weird performance inconsistencies stayed mostly.

Thank you for your thoughts. I'm definitely going to keep Zen 2 in mind as the final target when those prices drop. You're also right, that RAM is quite expensive, and given the likes of this I'm actually somewhat confident I could fit 512GBs in the budget. Definitely have to identify a suitable case, I did find a chassis at $160 on Newegg but it was massive, that price point seems about right though.

[Blue4130]

7 minutes ago, mldonkey said:

Thank you for your thoughts. I'm definitely going to keep Zen 2 in mind as the final target when those prices drop. You're also right, that RAM is quite expensive, and given the likes of this I'm actually somewhat confident I could fit 512GBs in the budget. Definitely have to identify a suitable case, I did find a chassis at $160 on Newegg but it was massive, that price point seems about right though.

That ram is DDR3 and won't work with zen CPU's

[mldonkey]

9 minutes ago, Blue4130 said:

That ram is DDR3 and won't work with zen CPU's

Good catch! I am still browsing around for RAM but I was not paying close attention I guess! I appreciate any resources that list RAM $/GB, so far best deal I've found (DDR4 this time) is about $320 for 256GB.

[mldonkey]

As a placeholder I found this Antec consumer case rather than server chassis for $100. Amazon has a Corsair RM850 used for $98, and Ebay has one for $54. So excluding the RAM, running total is 1909 for the 7601 version of the build. That leaves a bit of breathing room to splurge on 512GB or more of RAM.

[mariushm]

6 hours ago, mldonkey said:

Thank you this was incredibly useful as an entry point to the available options and prices. I think you're right, the best plan might be start small with a clear affordable upgrade path. I would love to hear your thoughts and recommendations (especially on compatibility) on the following build:

Base:

Dual AMD EPYC 7451 and MOBO $709

2x Coolers $72 (Any recommendations for smaller/better/cheaper coolers?)
RAM for $420 I can't tell if this is 2x 64GB (so 128GB) or 2x 32GB. If the former then that seems like a very good deal, and I'd get 1 per CPU for starting setup of 256GB RAM. The upgrade path here would be to 512GB RAM, which while short of the goal I think will be fine.

2x 4TB NVME SSD $576

4x 12TB HDD $320

Total: $2097 

 

I might switch the 7451's for 7601 for an extra $80.

 

The plan here would be to run tier caching on the SSDs and HDDs which reduces my overall SSD storage need actually. With RAID 5 on the HDDs. To compensate for the low ram I can run a 512GB swap file on the SSDs perhaps.
 

 

Contact the seller and ask him to give you a price for dual socket mobo AND two 7601 (or look for a Zen 2 gen cpu) and the coolers and ram compatible with the motherboard he can recommends for dual socket. May help you if you say you also saw the threads on ServeTheHome : https://forums.servethehome.com/index.php?threads/amd-epyc-7302p-supermicro-h11ssl-i-version-2.37913/page-16

 

For ram, if you plan to populate all ram slots eventually, it's not worth going over 2666 mhz, boards won't support more with naples. With 8 ram slots if you go with 32 GB sticks, you'd be limited to 256 GB per socket ... 64 GB sticks still pack a premium over 32 GB sticks and if you want 64 GB sticks mention it to the seller, because some mobos support 64 GB sticks only with revision 2 of a board and some boards don't support 64 GB sticks at all with the Naples generation (which 7601 and 7451 belong to)

 

There's also some other minor gotchas, like some motherboards may not even boot without at least 4 ram sticks per cpu socket, or you get serious latency issues with less than 4 sticks.

 

The board in the 709$ bundled package is rev 2 so it supports  up to 2666 mhz ram and supports up to 256 GB sticks but as I said, 64 GB and higher sticks will cost more, and you'll need at least 2 possibly 4 per socket.

 

He may be willing to give you a better price if you go with a bigger bundle, and promise to get back to him to buy more ram.

 

if it helps, dual socket cpus, naples  (7001 series) :

 

24 cores

AMD EPYC 7401 - PS7401BEVHCAF / PS7401BEAFWOF

AMD EPYC 7451 - PS7451BDVHCAF / PS7451BDAFWOF

 

32 cores

AMD EPYC 7501 - PS7501BEVIHAF / PS7501BEAFWOF
AMD EPYC 7551 - PS7551BDVIHAF / PS7551BDAFWOF

AMD EPYC 7571 - PS7571BDVIHAF
AMD EPYC 7601 - PS7601BDVIHAF / PS7601BDAFWOF

 

 

dual socket 7002  

 

24 cores

 

AMD EPYC 7352 - 100-000000077 / 100-100000077WOF
AMD EPYC 7402 - 100-000000046 / 100-100000046WOF
AMD EPYC 7F72 - 100-000000141 / 100-000000141WOF

 

32 cores

AMD EPYC 7452 - 100-000000057 / 100-100000057WOF
AMD EPYC 7502 - 100-000000054 / 100-100000054WOF
AMD EPYC 7532 - 100-000000136 / 100-000000136WOF
AMD EPYC 7542 - 100-000000075 / 100-100000075WOF
 

48 cores

 

AMD EPYC 7552 - 100-000000076 / 100-100000076WOF
AMD EPYC 7642 - 100-000000074 / 100-100000074WOF

 

64 cores

 

AMD EPYC 7662 - 100-000000137 / 100-000000137WOF
AMD EPYC 7702 - 100-000000038 / 100-100000038WOF
AMD EPYC 7742 - 100-000000053 / 100-100000053WOF
AMD EPYC 7H12 - 100-000000055 / 100-100000055WOF

 

 

 

tables are from cpu world ... ex https://www.cpu-world.com/CPUs/Zen/AMD-EPYC 7662.html

 

you can sort of compare performance looking on cpubenchmark.net , see https://www.cpubenchmark.net/cpu_list.php

 

 

[jaslion]

38 minutes ago, mldonkey said:

As a placeholder I found this Antec consumer case rather than server chassis for $100. Amazon has a Corsair RM850 used for $98, and Ebay has one for $54. So excluding the RAM, running total is 1909 for the 7601 version of the build. That leaves a bit of breathing room to splurge on 512GB or more of RAM.

You'll need an eatx case that can fit a long board like that. It also needs to have gpod airlfow and the antec doesnt as its a mostly closed panel in the front. You'll want a mesh case here.

 

As fot eatx it is NOT  A STANDARD!!!! So read carefully what fits. A popular "eatx" case is a 5000D as its very long and fits a lot of boards whilst not officially being one.

 

Do not cheap on a psu its literally what keeps your system alive. A rm850i is MUCH older.Id not get it. The rm850e is from this year so there being a 40$ cheaper used unit is quiestionable.

 

The memory you found should work btw.

 

I really WOULD NOT get a epyc zen 1 system as they are literally buggy.

[mldonkey]

Thank you for all the advice. I hope this thread will serve anyone looking to cheaply build a multicore system. I will post here (through a later edit) the final quote and setup I settled on. I recently got some excellent news allowing for a 10K budget for these purchases (thank you Fatou). So I think I will heed your advice on CPU generations and also spec out a beefier build with includes a good GPU and newer socket.

 

Common to both builds (8TB NVME SSD; 48TB HDD; 1000W PSU):

- 1000W PSU for $100 (e.g. Corsair RM1000x)

- An (E-)ATX chassis for $100

- 2x 4TB NVME SSD $576

- 4x 12TB HDD $320

 

$2,500 Build (64 cores@2.2GHz; 256GB RAM):

- 2x Noctua CPU cooler for $200 (you could get cheaper coolers)

- 2x 7601 and MOBO for $880

- 256GB DDR4 RAM for $345

 

$10,000 Build (96 cores@2.4GHz; 768GB RAM; NVIDIA GPU w/ 24GB VRAM):

- single 9654 and MOBO for $4000

- NVIDIA A10 for ~2000 (found this great listing at $1750)

- 768GB of DDR5 4800MHz RAM for $3360

- CPU cooler for $100

 

[Needfuldoer]

If you're going with used Epyc, make sure the motherboards you get actually support Rome and/or Milan. Some early platforms (first revision Supermicro 11, Dell PowerEdge R6415/R7415/R7425) don't have enough BIOS space to support them without breaking compatibility with Naples.

 

That's one of the things holding me back from upgrading my dual R730 homelab (each with dual E5-2697 v4s, 512 GB of RAM, and six 1.6TB SSDs) into one chassis (with the intention of carrying over all 32 of my 32 GB DIMMs and all 12 SSDs).