[Help] My First Multiuse Home Server!

[latot]

Budget (including currency): 2500USD

Country: Chile

Games, programs or workloads that it will be used for: Super variable workloads, the only thing I don't plan to do here is gaming. I'll run Databases, Servers, VMs, Mathematic and Physics simulations and more....... I know it is hard to have 1 server for all this, but let's think just to have most of them.

I'm using Linux!Gentoo in particular. I only need the server, no need for peripherals or a  monitor. I already have them.

I would like to avoid used parts, but! there is some of them that will be hard to get new, so I'm open to buy used parts, I know CPU and mobo for now will be used ones.

CPU

I have checked about this in another topic, a summary would be buy an used Epyc 7742 from Ebay, they come from 800USD to 1100USD, and some offers can go down a little but not too much of them.

Ram

For the Ram, I need ECC, 3200Mhz and a lot of ram, ideally start with at least 128gb of ram. I have chosen to use RDIMM or LRDIMM. I don't get very well which one would be the best for this. and I don't know if it is worth buying used ram, as my first build I have no idea if they usually come fine.

Other point of the ram is the CAS, there several server rams in ebay, but no all of them says the CAS, I found one of samsung describing only basics like this one: https://semiconductor.samsung.com/dram/module/rdimm/m393a8g40bb4-cwe/

So... If I can't get the CAS, it can be complex in the future to get a ram with similar characteristics to avoid instability. Any ideas about how to handle this?.

Mobo

For the mobo... I wanted to have flexibility to upgrade... usually I found two types of mobo in relation to ram, one cpu with 16 slots and two cpus with 16 slots, from what I read if has two cpus with 16 slots each cpu has half of the ram, and we only can use that 8 slots of you have the second CPU, is a tradeoff.

While there are mobos with 32 ram slots, they are harder to find and buy.

In my case.... I would like the 2 cpus and 16 slots, just because I would like to play with 2 cpus, only if I have several PCI 4.0 x16 to use.

Oks, putting the Ram aside, choosing a mobo has been hard, I have not been able to choose one.

First there is a Asus mobo ASUS KRPA-U16, but checking the previous version there is pretty bad reviews, I was not able to found the link, maybe the major reasons is that I was not able to found any opinions/reviews about that model while there is a gigabyte one MZ32-AR0 Motherboard REV 1.0 with a review: https://www.servethehome.com/gigabyte-mz32-ar0-amd-epyc-7003-motherboard-review/2/

I was thinking of going for a SuperMicro, the mobos 7001/7002 comes with PCI3 while 7002 and 7002/7003 comes with PCI4.

I'm not able to choose between the Gigabyte or SuperMicro, I'm not able to compare them due to my lack of experience, but there is something weird, is that GigaByte one (with one cpu) has more PCI4 than SuperMicro dual cpu! I'm pretty confused there. If you think both of them are great, I can choose from there.....

For now I'll have only one CPU, so it would be a good option to choose a mobo of 1 slot and if I need to upgrade in the future buy another mobo dual cpu.

Cooling CPU

I live in a country with very intense earthquakes.... So I need to... handle that in some way or avoid liquid cooling. I was thinking in Noctua...

Checkig this: https://ncc.noctua.at/cpus/model/AMD-EPYC-7742-1188

The NH-U14S DX-4677 seems to be the best option in https://www.amazon.com/-/es/NH-U14S-DX-4677-enfriador-prémium-LGA4677/dp/B0BSXP2G72 for 120USD but seems to not be too avaliable.

Case and Cooling

Oks... here... I have no idea, my top priority is the server efficiency... So if there is a way to look good is fine, but ideally not use too much budget in that.

The cooling needs to be silent.... So again I was thinking Noctua fans... if someone has a good but not that expensive idea I'm open to that.

For the case, no idea which one would be a good one, need to be EATX because practically all the mobos for Epyc are EATX or a private format. So looking for a case with good airflow.

I was thinking about using racks too, but I don't have the space for that right now, but it would be good to know at least where to search to think about it.

Storage

Oks, I want to use NVME with Raid.... but no idea which ones to use, the Mobos does not support too much NVME, have moro SATAs... maybe would be better use SATA instead of NVME? in order to use NVME the bease would be a PCI card extension for NVME with Raid... There are some Asrock expansion cards, but no idea which ones would be the best and handle the bootleneck due to the NVME super speeds.

Now I already have a 2TB NMVE to use, this storage can be thought of in the long term, so should be included in the actual budget.

Power Supply

A Epyc 7742 consumes at max 225W... Well I'm pretty confused here.

Pick as example: https://www.supermicro.com/en/products/motherboard/H11DSi

It supports max 240TDP... with two Epyc 7742... but that is 450TDP.... how the mobo supports less power than required for the dual cpu....

Putting that aside, I don't know the best power supply here, but take in consideration the server will be running 24/7 during months, so need to be stable and the power source be able to handle any voltage variability over the time.

Why not just go and look at a power supply titanium? I remember a LTT video where explain that some power supplies are not good but are constructed in a way to get the certifications, so I prefer to ask.

Extension Cards

I would need to extension cards right now.

A video card, how I will not play, just with a basic one, if there is something you can recommend I would appreciate it, just to avoid some bad products here.

An internet card... ideally with Wify 6..... think I'm using Linux...

UPS

No idea which one would be the best here... I can read the basic informacion but I lack the basics of what to see to be able to trust in a UPS.

Well, here we have some power outages from time to time, and voltage fluctuation, so it is important to have a UPS that is able to balance all of this.

I'm very tempted with the rack ones, and instead of a case, use a rack, but I have been having a hard time finding racks and where to buy that type of things.

Thx!

Edited June 27, 2023 by LogicalDrm
Fixed formatting

[brob]

Why do you want to RAID the NVMe drives?

 

Consult https://cultists.network/140/psu-tier-list/ for guidance on ATX and SFX PSU.

[Somerandomtechyboi]

36 minutes ago, latot said:

Ram

For the Ram, I need ECC, 3200Mhz and a lot of ram, ideally start with at least 128gb of ram. I have chosen to use RDIMM or LRDIMM. I don't get very well which one would be the best for this. and I don't know if it is worth buying used ram, as my first build I have no idea if they usually come fine.

Other point of the ram is the CAS, there several server rams in ebay, but no all of them says the CAS, I found one of samsung describing only basics like this one: https://semiconductor.samsung.com/dram/module/rdimm/m393a8g40bb4-cwe/

So... If I can't get the CAS, it can be complex in the future to get a ram with similar characteristics to avoid instability. Any ideas about how to handle this

Cas latency is completely and utterly irrelevant

 

If you want to ensure ram compatibility then buy the exact same rams with the same ics and same speed bin unless the mobo happens to be able to tune rams but i cant check cause noones done a bios overview of the krpa-u16

 

For example m393a2g40eb1 (16gb ecc ddr4, 4gbit samsung e die) or m393a4k40db2/3 (8gbit samsung d die), if the board supports ram tuning you can just buy a 2133 bin and run it at 3200 with no issues

 

 

Also i think servethehome would be a better place to ask for server hardware questions

[ToboRobot]

43 minutes ago, latot said:

Budget (including currency): 2500USD

Country: Chile

Cooling CPU

I live in a country with very intense earthquakes.... So I need to... handle that in some way or avoid liquid cooling. I was thinking in Noctua...

Checkig this: https://ncc.noctua.at/cpus/model/AMD-EPYC-7742-1188

The NH-U14S DX-4677 seems to be the best option in https://www.amazon.com/-/es/NH-U14S-DX-4677-enfriador-prémium-LGA4677/dp/B0BSXP2G72 for 120USD but seems to not be too avaliable.

If you go with an SP3 Epyc you need the right cooler https://noctua.at/en/nh-u14s-tr4-sp3

 

Is your budget sufficient in your country to purchase the parts you are looking at? 
 

[latot]

28 minutes ago, brob said:

Why do you want to RAID the NVMe drives?

Two main reasons, I would like first get speed, think like one lane per nvme would be great or something like that. And for backups.

I work with a lot of data, is pretty important be able to read/write massive amounts of data and in a faster way.

 

30 minutes ago, brob said:

Consult https://cultists.network/140/psu-tier-list/ for guidance on ATX and SFX PSU.

I'll need to lean more about them, I lack the knowledge of a lot of terms of there. Thx!

 

20 minutes ago, Somerandomtechyboi said:

Cas latency is completely and utterly irrelevant

 

If you want to ensure ram compatibility then buy the exact same rams with the same ics and same speed bin unless the mobo happens to be able to tune rams but i cant check cause noones done a bios overview of the krpa-u16

 

For example m393a2g40eb1 (16gb ecc ddr4, 4gbit samsung e die) or m393a4k40db2/3 (8gbit samsung d die), if the board supports ram tuning you can just buy a 2133 bin and run it at 3200 with no issues

 

 

Also i think servethehome would be a better place to ask for server hardware questions

Thx!

 

12 minutes ago, ToboRobot said:

Is your budget sufficient in your country to purchase the parts you are looking at? 

In my country.. you can't found this parts, I need to buy them in other countries and ship the parts. For now, the budget is able to handle major of all this.

[ToboRobot]

6 minutes ago, latot said:

 

In my country.. you can't found this parts, I need to buy them in other countries and ship the parts. For now, the budget is able to handle major of all this.

You are building a dual or single CPU system?

 

CPU 800 USED

Mobo 650 New
RAM 400

Cooling 100
GPU 200
PSU 300
UPS 300

Storage and Expansion cards +++

Plus shipping.

I guess what I am getting at is, do you need a server and have the experience to build and manage it, or should you build a desktop/workstation?

[brob]

19 minutes ago, latot said:

Two main reasons, I would like first get speed, think like one lane per nvme would be great or something like that. And for backups.

I work with a lot of data, is pretty important be able to read/write massive amounts of data and in a faster way.

 

Early studies of SSD in RAID arrays suggest that in some random access patterns they are no faster than single drives. 

 

If storage speed is important I'd suggest getting PCIe 4.0 NVMe units with hardware cache. PCIe 4.0 provides about 8GB/s bandwidth with m.2 drives. 

 

RAID is not backup. While it can provide some protection for hardware failures it does nothing to protect against soft issues, (accidental erasure, corruption, malicious attacks, etc.). Actual backups are still required. Unless downtime cannot be tolerated there seems little need for RAID.

 

[latot]

7 minutes ago, ToboRobot said:

You are building a dual or single CPU system?

I explain part of it above, I still does not have decided this important decision, for now I was thinking in single cpu and when I need to upgrade buy a dual cpu mobo.

 

8 minutes ago, ToboRobot said:

Mobo 650 New

I was thinking in a used mobo or is a bad idea?

 

8 minutes ago, ToboRobot said:

GPU 200

For a basic one, does it need to be 200USD? why not a cheaper one?

9 minutes ago, ToboRobot said:

Storage and Expansion cards +++

I already have storage, the raid part I'm thinking on it for the long term, so no need to go with the actual budget.

 

9 minutes ago, ToboRobot said:

Plus shipping.

Yeah... shipping and taxes, are a part of it.

 

10 minutes ago, ToboRobot said:

I guess what I am getting at is, do you need a server and have the experience to build and manage it, or should you build a desktop/workstation?

That is a hard question.... I uses for too much different things PCs..., The true I have asked some times what would be the main difference of a workstation and a server, I still don't have a clear answer to that question....

 

The main difference I found is that a workstation usually has heavier load per thread, while a server handle small loads but a lot of them. If we follow that rule the best case would be a workstation, but there is no too much CPUs with a lot of core to be able to do all or a lot from what I wrote above.... and I would need several workstations in order to achieve that, other point is the max ram of the workstation which is not too much. I'm open to new things if you have more info

 

Thx!

[latot]

12 minutes ago, brob said:

 

Early studies of SSD in RAID arrays suggest that in some random access patterns they are no faster than single drives. 

 

If storage speed is important I'd suggest getting PCIe 4.0 NVMe units with hardware cache. PCIe 4.0 provides about 8GB/s bandwidth with m.2 drives. 

 

RAID is not backup. While it can provide some protection for hardware failures it does nothing to protect against soft issues, (accidental erasure, corruption, malicious attacks, etc.). Actual backups are still required. Unless downtime cannot be tolerated there seems little need for RAID.

 

You are... Right! the backup is not perfect, backup to the cloud seems hard to me now because I use Sync.com and there is no linux support, but is good to take that in consideration.

 

About the NVMe with PCI 4.0 I agree with that, in that case I can't choose a SuperMicro 7001/7002 beause they have PCI 3, and for some reason the supermicro mobos has not too much PCI slots... even the dual cpu ones.

[Somerandomtechyboi]

15 minutes ago, latot said:

I was thinking in a used mobo or is a bad idea?

Very good idea to buy used mobo

I mean at their oldest maybe 5 years but most seem to be abit newer than that, there are many decade old mobos still running just fine if literally 90% of my hardware is anything to go by or atleast my p5q p6x58d-e and g31m-s2c cause i sold the others after fixing

[latot]

58 minutes ago, Somerandomtechyboi said:

Very good idea to buy used mobo

I mean at their oldest maybe 5 years but most seem to be abit newer than that, there are many decade old mobos still running just fine if literally 90% of my hardware is anything to go by or atleast my p5q p6x58d-e and g31m-s2c cause i sold the others after fixing

Great thx! just to know, them ram is that durable?

[Somerandomtechyboi]

Just now, latot said:

Great thx! just to know, them ram is that durable?

Yes pc parts are alot more durable and abusable than what most ppl think

 

You still need >2v to kill the cpu with vcore or imc voltage

Some good ram ics will flat out refuse to die well past 2v (bdie, rev e, etc.)

And most hardware is designed to last decades, even those crappy oem psus or lower end psus are made to last 10-15 years if my dazumba 450w and the plethora of shit oem psus i got are anything to go by

 

Also if you are in need of a high capacity psu you might get lucky with used deals on 1200-1600w psus, here in indonesia i can buy an andyson 1200w px (platinum) for 50$ used which is far better than some similarly priced 700w aerocool bronze units i can get new, it is advisable to not buy anything over 7 years old prefferably under 5 years old so you still got a decade or so left of psu life