Server Build Help Picking Parts

[R4M33Z]

Hi,

 

I am looking for some help with a new build that I am thinking of. I currently have a prebuilt system that I am running Freenas on but the hardware isn't up to scratch as it only has 16GB of RAM and I need more as ZFS is RAM hungry. The old system is not worth upgrading so I thought that I will start a new build and maybe use the current system as a backup system only. So here is my dilemma, there are so many parts to choose from and everyone has their own opinion on what parts are the best parts to pick.

 

I will start by telling you what I would like the system to do, I am currently using the system as a NAS that holds multiple datasets (this is pretty basic and most hardware will support this), I am also currently running a plex media server for my entertainment needs. I am looking to expand the system to use VirtualBox in a jail so that I can access my virtual machines from a web browser. This is where it starts to get interesting as RAM becomes more of an issue. I may also in the future want to expand the system with the plugins that Freenas contains.

 

So what I have been thinking; after watching the video that Wendell put out when he used to be at TekSyndicate, I figured that I should factor 64GB of RAM for my system as I will be running Virtual machines and running Freenas, this also opens up opportunities to use deduplication which is a RAM hungry feature. Currently I am using 4X4TB drives with redundancy and I am getting close to the 8TB limit so I would like to add 6x6TB drives. I will also have 2 SSD drives to speed up the jails operation and a further single SSD as cache for the system. There are so many choices for CPUs and motherboards, this is where I would like some help if someone would be so kind. I have been told by some people to use ECC RAM and some that say it is not necessary with Freenas, any input I am sure will help. As a budget, I do not have a budget yet as I would like to see how much the system would cost but I would imagine that I do not want to go more than £1500 without drives (I know how expensive these drives can be).

 

If you are interested, the current system is a HP Microserver with an intel celeron @ 2.3GHz, 16GB RAM, 4x4TB WD red drives, 500GB SSD for jails operation, HP motherboard.

 

CPU: ?

RAM: 64GB

Motherboard: ?

Hard-drives: 6X6TB NAS Drives (Don't know which ones), 3x500GB SSD (Haven't picked one yet)

PSU: ?

Case: ? (I was thinking about rack mounting in the future, would prefer to use SATA cables from the drives to the motherboard if we can find a motherboard with an appropriate number of SATA ports)

Cooler: ? (I am sure I would be able to find a suitable one after I have come to a decision on the CPU but recommendations are welcome)

 

Thank you for taking the time to reading my long post. I am new to this forum and this is my very first post.

[Schakal_No1]

for the psu (redundant/singe) and ram (ecc/standard) it is important to know how reliable the server has to be. Would a crash be critical? If not, go with a normal psu and regular ram. If crashes are not an option at all you need ecc and a redundant power supply.

 

For the CPU: If you are working with jails get as many cores as you can afford.

 

For the motherboard: I personally LOVE supermicro boards but they cost extra. But they are super reliable and have great features like remote access to the motherboard.

EDIT: 8 SATA cables is not a lot, so I do not see a limit there. You could even get SATA expension cards for PCIe, they are cheap.

 

Case/Cooler: Do you want rack mount or not? You only said you are thinking about it. This makes a big difference.

[R4M33Z]

Hi Schakal_No1 thanks for a speedy reply. In terms of the reliability of the system, this is basically a glorified home server and so does not have to be 100% reliable, therefore I do not think it would be necessary for ECC RAM. In London, I do not experience blackouts often so I think that I will be safe with a standard power supply but I may consider an external UPS for safe shutdowns later on down the road.

 

In terms of the chassis. I would in the future like to rack mount but currently I do not have the space for it, therefore, I think a standard chassis would be the way I go until I can get more space.

 

In terms of CPUs, what do you think I should be looking into as some people have said to look at Xeon E3s and some have said to look into just regular i3s. I feel that expandability and finding parts will be easier for an i3 but I would like a second opinion from someone. In terms of motherboard, I have heard of supermicro but always thought they just made chassis, I will look into this further.

 

Thanks for the food for thought.

[Schakal_No1]

the xeons will give you a higher maximum number of cores. Depending on the number of jails you want to be running in parallel 4 cores would be the absolute minimum I'd accept with no limit upwards (1-2 cores per jail would be recommended). You can get great deals on used xeons if you look on ebay, those are good for a long fucking time. i3s usually only have 2 cores which is why I would stay away from them.

 

The difference between xeon e3s and e5s is that e3s are meant for single cpu systems while e5s are dual cpu systems. Keep this in mind when shopping for used parts.

[Nuvi]

The freeNAS forum is the perfect place to read this but remember, ZFS itself functions entirely inside of system RAM, if your data is critical I'd keep away of non-ecc RAM. If you still decide to go with non-ecc ram test it really well, bad ram can and will make your pool unmountable and then there is no way to get any of your data back.

[Schakal_No1]

1 minute ago, Nuvi said:

The freeNAS forum is the perfect place to read this but remember, ZFS itself functions entirely inside of system RAM, if your data is critical I'd keep away of non-ecc RAM. If you still decide to go with non-ecc ram test it really well, bad ram can and will make your pool unmountable and then there is no way to get any of your data back.

not sure where you got the info of ram breaking a pool. One of the neat features of ZFS is its resistance to write holes which damage traditional raids, never saw a case of bad ram breaking a pool and I use tons of them at work.

[SCHISCHKA]

22 minutes ago, R4M33Z said:

this is basically a glorified home server and so does not have to be 100% reliable

then you do not need redundant drives or expensive server components. Get a ryzen 1700 or the upcomming threadripper. Put your existing drives into RAID0 and buy a decent backup solution.

 

 

[Nuvi]

11 minutes ago, Schakal_No1 said:

not sure where you got the info of ram breaking a pool. One of the neat features of ZFS is its resistance to write holes which damage traditional raids

Yeah it is! ZFS scrubs for exemple offer great protection against corrupted bits, unless you use bad RAM in which case ZFS will attempt to "fix" corrupted data that it thinks is from your hard disk and write that data back. But instead it is actually reading good data from your drive, corrupting it in RAM, and then writing the "fixed" data to your disk, trashing your pool along the way... 
I suggest you read this for a detailed explanation if your interested, and here is an example of one of many people who experienced data loss for using bad non-ecc RAM

[R4M33Z]

Nuvi thanks for that. I will look into ECC RAM which shouldnt be much of a push if I go for a xeon e3 CPU and supermicro board. I obviously do not want the data to become unreadable suddenly.