Design server by functional requirements

[JPotze]

Hello guys and thank you for reading,

 

TLDR; I want a streambox that can easely transcode (4k) hevc and h264 to multiple clients with plex and do some more.

 

I have a hard take making decisions on buying the best hardware for my needs. I have some wishes i would like to fullfill with a server and i dont know how to configure it. Maybe you would like to join in the debacle. 

I'll list the functional requirements and maybe you guys can tell me what i do right/wrong.

 

-Hold all my movie collection

-Stream, transcode (HEVC 10 bit to be future proof) with plex with minimum 3 transcoding streams and 1 direct stream at a time.

-Big datastore where i can make shares so all the computers in the house (maybe outside the hous) can dump data on

-Host minimum 1 linux vm, 1 windows vm and a mac vm that i can access with rdp from inside and outside my home

-be a receiver for backups of nas boxes of my friends (offside backup for them) 

-Make backups offside myself

That are pretty much my wishes in the order of priority. Now i found that the e3-1275 v5 is capable of playing 4k netflix and transcoding with the new internal video graphics that is supported by plex to hardware transcode or something. I dont have exact understanding how this works.

 

Can you please advice me how i can build better, cheaper or whatever? all advice is welcome

 

PCPartPicker part list: https://pcpartpicker.com/list/wc4w4C
Price breakdown by merchant: https://pcpartpicker.com/list/wc4w4C/by_merchant/

CPU: Intel - Xeon E3-1275 V5 3.6GHz Quad-Core Processor  ($336.44 @ OutletPC) 
Motherboard: Supermicro - MBD-X11SSH-LN4F Micro ATX LGA1151 Motherboard  ($211.98 @ Newegg) 
Memory: Crucial - 16GB (1 x 16GB) DDR4-2133 Memory  ($184.99 @ B&H) 
Memory: Crucial - 16GB (1 x 16GB) DDR4-2133 Memory  ($184.99 @ B&H) 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive  ($133.99 @ SuperBiiz) 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive  ($133.99 @ SuperBiiz) 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive  ($133.99 @ SuperBiiz) 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive  ($133.99 @ SuperBiiz) 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive  ($133.99 @ SuperBiiz) 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive  ($133.99 @ SuperBiiz) 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive  ($133.99 @ SuperBiiz) 
Storage: Western Digital - Red 4TB 3.5" 5400RPM Internal Hard Drive  ($133.99 @ SuperBiiz) 
Case: Fractal Design - Node 804 MicroATX Mid Tower Case  ($94.99 @ Newegg) 
Power Supply: EVGA - SuperNOVA G2 550W 80+ Gold Certified Fully-Modular ATX Power Supply  ($72.98 @ Newegg) 
Total: $2158.29
Prices include shipping, taxes, and discounts when available
Generated by PCPartPicker 2017-08-04 10:10 EDT-0400

 

 

The 2th question is, what OS to run.

run esxi and host a freenas vm that has direct acces to the disks with vt-d and make some vm's with the linux, windows and mac vm.

or

run freenas bare metal and run those vm's inside freenas. 

 

Thanks for the advice!

[cj09beira]

heve you though of running a ryzen chip as the server? you can get a full 8 core for less than that xeon will cost you

[JPotze]

5 minutes ago, cj09beira said:

heve you though of running a ryzen chip as the server? you can get a full 8 core for less than that xeon will cost you

i have thought about it but i have no knowledge of AMD and there architecture. And i really like supermicro and their IPMI capabilities.

[cj09beira]

7 minutes ago, JPotze said:

i have thought about it but i have no knowledge of AMD and there architecture. And i really like supermicro and their IPMI capabilities.

ya, you would loose the fancy management, on the other hand you would have double the compute power (it does support ecc).

so i guess it depends on if you need the extra perf enough to offset the loss of ipmi and other server features (as i am talking about a consumer board).

(its always good to have options, they make things more interesting)

[dalekphalm]

In general, your hardware seems fine.

 

A few notes about transcoding: If you're transcoding HEVC files into h.264 on the fly, you can just pretty much any current gen CPU (more cores + threads = better).

 

If you want to be transcoding while keeping HEVC, you'll be doing software encoding, unless the server has an iGPU or a discrete GPU w/ HEVC encoding support. That would basically be Skylake or newer w/ iGPU, or a Carrizo AMD APU (Which is very inferior to Ryzen).

 

With that in mind, a 4c8t Xeon should still be decent at software transcoding.

 

4 hours ago, JPotze said:

The 2th question is, what OS to run.

run esxi and host a freenas vm that has direct acces to the disks with vt-d and make some vm's with the linux, windows and mac vm.

or

run freenas bare metal and run those vm's inside freenas. 

 

As for this, I would go the ESXi route (I did this).

 

Install ESXi as the base OS, get an HBA (not a RAID Card, unless it can be flashed to "IT" mode), use PCI Passthrough, and assign that HBA directly to a FreeNAS VM.

 

I do it, and it works well.

 

The reason why I recommend this is that the VM functionality of FreeNAS is what I would call "Serviceable at best". It's not great. The options are more confusing and I found it a lot harder to setup a VM. Whereas ESXi was much more straight forward, the options made sense, and there were also tons of guides (even for FreeNAS) to get you running.

 

Plus, ESXi is free, as long as you don't need vCenter (which you probably don't).

[JPotze]

On 4-8-2017 at 8:34 PM, dalekphalm said:

In general, your hardware seems fine.

 

A few notes about transcoding: If you're transcoding HEVC files into h.264 on the fly, you can just pretty much any current gen CPU (more cores + threads = better).

 

If you want to be transcoding while keeping HEVC, you'll be doing software encoding, unless the server has an iGPU or a discrete GPU w/ HEVC encoding support. That would basically be Skylake or newer w/ iGPU, or a Carrizo AMD APU (Which is very inferior to Ryzen).

 

With that in mind, a 4c8t Xeon should still be decent at software transcoding.

 

As for this, I would go the ESXi route (I did this).

 

Install ESXi as the base OS, get an HBA (not a RAID Card, unless it can be flashed to "IT" mode), use PCI Passthrough, and assign that HBA directly to a FreeNAS VM.

 

I do it, and it works well.

 

The reason why I recommend this is that the VM functionality of FreeNAS is what I would call "Serviceable at best". It's not great. The options are more confusing and I found it a lot harder to setup a VM. Whereas ESXi was much more straight forward, the options made sense, and there were also tons of guides (even for FreeNAS) to get you running.

 

Plus, ESXi is free, as long as you don't need vCenter (which you probably don't).

Thank you for the info. I will look into a cheaper startoff point though. 

 

I was thinking. When i am going the esxi route. Then i have to have a HBA, so then i dont need 8 sata ports on the mobo right? cant i just configure the sata ports to do passtrough in Esxi? 

 

And wouldn't it be better to start off with 4 Disks in z1 raid en then later buy 4 more and do another z1. i think that would be just as efficient in space usage as 8 disks in z2. and i have the option to, if that excists in that time to buy 4 like 6tb or 8tb disks for cheap.

And wouldnt a z1 pool be faster then a z2 because of the parity compute?

 

 

[dalekphalm]

5 hours ago, JPotze said:

Thank you for the info. I will look into a cheaper startoff point though. 

 

I was thinking. When i am going the esxi route. Then i have to have a HBA, so then i dont need 8 sata ports on the mobo right? cant i just configure the sata ports to do passtrough in Esxi? 

 

And wouldn't it be better to start off with 4 Disks in z1 raid en then later buy 4 more and do another z1. i think that would be just as efficient in space usage as 8 disks in z2. and i have the option to, if that excists in that time to buy 4 like 6tb or 8tb disks for cheap.

And wouldnt a z1 pool be faster then a z2 because of the parity compute?

 

 

You cannot pass individual SATA ports through to a VM. You can pass entire SATA controllers, so if your motherboard has two controllers you might be able to use the secondary one. But I cannot guarantee how well that would work. 

 

As for 2x RAIDZ1 vs a single RAIDZ2? Yes technically they are equally space efficient. However the RAIDZ2 is better for redundancy since ANY two disks can fail. Whereas with 2x RAIDZ1 arrays you can only have a single failure in each array.

 

You could do either - totally up to you. If you can only get 4 disks to start, then that's a fine way to do it. 

[Erkel]

That boards LOM is a little buggy,  Has an issue with LSI raid cards also, dropping disks.

The CPU is over priced for the performance, I use the 1245 for reference in my go fast boxes.

Would go with a EVGA 750 G3 and disable eco mode so you have more headroom.

Also consider a couple of boot SSD's or HDD's to seperate you OS and data volumes. for 10 disks total.

 

[dalekphalm]

2 hours ago, Erkel said:

That boards LOM is a little buggy,  Has an issue with LSI raid cards also, dropping disks.

The CPU is over priced for the performance, I use the 1245 for reference in my go fast boxes.

Would go with a EVGA 750 G3 and disable eco mode so you have more headroom.

Also consider a couple of boot SSD's or HDD's to seperate you OS and data volumes. for 10 disks total.

 

I will agree with you about having a separate boot volume. Though ESXi or FreeNAS (either one) needs a very small amount of storage for the OS (2GB to 8GB each is generally fine).

 

I'd recommend doing a RAID1 array of a couple of small SSD's (Say, 60GB) for ESXi. Then use the remaining unused space on the RAID1 SSD array as a Datastore for ESXi, and create the FreeNAS VM virtual hard disk on there as well.

 

After that, you can use FreeNAS to create the RAIDZ1 (or RAIDZ2, or whatever) pool for the main storage array. You can then pass some (or all) of the storage array back to ESXi via iSCSI to use as more OS datastore storage for future VM's.

 

It's a bit convoluted, but works well once you've got it all setup. You just need to set boot order so that FreeNAS boots before any other VM's.

 

Personally, I used an SSD for ESXi, then a 2x 146GB RAID1 and a 2x 300GB RAID1 for my ESXi datastore storage for my VM's (including FreeNAS).

 

But my server setup allowed that, as I have 6x hotswap bays on the server case itself (powered by a Dell H200), and a 12-bay MD1200 SAS Expander (powered by an LSI 9207-8e HBA I believe) that I have my proper 6x 3TB RAIDZ1 array in. The SSD is plugged straight into a SATA port on the motherboard.