Server Sanity Check

[Aragorn-]

I feel like i've gone down a rabbit hole with this so i'd like some thoughts. Its a bit long winded, so apologies.
 

Currently i have a fairly basic home server. Its an A320 AM4 motherboard with a 2200G APU, and a LSI SAS controller connected to 4 drives in a hotswap cage. Its all currently in a little tiny 2U rack chassis. The server boots from an old Crucial M4 64gb ssd, and had 4GB of ECC RAM.

Software stack is an Ubuntu 18.04 server install, with ZFS running the storage array. Theres no virtualisation currently. The machine also hosts an instance of Kodi and is connected to my main television in the lounge for media viewing. There are various software packages running on there too, including sabnzbd and the usual accompaniments. It also hosts emoncms.

 

The machine has always been a bit short on ram, i had the 4gb stick spare when i built it, so chucked it in as a stop gap. Recently i've started wanting to make some changes, both hardware and software wise. As a starting point, i swapped the 4gb stick out for 2x8gb (still ECC) I also bought a pair of 256gb nvme ssd's which i want to use as a boot drive to replace the aging Crucial SSD. The crappy A320 board doesnt have m.2 slots, but i got some little riser boards which slot into the pcie1x slots and the board sees them and is happy to boot from them, so all good there.

 

Initially i planned to install a fresh copy of ubuntu onto the new SSD's and set up some virtualisation (Xen or KVM most likely). I wanted to move the downloadey stuff into its own VM so i could firewall it and ensure it only gets to the internet by a VPN. So far so good. I decided that while i was there, i might as well setup another VM to host Jellyfin or Plex, so i can share media with some family members. I've also been looking at some IP cameras, and would setup a VM to run the NVR software. So i started thinking about hardware requirements. I figured i'd swap the CPU for something with more cores, the 2200g is a bit weak and probably asking a bit much of it to be running all of the above, especially as the CCTV and media server can be fairly resource hungry, especially with internet streaming a lot of the media would need transcoded. I figured an 8 core Renoir would work (when they finally appear), but realised this would need a motherboard swap as Renoir isnt likely to work on the A320. Then amongst the process of that i realised that while i had ECC ram, the 2200g doesnt acutally support it, and the machine was running without any ECC. Its likely (though not confirmed) the Renoir APU's also wont support ECC.

 

So at this point i think ok, if i'm swapping the board and CPU, i could go 2700x (or 3700x depending on pricing) and stick a basic GPU in there to run the graphics. ECC is generally confirmed working on the non-APU Ryzen chips so that fixes that issue. Bearing in mind, the machine runs Kodi, so it needs something capable of decoding the usual media formats, but there are plenty options around for that. I just need a board with more slots, as i need to fit my SAS controller in (8x physical, so generally needs a 16x slot) as well as a 16x slot for the GPU. This is a proper faff as some boards split the CPU lanes from 16x into 8x/8x to run the pair of pcie slots. Others do weird stuff like sharing lanes with m.2 and you have to really scour the specs. The case also proves an issue at this point, as the bigger mATX boards that have the nicer features, are too wide to clear the drive caddys in the front of the chassis, and if i'm swapping the chassis, i dont need to stick to mATX, i could look at ATX options as well.

 

So ok, its doable, but i dont want to be revisiting this in another 6 months time. what else might i want to add to this machine? After some thought i realised the Plex VM might well need hardware accelleration, especially as i've been collecting more 4K stuff which will definitely need transcoded down for internet streaming, so potentially a GPU to run that. Perhaps a 10gbe NIC. Potentially some more storage added for the cameras, but my SAS controller has another 4 spare channels so okay there..

 

So with those specs in hand, i now need a board which can accept two GPU's plus an additional 8x card, plus have a slot free for a 10gbe nic (which are often 4x, but an open ended 1x slot would probably do in a pinch) I'd also need two m.2 slots. After a LOT of searching I found a small handful of AM4 boards that meet these criteria, and they're all £300+ top end X570 boards. And its still a compromise as they still dont have enough lanes, so the ssd's and sas controller would likely end up sharing the chipset lanes etc....

 

Then i realised for less than the price of one of those high end AM4 boards, i could buy X399, and currently the 1900x 8 core chip is the same price as a 2700 (and actually appears to be a bit faster due to higher clocks). So for the same money, i can go Threadripper. They're ATX boards ofcourse, but i need a new case regardless. Now clearly the geek in me says "do it!". And its clearly a better solution, as it has more lanes and less compromises. But it seems like a big jump from "stick a faster CPU in and add some ram" that i started out with. I could just pick a middle of the road B450 board with two 16x slots, but if i did that, and then after buying that and a 2700x ran into streaming issues i'd probably be quite pissed.

 

So thoughts? Do i just bite the bullet and go X399?

[Electronics Wizardy]

How much do you care about power usage? x399 will use more power than a am4 system. And you probalby don't need the computer power or ram either, so id stick with am4 here

 

Id probably drop the hba for now, just use the onboards sata ports.

 

Also look at proxmox if your running vms, makes managing vms work nicer.

16 minutes ago, Aragorn- said:

After some thought i realised the Plex VM might well need hardware accelleration, especially as i've been collecting more 4K stuff which will definitely need transcoded down for internet streaming, so potentially a GPU to run that.

how many streams at once? I can run transcodes form 4k to 1080 on my old dual l5640 system, so a new 8 core will easily do this with no gpu. 

 

So from what I see now, you need a slot for a gpu, and thats it for now. You can add the sas card if you need more drives, but the onboard sata works for now. You don't seem to need the 10gbe now either. And most cheap am4 boards let you have 2 of those, and there should be cheapish b550 boards that will let you ahve all 3

[Aragorn-]

I guess power matters, but I did look at this and a few reviews showed while it is more hungry, it's not that bad. A lot depends on the configuration as well. For instance running 4 dimms uses more power than two, regardless of threadripper or am4. However in this comparison, the 12 core 1920x was using 20w more than the 2700x. Thats acceptable, and i suspect the 1900x will use a bit less anyway.

https://static.techspot.com/articles-info/1678/bench/Power.png

 

Two of my existing drives 6tb drives are sas, so dropping the hba is out. I also like the flexibility it gives. Excellent deals often pop up with used SAS drives as they dont work in normal PC's
 

I did look at proxmox, i might well have a play with it, though i was a bit put out that they are using a custom kernel. It would have been much nicer to build it as an installable package that just sits on top of Ubuntu. If i get some time i'll install it on a spare machine and explore a bit.

 

I suspect initially only one or two streams, I'm hopeful the CPU will handle that fine. But the option to add a GPU if needed would be good. I do find it surprising that your old i5 can manage it unless its using QuickSync. Plex themselves suggest as a rough guide you need around 12000 passmark per 4k stream to transcode. Thats roughly an entire 2600x CPU. I tried it recently on a mates Plex server, and it pegged his quad core Xeon transcoding a 4k sample down to 1080p.

And yes, I realise I can just do b550 and have everything that I NEED at the moment but I want to future proof a bit, and b550 doesn't seem that much of a saving, as I'll still need a new chassis, and the boards aren't all that cheap. If it was a drop in CPU swap, then i'd just do that. I'd hate to rebuild it all, and then in 6 months time decide i need a GPU and realise i'm having to rebuild again.

[The_Ellimist]

Actually if you want TR4 mATX as an option there is one. Honestly if it fit current gen threadripper I'd be there.

 

https://www.asrock.com/mb/AMD/X399M Taichi/index.asp

 

its got 3 full x16 slots on it and a u.2 port which is kinda cool as well.

[Aragorn-]

1 hour ago, The_Ellimist said:

Actually if you want TR4 mATX as an option there is one. Honestly if it fit current gen threadripper I'd be there.

 

https://www.asrock.com/mb/AMD/X399M Taichi/index.asp

 

its got 3 full x16 slots on it and a u.2 port which is kinda cool as well.

Thanks. It would still need a new chassis. The one i have now is very short, and the hotswap drive cages i modified into it overlap part of the standard motherboard area. So a "full size" mATX board (9.6x9.6") wont fit. And if i'm going for a new chassis, i'll be as well to get one thats big enough to take a normal ATX motherboard, which means a much wider selection of boards.

[The_Ellimist]

4 hours ago, Aragorn- said:

Thanks. It would still need a new chassis. The one i have now is very short, and the hotswap drive cages i modified into it overlap part of the standard motherboard area. So a "full size" mATX board (9.6x9.6") wont fit. And if i'm going for a new chassis, i'll be as well to get one thats big enough to take a normal ATX motherboard, which means a much wider selection of boards.

Fair call. I'm rocking a thermaltake core G3 for a similar reason at the moment.