Which server software to use

[SorenHansen]

Hi all

 

I have finally put my "server" PC together, now comes what I dread most

 

Which server software to use, what are your recommandations, GUI prefered but I am used to a cli (although it have been many years ago (this many years ago: Debug G=C:800))

 

I have been sniffing at Windows server, but I'm no fan of it

I have tried Freenas, but the "new" GUI is unintuitive to me

I have tried Unraid, I like it, but the mindset needed I don't have

What I have had most success with (after countless hours scouering Youtube and the net) s Ubuntu server 20.04 LTS

I also looked at Vmware's Vsphere 6.7 but apparently my netcard are not officially supported

 

What I need could really be handled by a Nas, file sharing, video and picture server, backupserver and home streaming of movies

 

So filesharing, backup, video streaming, video and photo keeper are the main goal, but using it to store my games on and virtual machines for learning, would be a good runner up

 

Hardware:

 

Mobo     Asus ROG Crosshair VIII Dark Hero

CPU      Ryzen 9 5950x

Ram      Corsair Vengeance LPX 3000 mHz 32 Gb (will be swapped for 128 Gb faster ram later)

HDD      2x Samsung 970 EVO Plus 2 Tb, 2x Samsung 870 EVO 2 Tb, 6x Seagate Barracuda 2 Tb, 4x Seagate Barracuda 4 Tb, 1x Seagate Barracuda 8 Tb

Video     Zotax? GTX 1060 3Gb

Internet  1000/100

 

 

[maplepants]

If Ubuntu is what you know best, that's what you should use. It can easily handle all the tasks you've outlined, and there won't be any real downsides.

 

The main reason to use Windows Server is if Windows is what you know best or if you want to run some specific software. In general, managing a Windows Server is like running a Linux desktop. You can do it, but it's not the use case the OS was primarily written to solve so it will fight you in many places.

[charity]

liiiinuuuux in a good distro. lightweight and epic. all good servers use linux. 

[LogicalDrm]

-> Moved to Server and NAS

[Budgetclocker]

If you can't really decide on a dedicated OS but you like Linux, I'd personally recommend Proxmox. Similar to VMWare, in the fact that it's designed to host virtual machines. However, it's Full Debian and open source. It's not perfect, but it's free.99, and very nice.

[jde3]

Top industry servers by popularity are Red Hat Enterprise Linux (or clone like Oracle or Rocky), Ubuntu Server and SuSE Linux Enterprise Server. My personal favorite is FreeBSD Unix. They would be managed remote via automation or shell and tend not to have their own GUI's. - Ubuntu and FreeBSD have no subscription cost.

 

Proxmox, TrueNAS and Unraid etc are not really used much. (in serious industry) - Your choice in what you want to use depends on your intentions.. if you want to work in the industry some day you'll want to be familiar with the same stuff they use.

[Jarsky]

One of my servers has similar specs, I used UnRAID on it and use it for hosting services and as a test lab which i can build VM's and even have a full vCenter test lab for testing and continued practice (since I need to kee my VCP certifications up to date). The UI does about 90% of what I would use the shell for. It also supports mixed drives, so if you put all your drives in an array with 2 parity drives, you'd have a total of 24TB of usable space. You of course could swap out and upgrade drives over time.

 

The limitations of UnRAID?

Performance. It doesn't use traditiaonal raid and writes files contigously (whole files are written to a disk), so reading files you only have that 1 hard drives speed.

UnRAID doesnt have a proper cache system, it at most can have a write cache which is essentially new files stored on ssd before being moved to the array. There are no smarts that will move frequently accessed files into cache.

UnRAID does have a cost it would be a $109 license. It does have a 30 day free trial. 

 

 

TrueNAS Scale is a huge step up for enthusiasts and all-in-one boxes over TrueNAS Core. As it uses QEMU/LibVirt the same as UnRAID, rather than the very limited bhyve virtualisation from FreeBSD. It also uses Docker rather than Jails which has far wider community support. And being based on Debian its a very familiar platform for Linux users if you do need to poke around under the hood. The big advantage to TrueNAS Scale again is that most functions you could want are in the UI like UnRAID, and its QEMU VM manager requires far less terminal based XML editing compared to the old bhyve.

 

The limitations of TrueNAS?

Basically the restriction on ZFS expansion. If you want a large pool, it will be limited to  array-size= (n drives x smallest capacity drive) - parity drives. 

Personally I wouldnt build an 11 drive RAIDZ2 or RAIDZ3. Id probably get another disk and build 2 x 6 disk RAIDZ2's. The problem is due to mixed drives you'll lose a lot of your capacity using your current disks, but if you're looking to upgrade all the disks you can swap them out over time and expand the dataset once all the disks in each VDEV are updated.  So to start you could have 6 x 2TB in 1 VDEV giving you 8TB of usable space, and you could get another 8TB and have the 4x4TB + 2x8TB in another VDEV giving you 24TB....32TB in total in the zpool.

 

Proxmox is first and foremost a hypervisor, its primary use is for hosting VMs. Its built in container uses LXC (Linux Container) which while being powerful and very lightweight, has far less community support out there than Docker or Kubernetes. As its not a "NAS" OS, it also doesnt have native share management tools. You would need to install and configure services like Samba through terminal, and you could install something like Cockpit or Webmin to manage your shares/users through a UI. Or alternatively you can use an LXC container like Turnkey File Server to give it a native Fileshare role which includes Samba, Webmin and other UI tools. 

[jde3]

Do they call it QEMU? hmm.. (QEMU is an emulator and alone would take about 30 minutes to boot Windows 10.)

 

About the world of hypervisors:

 

Almost all hypervisors are built around Intel / AMD's vritulazation engines. These are the components that are doing a lot of work.

 

There are the commercial hypervisors. VMWare, VirtualBox, HtyperV and there is also Xen still out there. They are very full featured but I'm not going to talk a lot about these as you can find information out there.

 

Linux's is ordinarily called KVM and it is a combination of things. QEMU, Libvirt and the kernel bits KVM. Qemu runs 16bit boot code and other emulated machine features, it is a full emulator and can emulate x86 on arm and the like but for KVM it serves a minimal role as emulation is slow.. And Libvirt is the management tools for this for configuring the VM it's virtual hardware etc.. Everything Linux uses KVM. It uses several virtual disk container formats that are mostly loopback files on top of other filesystems. You can think of it like a bunch of different things that build up to a full featured hypervisor.. tho is has limited frontends by itself. Maybe Apache Tomcat would be a good analogue.

 

BHyve is FreeBSD's native hypervisor and it can use libvirt. At least there is some support for bhyve in libvirt. It has no emulation layer making it a minimal or light weight hypervisor hence it's requirement for only booting EFI OS's and it's limited support for emulated (slow) hardware.. It's why you sometimes have a more complicated guest install.. because it only supports doing things the fastest way damn the torpedoes..The reason for this is it serves a different role were a simplified hypervisor is desired. "Just the hypervisor ma'am." It typically uses ZFS as a virtual disk container giving it fewer filesystem layers and fast direct I/O to the disk. It's also been ported to MacOS and Illumos (Solaris) due to it's more simple implementation (Perhaps there is a Linux port as well?). You could think of it like lighttpd or nginx.

 

One is neither better or worse really than the other it just depends on your requirement needs and what you are trying to accomplish. It also really highlights the different design philosophy of the two OS's. Where Linux takes the layered approach building on dozens of existing technologies with lots of support and FreeBSD builds a purpose built custom performant solution.

 

 

 

 

[Pixel5]

we need to know what the purpose of this server is to recommend anything.

 

Also whats the mindset you need for unraid that you dont have?

[SorenHansen]

On 4/20/2022 at 3:34 AM, Jarsky said:

"TrueNAS Scale is a huge step up for enthusiasts and all-in-one boxes over TrueNAS Core. As it uses QEMU/LibVirt the same as UnRAID, rather than the very limited bhyve virtualisation from FreeBSD. It also uses Docker rather than Jails which has far wider community support. And being based on Debian its a very familiar platform for Linux users if you do need to poke around under the hood. The big advantage to TrueNAS Scale again is that most functions you could want are in the UI like UnRAID, and its QEMU VM manager requires far less terminal based XML editing compared to the old bhyve."

What you are saying is (if I understand you right) that as a newbee I'm better of with TrueNAS than Unraid, because it is so much more widespread ? And that what I have understood about dockersd can be used directly ?

 

On 4/20/2022 at 3:34 AM, Jarsky said:

"The limitations of TrueNAS?

Basically the restriction on ZFS expansion. If you want a large pool, it will be limited to  array-size= (n drives x smallest capacity drive) - parity drives. 

Personally I wouldnt build an 11 drive RAIDZ2 or RAIDZ3. Id probably get another disk and build 2 x 6 disk RAIDZ2's. The problem is due to mixed drives you'll lose a lot of your capacity using your current disks, but if you're looking to upgrade all the disks you can swap them out over time and expand the dataset once all the disks in each VDEV are updated.  So to start you could have 6 x 2TB in 1 VDEV giving you 8TB of usable space, and you could get another 8TB and have the 4x4TB + 2x8TB in another VDEV giving you 24TB....32TB in total in the zpool."

I didn't assume that I just could throw all the disks in a JBOD setup, so I would have to pool them up.

The 6x 2Tb I already figured out, the 4x 4Tb is also a nobrainer, the odd one out are the  8Tb disk (which I really don't mind forget about).

I understand that I'd be better off to switch the 8 Tb for 2x 4Tb and have 1 VDEV at 6x 2Tb and 1 VDEV s 6x 4Tb.

When upgrading each VDEV, do I need to upgrade all 6 disks at once ?

Is it possible (ofcourse it is, but is it difficult) to have my SSD's as some sort of highspeed cache for the "slow" spinning drives ?

When upgrading (because it WILL happend) what do I look after ? Servergrade spinning drives or SSD's

 

On 4/20/2022 at 3:34 AM, Jarsky said:

"Proxmox is first and foremost a hypervisor, its primary use is for hosting VMs. Its built in container uses LXC (Linux Container) which while being powerful and very lightweight, has far less community support out there than Docker or Kubernetes. As its not a "NAS" OS, it also doesnt have native share management tools."

I have been looking at Proxmox but I can't get a handle of WHAT it really is, so from what you say it is more like an environment where one create virtuel machines and run them for single purposes or a few combined purposes.

 

So .... as I see it, it would come down to TrueNAS vs Ubuntu Server, and the dealmaker would be what I feel most comfortable with, right ?

[SorenHansen]

12 hours ago, Pixel5 said:

we need to know what the purpose of this server is to recommend anything.

As written in my first post:

Filesharing, backup, video streaming, video and photo keeper are the main goal, but using it to store my games on and virtual machines for learning, would be a good runner up

12 hours ago, Pixel5 said:

Also whats the mindset you need for unraid that you dont have?

I don't know if the word "mindset" is the right term, but let me try this way:

 

Windows (with all it's flaws and limitations) are a no brainer

DOS 3.11 and OS2 was what I (besides Commodore 64 and Amiga) grew up with and thus are second nature

My first Linux was back in mid 1990's

I feel quite alright both in CLI and GUI on Ubuntu

 

But Unraid I feel that whatever I try, I feel lost ? The initial setup feels intuitive, but after that, I just . . . like don't know which programs to use for this or that application, and how to get to where I ca use them. It might be just me, or maybe the fact that I'm pushing 64, and only run on 7 cylinders

[Jarsky]

9 hours ago, SorenHansen said:

What you are saying is (if I understand you right) that as a newbee I'm better of with TrueNAS than Unraid, because it is so much more widespread ? And that what I have understood about dockersd can be used directly ?

Sorry to summarise my above post. 

TrueNAS Scale vs TrueNAS Core; Scale (built on Debian Linux) is typically a better choice for a home prosumer/enthusiast due to driver support and its more full featured hypervisor and using Docker. Contrast that to Core (built on FreeBSD) with its more limited hypervisor and using BSD Jails. There are Jails for most things, but Docker is more prevalant and many are more likely to create a Docker of their apps (for example on GitHub) than a Jail. 

 

9 hours ago, SorenHansen said:

I didn't assume that I just could throw all the disks in a JBOD setup, so I would have to pool them up.

The 6x 2Tb I already figured out, the 4x 4Tb is also a nobrainer, the odd one out are the  8Tb disk (which I really don't mind forget about).

I understand that I'd be better off to switch the 8 Tb for 2x 4Tb and have 1 VDEV at 6x 2Tb and 1 VDEV s 6x 4Tb.

When upgrading each VDEV, do I need to upgrade all 6 disks at once ?

When upgrading a VDEV, you upgrade the disks 1 at a time (using the zpool replace [pool] [olddisk] [newdisk] command). This is so it can rebuild the new drive and recalculate parity so you dont lose data.

However, the zpool will not increase in size until you replace all disks in the VDEV.

The space available on each disk in the VDEV is only as large as your smallest drive. 

 

9 hours ago, SorenHansen said:

Is it possible (ofcourse it is, but is it difficult) to have my SSD's as some sort of highspeed cache for the "slow" spinning drives ?

When upgrading (because it WILL happend) what do I look after ? Servergrade spinning drives or SSD's

Yes you can. You can have a SLOG device to move your ZIL to as a temporary write cache. And you can add an L2ARC cache drive to expand on the ARC which is a Read cache that caches frequently used files. ARC runs in memory and obviously a LOT of memory can get expensive, so an L2ARC on an NVMe is a good compromise. 

 

 

9 hours ago, SorenHansen said:

I have been looking at Proxmox but I can't get a handle of WHAT it really is, so from what you say it is more like an environment where one create virtuel machines and run them for single purposes or a few combined purposes.

 

So .... as I see it, it would come down to TrueNAS vs Ubuntu Server, and the dealmaker would be what I feel most comfortable with, right ?

 

Yeah I personally wouldnt run VMware unless you're going to be using a hardware RAID controller to build your VMFS datastores, or are going to seperate your storage from server roles. 

 

UnRAID is based on Slackware, its really flexible with its proprietary array solution, and feature packed with Virtual Machine and Docker tools etc..all into 1 nice package that runs off USB, but it gives up a lot in the way of performance. 

 

TrueNAS is a great all in 1 solution, if you want a UI, good storage performance, TrueNAS Scale has very much the same VM and Docker tools as UnRAID. Good solution if you need to run a few VM's

 

Ubuntu Server is great, especially if you dont need VM's and can containerise what you want to run. There are a few UI's out there that can make life easier as well like Webmin and Cockpit  - they even have ZFS Manager Plugins (though personally id just configure ZFS through terminal if you're going the ZFS route).  Theres a few threads ive replied to recently with info about going this route. 

 

Ultimately theyre all free to try, UnRAID has a 30 day free trial. So you can just configure them and have a play with them all before commiting to one of them.