What OS and FMS for a dual-purpose Backup/Render server?

[Lupino]

Hi, all.

I have my old computer collecting dust in the corner. It has the following specs:

•  MB: Gigabyte GA-Z87X-D3H
•  Intel Core i7-4770K 3.50GHz
•  HyperX HX318C10FK2/16 Fury, DDR3, 16GB (Kit 2x 8GB), 1866MHz, CL10, DIMM
• NVIDIA GeForce GTX 760, 4GB (MSi)
• Samsung SSD 840 EVO 250GB
• Seagate Desktop HDD.15 SATA III 4TB

From the last upgrade of my Windows Desktop PC, i have left:

• NVIDIA GeForce GTX 1060, 6GB (ASUS)
• BeQuiet! Dark Power Pro 750W PSU

With that, I want to create a server with two purposes:

1. Backup/long term storage for (mostly) Media and Software Development files
2. Renderfarm for Blender projects (well, with those GPUs, more like a Render-Backyard...)

For the first task, I ordered two 16TB HDD (Seagate Exos X X16) which should arrive in a few days.

For the second task, i intend to install both GPUs in the server and to use the old 4TB HDD as a temporary storage for the renderer. It is also less about performance; I want the system to run over night or in the background while I use my Desktop for more productive work (and gaming, of course).

The 250GB SSD would serve as the system drive with the OS and all applications.

 

I do neither have (the space for) a second display nor a second keyboard, so an ideal solution would be to set up the server to be controllable remotely via my local network. The files on the server need to be accessible (at least read and write) from both Windows (via File Explorer and WSL) and Linux (via OpenSSH).

 

My questions are:

• What is the best operating system on the Server for those intended purposes? I'm most familiar with Debian and its derivates (Kubuntu), but i'm a bit afraid of the long update delays.
• Are OSes specificly designed for NAS (like OpenMediaVault or TrueNAS) a valid or even the better option?
• Do I need GUI applications like Plex, Jellyfin, etc., for remote access, or is all that achievable with CLI tools and SSH access (on the server) only?
• The two 16TB HDDs should store the files redundantly. What RAID level is possible and recommended with this setup? As far as i understand, the main board only supports RAID Levels 0, 1, 5, and "10". Is it theoretically possible to use other levels (maybe software-controlled or something)?
• Does it make sense to encrypt the backup HDD(s)? Would that work with RAID?
• Would it be possble to use this machine also as a "recovery drive" for my main Windows PC (i thought of something like partitioning the 4TB HDD into two parts with one serving as Recovery Drive for Windows and the other one as temporary Blender storage)? Or, alternatively, would it be possible to "clone" the system drive of my Windows machine and store it on the server in such a way that i can use it to restore my Windows PC, if that should be necessary? This is because i'm still using Windows 10 on my Desktop and want a bullet-proof backup solution before I upgrade to Windows 11.

[AdamBGames]

42 minutes ago, Lupino said:

Hi, all.

I have my old computer collecting dust in the corner. It has the following specs:

•  MB: Gigabyte GA-Z87X-D3H
•  Intel Core i7-4770K 3.50GHz
•  HyperX HX318C10FK2/16 Fury, DDR3, 16GB (Kit 2x 8GB), 1866MHz, CL10, DIMM
• NVIDIA GeForce GTX 760, 4GB (MSi)
• Samsung SSD 840 EVO 250GB
• Seagate Desktop HDD.15 SATA III 4TB

From the last upgrade of my Windows Desktop PC, i have left:

• NVIDIA GeForce GTX 1060, 6GB (ASUS)
• BeQuiet! Dark Power Pro 750W PSU

With that, I want to create a server with two purposes:

1. Backup/long term storage for (mostly) Media and Software Development files
2. Renderfarm for Blender projects (well, with those GPUs, more like a Render-Backyard...)

For the first task, I ordered two 16TB HDD (Seagate Exos X X16) which should arrive in a few days.

For the second task, i intend to install both GPUs in the server and to use the old 4TB HDD as a temporary storage for the renderer. It is also less about performance; I want the system to run over night or in the background while I use my Desktop for more productive work (and gaming, of course).

The 250GB SSD would serve as the system drive with the OS and all applications.

 

I do neither have (the space for) a second display nor a second keyboard, so an ideal solution would be to set up the server to be controllable remotely via my local network. The files on the server need to be accessible (at least read and write) from both Windows (via File Explorer and WSL) and Linux (via OpenSSH).

 

My questions are:

• What is the best operating system on the Server for those intended purposes? I'm most familiar with Debian and its derivates (Kubuntu), but i'm a bit afraid of the long update delays.
• Are OSes specificly designed for NAS (like OpenMediaVault or TrueNAS) a valid or even the better option?
• Do I need GUI applications like Plex, Jellyfin, etc., for remote access, or is all that achievable with CLI tools and SSH access (on the server) only?
• The two 16TB HDDs should store the files redundantly. What RAID level is possible and recommended with this setup? As far as i understand, the main board only supports RAID Levels 0, 1, 5, and "10". Is it theoretically possible to use other levels (maybe software-controlled or something)?
• Does it make sense to encrypt the backup HDD(s)? Would that work with RAID?
• Would it be possble to use this machine also as a "recovery drive" for my main Windows PC (i thought of something like partitioning the 4TB HDD into two parts with one serving as Recovery Drive for Windows and the other one as temporary Blender storage)? Or, alternatively, would it be possible to "clone" the system drive of my Windows machine and store it on the server in such a way that i can use it to restore my Windows PC, if that should be necessary? This is because i'm still using Windows 10 on my Desktop and want a bullet-proof backup solution before I upgrade to Windows 11.

I think the best option for you would be to run a hypervisor like Proxmox or Unraid, then pass through devices for specific things. So for example, passing the drives to a Truenas scale VM, then the GPUs to say a Windows VM to run blender, connecting it back to the Truenas scale VM for larger storage access, via Samba.

2 drives can only be put into a raid 0 or 1, there is no other possible configurations you can have with too few drives.

In terms of encryption, if you were to use the Truenas Scale VM, it can create encryption keys for those drives, but it would also mean that moving the drives at any point would be incredibly difficult, possibly using a motherboard bios encryption could work, but that's up to you.

 

Yes, if the Truenas VM has a dedicated IP, you can use it via samba to create a recovery drive.

[Lupino]

Thanks for your reply, i will look into hypervisors.

 

10 minutes ago, AdamBGames said:

2 drives can only be put into a raid 0 or 1, there is no other possible configurations you can have with too few drives.

 

When I add two (or more) additional drives at a later time, is it possible to "upgrade" a RAID level? Ideally without losing any data?

 

[AdamBGames]

7 minutes ago, Lupino said:

Thanks for your reply, i will look into hypervisors.

 

 

When I add two (or more) additional drives at a later time, is it possible to "upgrade" a RAID level? Ideally without losing any data?

 

No, however, in TrueNAS, you can create additional VDevs, however those VDevs will work indipendantly when it comes to redundancy.

So for example, if we have 15 drives.

We can have those drives in 3x5 drive VDev Raid Z1s (Raid 5), meaning 1 drive failure PER group. But if 2 drives were to die in a single Raid Z1, that would lose data. It's not a good solution, but is basically the only solution if you want to upgrade later without doing a migration to another set of drives as a middle man.

The VDevs will look to the end user as a single drive which is mountable, but to ZFS it will be 3 seperate raids added together.

[Ralphred]

20 hours ago, Lupino said:

Backup/long term storage for (mostly) Media and Software Development files

When you use the words "long term" and then have "ddr3" in the spec list my first reaction is to recommend a ubiquitous software raid and volume management system, that way if something goes wrong with the server itself, the data can be transferred to a "new server" without having to try and find a compatible hardware raid controller.

We all know how different raid levels work, so what you want to think about for long term storage is how much data will be lost if there is a catastrophic raid failure. From that point of view if you extend your raid 5/6 beyond 3/4 disks your entire data set will be lost after 2/3 drive failures. There is no reason (storage given to raid vs storage provided by raid) to extend a raid 1 - meaning dataset lost due to catastrophic failure will always limit self to that (raid set) of the whole.

Now, when we are dealing with spinny disks we get to decide which part of the physical disk get used, and for proper long term storage default to the old idea of a LAID array, the idea being that it's essentially a set of raid 1 arrays where the number of arrays = number of disks. To give a proper understanding of how it works assume we have three 2TB disks, each split into 2 partitions (of 1 tb each) 1st half A and second half B. From here we formulate 3 raid 1 arrays so that:

1. Part 1B + Part 2A = array 1
2. Part 2B + Part 3A = array 2
3. Part 3B + Part 1A = array 3

What this allows over a Raid 2,3,4,5 or 6 array is that only 2 of the disks need to be powered up for writing of new data, the rest are only woken up for reading old data. Once this is extended to more than 4 disks, if you lose two of them (as with any set of raid 1 arrays) you may not lose any data, but apposed to standard raid 1 if you lose two disks in the same array, you only lose 1/2 a disks worth of data. This is where you can be clever with the next layer, logical volume management (LVM2).

LVM lets you collect all your arrays and present them as 1 disk, but you can set it up so that when one of the arrays become "degraded" is will move all that arrays data away from that array, when you come to replace the damaged disk the rebuild will be much quicker as the data is already gone, and you downgrade the risk of a secondary drive failure during the "extra rebuild load", something you can't do with raid 2-6.

 

21 hours ago, Lupino said:

• I'm most familiar with Debian ..snip.. but i'm a bit afraid of the long update delays

Stick to what you know* when doing new things, the backend stuff (software raid, LVM, samba4 for windows shares, remote access tools) is all very mature and not an issue to not have bleeding edge. Blender and GPU compute drivers, well that's a different story. The requirement of "stable backend - bleeding edge front end" screams Gentoo, but for the uninitiated that's quite a task.

 

As far as management, if you use the lower level tools everything is possible in the shell, so just ssh. If you want to do GUI management then running X in a vnc server works fine for remote login from any OS.

 

21 hours ago, Lupino said:

Or, alternatively, would it be possible to "clone" the system drive of my Windows machine and store it on the server in such a way that i can use it to restore my Windows PC

Yes, that's totally doable, but it's not necessary to have a whole "clone drive", you can literally "clone the drive into a file", in your main storage area. The easiest way would be to make a bootable usb with a silly little "Clone and overwrite, new clone, restore clone" menu.

 

*If you do it right, you should be able to pull the cables, switch out the motherboard, plug in a new one and switch it on and it'll "just werk".