Rebuilding Home Server - looking for some feedback no Proxmox / TrueNAS setup.

[Simon Bruun]

Hi 

 

So I'm thinking about rebuilding my home server.

It is a Intel I7-4790k (4 core, 8 threads), 32GB Ram - Windows 10 box

With some Disks working as my NAS. 

I also use it for occasional projects, and have an IIS , and SQL Server Express running on it. And also Minecraft from time to time. 
The disk are running in a Windows Storage Space setup. 

 

I never really had any problems with it, and current uptime is 280days ^^ 

 

So why rebuild ?

1. I don't have the greatest feeling about the Windows Storage Space thing. but can't put words to it. 

2. Open op for new possibilities with virtualizations and containers. 

3. For fun. 

 

I made some Powerpoint illustrations to show what i'm thinking about doing, and I would like to hear you guys thoughts on it. and what I might have missed. 

As-Is setup. 

To the left is the "Server" and to the right is my PC

there are these disk

Server:

3 x 250GB SSD

3 x 6TB HDD

PC:

1 x 2TB nvme 

1 x 1TB nvme

4 x 1TB SSD

 

Initial thoughts of the new setup

Install Proxmox as main hypervisor

Install TrueNas on top Proxmox 

(I'm not sure about this, or maybe just go with TrueNas, and use its virtualization features)

I'll use one 250SSD as Proxmox boot drive

Use the rest for TrueNas, where the 2 SSDs would be cashes. 

I'll also need to figure out how to reserve disc space for future virtual machines and Containers. 

I've decided to get rid of Windows Storage Spaces on the PC to, and just have a drive letter for each my discs. 

 

 

How to setup my backup option 1

Assuming 1 redundant disc on the server I'll have aprox 12TB available 

I was thinking to create an equal "partition" on the TrueNAS server for each of my discs on the PC. 

And just make a 1:1 copy to have a backup. And then have the 7TB as free space to play with.

 

How to setup my backup option 2

Sort of the same principle as option 1, but instead all the discs are one big space, and I just create a "folder" matching each of my drives on the PC.

 

 

 

 

I like have plenty of space on my PC, but could agree to move one, maybe to of the 1TB SSDs to the server too, if there would some good reason for it. 

My PC and my Server are LAN Connected with 1Gbps. could an upgrade to 2.5Gbps or 10Gbps befit me in any way ? 

 

PS. I've had this thought, but sort of put it in the back of my head.

All this is quite technical, but I don't doubt that I can manage to get it up and running. But what I am afraid of, is when I return to it in 2 years for what ever reason, I don't remember half of it, and will be afraid to touch it. 

So I did have this idea, of just to keep my "Win10 server", and get rid of Storage space, and just do plain NTFS drives, and have some sync tool taking care of my backups/redundancies. As basic as it can be. 

 

Any feedback is appreciated  

 

[Electronics Wizardy]

6 minutes ago, Simon Bruun said:

Install Proxmox as main hypervisor

Install TrueNas on top Proxmox 

(I'm not sure about this, or maybe just go with TrueNas, and use its virtualization features)

Id personally just use proxmox with zfs on the host for storage. Running a truenas vm for storage just seems to be annoying. But truenas will also work fine for a vm host.

 

6 minutes ago, Simon Bruun said:

I'll also need to figure out how to reserve disc space for future virtual machines and Containers. 

 

What do you mean by reserve disk space. Just leave the disk space unused and vms can use it.

 

7 minutes ago, Simon Bruun said:

 

How to setup my backup option 1

Assuming 1 redundant disc on the server I'll have aprox 12TB available 

I was thinking to create an equal "partition" on the TrueNAS server for each of my discs on the PC. 

And just make a 1:1 copy to have a backup. And then have the 7TB as free space to play with.

Id make one big pool for the PC to use, and mount it as one drive letter. 

 

Then you can back it up on the server. I'm a fan of PBS for this as its also good for backing up vms.

 

8 minutes ago, Simon Bruun said:

My PC and my Server are LAN Connected with 1Gbps. could an upgrade to 2.5Gbps or 10Gbps befit me in any way ? 

 

File copies will be faster with faster networking. Also with 1gbe going ssds for storage will provide very little benfit.

 

9 minutes ago, Simon Bruun said:

So I did have this idea, of just to keep my "Win10 server", and get rid of Storage space, and just do plain NTFS drives, and have some sync tool taking care of my backups/redundancies. As basic as it can be. 

 

Id be tempted to do this. I kinda like storage spaces, but I'm kinda weird. 

 

You can use veeam for backups it works pretty well generally. 

[LIGISTX]

1 hour ago, Electronics Wizardy said:

Id personally just use proxmox with zfs on the host for storage. Running a truenas vm for storage just seems to be annoying. But truenas will also work fine for a vm host.

Truenas virtualized works fine, just need to pass through a HBA. Proxmox can certainly server up ZFS SMB shares, but truenas is built to do it. Truenas can also be a hypervisor…. But proxmox is built to do it. 
 

Neither are great at both, but both are great at each. 
 

1 hour ago, Simon Bruun said:

Use the rest for TrueNas, where the 2 SSDs would be cashes

Truenas and ZFS in general don’t have a cache architecture. It uses RAM for ARC cache. Yes, L2ARC is a thing, but for a home user you don’t need it, and don’t want it. The answer is more RAM. SLOG and ZIL also exist, as so special metadata devices, but…. You also don’t want these for a home setup as they likely won’t help anything, and would just add more points of failure.

 

Another thing to remember about ZFS; you can’t easily add more storage. So I would buy more drives up front and create a large enough vdev that you know for certain will last you many years. The only way to add more space is add another vdev (which h requires it’s own redundancy), or replace all drives in the current vdev with larger capacity.

 

ZFS is great, but it isn’t “the economical choice”. Just be aware of this and build accordingly

 

1 hour ago, Simon Bruun said:

I like have plenty of space on my PC, but could agree to move one, maybe to of the 1TB SSDs to the server too, if there would some good reason for it. 

My PC and my Server are LAN Connected with 1Gbps. could an upgrade to 2.5Gbps or 10Gbps befit me in any way ? 

Direct fiber can be nice, I run direct fiber from truenas to my PC. But it’s not really “worth it”. I also wouldn’t move SSD’s to the server. SATA is much lower latency then going over the network… keep the SSD’s local, it’ll be a much better time. Plus, some things don’t like to run over SMB. You could set up iSCSI mounts, but if you do that, might as well just keep the SSD’s local anyways since the iSCSI mounts can only be used by a single client device anyways. 

[Simon Bruun]

13 hours ago, Electronics Wizardy said:

Id personally just use proxmox with zfs on the host for storage. Running a truenas vm for storage just seems to be annoying. But truenas will also work fine for a vm host.

Thanks, I haven't thought about that, and I don't think I'll need anything to elaborate, so maybe just using proxmox for it, is fine for me. 

 

13 hours ago, Electronics Wizardy said:

What do you mean by reserve disk space. Just leave the disk space unused and vms can use it.

Didn't mean anything by it, just my lack of knowledge on display here xD.

 

13 hours ago, Electronics Wizardy said:

Id make one big pool for the PC to use, and mount it as one drive letter. 

 

Then you can back it up on the server. I'm a fan of PBS for this as its also good for backing up vms.

Thanks for the input, haven't heard about the PBS (Proxmox backup server I assume) before. 

 

13 hours ago, Electronics Wizardy said:

File copies will be faster with faster networking. Also with 1gbe going ssds for storage will provide very little benfit.

 OK, i'll consider it, but can always be done as a side project I guess. 

13 hours ago, Electronics Wizardy said:

Id be tempted to do this. I kinda like storage spaces, but I'm kinda weird. 

 

You can use veeam for backups it works pretty well generally. 

OK, thanks for that, I can't say that I had any problems with Storage Spaces, so your input might convince me stay put.  

I'll give veeam a look.

 

11 hours ago, LIGISTX said:

Neither are great at both, but both are great at each. 

Thanks, yeah, that is sort of the feeling I'm getting, and was not sure what to do. Thinking maybe just use Proxmox, and do a share from there without TrueNas

 

11 hours ago, LIGISTX said:

Another thing to remember about ZFS; you can’t easily add more storage. So I would buy more drives up front and create a large enough vdev that you know for certain will last you many years. The only way to add more space is add another vdev (which h requires it’s own redundancy), or replace all drives in the current vdev with larger capacity.

holy shit dude, I took that as a given, that you could add more dices.  Hmm damn, that might be a showstopper for me, and just stick with storage spaces. 

Thank you for pointing that out. and thanks for info on the cache discs.

 

11 hours ago, LIGISTX said:

Direct fiber can be nice, I run direct fiber from truenas to my PC. But it’s not really “worth it”. I also wouldn’t move SSD’s to the server. SATA is much lower latency then going over the network… keep the SSD’s local, it’ll be a much better time. Plus, some things don’t like to run over SMB. You could set up iSCSI mounts, but if you do that, might as well just keep the SSD’s local anyways since the iSCSI mounts can only be used by a single client device anyways. 

man, so glad i asked before going further with this. 

 

 

Thanks for your feedback.

 

I guess I'm swayed more towards staying on my win10 "server", 

What I think I will miss the most is the Virtualization options of Proxmox, but haven't really got a "use case/home project" for it yet. 

[LIGISTX]

3 hours ago, Simon Bruun said:

holy shit dude, I took that as a given, that you could add more dices.  Hmm damn, that might be a showstopper for me, and just stick with storage spaces. 

Look into unraid. That may be a better solution for your needs. 

[Simon Bruun]

3 hours ago, LIGISTX said:

Look into unraid. That may be a better solution for your needs. 

Checked up on unraid, and I think you are right, this seems to fit perfectly - Thank you !

 

checked out Raid Owls video on it, and looks good to me

 

[Simon Bruun]

So I've been messing around with Unraid the last few days now, and it started off pretty good. 

But now ended up with sort of bad feeling about it. 

 

Not really sure what I'm going to do now, but thought I would share.

 

As stated, started off pretty good, with not too many hiccups, I got it up and running. and build my array, and set up some shares, and even tried Container and VMs.

And do like the power it gives me when it is working.

 

things that bothered me

• I realized that parity takes forever (7-8hours on a 6TB disc), but I think that is normal, but I lost my parity-check progress as I thought it would continue after a reboot. 
• I have not yet been able to select any options in blue boot menu that shows up when it starts, it just picks option 1 and starts the unraid server.
•  Started a file sync tool on my PC to copy data to the unraid server, and at some point my memory log went to 100% and I think it bugged down after that, and needed to reboot to clear the memory. 
• after I asked for the server to reboot it would not boot back up. and had to leave it for the next day. 
• next day it did boot again, and tried turning off fast-boot in uefi, in hope that it would allow me to use the menu for other than option 1. 
  • i'm not sure what it was, but now the machine won't boot again, power goes on, and keyboard lights up, but no signal to monitor. 
  • Maybe clearing CMOS or waiting a day and try again will bring it back up.
  • EDIT: Clearing CMOS got me running agian 

Most of the problems I've googled and there are discussions about problems like mine. That is sort of a good thing, so I know it can be fixed (maybe), but then again. I don't like it is that filled with instability and tinkering to have it running.  

 

What are you experiences with unraid, Did it work out of the box - no problems ?

Or did you have rough startup phase before stability ?

 

Do you guys think I should crack on, and get this running, or abandon ship ?