Help! Trying to decide what to install and how to configure my home server.

[jncunha]

Hi,

 

As the title says, I cannot choose the best OS/configuration to my home server.

 

• Network file share
• Virtualization (Win, Linux, Metasploit, etc, mainly for pentesting)
• Torrent box
• Plex
• VPN server
• Web server
• OwnCloud (or similar service)
• Local data backup (CrashPlan in various devices and RAID setup in server)

 

This is what I need the server for. My system is the following:

 

• Asus P6T Deluxe v2
• i7 920 (3,8 GHz OC)
• Noctua NH-U12P SE2
• Radeon HD 5870
• Corsair HX750
• G.skill Ripjaws 6 GB DDR3 1600 (PC3 12800)
• Samsung 850 EVO 250 GB
• 4x WD Red 3 TB

 

Now my biggest problem is what SO I should run to make all this happen.

I want to give it a try at Linux. At least for learning purposes. I just don't know which distro I should use. I started with Ubuntu Server but I start reading comments with other opinions regarding home servers. Either way, I have not discarded Ubuntu Server from the possibilities.

 

I'm currently having a hard time choosing between FreeNAS, CentOS, Ubuntu Server or even Amahi. I don't care about the easiest route. I want this to serve me the best way.

 

I don't want this to be mainly a NAS. I do want all my devices to back up through CrashPlan or similar service to the home server and then all the backups stored in RAID.

At the same time I'm going to use it to host small websites, run some VM's for exploiting tests and to be a media server for my living room.

 

I've been told to run the the OS and on top of it to virtualize every application so I can isolate them.

 

What do you guys think? What's my best solution? I imagine that FreeNAS is mainly for a server that is a backup machine. I don't want it to be stuck to that functionality or to slow down the other server services.

 

Is there any place I can refer to do this in a organized way? I've never set up a home server before.

 

Thank you for all the help.

[Vitalius]

7 minutes ago, jncunha said:

I'm currently having a hard time choosing between FreeNAS, CentOS, Ubuntu Server or even Amahi. I don't care about the easiest route. I want this to serve me the best way.

 

I don't want this to be mainly a NAS. I do want all my devices to back up through CrashPlan or similar service to the home server and then all the backups stored in RAID.

At the same time I'm going to use it to host small websites, run some VM's for exploiting tests and to be a media server for my living room.

 

I've been told to run the the OS and on top of it to virtualize every application so I can isolate them.

 

What do you guys think? What's my best solution? I imagine that FreeNAS is mainly for a server that is a backup machine. I don't want it to be stuck to that functionality or to slow down the other server services.

 

Is there any place I can refer to do this in a organized way? I've never set up a home server before.

 

Thank you for all the help.

FreeNAS isn't easy to virtualize other OSes in (like Windows). Amahi is probably too bare bones. If you want to use Amahi you may as well use your own linux build.

If I were you, I would just use Debian. It's the middle of "does everything for you" and "you do everything yourself". i.e. in terms of ease of use it goes Ubuntu > Debian > Arch/Gentoo.

FreeNAS has plugins for it that are run in jails. Jails are isolated environments that are effectively virtual machines, but jails don't support Windows and other OSes like it. They're always unix or linux. But you can run VirtualBox in a jail. https://forums.freenas.org/index.php?threads/virtualbox-in-a-jail-in-freenas.20185/ So that's an option, it just takes added work.

FreeNAS is focused on storage, yes, but it has plugins like ownCloud and Plex Media Server that are literally one click to install in their own jail. They can be a bear when configuring the permissions for the files so that the jails have access to them like they should. There are guides though. https://forums.freenas.org/index.php?threads/tutorial-how-to-install-plex-in-a-freenas-9-3-jail-updated.19412/

There's a huge caveat to FreeNAS and plugins though. And that is that to use plugins, you must use ZFS as your file system. When setting up FreeNAS for the first time, you will be asked what you want to format your drives as. ZFS or UFS. ZFS is a powerful and feature filled file system. UFS is more general and easy to use. 

ZFS has certain hardware requirements. Specifically, minimum: 8GB of RAM. And it is beyond highly recommend that RAM is ECC RAM. Your server does not fulfill either of those. You will not get help on FreeNAS' forum because there is no good way to troubleshoot an issue with a system that did not follow those requirements and still used ZFS.

You can still use ZFS with that system. There are people who have used ZFS on FreeNAS without issue with less than 8GB of RAM and non-ECC RAM. That's not really the point though. Some people run a system for years with that setup and never have issues. Others wake up one day and all their data is gone or 100% corrupt. You don't want to mess with that if you care about your data at all.

This guide is fairly old, but should be helpful if you do decide on Debian. 

If you want something more along the lines of a complete guide, you should try Arch or Gentoo. Both require you to essentially manually do everything in terms of installation. It's a good learning experience, but it's easy to mess up. 

The Arch and Gentoo wikis are fairly detailed but don't hold your hand. https://wiki.archlinux.org/index.php/KVM <for virtualizing OSes like Windows and other Linux systems.

https://wiki.gentoo.org/wiki/QEMU <Same, but for gentoo.

Generally speaking, the guides I linked above for gentoo and arch can work for other linux distros (like Debian). You just have to find Debian's packages that do the same thing the arch/gentoo packages do and remember to adjust commands accordingly. i.e. debian's package manager uses apt-get while arch uses pacman. 

https://wiki.debian.org/QEMU Usually if you google the thing and the distro name, the distro's wiki has a page on how to install it and what to consider.

[jncunha]

Wow great answer. Thank you very much for your help!

 

Although FreeNAS has its advantages I think I'll try Debian as you said.

Would you simply install all these applications in Debian or would you run something to virtualize everything that runs on the home server? What would you use?

 

Thank you.

[Vitalius]

32 minutes ago, jncunha said:

Wow great answer. Thank you very much for your help!

 

Although FreeNAS has its advantages I think I'll try Debian as you said.

Would you simply install all these applications in Debian or would you run something to virtualize everything that runs on the home server? What would you use?

 

Thank you.

You're welcome.

I would just install all these applications on Debian. There is merit to isolating them, but odds are, unless you do something wrong in installing them, there won't be an issue with them all running on the same computer.

I currently have Plex Media Server installed on my Arch Server (and I had it installed on my Debian server before that). I will one day care to deal with ownCloud, but my internet speed means that isn't worth doing at this time. 

I would virtualize the OSes you want to use for testing using QEMU, and I would install Samba if your main computer (i.e. the one you are on now or whatever you use to browse the web, watch content, or wtv) is Windows. Configuring Samba is fairly easy. If you use Apple, you'll want AFP for sharing files over the network to it, and if you use Linux, Debian comes with NFS installed. 

Alternatives to virtualizing with QEMU: VirtualBox, VMware, Xen, and KVM.

Note: there is AQEMU which is a GUI for QEMU in case you prefer GUIs over CLI.

You'll want to install LAMP (Linux, Apache, MySQL, PHP) for a web server (you can swap out Apache, MySQL, or PHP for your preferred web daemon, database software, or Server-Side scripting language respectively). 

Other options for Web Daemons: nginx and lighttpd.
Other options for database software: MariaDB, Oracle, and Percona.
Other options for Server-Side scripting language: CGI/Perl, ASP, Java, and Cold Fusion.

For VPN, I know of two options. Either just using OpenVPN, or using SoftEther. I've used both. Both work well.

Here's a list of plugins for FreeNAS to give you an idea of what other software you might want to use: 
 

[DeletedAccount]

In my opinion the easiest method of virtualisation is to install Vmware ESXi. It can be easily controlled from a windows application and new servers can be deployed quickly and easily.

[Eniqmatic]

Exactly. Why choose one OS? Install ESXI, Hyper-V or another hypervisor and virtualise, that way you don't have to choose one.

[jncunha]

1 hour ago, Thetechnoman1 said:

In my opinion the easiest method of virtualisation is to install Vmware ESXi. It can be easily controlled from a windows application and new servers can be deployed quickly and easily.

 

55 minutes ago, Eniqmatic said:

Exactly. Why choose one OS? Install ESXI, Hyper-V or another hypervisor and virtualise, that way you don't have to choose one.

 

So you're both talking about installing ESXI as the "main OS" and virtualize all the OS's I want with each independent application? I'm sorry for the noob question. I'm still trying to grasp all of this. I still haven't really understood how ESXI works as the "base" of the server.

[Mikensan]

If you're wanting to run some virtual machines you're going to need more RAM than 6gb. The host OS is going to need RAM on its own and that doesn't leave anything for VMs.

 

If Crashplan is a must, stay away from FreeNAS. I've fought the good fight to try and get crashplan working - their current plugin is not up to date and Crashplan made significant enough changes that the guides out there do not work. There is a port but it requires the host OS have some linux binaries/kernel whatever installed which FreeNAS does not. FreeNAS does not play nice with changes done outside of the web front unless it's within the Jail. Only way I've found is to mount a share to another computer and then use crashplan.

 

Otherwise FreeNAS can do everything else you want and is a terrific solution. It does want direct communication with the disks, so you cannot use a RAID card and if nesting it inside of ESXi you need a HBA card or something to directly assign to the FreeNAS vm.

 

For the most flexibility and immense support / guides / tutorials out there, use Linux. Any debian based (including ubuntu) distro is going to work great. I would maybe setup Linux + mdadm + btfrs + KVM + docker. It's going to require some elbow grease but it's been done a million times so there's a lot of support. Honestly I've given some thought about switching from FreeNAS - but I'm spoiled.

 

Or honestly, you could run Windows Server 201X and that'll do everything you want it to. I believe with 2016 you don't even need a Hyper-V specific installation to get Hyper-V. I have little to no experience with Hyper-V but somebody else could explain further on it.

[Mikensan]

4 minutes ago, jncunha said:

 

 

So you're both talking about installing ESXI as the "main OS" and virtualize all the OS's I want with each independent application? I'm sorry for the noob question. I'm still trying to grasp all of this. I still haven't really understood how ESXI works as the "base" of the server.

ESXi is just a hypervisor. If you've ever used vbox / KVM / vmware player / vmware workstation - it is the same concept except there is no windows in between it and the "bare metal."

 

So you install ESXi on a box, then go to your personal computer and remotely manage it and setup virtual machines. You can download and install vmware desktop client to manage it, or I think the latest version of 6.0 allows you to manage it from a website hosted off of the machine you install esxi to (no setup required, natively installed).

[Eniqmatic]

10 minutes ago, jncunha said:

 

 

So you're both talking about installing ESXI as the "main OS" and virtualize all the OS's I want with each independent application? I'm sorry for the noob question. I'm still trying to grasp all of this. I still haven't really understood how ESXI works as the "base" of the server.

Yes, ESXI is what is known as a "hypervisor". In essence, it allows for a much smarter provisioning of hardware because you share the resources out to multiple "virtual machines". For example, instead of having one physical machine that has 8 cores and 8GB of RAM that you would probably never utilise, you could have 3 virtual machines each running 2 cores and 2GB of RAM. It makes much more use out of the available hardware. There are many many many more advantages, but in very very basic terms this is why you would do it.

So in answer to your question, yes it would allow you to choose different OS's for different applications, depending on what suits your needs better, all contained within the same physical host.

[jncunha]

1 hour ago, Mikensan said:

If you're wanting to run some virtual machines you're going to need more RAM than 6gb. The host OS is going to need RAM on its own and that doesn't leave anything for VMs.

 

If Crashplan is a must, stay away from FreeNAS. I've fought the good fight to try and get crashplan working - their current plugin is not up to date and Crashplan made significant enough changes that the guides out there do not work. There is a port but it requires the host OS have some linux binaries/kernel whatever installed which FreeNAS does not. FreeNAS does not play nice with changes done outside of the web front unless it's within the Jail. Only way I've found is to mount a share to another computer and then use crashplan.

 

Otherwise FreeNAS can do everything else you want and is a terrific solution. It does want direct communication with the disks, so you cannot use a RAID card and if nesting it inside of ESXi you need a HBA card or something to directly assign to the FreeNAS vm.

 

For the most flexibility and immense support / guides / tutorials out there, use Linux. Any debian based (including ubuntu) distro is going to work great. I would maybe setup Linux + mdadm + btfrs + KVM + docker. It's going to require some elbow grease but it's been done a million times so there's a lot of support. Honestly I've given some thought about switching from FreeNAS - but I'm spoiled.

 

Or honestly, you could run Windows Server 201X and that'll do everything you want it to. I believe with 2016 you don't even need a Hyper-V specific installation to get Hyper-V. I have little to no experience with Hyper-V but somebody else could explain further on it.

 

58 minutes ago, Mikensan said:

ESXi is just a hypervisor. If you've ever used vbox / KVM / vmware player / vmware workstation - it is the same concept except there is no windows in between it and the "bare metal."

 

So you install ESXi on a box, then go to your personal computer and remotely manage it and setup virtual machines. You can download and install vmware desktop client to manage it, or I think the latest version of 6.0 allows you to manage it from a website hosted off of the machine you install esxi to (no setup required, natively installed).

 

49 minutes ago, Eniqmatic said:

Yes, ESXI is what is known as a "hypervisor". In essence, it allows for a much smarter provisioning of hardware because you share the resources out to multiple "virtual machines". For example, instead of having one physical machine that has 8 cores and 8GB of RAM that you would probably never utilise, you could have 3 virtual machines each running 2 cores and 2GB of RAM. It makes much more use out of the available hardware. There are many many many more advantages, but in very very basic terms this is why you would do it.

So in answer to your question, yes it would allow you to choose different OS's for different applications, depending on what suits your needs better, all contained within the same physical host.

I've learned more in a forum post than in my search in the last week. Thank you so much guys.

 

Another question. I understand the concept of using ESXI to be the "base" of everything that is a service (Transmission, OpenVPN, LAMP, etc). My question now is regarding setting up a local data backup, probably with RAID. Would that be configured in an independent virtual machine? Can the Hypervisor have access to the drives? Is the speed reading and writing information the same as if it was configured in a OS directly on the hardware? Maybe now I'm starting to make some really dumb questions. Bare with my ignorance please.

[Eniqmatic]

6 minutes ago, jncunha said:

 

 

I've learned more in a forum post than in my search in the last week. Thank you so much guys.

 

Another question. I understand the concept of using ESXI to be the "base" of everything that is a service (Transmission, OpenVPN, LAMP, etc). My question now is regarding setting up a local data backup, probably with RAID. Would that be configured in an independent virtual machine? Can the Hypervisor have access to the drives? Is the speed reading and writing information the same as if it was configured in a OS directly on the hardware? Maybe now I'm starting to make some really dumb questions. Bare with my ignorance please.

The way virtual machines work is by using "virtual disks". You simply add a virtual disk to a VM, it treats it like a physical disk.

Using ESXI as an example, it uses what's known as "datastores". These can either be local disk storage or network storage (iSCSI/NFS). Since local storage is what you will probably want to do, since it is your first look at virtualization, forget network storage for now.

 

So, in ESXI you would create a datastore on any physical disk you have in the system. Think of it like partitioning a disk in windows. You tell ESXI that you want to use "disk 1" to create a datastore called "datastore-1" and it will do it. From there, you would then create a virtual machine, with a 20GB disk and store it on "datastore-1". Sounds complicated, but really isn't at all. Its just easier to see it rather than trying to explain it. Perhaps I'm just rubbish at explaining! You will note above I said "physical disk", well this can also be a RAID disk. You would just create your RAID in your motherboard BIOS like you would normally, and ESXI will pick this up and treat it as one physical disk for you to create a datastore on.

 

Hope that helps.

[jncunha]

15 minutes ago, Eniqmatic said:

The way virtual machines work is by using "virtual disks". You simply add a virtual disk to a VM, it treats it like a physical disk.

Using ESXI as an example, it uses what's known as "datastores". These can either be local disk storage or network storage (iSCSI/NFS). Since local storage is what you will probably want to do, since it is your first look at virtualization, forget network storage for now.

 

So, in ESXI you would create a datastore on any physical disk you have in the system. Think of it like partitioning a disk in windows. You tell ESXI that you want to use "disk 1" to create a datastore called "datastore-1" and it will do it. From there, you would then create a virtual machine, with a 20GB disk and store it on "datastore-1". Sounds complicated, but really isn't at all. Its just easier to see it rather than trying to explain it. Perhaps I'm just rubbish at explaining! You will note above I said "physical disk", well this can also be a RAID disk. You would just create your RAID in your motherboard BIOS like you would normally, and ESXI will pick this up and treat it as one physical disk for you to create a datastore on.

 

Hope that helps.

Nope you are great explaining. I got it. Basically you give that VM a specific "partition" to work on. Sounds great.

 

For my final question...

Having my CPU, RAM, drives and desired services, how would you split the VM's? Number of cores, available RAM, disk space, etc, for each VM?

Which OS and services in each VM?

Kind of lost in how to organize this. I'm open to any suggestion. Don't care if I use Ubuntu or Debian. Never tried to do anything like this in my life...

(J/K about final question. More will come lol.)

[Eniqmatic]

6 minutes ago, jncunha said:

Nope you are great explaining. I got it. Basically you give that VM a specific "partition" to work on. Sounds great.

Kind of. When you assign a virtual machine a disk, if you look at it in the datastore, the disk is actually one file. So, if you gave a VM a 20GB disk, if you browse the datastore you would see 1 file called "virtual-machine-1" which would be a 20GB file. However, once you enter the VM, its like a normal physical disk. So think of a datastore as a partition, and virtual disks as a file on that partition. That's probably the easiest way to think of it when your starting out.

 

8 minutes ago, jncunha said:

For my final question...

Having my CPU, RAM, drives and desired services, how would you split the VM's? Number of cores, available RAM, disk space, etc, for each VM?

Which OS and services in each VM?

Kind of lost in how to organize this. I'm open to any suggestion. Don't care if I use Ubuntu or Debian. Never tried to do anything like this in my life...

(J/K about final question. More will come lol.)

The hypervisor makes it easy to assign resources exactly as you want them. You can create or edit an existing VM and say "Hey, you can have 2 cores and 4GB of RAM as you will be quite busy" then go to another machine and say "Well you are rarely used and don't do much intensive tasks, you can have 1 CPU and 1GB of RAM." That way you only use what you actually need, rather than wasting tons of resources that never get used. No point a machine having 16GB of RAM when you barely go over 2GB. Make sense?

 

Honestly, the easiest thing for you to do right now is to actually just install a hypervisor on some spare hardware. I would personally recommend ESXI (free) or Hyper-V if you want to use Windows. I prefer ESXI however but choice is yours.

 

There are tons and tons of getting started guides out there, install ESXI (or whichever you choose) and just have a mess around with it to get an understanding. Its by far in the best way. I can sit here and explain things to you all day, but the best way (at least for me) to learn is to see it for yourself.

[jncunha]

7 minutes ago, Eniqmatic said:

Kind of. When you assign a virtual machine a disk, if you look at it in the datastore, the disk is actually one file. So, if you gave a VM a 20GB disk, if you browse the datastore you would see 1 file called "virtual-machine-1" which would be a 20GB file. However, once you enter the VM, its like a normal physical disk. So think of a datastore as a partition, and virtual disks as a file on that partition. That's probably the easiest way to think of it when your starting out.

 

The hypervisor makes it easy to assign resources exactly as you want them. You can create or edit an existing VM and say "Hey, you can have 2 cores and 4GB of RAM as you will be quite busy" then go to another machine and say "Well you are rarely used and don't do much intensive tasks, you can have 1 CPU and 1GB of RAM." That way you only use what you actually need, rather than wasting tons of resources that never get used. No point a machine having 16GB of RAM when you barely go over 2GB. Make sense?

 

Honestly, the easiest thing for you to do right now is to actually just install a hypervisor on some spare hardware. I would personally recommend ESXI (free) or Hyper-V if you want to use Windows. I prefer ESXI however but choice is yours.

 

There are tons and tons of getting started guides out there, install ESXI (or whichever you choose) and just have a mess around with it to get an understanding. Its by far in the best way. I can sit here and explain things to you all day, but the best way (at least for me) to learn is to see it for yourself.

Awesome help. I can't thank you enough. I'm going to do exactly how you said. As soon as I get home I'm going to format the server and install ESXI. From there I'm going to read more and more.

 

Thank you!!

[Eniqmatic]

10 minutes ago, jncunha said:

Awesome help. I can't thank you enough. I'm going to do exactly how you said. As soon as I get home I'm going to format the server and install ESXI. From there I'm going to read more and more.

 

Thank you!!

Sure!

 

Any problems or you want to know more I'm sure everyone here will be happy to help.

[Wallify]

I think virtualization is the way to go, but what I would do is install server 2012, and run a few VMs for the applications.  ESXi can be a pain to manage if you don't know what your doing, and will require another computer on the network.  I would also suggest getting at least 16 GB of ram though.  If your going to run VMs you going to need ram for that on top of the host.  If budget was the issue I would hold off on the RAID configuration and get more RAM.  Crashplan is something like $60 a year and will backup all hard drives in your machine, and you can just keep adding them as needed (I'm up to 8TB in mine).  A note about Crashplan though is the initial upload takes forever and a day, and pulling it all back down is not much better.  Still for the money it cant be beat.  And if your having issues with it, submit a ticket, they have loads of hidden settings to tweak it to significantly better performance.  

[Eniqmatic]

8 hours ago, Wallify said:

I think virtualization is the way to go, but what I would do is install server 2012, and run a few VMs for the applications.  ESXi can be a pain to manage if you don't know what your doing, and will require another computer on the network. 

It's no harder to manage than Hyper-V. What makes you think it would require another PC on the network to manage?

[jncunha]

So after a night trying with ESXi I think I managed to do something useful.

 

First there was a problem with ESXi installation because the ISO didn't contain my network adapter driver so I had to create a customized ISO. Took me a about an hour to figure out how to do it.

 

Now that I have ESXi configured with a static IP address I'm creating these VM's:

 

1 - Web server

• LAMP stack
• OpenSSH
• VSFTPD
• etc

 

2 - Media server

• Plex
• Subsonic
• Couch potato
• Transmission
• etc

3/4/5/etc - Various OS's for learning a bit about pentesting

 

I really don't know if I need OpenSSH in my web server but I'll look more into it. All of these VM's seem ok to me. One machine to all that is web server related and another one to act as a media server for my TV and devices. Then I will have various machines to install temporary OS's for tests.

 

My confusion starts when I want services like a local data backup, a network file share, VPN server, ownCloud or even CrashPlan. Should these "live" all inside the same VM? Should I split these services in different VM's? I'm just trying to figure out the best way to organize ESXi for functionality.

[Eniqmatic]

Glad you seem to have got on quite well! How are you finding it?

 

VPN server I would split into it's own, just from a security point of view (not that it really would matter, just something I would prefer to do). As for the rest, no reason you can't run them from the same VM.

[jncunha]

4 hours ago, Eniqmatic said:

Glad you seem to have got on quite well! How are you finding it?

 

VPN server I would split into it's own, just from a security point of view (not that it really would matter, just something I would prefer to do). As for the rest, no reason you can't run them from the same VM.

I'm loving ESXi to be honest. From a noob perspective it's awesome, since I can screw up my VM's however the hell I want and still I can delete it or return to a previous "clean" snapshot, without having to reconfigure the whole server again.

 

Ok, I'll take what you said into consideration and create a VM only to host the VPN server.

 

On the other hand, l will put the "backup services" inside another VM. My question now is, how can I say to the guest OS to use my WD Red drives as a RAID array and configuring it to make daily backups from all my devices. Maybe using CrashPlan to help with that since it can automate the daily backup? Will the guest OS have trouble finding those drives on the system since it can only see the space I gave it on the SSD? Probably configuring a VM with FreeNAS would help me set all of this but I see a lot of guides indicating that it's not a good idea to virtualize FreeNAS. In another point, I don't have the minimum recommended hardware for FreeNAS. I only have 6 GB of RAM and they are not ECC.

 

Thank you the help!

[Eniqmatic]

1 minute ago, jncunha said:

I'm loving ESXi to be honest. From a noob perspective it's awesome, since I can screw up my VM's however the hell I want and still I can delete it or return to a previous "clean" snapshot, without having to reconfigure the whole server again.

 

Ok, I'll take what you said into consideration and create a VM only to host the VPN server.

 

I will put the other "backup services" inside the same VM. My question now is, how can I say to the guest OS to use my WD Red drives as a RAID array and configuring it to make daily backups from all my devices. Maybe using CrashPlan to help with that since it can automate the daily backup? Will the guest OS have trouble finding those drives on the system since it can only see the space I gave it on the SSD? Probably configuring a VM with FreeNAS would help me set all of this but I see a lot of guides indicating that it's not a good idea to virtualize FreeNAS. In another point, I don't have the minimum recommended hardware for FreeNAS. I only have 6 GB of RAM and they are not ECC.

 

Thank you the help!

That's good, welcome to the world of virtualization!

 

As for creating backup storage space there are a few options.

 

1. Create a RAID at BIOS level on the host ESXI machine, then create a datastore in ESXI using this space. Simply add another disk to the VM that is on your large datastore. For example, say you have 1 x 256GB SSD, and 4 x 1TB WD Reds. You can create a datastore on the SSD, with 250GB free space. You can then create a RAID array across your 4 x 1TB disks, then in ESXI create a datastore on this "logical disk" (it will appear as 1 disk in ESXI, but you will know it is 4 disks, same as any array works). Now you have 1 datastore with 250GB of free space, and one datastore with 3TB of free space (assuming you RAID 5). Now you provision a machine, create a virtual disk and install the OS, you could store this on the SSD drive, assuming you size the OS partition correctly. Setup OS all correctly. Then go back, add a second virtual disk, this time creating a 2TB virtual disk, placing it on the RAID datastore. This will then appear in the VM just a secondary disk would, you can then use it for backups.
2. Option 2 is you do not create a RAID array and you can do what is known as "hardware passthrough" whereby you can directly attach each individual disk (your 4 x 1TB disks) directly to the VM, and use the disks in that way. If you do it this way, no other VM can use those disks at all, you will not be able to create a datastore on them. 

As for virtualizing FreeNAS, it can be done if you follow best practices. People (incorrectly) have it in their heads that "you shouldn't virtualize FreeNAS ever, it will die a horrible death" when this really isn't the case. You should take precautions and as I say follow best practices when doing so. You will need a minimum of 8GB of RAM, preferably more, and you should be doing hardware passthrough as mentioned above when virtualizing FreeNAS. If you do not and you want to use virtual disks, you should disable ZFS scrubbing. FreeNAS likes to have direct access to the disks, hardware pass through gives you this, but virtual disks (and creating RAID arrays in the BIOS) does not. Given your current hardware, I wouldn't go down the route of virtualizing a FreeNAS installation at this moment in time.

[Electronics Wizardy]

3 hours ago, Eniqmatic said:

That's good, welcome to the world of virtualization!

 

As for creating backup storage space there are a few options.

 

1. Create a RAID at BIOS level on the host ESXI machine, then create a datastore in ESXI using this space. Simply add another disk to the VM that is on your large datastore. For example, say you have 1 x 256GB SSD, and 4 x 1TB WD Reds. You can create a datastore on the SSD, with 250GB free space. You can then create a RAID array across your 4 x 1TB disks, then in ESXI create a datastore on this "logical disk" (it will appear as 1 disk in ESXI, but you will know it is 4 disks, same as any array works). Now you have 1 datastore with 250GB of free space, and one datastore with 3TB of free space (assuming you RAID 5). Now you provision a machine, create a virtual disk and install the OS, you could store this on the SSD drive, assuming you size the OS partition correctly. Setup OS all correctly. Then go back, add a second virtual disk, this time creating a 2TB virtual disk, placing it on the RAID datastore. This will then appear in the VM just a secondary disk would, you can then use it for backups.
2. Option 2 is you do not create a RAID array and you can do what is known as "hardware passthrough" whereby you can directly attach each individual disk (your 4 x 1TB disks) directly to the VM, and use the disks in that way. If you do it this way, no other VM can use those disks at all, you will not be able to create a datastore on them. 

As for virtualizing FreeNAS, it can be done if you follow best practices. People (incorrectly) have it in their heads that "you shouldn't virtualize FreeNAS ever, it will die a horrible death" when this really isn't the case. You should take precautions and as I say follow best practices when doing so. You will need a minimum of 8GB of RAM, preferably more, and you should be doing hardware passthrough as mentioned above when virtualizing FreeNAS. If you do not and you want to use virtual disks, you should disable ZFS scrubbing. FreeNAS likes to have direct access to the disks, hardware pass through gives you this, but virtual disks (and creating RAID arrays in the BIOS) does not. Given your current hardware, I wouldn't go down the route of virtualizing a FreeNAS installation at this moment in time.

Id probably stay away form esxi and run something kvm based, like centos with libvirt and kvm. You can then run software raid which would be better than hardware raid, you can also run zfs on the base system.

 

Also kvm is free for the full version, while esxi is limited in the freeversion, and you have to pay for thing like vsphere is you want to make a cluster. ovirt is great if you want a easy cluster of kvm systems.

[b1uew01f]

You're making this way more complicated than it needs to be with vm's with little to no actual benefot beyond learning. The purpose you identified that requires a VM is that you want to play woth multiple OS's; however you seem to have got confused about adding all services to seperate VM's which is not particularly logical at all - it provides no real security benefit (particularly as you likely end up usibg the same login across them all) and means youve got a massive PITA to update it and manage it all.

 

The uses you described can all be achieved on a single machine, there may be slight benefit to multiple vms for restricting resources but i wouldnt. Keep the vm's for testing and playingand remember you can still screw anything system wide.

[Eniqmatic]

1 hour ago, Electronics Wizardy said:

Id probably stay away form esxi and run something kvm based, like centos with libvirt and kvm. You can then run software raid which would be better than hardware raid, you can also run zfs on the base system.

 

Also kvm is free for the full version, while esxi is limited in the freeversion, and you have to pay for thing like vsphere is you want to make a cluster. ovirt is great if you want a easy cluster of kvm systems.

He is learning the basics of a hypervisor and how it works, KVM isn't the easiest way to do that. ESXi, Hyper-V or even to an extend Xen is the easiest way to learn.

 

He is learning and has a single machine with 6GB of RAM, do you think he is going to be clustering at this point?