Optimal Storage Allocation for VMs

[TechNoob9]

Recently got started with a home lab and playing around with multiple VMs. Currently I have 1 SSD (for proxmox and Isos) and then a bunch of HDD.

 

If I wanted to spin up multiple VMs, is the best practice to have 1 small dedicated SSD per guest OS per VM, then have any additional storage (i.e., an additional SSD or HDD?). So if I have 4 VMs, then I should have 1 SSD dedicated to each VM, plus an additional drive if needed for additional storage.

 

Or are there instances of running multiple VMs on one SSD? I feel this is a bit more risky as if that fails, then you lose two VMs instead of one. But I also understand the cost of storage/only so many SATA ports.

[manikyath]

having a dedicated SSD per VM is sort of.. unnecessary?

 

ideally you have a fast array of SSD's with some form of parity (to protect against data loss) and then stick the VM's on top of that, each with exactly the allocation they need.

[Electronics Wizardy]

Normally I'd say its best to have one big array of drives, and then put virtual disks on the drive array. Then you can use raid so drive failure doesn't stop vms. Also vms use little storage typically, so having a whole ssd for a vm thats using only 50GB of less is kinda wasteful.

[LIGISTX]

3 hours ago, TechNoob9 said:

Recently got started with a home lab and playing around with multiple VMs. Currently I have 1 SSD (for proxmox and Isos) and then a bunch of HDD.

 

If I wanted to spin up multiple VMs, is the best practice to have 1 small dedicated SSD per guest OS per VM, then have any additional storage (i.e., an additional SSD or HDD?). So if I have 4 VMs, then I should have 1 SSD dedicated to each VM, plus an additional drive if needed for additional storage.

 

Or are there instances of running multiple VMs on one SSD? I feel this is a bit more risky as if that fails, then you lose two VMs instead of one. But I also understand the cost of storage/only so many SATA ports.

Think of data centers that run thousands, potentially millions of VM’s… at least many dozens or hundreds per host. They certainly do not have dedicated drives, at least not normally. 
 

You can run all the VM’s off the same SSD Proxmox host runs on. FYI, the way Proxmox sets up the datastore is not conducive to this as it allocates 50% to itself… which is dumb. I will link a video below on how to fix this. You have to do this prior to installing any VM’s tho, so if you already have VM’s, back them up, do this, then reload them. 
 

But the general idea is, install VM’s to the same boot drive as proxmox, and use your spinning rust as an array for them to use if/when needed for mass storage. I personally run TrueNAS as a VM, so I present my ZFS array to my VM’s via NFS or SMB, if you run the array in proxmox, you can just directly attach storage to VM’s, either way is fine; virtual networking is all 10gig or faster anyways, so performance differences are pretty moot. 
 

the part you want starts at 14:50, but if your new to proxmox, may be worth just watching the entire video, as well as videos from other creators just to get your bearings. 

[TechNoob9]

5 hours ago, manikyath said:

having a dedicated SSD per VM is sort of.. unnecessary?

 

ideally you have a fast array of SSD's with some form of parity (to protect against data loss) and then stick the VM's on top of that, each with exactly the allocation they need.

Would you keep that separate from the SSD that has Proxmox installed or put everything on the same drive? I know @LIGISTX suggesting to put them all on the same drive, but that seems like it could be risky. (unless it's standard practice).

[manikyath]

25 minutes ago, TechNoob9 said:

Would you keep that separate from the SSD that has Proxmox installed or put everything on the same drive? I know @LIGISTX suggesting to put them all on the same drive, but that seems like it could be risky. (unless it's standard practice).

IMO the host OS should always be on a separate drive from your "data" (VM storage is "data" in this sense)

 

but this gets into a matter of personal preference, the actual hardware you're working with, budgets, and your plans for disaster recovery.

[LIGISTX]

56 minutes ago, TechNoob9 said:

Would you keep that separate from the SSD that has Proxmox installed or put everything on the same drive? I know @LIGISTX suggesting to put them all on the same drive, but that seems like it could be risky. (unless it's standard practice).

It really doesn't matter, especially for a home setup.

 

In the enterprise, no, they wouldn't have VM's share a boot drive as the host, but that isn't because of some unsafe risk - itll be because they will run some 256 GB boot drives, and an array of SSD's for VM's to live on, all while having a high availability mechanism in place... this is stuff most homelabers are not going to have, and thus don't need to architect our setups this way.

 

Think to yourself "what are the failure modes", and then determine what seems best for you. If your proxmox host dies, how do you recover? If your VM drive dives, weather it be the same as the proxmox host or not, how do you recover? Think through this, have backups in place, and then you can do what seems best for you. Personally, I run a Z1 Mirror for proxmox boot, and that drives also has my VM's... and those VM's are also backed up via PBS to my ZFS truenas spinning rust array... Personally I would much rather have 2 drives in a mirror, and on them have my proxmost host AND my VM's, then splitting them up and having to deal with redundancy for both. In the enterprise, they would just have redundancy for both...

 

TLDR; there is no "inherent risk" doing it all in one, it just means if that drive dies and you have no backups, you lose your proxmox host and your VM's. But if you split them up and your VM drive dies.... you still lose your VM's if you have no backup. So moral of the story is, have backups