Kadah

Mistakes were made
 
Trying to do too many things in the same window.
 
No planning or strategy. Seems like it was mostly a general plan of action and them figuring out the details during. That's fine if Monday isn't a thing.
 
Replacing core network hardware but not configuring it beforehand when there was no time constraints.
 
Not labeling everything so not to spend lots of time looking for a cable or port.
 
Not just removing the door of the server rack. Usually those just easily lift off for this exact reason. Prop the server room door open and get that keyboard out of the way. That room is painfully small and it's like they tried to make it harder.
 
If servers are getting dusty, fix air filtration.
 
Unless reorganizing a rack, remove servers from service and the rack for maintenance and inventory one at a time. Don't further complicate everything by doing all at once plus lots of other stuff at the same time too. Kinda goes with the first point, do the different things in different maintenance windows.
 
If you have A thru Z tasks which could be independent of each other to varying extents, then you start A to Z all at once and get stuck on G and M for a very long time, you'll have to still finish and put back in to service the other 22 as well before Monday.

OEM Microsoft licenses do not have transfer rights, the license is locked to the PC and MS defines that as the mobo for Windows 10. This is why OEM licneses are cheaper, they have this restriction. (If you upgrade from an OEM license of 7 or 8 to 10, that 10 license will also not have transfer rights.)
 
Retail Windows licenses have transfer rights, though if you reactivate too many times by MS's determination and hyper complex rules, they will stop you and require you to buy another license.
 
Microsoft licencing is overly complex for business use, this may be intentional. Linus's use is an edge case. My guess is that what he needs a per user volume license, BUT the reactivations on that get even more complex. The only correct answer here is "talk to a MS licensing partner then blame them if you ever end up out of compliance during an audit after licensing how you were instructed." At a previous job, they got audited by MS and were found out of compliance by around a dozen devices/users on their volume license and were fined over $2 million. I don't know if MS still does those audits now, but making sure things remained in compliance after that was a massive headache.
 
One gotcha about MS volume licensing is that its more expensive than buying individual licenses, and its more complicated than individual licenses. Unless you have 1000's of workstations, it will not be worth the hassle, so just buy your machines from your OEM with Windows pre-licensed.
 
OSX is licensed not to the user but to the device. If you want to virtualize OSX, its allowed by the OSX terms but you have to do it on Apple hardware, which is a huge pain in the ass for software development and testing as Apple killed the xServe forever ago, Mac Pro haven't been viable for years, the Mac Mini might be dead, and new Macbook Pros are a terrible choice for actual computational workloads like a build farm.

OEM Microsoft licenses do not have transfer rights, the license is locked to the PC and MS defines that as the mobo for Windows 10. This is why OEM licneses are cheaper, they have this restriction. (If you upgrade from an OEM license of 7 or 8 to 10, that 10 license will also not have transfer rights.)
 
Retail Windows licenses have transfer rights, though if you reactivate too many times by MS's determination and hyper complex rules, they will stop you and require you to buy another license.
 
Microsoft licencing is overly complex for business use, this may be intentional. Linus's use is an edge case. My guess is that what he needs a per user volume license, BUT the reactivations on that get even more complex. The only correct answer here is "talk to a MS licensing partner then blame them if you ever end up out of compliance during an audit after licensing how you were instructed." At a previous job, they got audited by MS and were found out of compliance by around a dozen devices/users on their volume license and were fined over $2 million. I don't know if MS still does those audits now, but making sure things remained in compliance after that was a massive headache.
 
One gotcha about MS volume licensing is that its more expensive than buying individual licenses, and its more complicated than individual licenses. Unless you have 1000's of workstations, it will not be worth the hassle, so just buy your machines from your OEM with Windows pre-licensed.
 
OSX is licensed not to the user but to the device. If you want to virtualize OSX, its allowed by the OSX terms but you have to do it on Apple hardware, which is a huge pain in the ass for software development and testing as Apple killed the xServe forever ago, Mac Pro haven't been viable for years, the Mac Mini might be dead, and new Macbook Pros are a terrible choice for actual computational workloads like a build farm.

The main issue is that ESXi likely does not support that mobo and won't be able to make a datastore. Whiteboxing ESXi is hit and miss, mostly miss.

I used to do it all the time, including production business use cause no budget and such at the time. It was really dumb, and caused numerous issues later on, but it worked mostly for 6+ years. Now I've can get budgets (after 6 months of requesting...) and have an entire rack for vSphere. 
 
Whitebox ESXi either works or not, pretty just has to try it and see. You can try checking 3rd party HCL lists for whiteboxing and for any driver packs that might help, but the one I used to use hasn't been updated since 4.1.
 
No idea what OP's actually intended load is yet, so no idea what would work. You can get away with a lot on small deploys. 

True but for any virtual host using desktop parts is never recommended. If you have problems installing and running ESXi on the hardware you have you shouldn't be using it for this kind of usage at all, ESXi or anything else.
 
Totally fine thing to do for home or test lab stuff but not the smartest thing to do for business use. Refer back to my declared bias towards VMware, there is a reason why systems running ESXi are so reliable and its not all down to ESXi itself.

ESXi is free, there are no VM limits. Only the hardware you have limits how many you can run. The paid features from VMware only come in to play with multiple hosts and requirements beyond just simply virtualizing.
 
Virtualizing FreeNAS is also not recommended, you can do it but you shouldn't. If you are going to do this you need to know more than the basic theory behind virtualization.
 
In my personal opinion if you are looking at building a system for production business use you first point of call should be ESXi. Yes I'm biased, yes other options work very well but VMware is the most widely used, best supported and the most trusted. 

ISPs by default block port 25 now days to prevent wide spread deployment of open mail relays on home internet connections. Just give your ISP a ring and ask for it to be unblocked.

Also FYI the Nic cards actually have a on board web interface and serial controller

Doesn't need to, treat it as a physical to virtual if a direct VM conversion is not possible.

Doesn't a Windows Server license cost like $800 or something? So wouldn't most of your budget go towards that?

This would be very expensive. Easily a few thousand for the cards plus requiring dual port SAS drives (think $400/drive as cheap). Unless you need the 24/7/365 uptime, you'll not need redundant hot raid controllers. All good raid controllers do not store the array config on the card but on the array itself, this allows you to have a cold spare in the event of a failure if you have the budget for it and cannot withstand the downtime to overnight a replacement.
(Personally, I've have none of my highend HP raid cards actually fail since I switch away from LSI 6 years ago.)
 
You did not say what the operational storage was, but if its something like a highly active SQL server, then that can change a lot of things. There are a lot of things you can do to make solid reliable storage, but money is going to be a very major factor.

I use an SSD in an ESX 5.1 - server for cache and makes a very noticeable difference (the Datastore is still on regular SAS - disks but the SSD is used as cache).

I've had arrays import from P420's on to P812's. Wasn't intentional as the drives were going to be wiped. Normally you would clone data from the old array to a new one in place or from a backup.
 
Same for raid migrations, unless the downtime can not be tolerated, it's much faster to import from back up on to a new array than to expand. On my first level backup sever, the last expansion of adding 3 docks took over 12 days while an import from backup would have taken less than 2. 2 days without the backup system was not a good option, so the slow expansion was preferred in that case.
 
ZFS does not have equivalent expansion capabilities, only adding of addition storage to the end of the data set. (Think LVM)
This is why I use it on VMs with over provisioning. Define the vmdk with the max I think it will need before the VM well need to be replaced or redesigned to support more, and add additional storage to the host (usual SAN).