Jump to content

VMWare ESXI 6.0.0U2 NVIDIA GPU Passthru

Hello LTT community 

 

I have some interesting new to share with the tech community and what better place to share it than here.

 

I have been doing a lot of research on hypervisors and specifically GPU pass through to virtual machines, in my research i have found a number of skeptical and contradictory articles and posts.

 

I plan to discuss and share my findings, first off ill list my system specs, installation of esxi 6.0.0 (aka vsphere), the creation of my windows vm, pci passthrough and finally gaming and benchmarks ( graphs to come in the future)

 

The System:

Bequiet Silent Base 800 

Asus x99 deluxe 

Intel I7 5930K (OC'd to 4.1GHz) esxi stability is important so im not pushing it.

Kingston fury x 16gb 4x4 2133mhz kit (upgrading to 32gb 4gbx8 2666 gskill kit)

Bequiet Dark Rock Pro 2 Cpu cooler ( minor case mod to fit, one of the tabs on the side panel fan cover had to be cut down)

EVGA GTX 970 SSC (for pass thru)

PNY GTX 750ti (for esxi since the x99 platform has no igpu)

Corsair HX 850i 850w platinum powersupply 

samsung ssd's; 256gb 950 pro for VM's and 250gb 850 evo for storage

and a few other drives not configured in vmware yet

Other components include an APC 1500va UPS for good measure

 

ok now on to the installation, 

 

I started by downloading the free version of esxi

the fired up vmware workstation to create a USB image of esxi so i didnt eat up my valuable drives

instructions i followed are here  http://www.vladan.fr/install-esxi-6-to-usb-as-destination-or-have-it-as-source/

 

once that was create i installed the usb in an nzxt internal usb extension that also has 2 usb "A" ports

fire up the PC and mash F8 to get boot devices and pick the esxi usb

 

next go to the address esxi gives you as there is no gui so your gonna need a laptop or a second computer to configure esxi

from the web gui is how you configure esxi for pci pass thru and create your vm

 

First you have to create a data store, i called mine VM's and Games as the evo would be attached to the vm as a secondary vdisk just for games

click on storage, click create new data store and follow the steps and add the drives you want

now click on virtual machines, click create virtual machine, the nice part about creating a vm with esxi is you have the ability to use the client machine as the CD/DVD drive so you can mount and install images from a laptop remotely

my vm is using 8gb of ram and running on 4 virtual cores and as a side note use vmxnet3 for your network as e1000 and e1000e have issues getting to the internet

once your virtual machine is up and running with Windows 10, (im using build 1607 for my vm)  shut down your vm now we will add pci devices

click on manage right under host then click the hardware tab, note that not all devices can be passed thru, on the devices that say disabled you can check the box and click toggle passthrough and that device will be unavailable to the host and now be available to add to your virtual machines.

once youve toggled all the devices you want you will have to reboot the host before the devices will be passed through

 

Once the host is back up and before you start your vm, click on virtual machines and click on your vm, click edit and add other device, select the pci devies you wish to add to your vm the click save

now before you boot up you vm and this is IMPORTANT if you OWN an NVIDIA Consumer GPU (GEFORCE CARDS) you will have to edit the config file and add 2 lines.

to get to the config click virtual machines, click the machine you want to edit, click edit settings, click the VM options tab, click advanced and scroll down and click edit configuration

now that the config is open click add parameter and in the KEY: hypervisior.cpuid.v0 Value: FALSE and the second parameter is KEY: pciHole.start Value: 2048 

not that in the first parameter that is a ZERO after the "v"

 

Now your ready to start your VM fingers crossed you have an output on your display and your gpu passthrough is working correctly 

 

once your back into your vm you will need to download and install nvidia drivers and in device manager uninstall and delete the SVGA adapter.

Now running games is to my eye showing now loss, i have yet to notice any frame rate dips 

Playing the Division i was getting between 35 and 45 fps on ultra @ 1440P and on a base windows install i was getting the same, i will however post some benchmarks of my vm and try to get some benchmarks of the pc running these games on windows on the baremetal 

 

if you have any questions dont hesitate to comment or PM me

 

Thanks

 

 

 

Link to comment
Share on other sites

Link to post
Share on other sites

These two parameters were enough for you to make GTX work with ESXi? I am trying with GT710, but without luck: i was only able to get rid of E43 after reinstalling the drivers, but external display was still not working. Damn NV...

 

Spoiler

99bd8068a77d52a159a6ca8b5bee86f6.jpg

 

Link to comment
Share on other sites

Link to post
Share on other sites

  • 2 years later...

Thanks for that guide!

 

I've actually been following this to get a GTX560ti working on an old Apple Xserve. So far the furthest i've gotten is for windows to recognize the card and GPU Z to see as if its acting normal. my issue being that when I try to connect a display or trick windows into a dummy display, I get a blue screen on the VM with a "nvlddmkm.sys" error. I'd love someones suggestions on what to try? currently working on rolling the driver back to 388.13 if that doesn't work though, i'm out of ideas.

 

working on windows 7 Pro SP1 

 

Xserve specs:

2x Xeon E5462 @ 2.80 GHZ

12GB of ECC RAM

GTX 560ti attached externally via a PCI-E Extender.

Link to comment
Share on other sites

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×