(Guide) Using an NVIDIA Tesla K80 for Gaming on Windows

[2s0ckz]

Given the ongoing GPU shortage, I have seen several posts around the internet about using an NVIDIA Tesla K40 (the datacenter version of the GTX Titan Black, with 12 GB of VRAM) for gaming, so I wanted to share my experience with the Tesla K80, which is essentially two K40s in one card.

These cards can be found pretty cheap on eBay and Amazon right now, and they are absolute monsters with 4492 CUDA cores and 24 GB total memory. I bought mine off of Amazon (https://www.amazon.com/Dell-Tesla-K80-Accelerator-Refurbished/dp/B07GJ45V3D) about 2 months ago for $200, but they are still going for $300 at the link above.

 

Basic Considerations

1. The Tesla K80 draws 300W and uses a CPU 8-pin cable, so you'll need a decent power supply with two CPU outputs.

2. You will need a BIOS with the option to enable "Above 4G decoding" (I’m using an ASUS Prime Z490-A mobo).

3. You will need to be running at least Windows 10 version 20H2 (I'm running 21H1).

4. You will need a CPU with integrated graphics (I'm using an Intel i9 10850-k), or a second GPU with display output.

 

Cooling the K80

Because Tesla cards are designed for servers and use passive cooling, you will need to rig up some DIY active cooling. One option is to buy a 3D printed adapter for a blower fan off of eBay.

What I did was I removed the heatsink shroud via the eight 1.5mm hex screws on the sides, and then peeled off the clear plastic cover that was glued on the inside. This exposes the heatsink on the rear of the graphics card.

I bought a full-length GPU cooler for $50 (https://www.amazon.com/anidees-Graphic-GIGABYTE-Addressable-AI-GP-CL/dp/B07Y6CY6GL/ref=sr_1_4?dchild=1&keywords=gpu+cooler&qid=1621551180&sr=8-4), which blows air into the graphics card through the opening where the clear plastic used to be. If you're not worried about aesthetics, there are some cheaper GPU coolers that go for about $20.

This setup keeps both GPUs at about 35°C on idle, and around 60°C under load, but it uses 4 PCI slots.

 

 

Enabling Graphics

The Tesla K80 is a computing GPU, so Windows will not recognize it as a graphics processor by default, though it can be used for computations and neural network training, etc. In order to trick Windows into using the K80 for graphics, these are the steps that I followed:

1. Enable "Above 4G decoding" in the BIOS.

2. Install the latest NVIDIA Tesla K80 driver for Windows (https://www.nvidia.com/Download/driverResults.aspx/174135/en-us)

3. Reboot

4. Modify the Windows Registry:

   1. Go to the start menu and type in "Regedit", enter.

   2. Navigate to: computer\HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{4d36e968-e325-11ce-bfc1-08002be10318}\0001

   3. Export your registry to make a backup.

   4. Delete the entry: AdapterType.

   5. Create a 32bit dword: EnableMsHybrid and give it a value of 1.

5. Reboot

6. Switch the GPU from compute (TCC) to graphics (WDDM) mode in the command prompt:

   1. Go to the start menu and type in "CMD".

   2. Right-click on "Command Prompt" and run as administrator.

   3. Run nvidia-smi -L to get a list of GPUs and their ID numbers.

   4. Run nvidia-smi -g {ID} -dm 0, where {ID} is the ID of the GPU that you want to use for graphics.

7. Reboot

8. Assign the game executable to run using the K80:

   1. Right-click on your desktop and go to the display settings.

   2. Scroll down and click on "Graphics Settings".

   3. Find the .exe file of the game you want to run using the K80.

   4. Click on the game in the list and select "Options" and choose the "High performance" NVIDIA Tesla K80 GPU.

 

Overclocking (edit)

My K80 came with a GPU clock limit of 562 MHz and a memory clock limit of 2505 MHz. I found the GPU to remain stable (without any modifications to voltage) at a boost clock of 849.5 MHz and a memory clock of 3505 MHz.

These settings can be changed either using MSI Afterburner or by flashing the VBIOS using these instructions: https://linustechtips.com/topic/1058561-simple-tutorial-ish-for-kepler-and-probably-maxwell-ii-bios-tweaker/

Conclusions

Overall, I think this could be a pretty good option for anyone who hasn't been able to get their hands on a new GPU. These cards aren't that useful to crypto miners, so they've generally been available. Also, many data centers are getting rid of these cards in favor of newer options, increasing their availability.

 

Personally, I only switched one of the two GPUs in my K80 to WDDM mode, because I primarily use this card for scientific computing. Similarly, I only overclocked the WDDM GPU (by flashing the VBIOS; MSI Afterburner will overclock both GPUs). Essentially, I now have one 12GB K40 for scientific computing and a second 12 GB K40 for gaming. For the games that I've tested, it operates at a pretty decent average of 60 FPS on high settings.

 

Sources:

https://forum.level1techs.com/t/gaming-on-my-tesla-more-likely-than-you-think/171185?fbclid=IwAR2QMYGD-LFsX7mMjBEspR48zMUR010cqSEEBe8b49YjHTY7aAIZcFSYfzY

https://www.overclock.net/threads/gaming-on-a-tesla-m40-gtx-titan-x-performance-for-a-third-the-price.1777611/

[SorryBella]

now the 12 year olds can unironically run nvidia teslas in their "10000$ oVaHkiLL gAeMiNg StRiMiNg Pc NiNjA eDiTiOn" and not being embarrased by hundreds of people who can smell their bullshit. /sarcasm

 

But in all seriousness,

9 minutes ago, 2s0ckz said:

but they are still going for $300 at the link above.

I can find em off ebay at 260, so thats a tip for everyone looking at this thread. But at normal market scenario, this thing might be pale in raw value compared to something like a 1080Ti.

[Skiiwee29]

<-- Moved to Guides and Tutorials -->

[decolon]

IIRC Geforce Now cloud gaming service uses Tesla K80s (at least back when I was a beta tester for it) and games ran great. Definitely over 100fps at 1080p max settings for competitive games

[Mnky313]

Does anyone know if this will work with the Tesla M40?

I've got some I picked up for cheap to mine on and once the profitability is bad enough I'll probably try and repurpose them to game streaming cards (they are in a server)

I have too much critical shit on the server to reboot it and try this XD, ill get around to testing eventually.

Quote

You will need to be running at least Windows 10 version 20H2

Also this could be problematic for me. Unless I can get Hyper-V to pass a GPU to a VM....

Also if you are memory overclocking I would disable ECC support, this can also be done through nvidia-smi, command is:

nvidia-smi -e 0

I believe. I did this and have the M40s at +650MHz mem for mining.

and thanks for the link to the kepler/maxwell BIOS tweaker

[2s0ckz]

On 6/1/2021 at 4:29 PM, Mnky313 said:

Does anyone know if this will work with the Tesla M40?

I've got some I picked up for cheap to mine on and once the profitability is bad enough I'll probably try and repurpose them to game streaming cards (they are in a server)

I have too much critical shit on the server to reboot it and try this XD, ill get around to testing eventually.

Also this could be problematic for me. Unless I can get Hyper-V to pass a GPU to a VM....

Also if you are memory overclocking I would disable ECC support, this can also be done through nvidia-smi, command is:

nvidia-smi -e 0

I believe. I did this and have the M40s at +650MHz mem for mining.

and thanks for the link to the kepler/maxwell BIOS tweaker

 

Hey, thanks for your tutorial on overclocking! And, yes you're right, I forgot to mention to disable ECC support, though I did that via the NVIDIA Control Panel.

 

I believe this should work for an M40, since I based this partly off of a guide that was originally meant for the M40. Here's the link: https://www.overclock.net/threads/gaming-on-a-tesla-m40-gtx-titan-x-performance-for-a-third-the-price.1777611/

[ErnestJ]

Any idea how to run game on both card cpu ? Only running on one.

[JEF1056]

On 7/6/2021 at 2:56 AM, ErnestJ said:

Any idea how to run game on both card cpu ? Only running on one.

You can't. The two GPUs on the card aren't linked with a SLI or NVLink bridge. Not only that, but I don't really advise running both the cores on the K80 since they get extremely hot without adequate cooling.

[ErnestJ]

On 7/16/2021 at 6:20 AM, JEF1056 said:

You can't. The two GPUs on the card aren't linked with a SLI or NVLink bridge. Not only that, but I don't really advise running both the cores on the K80 since they get extremely hot without adequate cooling.

So then working on one chip what other doing ?

Why its resting then i want to make it game lol. I am not worried about heat, because i modified heatsink and added some delta fans .

 

What if instead of

Quote

Create a 32bit dword: EnableMsHybrid and give it a value of 1.

I will do

 

Quote

Create a 64bit dword: EnableMsHybrid and give it a value of 2.

Is that make card do job harder if im using win 10 64 bit xeon phi workstation ?

Maby that step make all two chips on card work instead of just one ?

[gio_kiborg]

Thanks for the nice guide,

I followed it (and one from level 1 forums referenced) one thing was slightly different: I could not find Adapter type entry in registry key, just added EnableMsHybrid and then nvidia-smi -g {ID} -dm 0  command, after the reboot I was able to specify high performance option K-80 in windows.

I have several of questions:

1) Is it possible to specify k80 as default render gpu in windows (so I do not need to specify each exe to run?

2) does physx work? I installed physX software and configured in Nvidia Driver, but not sure if it works, in Fluid Mark only CPU physx is available and by FPS it is clear that it shows correctly,

 

 

3) there is option in Nvidia Driver to specify Graphics or Compute for each of the GPU, does it mean that nvidia-smi -g {ID} -dm 0  command is not mandatory and it could be specified in a driver?.

 

 

4) As I understand SLI is not enabled/possible, but some games can use dx12 feature to render on any cards, will this work for 2x k80?

Otherwise it is working fine, I cranked up the sliders in MSI Afterburner and so far it is stable (I played Metro Exodus intro 1080P high no hairworks or advanced physX about 80 fps,

Strange thing is that built in benchmark of the same game with similar settings only shows about 35 fps average

 

[JEF1056]

On 7/29/2021 at 10:31 AM, ErnestJ said:

So then working on one chip what other doing ?

Why its resting then i want to make it game lol. I am not worried about heat, because i modified heatsink and added some delta fans .

 

What if instead of

I will do

 

Is that make card do job harder if im using win 10 64 bit xeon phi workstation ?

Maby that step make all two chips on card work instead of just one ?

I want to clairify something. The tesla k80 is a COMPUTE card, and not intended for gaming. In a machine learning application, we don't need a SLI/NVLink Bridge in order to make use of both GPUs on the card. Instead, we can design our programs to effectively split models in two or train two different models on each GPU and choose the best result from them. In this way, we can train on two or more GPUs effectively and see a correlation between more cards and more speed in that task.

 

When you are gaming, the game needs to render one frame at a time. This makes it difficult to use multiple cards that don't share the same compute resources to render that game. As such, the SLI/NVLink bridge facilitates communication between cards and in very specific use cases allows some games to take advantage of both cards' speed.

 

Now that the reason why you can't use both cards simultaneously is cleared up, I need to clarify some other misconceptions you appear to have.

Quote

Create a 32bit dword: EnableMsHybrid and give it a value of 1.

This is not a "performance" switch. "EnableMsHybrid" is a feature of windows that allows a graphics card to work as a "high-power" GPU in a system, where the powerful graphics card pushes frames through the integrated grpahic's card's framebuffer. Changing its value to 2 doesn't do anything and changing it to 64 bits might not even work at all.

 

Next, your claims regarding the xeon phi workstation. DO NOT. USE. IT. FOR. GAMING.
Linus actually did a great video on this exact concept: https://www.youtube.com/watch?v=fBxtS9BpVWs

The TLDR of it is that unless your task is easily paralellizable (like machine learning) you are better off with a machine that only has 2-4 cores that run at higher speeds. Gaming does not beinifit from more cores like machine learning does.

 

In summary, how can you game on other GPU while you're gaming on one half of a K80? The answer, if you're the only one using it, nothing. If you have a friend, you might want to consider using a hypervisor like ESXI to deploy two virtual machines, each with its own half of the GPU, to play games on the same physical computer.

[sorenwp]

How is gaming performance on K80?
- Can you share some benchmark / FPS 

[Konstantin Naumenko]

Hi everyone. I have a similar configuration as the author, but I use two TESLAs K40 12Gb in a desktop computer. I also have a different cooling system, similar to the server's one with 5 coolers and I had to shut off some holes on a case to let the air flow through the video processors, but I think the author's one is better. I use my two TESLAs in windows 10 both for games and applications. I found a slightly different way to setup it by experiment which is better, I think, because you don't have to edit the registry. You are welcome to use it. Here it is:

 

To let it be found and normally installed by Windows 10:

1. Enable "4G Decoding" in the Boot options in BIOS
2. Disable "Fast Boot" in Windows' current Power plan settings

3. Download and install the latest driver from NVIDIA
4. Disable "Error correction Code" in the Nvidia Control Panel (Optional. Have some impact on the calculation speed)

 

To use it for graphics:
1. Run "nvidia-smi -L" to get a list of GPUs and their ID numbers (at the command prompt as administrator)
2. Run nvidia-smi -g {ID} -dm 0, where {ID} is the ID of the GPU that you want to run in WDDM mode

Note: Run nvidia-smi -g {ID} -dm 1, where {ID} is the ID of the GPU that you want to turn back in TCC mode if needed

 

To use it for graphics in a specific application:
1. Run "Graphics settings" from the Windows Control Panel
2. Choose an app to set preference by browsing to its' .exe file
3. Click the application bar, that appeared and select "Options"
4. Choose "High Performace" from the options displayed

Note: Some applications do not require this and will use TESLA by default

 

Best regards

[Legal_EasE]

On 5/20/2021 at 9:35 PM, 2s0ckz said:

Given the ongoing GPU shortage, I have seen several posts around the internet about using an NVIDIA Tesla K40 (the datacenter version of the GTX Titan Black, with 12 GB of VRAM) for gaming, so I wanted to share my experience with the Tesla K80, which is essentially two K40s in one card.

These cards can be found pretty cheap on eBay and Amazon right now, and they are absolute monsters with 4492 CUDA cores and 24 GB total memory. I bought mine off of Amazon (https://www.amazon.com/Dell-Tesla-K80-Accelerator-Refurbished/dp/B07GJ45V3D) about 2 months ago for $200, but they are still going for $300 at the link above.

 

Basic Considerations

1. The Tesla K80 draws 300W and uses a CPU 8-pin cable, so you'll need a decent power supply with two CPU outputs.

2. You will need a BIOS with the option to enable "Above 4G decoding" (I’m using an ASUS Prime Z490-A mobo).

3. You will need to be running at least Windows 10 version 20H2 (I'm running 21H1).

4. You will need a CPU with integrated graphics (I'm using an Intel i9 10850-k), or a second GPU with display output.

 

Cooling the K80

Because Tesla cards are designed for servers and use passive cooling, you will need to rig up some DIY active cooling. One option is to buy a 3D printed adapter for a blower fan off of eBay.

What I did was I removed the heatsink shroud via the eight 1.5mm hex screws on the sides, and then peeled off the clear plastic cover that was glued on the inside. This exposes the heatsink on the rear of the graphics card.

I bought a full-length GPU cooler for $50 (https://www.amazon.com/anidees-Graphic-GIGABYTE-Addressable-AI-GP-CL/dp/B07Y6CY6GL/ref=sr_1_4?dchild=1&keywords=gpu+cooler&qid=1621551180&sr=8-4), which blows air into the graphics card through the opening where the clear plastic used to be. If you're not worried about aesthetics, there are some cheaper GPU coolers that go for about $20.

This setup keeps both GPUs at about 35°C on idle, and around 60°C under load, but it uses 4 PCI slots.

 

 

Enabling Graphics

The Tesla K80 is a computing GPU, so Windows will not recognize it as a graphics processor by default, though it can be used for computations and neural network training, etc. In order to trick Windows into using the K80 for graphics, these are the steps that I followed:

1. Enable "Above 4G decoding" in the BIOS.

2. Install the latest NVIDIA Tesla K80 driver for Windows (https://www.nvidia.com/Download/driverResults.aspx/174135/en-us)

3. Reboot

4. Modify the Windows Registry:

   1. Go to the start menu and type in "Regedit", enter.

   2. Navigate to: computer\HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Class\{4d36e968-e325-11ce-bfc1-08002be10318}\0001

   3. Export your registry to make a backup.

   4. Delete the entry: AdapterType.

   5. Create a 32bit dword: EnableMsHybrid and give it a value of 1.

5. Reboot

6. Switch the GPU from compute (TCC) to graphics (WDDM) mode in the command prompt:

   1. Go to the start menu and type in "CMD".

   2. Right-click on "Command Prompt" and run as administrator.

   3. Run nvidia-smi -L to get a list of GPUs and their ID numbers.

   4. Run nvidia-smi -g {ID} -dm 0, where {ID} is the ID of the GPU that you want to use for graphics.

7. Reboot

8. Assign the game executable to run using the K80:

   1. Right-click on your desktop and go to the display settings.

   2. Scroll down and click on "Graphics Settings".

   3. Find the .exe file of the game you want to run using the K80.

   4. Click on the game in the list and select "Options" and choose the "High performance" NVIDIA Tesla K80 GPU.

 

Overclocking (edit)

My K80 came with a GPU clock limit of 562 MHz and a memory clock limit of 2505 MHz. I found the GPU to remain stable (without any modifications to voltage) at a boost clock of 849.5 MHz and a memory clock of 3505 MHz.

These settings can be changed either using MSI Afterburner or by flashing the VBIOS using these instructions: https://linustechtips.com/topic/1058561-simple-tutorial-ish-for-kepler-and-probably-maxwell-ii-bios-tweaker/

Conclusions

Overall, I think this could be a pretty good option for anyone who hasn't been able to get their hands on a new GPU. These cards aren't that useful to crypto miners, so they've generally been available. Also, many data centers are getting rid of these cards in favor of newer options, increasing their availability.

 

Personally, I only switched one of the two GPUs in my K80 to WDDM mode, because I primarily use this card for scientific computing. Similarly, I only overclocked the WDDM GPU (by flashing the VBIOS; MSI Afterburner will overclock both GPUs). Essentially, I now have one 12GB K40 for scientific computing and a second 12 GB K40 for gaming. For the games that I've tested, it operates at a pretty decent average of 60 FPS on high settings.

 

Sources:

https://forum.level1techs.com/t/gaming-on-my-tesla-more-likely-than-you-think/171185?fbclid=IwAR2QMYGD-LFsX7mMjBEspR48zMUR010cqSEEBe8b49YjHTY7aAIZcFSYfzY

https://www.overclock.net/threads/gaming-on-a-tesla-m40-gtx-titan-x-performance-for-a-third-the-price.1777611/

Super sweet you made this work. I tried and failed earlier this year. I needed the registry hacks I think.  Anyway, I would consider doing it again with a K40 this time since you only get half the k80 anyway, but is it really worth it? What about benchmarks? How does it compare to a gtx 980? 

[ItsDarkMyG]

I have a tesla K40, but im wondering if i can use it through my Quadro P620, any ideas?

 

[InertEvening67]

On 1/8/2022 at 4:02 PM, ItsDarkMyG said:

I have a tesla K40, but im wondering if i can use it through my Quadro P620, any ideas?

 

I think as long as the pc recognizes the p620 as a graphics card you should be fine but i'm just guessing

[InertEvening67]

On 5/21/2021 at 3:35 AM, 2s0ckz said:

 

1. You will need a BIOS with the option to enable "Above 4G decoding" (I’m using an ASUS Prime Z490-A mobo)

i have that "Above 4G decoding" but i don't know if the bandwidth of the pci-e slot is enough for the tesla m40, i read somewhere else that "addressable BAR space needs to be above 32GB" but i have no idea if my consumer-grade system is compatible with that "bar" requirement, could anyone help me?

my pc now: https://pcpartpicker.com/list/NdmnsX

[ItsDarkMyG]

My k40c always has a code 10 error in device manager, not sure what to do.

[SuperGarfield1]

On 8/24/2021 at 1:06 AM, sorenwp said:

How is gaming performance on K80?
- Can you share some benchmark / FPS 

Anyone got any more on this????

 

I'm also wondering the same....

[Konstantin Naumenko]

Hi, ItsDarkMyG!

Do you have windows 10?

Did you turned on 4g decoding in bios settings?

Did you turned off Fast Boot?

[ItsDarkMyG]

On 2/7/2022 at 2:07 PM, Konstantin Naumenko said:

Hi, ItsDarkMyG!

Do you have windows 10?

Did you turned on 4g decoding in bios settings?

Did you turned off Fast Boot?

i am running windows 10, above 4g decoding is enabled, fast boot is enabled but when disabled it makes no difference.

[Ravol]

Hi, I have a problem. When I run an "nvisia-smi" command in cmd, I get the following error: 
Failed to initialize NVML: Not Found

[AcNothing]

On 2/17/2022 at 2:53 AM, Ravol said:

Hi, I have a problem. When I run an "nvisia-smi" command in cmd, I get the following error: 
Failed to initialize NVML: Not Found

 

[Konstantin Naumenko]

On 2/13/2022 at 3:51 PM, ItsDarkMyG said:

i am running windows 10, above 4g decoding is enabled, fast boot is enabled but when disabled it makes no difference.

It's strange. Make sure Fast Boot is disabled in the current enabled power plan. I have 2 Teslas k40m and the above method is actually working every time. Try to reinstall the drivers. It should be named: 463.15-data-center-tesla-desktop-win10-64bit-dch-international.exe or later

[ItsDarkMyG]

On 2/23/2022 at 8:11 PM, Konstantin Naumenko said:

It's strange. Make sure Fast Boot is disabled in the current enabled power plan. I have 2 Teslas k40m and the above method is actually working every time. Try to reinstall the drivers. It should be named: 463.15-data-center-tesla-desktop-win10-64bit-dch-international.exe or later

okay!