GPU Pass through to VM in FreeNAS

[Turretgaming]

So I just got an Ubuntu VM running in FreeNAS and I know you can pass a NIC or drives to the VM. Now lets say I want to have a gpu on the VM for some better number crunching power. If i just slapped the gpu in the system then edit the vm and add the gpu as a device would it work or would freenas not even see it as a device to pass to the vm? Any and all help is much appreciated 

[Radium_Angel]

2 minutes ago, Turretgaming said:

So I just got an Ubuntu VM running in FreeNAS and I know you can pass a NIC or drives to the VM. Now lets say I want to have a gpu on the VM for some better number crunching power. If i just slapped the gpu in the system then edit the vm and add the gpu as a device would it work or would freenas not even see it as a device to pass to the vm? Any and all help is much appreciated 

PCI passthrough, as it's called, is handled by the VM itself, assuming you have the new enough hardware to do this.

 

[Turretgaming]

Just now, Radium_Angel said:

PCI passthrough, as it's called, is handled by the VM itself, assuming you have the new enough hardware to do this.

 

the cpu is haswell (4130 i3 but looking to make it a Xeon E3-1231 V3 soon) but the gpu will be a 1060 6gb and if i added it the system and booted up in freenas and started the Ubuntu vm it will just work or do i have to configure anything?

[samcool55]

Noope, won't work.

You need IOMMU to be able to assign gpu's to VM's like you want to do.

Thing is, it's only available on i5's and i7's.

 

I have a 4170 myself and tried gpu passthrough with unraid and it didn't work because the stupid cpu doesn't support iommu...

 

Don't know about the xeon tho.

[Turretgaming]

1 minute ago, samcool55 said:

Noope, won't work.

You need IOMMU to be able to assign gpu's to VM's like you want to do.

Thing is, it's only available on i5's and i7's.

 

I have a 4170 myself and tried gpu passthrough with unraid and it didn't work because the stupid cpu doesn't support iommu...

 

Don't know about the xeon tho.

so an i7 4770k might work as its an i7 on the same platform 

[Electronics Wizardy]

the xeon has support for vt-d

 

Can you run proxmox or anouther linux based hypervisor, it will work much better here, bhvye supports it, but linux kvm supports it better and will work better here.

 

11 minutes ago, Radium_Angel said:

PCI passthrough, as it's called, is handled by the VM itself, assuming you have the new enough hardware to do this.

 

Its handleded by the hypervisor, the vm just gets the devices.

[Electronics Wizardy]

Just now, Turretgaming said:

so an i7 4770k might work as its an i7 on the same platform 

the 4470k has vt-d disabled, its enabled on the xeons, 4790k and 4770

[Turretgaming]

6 minutes ago, Electronics Wizardy said:

the 4470k has vt-d disabled, its enabled on the xeons, 4790k and 4770

https://en.wikipedia.org/wiki/List_of_IOMMU-supporting_hardware

 

this says the i3 4130 has iommu but the i7 4770k doesnt and the i7 4770 non k does and no mention of the xeon chips but i cant see why that would be disabled as xeons are used for vm work environments all the time.

 

also wats vt-d? 

and this is the mobo im using

https://www.newegg.com/Product/Product.aspx?Item=N82E16813157414

 

so since im running the linux os in a vm on freebsd (freenas) it work? and i was using unraid it would work as its linux talking to linux?

[dalekphalm]

1 hour ago, Turretgaming said:

So I just got an Ubuntu VM running in FreeNAS and I know you can pass a NIC or drives to the VM. Now lets say I want to have a gpu on the VM for some better number crunching power. If i just slapped the gpu in the system then edit the vm and add the gpu as a device would it work or would freenas not even see it as a device to pass to the vm? Any and all help is much appreciated 

Can FreeNAS even utilize a GPU in any meaningful way besides standard video output?

[Turretgaming]

9 minutes ago, dalekphalm said:

Can FreeNAS even utilize a GPU in any meaningful way besides standard video output?

I want to pass the gpu to Ubuntu. I went over it in my main post. the system is running freenas and the vm is Ubuntu and i want to pass a gpu to the Ubuntu vm

[dalekphalm]

21 minutes ago, Turretgaming said:

I want to pass the gpu to Ubuntu. I went over it in my main post. the system is running freenas and the vm is Ubuntu and i want to pass a gpu to the Ubuntu vm

Oh I see - FreeNAS is your bare metal OS, and Ubuntu is the VM. I misinterpreted what you said as the reverse of that.

[Turretgaming]

22 minutes ago, dalekphalm said:

Oh I see - FreeNAS is your bare metal OS, and Ubuntu is the VM. I misinterpreted what you said as the reverse of that.

whoops sorry about that. But do you have any idea if it will work or if its even possible?

[dalekphalm]

22 minutes ago, Turretgaming said:

whoops sorry about that. But do you have any idea if it will work or if its even possible?

No idea - FreeNAS VM Support is kinda crap. The interface sucks and it's more difficult to do than in other systems.

 

I actually run my FreeNAS server as a VM ontop of ESXi - which definitely has PCIe passthrough support.

 

From my very limited research, FreeNAS might not even support GPU passthrough:

The VM subsystem that FreeNAS uses - bhyve - does support GPU passthrough, but I don't know if it's implemented on FreeNAS.

[Turretgaming]

1 hour ago, dalekphalm said:

No idea - FreeNAS VM Support is kinda crap. The interface sucks and it's more difficult to do than in other systems.

 

I actually run my FreeNAS server as a VM ontop of ESXi - which definitely has PCIe passthrough support.

 

From my very limited research, FreeNAS might not even support GPU passthrough:

The VM subsystem that FreeNAS uses - bhyve - does support GPU passthrough, but I don't know if it's implemented on FreeNAS.

I'm running 12.1 and I just got home and slapped the 1060 6gb in and a quick look showed that the system booted and the VM started but I can't see an option to pass a GPU to it. I think it's cause the mobo is using the 1060 for FreeNAS so after work I'll try to tell the mobo to use on board GPU in my i3 4130 and it should leave the 1060 free for the VM to snag it.

[dalekphalm]

1 hour ago, Turretgaming said:

I'm running 12.1 and I just got home and slapped the 1060 6gb in and a quick look showed that the system booted and the VM started but I can't see an option to pass a GPU to it. I think it's cause the mobo is using the 1060 for FreeNAS so after work I'll try to tell the mobo to use on board GPU in my i3 4130 and it should leave the 1060 free for the VM to snag it.

Yes you definitely need to run FreeNAS off of the onboard video. Don't plug anything into the 1060 (at least, unless it's another monitor that will only be used by Ubuntu).

[Turretgaming]

20 hours ago, dalekphalm said:

Yes you definitely need to run FreeNAS off of the onboard video. Don't plug anything into the 1060 (at least, unless it's another monitor that will only be used by Ubuntu).

ya tried it that way and still failed. I might backup the zfs file for FreeNAS and load up Ubuntu as the baremetal os and run vmware and pass 2 cores and 10gb ram to freenas and keep the other 2 cores for Ubuntu.

[Electronics Wizardy]

1 hour ago, Turretgaming said:

ya tried it that way and still failed. I might backup the zfs file for FreeNAS and load up Ubuntu as the baremetal os and run vmware and pass 2 cores and 10gb ram to freenas and keep the other 2 cores for Ubuntu.

why not run proxmox? You get the same zfs filesystem, and a good hypervisor.

 

Don't run vmware on ububntu, just run exsi if you want that.

 

If you want you can probably get it to work in freenas if you want to mess with bhyve, but really its easier to run proxmox here.

[Turretgaming]

22 minutes ago, Electronics Wizardy said:

why not run proxmox? You get the same zfs filesystem, and a good hypervisor.

 

Don't run vmware on ububntu, just run exsi if you want that.

 

If you want you can probably get it to work in freenas if you want to mess with bhyve, but really its easier to run proxmox here.

i might try that but i need a new cpu but the e3 1240 v3 and e3 1231 v3 or e3 1286 v3 or e3 1245 v3 that i want are all over $300-$500 new or i can find them on ebay for $150-$500 but idk if i can trust ebay. does anyone know where i can get them at a fair price and not have to worry about the quality. My 4130 is currently using all core multiplier of 3.4ghz to pin the cpu at 3.4ghz so i get max performance at all times cause i max out the cpu doing this stuff.

[Electronics Wizardy]

1 minute ago, Turretgaming said:

i might try that but i need a new cpu but the e3 1240 v3 and e3 1231 v3 or e3 1286 v3 or e3 1245 v3 that i want are all over $300-$500 new or i can find them on ebay for $150-$500 but idk if i can trust ebay. does anyone know where i can get them at a fair price and not have to worry about the quality. My 4130 is currently using all core multiplier of 3.4ghz to pin the cpu at 3.4ghz so i get max performance at all times cause i max out the cpu doing this stuff.

How about just running windows on the host or whatever os you want to run gpu programs with?

[Turretgaming]

Just now, Electronics Wizardy said:

How about just running windows on the host or whatever os you want to run gpu programs with?

that could work but again im limited by the cpu sadly 

[Electronics Wizardy]

Just now, Turretgaming said:

that could work but again im limited by the cpu sadly 

What are you running on there?
 

Id just get something like this if you want a faster cheap cpu .https://www.ebay.com/itm/Intel-Xeon-E3-1220v3-Quad-Core-3-10GHz-SR154-8MB-LGA-1150-CPU-Processor/113611507787?epid=1223020354&hash=item1a73c6084b:g:bMUAAOSwXINcVVBS:rk:2:pf:0

[Turretgaming]

is it trustworthy though? i havent used ebay before and im worried about getting ripped off

[Turretgaming]

10 minutes ago, Electronics Wizardy said:

What are you running on there?
 

Id just get something like this if you want a faster cheap cpu .https://www.ebay.com/itm/Intel-Xeon-E3-1220v3-Quad-Core-3-10GHz-SR154-8MB-LGA-1150-CPU-Processor/113611507787?epid=1223020354&hash=item1a73c6084b:g:bMUAAOSwXINcVVBS:rk:2:pf:0

is it trustworthy though? i havent used ebay before and im worried about getting ripped off

and are any of these better price to performance ive been looking on intel ark for a good cpu and these look good but idk if the sellers are good. id love to get one of them but idk if i can trust the seller. if you think any if these are better please let me know.

 

https://www.ebay.com/itm/Intel-Xeon-E3-1240-v3-SR152-3-4GHz-Quad-Core-LGA-1150-CPU-Processor-km/401682754559?epid=1328652074&hash=item5d86286bff:g:maAAAOSwDJZb5FkU:rk:1:pf:0

 

https://www.ebay.com/itm/Intel-Xeon-E3-1240L-V3-2-0-GHz-LGA-1150-4C-8T-25W-8MB-Cache-Processor/283269447879?hash=item41f42cc0c7:g:4xoAAOSwxxVb9MZI:rk:3:pf:0

 

https://www.ebay.com/itm/Intel-Xeon-E3-1245V3-Quad-Core-3-40GHz-CPU-Processor-SR14T/143108879975?epid=1279155286&hash=item2151f43667:g:2gEAAOSwNe9cULws:rk:2:pf:0

 

https://www.ebay.com/itm/Intel-Xeon-E3-1245-V3-Haswell-3-4-GHz-8MB-L3-Cache-LGA-1150-84W-CPU/254038671107?hash=item3b25e27f03:g:0HkAAOSwYd5cIR9V:rk:4:pf:0

[Electronics Wizardy]

9 minutes ago, Turretgaming said:

is it trustworthy though? i havent used ebay before and im worried about getting ripped off

For used stuff its your best source, and you can get a refund if it doesn't come to you. Its pretty good.

 

Id get the e3 1240 v3 there if you want to spend a bit more.

[Turretgaming]

Just now, Electronics Wizardy said:

For used stuff its your best source, and you can get a refund if it doesn't come to you. Its pretty good.

 

Id get the e3 1240 v3 there if you want to spend a bit more.

ya it has hyper-threading which can be useful