GPU passthrough is the thing, but it's a pain to setup (especially if you want to pass your GPU to the VM and back without rebooting / logging out and back in)
Edit: skipped through part of your comment, sorry. GPU passthrough works, but not without more effort than just dualbootinh