Nvidia is always better for audio - AI noise cancellation program now available.

[williamcll]

Nvidia recently uploaded a Beta release for a GPU-driven voice cancellation program for RTX series cards, download it here: https://developer.nvidia.com/rtx/broadcast_engine/secure/NVIDIA_RTX_Voice.exe

Quote

Nvidia's GeForce RTX GPUs have been out for quite some time now, and though more and more games have begun to support the real-time ray tracing tech they enable, their value proposition is still a bit shaky. Nvidia wants to sweeten the pot a bit now, though, with its new "RTX Voice" plugin.

This plugin is available for download now, and it aims to remove "distracting background noise" from your recording device. Now that everybody who can is working from home, RTX Voice could be a lifesaver -- if you're hammering away at a work-related document while taking part in a conference call, the tech could help you suppress that sound and avoid distracting others. Of course, the average work-from-home employee probably doesn't have an RTX GPU, which is a strict requirement here. RTX Voice relies on the AI capabilities built into the cards to function, meaning gamers are much more likely to benefit from it. Even so, Nvidia is currently supporting a nice mix of apps with RTX Voice, including gaming-centric software like Discord and Twitch Studio and productivity-focused apps like Zoom and Slack. The full supported app list is as follows:

• OBS Studio
• XSplit Broadcaster
• XSplit Gamecaster
• Twitch Studio
• Discord
• Google Chrome
• Skype
• WebEx
• Zoom
• Slack

Setting up and using RTX Voice is relatively simple, according to Nvidia. Just download the app here, install it on a Windows 10 system with Nvidia drivers 410.18 or newer, and boot the app up. You'll see a screen that prompts you to select whatever input and output devices you prefer to use, and then you'll need to check the box labeled "Remove background noise from my microphone."

After that, open up a supported app and navigate to its audio settings interface. There, you must change your input and output devices to "NVIDIA RTX Voice." If you still need help setting RTX Voice up for a specific app, Nvidia has detailed instructions for each available right here.

Source: https://www.techspot.com/news/84881-nvidia-rtx-voice-app-uses-ai-improve-noise.html

https://www.nvidia.com/en-us/geforce/guides/nvidia-rtx-voice-setup-guide/#Testing-RTX-voice

Thoughts: Useful, considering Discord's own voice cancellation software mutes my voice entirely, though the filesize do look pretty massive for a single purpose software.

[WereCatf]

Eposvox sure seemed mightily impressed by this thing. I wish I could use it as well, but alas, I am not willing to sell a kidney to buy an RTX-card.

[RejZoR]

I don't care about noise cancellation, but give me path traced 3D audio in games and then I'm gonna be sold...

[LAwLz]

30 minutes ago, huilun02 said:

Whats the perf impact when using the same card for gaming?

My guess is that this is running on the tensor cores so unless you use other AI features running it should have close to 0 performance impact. 

But that's a big assumption from me. But in general, audio software uses very little performance so it shouldn't have a major impact even if it runs on the CPU for example. 

[Arika]

I dont care about this....UNLESS it can remove the loud ass clicking of the MX Blue keyboards that everyone i play with has to smash their fingers on 100% of the time we play.

in that case; how to i install this on other people's computers?

[tim0901]

3 hours ago, Arika S said:

I dont care about this....UNLESS it can remove the loud ass clicking of the MX Blue keyboards that everyone i play with has to smash their fingers on 100% of the time we play.

in that case; how to i install this on other people's computers?

It can be enabled on both incoming and outgoing audio, so yes it can mute other people's keyboards!

[AkatsukiKun]

13 hours ago, tim0901 said:

It can be enabled on both incoming and outgoing audio, so yes it can mute other people's keyboards!

Cherry Blues we out here.

Anyways how well does it do for fan noise, my room does get quite hot and running and I use a table fan relatively close to me to get some air around.

[AlTech]

@williamcll I personally dislike the title. Nvidia isn't better for audio. They're just doing different things than AMD.

 

AMD has TrueAudio Next which uses an asic block on some GPUs and Stream Processors on newer GPUs to allow devs to do stuff with computational audio, spatial audio, etc.

 

AMD also has a bunch of other stuff with GPUOpen.

[williamcll]

1 hour ago, AluminiumTech said:

@williamcll I personally dislike the title. Nvidia isn't better for audio. They're just doing different things than AMD.

 

AMD has TrueAudio Next which uses an asic block on some GPUs and Stream Processors on newer GPUs to allow devs to do stuff with computational audio, spatial audio, etc.

 

AMD also has a bunch of other stuff with GPUOpen.

See, I cut the joke into two

The other part was "AMD cards are better at networking"

[WereCatf]

On 4/20/2020 at 6:17 AM, AkatsukiKun said:

Anyways how well does it do for fan noise, my room does get quite hot and running and I use a table fan relatively close to me to get some air around.

Eposvox demonstrated how well it removed all keyboard-noise, fan-noise and even a running vacuum next to the microphone. Discord released a kinda-sorta same thing, but when Eposvox demonstrates Discord's thingy, there's a noticeable change in his voice for the worse. At least I wasn't nearly as impressed with Discord's system.

 

 

[GreyestGoat]

Intel is always better at winter nights.

[Dellenn]

That RTX demonstration was really impressive. 

 

When he's talking about performance impacts... I would assume that's only if you are using DLSS in a game as that function also uses the Tensor cores. I can't imagine that CUDA performance would be impacted by this.

[Letgomyleghoe]

On 4/21/2020 at 4:16 AM, GreyestGoat said:

Intel is always better at winter nights.

wUt

[orbitalbuzzsaw]

Gaming Noise Cancelling^TM

[FezBoy]

no linux support -_-

 

 

Edited April 24, 2020 by FezBoy
Had to use a zerowidth space for it not to change the -_- into an emoji

[SpaceGhostC2C]

On 4/19/2020 at 2:42 AM, WereCatf said:

I wish I could use it as well, but alas, I am not willing to sell a kidney to buy an RTX-card.

On 4/21/2020 at 2:46 PM, Dellenn said:

When he's talking about performance impacts... I would assume that's only if you are using DLSS in a game as that function also uses the Tensor cores. I can't imagine that CUDA performance would be impacted by this.

 

... and... it's gone.

 

 

On 4/19/2020 at 11:17 PM, AkatsukiKun said:

Anyways how well does it do for fan noise, my room does get quite hot and running and I use a table fan relatively close to me to get some air around.

It should do well, as relatively constant, repetitive noises are the easiest to filter.

[StDragon]

On 4/19/2020 at 1:44 AM, RejZoR said:

I don't care about noise cancellation, but give me path traced 3D audio in games and then I'm gonna be sold...

Ray traced audio. Would make sense to also perform these calculations along with visual ray traced within the same GPU. That said, Microsoft will have dedicated audio hardware acceleration for the next XBox. That revelation is really off putting given how DirectX stopped supporting audio acceleration in Windows Vista/7. Meaning it was only available in Windows XP, and that SoundBlaster EAX was used as well for offloading the main CPU for effect. Granted, it wasn't true path 3D. It was more of an equivalent of "rasterized" audio effects handled by the DSP based on instructions fed to it by the CPU; at least that's how I understand it.

 

Apparently Microsoft's decision to remove Direct Audio acceleration was due to (their claim) instabilities that led to a lot of kernel panics (BSOD). So the next version of Windows thereafter had the audio sub-system redone which would break this function.

 

So, unless I'm misinformed, is audio hardware acceleration now a thing again with the latest DirectX API and by extension Windows 10? Or is this all moot anyways because it's best to offload the calculations through the GPU first which apparently is counter to the dedicated chip solution that XBox is slated to have?

[Tech Enthusiast]

On 4/19/2020 at 12:04 PM, Arika S said:

I dont care about this....UNLESS it can remove the loud ass clicking of the MX Blue keyboards that everyone i play with has to smash their fingers on 100% of the time we play.

in that case; how to i install this on other people's computers?

Well.

It does.

 

Simple as that. I could not believe it, but when I recorded a Test Video Call from myself.... there was ZERO key presses in the playback. Not "quite silent", but seriously: They where gone. It was my voice only and nothing else.

 

It kind of feelt like a little bit of magic, so I am gonna keep using it for sure.

[RejZoR]

1 hour ago, StDragon said:

Ray traced audio. Would make sense to also perform these calculations along with visual ray traced within the same GPU. That said, Microsoft will have dedicated audio hardware acceleration for the next XBox. That revelation is really off putting given how DirectX stopped supporting audio acceleration in Windows Vista/7. Meaning it was only available in Windows XP, and that SoundBlaster EAX was used as well for offloading the main CPU for effect. Granted, it wasn't true path 3D. It was more of an equivalent of "rasterized" audio effects handled by the DSP based on instructions fed to it by the CPU; at least that's how I understand it.

 

Apparently Microsoft's decision to remove Direct Audio acceleration was due to (their claim) instabilities that led to a lot of kernel panics (BSOD). So the next version of Windows thereafter had the audio sub-system redone which would break this function.

 

So, unless I'm misinformed, is audio hardware acceleration now a thing again with the latest DirectX API and by extension Windows 10? Or is this all moot anyways because it's best to offload the calculations through the GPU first which apparently is counter to the dedicated chip solution that XBox is slated to have?

Some of audio stuff has access to audio processor directly. But nowhere near the extent we used to have under DirectSound3D as far as I can tell. But if graphics can have all the access and we don't see 300 million BSOD's every minute from it, so should audio. We're not only stagnating with audio, we went basically 2 decades back in time with it.

 

As for this feature, I wonder how my Sound Blaster AE-9 Noise Cancelation feature fares against this thing. LTT guys tested this RTX thing, but Creative has been around for a very long time sporting these features and no one really tried them or compared them. And I don't use mics in games enough to remember how it works. I do know it has filtering and it worked when I was recording something. Just not to same extent...

[Tech Enthusiast]

On 4/19/2020 at 3:23 PM, tim0901 said:

It can be enabled on both incoming and outgoing audio, so yes it can mute other people's keyboards!

Short, but important info.

You can indeed block OTHER peoples keyboards as well with this little magic trick.

 

I rarely get excited about software nowadays, but this little gem is just amazing and works flawlessly (in my limited testing obviously).

It simply does what it claims and it does it easily and perfectly.

 

If you have an RTX GPU (or can be bothered to comment out the 3 lines needed to make it not RTX only), get it. It is worth your time, if you are on voice calls a lot.

[StDragon]

1 hour ago, RejZoR said:

As for this feature, I wonder how my Sound Blaster AE-9 Noise Cancelation feature fares against this thing. LTT guys tested this RTX thing, but Creative has been around for a very long time sporting these features and no one really tried them or compared them. And I don't use mics in games enough to remember how it works. I do know it has filtering and it worked when I was recording something. Just not to same extent...

I suspect there really is AI being used here and it's not just marketing. I've demoed it myself, pretty amazing, though on my 2060 Super it only shows to be using 1% GPU per Windows Task Manager.

 

It doesn't seem to be a simple matter of frequency filtering that you'd normally expect from a DSP. This seems to leverage actual intelligence in heuristics based on real-time statistics. For example, it biases audio to keep based on the average frequency of human speech and the higher amplitude one would expect being close to the mic. The rest is AI filtering out everything else. Purely a guess.

 

It's not perfect, but for beta it's damn effective!

[Tech Enthusiast]

Just got off another call.

After I showed how it worked, half a dozen people instantly installed this software.

I had a text file open and mashed my keyboard, without RTX Voice. Then showed them the same thing with slider turned right. 

 

"It's not perfect" may be correct, but if I have ever seen noise cancellation come close to perfection, this is it.

[RejZoR]

I frankly don't believe there is any "Ai" involved. Then again they throw Ai at anything these days. Ai banana, Ai Yogurt, Ai hamsters, Ai toilet paper...

 

It's literally just filter that checks the volume levels and filters stuff. Coz you speaking into mic will always be above everything, even if that everything is super loud. And then it's just up to the filter to sort it out.

 

I'd be really interested to see this RTX Audio thing compared to something like Sound Blaster AE-5/AE-7/AE-9 soundcard and it's Noise Cancelation filter and Denoiser.

[LAwLz]

1 minute ago, RejZoR said:

I frankly don't believe there is any "Ai" involved. Then again they throw Ai at anything these days. Ai banana, Ai Yogurt, Ai hamsters, Ai toilet paper...

 

It's literally just filter that checks the volume levels and filters stuff. Coz you speaking into mic will always be above everything, even if that everything is super loud. And then it's just up to the filter to sort it out.

 

I'd be really interested to see this RTX Audio thing compared to something like Sound Blaster AE-5/AE-7/AE-9 soundcard and it's Noise Cancelation filter and Denoiser.

There is definitely AI involved here.

This is not just a simple filter which removes X or Y. This can filter out sounds that are louder than your voice. It also filters out noise even if you are not speaking, which would not be possible with a "remove everything quieter than my voice" filter.

 

The AI portion is that they trained an AI to detect the frequencies and patterns of human speech. The training stage most definently used machine learning. Otherwise it would not the way it does.

The actual execution happening on your PC does not seem to involve AI though. Once Nvidia had the pattern classifications it seems like they stuck it inside an application that can run it on CUDA cores and then shipped that.

[cj09beira]

8 hours ago, SpaceGhostC2C said:

 

... and... it's gone.

 

 

It should do well, as relatively constant, repetitive noises are the easiest to filter.

you dont need an rtx card, it just uses the cuda cores, at most they used tensor cores to train the algo