AI tool for generating deep fake voice acting in games

[Braveheart101]

 

Summary

 There's an AI tool available which makes it possible for people to generate voice lines using voices already existing in games. It's been released for a number of popular games, like Skyrim, Fallout 4, Oblivion, and more.

 

https://www.youtube.com/watch?v=xqJ-ujWvu7s

 

Quotes

Quote

AI tool for high quality voice acting synthesis using in-game voices from Bethesda games.. 

Source: https://www.nexusmods.com/skyrimspecialedition/mods/44184

 

 

My thoughts

Audio deepfakes is not a new concept, there've been plenty of examples of the tech used before, but this is the first time that I'm aware of where it's been used directly for content creation in gaming. There's questions around who owns the voices that are being used, or if that is even an issue at all, seeing as audio splicing tools already exist and have been used plenty, in the past. It'll be interesting to see how it evolves. It seems several big modders and content creators have picked up on it.    

 

I'm personally excited, as it should make modding a whole lot better. Content creators will be able to do much more, with this.

 

Sources

https://www.nexusmods.com/skyrimspecialedition/mods/44184

https://www.youtube.com/watch?v=xqJ-ujWvu7s

[StDragon]

Voice avatars! Being able to pick your favorite celebrity and have your voice in-game would be hilarious (and spooky). Not sure if the tensor cores could do it in real-time however.

[Franck]

16 minutes ago, StDragon said:

Voice avatars! Being able to pick your favorite celebrity and have your voice in-game would be hilarious (and spooky). Not sure if the tensor cores could do it in real-time however.

For sure it can. The way AI works is the training time that is long to do. Once you have trained the model enough you have a kind of formula which is usually very very fast.

Don't forget that also that most if not all game with voice have subtitles which is a text file somewhere that could also be preloaded and you generace the voice line in an audio clip before the game need it.

 

I used to work for a company as a dev and we were recording thousands of hour of voice from a specific voice actor so we could create and emulated voice for that language and we did so for over 80 languages. So we could feed text in the system and the voice audio was generated live. This was used for automatic phone systems and most if not all are not prerecorded, they use VXML and voice pattern recognition which have a very similar result but use entirely different method than AI. AI would be much faster than the method we used which needed a certain amount of sounds to be repeated by the actor. Pretty sure you need more than the 1 sentence you see in spy movies where they copy the voice of someone but the principle is as simple as that.

[Braveheart101]

Never thought I'd hear the Whiterun guard tell me to visit the LTT store:

https://vocaroo.com/12yMXec4HS5f

There's a big thread over it on reddit: 

 

[suicidalfranco]

with projects like this one:

https://github.com/sebastianstarke/AI4Animation

Combined with this AI, could this spell the end of video game acting

Though i would really just love to see the end of awful collision detection, specially with stair steps

[Kisai]

This is nothing new. This is in fact, a suggestion that shows up in voice training, to use fully-voiced games for training voice models. It doesn't happen because copyright (despite a lot of voice training using youtube videos.)

 

That said, it would be considered copyright infringement if you use this tool to produce anything that doesn't go back into the game. Eg don't produce a line and make a machinma with another game. I doubt Bethesda will go after anyone who does this, but the professional Voice actors certainly will, as their didn't agree to this. So don't use it to make your Fallout Visual Novel.