You want the moves like Jagger - Deepfake now works with fullbody

[williamcll]

Yandereify was one of the most popular apps back in July that made use of machine learning to deepfake a single photo into a video and it spawned countless memes. Now Students in ShangHai have developed a new program that can take a fullbody picture and impose it onto another.

Quote

We tackle human image synthesis, including human motion imitation, appearance transfer, and novel view synthesis, within a unified framework. It means that the model, once being trained, can be used to handle all these tasks. The existing task-specific methods mainly use 2D keypoints (pose) to estimate the human body structure. However, they only express the position information with no abilities to characterize the personalized shape of the person and model the limb rotations. In this paper, we propose to use a 3D body mesh recovery module to disentangle the pose and shape.

It can not only model the joint location and rotation but also characterize the personalized body shape. To preserve the source information, such as texture, style, color, and face identity, we propose an Attentional Liquid Warping GAN with Attentional Liquid Warping Block (AttLWB) that propagates the source information in both image and feature spaces to the synthesized reference. Specifically, the source features are extracted by a denoising convolutional auto-encoder for characterizing the source identity well. Furthermore, our proposed method can support a more flexible warping from multiple sources. To further improve the generalization ability of the unseen source images, a one/few-shot adversarial learning is applied. In detail, it firstly trains a model in an extensive training set. Then, it finetunes the model by one/few-shot unseen image(s) in a self-supervised way to generate high-resolution (512 × 512 and 1024 × 1024) results.

Source: https://github.com/iPERDance/iPERCore

https://arxiv.org/pdf/2011.09055.pdf

Thoughts: True, the animation is still kind of awkward in some situations, but it's a huge step from just deepfaking a face, I can see some industrial use case for this such as emulating people.

[Moonzy]

trump doing the chika dance is one of the most cursed things i've seen

 

delet

[sub68]

basically mmd on crack.

seriously this doesn't surprise me.

people have been doing this though tracking points by hand or making models of the thing they need to do.

1 minute ago, Moonzy said:

trump doing the chika dance is one of the most cursed things i've seen

try me

 

[wall03]

4 minutes ago, Moonzy said:

trump doing the chika dance is one of the most cursed things i've seen

 

delet

where hand

[SpaceGhostC2C]

I mean... "deepfake"... maybe, one day... Right now it looks like a Trump model for Totally Accurate Battle Simulator... all fake and no deep  

We've had motion capture for decades, so the contribution is... generating a terrible texture from one photo?

 

19 minutes ago, williamcll said:

it's a huge step from just deepfaking a face

I don't know, the face-only ones do look better than a meme hand-drawn in paint...

[TetraSky]

Nice. With this, death is no longer an issue for actors, they will be forever.

[minibois]

 

while on one hand this is awesome, it's also super scary what can be done with it. glad it's mostly used for memes now!

[Kisai]

2 hours ago, SpaceGhostC2C said:

I mean... "deepfake"... maybe, one day... Right now it looks like a Trump model for Totally Accurate Battle Simulator... all fake and no deep  

We've had motion capture for decades, so the contribution is... generating a terrible texture from one photo?

That's still at a level that would work for a Mortal Kombat clone, though to be honest, this probably won't fool anyone. The point of "deep fake" is to make something convincing, and this is not. Not yet anyway.

 

 

2 hours ago, SpaceGhostC2C said:

I don't know, the face-only ones do look better than a meme hand-drawn in paint...

Just to put the cards on the table, this tech does have some very obvious implications. It might be easier to "rough out" an animation sequence with it, rather than have someone storyboard just the key frames. Like existing 2D cartoons often have to 'trace over' 3D models to begin with, so if you can omit that stage and go directly from reference to trace, you save half the time.

 

[CarlBar]

3 hours ago, Moonzy said:

trump doing the chika dance is one of the most cursed things i've seen

 

delet

 

I've never even heard of the chika dance before and i still 100% agree. Obviously capable dancer on the mo cap side of things though. or is being an insane dancer just a normal thing in japan, (i assume it's japan anyway).

[JZStudios]

3 hours ago, Moonzy said:

trump doing the chika dance is one of the most cursed things i've seen

 

delet

You ever seen his actual dance? He got moves.

 

[StDragon]

3 hours ago, Kisai said:

That's still at a level that would work for a Mortal Kombat clone, though to be honest, this probably won't fool anyone. The point of "deep fake" is to make something convincing, and this is not. Not yet anyway.

 

 

Just to put the cards on the table, this tech does have some very obvious implications. It might be easier to "rough out" an animation sequence with it, rather than have someone storyboard just the key frames. Like existing 2D cartoons often have to 'trace over' 3D models to begin with, so if you can omit that stage and go directly from reference to trace, you save half the time.

 

These are crappy deep fakes. Look around YT and you can find some really good ones.

 

The right tools in the hands of professionals could really disrupt the narrative. Within the next few years or so, it could put into question as to if video and audio can be trusted or admissible in court as evidence.

[ARikozuM]

And now every politician and person of influence will have a "get out of jail free" card... This century is proving to be the death of mankind. 

[spartaman64]

12 hours ago, Moonzy said:

trump doing the chika dance is one of the most cursed things i've seen

 

delet