Gradient Dude

This media is not supported in your browser

VIEW IN TELEGRAM

Snap has released a new model for animating the entire human body (not just the face). Looks pretty good.

The principle is similar to their previous method - First order motion model for animation of heads. The difference is that (a) the background motion is explicitly modeled here; and (b) instead of regressing local affine transformations for a set of keypoints, this method learns to find heatmaps of different body parts in unsupervised way and
the transformation matrix of each body part is computed by applying principal component analysis (PCA) to the predicted heatmaps.

More details on the project website. Most importantly, there is code and pretrained weights. So go ahead and animate!

P.S. 2 years ago another method for animating the whole body "Everybody Dance Now" was released, but there you had to retrain the network for each new person.

www.group-telegram.com/ye/gradientdude.com/290

2.5K viewsMay 4, 2021 at 13:14

Snap has released a new model for animating the entire human body (not just the face). Looks pretty good.

The principle is similar to their previous method - First order motion model for animation of heads. The difference is that (a) the background motion is explicitly modeled here; and (b) instead of regressing local affine transformations for a set of keypoints, this method learns to find heatmaps of different body parts in unsupervised way and
the transformation matrix of each body part is computed by applying principal component analysis (PCA) to the predicted heatmaps.

More details on the project website. Most importantly, there is code and pretrained weights. So go ahead and animate!

P.S. 2 years ago another method for animating the whole body "Everybody Dance Now" was released, but there you had to retrain the network for each new person.

BY Gradient Dude