[R][P] Talking Head Anime from a Single Image
I trained a network to animate faces of anime characters. The input is an image of the character looking straight at the viewer and a pose, specified by 6 numbers. The output is another image of the character with the face posed accordingly.
I created two tools with this network.
Using a face tracker, I could transfer human face movements from existing videos to anime characters. Here are some characters impersonating President Obama:
The approach I took is to combine two previous works. The first is the Pumarola et al.’s 2018 GANimation paper, which I use to change the facial features (closing eyes and mouth, in particular). The second is Zhou et al.’s 2016 object rotation by appearance flow paper, which I use to rotate the face. I generated a new dataset by rendering 8,000 downloadable 3D models of anime characters.
You can find out more about the project at https://pkhungurn.github.io/talking-head-anime/.