Does deepfake have any application for animation?

Answer 1

Yes, there are several ways that the deep learning technology used in deep fakes can be applied to animation.

It depends on the model (learning/AI) type however. A 3D morph-able model position and place technique can be utilized to produce object sequences from a piece of video. Any of these objects in a sequence can be converted into a single object that retains that frame position of the points and can be driven later with "endomorphs" (LightWave3D) or "blendshapes" (maya).

I actually had a tool written last summer for a movie that does exactly this. It looks at a mp4 video and then produces a set of 3D models (obj sequence) from that video where each frame the computer does its best to match the position and placement of the "face" that is a generic face model and then distorts it appropriately to match. This is the "morph-able model" part of the position and place technique. Once complete, I can take that object sequence and turn all of them into a series of endomorphs in LightWave3D Layout or select any of them from the video frame numbering and put those into the LightWave3D object file I want to save out. From there I can use the morph mixer tool in Layout to animate a face, without actually having to model the morphs themselves, using this technique.

What's even better is that one I have this, I can throw it into a tool like TAFA (http://macreitercreations.com/ << ignore the bad website, its hands down the most powerful/fastest facial animation tool on the planet - trust me, its TOO easy. The developer ran into some troubles a number of years ago and hasn't had the time to update it, but it still works wonderfully) and use those morphs to animate a face easily.

A great technique is to record the basic facial expressions (phenoms) of yourself with your head straight on, convert to a model with the morphs or blendshapes in it through this tool I have or a similar tool, then use the results to drive facial animation using morphs rather than bones, and its its a heck of a lot faster using TAFA than any other way I've tried or seen it done.

If you need more information, please contact me.