I’ve seen numerous guides online discussing face and facial landmark detection. Sometimes these guides also overlay an image attached to one or multiple of these landmarks. None however, promise real-time mapping of 3D models to 3D face meshes. Yet Snapchat manages to do this with nearly zero latency on mobile devices. What are some tricks to achieve similar performance?

I just thought of this as a complex, multifaceted problem. I would love and try to recreate Snapchat lenses in an open source project.

Anyone with experience with similar problems who can elaborate on some of the key components?