Last month FBI warned about scammers impersonating someone else by using deepfakes in interviews to get remote IT jobs. While the purpose of these actions has nothing to do with getting the job, it does present a new security issue.
Now researchers have discovered a surprisingly simple way to detect deepfakes in video calls. The hack was shared by Metaphysic.ai, a London-based startup behind viral Tom Cruise deepfakes.
The company used DeepFaceLive, a popular fake video app, to transform a volunteer into various celebrities. Most of the impersonations were impressive when viewed from the front. But once the faces were turned a full 90 degrees, the images distorted, and the spell was broken.
The team believes the flaws arise because the software uses fewer reference points to estimate side views of faces. This forces the algorithm unable to guess what it would look like. So the simple trick to detect deepfakes in real-time is to ask the suspect to turn on their side.
As the team explains, most deepfake systems don’t work well in acute-angle scenarios. This is because almost all the reference points used by the facial recognition algorithm are assigned to the front of the face. Thus, when someone shows their side of the face in the profile of the deepfake, they start to get lost.
“Typical 2D alignment packages consider a profile view to be 50% hidden, which hinders recognition, as well as accurate training and subsequent face synthesis. Frequently the generated profile landmarks will ‘leap out’ to any possible group of pixels that may represent a ‘missing eye’ or other facial detail that’s obscured in a profile view:”
Another technique that Metaphysic proposes to identify deepfakes is to ask the person with whom we are making a video call to move their hand in front of their face. This “creates a critical situation for the model, which is likely to demonstrate poor latency and quality of superimposition over the deepfaked face:“
Although these weak spots can be strengthened, but it takes a lot of work. For now, though, you can ask a person to turn their head 90 degrees or wave their hand at the camera to find scammers.
