Deepfakes – AI-generated synthetic media that imitate real people – have rapidly risen from internet curiosities to serious security threats, resulting in more cases of AI-fueled identity fraud. Fraudsters are increasingly using deepfakes in security-sensitive processes like remote identity verification and digital onboarding for banks and services, making solutions that offer AI deepfake detection an obvious solution.
A recent industry report found that deepfake-based identity attacks are occurring at an alarming rate — once every five minutes in 2024.
In this recent paper about AI deepfake detection approaches and types of detectors, the researchers share insights from a real-world experiment evaluating state-of-the-art deepfake detection models on real production data. The findings reveal how deepfake attacks are becoming more sophisticated and underscore the need for a multi-layered defense that goes beyond deepfake detection alone.
Understanding deepfakes
Deepfakes refer to digital content (usually images, voice clones, or videos) manipulated by AI to realistically impersonate someone or create a synthetic person. Thanks to advances in generative models, the most convincing deepfakes are nearly indistinguishable from authentic images.
They come in various forms, from partially edited faces to fully fictitious personas generated by neural networks. Below are some common types of deepfakes encountered today:
-
Partial face morphing: Blending or altering specific facial regions of one person with another. e.g., an attacker might subtly merge their face with a target’s face, producing a composite image that resembles both. This technique is sometimes used in ID photo fraud, allowing the picture to pass a cursory likeness check.
-
Fully AI-generated faces: Creating an entire face from scratch using AI (e.g., StyleGANs or diffusion models). These synthetic faces do not correspond to any real person and often look completely authentic. A deepfake of this sort can produce a brand-new digital identity. e.g., the images from ThisPersonDoesNotExist showcase fully AI-generated people.
-
Face swapping: Replacing one person’s face with another’s in an image or video. In this case, the deepfake model maps the “donor” face onto the impersonator’s head, making it appear as if the impersonator is actually the donor. This is popular in deepfake videos where an impersonator’s face is overlaid with a celebrity’s face.
-
Face reenactment / Lip syncing: Altering a person’s facial expressions or mouth movements to match someone else’s speech or actions. In these deepfakes, the target’s face remains the same but is animated by another source (e.g., making a still photo speak or altering what a person appears to say in a video). This can produce convincing fake speeches where the person’s lips move in sync with dubbed audio.
Most deepfakes seen in real-world fraud scenarios are generated using readily available tools – often commercial or subscription-based deepfake software or mobile apps.
The barrier to creating believable deepfakes has lowered, with user-friendly services allowing anyone to swap faces or generate avatars with a few clicks. In fact, many of these services now offer customer support to aid clients in their use.
In actual production environments, we have observed both low-effort “deepfakes” and simple Photoshopped images (e.g., a static photo of a person’s face crudely pasted onto another person’s body), as well as highly realistic AI-generated content that leaves no obvious visual traces.
How deepfakes enter a system
When defending against deepfake attacks on identity systems, it’s important to understand how attackers introduce deepfakes into the verification process. Broadly, there are two primary attack vectors:
-
Presented in front of the camera (“Presentation Attacks”): In this method, the deepfake is physically presented to the camera or sensor as if it were a live person. e.g., an impostor might hold up a deepfake video or display a synthetic face on a 4K screen during a selfie identity check. Robust face biometric systems deploy Presentation Attack Detection (PAD) measures to catch these – for instance, checking for signs of a screen replay or 2D image vs. a real 3D face. Without PAD, a deepfake video or even a high-quality photo on a phone screen could be falsely accepted as a live person.
-
Injected into the system during the live capturing process (“Injection Attacks”): In an injection attack, the manipulated image is inserted digitally into the system, rather than being shown to a camera. The attacker bypasses the normal capture process and feeds the deepfake data directly into the software or network.
Pinpointing the exact method of injection into production environments is challenging. However, we have observed that injected content often arrives via virtual camera software, emulators that simulate genuine camera input, and even through tools that manipulate application behavior at runtime. In some cases, deepfake content has been injected using direct hardware screen captures. This wide range of techniques reinforces the need for robust injection detection measures capable of addressing a variety of attack vectors.
In summary, presentation attacks fool the system with a fake image in front of the camera, whereas injection attacks fool the system by sneaking fake data behind the scenes. Both methods are being used by fraudsters to try to spoof online onboarding and biometric login systems.
Key findings from testing 3rd party models
Mitek’s dedicated fraud detection team evaluated several current, state-of-the-art deepfake detection models from across the industry, using a dataset compiled from approved, real-world data across multiple identity verification environments. This dataset encompassed both, genuine users and confirmed deepfake attack attempts that had been identified “in the wild”. The assessment aimed to measure how well existing, 3rd party detectors—predominantly trained on academic datasets—perform when confronted with real-world samples. The results were eye-opening. The models exhibited a Bonafide Presentation Classification Error Rate (BPCER) exceeding 30%, meaning over 30% of legitimate user sessions were mistakenly flagged as deepfakes. Simultaneously, the FAR (false acceptance rate) ranged between 60% and 90%, indicating that the majority of deepfake attacks were not detected and would have been accepted as genuine by these external models. In simpler terms, these detectors either overreacted by misclassifying real users or underreacted by missing many deepfake attacks—a stark contrast to the 90%+ accuracy often reported in controlled lab settings.
The assessment also highlighted several key challenges affecting detector generalization. First, the diverse data distributions across organizations—owing to variations in camera quality, lighting, user behavior, and demographics—mean that a model fine-tuned for one scenario often fails when applied to another. Second, deepfake attack methods are constantly evolving; attackers quickly switch from one generation technique to another, rendering models trained on known artifacts less effective. Lastly, there is a significant gap between public academic datasets (like FaceForensics++ or DFDC) and real-world deepfake attacks, which tend to employ different software, compression techniques, and evasion strategies. This mismatch underscores the inherent difficulty of generalizing deepfake detectors to unseen conditions, reaffirming that even the most advanced models today may struggle to cope with the diversity and adaptability of real-world deepfake attacks.
Insights from 3rd party studies on AI deepfake detection software
Our findings align with recent research studies that examine why AI deepfake detection is so brittle and how it might be improved. Here are a few relevant insights from the literature:
-
Generalization is poor in “zero-shot” settings: Academic studies have found that deepfake detectors that perform well on one dataset often perform poorly when tested on a new dataset without any fine-tuning. This suggests that no detector today is truly “one-size-fits-all” – some amount of retraining or adaptation is needed when confronted with new kinds of deepfakes. Zero-shot learning (ZSL) refers to the ability of a model to recognize and classify objects or images from classes it has never seen curing its training phase, this is further complicated as deepfakes become more prevalent.
-
Detectors often learn surface artifacts, not underlying essence: Researchers have observed that many deepfake detectors latch onto peculiarities of the specific generative method used to create fakes, rather than learning general characteristics of fake vs real images. e.g., a model might learn the noise signature left by a particular GAN or the slight warping from a certain face swap technique. This works well for catching fakes made by that same method, but it doesn’t generalize – a different generator that doesn’t have those artifacts will evade detection. In effect, the detector is picking up synthesis-specific features (the “fingerprints” of the fake generator) instead of universal indicators of inauthenticity. This explains why cross-dataset performance is so poor; each dataset’s fakes were made by different tools, so detectors trained on one set fail on another. Ideally, detectors need to identify more intrinsic signs of manipulation—if such clues exist—rather than relying on superficial signatures.
Why AI deepfake detection alone is not enough
There are several techniques that should be in place to increase the quality and precision of deepfake detection. One key approach is using diverse training data. Detectors should be trained on a broad mix of data—not only curated academic deepfakes but also examples from real production incidents and additional synthetic data designed to mimic attacker behavior. This mix, covering various deepfake types such as face swaps, morphs, fully AI-generated faces, and lip-sync videos helps the model learn a wide spectrum of attacks and reduces potential blind spots.
Another important technique is data augmentation and simulation. By artificially introducing conditions like camera noise, motion blur, lower resolutions, and screen-recording artifacts into training samples, the model can better handle the variations seen in real-world scenarios.
Finally, continuous learning is essential. Deepfake detection systems must continuously update and adapt by incorporating real-world feedback and new attack data, ensuring that the detection system remains current and effective against evolving deepfake techniques.
Given the limitations discussed above, it’s clear that content-based deepfake detection cannot be the only line of defense. If an organization simply deploys a deepfake detection algorithm and assumes the problem is solved, they are at high risk. Attackers can and will find ways around detection models – either by employing novel fakes the model doesn’t recognize, or by avoiding presenting obvious deepfakes at all (circumventing the mechanism). To truly secure biometric systems against AI-generated imposters, we need a layered approach. In practice, this means combining deepfake detection with other protective measures designed to counter the specific attack vectors described earlier. Two critical layers are:
-
Presentation Attack Detection (PAD): This is the technology used to detect spoofing when a fake is physically presented to the camera. PAD includes liveness checks and other signals to ensure the subject in front of the device camera is a real live person, not an imposter showing a doctored image or video. In the context of deepfakes, a good PAD system can catch someone trying to play a deepfake video or display a static mask. It serves as a gatekeeper to block many simple deepfake presentation attacks before the content even gets analyzed. While PAD itself can sometimes be bypassed by very sophisticated attacks, it significantly raises the bar by forcing the attacker to simulate a live presence convincingly.
-
Injection attack detection: To handle the more covert threat of injected deepfakes (where the fake imagery is fed directly into the system), specialized injection attack detection mechanisms are needed. These focus on verifying the authenticity of the data source and channel. Techniques here include detecting the use of virtual cameras or emulators, ensuring the data originates from the expected (and therefore trusted) camera hardware (e.g., via cryptographic signing of camera frames or hardware binding), and spotting anomalies in the data stream that indicate tampering. The idea is to ensure the selfie or video being analyzed actually came from a live capture at that moment, and not from a pre-recorded deepfake file. By implementing injection attack detection, even if an attacker crafts a perfect deepfake that would fool the AI detector, they still can’t easily insert it into the process without tripping an alarm. This layer is a backstop against attackers who try to bypass the camera/feed integrity.
In essence, AI deepfake detection analyzes voice, image, and video content for AI manipulation but must be complemented by PAD (which verifies how the content is captured or presented) and data integrity checks (which verify where the content is coming from). This type of multi-layered defense addresses the problem from multiple angles. Even if a deepfake slips past one layer, another layer can catch it. This defense-in-depth approach is considered best practice by many security experts, especially as deepfake generators become more sophisticated. Relying on just deepfake detection alone is not enough; it’s like locking your front door while leaving the windows wide open.
The road ahead: mitigating deepfake threats with multi-layered security
The evolution of deepfakes has reached a point where they pose a serious and escalating threat to security systems. As our own testing has demonstrated, even state-of-the-art deepfake detectors can be bypassed by some of the wide-ranging and hyper-realistic deepfakes encountered in real-world attacks. Worse still, attackers are innovating quickly – using new tools, testing against detection models, and finding gaps. Simply deploying a deepfake detection model and hoping for the best is not a viable strategy. Deepfakes need to be treated as a new form of adversary in identity verification and new forms of resilient defenses need to be implemented accordingly.
A robust security strategy against deepfake fraud must be multi-layered. Deepfake detection algorithms are an important piece of the puzzle, but they should operate in concert with strong presentation attack detection and injection attack prevention. This way, even if the deepfake content itself doesn’t trigger the detector, the manner in which it’s introduced can be caught. By layering defenses, organizations can significantly reduce the risk of a deepfake slipping through and causing damage.
Looking ahead, continued research and innovation will be essential. The industry and academic community are actively working on improving detection generalization, exploring new verification signals, and hardening systems against injections. There is also a need for sharing of threat intelligence – as new deepfake attack patterns emerge, collectively learning and updating defenses will be key. As a collective industry, we’re essentially in an arms race with deepfake creators, and to stay ahead, defenders must be as adaptive and creative as the attackers.
In conclusion, deepfakes are here to stay, and their impact on security will only grow. But by understanding how these attacks work and deploying a layered defense that goes beyond just detection, we can significantly mitigate the risk. The takeaway for any security-conscious organization is clear: don’t rely on deepfake detection alone. Treat it as one layer in a broader anti-spoofing strategy. Strengthen your onboarding and authentication processes with liveness checks, integrity checks, and continuous updates as new deepfake threats emerge. With vigilance and a multi-faceted approach, we can continue to securely verify identities – even in the age of deepfakes – and uphold trust in digital systems.
Denis Kondranin, Sr. Machine Learning Engineer at Mitek, contributed valuable insights and data from our internal testing that helped shape several key findings in this blog.
About Anastasia Molotkova - Product Manager at Mitek
Anastasia Molotkova is a certified product leader, specializing in AI-driven cybersecurity solutions that address emerging threats like deepfakes and generative AI fraud. She leads the development of innovative biometric technologies, including injection attack detection, deepfake detection and facial liveness detection, helping to set new security standards in digital onboarding and authentication.