Deepfakes – an emerging technology creating fake content and media with serious implications
In May of 2019, a Youtuber called “ctrl-shift-face” uploads a video interview of actor and SNL alum Bill Hader doing celebrity impressions. At first glance, it appears Hader is using his comedy chops and impersonating famous people to drum up press for a new creative project that’s releasing soon. Hader launches into the famous “Arnold Schwarzenegger” voice, inflecting the Austrian superstar’s European drawl.
Though Arnold’s timbre has been replicated by hundred of comics looking for laughs, what’s unique about this video isn’t Hader’s impression (which is pretty good), but what happens when he impersonates Arnold’s voice – his face eerily becomes the Governor’s, as if he was wearing a digital mask. The squinty eyes, furrowed brow, chiseled chin – all of it is superimposed on to Hader’s head in the altered video, as if Schwarznegger is sitting in the studio talking to the audience.
The video is entertaining, but, there’s an eerie quality about how similar Hader looks to Schwarzenegger. Comments underneath the clip say things like “After he started looking like Arnold, I forgot what he actually looks like,” “I thought I was hallucinating,” and “he even looks like him lol.”
The technology used to create the video is called “DeepFakes,” and has become a popular digital tool to create synthetic media. In the context of the Deepfake creator’s video, it appears harmless. But now, consumers, businesses, and governments a like are debating the implications of DeepFakes (our own CTO Stephen Ritter has talked about this here), especially after its recent uses in nefarious purposes like fake news and deceptive media that could potentially affect national elections.
As a part of Mitek’s Fraud series, we’ll discuss in this blog: What is a Deepfake? Where did Deepfake technology come from? How do you make a Deepfake? How do you spot a Deepfake? Are there Deepfake detectors? Can Deepfakes be used in new account opening and account takeover fraud? How does a company fight Deepfakes in 2020?
Though Deepfakes are apart of the increasing fraud trends on digital platforms, this is only one of the tools fraudsters use everyday for cybercrime in order to steal millions of dollars, identities, and livelihood. Download the latest White Paper “Fraud trends and tectonics” where Mitek’s Global Head of Product, Sanjay Gupta, talks about 2020 fraud trends, detection tools, and what emerging technology businesses can use to protect themselves.
What is a Deepfake?
Quick summary: A Deepfake is an emerging technology created by machine learning and artificial intelligence used to alter videos, emulate forgeries of people doing or saying malicious things, creating convincing synthetic audio, and other forms of fake content where humans are present. Similarly, Deepfakes can be generated from using existing images to create something places, people, and things that are entirely synthetic.
To date, Deepfake technology have been used to create fake content, where: biometrics like facial expressions are generated and superimposed onto another person’s body in fake videos; human voices matching the timbre and pitches of celebrities make like Jay-Z singing Billy Joel in Deepfake audio recordings ; or politicians saying things they’ve never said before. As the technology gets better, fraudsters are beginning to use malicious Deepfakes for cybercrimes and corporate espionage for more than just "fake news" (see the section below on Fraud).
Where did Deepfake technology come from?
In a world where everyone now *hopefully* questions the validity of the content they see on the internet, the first known Deepfakes were fake videos generated by artificial intelligence published at the start of 2017 by a Reddit user, deepfake, to the platform. Today, the user has been credited as Deepfakes’ creator, bringing the overall practice into public view. [Editor’s note: this user is separate than redditor ctrl+shift+face described in the introduction… however there’s no evidence to suggest they aren’t the same person].
deepfake used open source images libraries like Google image search, social media websites, stock photos, tensorflow, and YouTube videos to create a machine-learning algorithm which allowed the user to insert people’s faces onto pre-existing videos frame by frame.
Although there are glitches and obvious catches a user can notice, the videos are quite believable and are only getting more convincing as more users continue to experiment. At the time, deepfake even created and released an app called “FakeApp” making it more accessible for basic, less tech-savvy users to participate in creating fake content from funny videos to more malicious ones. Today, there are dozens of Deepfake generators, with even some larger software companies releasing Deepfake technology and artificial intelligence capabilities that are making it even easier.
How do you make a Deepfake?
In order to make a Deepfake video, you’ll need a couple of things: a super-powered computer, a couple of artificial intelligence and machine learning programs, and hundreds of thousands of images of two selected people. We didn’t say these would be easy to come by .
Here’s the process:
Another method to create Deepfakes use a generative adversarial network (Gan) . The notable difference here is the Gan creates an entirely new image/video that looks incredibly real, but is entirely fake. Here’s how it works:
How do you spot a Deepfake?
Since its inception, Deepfakes have gotten harder to spot as technology continues its inexorable march of minute by minute improvements. Here’s some ways you can spot one:
Don’t blink or you’ll miss it: Deepfake faces may not blink normally due to the fact that a majority of images run through the artificial intelligence creation show people with their eyes open.
Would you kiss those lips? If the lip synching appears off, it may not just be the sound bar. Compare the sound on a different video for a Deepfake sanity check.
Smile – you’re on camera: teeth can be a tip-off because some algorithms may have a hard time rendering them individually. Keep your eye out for a cartoonish slab of pearly whites.
Cover up and wear a hat: Patchy skin from the compression process, and hair that doesn't look properly rendered or moving with head nods or wind is a way to catch a Deepfake.
Watch a lot of them: The more Deepfakes you watch, the better you get at seeing which are real or fake.
Despite all of these tips to spot a Deepfake, Moore’s law doesn’t just apply to the good guys making technology. Because Deepfakes can be made into malicious content so quickly by more enterprising fraudsters or bad actors, many of the ways to spot Deepfakes from this list may be entirely antiquated by the time you’re done reading this 2000 word blog.
Tomorrow, human’s only option may be to use Deepfake detectors run on artificial intelligence and machine learning to capture… Deepfakes that are running on artificial intelligence and machine learning. The irony is not lost on us. Governments, universities, and tech firms are all funding research to create new Deepfake detectors. And Recently, a large consortium of tech companies have kicked off the Deepfake Detection Challenge (DFDC) to find better ways to identify manipulated content and build better detection tools.
Can Deepfakes be used for fraud? How?
In 2019, Instagram has 50 billion images on the platform. Google likely has even more selfies of people in the petabytes it stores (layman’s terms: a lot of pics). Needless to say, most people have some type of digital fingerprint – whether that’s a Linkedin account profile picture, or family photos you’ve shared on Facebook (these items encompass a behavioral biometric profile). And as described above, all of these pictures are potential inputs for AI to begin creating convincing Deepfakes for deceptive media.
So far, Deepfakes have been used mostly for celebrity-bating, political dirty tricks, pornography and one notorious case of corporate extortion: in a report from The Wall Street Journal, the CEO of an energy company believed he was on the phone with his boss, Mr. CEO, when he followed the orders from the other end of the phone to immediately transfer €220,000 (approx. $243,000) to the bank account of a Hungarian supplier. If you’ve read this far, you know what happens next: the audio on the phone was actually an audio Deepfake the fraudsters had created.
As the technology improves and becomes commoditized, it could be used for identity-theft and other cybercrimes including fraudulent account opening and account takeover - two of 2020's top fraud trends. Here’s some a few examples:
What are the solutions for Deepfakes?
Detecting deepfakes is a hard problem. Simplistic Deepfakes can, of course, be detected by the naked eye if they’re bad. Some detection tools can even spot the faulty characteristics we wrote about But, artificial intelligence that generate Deepfakes are getting better all the time, and soon we will have to rely on Deepfake detectors to flag them to us. Still ironic!
To counter this threat, it’s important to make sure companies and providers user facial authentication software that includes certified liveness detection. And because sophisticated Deepfakes can spoof common movements like blinks and nods, authentication processes will need to evolve to guide users through a less predictable range of live actions.
For high-risk/high-value transactions such as money transfers or attempts to change account details, passive methods can be combined with active methods. These might include two-factor authentication using a one-time token or confirmation via an alternate channel, like SMS, that it’s the legitimate user making a change or transaction.
In addition, account holders’ identities may need to be reverified at various moments – even after their account has been set up. Some companies routinely invoke identity verification whenever a dormant account suddenly becomes active for high-value transactions, or when passive analytics indicate elevated fraud risk.
One way to do this is to request a current selfie, then compare it to the biometric data stored from onboarding (where storage is allowed by regulations and permissioned by the customer). In very risky situations, you could also request a new snapshot of the originally submitted government issued physical ID, and take a few seconds to verify the authenticity of the document and compare the photo on the ID against the selfie.
In text citations:
 Jay-z Takes Action Against 'deepfakes' Of Him Rapping Hamlet and Billy Joel
Ben Beaumont-Thomas - https://www.theguardian.com/music/2020/apr/29/jay-z-files-takes-action-against-deepfakes-of-him-rapping-hamlet-and-billy-joel
 What Are Deepfakes – and How Can You Spot Them?
Ian Sample - https://www.theguardian.com/technology/2020/jan/13/what-are-deepfakes-and-how-can-you-spot-them
 Shen, T., Liu, R., Bai, J., & LI, Z. (2018). “Deep Fakes” using Generative Adversarial Networks (GAN). Retrieved from “Deep Fakes” using Generative Adversarial Networks (GAN)