How Criminals Use AI in Cybercrime and How to Defend Against It

Quick answer: Criminals use AI mainly to make old attacks faster and more convincing: fluent phishing in any language, cloned voices and deepfake video for impersonation, fake profiles and documents for fraud, and help with code and reconnaissance. The FBI warned in December 2024 that criminals use generative AI to increase the believability of fraud. Defences are largely the same as before, applied more strictly: verification of payment and access requests through a second channel, phishing-resistant MFA, staff training on deepfakes, and email authentication.

Key takeaways

• AI improves the quality and scale of social engineering more than it creates new attacks.
• Deepfake video calls have been used for large frauds, such as the Arup case in Hong Kong.
• A call-back to a known number defeats most voice and video impersonation.
• AI vendors publish reports on accounts they disrupted for malicious use.
• Payment and access requests need process controls, not just awareness.

Table of Contents

• Which AI Attack Methods Are Common?
• Which Real Cases Show the Risk?
• How Do You Stop Deepfake Payment Fraud?
• How Do You Check a Suspicious Email?
• What Should Organisations Do?
• What Do Government Agencies Advise?
• How Do You Find Lookalike Domains?
• Can Attackers Target AI Systems Directly?
• References

Which AI Attack Methods Are Common?

Which Real Cases Show the Risk?

• In 2024 a finance employee at engineering firm Arup in Hong Kong transferred about HK$200 million after a video call in which other participants were deepfakes, as CNN reported after Arup confirmed the incident.
• The FBI's December 2024 announcement described criminals using AI-generated text, images, audio and video in fraud schemes.
• OpenAI's threat reports describe disrupted accounts using its models for scams, influence operations and help with malicious code.

How Do You Stop Deepfake Payment Fraud?

1. No payment or bank detail change is approved on the strength of a call, video or email alone.
2. Confirm by calling back on a number from your own records, not one in the message.
3. Require two people to approve payments above a threshold.
4. Treat urgency and secrecy as warning signs.
5. Agree code words with family members for emergency money requests.

How Do You Check a Suspicious Email?

Look at the authentication results in the message headers. A legitimate message from a well-configured domain usually passes SPF, DKIM and DMARC:

Authentication-Results: mx.example.com;
  spf=pass smtp.mailfrom=billing.vendor.com;
  dkim=pass header.d=vendor.com;
  dmarc=pass header.from=vendor.com

For your own domain, publish a DMARC policy so others can reject spoofed mail:

dig +short TXT _dmarc.example.com
# a strict policy looks like:
# "v=DMARC1; p=reject; rua=mailto:[email protected]"

A pass does not prove intent: attackers use lookalike domains that pass their own checks, so read the domain carefully.

What Should Organisations Do?

• Train finance, HR and help desk staff with deepfake examples.
• Move to phishing-resistant MFA for privileged and finance users.
• Enforce DMARC on your domains and monitor lookalike domains.
• Write a verification procedure and test it with exercises.
• Report incidents to CERT-In within six hours, and fraud to cybercrime.gov.in or 1930 in India.

Read our guides on AI spear phishing and MFA bypass, or talk to a WebAsha training advisor.

What Do Government Agencies Advise?

In September 2023 the NSA, FBI and CISA published a joint sheet, Contextualizing Deepfake Threats to Organizations. Its recommendations cover verification, detection and response planning:

How Do You Find Lookalike Domains?

AI makes it easy to write convincing emails, but they still need a sender domain. The open-source dnstwist tool generates typo and lookalike variants of your domain and shows which are registered:

pip install dnstwist
dnstwist --registered --format csv example.com > lookalikes.csv
# check whether a suspicious lookalike has mail servers set up
dig +short MX examp1e.com

A lookalike with MX records is ready to send or receive mail. Add such domains to email filters and consider registering the most obvious ones yourself.

To triage a reported message quickly, a short script can read the authentication results from a saved .eml file:

import email, sys
from email import policy

msg = email.message_from_file(open(sys.argv[1], encoding="utf-8", errors="ignore"), policy=policy.default)
print("From:       ", msg["From"])
print("Reply-To:   ", msg["Reply-To"])
print("Return-Path:", msg["Return-Path"])
for h in msg.get_all("Authentication-Results", []):
    print("Auth:       ", " ".join(h.split()))

A Reply-To that differs from From, or a failed DMARC result, is a reason to verify before acting.

Can Attackers Target AI Systems Directly?

Yes. As organisations adopt AI, the models themselves become targets. NIST's AI 100-2 E2025 taxonomy groups attacks into evasion, poisoning, privacy attacks and, for generative AI, misuse such as prompt injection. The OWASP Top 10 for LLM Applications turns these into practical risks for developers. Our guide to prompt injection covers the most common one.

References

• FBI IC3 PSA, December 2024
• CNN, Arup deepfake fraud
• OpenAI, disrupting malicious uses of AI
• MITRE ATLAS
• NSA, FBI and CISA, Contextualizing Deepfake Threats
• dnstwist
• NIST AI 100-2 E2025
• OWASP Top 10 for LLM Applications