AI for Ethical Hackers and Cybercriminals: A Dual-Use Map and What Defenders Can Do
Discover how AI tools like WormGPT, Code Llama, and AutoGPT are being used by both ethical hackers and cybercriminals for red teaming, phishing, malware creation, and exploitation. Learn key use cases, risks, and how to stay protected.
Quick answer: AI speeds up both sides of security. Defenders use it for scripting, simulations, summarising and attack surface mapping, while criminals use it for convincing phishing, faster malware changes, deepfake fraud and reconnaissance. Counter with phishing-resistant MFA, behaviour-based detection, out-of-band payment checks, patching and least privilege for AI tools.
Key takeaways
- AI is dual-use: the same capability speeds up defenders and attackers.
- Better phishing and deepfakes call for phishing-resistant MFA and out-of-band payment verification.
- Behaviour-based detection beats signatures against rapidly changed malware.
- Give AI tools least privilege and treat their input and output as untrusted.
Why is AI a dual-use technology?
A tool that writes code, drafts convincing text, mimics a voice or reads thousands of pages in minutes helps whoever uses it. In security that means defenders and attackers get faster at the same things. This article maps the main categories of AI use on each side and, for each, what defenders can do. It does not give instructions for attacking anyone. Any testing you do must be on your own lab or with written authorisation, since the IT Act applies whatever tool you use.
Quick view: the same capability, two uses
| AI capability | Defensive and authorised use | Criminal use | Best counter |
|---|---|---|---|
| Code assistance | Speed up scripts, log parsers and secure code review | Faster malware variants and obfuscation | Behaviour-based EDR, application control |
| Text generation | Phishing simulations and awareness training | Convincing, personalised phishing at scale | Phishing-resistant MFA, context-aware mail filtering, reporting culture |
| Voice and video synthesis | Permissioned drills that test payment verification | Impersonating executives or relatives for fraud | Out-of-band call-back, dual approval for payments |
| Automated research agents | Mapping your own attack surface | Finding exposed systems and building target lists faster | Reduce exposure, patch, monitor internet-facing assets |
| Fuzzing and analysis | Finding bugs for responsible disclosure | Finding bugs to exploit before patches | Faster patching, defence in depth |
| Attacks on AI systems themselves | Testing your own chatbots for prompt injection | Hijacking assistants to leak data | Least privilege for AI tools, input and output controls |
1. Code assistants
Defenders: assistants help write detection scripts, review code for common flaws and explain unfamiliar code. Treat the output as a draft, since it can contain bugs and insecure patterns.
Attackers: the same speed-up applies to writing and rewriting malicious code, which weakens signature-only antivirus.
What to do: rely on behaviour detection rather than file signatures alone, restrict script execution where you can, and log process creation.
2. Generative text and phishing
Poor spelling used to be a giveaway. Language models remove that tell, and can personalise messages with details gathered from public profiles. Underground tools such as WormGPT were reported in 2023, and a stream of clones followed. Their claims often outrun their quality, but the direction is clear.
What to do:
- Use phishing-resistant MFA (passkeys or hardware security keys) so a stolen password or one-time code is not enough.
- Authenticate your own domains with SPF, DKIM and DMARC.
- Train staff to verify requests by another channel, especially for money or credentials, and make reporting easy.
- Run authorised simulations using a proper platform with written approval.
3. Deepfake voice and video
Synthetic voice and video have been used in payment fraud, including a widely reported 2024 case in Hong Kong in which an employee was deceived on a video call with fake colleagues. The lesson is not to trust a face or voice alone.
What to do: require a second, independent confirmation for payments and credential resets, such as a call-back to a known number, and use dual approval for large transfers. Agree a code phrase for high-risk requests inside families and teams.
4. Automated reconnaissance
Agents that combine search, scanning data and summarisation shorten the time between "a service is exposed" and "someone has noticed". Defenders can use the same approach on themselves, running authorised discovery on their own assets and fixing what they find.
What to do: keep an inventory of internet-facing systems, patch fast, close unneeded ports and monitor for scans against you.
5. What is AI-assisted bug finding?
Fuzzing and code analysis, with or without machine learning, find vulnerabilities faster. Researchers use this for responsible disclosure, and criminals can use it too. Faster patch cycles and layered defences matter more as discovery speeds up.
6. What are attacks on AI systems?
When a company deploys a chatbot or agent connected to internal data, attackers target it. Prompt injection hides instructions in text the model reads. See the OWASP project for guidance on risks to LLM applications: OWASP.
What to do: give AI tools the least access they need, treat their input and output as untrusted, require human approval for sensitive actions and log what they do.
What this means for your career
Defenders who understand how AI is used against them detect and respond better. Ethical hackers who use AI should do so inside a lab or contract, keep client data out of public tools and verify every output. Learning the fundamentals, such as networking, Linux, web security and scripting, matters more than ever because they let you judge what AI produces.
Next steps
To build the fundamentals, see the AI Security and LLM Security course. Related reading: how hackers use AI for cybercrime and AI tools used by ethical and black-hat hackers.
Frequently Asked Questions
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Wow
0
Sad
0
Angry
0