Offensive AI: How Criminals Are Already Using Artificial Intelligence

For decades, one of the best lines of defense against phishing was… the attacker’s poor Polish. “Dear customer, your account has been blocked, click here immediately” - typos, strange syntax and language collages revealed fraud faster than any filter. This era is over. Generative AI has lowered the barrier to entry for attackers so dramatically that the signals we taught employees to rely on largely no longer work. Let’s see how criminals actually use AI - without science fiction, based on what we see in attacks and tests.

AI doesn’t create new attacks - it scales old ones

Let’s start with a key distinction, because there are many myths surrounding “AI in the hands of hackers”. Artificial intelligence, for the most part, does not invent new categories of attacks. What it does is more dangerous in practice: it scales and makes more credible attacks that we have known for years. It removes three traditional limitations from the attacker:

• Language barrier - perfect text in any language, in any tone.
• Time barrier - what used to take hours (researching the victim, writing content) takes seconds.
• Skills barrier - a person without technical knowledge receives an “assistant” who suggests next steps.

The result: more attacks, better suited, cheaper to carry out. It’s a quantitative change that becomes qualitative.

Phishing and social engineering without bugs

The most immediate effect is the end of phishing that revealed itself with language. The language model will write the message in impeccable Polish, in a corporate style tailored to the target’s industry - and if you give it some public information about the company, it will personalize the content so that it is difficult to distinguish it from real correspondence.

However, spear phishing on a large scale is more dangerous. Until now, a targeted, refined attack on a specific person was expensive, so it was reserved for high-value targets. AI means that everyone can receive a tailor-made message: mentioning their position, the last project visible on LinkedIn or the name of their superior. What we describe in the article why anti-phishing training is no longer sufficient, AI only accelerates - because the last cheap warning signal disappears.

Voice and video deepfakes: real-time social engineering

Voice cloning today is a matter of several dozen seconds of an audio sample - and this is provided by recordings from webinars, interviews, podcasts or voicemails. The attacker calls a finance department employee in the “president’s voice” with an urgent request for a transfer, or joins a video conference as a fake board member. This is not a theory: there are known cases in which employees authorized multi-million-dollar transactions after a video call with “management” that was entirely a deepfake.

This is an evolution of the classic president fraud (BEC) and vishingu: there is an audio-video layer that overcomes the “I’ll call and check” reflex. The topic is so important that we devote a separate article to it - voice and video deepfake in attacks on companies.

AI-powered malware

There’s a lot of hype around “AI-written malware” - let’s make it real. The models do not (yet) generate breakthrough new, undetectable viruses. The real impact is more mundane but significant:

• Development acceleration - the attacker assembles working code faster, obfuscates it and adapts it to the victim’s environment.
• Polymorphism - AI makes it easier to generate multiple variants of the same malware, making signature-based detection difficult (which is why defense shifts towards behavior analysis).
• Support at every stage - from writing phishing emails delivering the payload to lateral movement scripts.

So it’s not about “AI creates the perfect weapon”, but about a less skilled attacker achieving a result that previously required a team.

Automated reconnaissance and OSINT

Before an attack comes reconnaissance. AI radically accelerates intelligence gathering: it summarises employee profiles, combines data from multiple sources, maps the organisation and identifies promising social-engineering targets - all from publicly available information. This is the same digital footprint we describe in the context of OSINT, compressed from days into minutes. The attacker receives a ready-made brief: whom to approach, whom to impersonate and which pretext to use.

Underground models and jailbreaks

Commercial models have safeguards (guardrails) that refuse to help with obviously harmful tasks. Criminals get around this in two ways. The first are jailbreaks - crafted commands that trick the model into breaking its own rules; it’s an arms race in which defense and offense chase each other. The second is underground models (under the banner of “WormGPT”/“FraudGPT”) - tools advertised on forums without any restrictions, sold on subscription as “AI for crime”. Their quality is sometimes overestimated, but the very fact of the existence of the market shows the direction: crime as a service gets an AI layer.

Deception on a scale: fake personas and content

Generative AI is a factory of credible fakes, not only in e-mail. Fake social media profiles with photos of generated “people” who do not exist; fake reviews; fake recruitment websites; cloned sites. This builds a background of trust for the actual attack - the victim “checks” the attacker and finds a consistent but completely made-up identity.

What does this mean for defense

As AI nullifies some of the old warning signals, defense must shift the burden from “identification by appearance” to processes and verification through an independent channel:

• Out-of-band verification for sensitive operations. No transfer or change of bank details should depend on one message or one phone call - even if the voice is correct. Call back the known number, confirm via the second channel.
• Strong authentication resistant to phishing. Since credentials will be phished more and more effectively, MFA (preferably passkeys) limits the effects of password hijacking.
• Training update. Stop teaching “typos recognition”. Teach the principle of limited trust and verification procedures, because that’s what works when the content is perfect.
• Defense based on behavior, not signatures. Due to polymorphism and personalization, the role of anomaly detection is growing - we write about how AI supports defenders in AI on the side of defense.
• Social engineering tests including AI. Controlled exercises (including deepfakes) show the real resilience of the organization before the attacker does.

Summary

Offensive AI is not a futuristic bogeyman, but an everyday reality that has already changed the economics of attack: perfect phishing, voice cloning, faster reconnaissance and crime as a service with an AI layer. The good news is that the foundations of defense remain the same - strong authentication, independent channel verification, permissions control and behavior detection. The bad: there is no longer the cheap margin of error that scammers’ typos gave us.

If you want to see how your organization fares against AI-powered social engineering attacks - including deepfake and realistic phishing tests - let’s talk. Security testing and implementation support AI is one of our specializations.

Frequently asked questions (FAQ)

Does AI really create “undetectable” viruses? Not as much as the headlines suggest. AI speeds up the creation and modification of malicious code and lowers the required skills, but it does not magically generate weapons that bypass all security measures. The biggest real impact is on the side of social engineering and scale, not the malware itself. Defense based on behavior analysis (not just signatures) remains effective.

Where do criminals get voice samples for cloning? From publicly available recordings: webinars, interviews, podcasts, conferences, social media videos, and even voicemail greetings. Nowadays, several dozen seconds of material are enough to reliably clone the timbre of a voice. Therefore, it is crucial to verify identity by procedure, and not by “recognizing by voice”.

Do our anti-phishing trainings still make sense? Yes, but their content needs to be changed. Teaching “recognizing language errors” has lost its value because AI writes error-free. Effective training emphasizes the principle of limited trust, verification of sensitive requests through an independent channel and knowledge of procedures (e.g. confirming changes to bank details) - because this works regardless of the quality of the content.

How to prepare your company for AI-powered attacks? Shift defense from visual signals to processes: introduce out-of-band verification for financial transactions and data changes, implement phishing-resistant authentication, update training and consider controlled social engineering tests (including with deepfakes). It is also worth strengthening behavior-based detection in security monitoring.