From convincing phishing emails to cloned voices and fake video calls, artificial intelligence is changing how online fraud is carried out. As criminals use AI to imitate trusted people, brands and organisations, familiar warning signs are becoming less reliable. Recent cybersecurity research suggests that detecting social engineering increasingly requires more than judging how genuine a message appears.
An email arrives from a familiar brand asking a customer to update payment details. The language is professional, the logo looks genuine and the instructions resemble a routine account notification.
Elsewhere, an employee receives a call from someone claiming to be from the IT department. The caller knows the organisation, refers to an existing process and sounds convincing.
Both interactions could be legitimate. They could also be social-engineering attempts designed to obtain sensitive information or persuade somebody to authorise an unwanted action.
For years, poor grammar, unusual phrasing and suspicious-looking communication helped people recognise potential scams. Artificial intelligence is making several of those indicators less dependable.
Generative AI can produce polished messages, adapt language for different audiences, create convincing visual material and imitate voices. These capabilities are making fraudulent communication resemble the automated messages and digital interactions people already encounter daily.
The underlying crime is not new. Social engineering has historically relied on manipulating trust, urgency, authority and familiarity. What is changing is the speed and scale at which convincing material can be produced.
Microsoft’s Digital Defense Report 2026 recorded more than 46 million business-email impersonation attacks and found that 93% of observed voice-phishing attacks kept victims engaged long enough for social engineering to begin.
Meanwhile, the FBI’s 2025 Internet Crime Report, released in 2026, recorded 22,364 complaints containing AI-related information, with adjusted losses exceeding $893 million.
The FBI received more than one million internet crime complaints overall, with reported losses reaching approximately $20.9 billion.
These findings do not mean every online scam involves AI. They indicate that AI is becoming another tool in an already substantial fraud ecosystem, complicating the distinction between genuine and fraudulent communication.
The challenge is particularly relevant for organisations communicating with customers through email, social media, messaging platforms and increasingly automated service channels.
1. Phishing Emails Are Becoming More Convincing
Traditional phishing often involved poorly written emails, generic greetings or obvious inconsistencies in company branding.
Those warning signs have not disappeared, but relying on them alone is becoming less effective.
Generative AI can rewrite messages in professional language, produce variations for different audiences and incorporate familiar corporate terminology. It can also help create visually consistent communication using publicly available brand elements.
Hoxhunt’s 2026 Phishing Trends Report, drawing on more than 50 million data points from over four million users, found a sharp increase in suspected AI-assisted phishing towards the end of 2025.
In sampled attacks, phishing emails displaying indicators of AI assistance accounted for 4% of reported phishing in November 2025. That proportion increased to 56% in December before declining to 40% in January 2026.
The study examined reported phishing that had bypassed email-security filters. Researchers assessed characteristics consistent with AI assistance rather than independently proving the production method of every message.
Nevertheless, the findings highlight how phishing content is changing.
“AI is making social engineering faster and more scalable. When we keep training timely and relevant, people can adapt just as fast as the threats,” said Maxime Cartier, VP of Human Risk at Hoxhunt.
The problem extends beyond improved writing.
Microsoft documented campaigns in 2026 that impersonated established AI companies, including OpenAI’s ChatGPT and Anthropic’s Claude.
In one campaign, attackers used ChatGPT branding and a supposed payment-update requirement to direct recipients towards fraudulent destinations. Microsoft observed related activity reaching as many as 100,000 emails in a single day.
Another campaign used Claude branding and account-policy warnings to target users across more than 2,000 organisations, including targets in India.
Such campaigns demonstrate how attackers can exploit the familiarity of products that people increasingly use for legitimate work.
For marketing teams, this creates a reputational concern alongside the cybersecurity risk.
Brands spend considerable resources developing recognisable identities, consistent communication and frictionless customer experiences. Those same visual elements can be reproduced in fraudulent messages.
A familiar logo or well-written notification may therefore make an interaction appear credible without establishing that the sender is authentic.
The risk is especially relevant when customers regularly receive automated payment reminders, subscription alerts and promotional messages from genuine companies.
2. A Familiar Voice or Face Is No Longer Enough
The challenge becomes more complicated when impersonation moves beyond written communication.
AI-generated audio and video can reproduce characteristics associated with real people, weakening the assumption that hearing a familiar voice or seeing somebody on screen confirms their identity.
Pindrop’s 2026 Deepfake Readiness Index, based on a survey of 250 US security leaders at organisations employing at least 1,000 people, found that 74% had encountered or suspected a deepfake attack during the previous year.
However, only 10% reported having purpose-built tools to address deepfake threats.
Among organisations that experienced or suspected such attacks, approximately one in four reported costs of at least $1 million from a single incident, including associated consequences.
“Deepfakes turn our most instinctive signals of identity, a familiar face and voice, into an attack surface,” said Elie Khoury, SVP of Research at Pindrop.
The findings reflect reported experiences and suspicions among surveyed security leaders, rather than a verified count of all deepfake incidents.
They nevertheless highlight a challenge for businesses accustomed to treating voice and video communication as additional forms of reassurance.
Consider a finance employee receiving an urgent payment request during a video meeting. The person appearing on screen resembles a senior executive, and the request refers to a familiar business transaction.
Even when the interaction appears credible, the identity of the person issuing the instruction may still require independent confirmation.
The FBI documented a similar combination of visual impersonation and institutional credibility in July 2026.
The bureau warned that criminals were using AI-generated videos impersonating FBI personnel and directing previous fraud victims towards spoofed Internet Crime Complaint Center websites.
In these cases, synthetic media was part of a wider deception involving familiar authority figures and apparently legitimate digital destinations.
The FBI’s annual report also recorded more than $30 million in losses associated with business-email-compromise complaints involving AI.
Such examples show why social engineering cannot always be identified by examining a single piece of content.
A convincing message, familiar voice and professionally designed website can appear together, creating an impression of authenticity even when the underlying request is fraudulent.
3. Social Engineering Is Moving Beyond a Single Message
The growth of AI-assisted deception is also changing how fraud attempts develop across communication channels.
A conventional phishing email is usually a fixed message. A live conversation allows an attacker to respond to hesitation, introduce additional information or create pressure around an immediate decision.
That adaptability makes phone-based and multi-channel attacks particularly relevant.
CrowdStrike’s 2026 Threat Hunting Report found that voice-phishing intrusions doubled during the first half of 2026 compared with the second half of 2025.
The company also recorded a fifteenfold increase in monthly device-code phishing attempts during the first half of the year, illustrating growing abuse of legitimate authentication processes.
These statistics cover broader threat activity and do not establish that every observed intrusion involved generative AI.
However, CrowdStrike identified AI as an increasingly relevant capability within modern attacker operations.
“AI is now embedded in modern adversary operations. It is changing how attacks are planned, executed, and scaled while expanding the attack surface organizations must defend,” said Adam Meyers, Head of Counter Adversary Operations at CrowdStrike.
Microsoft’s research similarly suggests that attackers are combining familiar communication methods with trusted digital infrastructure.
A fraudulent interaction may begin with an ordinary-looking email and continue through a phone call or collaboration platform. The attacker may reference an actual department, established workflow or publicly available employee information.
For example, an employee could receive a call from someone impersonating internal technical support. The caller might refer to a genuine software system and request assistance with what appears to be a routine account issue.
The fraud becomes harder to judge because legitimate information is mixed with misleading instructions.
This also creates a distinction between AI-assisted attacks and fully autonomous scams.
Current research shows AI being used to improve content, support preparation and accelerate certain malicious activities. It does not establish that most social-engineering campaigns operate autonomously without human involvement.
Attackers may still rely on people to conduct calls, establish trust and decide when to escalate an interaction.
For organisations, the threat increasingly involves the credibility of an entire communication sequence rather than an isolated suspicious email.
4. Why Brands May Need to Rethink How They Establish Trust
As fraudulent messages become more polished, cybersecurity guidance is placing greater emphasis on verifying requests through independent channels.
The FBI advises recipients of suspicious communications to contact organisations using independently obtained official information rather than telephone numbers or websites supplied within the questionable interaction.
For businesses, this shifts attention towards the processes surrounding sensitive decisions.
Payment changes, credential resets and requests involving confidential information can require additional verification, even when the initial communication appears to come from an authorised employee or familiar contact.
A finance team, for instance, can independently confirm an unexpected change in a supplier’s banking details before releasing payment.
A customer-service operation can apply stronger identity checks before allowing access to an account, rather than relying solely on personal information supplied during a call.
These measures reduce dependence on whether an individual correctly identifies AI-generated content.
The implications extend to customer communication and marketing.
Brands increasingly interact with consumers through automated emails, messaging assistants, chatbots, social platforms and digital payment interfaces.
As those interactions become more common, distinguishing official communication from impersonation becomes part of maintaining customer trust.
Organisations may need to communicate more clearly which channels they use, how they handle sensitive requests and when customers should expect additional verification.
This is particularly relevant for financial services, ecommerce, travel and subscription-based businesses, where customers regularly receive payment notifications, promotional offers and account-related communication.
At the same time, additional security requirements must be balanced against customer convenience. Excessive verification can create friction, while insufficient safeguards may expose customers and employees to impersonation.
The objective is not to treat every digital interaction as fraudulent but to ensure that important decisions are supported by reliable confirmation.
The research also indicates that defensive capabilities are developing alongside the threats.
AI can support threat detection, investigation and security training, while Hoxhunt’s findings suggest that timely, relevant training can improve employees’ ability to recognise and report suspicious activity.
The available evidence does not show that AI makes every social-engineering attempt successful. Nor does it suggest that traditional indicators of fraud have become entirely irrelevant.
Instead, it highlights the limitations of relying mainly on appearances.
Correct grammar does not establish that an email is authentic. A familiar voice does not independently confirm identity. A recognised logo or legitimate-looking website does not prove that a payment request is genuine.
As AI reduces some of the visible differences between authentic and fraudulent communication, organisations are being pushed to distinguish between how convincing an interaction appears and whether its source and instructions have been verified.
For brands, employees and consumers, that represents a significant change in digital trust.
AI has not changed the basic objective of social engineering: persuading somebody to act on a false premise. It is changing how convincingly that premise can be presented. The challenge now is to verify who is making a request and why, even when the message looks entirely familiar.
Disclaimer: All data points and statistics are attributed to published research studies and verified market research. All quotes are either sourced directly or attributed to public statements.
Related Articles
