Your team is already using AI. So are the people trying to take your money.
Half of Australian employees now use AI regularly at work, according to a 2025 University of Melbourne and KPMG study. It drafts emails, summarises meetings and tidies up a tricky message to a client.
The time savings are real. But scammers have the same tools, and they're using them.
In late October, the Australian Signals Directorate released its Annual Cyber Threat Report 2025–26. AI sits at the centre of it. Small businesses that reported a cybercrime put the average cost at $55,400. For a business running a 10% net margin, that's the profit on $554,000 of sales.
So there are two questions worth asking. Would your team spot a scam written by AI? And do you know what your team is putting into AI?
Scams no longer look like scams
For years the advice was simple: look for bad spelling, odd phrasing and generic greetings. AI-written scams don't have them.
AI can write a clean, convincing email in seconds. It can copy the style of your supplier or your director, using details from your website and social media. Voice cloning tools can copy a person's voice from a short recording. A call from "the boss" asking for an urgent payment can now sound exactly right.
The scams themselves haven't changed:
A supplier's bank details "change".
An invoice needs paying today.
The director needs a favour before a meeting.
What's changed is how real they look, and how many can be sent at once.
What's going into AI?
The second risk sits inside your business. When a staff member pastes a client list, payroll report or contract into a free AI tool, that information has left your control. Some tools keep what you type and use it to train future versions.
Australia's privacy regulator, the OAIC, warns that personal information entered into a generative AI tool is very hard to track and may be impossible to remove. Its advice is to keep personal information out of publicly available AI tools altogether.
Most staff haven't been told. Only 30% of Australian employees say their organisation has a policy on generative AI. And 48% of those using AI admit to using it in ways that break company policy, including uploading sensitive company information into free tools like ChatGPT.
TIP: Business versions of AI, like Microsoft 365 Copilot, keep your prompts and files inside your Microsoft 365 environment, and Microsoft doesn't use them to train its models. Free tools don't give you that protection.
Doesn't my IT support handle this?
Good email filtering stops a lot. But these scams are built to get past technology and reach a person.
An email sent from a supplier's real mailbox, after it's been hacked, looks genuine to most filters. A cloned voice on a phone call never touches your email system at all.
And no filter decides which AI tools your team should use, or what they can share with them. That's a business decision, and someone has to make it.
Would your team spot it?
If a supplier emailed new bank details today, would your team call them on a number you already have before paying? If "you" rang your bookkeeper asking for an urgent transfer, would they check? Do you know which AI tools your staff use, and what goes into them?
If any answer is "not sure", start with these three steps:
Call to confirm. Check every bank detail change and urgent payment request by calling a number you already hold. Never use the number in the email.
Turn on multi-factor authentication for email, banking and your accounting software.
Write down your AI rules. List which AI tools your team can use, and what must never go into them: client details, payroll data and passwords.
AI hasn't changed what keeps a business safe. It has made skipping the basics riskier.
If you'd like to see how your team handles a realistic AI-written scam, our cyber awareness training runs safe phishing simulations to find out. Or for help setting sensible rules for AI, book a call with our team.
Written by James Bartrop,
CEO of ShadowSafe
Further reading: why your business needs an IT policy
