The AI Security Problem Australian Businesses Can't Patch Their Way Out Of

Australia's own cybersecurity agency has confirmed something worth sitting with: prompt injection attacks on AI systems cannot be reliably fixed inside the model itself. There is no patch coming for this one.

The Australian Signals Directorate (ASD) published guidance this week on the careful adoption of agentic AI services, concluding that prompt injection is architecturally inherent to how large language models work. The issue is that LLMs process user instructions and external content (documents, emails, web pages) inside the same context window. The model cannot reliably tell the two apart, so malicious instructions hidden inside a document or email can override what the actual user intended. ASD compared this to the early days of phone phreaking, where signaling and voice traveled the same channel, making certain attacks structurally possible regardless of how careful individual users were.

For Australian businesses, this is immediately relevant. Microsoft Copilot for Microsoft 365, ChatGPT Enterprise, and similar tools are now reading emails, files, and shared documents on behalf of staff. A successful prompt injection could cause an AI assistant to exfiltrate data, send messages the user never wrote, or take actions the user never authorised. ASD's recommended response is not to avoid AI tools, but to focus security controls on the "harness," the software layer surrounding the model: apply least-privilege access to what the AI can see and do, require human approval before the AI takes high-impact actions, log all AI interactions, and delete rather than summarise stale context from AI sessions. In practice, many Australian businesses are adopting Copilot through bundled Microsoft licensing via their IT provider, which means they often have limited visibility into what harness controls are in place and what data the AI is currently permitted to touch. That is the conversation worth having now.

Written by Caleb Attard, Technical Consultant, All IT Services. All IT is a Sydney-based managed IT provider supporting businesses across hospitality, not-for-profit, and professional services.