Prompt Injection, Deepfakes, and Model Theft: A Business Leader's Guide to AI Cybersecurity in 2026

The New Threat Landscape: AI as Both Weapon and Target

AI is no longer just a productivity tool. It is now an attack vector and an attack target simultaneously. The Australian Signals Directorate's ACSC released joint guidance in 2025 urging organizations to adopt agentic AI "carefully," noting that while it enables powerful automation, it "introduces significant security risks" that traditional security controls were not designed to address. For business leaders, this means three things: your existing security stack has gaps, your adversaries are already using AI, and your AI systems are themselves vulnerable.

1. AI-Powered Phishing and Deepfake Social Engineering

The most immediate AI threat to most organizations is not a zero-day exploit — it is social engineering supercharged by generative AI.

In early 2024, the Hong Kong branch of Arup, a global engineering firm, lost approximately US$25.6 million when a finance employee was deceived by a deepfake video conference. The call featured digitally recreated versions of the company's CFO and other executives, all generated in real time. The employee, believing the call was legitimate, authorized 15 wire transfers to five separate bank accounts.

This is not a hypothetical risk. It has already happened, and the cost was significant. What makes AI-powered phishing different from traditional phishing:

• Scale and personalization: Large language models can scrape a target's LinkedIn, social media, and published content to generate highly convincing, individually tailored phishing emails at a rate of thousands per hour.
• Deepfake voice and video: Tools can clone a person's voice from as little as three seconds of audio. Commercially available services like ElevenLabs, while legitimate, demonstrate how accessible this technology has become.
• Reduced language barriers: Previously, phishing emails were often identifiable by poor grammar or unnatural phrasing — especially in non-native language attacks. AI eliminates these telltale signs.

What to do about it: Implement out-of-band verification for any financial transaction or credential change request, regardless of how authentic the communication appears. Train staff specifically on deepfake awareness — not just traditional phishing. Consider deepfake detection tools such as Reality Defender or Pindrop for voice-based verification channels. Budget approximately US$15,000–$50,000 annually for enterprise deepfake detection depending on your organization's size and risk profile.

2. Prompt Injection and AI Agent Security

Prompt injection is the single most critical vulnerability in AI systems today. OWASP ranked it as the number one risk in their Top 10 for Large Language Model Applications.

A prompt injection attack works by embedding hidden instructions in content that an AI system processes — a document, a web page, an email, or even a image. When the AI reads that content, it follows the injected instructions instead of (or in addition to) its original programming. This can cause the AI to:

• Exfiltrate sensitive data from its context window
• Bypass safety guardrails and content filters
• Execute unauthorized actions if the AI agent has tool access (e.g., sending emails, modifying files, making API calls)
• Provide manipulated outputs that mislead decision-makers

The risk is amplified enormously with agentic AI — systems that can autonomously take actions rather than just generate text. If an AI agent has access to your CRM, email, file system, or financial tools, a successful prompt injection doesn't just produce a bad answer. It can trigger real-world actions with real-world consequences.

The ACSC's 2025 guidance on agentic AI adoption specifically calls out this risk, recommending that organizations implement strict boundaries on what autonomous AI agents can access and execute.

What to do about it: Never give AI agents unrestricted access to sensitive systems. Implement the principle of least privilege — agents should have the minimum permissions necessary for their task. Use input sanitization and output filtering. Deploy guardrail frameworks such as NVIDIA's NeMo Guardrails or Llama Guard to detect and block injection attempts. Test your AI systems with adversarial prompt testing tools like Garak (developed by NVIDIA) before deployment. Budget US$5,000–$20,000 for initial AI red-teaming depending on system complexity.

3. Model Theft and Intellectual Property Risks

Your AI models — whether custom-trained or fine-tuned — represent significant intellectual property and competitive advantage. They are also targets.

Model theft can occur through several vectors:

• API extraction: Attackers make repeated queries to your model's API to reconstruct its behavior, effectively cloning its capabilities without stealing the weights directly. Research has shown this is feasible with thousands of carefully crafted queries.
• Insider threats: Employees or contractors with access to model weights, training data, or architecture details can exfiltrate them.
• Supply chain compromise: Dependencies in ML pipelines (Python packages, pre-trained components, model hubs) can be trojanized to steal models or data.

The financial impact extends beyond the development cost. A stolen model means stolen competitive advantage, potential regulatory exposure if training data contained personal information, and reputational damage.

What to do about it: Implement rate limiting and query monitoring on all model-serving APIs to detect extraction patterns. Use watermarking techniques to prove ownership if a model is stolen. Apply the same access controls to ML infrastructure as you would to any crown-jewel intellectual property — encryption at rest, audit logging, role-based access.

4. Governance Frameworks: What Businesses Need

Technology alone will not solve this. The ACSC, NIST, and OWASP all emphasize that AI security requires governance — policies, accountability structures, and risk management processes.

The NIST AI Risk Management Framework (AI RMF 1.0, updated in 2025 as NIST AI 100-2e2025) provides a practical structure organized around four functions: Govern, Map, Measure, and Manage. This is not a compliance checkbox — it is a framework for making informed decisions about AI risk.

At minimum, your organization should:

• Establish an AI security policy defining acceptable use, approval processes, and incident response procedures for AI-related breaches
• Maintain an inventory of all AI systems in use, including third-party and shadow AI tools
• Conduct AI-specific risk assessments before deploying new AI capabilities, particularly agentic systems
• Assign clear ownership — someone in your organization should be accountable for AI security, whether that is a CISO, a dedicated AI security lead, or a committee
• Review vendor security for any third-party AI services, including data handling, model training on your inputs, and breach notification terms

The cost of establishing a basic AI governance program is modest compared to the cost of an incident — expect US$20,000–$80,000 for initial framework development, policy creation, and staff training, scaling with organization size.

FAQ

Conclusion

AI is reshaping cybersecurity faster than most organizations are adapting. The threats are real, documented, and already causing significant financial losses. But the response does not have to be overwhelming. Start with an AI system inventory, implement out-of-band transaction verification, deploy guardrails on any AI agent with tool access, and establish a governance framework aligned with NIST or ACSC guidance. The cost of prevention is a fraction of the cost of a single incident.

If your organization is deploying AI — or if your employees are using AI tools without oversight — you need a security assessment. Visit consult.lil.business for a free cybersecurity assessment tailored to your AI adoption stage. We will identify your gaps, prioritize your risks, and give you a clear roadmap for secure AI deployment.

References

1. Australian Signals Directorate ACSC — Guidance for Adoption of Agentic AI Services
2. Australian Signals Directorate ACSC — Using AI to Strengthen Cyber Defence
3. NIST AI Risk Management Framework (AI 100-2e2025)
4. OWASP Top 10 for Large Language Model Applications
5. Australian Signals Directorate ACSC — Frontier AI Models and Their Impact on Cyber Security