AI Security
Testing the AI you build and connect
When you build AI into a product or connect an agent to your systems, it opens attack routes a standard penetration test does not look for: prompt injection, data leakage, insecure tool use and agents doing more than they should. We scope and oversee testing of the AI layer specifically, delivered through CREST accredited partners, so a feature you shipped does not become the way in.
Definition
What is AI penetration testing?
AI penetration testing is security testing aimed at the AI in your systems: the models, the applications built on them, and the agents connected to your tools and data. It looks for failures that standard tests miss, such as prompt injection, data leakage and agents being steered into actions they should not take.
When you build AI into a product or connect an agent to your systems, it opens attack routes a standard penetration test does not look for. We test the AI layer specifically, so a feature you shipped does not become the way in.
Why AI
Why AI needs its own testing.
A traditional penetration test checks networks, applications and configuration. It does not probe whether a model can be talked into ignoring its instructions, whether a prompt can pull back data it should not, or whether an agent can be steered into misusing the tools it is connected to. Those are AI-specific failures, and they need AI-specific testing.
The risk grows the moment a model is given access to anything: your documents through retrieval, your systems through tools and APIs, or the ability to act through an agent. A chatbot that can only chat is low risk. An assistant that can read your data and take actions on your behalf is a different proposition, and it needs testing to match.
What we test
What we test.
Testing is carried out independently, through CREST accredited partners that we select and oversee, with Threat Protect as your single point of accountability. The findings feed straight into how you govern AI across the organisation.
- LLM applications, against the OWASP Top 10 for LLM applications.
- AI agents and their access to tools, APIs and MCP servers.
- Retrieval and data-leakage paths, including RAG pipelines.
- Guardrail and jailbreak resistance, through structured AI red teaming.
The failures
The failures AI testing looks for.
The OWASP Top 10 for LLM applications is the baseline, and a few failures come up again and again in real systems.
Prompt injection
Instructions hidden in user input, a web page or a document that the model follows instead of your own, overriding its guardrails or redirecting what it does. Indirect prompt injection, hidden in content the model retrieves, is the version most teams miss.
Data and knowledge leakage
The model returning data it can technically reach but should not reveal: other users’ information, system prompts, secrets, or content pulled back through a retrieval pipeline that was never scoped to the person asking.
Insecure output handling
What your application does with the model’s output. If the response is trusted and passed to a browser, a database or a shell without checks, the model becomes a route to the classic web and injection flaws.
Excessive agency and tool misuse
Agents and AI features given more permissions than they need, so a successful manipulation lets the model do more than read: send mail, change records, move money, or call tools in ways you did not intend.
Red teaming
AI red teaming.
AI red teaming is structured adversarial testing of an AI system: deliberately trying to make it misbehave, leak data, or bypass its guardrails, to find the weaknesses before an attacker does. It complements penetration testing by focusing on how the model itself can be manipulated, rather than only the application around it.
It is most useful once the obvious issues are fixed. If a test keeps returning the same findings, closing those comes first; red teaming answers the harder question of whether a determined adversary can still get the model to do something it should not.
Penetration testing
How this relates to our penetration testing.
This is the AI-specific layer on top of our standard testing, not a replacement for it. If you need your network, applications and infrastructure tested as well, that is our main penetration testing service, and the two are usually scoped together so nothing falls between them.
Related
When you need it
When you need AI testing.
If you have shipped or connected AI, it is part of your attack surface and should be tested like any other application, with the AI-specific failures added on top.
- A customer-facing chatbot or assistant.
- An internal copilot connected to your documents or systems.
- An agent that can take actions through tools, APIs or MCP servers.
- A RAG pipeline that retrieves from your own data.
- Any AI feature that handles sensitive input or acts on its output.
FAQ
Questions buyers actually ask
What is AI penetration testing?
It is security testing aimed at the AI in your systems: the models, the applications built on them, and the agents connected to your tools and data. It looks for failures that standard tests miss, such as prompt injection, data leakage and agents being steered into actions they should not take.What is LLM penetration testing?
It is penetration testing focused on applications built on large language models. It probes the model and its surrounding application for issues like prompt injection, insecure output handling, data leakage through retrieval, and weak guardrails, commonly framed around the OWASP Top 10 for LLM applications.What is AI red teaming?
AI red teaming is structured adversarial testing of an AI system: deliberately trying to make it misbehave, leak data, or bypass its guardrails, to find the weaknesses before an attacker does. It complements penetration testing by focusing on how the model itself can be manipulated.Is this different from a normal penetration test?
Yes. A normal penetration test covers networks, applications and configuration. AI testing adds the model-specific and agent-specific risks on top. For AI products we usually scope both together so nothing falls between the two.What is prompt injection?
Prompt injection is when instructions hidden in user input, a web page or a document are followed by the model instead of your own, overriding its guardrails or redirecting what it does. Indirect prompt injection, hidden in content the model retrieves rather than typed by the user, is the version most teams overlook.Do we need AI testing if we only use third-party AI tools?
If you only buy AI rather than build or connect it, most of this testing does not apply, and the priority is governance: an inventory, a policy and supplier assurance. AI testing becomes relevant when you build a product on a model, connect an agent to your systems, or expose an AI feature to users.How does AI testing relate to the OWASP Top 10 for LLM applications?
The OWASP Top 10 for LLM applications is the common baseline for this work. It catalogues the main risk classes, such as prompt injection, insecure output handling, sensitive information disclosure and excessive agency, and gives testing a shared reference point. We test against it and beyond it, depending on how your system is built.
Scope AI and LLM testing.
Tell us what you have built or connected. We will scope AI-specific testing through accredited partners, alongside your standard testing where it helps.
Book a call