Traditional penetration testing examines software behavior, access controls, data flows, and infrastructure under hostile conditions. Artificial intelligence systems add another layer because model output can influence application logic, retrieve private content, and trigger external actions.

A tester must examine the model and every connected component, including prompts, data sources, tools, interfaces, and monitoring. Because of this expanded reach, the attack vectors and proof required to produce a valuable security report are evolving.

The Testing Scope Extends Beyond the Model

A conventional assessment often centers on application entry points, such as login pages, APIs, file uploads, and administrative functions. AI security testing examines those areas alongside model instructions, retrieved content, generated output, and connected functions.

The tester traces data across trust boundaries to determine whether a model response can expose information, alter records, or trigger an unauthorized action.

A jailbreak test asks whether a user can bypass the model’s behavioral controls. That question matters, but it does not show what the surrounding application permits after the model generates a response. A harmless-looking answer can still create risk when the application renders it as markup, stores it in a database, sends it by email, or passes it to another service.

Prompt Injection is One Attack Path

When predetermined instructions override or alter the behavior of a model, this is called prompt injection. Direct attacks come from a user’s message. Indirect attacks appear inside content the system retrieves, such as a document, support ticket, web page, or uploaded file.

Testing should determine whether an attacker can extract hidden instructions, expose sensitive context, or make the model ignore business rules. It should also examine whether retrieved content can change the model’s task. A system that treats external text as trusted instructions gives attackers a path into the application’s decision process.

Output Handling Creates Application Risks

Model output becomes dangerous when developers treat it as safe data. A response can contain unsafe markup, a malicious link, an injected command, or values that alter a downstream request. The risk depends on where the output goes and how the receiving component interprets it.

A useful assessment tests each destination separately. A response displayed in a browser requires different checks than one exported to a PDF, sent to a team messaging channel, stored in a database, or submitted to an external function. The tester also checks whether repeated prompts can generate excessive model calls and cause cost-based denial of service.

Retrieval and Access Controls Need Review

Retrieval-augmented systems search documents or records before generating a response. Traditional access control still applies, but the retrieval process introduces additional failure points. A tester checks whether search results cross tenant boundaries, ignore document permissions, expose deleted content, or reveal metadata through citations and summaries.

The assessment should compare a user’s authorized records with the context retrieved by the system. It should also test whether prompt instructions can force broader searches or expose content through indirect questions. Secure retrieval requires permission checks before content reaches the model, rather than relying on it to hide restricted information.

Connected Tools Change the Impact

An AI feature that can call tools has authority beyond conversation. Those tools might update records, send messages, create tickets, run searches, or perform transactions. The security question becomes whether the model can select an unsafe function, provide untrusted arguments, or act without the user’s required approval.

Testers should map every function, its permissions, and its input validation. They should try unauthorized actions, parameter manipulation, repeated execution, and action chaining. A low-risk prompt flaw becomes a serious vulnerability when it reaches a privileged tool without an independent authorization check.

How the Method Differs From a Traditional Test

Traditional penetration testing usually follows an application’s routes, roles, and technical controls. AI testing adds adversarial conversations, poisoned documents, context manipulation, output analysis, and tool-use scenarios. Testers must preserve the exact prompts, retrieved content, model responses, and downstream requests used for each finding.

The report also needs more than a vulnerable endpoint and a reproduction step. It should explain the trust boundary, the model’s authority, the affected data, the resulting action, and the failed control. This evidence helps engineering teams separate a model limitation from an application vulnerability.

Conclusion

AI model testing requires a full-system view. Jailbreak attempts belong in the assessment, but testers also need to inspect retrieval permissions, output handling, tool authorization, data separation, logging, and cost controls.

The practical next step is to map every model input and output path, mark each trust boundary, and test the highest-impact action first. That approach connects model behavior to measurable application risk and gives development teams findings they can fix.