Augustus: LLM Jail Breaking

Introducing Augustus

Praetorian Guard Platform now includes Augustus, a powerful LLM vulnerability scanner designed to help security teams understand and secure their AI attack surface. This new capability brings enterprise-grade security testing to artificial intelligence systems, enabling penetration testers and security professionals to discover vulnerabilities before attackers do.

Augustus works seamlessly with Julius, our LLM fingerprinting tool, to provide complete AI security coverage. Once Julius identifies which AI services are running across your environment, Augustus steps in to test those systems for security weaknesses. This integrated approach means you can discover, assess, and remediate AI vulnerabilities in a single unified workflow.

Comprehensive Attack Coverage

The scanner includes comprehensive coverage of modern LLM attack techniques, with over 46 specialized security probes covering everything from classic jailbreak attempts to sophisticated adversarial attacks. Augustus understands the full spectrum of AI security risks, including prompt injection vulnerabilities that could allow attackers to manipulate AI behavior, training data extraction attempts that could expose sensitive information, and encoding evasion techniques that try to slip malicious content past security filters.

Augustus employs 28 distinct detection strategies to identify successful attacks, including advanced techniques like LLM-as-judge evaluation where one AI system helps evaluate the security of another. The scanner also includes five buff transformations for payload obfuscation, allowing security testers to see if their defenses can catch attacks that have been encoded or disguised. This depth of testing capability ensures that vulnerabilities won't slip through the cracks.

Broad Platform Compatibility

What makes Augustus particularly powerful is its broad compatibility. The tool works with 19 different LLM providers right out of the box, including major platforms like OpenAI, Anthropic Claude, Azure OpenAI, AWS Bedrock, Google Vertex AI, Cohere, Mistral, and Ollama. Whether your organization runs AI services in the cloud or on-premises, Augustus can test them. This provider-agnostic approach means security teams can standardize their AI testing methodology across their entire technology stack, rather than learning different tools for different platforms.

MCP Server Security Testing

Model Context Protocol (MCP) servers introduce a distinct attack surface — tool poisoning, injection, broken object-level authorization (BOLA), and identity abuse — that traditional scanners do not cover. Guard integrates the Augustus MCP security testing suite end-to-end: Julius fingerprints MCP endpoints, Guard invokes Augustus with the appropriate configuration, probes run, and risks are created as well-formed findings with supporting evidence.

MCP server fingerprinting and automatic scan routing

Julius includes an mcp-server probe that identifies Model Context Protocol servers (JSON-RPC 2.0 over the Streamable HTTP transport, spec 2025-06-18). When Julius detects an MCP server during surface discovery, it sets Type = "mcp" on the asset, distinguishing it from generic LLM endpoints. Augustus picks up these typed MCP assets and automatically routes them into the MCP scan path — no additional configuration is required. The Augustus MCP generator configuration emitted by Julius is forwarded correctly, preserving probe scope throughout the pipeline.

This completes the end-to-end Julius-to-Augustus MCP pipeline: discovered MCP servers are added to your attack graph and become eligible for Augustus security testing immediately upon detection.

Automatic MCP routing in agentic workflows

MCP server routing now applies to both the deterministic and agentic execution paths. When an orchestrator agent discovers an endpoint fingerprinted as an MCP server during a hunt, it is automatically handed to Augustus for security testing — the same way it would be on the deterministic path. Augustus selects MCP-specific probes (mcptool.*, mcptransport.*) automatically; no additional configuration is required.

Previously, the agent orchestration layer did not recognize MCP targets, so agentic hunts left MCP servers untested. MCP servers discovered during agentic workflows are now covered without any change to hunt configuration.

Agent-driven MCP probing

The Augustus agent can now invoke MCP server probes directly. When the agent detects an MCP endpoint, it passes generator_type: mcp to target that endpoint without requiring manual scan configuration. Combined with auto-generated scan configs emitted by Vespasian's MCP probe on server detection, the full probe lifecycle — from discovery through active testing — requires no manual setup.

Guard credential passthrough

Credentials attached to an asset or integration in Guard are automatically looked up and forwarded to MCP and HTTP probes at scan time. Authenticated tests use the stored credentials without additional configuration, and probes that require authentication will not produce misleading clean results due to missing credentials.

Full JSON Schema traversal for MCP tool probing

Augustus now traverses the complete input schema tree when building probe arguments for MCP tools. Previously, only top-level properties were read, which caused Augustus to silently undercount coverage for servers that use nested schemas, discriminators, or JSON Schema composition keywords. As a result, some tools appeared covered in scan reports while actually producing schema-validation errors that prevented meaningful probing.

Augustus now resolves the following constructs at any depth before generating probe arguments:

• Nested properties
• $ref references
• allOf, anyOf, and oneOf composition keywords
• if/then conditional constructs

This change ensures that every tool reported as covered has been exercised with structurally valid arguments. Scan results on servers with complex tool schemas will reflect the actual tested surface rather than an artificially inflated one.

Active MCP security testing across the OWASP MCP Top 10

Augustus actively probes identified MCP endpoints for vulnerabilities across the OWASP MCP Top 10 attack classes. Fingerprint data collected by Julius flows directly into Augustus test configuration, so discovery and exploitation operate as a single coordinated workflow. MCP servers are treated as a distinct surface from LLM inference endpoints, and the probe families are scoped accordingly.

Augustus runs toolsec.* and mcptransport.* probe families against discovered MCP servers, covering:

• Tool poisoning
• Prompt injection via tool responses
• Broken object-level authorization (BOLA)
• Transport-level issues

Each probe emits a finding with proving evidence attached.

MCP authentication and authorisation probing (MCP07)

Augustus includes dedicated probes for authentication and authorisation weaknesses at the MCP layer. These probes detect:

• Unauthenticated access at the HTTP transport layer — servers that accept MCP requests without requiring valid credentials.
• Missing or bypassable authorisation on tool calls — tool endpoints that do not enforce access controls, allowing callers to invoke tools they should not have access to.
• Privilege-escalation paths across MCP sessions — conditions where a lower-privileged session can be used to exercise capabilities intended for higher-privileged callers.

These checks map to MCP07 in the OWASP MCP Top 10 and complement the existing transport and tool-surface probe families. No additional configuration is required; auth probes run automatically when an MCP asset is tested.

Credential leakage across all MCP surfaces

Secret-detection coverage now extends to all MCP response surfaces. Previously, credential leakage detection was limited to tool responses. A server advertising plaintext credentials in an MCP resource or prompt template would not produce a finding. Augustus now scans:

• MCP resources
• Prompt templates
• All other non-tool response surfaces

If any of these surfaces expose plaintext credentials or other secrets, Augustus raises a finding with evidence attached. No change to scan configuration is required.

MCP reconnaissance panel

Discovered MCP servers surface a dedicated LLM-Recon panel in the Guard UI. The panel shows:

• Server identity information
• Tool, resource, and prompt catalog
• Identity reconnaissance results

Per-asset credentials and model pre-flight

Credentials are scoped per asset, and model availability is verified before probes execute. This prevents silent misconfiguration from producing a clean result when the underlying target or model is unreachable.

PDF Attack Probe Coverage

Augustus includes multimodal document attack probes (pdf.*) that send adversarial content embedded in PDF payloads to the target model. The following provider support is available:

• Anthropic — supported since initial release.
• Google Gemini and Vertex AI — supported as of the Augustus PDF transport update. Native document content blocks are now wired through both Google generators, so pdf.* probes execute against Google-hosted models without silently dropping document payloads.

If you are assessing AI systems built on Google's model family, no additional configuration is required — PDF probes will run automatically when a Gemini or Vertex AI target is configured in your scan.

MCP Security Write-Ups

MCP security findings from Augustus carry structured, self-contained security write-ups directly on each probe. Security teams no longer need to consult external references to understand what was found, why it matters, and how to address it.

Each MCP probe finding includes:

• Vulnerability description and impact — a curated write-up explaining the security issue and its potential consequences.
• Remediation guidance — actionable fix guidance included directly in the finding.
• CWE taxonomy and references — findings are tagged with relevant CWE identifiers and supporting references.
• CVSS v4.0 scoring — a conservatively-scored CVSS v4.0 vector for consistent severity assessment.
• Verification write-up — a dedicated, reproduction-oriented field that describes exactly how the probe confirmed the vulnerability and the steps needed to reproduce it manually. This field is separate from the finding description and provides a self-contained path to validate and escalate each result.

Write-ups are implemented across the following MCP probes:

When reviewing MCP scan results in the Praetorian Guard Platform, the write-up for each finding — including the verification and reproduction steps — is available inline. The Verification field in a probe's RiskInfo provides the reproduction-oriented prose needed to triage, validate, and escalate findings without additional research.

Finding Quality and Noise Reduction

Aggregated Origin validation findings

Origin header bypass variants are now reported as a single consolidated finding per server. Previously, each bypass variant produced a separate row, which could result in ten or more findings for a single server — including benign ones — dominating scan output. The aggregated finding retains all relevant evidence while eliminating the noise.

Tool-surface false-negative fixes

Four measured false negatives in the path-traversal, injection, SSRF, and response-leak probes have been resolved. In each case, the probe was reaching its target and receiving a response containing valid evidence, but that evidence was being discarded before a finding was raised. These probes now correctly surface results when evidence is present.

Calibrated Findings

Finding titles, descriptions, and severity justifications are grounded in the evidence collected by each probe. Language is consistent with pentest report standards — impact claims reflect what the probe actually confirmed rather than overstated or speculative conclusions. When reviewing Augustus findings, the stated severity and description correspond directly to the observed behavior.

Probe Error Reporting

Probes that fail before reaching the model — due to an authentication failure, timeout, or transport error — are now reported as errors in scan results rather than being recorded as silent passes. This change prevents a broken scan configuration from appearing as a clean result with a safe score. If you see probe errors in a scan report, review the affected target's credentials and connectivity before treating the scan as complete.

Enterprise-Ready Reporting

For enterprise security workflows, Augustus generates detailed evidence in multiple formats including JSON, HTML, and tabular reports. These reports make it easy to share findings with development teams, track remediation progress, and demonstrate compliance with emerging AI governance frameworks. The automated nature of the scanning means security teams can continuously monitor their AI systems without manual intervention, catching new vulnerabilities as code changes or new attack techniques emerge.

Security and Control

Augustus is designed with security and control as foundational principles. The scanner operates in a read-only, observation-based mode, sending carefully crafted test prompts to your AI systems and analyzing the responses to determine if vulnerabilities exist. Augustus does not modify your AI models, change configurations, or take any autonomous actions against your systems. It simply asks questions and listens to the answers, much like a security researcher would during a manual assessment.

When Augustus runs, it only interacts with the specific AI endpoints you have authorized for testing within Praetorian Guard Platform. The tool cannot reach beyond your defined scope, and all testing activity is logged and auditable within the platform. Your security team maintains complete visibility into what Augustus is testing, when tests occur, and what results are generated. This transparency ensures that AI security testing fits seamlessly into your existing governance and change management processes.

It is worth noting that Augustus itself does not use AI to make decisions about your environment or take remediation actions. While the scanner does employ LLM-as-judge techniques to help evaluate whether certain attack attempts succeeded, this evaluation happens in an isolated context purely for classification purposes. The judgment calls about what to fix and how to prioritize remain entirely with your security team. Augustus provides the evidence and insights; humans make the decisions about what happens next.

Unified AI Security View

Augustus represents a significant step forward in AI security for the Praetorian Guard Platform, bringing our total security scanning capabilities to 87 distinct tools. Security teams can now confidently deploy AI systems knowing they have the same rigorous testing available for artificial intelligence that they have always had for traditional web applications and infrastructure.

You will find Augustus results integrated directly into the Praetorian Guard Platform UI alongside Julius fingerprinting data, making it easy to see which AI services are exposed and which ones have security concerns requiring attention. This unified view helps security teams prioritize their remediation efforts and maintain a comprehensive picture of their AI attack surface.