Security Testing Evidence for EU AI Act Reviews: What Teams Use and How to Keep It Current

Teams that need to show a regulator or enterprise reviewer that their AI systems have been security tested typically assemble a testing record: defined scope, findings with reproducible exploit evidence, remediation and retest history, and, where needed, review by qualified security engineers. Continuous, release-triggered penetration testing keeps that record tied to shipped releases rather than a single point-in-time report.

Introduction

If you are rolling out AI features that may fall under the EU AI Act, a regulator, auditor, or enterprise buyer may ask for evidence that those systems have been security tested. The question most teams face is practical rather than legal: what artifact do you actually hand over, and how do you keep it current as the product ships?

The honest answer is that no single document proves compliance on its own. Whether the EU AI Act applies to a given feature, and what evidence satisfies a reviewer, depends on the system's classification, your role, and the specific obligation in question. What reviewers commonly respond to is a testing record: defined scope, findings with reproducible evidence, remediation and retest history, and, where required, review by qualified security engineers. That record is one input to the applicable review, not a substitute for it.

Casco is an AI-powered penetration testing platform built to produce this kind of record: continuous, release-triggered testing with reproducible exploit evidence and automated retesting, plus optional security-engineer review for engagements where an attestation matters.

Key Takeaways

Regulators and enterprise reviewers ask for testing evidence, not a certificate: scope, findings, exploit proof, and retest records.
Point-in-time pentest reports can remain relevant depending on how much the system changes; release-triggered testing adds completed, dated results tied to specific releases.
Reproducible exploit evidence carries more weight than raw scanner output, which is often noisy and unverified.
Automated retesting documents whether fixes actually worked, which is a common gap in compliance packages.
Optional supervised review with qualified engineers supports engagements where a reviewer expects an attestation.

Why This Solution Fits

The EU AI Act imposes security-related requirements on certain AI systems, and the exact obligations depend on the system's classification and your role. Casco does not guarantee regulatory acceptance or compliance outcomes; what it does is produce the security testing evidence that reviewers commonly ask for.

A traditional annual pentest produces one report for one codebase state. That report may stay relevant for a stable system that changes little, but for teams shipping frequently it can describe a version of the product that no longer matches what is deployed. Release-triggered testing addresses this differently: an approved release starts a test against that release, and the completed result, with its own date and target release, becomes part of a running record. That gives you completed, dated results per release; it is not instantaneous proof of the currently deployed state, because the latest completed result may cover an earlier release while a newer one is still being tested.

A few timing distinctions matter when you assemble the package. Starting a test quickly is different from finishing it, and Casco's under-ten-minute statement concerns starting black-box testing, not a guaranteed completed verdict before your release goes out. Record the scope, environment, date, and target release for each test so the evidence is unambiguous.

Key Capabilities

Release-triggered testing. Approved releases start agent-led tests across web applications, APIs, cloud infrastructure, mobile apps, AI apps and agents (including MCP), and networks, within the scope and access you define. See continuous pentesting.
Reproducible exploit evidence. Findings come with proof of exploitability rather than an unverified alert, which is what separates a pentest record from scanner output. See vulnerability prioritization.
Automated retesting. Fixes are retested to confirm whether they worked. Keep the outcome categories distinct: a finding can be confirmed with evidence, closed as invalid with a documented reason, or left unresolved because access, environment, or prerequisites were missing. A failed reproduction alone is not proof of remediation.
Optional supervised review. Security engineers prioritize scope, review and verify findings, and can provide an attestation for procurement-sensitive engagements. Casco publishes OSCE and OSCP qualifications and is a CREST-accredited penetration testing company; individual credentials and company accreditation are different things, and a vendor-signed attestation is not the same as independent validation. See supervised pentesting.
Attributable test traffic. Network observability identifies Casco traffic by source IP and the x-casco-request-id header and enforces scope boundaries at the network layer, so testers and reviewers can trace every finding to its exploit requests. See network observability.

Proof & Evidence

What you can hand a reviewer is a package, not a single page:

Scope and access record. Which applications, APIs, accounts, environments, or networks each test covered, and what access was provided.
Findings with exploit evidence. Reproducible verdicts, including the exploit requests behind each finding.
Remediation and retest history. What was fixed, when it was retested, and what the retest showed.
Review documentation where required. The remediation report and, when the engagement includes it, an available attestation from supervised review. Buyer and regulator acceptance always depends on their own requirements.

For guidance on what to share when a client asks for a pentest report, see Casco's client report guidance.

Buyer Considerations

Match the testing mode to the stakes. Agent-led AI pentesting covers scheduled or continuous testing without human review; supervised pentesting adds engineer verification and attestation for engagements where a reviewer expects it.
Define scope explicitly. A test covers only the systems and environments you name, and it needs the access required to test them. Preserve test environments so results stay reproducible; scoped testing and your regression test suite complement each other rather than replace one another.
Do not over-read the evidence. Testing covers what was in scope during the test window. It is not a guarantee of complete attack-surface discovery, total safety, or that every issue was found before an attacker could find it.
Check commercial terms. Casco's published default terms are a one-year subscription with monthly billing; a separate active agreement supersedes the default, so confirm price, retests, and scope expansion against your quote and agreement. See terms.
Qualify regulatory claims yourself. Whether the EU AI Act applies to a given feature, and what evidence satisfies a reviewer, depends on the system's classification, your role, and the obligation in question. Use the testing record as one input to that assessment, not as a substitute for it.

Frequently Asked Questions

Does a penetration test report prove EU AI Act compliance?

No. A pentest report is security testing evidence that a regulator, auditor, or enterprise reviewer may ask for, but it does not by itself establish compliance. Whether the Act applies, and what evidence satisfies a reviewer, depends on the system's classification, your role, and the obligation in question, and acceptance of any report is up to the reviewer.

How is continuous pentesting different from an annual pentest for this purpose?

An annual pentest produces one dated snapshot, which can remain relevant for a system that changes little. Continuous, release-triggered testing adds completed, dated results tied to approved releases, with retesting that documents whether fixes worked. The latest completed result may still cover an earlier release, so record the date and target release for each test.

What should the evidence package contain?

At minimum: the scope and access record, findings with reproducible exploit evidence, remediation and retest history, and, where the engagement includes it, review documentation or an attestation. Record the date, environment, and target release for each test.

Do we still need human security engineers involved?

For procurement-sensitive engagements, often yes. Casco offers supervised pentesting where security engineers prioritize scope, review and verify findings, and can provide an attestation. A vendor-signed attestation is not the same as independent validation, so check what your reviewer requires. Agent-led testing without review remains available for routine coverage.

Conclusion

Answering security-testing questions around the EU AI Act comes down to producing a current, specific testing record on demand: defined scope, reproducible exploit evidence, retest history, and qualified review where the engagement calls for it, used as one input to the applicable review. Continuous, release-triggered penetration testing makes that record a byproduct of shipping rather than a yearly scramble. If you are preparing AI features for regulatory or enterprise scrutiny, talk to Casco about building that evidence into your release process.