Designing an AI Governance Plane for Offensive Security

Responsible AI use becomes credible when every consequential action can be traced to current authority, constrained by enforceable boundaries, and supported by evidence.

When an AI suggestion crosses the client’s boundary

A penetration tester is reviewing a client’s online shop. The client has authorized testing of the shop, but its payment system—operated by another company—is outside the agreed scope.

The AI assistant notices a connection to that payment system and suggests checking it for weaknesses. The tester accepts the suggestion, and the testing tool sends a request.

Without an enforced scope check, the workflow can look like this:

Each step may seem like a reasonable continuation of the previous one.

Together, they cross a boundary the client never authorized. Being logged in, having access to a tool, or being able to reach a system does not grant permission to test it.

Where should that progression stop—and what should demonstrate that it stopped?

The proposed governance plane checks authority before a proposed action can reach an execution tool. The check must establish whether the actual system, action, timing, and data use fall within current permissions. A tester’s confirmation cannot supply missing client or third-party authorization.

With those controls, the same workflow should look like this:

The assistant’s proposal remains available for human review, but it cannot expand the engagement scope.

Any extension would require a separate authorization process and a fresh evaluation before execution.

What the Rejection Record Should Show

A rejection record makes the boundary visible. It should explain the decision, identify the authority checked, and distinguish a blocked proposal from an executed action.

For the fictional scenario above, a simplified record could read:

Scenario:               Synthetic online-shop engagement
Requested action:       Test the connected payment system
Authority checked:      Engagement scope v1
Applicable restriction: Payment system explicitly excluded
Decision:               DENY
Executable permission:  None issued
Dispatch status:        Not dispatched

This illustrates the information a rejection record should contain. Demonstrating that dispatch was prevented also requires observations from the enforcement and execution boundaries; the denial record alone is insufficient.

This article presents a reference architecture for governed AI use in offensive security. It draws on ISO/IEC 42001’s organizational management perspective, CREST’s responsible-AI requirements, and its supplementary provisions for AI-enabled penetration testing. The controls and approval gates described here are engineering interpretations. Their effectiveness must be demonstrated through implementation, testing, and operating evidence; the architecture alone does not establish conformity or accreditation.

Two Foundations: Organizational Governance and Testing Authority

The opening example raises a technical question: how do we prevent an AI-assisted workflow from crossing an authorized boundary? Answering it requires decisions about who grants permission, which restrictions apply, and what evidence demonstrates that those restrictions held.

ISO/IEC 42001 and CREST’s AI-enabled penetration-testing requirements bring complementary perspectives to those decisions.

ISO/IEC 42001 establishes the organizational foundation. It specifies requirements for an AI management system: how an organization establishes, implements, maintains, and continually improves its management of AI. That includes organizational context, leadership, planning, support, operation, performance evaluation, and improvement. [ISO/IEC 42001 overview]

For an offensive-security architecture, this leads to questions such as:

• Who is accountable for approving an AI use case?
• How are risks and impacts assessed, treated, and reviewed?
• Who approves the use of data and suppliers?
• What changes require reassessment?
• How are failures investigated and corrective actions verified?

These decisions provide authoritative inputs to the proposed governance plane. The system needs a reliable way to establish which decisions are approved, what they cover, and whether they remain current. It cannot manufacture organizational approval from a configuration setting.

CREST brings the service-delivery perspective. Domain 7 of its Company General Requirements establishes organizational requirements for responsible AI use. Annex B of its Penetration Testing Standard adds supplementary requirements for AI-enabled penetration testing when adopted or required by the relevant accreditation pathway. This annex is distinct from CREST’s accreditation for security testing of AI technologies. [CREST’s explanation of the two areas]

These service expectations translate into concrete control needs: explicit engagement scope, competent human oversight, protected testing artifacts, controlled actions, and validated findings and reports.

In the opening scenario, those needs translate into a simple requirement: discovering a connected payment system must not silently add it to the client’s authorized scope.

The two perspectives meet at the point where organizational permission becomes a specific action.

Our proposal translates applicable requirements into architectural invariants: conditions that must remain true across every relevant path. An action must stay within current authority. Data must stay within approved processing boundaries. A model’s suggestion must not become permission. Missing or unverifiable critical state must stop the affected action.

Each invariant then needs an enforcement point, defined failure behavior, an accountable owner, and evidence that can test the claim. That is the bridge from standards to architecture—and the subject of the next section.

From Requirements to Verifiable Controls

“AI must stay within the authorized testing scope” is a useful requirement. To turn it into architecture, we need to establish where that boundary is enforced, what happens when permission is unclear, and how we demonstrate that an unauthorized action was blocked.

The method has four steps:

Each step makes the requirement more concrete.

We can follow the method using the online-shop example.

Define the Requirement and Its Owner

Testing must remain within the client’s authorized scope. The engagement owner is accountable for establishing that scope from valid client and, where necessary, third-party authorization.

The architecture must consume an authoritative scope record containing the permitted systems, exclusions, testing windows, and relevant restrictions. Neither the AI assistant nor a discovered connection can amend that record.

At this stage, we also record the requirement’s source and applicability. This distinguishes the external requirement, the engineering interpretation, and the organization’s approved implementation decision.

State the Invariant

An invariant is a condition the design must preserve across every relevant execution path. Here, it is:

No testing action may be dispatched unless its actual destination and requested effect are covered by current, verifiable authority.

The words matter. Actual destination means a permitted starting point cannot silently authorize a different system reached through a redirect or subsequent step.

Destination checks must use the identity relevant to the engagement, such as the application, tenant, hostname, or resolved network endpoint. An allowed IP address alone may not distinguish an authorized application from another tenant or service. Redirects and subsequent connections must remain subject to the same scope restrictions.

Current authority means an earlier approval cannot survive expiry, revocation, or a material scope change without re-evaluation.

If the system cannot establish permission, the action must remain blocked.

Enforce the Boundary Before the Effect

The proposed architecture places an authorization check before dispatch. It evaluates the proposed action against the current scope, actor permissions, delegation, timing, and applicable restrictions.

For the payment-system request, the check finds that the destination is excluded and denies dispatch. The execution path must enforce that decision: recording a warning while allowing the tool to proceed would leave the invariant unprotected.

For an allowed action, permission must be bound to the reviewed context. Changing the destination, parameters, artifact, or authority requires re-evaluation. Consequential continuation must also remain within those boundaries.

This creates two architectural responsibilities: decide whether the action is permitted, and enforce that decision wherever the action could occur.

Define the Verification Evidence

A rejection record explains a decision. Demonstrating enforcement also requires evidence that the denied action produced no downstream dispatch.

The verification plan should therefore include:

Evidence should link the requirement, control version, test conditions, observed outcome, and limitations without retaining unnecessary client content.

The same method applies beyond engagement scope:

Each requirement-to-evidence chain needs an accountable owner and a clear distinction between design intent, implemented behavior, and demonstrated effectiveness.

• A diagram establishes the intended boundary,
• implementation must enforce it,
• controlled evidence must demonstrate that it held.

This traceability also supports change review. When a supplier, tool, scope, or policy changes, the organization can identify which invariants, controls, tests, and approvals need reassessment.

The Governance Plane and Its Authority Boundaries

The previous section translated requirements into controls. Those controls now need a place in the architecture—and a clear relationship with the people who hold decision authority.

The governance plane connects approved organizational decisions and engagement permissions to technical enforcement. It must establish whether an action is authorized, constrain its execution, and preserve evidence of the decision and outcome. This is a logical set of controls, with enforcement points across provider routing, tool invocation, credential access, execution environments, and network egress. Every path capable of producing a consequential effect must be covered.

Separate Governance, Application Controls, and Infrastructure

Organizational governance establishes approved AI uses, risk decisions, engagement requirements, and criteria for accepting evidence. The application governance plane translates those decisions into authorization checks, data restrictions, routing, and execution controls. Shared infrastructure supplies identity, hosting, networking, and isolation. Engagement delivery remains responsible for testing decisions, finding validation, and report quality.

These responsibilities may sit with different teams or overlap in a smaller organization. Their boundaries must remain explicit: infrastructure access does not authorize a testing action, and application configuration cannot create client permission.

This separation prevents an infrastructure fact—such as successful login—from being mistaken for permission to act.

Trace the Action from Request to Execution

The proposed flow keeps an AI suggestion separate from an executable action:

Logical control flow. Revocation prevents new dispatch commitments and triggers the approved response for active work. Stopping cannot undo committed effects; uncertain outcomes remain recorded for reconciliation. The coordination mechanism and enforcement boundaries require implementation and verification.

Every relevant check must pass. Human review adds professional judgment, but it cannot compensate for missing client permission. An approval must identify the reviewed action: its destination, parameters, artifact version, permitted effects, and applicable limits. These define what may be attempted; they do not guarantee the actual outcome. A recheck before dispatch detects changes since review, but leaves a race if authorization can change before execution is committed. The design must coordinate permission consumption, revocation, and dispatch commitment so that a stale or replayed approval cannot authorize a new action. Permission may have expired, scope may have changed, or a critical control may have become unavailable.

Treat AI Output as Untrusted Input

An assistant may identify a connection, suggest a technique, or draft a plan. Those outputs are proposals for evaluation. An unsafe proposal may result from a model error, a tester’s mistaken assumption, or malicious instructions embedded in retrieved content. None of these sources can grant authority.

A statement such as “this system belongs to the client” cannot establish authorization merely because it appears in model output or a retrieved document. Scope changes must come through the accountable authorization process.

In the opening example, the payment system therefore remains excluded even if the assistant considers testing it useful.

Enforce the Execution Boundary

The executor performs the technical work. It needs an engagement-specific environment, constrained access, permitted destinations, resource and time limits, monitoring, and a stop mechanism.

Required execution boundaries. The enforcement diamond represents controls distributed across tool access, credentials, isolation, and network egress. Every consequential path must be covered, and blocked paths must be demonstrated for the implemented configuration.

Placing a gateway in front of an executor does not establish containment by itself. The design must identify every consequential path—including shell access, network calls, plugins, child agents, and credential use—and demonstrate that each is mediated or constrained.

The stop path also needs explicit design. If authority is revoked or isolation fails, the architecture must prevent new actions and contain active work according to the approved recovery procedure. Stopping cannot undo an effect that has already occurred, so uncertain outcomes must remain visible.

Validate Results Before Release

Execution results do not automatically become validated findings or client deliverables.

The governance plane should preserve enough attributable metadata to connect authority, decision, dispatch, and outcome. Sensitive testing artifacts require controlled custody and lifecycle handling.

These boundaries define the reference architecture. Their effectiveness would need to be demonstrated through implementation and verification, including denied, degraded, interrupted, and recovery scenarios.

Five Gates for Authorization and Release

An organization may approve an AI assistant for penetration testing. That decision alone does not authorize it to process every client’s data, use every available tool, or act against every reachable system.

The architecture uses five cumulative gates. Each answers a different question and has an accountable owner. Runtime checks may evaluate approved policy automatically, while the testing activity remains subject to the required human review and control. Permission extends only as far as the conditions supporting it.

The first four gates constrain whether work may proceed. The fifth governs what may be released and when the engagement can be considered complete. Passing an earlier gate does not grant permission at a later one.

Gate 1: Approve the Organizational Use

This establishes the approved use, organizational responsibilities, competence requirements, and applicable legal, privacy, and client constraints. It also requires current risk and impact assessments, treatment decisions, and appropriate residual-risk acceptance.

For our online-shop example, the organization might permit supervised AI assistance during penetration testing under defined conditions. That permission still needs a platform capable of enforcing those conditions.

Gate 2: Validate the Platform Capability

This concerns the actual combination of models, providers, tools, accounts, configuration, and environment. The relevant owners need evidence that authorization, data restrictions, isolation, monitoring, stop mechanisms, and recovery behave as required.

Permission is bounded by the evaluated capability and configuration. Adding a new tool or changing a provider’s data handling can require reassessment before continued use.

Gate 3: Authorize the Engagement

The engagement gate establishes valid client and relevant third-party authority, the systems included and excluded, testing windows, allowed techniques, impact limits, data permissions, and communication arrangements.

In the opening example, this gate permits testing the online shop and excludes the third-party payment system. Organizational approval of AI and platform readiness cannot override that exclusion.

Gate 4: Authorize the Specific Action

The action gate connects the preceding decisions to a concrete request. It checks the tester and workload identities, delegation, current scope, proposed destination, action details, required human review, and the health of critical controls.

Approval must cover the action actually dispatched. A changed destination, parameter, artifact, or expected effect can invalidate the affected decision.

Organization permits supervised AI assistance ...... PASS
Platform capability is validated for the use ....... PASS
Engagement excludes the payment system ............. CURRENT RESTRICTION
AI proposes testing that payment system ............ DENY
                                                      |
                                                      v
                                                  No dispatch

Gate 5: Approve Release and Closeout

Completed execution does not establish that a finding is correct or a report is ready. Findings need competent validation, appropriate peer quality review, and authorized release through an approved delivery path.

Closeout also requires cleanup, credential revocation, retention and deletion handling, and an explicit record of unresolved residue. A successful test run cannot substitute for those decisions.

Keep Authorization Current

These gates cannot be checked once and forgotten. Each decision needs a controlled reference identifying its scope, approving authority, conditions, revision, effective dates, and revocation state.

Before dispatch and consequential continuation, the architecture must verify the applicable decisions. If scope changes, approval expires, a material supplier or configuration change affects the approved boundary, or required authorization or audit evidence becomes unavailable, the affected work must be denied or held. Active work may need suspension and containment.

The same principle applies after a restart or recovery: restoring a record must not revive revoked authority. Where the design uses single-use action permissions to prevent duplicate execution, restoring a consumed permission must not make it reusable.

When Authority Changes or Systems Fail

The opening example involved a clear exclusion: the payment system was outside the authorized scope. Real engagements also encounter uncertainty. A client withdraws permission while work is running. A provider changes its configuration. A connection drops after a testing request has been sent.

The architecture must preserve the same boundaries under these conditions. Uncertainty must not silently become permission.

The appropriate response depends on when the problem occurs:

Permission Is Withdrawn During Execution

Suppose the client asks the tester to stop testing the online shop. A previously approved action may be waiting for dispatch, while another is already running.

The architecture must distinguish those states. Revocation established before dispatch commitment must prevent dispatch. For work already committed or underway, the system must invoke the approved stop or containment procedure, prevent unauthorized continuation, and record what could and could not be stopped.

A stop request cannot undo an effect that has already occurred. Evidence must preserve the timing and observed outcome rather than imply that everything was prevented.

A Provider or Configuration Changes

An approved model route depends on more than a product name. Its account, configuration, processing location, data handling, and other relevant conditions form part of the evaluated boundary.

If a material change affects the approved processing or security boundary, or the system cannot verify that the boundary still holds, the affected route must be held pending reassessment. An outage must not trigger silent switching to another model, account, region, or provider.

Any alternative needs its own validated permission for the purpose and data involved, together with current action authority. Availability pressure cannot supply that permission.

A Critical Control Becomes Unavailable

The authorization service might stop responding. Required audit recording might fail. The executor might lose its ability to demonstrate isolation or receive stop instructions.
For a new consequential action, missing critical control state means no dispatch. Active work requires the approved safe-stop or containment response, appropriate to the action and environment.

A cached decision must not become an indefinite substitute for current authority. Any permitted freshness interval or bounded operating behavior needs explicit justification, approval, and verification.

The Connection Fails After Dispatch

A timeout creates a different problem. The request may never have reached the executor—or it may have completed while its response was lost.

Automatically repeating it could cause a duplicate effect. The proposed architecture therefore distinguishes permission, dispatch commitment, and observed completion. An uncertain outcome remains unresolved until attributable evidence supports reconciliation.

A retry requires an explicit decision under current authority and an approved duplicate-prevention or reconciliation mechanism. “No response received” cannot be treated as “nothing happened.”

Recovery Restores Obsolete State

Restoring a backup can restore old policy, revoked assignments, or an approval already consumed by an earlier action.

Recovery must reconcile restored records with current authoritative state before affected capabilities resume. Revoked permission must remain revoked. Consumed single-use action permissions must remain unusable. Unresolved actions must remain unresolved until their outcomes are established.

If that reconciliation cannot be completed, the affected capability stays stopped.

These behaviors need verification at the points where failures matter: between approval and dispatch, during execution, and across restart and recovery. Tests should establish both that prohibited actions remain blocked and that the evidence accurately describes any effects already committed.

Evidence That Supports Assurance

To assess the payment-system rejection, a reviewer needs to establish which permission applied, why the action was denied, and whether dispatch was prevented. That review should not require a copy of the client’s application, the tester’s complete conversation with the assistant, or sensitive vulnerability details.

The architecture separates evidence by purpose and sensitivity. It preserves the connection between authority, decision, action, and outcome while keeping restricted engagement material under controlled custody.

Distinguish Telemetry, Audit, and Evidence

These records answer different questions:

A dashboard showing no errors does not establish that scope enforcement worked. A denial event records a decision, but supporting evidence must also establish that the execution path respected it.

Connect Authority, Decisions, and Outcomes

The proposed evidence chain connects the records needed to assess an action:

Each reference must identify the relevant revision and context. A reviewer should be able to distinguish the permission that existed at the time from a scope record changed later.

For the fictional payment-system request, the chain might establish:

Authority reference:    Synthetic scope v1 excludes payment system
Decision:               DENY — destination outside authorized scope
Executable permission:  None issued
Dispatch:               Prevented
Supporting evidence:    No dispatch observed at the monitored enforcement
                        and execution boundaries during the defined test

The verification record must identify the tested configuration, monitored boundaries, alternative paths assessed, and limitations. A component’s assertion that it blocked a request is insufficient if another path could still dispatch it.

Restrict Access to Engagement Evidence

Prompts, attachments, screenshots, tool output, and draft findings can expose credentials, personal information, application internals, or exploitable weaknesses. Collecting them indiscriminately creates another sensitive dataset to protect.

The architecture therefore proposes minimized decision metadata for routine audit, with purpose-limited references to restricted source evidence:

Content capture requires an approved purpose, classification, access rules, retention, and deletion arrangements. Identifiers and hashes also need assessment: they can reveal sensitive relationships, and a hash alone establishes neither authority nor the truth of the underlying claim.

Validate AI-Assisted Findings

An AI-generated explanation is an input to professional review. It does not establish that a vulnerability exists, that its severity is correct, or that the claimed impact was demonstrated.

Evidence should preserve that distinction. A finding needs support for its affected system, conditions, reproducibility, impact, and limitations. Where validation is incomplete, that uncertainty must remain visible.

The release decision must also account for approved AI-use disclosures and client restrictions. Internal permission to process an artifact does not automatically permit including it in a report or exporting it elsewhere.

Maintain Evidence Controls Through Closeout

Evidence handling continues after testing ends. The proposed lifecycle covers temporary files, caches, retrieval indexes, exports, backups, and supplier-held copies, as applicable.
Closeout needs attributable confirmation of cleanup, access revocation, retention or authorized holds, deletion, and unresolved residue. A deletion request sent to a supplier must remain distinguishable from verified completion.

Custody and integrity protections also need a stated trust boundary. Records held under one administrator’s control do not, by themselves, demonstrate resistance to compromise of that administrator. Assurance claims must reflect the protections actually implemented and tested.

Evidence should be sufficient for a defined claim, proportionate to its sensitivity, and explicit about its limitations. Reviewers should be able to assess whether a boundary held while accessing only the engagement material their role and purpose require.

Start with One Workflow

The practical starting point is one consequential AI-assisted workflow: a request that could send client data to a provider, invoke a testing tool, or affect a target system.

Choose that workflow and trace it from the human request to the possible effect. Then answer six questions:

1. Who owns the decision? Identify who may authorize the use, approve the engagement scope, and review the proposed action.
2. What permission applies? Establish the approved purpose, systems, techniques, data use, and time limits.
3. Where is the boundary enforced? Identify the point that can prevent dispatch, including alternative paths through tools, plugins, or direct access.
4. What must cause a stop? Define the response to missing permission, changed scope, revoked approval, unavailable controls, and uncertain outcomes.
5. How will you demonstrate enforcement? Specify positive, denied, degraded, and recovery scenarios with observable results.
6. What evidence is necessary? Preserve attributable decisions and outcomes under appropriate access and retention controls.

For the online-shop example, this produces a concrete first verification objective:
When the assistant proposes testing the excluded payment system, the control denies the request, no testing request reaches that system, and the evidence explains which scope restriction caused the denial.

Writing that objective exposes the architecture you need: an authoritative scope source, a decision point, an enforceable execution boundary, and evidence connecting them. It also exposes unresolved responsibilities before they become operational gaps.

The same method applies across AI-assisted offensive-security workflows. Each applicable requirement becomes an invariant, each invariant receives an enforcement mechanism, and each claimed control needs evidence.

The question to take into your next architecture review is:

Can we demonstrate that this AI-assisted action stays within current authority—even when permissions change or systems fail?

Developed as part of SPARK42’s work on governed AI-assisted offensive security, this reference architecture is shared to support implementation, testing, and discussion across the industry.