An AI Governance Framework for Engineering Teams (Template)

An AI governance framework connects policies, decision owners, technical controls, and evidence across an AI system's lifecycle. For engineering teams using LLMs, it answers: which data can reach which deployment, who can spend money, what an agent can execute, and how you prove those restrictions work.

This template includes a downloadable checklist, copyable policy, inventory, RACI, risk record, and 13 controls. A control is complete when its required tests pass and the evidence is saved.

Policy states the rule. Controls translate it into requirements: an approved model list, for example. Enforcement checks requirements before data or authority crosses a boundary. Evidence records decisions and tests.

AI governance framework template: policy and checklist

Start with one inspectable workflow. Copy the policy and assign owners. The checklist's Copy and Download as Markdown buttons export the checklist only; other artifacts are copyable below. Tiers and schedules are suggested defaults. Adapt them and obtain qualified legal classification.

# AI governance policy
Policy owner: [role and current assignee]
Approver: [role and current assignee]
Version and effective date: [version, date]
Next review: [date]
 
Scope: Employees, contractors, services, coding assistants, LLM APIs,
retrieval systems, agents, embedded SaaS AI, and tool/MCP integrations
processing company data or acting on company systems.
 
Every production AI workflow requires:
- Inventory, business/technical owners, and backups.
- Internal risk tier, legal classification, and risk record.
- Approved data classes, deployments, accounts, features, and fallbacks.
- Organization-managed credentials and bounded spending/execution.
- Approved tool permissions and pre-execution approval points.
- Versioned evaluations, acceptance criteria, and release decision.
- Tested controls, saved evidence, and rollback or disable path.
 
GOV-01 through GOV-13 form this policy's control library.
Exceptions require owner, justification, compensating controls,
approval, technical scope, and expiry. Expiry removes access or blocks
release until renewed. Exceptions cannot waive law or contracts.
 
Release decision: [approve / reject / expiring exception]
Residual risks and authorized risk acceptor: [description, assignee]
Exception ID and expiry: [if applicable]
Evidence bundle: [internal URL]

Scope the workflow and assign decision owners

Consider a support agent with an approved primary deployment, an unapproved fallback receiving customer tickets, and permission to send replies. Model approval alone misses the fallback, data, and sending authority. The reply also needs evaluation.

Inventory purpose, affected users, data, deployments, provider account, tools, permissions, and approval points. Reconcile procurement, identity grants, repositories, and traffic. For discovery beyond registered workflows, see how to find unsanctioned AI use.

system_id: support-draft-agent
status: pending-review
purpose: Draft support replies for human review
owners: {business: support-director, technical: platform-team, backup: platform-on-call}
internal_risk_tier: tier-2
legal_classification: pending-legal-review
data_classes: [internal]
model_deployments: [bedrock/claude-sonnet-5@eu-central-1]
tool_permissions: [ticket.read, reply.draft]
key_ids: [inventory-key-id]
budget_and_logging_record: approved-controls-entry
location_and_vendor_terms: vendor-register-entry
risk_and_eval_record: support-drafts-v1
policy_and_approval_record: GOV-2026-09-pending
incident_and_retirement_record: support-runbook
next_review: 2026-12-30

This illustrative internal schema is not gateway configuration. Link detailed records; store key IDs, never secrets. Pending reviews block dependent production uses.

Classify by data, impact, and authority

Internal tiers set review depth, not legal classification. Consider data, impact, autonomy, reversibility, and permissions.

Copy the RACI

R performs work; A owns the decision; C is consulted; I is informed. Assign people and backups. Small teams can combine roles while preserving independent consequential-action approval.

The reviewer must have action-approval authority. A model cannot approve itself. Assign one incident lead per incident type, including quality incidents.

activity,engineering_sponsor,system_owner,platform_engineering,security_privacy,legal_compliance,finops,independent_reviewer
Framework and risk appetite,A,C,R,C,C,C,I
Inventory and ownership,I,A/R,C,C,C,I,I
Vendor data/security terms,I,R,C,A,C,I,I
Legal classification,I,R,C,C,A,I,I
Production workflow approval,A,R,C,C,C,C,C
Keys allowlists and regions,I,C,A/R,C,I,I,I
Spending budgets,I,R,C,I,I,A,I
Consequential action approval,I,A,C,C,C,I,R
Security/data incident containment,I,R,R,A,C,I,I
Control sampling and evidence review,I,R,R,C,C,C,A
Retirement and revocation,I,A,R,C,C,I,I

The 13 controls: rules you can enforce and test

Each control specifies owner, rule, enforcement, tests, and evidence. CSV review triggers complete the register. A forbidden operation's failure counts only when you identify the intended denial.

control_id,control,owner,review_trigger,policy,enforcement_point,positive_test,negative_test,evidence_url,last_tested,next_review,status,exception_id,exception_expiry
GOV-01,Inventory,system-owner,New tool/data/scope or monthly reconciliation,,,,,,,,not-tested,,
GOV-02,Roles,engineering-sponsor,Personnel change or quarterly role review,,,,,,,,not-tested,,
GOV-03,Approval,security-privacy,Model/alias/provider/terms/fallback change,,,,,,,,not-tested,,
GOV-04,Data,security-privacy,New data source/purpose/destination,,,,,,,,not-tested,,
GOV-05,Keys,platform,Onboarding/offboarding/privilege change,,,,,,,,not-tested,,
GOV-06,Budgets,finops,Workload/cost change or monthly budget review,,,,,,,,not-tested,,
GOV-07,Logging,security-privacy,Logging/viewer/purpose/retention change,,,,,,,,not-tested,,
GOV-08,Quality,system-owner,Model/prompt/data/tool/user/threat change,,,,,,,,not-tested,,
GOV-09,Actions,system-owner,New action/permission/autonomy,,,,,,,,not-tested,,
GOV-10,Incidents,incident-lead,Incident/failed exercise/dependency change,,,,,,,,not-tested,,
GOV-11,Residency,platform,Region/provider/storage/cache/tool change,,,,,,,,not-tested,,
GOV-12,Changes,system-owner,Material change/exception expiry/scheduled review,,,,,,,,not-tested,,
GOV-13,Audit,independent-reviewer,Monthly metrics/quarterly sampling/failed tests,,,,,,,,not-tested,,

Approve data destinations and vendor terms

Personal data needs privacy review even in documents labeled Internal. Masking one value does not declassify a document.

GDPR minimization and storage limitation require necessary data and purpose-limited retention. Set separate schedules for metadata, approvals, policy/tests, and justified payloads. Exclude sensitive values from tags and traces.

Copy a risk and evaluation record

NIST's Generative AI Profile release identifies confabulation, misinformation, harmful content, and cybersecurity misuse. Choose workflow-specific tests: citation checks, generated-code security tests, or matched-case review for unequal treatment.

Risk ID and workflow: [IDs]
Intended and prohibited use: [scope]
Affected people and harm scenario: [who is exposed, what fails]
Impact and likelihood rationale: [evidence, assumptions, uncertainty]
Required controls and owner: [IDs, enforcement, assignee]
Evaluation set/version and method: [cases, metric or rubric]
Acceptance criterion and rationale: [release requirement]
Observed result and evidence: [result, limitations, internal URL]
Residual risk and authorized acceptor: [exposure, role, assignee]
Release decision: [block / approve / expiring exception]
Exception scope and expiry: [if permitted]

Illustrative support-agent example:

Workflow: Draft policy-grounded replies for staff review.
Risk: Invented refund entitlement misleads customers or creates unauthorized commitments.
Criterion: No unsupported refund commitments in the release set; missing evidence triggers review.
Control: Verify claims/citations against policy sources; require approval before sending.

Monitor coverage limits in production. Failed criteria block release pending remediation or authorized expiring exception. OWASP warns that RAG does not fully mitigate prompt injection, including malicious document/website instructions. Enforce retrieval and action authorization in code.

Separate location from retention

GDPR permits international transfers with applicable safeguards; contracts may require narrower locations. Review gateway, inference, storage, caches, and tools. An EU model does not constrain external tool destinations.

Exception ID and workflow/control/risk IDs: [IDs]
Requester, owner, independent approver: [roles and assignees]
Justification and affected data/actions: [scope]
Risk and residual exposure: [description]
Compensating controls and tests: [implementation, evidence]
Technical scope: [key ID, deployment, tool, permissions]
Start and expiry: [dates]
Monitoring and accountable operator: [checks, assignee]
Revocation: [automated access removal or release block]
Renewal evidence and legal/contract review: [internal URLs]

Map the framework to NIST, ISO 42001, and the EU AI Act

Map one control library to applicable standards and laws.

As of September 2026, NIST is revising AI RMF 1.0. GOVERN is cross-cutting; the Core calls for testing before deployment and regularly in operation. ISO/IEC 42001 supports third-party AIMS certification, which requires management-system assessment, not a gateway purchase or copied policy.

What EU AI Act obligations apply in September 2026?

The Digital Omnibus on AI is enacted. Regulation (EU) 2026/1744 amended deadlines, reflected in the Commission timeline.

High-risk dates concern Chapter III Sections 1, 2, and 3, with the Article 6(5) exception. Check existing-system transitions; Article 111(3) gives the legacy GPAI deadline.

Classification depends on intended purpose. Distinguish GPAI model providers, integrating system providers, and deployers: API consumption does not automatically make you a GPAI provider.

Article 50 assigns disclosure/content-marking duties with exceptions. The amendment retains staff AI-literacy support, without requiring a guaranteed individual level.

Article 26(6) requires purpose-appropriate retention of automatically generated high-risk system logs under deployer control for at least six months, unless applicable law provides otherwise. Apply the high-risk timeline and privacy rules: this is not universal six-month prompt storage.

Enforce model-traffic controls through Requesty

We centralize approved models, key/user/team/org spend caps, guardrails, regional gateways, and evidence exports on the model path. Your application authorizes actions and evaluates outputs.

Product behavior checked September 26, 2026.

Access precedence is key list, applicable group lists, group default, organization Approved Models, then full catalog if unconfigured. Keep overrides within data/vendor policy; test effective permissions, including embeddings, images, and audio.

Enterprise groups default to per-member limits; pooled budgets require support-enabled Group Budget mode. Enterprise provides scoped logs/analytics, audit logs, and SSO; pay-as-you-go members see org-wide logs/analytics.

Guardrails Report forwards unchanged; Mask replaces detected inputs and restores matching response values, without scanning new sensitive output. ZDR stops new payload storage, retaining telemetry; it does not delete historical payloads or set provider retention.

Test the approved and forbidden paths

1. 1
   Create a scoped test key

   Allow one chat deployment; exclude another catalog-supported deployment. Use an EU-restricted test organization, or test an existing approved EU restriction without changing production settings. Record effective policy/version and key ID. Choose deployments from the public catalog; use harmless input.

2. 2
   Run three requests and record responses

   Inject REQUESTY_API_KEY from your secret manager. Replace the excluded-deployment placeholder. The EU deployment below was catalog-verified September 26, 2026; these are proposed tests, not observed results.

   Shell

   Copy

   umask 077
   EVIDENCE_DIR=$(mktemp -d governance-test.XXXXXX)
   ALLOWED="bedrock/claude-sonnet-5@eu-central-1"
   DENIED="REPLACE_WITH_SUPPORTED_EXCLUDED_DEPLOYMENT"
   probe() {
     curl --silent --show-error \
       --output "$EVIDENCE_DIR/$1.json" \
       --write-out '%{http_code}\n' \
       "$2/chat/completions" \
       -H "Authorization: Bearer $REQUESTY_API_KEY" \
       -H "Content-Type: application/json" \
       -d "$(printf '{"model":"%s","messages":[{"role":"user","content":"Reply with OK."}]}' "$3")" \
       > "$EVIDENCE_DIR/$1.status"
   }
   probe allowed https://router.eu.requesty.ai/v1 "$ALLOWED"
   probe excluded https://router.eu.requesty.ai/v1 "$DENIED"
   probe region https://router.requesty.ai/v1 "$ALLOWED"
   cat "$EVIDENCE_DIR/"*.status
   printf 'Responses saved in %s\n' "$EVIDENCE_DIR"

   Require HTTP 200 and a valid completion for the positive control. The error reference documents model-policy denial as HTTP 403, router origin, Provider blocked by policy; Access Lists also documents provider violates policy. Inspect the body for that intended denial.

   The allowed model through the forbidden gateway should fail under the region restriction. Review its response; do not assume an undocumented regional error code.

3. 3
   Archive evidence; test actions separately

   Save responses/statuses, timestamp, policy/version, key ID, deployments, endpoints, and reviewer verdict. Require the positive control before passing negative tests. Test aliases, modalities, fallbacks, and a forbidden action in your executor.

Compliance exports cover chat completions; traffic counts are provider attempts and exclude pre-provider blocks. Snapshots show current configuration, so retain change history and report copies: reports are generated on demand without an archive. The JSON hash identifies the report, not a signature or tamper evidence; protect your evidence store.

ZoomInfo routes access for more than 1,300 engineers through Requesty, keeping provider credentials inside the gateway and issuing Requesty keys.

"Every engineer wanting to access AI at ZoomInfo now has to go through Requesty!"

Arkady Landes, Senior Cloud DevOps Manager, ZoomInfo

Put the approved path into operation

Create an account and follow the governance documentation. For pooled budgets, scoped logs, SSO, and audit logs, talk to us about Enterprise.

Roll it out and prove it keeps working

Suggested first-month sequence for a bounded rollout, not a compliance deadline:

Measure coverage with honest denominators

Track expired exceptions, versioned-evaluation regressions, and detection-to-verified-containment time. Suggested cadence: weekly spend/scan-error review during rollout, monthly inventory/access/metrics reconciliation, quarterly independent sampling, and immediate reassessment after material change or incident.

Start with one workflow, output risk, and scoped identity. Prove one forbidden model and one forbidden action fail, then save the evidence. That is a framework your team can operate.

Sources

Legal milestones and product behavior checked September 26, 2026.

• NIST AI RMF, Core, and Generative AI Profile.
• ISO/IEC 42001.
• Enacted AI Omnibus and Commission timeline.
• OWASP prompt-injection guidance and Requesty governance evidence.