What is AI Governance
AI governance is the working rules for who may use which AI, on which data, and whether it may change a live business system — plus a record of what happened.
AI governance is the set of rules, enforced in the software people actually use, that decide who may use which AI, on which company data, and whether that AI is allowed to change a live business system — plus a record of what happened afterwards.
A training video is not that. An acceptable-use PDF is not that. An admin toggle the model can ignore is not that. If an unapproved change can still succeed, you have guidance, not governance.
People use the phrase for three different things, and they get mixed up:
- A public commitment — for example the OECD AI Principles.
- A company committee with a risk register.
- The runtime that actually stops a change to CRM, ERP, or a customer message.
All three are real. Only the third one would have blocked an unlogged field change that later showed up in a forecast.
Gartner’s TRiSM language is about that third layer: trust, risk, and security around the systems that run — not a quarterly slide about principles. The NIST AI Risk Management Framework says the same thing in public-sector language: Govern, Map, Measure, Manage. Mapping systems and measuring incidents still fail if the product people click can write to Salesforce without a named signer.
Words you’ll hear
- Live business system. CRM, ERP, HR, billing — the tools that hold official numbers and customer records. At work, this is where a fluent sentence becomes a fact other teams will inherit.
- Write / write-back. The AI is allowed to change that system, not only draft a suggestion. See What is write-back governance. At work, a next-step note and an Amount field are not the same risk class.
- Human-in-the-loop. A person must approve before the job can finish. See What is human-in-the-loop AI. At work, the gate shows the payload in the language of the live system, not a wall of prompt text.
- Named signer. The identity that authorised the change. At work, “someone in the channel clicked yes” is not a signer.
- Fail-closed. Missing approval means nothing happens. Fail-open means the change goes through unless someone happens to stop it.
- DPIA. A data-protection impact assessment — thinking through purpose, risk, and personal data before you turn a tool loose. UK ICO guidance on AI and data protection still wants a lawful basis and purpose when the “user” is an AI.
- Shadow AI. Personal ChatGPT for work because the official path is missing. See What is shadow AI.
- Inventory. A list of where AI actually runs. The US plan described in OMB M-24-10 puts a named owner and an inventory first, not a PDF.
- Least privilege. Only the data and tools required for this job. A workstream is how that instinct becomes a company object.
Model safety is adjacent and different. Safety is about what a model will say in the abstract. Enterprise governance is about what your people and tools may do with your systems and data. You can have a carefully aligned model and still have ungoverned CRM writes.
Why you should care
Without working rules, AI becomes a side effect. A field moves. A journal posts. A customer is told a policy the company does not hold. Nobody can say who allowed it. In February 2024, a British Columbia tribunal held Air Canada responsible for a chatbot that invented a bereavement-fare policy. CBC reported that the airline’s argument — the chatbot is a separate legal entity — failed. A customer-facing commitment without a working gate is still the company’s commitment.
It affects you if you:
- Own a number. Forecasts and close packs inherit whatever changed.
- Own a customer relationship. Model output that becomes a commitment is still the company’s commitment.
- Own risk or legal. Privacy law does not pause for a chatbot. GDPR still applies to purpose, minimisation, and erasure.
- Are asked “who is in charge of AI here?” An inventory and a named owner beat a principles slide.
Good governance in practice is four working rules:
- People and rights. Humans and AI tools are both actors. Roles decide what they may start, see, and sign.
- Data at question time. Purpose and minimisation still apply when an AI is the one looking.
- Action rights. Read-only is a control. Unrestricted tools are an incident waiting for a bad prompt.
- Evidence and spend. Chat scrollback is not a management system. Uncapped spend is a budget failure and often a security failure. See What is AI token economics.
Blocking consumer ChatGPT at the office network, while people use personal phones, is not governance. It is a shadow AI problem with extra steps.
What changes by role
Finance. Governance is whether an AI-proposed journal can post, against which checklist, with which signer, and whether the spend of the run was capped. “Unlimited AI” is not a control. Surprise inference bills are a governance failure that looks like a cloud invoice.
Legal. Lawful basis, purpose limitation, customer-facing language, and reconstructable authorisation. Legal also has to separate the OECD-style public commitment from the runtime. A principles page does not implement Article-style oversight. For higher-risk systems, EU AI law Article 14 talks about effective oversight: people must be able to interpret outputs and interrupt the system. A footer that says “generated by AI” is not that.
Operations. Isolation of jobs, connector scope, and a place to put a paused run. Ops already runs change control. Governance is change control that includes a model as a proposer.
Go-to-market. The difference between a draft email and a sent commitment; between a suggested next step and a changed Amount. GTM feels friction first. The honest metric is time-to-approved-write, not time-to-first-answer.
Security. Identity of the connected user, read versus write, prompt injection as a path to a tool call, and the new store created by logs and indexes. The OWASP Top 10 for LLM applications treats retrieval and tool use as a security surface, not only a quality issue. Network DLP helps with paste-out. It does not quote a CRM change.
What people get wrong
Governance as a committee. Useful for risk registers. Useless if the product can still write.
Governance as model safety. Refusals on public-web questions do not bind Salesforce.
Governance as a secure web gateway. Necessary for some paste-out paths. Insufficient for writes, approvals, and causal history.
Governance as blocking. Blocks without a sanctioned path train people onto phones. See What is shadow AI.
Theatre. A checkbox, a prompt that says “ask first,” or an admin toggle the model can ignore.
Good looks like: connectors default to read-only; writes are quoted; a named signer cannot be waived by the model; evidence lives on a lifecycle graph; spend has a ceiling; scope follows the job. Failure looks like a PDF, a blocked URL, and a personal API key in a wiki.
Adjacent concepts: write-back governance is the write subset. Human-in-the-loop is the gate. Workstreams are the isolation unit. An enterprise AI operating system is the product shape that makes those rules the default path.
How this shows up in Nimbus
Nimbus treats governance as how work is released, not as a sidecar policy engine.
Connectors — secure links to live systems — default to read-only. When a change is proposed, the product shows the intended action and waits. A named person must sign. The model cannot waive the gate. Missing approval is fail-closed: nothing happens.
Scope is the workstream: one job, with the playbooks, systems, teams, and budget that belong to that job. Evidence is the Lifecycle Graph. The company wiki is the asserted policy the run must cite. Model routing does not bypass the gate.
See Governance. For scoring vendors: How to evaluate AI governance platforms.
Questions people actually ask
Is AI governance the same as making the model “safe”?
No. Model safety is about what the model will say in the abstract. Enterprise governance is about what your people and tools may do with your systems and data.
Can we rely on the secure web gateway?
Network controls help with paste-out. They do not quote a CRM change, bind an approver, or store a causal history. Use both.
Must a person always approve?
For many operational writes, yes. For read-only analysis, maybe not. The mistake is calling a system “human-approved” because a human could look, while changes proceed on model initiative.
Do the OECD AI Principles require a specific product?
No. They are a public commitment. A product can make evidence cheaper to produce. The commitment does not implement a gate.
Is a DPIA enough to go live?
It is necessary thinking, not a runtime. You still need identity, scope, fail-closed writes, and a record. The DPIA should describe those controls, not replace them.
Does blocking ChatGPT count as governance?
It is a network control. Without a sanctioned path that can see the right files, people use personal phones. Blocking can tighten after substitution exists.
How is this different from IT change management?
It is the same instinct — who may change production, with what evidence — applied to a proposer that speaks English. Existing CAB processes rarely see model-initiated payloads unless the product emits them.
Who should be the named owner of AI?
Someone who can inventory systems and stop a write path, not a volunteer “champion” with no authority over CRM. Federal-style guidance starts with inventory and ownership for a reason.
Can we govern only customer-facing chatbots and ignore internal copilots?
Internal tools still process personal data and still write to live systems. Air Canada was customer-facing. Ungoverned CRM hygiene is an internal path to the same class of invented fact.
Do we need the EU AI Act if we are not a high-risk provider?
You may still have GDPR duties, sector rules, and customer contracts. Oversight and records are useful even when a specific Act title does not apply. Do not claim “Act compliant” because you have a button.
Where does spend fit?
Uncapped inference is a control failure. Quotes, ceilings, and attribution by job are governance of a scarce, abusable resource. See What is AI token economics.
Is an acceptable-use policy still worth writing?
Yes, as communication. No, as enforcement. Write the PDF. Then put the same rules in the product people actually use.
Related reading
What is write-back governance, What is shadow AI, and What is an enterprise AI operating system.
Sources
See what governed AI looks like on your stack.
Connect your tools, run a workstream, and keep every decision on your ledger - free for 7 days.