AI model registry: the foundation of AI governance

An AI model registry answers a simple but often difficult question: which AI systems operate in the organisation, who owns them and what evidence supported approval. Without a registry, governance depends on surveys and individual memory.

The scope should not stop at in-house ML. Include external LLM APIs, AI features inside SaaS, open-source models, RAG systems, agents and automation that acts on model output.

Register the use case

The same model may summarise public text or influence a customer decision. Risk differs, so register the model together with the use case, data, affected people and autonomy.

A minimum record contains:

• business and technical owners;
• purpose, users and dependent systems;
• supplier, model, version and region;
• input and output data categories;
• RAG sources, memory and available tools;
• impact tier and required controls;
• evaluation results, approval date and review deadline;
• fallback, retirement and incident-response plans.

Add automated discovery

An intake form will not reveal every deployment. Supplement it with SaaS spending, proxy and DNS logs, repositories, CI secrets, OAuth application catalogues and traffic to model APIs.

Discovery of an unknown tool is not automatically a policy breach. Confirm the owner, data and actual use, then decide whether to approve, restrict, migrate or disable it.

Risk-based gates

A public-document assistant does not need the same process as a model that evaluates customers. Define tiers with proportional requirements.

For high-impact uses, require an impact assessment, quality and security tests, data review, human oversight, monitoring, an appeal route and formal acceptance. Attach evidence or links to the record rather than a “tested” checkbox.

A model version is a change

Suppliers may update models, filters, retention or regions. The registry should receive change signals and trigger re-evaluation when impact crosses a threshold.

Pin model versions where available. Retain test sets and results for behavioural comparison. Use a structured LLM evaluation process before promoting the change.

Lifecycle and retirement

Give records explicit states: proposed, pilot, approved, restricted, retiring or closed. Remove inactive keys, indexes and training data according to retention policy.

Quarterly, verify ownership, activity, contract, model and evidence. Alert when a review expires or a deployed version loses support.

Programme metrics

Measure detected AI traffic covered by records, discovery-to-decision time, systems with current evaluations, ownerless deployments and retirement time. A larger inventory can merely indicate more administration; it does not prove risk control.

The model registry links engineering, risk, procurement, security and legal work. It creates the most value when it reacts to change automatically and drives a concrete decision.

---

The minimum production record

The registry should connect a model version to its use, owner, provider, data sources, processing location, system prompt, tools, evaluation and approval date. For APIs named “latest”, retain the resolved version identifier when the provider exposes it. Without that, an incident cannot be reproduced or a regression compared.

Changes to model, prompt, RAG index or tools should trigger a risk-based impact review. “Retired” does not mean deleted: define retention for logs, replicas, embeddings and training data. The registry should feed monitoring and incident response, not remain a spreadsheet refreshed annually.

Sources: NIST AI RMF, NIST GenAI Profile, OWASP GenAI Data Security.