Open-weight AI in the enterprise: five risks to review before deployment
Microsoft and NVIDIA propose a shared risk framework for open-weight models. We turn it into questions organisations should answer before self-hosting a model.
- AUTHOR
- Karol Rapacz / CEO Breachroad · OSCP · PNPT
- PUBLISHED
- 19 October 2026
- READING TIME
- 9 min read
- TOPIC
- AI Security
Downloading a model and running it on your own infrastructure can improve control over data, cost and availability. It also transfers responsibilities that a managed service may previously have carried. The label “open-weight” does not decide whether a deployment is safe.
On October 16, Microsoft and NVIDIA published a joint framework for managing open-weight model risk in enterprise deployments. Grounded in ISO and the NIST AI Risk Management Framework, it groups risk into five categories: cybersecurity, data protection, content, human–AI interaction and bias.
The authors stress that open or closed labels do not determine risk by themselves. The specific model, use case, deployment and safeguards matter. A model used to draft internal documents and the same model embedded in a customer workflow with tool access have very different impact profiles.
Breachroad’s conclusion: the AI team should not make the “can we use this model?” decision alone. Process owners, security, privacy, legal and IT need to agree on the boundaries of the whole system, not just model parameters.
Open-weight does not automatically mean open source
An open-weight model makes trained weights available, allowing organisations to run, evaluate and sometimes adapt it in their own environment. That does not guarantee full access to training data, code, process documentation or unrestricted usage rights. A licence may constrain use, while artifact visibility may reveal little about provenance.
A review should therefore cover at least the licence, download source, checksum, file format, model documentation, declared data and limitations. The ability to download a file is not approval to use it with company information.
Five categories turned into questions
| Area | Question before deployment |
|---|---|
| Cybersecurity | Could the artifact, inference server or integrations enable code execution, secret exposure or tool abuse? |
| Data protection | What data enters prompts, logs, caches, conversation memory and fine-tuning sets? |
| Content | How will the process detect harmful, illegal, confidential or simply incorrect output? |
| Human–AI interaction | Do users know when AI is involved, understand its limits and have a route to challenge a decision? |
| Bias | Does quality or error rate vary across languages, groups or cases material to the organisation? |
Microsoft and NVIDIA caution that the taxonomy is not a ready-made policy or universal checklist. Organisations must connect it to the use case, evaluations, mitigations and named risk owners.
Three deployments, three responsibility profiles
Managed API
The provider may handle parts of infrastructure, filtering, updates and monitoring. The customer still owns the data sent to the service, application permissions, process compliance and response to incorrect output.
Self-hosted model
The organisation assumes responsibility for artifact provenance, safe formats, the inference library, containers, updates, GPU isolation, logging and administrative access. A private deployment may reduce data transfer to a provider, but it does not eliminate leakage inside the organisation’s own environment.
Agent connected to tools
A model that can reach email, files, CRM, source code or payments has far more impact than a question-answering chatbot. Review permissions, action approval, prompt-injection resistance, tenant boundaries, cost limits and safe shutdown.
The joint framework focuses mainly on the model layer. Its authors explicitly note that the complete system also requires assessment of the application, data, tools, runtime and operational processes.
A minimum production decision record
- Use case and owner. State the permitted task, prohibited uses, users and the person accountable for the process outcome.
- Provenance. Record the repository, version, licence, hash and update channel. Do not rely on an unverified mirror.
- Data. Define information classes allowed in input, log retention, conversation memory and fine-tuning rules.
- Evaluation. Test quality, refusals, leakage, multilingual behaviour and misuse cases using data representative of the process.
- Execution boundaries. Run the model with minimum privileges and separate it from secrets, the host and unnecessary network access.
- Human oversight. Identify decisions that require approval and provide routes for appeal and correction.
- Monitoring and rollback. Define metrics, alerts, a fallback version, service shutdown and incident handling.
An evaluation is not a permanent certificate. Results depend on prompts, configuration, version, language and context. Changing the model or adding a tool may require re-testing the entire use case.
Facts and recommendations
The five categories, the dependency of risk on use case and deployment, and the distribution of controls across layers come from the Microsoft and NVIDIA paper. It does not claim that every open-weight model is inherently safer or less safe than a closed service.
The decision record, proposed owner group and minimum approval evidence are Breachroad recommendations. Our analysis of the open-weight model wave provides broader context, while the guide to SafeTensors and Pickle risk covers artifact formats. We help organisations translate these principles into roles and daily decisions through cybersecurity and secure-AI training and secure AI implementation workshops.
