An AI support assistant that drafts a reply and one that can issue a refund may use the same model. Their risks are different because their authority is different. A useful AI risk assessment starts with that difference. A label such as medium cannot tell a reviewer what the system can do or how a failure would reach a customer.
Assess the workflow before choosing the score
NIST’s voluntary AI Risk Management Framework allows qualitative, quantitative and mixed methods, and considers effects on people as well as organizations. ISO/IEC 42001 is an AI management system standard. Neither requires every AI risk to be reduced to money. A financial estimate can inform a spending decision while safety, privacy and rights concerns remain explicit constraints. NIST AI RMF Core.
For a budget discussion, ask a narrower question: which loss could this control reduce, and what evidence supports that expectation? A heat map may help present the answer. It cannot replace the scenario, the control test or the assumptions behind an estimate.
The deployment changes the risk
Likelihood and impact categories can support triage when their meaning is defined. Problems arise when the category hides information the decision needs. A system that can read customer records, an agent that can export them and a chatbot that cannot access them should not inherit one rating merely because all three use generative AI.
AI systems introduce unfamiliar failure paths, but they also depend on familiar controls: authentication, authorization, change management and recovery. Begin with what can be observed. Which inputs are untrusted? Which records can the system retrieve? Which actions require approval? Where can an output leave the organization?
Suppose an agent reads a malicious instruction inside a support attachment. The consequence depends on whether it can retrieve other customers’ records and send them outside an approved destination. Testing those boundaries produces evidence about this deployment. A generic model benchmark does not answer the same question.
Several features may share a model or tool service. Assess which functions depend on it and what happens when it fails; a shared provider does not mean every feature fails identically.
Follow the failure from input to consequence
Estimate the loss you can define
For financial questions, FAIR provides a structure based on loss-event frequency and loss magnitude. It can help organize a scenario such as an unauthorized export through an agent. Use it alongside assessments of harms that a dollar estimate cannot adequately represent. FAIR Institute overview.
Define the event first: an agent exports customer records to an unauthorized destination. Then identify the conditions that would allow it, the frequency assumptions and the potential consequences. Estimate response work and business interruption without double counting. Model downstream legal or customer reactions only where they are plausible, with their own uncertainty.
For this example, test the agent with a malicious attachment, an unauthorized customer identifier and an external destination. Record whether retrieval and export were refused, whether approval could be bypassed, and what the audit record shows. A passed test supports a narrow claim about the tested configuration. It does not establish the annual probability of an attack succeeding.
Illustrative financial model for one defined AI loss scenario. A P90 figure depends on the inputs, time horizon and assumptions used in the simulation.
The loss estimate is conditional on the evidence. If the team has little information about attack frequency, show that uncertainty and test how much the decision depends on it. More simulation runs improve numerical stability; they do not create missing operational evidence.
Use the result to choose the next action
The best next step may be a permission change, a stronger approval boundary or a test that resolves an important unknown. Funding priorities should also reflect mandatory obligations and potential harm to people, even when those effects are difficult to price.
Build the scenario
Describe the failure. Name the input, the action and the affected party. “An agent exports another customer’s records” gives a test team something concrete to investigate.
Test the authority. Check retrieval permissions, allowed destinations and approval enforcement in the actual workflow. Include refusals and recovery after a failed tool call.
Keep financial assumptions visible. Record frequency and loss estimates separately from observed test results. A failed test shows a weakness; it does not tell you its annual incident rate.
Account for shared dependencies. A model outage, compromised tool or identity service may affect several scenarios. Document that overlap before building a portfolio estimate.
Make and revisit the decision
Apply decision constraints. Address mandatory requirements and unacceptable harm before comparing expected financial benefits. Money is one view of the decision.
Compare a specific control. Explain how narrower access, human approval or a tested fallback changes the scenario. Re-run the relevant tests before assuming the exposure has fallen.
Investigate sensitive assumptions. If the recommendation reverses under a plausible recovery-time estimate, measure recovery rather than presenting one precise result.
Review relevant changes. New tools, broader access, changed data sources or a new model can alter the workflow. Decide which changes require testing and who can approve the release.
A useful assessment explains what would change the decision
If reducing the agent’s access removes the sensitive-data path, test that change and update the scenario. If the feature cannot meet an important safety requirement, a low financial estimate does not settle the issue. Keep the assessment attached to the deployed workflow and revisit it when its authority changes. Our guide to risk assessment methods explains how to document that decision.
To put the assessment into an operating process, use the enterprise AI governance guide to connect scenarios and control evidence to accountable owners, deployment decisions and monitoring.
Explore Zania’s first-party risk management →
Sources:
Share
