From demonstration to operating service, the AgentOps control model

A demonstration answers: can it work? An operating service answers: who is responsible when it does not, and how will we know?

Six things become required that were optional the day before.

A named owner with a delegate. The Treasury Board of Canada Secretariat’s Guide on the Use of Agentic Artificial Intelligence, last modified 22 May 2026, is specific. It names a role accountable for the agent’s outcomes and monitoring, with a documented escalation path and a designated delegate during absences or transitions. It adds: “Accountability always ultimately rests with the designated human owner, even when the agent acts autonomously within approved permissions.” A service whose owner is on leave and has no delegate is unowned, which is a different thing from lightly supervised.

Bounded autonomy, written down. The guide asks that agents “run with tight, explicit parameters that limit data, tools, permissions and scope.” It wants each agent at a labelled activity permission level, for example ‘draft only’ or ‘read only’, so the agent can be useful without broad powers of general application. A demonstration runs with whatever credentials were convenient. A service runs at a level someone signed off on.

Recoverability as a design property. The guide says to design agents so they can be guided easily, paused or stopped when needed, and quickly returned to a safe and stable state. It also wants a pause and disable mechanism “external to the AI agent or agentic system”, with a recovery plan for unintended actions. External, because a stop control inside the thing you are stopping is a request.

Logging sufficient to reconstruct. The guide says: “Keep logs of tools used, actions taken, approvals given and key inputs/outputs.” For tool calls, it adds the tool name, the time, the outcome of the call and the type of parameters passed. Every field is cheap at build time and unobtainable afterwards.

A standing comparison against human performance. The guide asks institutions to “occasionally have the agentic AI tasks performed manually by human experts for comparison with the agent’s performance.” It notes that this “can indicate the need for retraining of the humans or the AI agent, depending on the results.” It adds “regular audits and evaluations” and “occasional spot checks” to “catch slippage early.” This control is easy to skip, and it is the only one that detects quiet degradation.

A gate on state altering actions. The guide’s operative sentence is this. When an agent’s action “alters a system’s state (send, publish, approve, spend, update records),” the guide says to “include a review/confirmation step in the design.” The exception: “unless the expected impact is demonstrably low and the action is trivially reversible.” Note that the trigger is the action type, and that the exemption requires low impact and trivial reversibility together. Reversibility alone is not a pass.

A financial institution does not need to wait for artificial intelligence specific rules. They are already covered, twice over.

The Office of the Superintendent of Financial Institutions (OSFI) released the final revised version of its model risk management guideline on 11 September 2025. It takes effect for all federally regulated financial institutions on 1 May 2027. It defines a model as “an application of theoretical, empirical, judgmental assumptions or statistical techniques, including AI/ML methods, which processes input data to generate results.” It also anticipates the growing prevalence of models characterized by dynamic self learning and autonomous decision making. The earlier version covered deposit taking institutions. This one covers all models at all federally regulated financial institutions.

The third party risk management guideline has been effective since 1 May 2024, with the expectation that third party arrangements commencing on or after that date adhere to it.

Read together: an agent is a model, and the supplier of that agent is a third party. Both lifecycles already have owners, inventories, validation expectations and review cadences inside Canadian financial institutions. The work is to route agents into the machinery that exists, before someone builds a parallel AgentOps function.

The same reasoning generalizes. Take a regulated Canadian organization that already runs model governance, third party risk, change management and incident response. An agent that sits outside all four is ungoverned because nobody asked which of the four it belongs to, not by accident.

The National Institute of Standards and Technology Artificial Intelligence Risk Management Framework, version 1.0, January 2023, organizes the work under four functions. Govern “cultivates and implements a culture of risk management.” Map “establishes the context to frame risks related to an AI system.” Measure employs quantitative and qualitative techniques, or a blend of both, “to analyze, assess, benchmark, and monitor AI risk.” Manage “entails allocating risk resources to mapped and measured risks on a regular basis.” Its generative artificial intelligence profile followed in July 2024 as a cross sectoral companion.

The framework is useful as a shape for the operating model. It is not a control list, and treating it as one produces a spreadsheet nobody reads.

ISO/IEC 42001:2023, published in December 2023 by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC), is a certifiable management system standard for artificial intelligence. It is the right instrument if the organization wants an auditable management system. It is paywalled, which means anyone quoting clause level text to you without a licence is quoting something else.

Article 14 of Regulation (EU) 2024/1689 sets the European Union’s human oversight obligations. The people assigned to oversight must be enabled, “as appropriate and proportionate”, to “otherwise disregard, override or reverse the output”. They must also be able to “interrupt the system through a ‘stop’ button or a similar procedure that allows the system to come to a halt in a safe state.” That is good design, whatever jurisdiction you operate in.

The application dates have moved. Regulation (EU) 2026/1744 was published in the Official Journal on 24 July 2026. It set the date of application for the obligations in Chapter III Sections 1, 2 and 3. For systems classified as high risk under Annex III, the date is 2 December 2027. For those under Annex I, it is 2 August 2028. The original dates were 2 August 2026 and 2 August 2027 respectively. Article 14 sits in Chapter III Section 2, so it moved with them.

And the Canadian guide is guidance, not a directive: it points readers to the Directive on Automated Decision Making for binding obligations. Treating it as mandatory internally is a defensible choice. Describing it as mandatory to an executive committee is not.

For each agent in service: the named owner and delegate. The permission level, written. The list of state altering actions and the gate on each. The log fields, verified by reconstructing one week of activity from them. The human comparison cadence and who runs it. The external stop control and the last date it was tested.

Six lines per agent. An organization that cannot fill in six lines for an agent running in production does not have an operating service. It has a demonstration that nobody turned off.

Previously: bilingual evaluation for Canadian government AI workflows. Next in this series: how to write a procurement-ready AI pilot statement of work.