Workflow pattern
A governed research assistant for policy and program teams
Analysts can spend more time locating evidence than judging it. This workflow puts retrieval and first-draft synthesis on an approved corpus, keeps every citation bound to the passage it came from, and leaves the judgment where it belongs. Eight weeks from candidate workflow to a production decision.
The friction
The same instruments, re-read by a different person, every time
Policy and program teams rarely lack information. They lack a citable route back to information the organization has already read.
A request arrives: what does the instrument say, what changed at the last review, what did we tell the minister last time. An analyst opens the act, the regulation, the directive, three guidance documents and two prior briefings. That reading is re-reading. A colleague did it before, for a question that was almost the same. What mattered about it was never captured in a form anyone could reuse.
The cost shows up as turnaround: a question that is an hour of judgment becomes days of elapsed time. It also shows up as inconsistency. Two analysts working from different subsets of the same corpus produce different answers, and nobody can reconstruct which passages drove the difference.
General-purpose AI tools make this worse. A model answering from its training data produces prose that reads like a briefing and traces to nothing. In policy work an uncitable answer is unusable, and a persuasive one carries additional risk.
Note
What decides whether AI can be used for policy research
The corpus, the citations, the approver and the handling of conflict decide whether an institution can use AI for policy research. The choice of tool does not.
Sources
Approved-source retrieval starts with a corpus somebody owns
Before anyone builds retrieval, the client defines, classifies and signs off the corpus. It is the most consequential design step, and it is a client decision we document rather than a technical one we make.
We work with the accountable business owner and the information-management function to produce a source register. Every source carries five attributes: what it is, who owns it, its classification, its authoritative status, and how we know the version is still in force. Anything missing one of the five stays out until someone resolves the gap.
- Authoritative instruments
- Acts, regulations, directives and published guidance, taken from the version the institution treats as authoritative rather than a copied snapshot.
- Institutional interpretation
- Approved interpretation notes, program guidance and prior briefings, attributed to the unit that owns each.
- One classification level
- A single level for the first release. Mixing levels in one index is an access-control problem disguised as a retrieval problem.
- Exclusions, in writing
- Drafts, personal files, unapproved wikis, anything with an open retention question. The exclusion list is a deliverable.
- Currency rules
- How superseded versions are marked, how new versions enter the index, and how long a stale document may remain retrievable.
The register lets a security reviewer, a privacy officer and a records manager each answer their own question in one sitting. The control model around it is on security and governance.
Grounding
Citations survive to the passage, and conflicts survive to the reader
Two behaviours separate a research assistant an institution can defend from one it cannot. It shows where each statement came from. It refuses to quietly resolve a disagreement between approved sources.
Approved source
Assistant response
Impact Level III systems require peer review by at least one qualified expert, and the final administrative decision must be made by a human rather than by the system.
- Directive on Automated Decision-Making, s.6.3
- Guide to Peer Review of Automated Decision Systems, s.2
Conflicting guidance detected in a third source. Routed to a human reviewer rather than resolved automatically.
Exact citation preservation
The assistant binds every statement to the passage that supports it: document, version, section and the span of text itself, rather than a document-level link. The analyst can open it in place. Where it cannot bind a statement to a passage, it does not write the statement. It reports that the corpus does not answer the question. We test citation integrity rather than assume it, because pointing at the right document while paraphrasing something it does not say is the failure mode that matters.
Conflicting and superseded sources
Approved sources can disagree: an instrument and its guidance drift apart, or someone updates a directive and not its interpretation note. A system tuned to produce one confident answer averages them, invisibly. Ours does the opposite. When retrieved passages conflict, the draft says so, presents each position with its own citation and date, and routes the question to the reviewer as an open item. Where a newer source supersedes an older one, the assistant surfaces the current version and notes the prior one. The change between versions is often what the request is about.
Obligations
Records, retention and what the Directive covers
A research assistant creates records. Queries, retrieved passages, drafts and reviewer edits are information under the institution’s control. They inherit the same obligations on retention and disposition, and on access, as the rest of your holdings. That is designed in, not discovered after go-live.
Information management
- Query and output logs mapped to an existing records class, with a retention period the records office sets, not a platform default.
- Disposition applied on schedule, including derived artifacts such as embeddings and caches, which are easy to overlook.
- Logs searchable for access-to-information and privacy requests, because they will be subject to them.
- A documented decision on whether drafts are records of business value, taken with the records office before the pilot ends.
The Directive on Automated Decision-Making
The Directive applies to production systems that make or support administrative decisions or related assessments about clients; test and research systems are excluded. An assistant that retrieves and drafts for an analyst is usually outside that scope. The applicability screen settles it, and it runs in the governance sprint before any build begins. Where the Directive applies, impact level drives peer review obligations.
Source
Sources: the Treasury Board Directive on Automated Decision-Making, modified 24 June 2025. The Guide to Peer Review of Automated Decision Systems, 2025, under which Impact Levels II to IV require peer review.
Boundaries
What this assistant is not
We enforce the boundary in prompts, tool permissions, evaluation cases and the interface itself, rather than asserting it in a disclaimer.
Warning
Three things it must never do
- It does not advise. It reports what approved sources say. It does not recommend a policy position, a course of action or an interpretation, and it does not characterize the strength of an argument.
- It does not decide. Nothing it produces is an assessment about a person, an entitlement or a case. It has no authority and no write access to any system of record.
- It does not summarize away a conflict. Resolving a disagreement between approved sources is analytical work with accountability attached. The assistant surfaces the conflict and stops.
It is not a substitute for counsel, and not a source of anything outside the approved corpus. Asked something the corpus does not cover, the correct behaviour is a refusal that explains the boundary. We score that refusal behaviour like any other.
The human approval point
One named role approves before anything leaves the workflow: the analyst who owns the product. The draft arrives with citations attached, open conflicts listed and low-confidence sections marked. The analyst edits, rejects or approves. The action is recorded against that version of the draft. Nothing reaches a decision-maker or an external party without passing that point.
Evidence
How we prove it works before anyone depends on it
Research assistants are judged on dimensions that general benchmarks do not cover. We build a golden set from real past briefings and the approved answers your analysts produced, and score against it every release.
| Dimension | What is measured | Failure it catches |
|---|---|---|
| Citation accuracy | Whether each citation resolves to a passage that supports the statement, at the section level. | Confident prose attached to a real document that does not say it. |
| Groundedness | Share of answers fully supported by the cited sources, with unsupported claims counted as defects. | Model knowledge leaking in as if it were institutional knowledge. |
| Completeness | Coverage against a golden set of past briefings: did the draft find the instruments the approved answer relied on. | Plausible, well-cited drafts that quietly miss the governing provision. |
| Conflict detection | Recall on seeded cases where two approved sources disagree or one is superseded. | Averaging a disagreement into a single confident answer. |
| Refusal behaviour | Correct refusal rate on out-of-corpus questions, and false-refusal rate on questions the corpus does answer. | Answering from outside the boundary, or becoming useless by refusing everything. |
| Currency | Whether the version retrieved is the version in force on the date of the query. | Correct citations to an instrument that was replaced last quarter. |
Bilingual work
In federal and provincial settings the corpus is often bilingual and the output must be too. Where both language versions of an instrument are authoritative, machine-translating an English answer is not an acceptable design: a French question must retrieve and cite the French passage. We test French and English as separate sets with their own thresholds and report the gap, not an average that hides it. Alexander Leslie, a native bilingual speaker of English and French, leads French-language engagements. We design French-language evaluation with the client, and independent qualified reviewers perform it. The client names those reviewers, or we contract them, and the acceptance pack identifies them.
Operating measures
- Time to first defensible draft, against the baseline measured before the pilot.
- Citation accuracy and groundedness, reported monthly with the threshold beside the result.
- Analyst edit distance: the share of the draft that survives review, trended over time.
- Conflicts surfaced and how they were resolved, showing where the corpus needs attention.
- Refusals and out-of-corpus questions, which tell you what to approve into the corpus next.
These carry into steady state through Managed AgentOps, where the same suite runs as a regression test on every model, prompt or corpus change.
Direct answers
Questions reviewers ask first
How can government teams use AI for policy research?
Use it for retrieval and first-draft synthesis over an approved, classified corpus, never for advice or decisions. Define the sources and their owners, keep exact citations to the passage, surface conflicts instead of resolving them, and require a named analyst to approve every output. Run an Algorithmic Impact Assessment applicability screen first. A research assistant supporting an analyst usually sits outside the Directive on Automated Decision-Making, but the screen is what establishes that. It belongs in the record before the build starts.
How can an AI assistant cite approved sources?
By retrieving only from an approved index, and binding every statement to the retrieved passage that supports it: document, version, section and text span. A reviewer can then open the source in place. The system produces no statement it cannot bind to a passage. We then test citation integrity directly: a citation that resolves to the right document but does not support the sentence counts as a defect. We test currency too, so the version cited is the version in force.
How do you evaluate hallucination in a research assistant?
With a golden set built from your own past briefings and their approved answers, scored on four things. Groundedness is the share of answers that the cited sources fully support. We measure citation accuracy at the passage level. Completeness asks whether the governing instruments were found at all, and refusal behaviour covers the questions the corpus cannot answer. We agree thresholds at acceptance and re-test them on every model, prompt or corpus change. A system that passed in March is not evidence about the system running in September.
Next step
Bring one recurring research question
If your team answers a version of the same question every month, that is the candidate. Thirty minutes, scored against the seven sprint dimensions, and a straight answer.
Context for public-sector buyers is on governed AI for government, and for federally regulated financial institutions on Office of the Superintendent of Financial Institutions (OSFI) Guideline E-23 and AI agents. Delivery runs through the Opportunity and Governance Sprint and the six-week Sovereign Knowledge Agent Pilot, on the control model in security and governance.