How to complete an Algorithmic Impact Assessment

The first half is 65 risk questions producing a raw impact score out of 169. The second half is 41 mitigation questions producing a mitigation score out of 77. The risk half is weighted towards data and towards impact on people, which is where the design decisions that move a score sit.

The 41 mitigation questions split into 4 on consultations, worth 10, and 37 on de-risking and mitigation, worth 67.

Mitigation does not reduce your score proportionally. If the mitigation score reaches 80 percent or more of the maximum attainable mitigation score, the tool deducts 15 percent from the raw impact score. Below that threshold, it deducts nothing and the current score equals the raw impact score.

Eighty percent of 77 is 62. A project sitting at 61 gets no credit for any of it. The same project at 62 drops 15 percent of its raw impact score in one step. A project near a level boundary can change level on mitigation evidence alone, without the system changing at all.

What score produces which impact level?

Two consequences follow for delivery. Answer the mitigation half early enough to know whether 62 is reachable, because if it is not, the effort spent on partial mitigation buys no score. And treat the data and impact sections as design inputs rather than as a form filled in at the end, because together they carry 107 of the 169 available points.

Read the approval row before the peer review row. A Level III project usually takes longer because a deputy head has to sign, and a Level IV project needs Treasury Board; the reviewer is rarely the cause. Those are calendar items owned by your institution rather than by a supplier, and they belong in a plan before anyone writes code.

Note also what does not change. Peer review is one expert at both Level II and Level III. Two experts is a Level IV requirement, and it should not be read as a Level III one.

1. Complete it once at the beginning of design. The score at this point tells you which approval authority you are heading for and whether the design needs to change to avoid a level you cannot resource.
2. Book the peer review during design, not after acceptance. The guide suggests budgeting one to three months, and recommends running it after privacy and security assessments are complete or underway.
3. Complete it a second time before production. The two completions are separate by design, so a score that moves between them is expected rather than a finding.
4. Publish three things before the system launches. First, the assessment results on the Open Government Portal, in both official languages and in an accessible format. Second, the peer review report or a justified plain-language summary of it. Third, the plain-language notice through every service delivery channel in use.
5. Update it when scope or functionality changes. The Directive requires review, approval and update on a scheduled basis and when the system changes.

Who can act as a peer reviewer?

Eligible reviewers include experts from a federal, provincial or municipal government institution and faculty at a post-secondary institution. Researchers at civil society organizations also qualify, as do third-party vendors and individuals at other external organizations. So do members of a data and automation advisory board specified by the Treasury Board Secretariat. Reviewers disclose conflicts of interest, and the guide recommends choosing experts external to your own department and avoiding reciprocal reviews between departments. Hiring a firm is one route. It is not the only one.