THE INTELLIGENT AI GOVERNANCE LEADERSHIP PROGRAMME
Separate the claim from the permission.
20-Session Programme · 25 Minutes Per Day · Free · For leaders in Nigeria’s oil and gas sector
WEEK 4 OF 4 · RELEASE DECISIONS, THE BOARD AND THE NINETY DAYS · SESSIONS 16–20 · WORK AT YOUR OWN PACE
Completed Case · Chapter 27SESSION 16
A Release Decision for Predictive Maintenance
Why This Matters
Week 4 is about the moment someone has to sign. For fifteen sessions you have built the parts: the claim, the counterfactual, the owners, the evidence contract, the metrics, the plant boundary and the incident route. Today you watch them combine into one release decision, using the book's completed predictive-maintenance case. It is a constructed teaching case: every count, cost, operating constraint and outcome is assumed, and it is not an account of an actual operator. What makes it worth your time is the ending. A technically sound model receives a narrower permission than its sponsor asked for, and that is the right answer. You will then write your own release decision in the same form.
25-Minute Module
0:00 – 5:00
The requested claim, and why it was rejected
- An industrial enterprise proposes a model to warn of unplanned compressor failures within 14 days. The first requested release would automatically postpone low-ranked inspections and accelerate high-ranked work. The validation team objects that the evidence supports ranking more strongly than inspection deferral.
- The sponsor claims a 30% reduction in annual downtime. Model-ranked assets do contain many subsequent work orders, but those records include repair narratives written after the fault was discovered. In plain terms, the model was partly reading the answer.
- The claim is rejected in that form: the prediction information boundary is wrong and the intervention benefit is unmeasured. The claim is rejected, not the work.
- The revision: the target becomes confirmed unplanned failures; features are reconstructed as available at each prediction cutoff; post-outcome narratives are excluded; overlapping event windows are separated; validation is time-based. Deployment is restricted to one equipment family in defined steady operating modes.
When a sponsor quotes a downtime figure, ask what the model knew at the moment it made each prediction.
5:00 – 11:00
Evidence and its limitation
- The final test uses the confusion matrix from Chapter 10, which you met in Session 7. Every figure is constructed:
Measure Count or calculation Equipment periods; confirmed failures 10,000; 100 Failures detected and missed 80 detected; 20 missed False alerts and true negatives 495; 9,405 Alerts the system creates 80 + 495 = 575 Recall 80 / 100 = 80%; illustrative Wilson interval approximately 71.1% to 86.7% Precision 80 / 575 = approximately 13.9% - The book treats the 100 failures as independent for the interval, and says a real asset dataset would require an assessment of that assumption.
- This supports a bounded discrimination claim. It does not show that automatically deferring low-ranked inspections is safe. Startup conditions are underrepresented, so they stay excluded: the system must detect and flag the operating mode, not silently apply the steady-operation model to startup.
- THE DOCTRINE
A failure detected after the practical intervention deadline is not a useful early warning, even if the statistical label is correct.
- There is no prospective evidence yet that alerts arrive in time, so the next phase measures warning lead time and the proportion of alerts producing an actionable engineering finding.
The test shows the model can rank. Whether anyone can act in time needs separate evidence.
11:00 – 17:00
The economic model and review capacity
- Every figure is the book's teaching assumption, not a measured return or an industry average. Review: 575 alerts × one hour × $60 = $34,500 a year. Add $80,000 service and support and $120,000 first-year implementation: first-year cost $234,500. Each prevented failure avoids an assumed $15,000.
Assumed failures prevented Potential benefit First-year scenario net value 10 $150,000 −$84,500 20 $300,000 $65,500 35 $525,000 $290,500 50 (the book's financial scenario) $750,000 $515,500 - Break-even is 234,500 / 15,000 = 15.6333, so at least 16 prevented failures. That is not a prediction that 16 will be prevented; prevention is precisely the unresolved question. Finance also checks the avoided loss is not already attributed to another programme and reflects the relevant margin or cost basis.
- So the scenario can justify a bounded pilot, not the production return. The investment paper presents prevention as unresolved and makes prospective evidence a condition for scale.
- Review capacity: 575 alerts over 50 operating weeks averages 11.5 a week. The average is manageable; peaks may not be. The plan sets a queue limit and escalation path, and keeps required inspections regardless of rank. The trial records review time, lead time, findings, interventions and unalerted failures, with a verification sample of unalerted cases inside the existing inspection programme.
If the case turns on how many failures are prevented, the case is that assumption, not the model.
17:00 – 22:00
Controls, defeats and the decision
- Controls: invalid or unavailable signals are rejected and abstentions recorded; the interface shows horizon, scope and measurements used; excluded modes are flagged; a model or feature change triggers regression and domain review.
- Approval is defeated by inability to enforce the mode restriction, a sensor-unit mismatch, an uncontrolled change to inspection authority or loss of the prediction-to-outcome link. A suspected serious hazard can trigger immediate restriction. The drill confirms that stopping the service leaves maintenance and protection systems working and that queued recommendations cannot later amend work orders.
- OPERATIONS · THE WORKED CASE
- Operations. The book's decision: a time-limited advisory pilot for the defined equipment family and modes; automatic deferral rejected. The maintenance superintendent keeps the schedule, the engineering authority preserves the inspection procedure, and the service cannot close, postpone or cancel a required task.
- The approval names the business owner, technical owner, engineering authority and independent reviewer. Expansion needs evidence on lead time, intervention outcomes, review capacity, current calibration and transfer to other operating conditions.
ENTERPRISE · THE PARALLEL- Enterprise. The anatomy is identical for an enterprise system: requested claim, revised claim, evidence and limitation, labelled scenario, review capacity, controls and defeats, decision. What changes is the authority at stake. Session 17 shows it on the book's second case.
- THE DOCTRINE
A technically sound result can support a narrower approval than the sponsor requested. That is a successful governance outcome: useful evidence produces a proportionate permission and a clear next experiment.
- Read one line twice: the decision can change even if the retrospective score stays the same.
The sponsor got the permission the evidence could carry, and a route to more.
22:00 – 25:00
Before you close the laptop
- Write down your sponsor's claim as the sponsor would say it, and beside it the narrower claim your evidence can actually carry.
- This session produces Section 16 of your 90-Day AI Governance Plan: your release decision in the form of the completed case.
- If your company would today grant the sponsor's full request, say so, and say what would have to change.
Write the decision you would defend, not the one that would please the steering committee.
Key Terms
Prediction information boundary
The line between what was knowable at the moment of prediction and what became known later. Data from beyond it makes test results look better than deployment will be.
Bounded discrimination claim
A claim that the model ranks cases within a tested scope, with stated recall, precision and uncertainty. It does not establish that acting on the ranking is safe.
Defeat condition
A stated condition that withdraws an approval, such as a sensor-unit mismatch or an uncontrolled change to inspection authority.
Verification sample of unalerted cases
Cases the model did not flag, checked within the existing inspection programme, so outcomes are not observed only on model-selected assets.
Self-Check — Before You Submit
1. In the book's case, why was the sponsor's 30% downtime claim rejected in that form?
- A. Because 30% exceeds the industry benchmark
- B. Because precision of about 13.9% rules out any use
- C. Because the test set was too small
- D. Because records included repair narratives written after fault discovery, and the intervention benefit was unmeasured
D. Because records included repair narratives written after fault discovery, and the intervention benefit was unmeasured — The book rejects the claim on the information boundary and the unmeasured benefit. It cites no industry benchmark, and low precision limited the permission rather than preventing an advisory pilot.
2. Under the book's constructed assumptions, what is first-year scenario net value if only 10 failures are prevented?
- A. $65,500
- B. −$84,500
- C. $150,000
- D. $515,500
B. −$84,500 — 10 × $15,000 = $150,000, minus first-year cost of $234,500, gives −$84,500. $150,000 is gross benefit before cost; $515,500 rests on fifty prevented failures, the assumption the case calls unresolved.
3. A fictional gas processing company's reliability team shows a failure-warning model with strong retrospective recall and asks that it automatically postpone low-ranked inspections from next month. What do you authorise?
- A. Full release, because strong recall shows low-ranked assets are safe to defer
- B. Release with deferral, provided the vendor certifies the model
- C. A time-limited advisory pilot with required inspections retained, deferral rejected, and prospective evidence as the condition for expansion
- D. Rejection until the model reaches 100% recall
C. A time-limited advisory pilot with required inspections retained, deferral rejected, and prospective evidence as the condition for expansion — This applies the book's decision: a ranking test does not show deferral is safe, so authority is narrowed while evidence is gathered. A vendor certificate is not local evidence, and demanding perfect recall refuses a proportionate permission.
Today’s Assignment
Write Section 16 of your plan: the release decision for your company's chosen AI system, in the form of the book's completed case.
- State the requested claim as the sponsor puts it, then the revised claim. Name what the system knows when it produces an output, and anything in your test data that crossed that boundary.
- Give the evidence with its denominators, the conditions excluded and what the result does not show.
- Write an economic scenario with every input labelled as assumed, its pivotal assumption and break-even, and what finance must check before counting a benefit.
- Plan review capacity, a queue limit and escalation, then list controls, defeat conditions and the drill showing that stopping the system leaves existing processes working.
- Write the decision: what is authorised, what is rejected, owners by role and name, and the evidence required for expansion. Say honestly where your company would currently grant more than the evidence supports. If you have no AI system yet, use the proposal under consideration or a tool staff already use informally.
YOUR 90-DAY PLAN
Section 16 — Our release decision written in the form of the completed case.
How this is marked — the rubric
| Requested and revised claims distinguished, with the information boundary and exclusions stated | 25 |
| Evidence reported with denominators and what it does not show | 20 |
| Economic scenario labelled as assumed, with pivotal assumption, break-even and finance checks | 20 |
| Review capacity, controls, defeat conditions and a stop drill specified | 15 |
| A decision no wider than the evidence, with named owners and conditions for expansion | 20 |
| Total · pass mark 70 | 100 |
Submitting session 16
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 16, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.Capabilities Covered
Release decision anatomyNarrowing a claimScenario versus evidenceReview capacity planningDefeat conditions
Completed Case · Chapter 28SESSION 17
A Release Decision for a Regulatory Assistant
Why This Matters
Yesterday's case was a model that ranks equipment. Today's is an assistant that writes. The book's second completed release decision concerns an assistant that helps a Nigerian energy enterprise prepare a regulatory submission. It is a constructed case with assumed documents, tasks and counts. It does not describe a named organisation or assert any particular regulator's detailed submission requirements. I teach it because its sponsor's request is one many of you will receive: let the assistant find the records, draft the submission and send it once a manager approves in the conversation. By the end you will know why that last step is the one to refuse, and you will draft the system card and release record for your own system.
25-Minute Module
0:00 – 5:00
Divide the workflow
- The sponsor proposes that the assistant locate required records, draft the submission and send it after a manager says approved in the conversation. The principal risks: incorrect applicability, obsolete source use, missing evidence, confidentiality and unauthorised submission.
- Separate five stages: record retrieval, evidence extraction, drafting, professional review and external submission. The first three can be evaluated as bounded tasks. Review and submission require identified authority.
- THE DOCTRINE
A general approval of the assistant should not cover every stage automatically. Evaluate the bounded tasks; give review and submission to identified authority.
- The corpus holds controlled policies, inspection records and a hypothetical submission checklist, with revisions and effective dates recorded. The assistant must distinguish a current submission from reconstruction of a historical record, because the applicable version can differ.
When a request arrives as one workflow, your first job is to cut it into the decisions inside it.
5:00 – 11:00
The evaluation protocol and the two numbers
- Assume 300 held-out tasks: routine extraction, conflicting revisions, absent evidence, table and unit interpretation, unauthorised-access attempts and malicious document instructions. Domain reviewers set reference decisions and decide which errors are material.
Measure (constructed) Calculation or count Tasks; completed; deferred 300; 270; 300 − 270 = 30 Materially correct 252 Correctness among completed tasks 252 / 270 = 93.33% Correct completion over all tasks 252 / 300 = 84% Completed outputs with material errors 270 − 252 = 18: eight obsolete revisions, six omitted conditions, four misread table units - Both percentages are needed, and the organisation must examine whether the 30 deferrals are appropriate and can be handled.
- The aggregate 93.33% conceals failure patterns central to the intended use. Eight of the eighteen errors are obsolete revisions, one of the principal risks named at the start.
Whenever someone gives you one percentage, ask for the other denominator and the errors behind it.
11:00 – 16:00
Repair the source first; hold the submission authority
- The retrieval metadata does not identify effective dates reliably. The team repairs the corpus and parsing, then retests on a new held-out sample and the known-failure regression set. It does not claim that retesting the same repaired examples gives an unbiased population estimate.
- The assistant displays the source revision and exact supporting passage for consequential statements, and produces a gap list for missing records rather than a plausible assertion. The professional reviewer checks the pack against the actual governing checklist.
- Permission tests use distinct identities and deliberate attempts to reach another project's records. A detected breach blocks release until the retrieval boundary is corrected, and the remedy is enforced outside the model.
- In the initial release the assistant cannot submit externally. It prepares a draft and evidence index; an authorised person signs off and submits through the approved channel; the final pack is retained with sources and approval record.
- THE DOCTRINE
If submission is later automated, approval must bind to the final document and recipient. A conversation-level approval given before further revisions is insufficient.
When the defect is in the documents, a better model only produces the wrong revision more fluently.
16:00 – 21:00
What the approval says, and the record that holds it
- The accepted purpose is drafting and evidence assembly. It excludes legal interpretation without review, fabrication of missing records, alteration of certified inspection evidence and autonomous submission. Monitoring tracks material corrections, deferrals, retrieval freshness and permission failures; the source owner controls updates and withdrawals.
- Appendix F's system card indexes this: identity, owners, purpose, users, authority and exclusions, dependencies, data, classification, evidence, limitations, human review, monitoring, containment, approval with named authority, date, expiry and accepted residual risk, change triggers and incidents. Do not delete a material limitation simply to achieve one page. The release record adds an immutable ID, configuration identifiers, scope, evaluation, readiness, approvals, conditions, rollback and monitoring start.
Change class What it covers Editorial Changes presentation without altering a material claim or authority Evidence-relevant Changes model behaviour, data, prompt, retrieval or threshold Authority-relevant Changes actions, destinations, population or operating consequence - OPERATIONS · THE WORKED CASE
- Operations. In the Chapter 27 case a model or feature change triggers regression and domain review: evidence-relevant. Letting the service postpone a required inspection would be authority-relevant.
ENTERPRISE · THE PARALLEL- Enterprise. Here the effective-date repair is evidence-relevant and was retested. Granting direct submission is authority-relevant: more autonomy requires stronger evidence and a new authority decision.
- THE DOCTRINE
A sponsor should not classify a permission expansion as editorial merely because it is achieved through a small configuration edit. Record who classifies the change and how disagreement is resolved.
The size of the edit tells you nothing. The class of the change tells you who must decide.
21:00 – 25:00
Before you close the laptop
- Write down the stages inside your system's workflow, marking bounded tasks and stages that need identified authority.
- This session produces Section 17 of your 90-Day AI Governance Plan: the system card and release record for your system.
- The book's approved release narrows the assistant to a useful preparatory role that preserves professional responsibility. That is the shape of a good first release.
A card that has never shown a limitation is not yet telling the truth.
Key Terms
System card
A short index to the evidence for one AI system: owners, purpose, authority and exclusions, evidence, limitations, review, containment, approval and change triggers.
Release record
The immutable record of what was released: configuration identifiers, scope, evaluation, readiness, approvals, conditions, rollback and monitoring start.
Authority-relevant change
A change to actions, destinations, population or operating consequence. It needs an authority decision however small the edit.
Gap list
The assistant's explicit list of missing or conflicting records, produced instead of a plausible statement that fills the space.
Self-Check — Before You Submit
1. In the book's case, 252 of 270 completed tasks are materially correct, out of 300 tasks. Which reporting is sound?
- A. 93.33% only, since deferred tasks were never attempted
- B. 84% only, as the more conservative figure
- C. Both 93.33% and 84%, with the 30 deferrals examined and the pattern of the 18 errors shown
- D. The 18 errors only, since percentages mislead
C. Both 93.33% and 84%, with the 30 deferrals examined and the pattern of the 18 errors shown — The book says both figures are needed and that the aggregate conceals failure patterns. Either percentage alone hides the deferrals or the errors.
2. At a fictional upstream company, the head of regulatory affairs approved a draft in the assistant's chat yesterday. Two tables were revised this morning, and the pilot lets the assistant send once approval is recorded. What should happen?
- A. An authorised person signs off the final revised pack and submits it through the approved channel, with approval bound to that document and recipient
- B. The assistant sends, because the right person approved
- C. The assistant sends, with the revisions disclosed in a covering note
- D. The assistant rechecks the tables itself and sends if they pass
A. An authorised person signs off the final revised pack and submits it through the approved channel, with approval bound to that document and recipient — A conversation-level approval given before further revisions is insufficient. The assistant's own check does not replace the authorised signatory, and disclosure after sending does not authorise the submission.
3. A sponsor reports a one-line configuration edit as editorial. It lets the assistant email drafts directly to an external address. How should it be classified?
- A. Editorial, because one line changed
- B. Evidence-relevant, because the prompt may have changed
- C. Editorial, provided the drafts are unchanged
- D. Authority-relevant, because it changes actions and destinations
D. Authority-relevant, because it changes actions and destinations — Appendix F classifies by effect, not size. A new external destination is authority-relevant, and the book warns against calling a permission expansion editorial because the edit was small.
Today’s Assignment
Write Section 17 of your plan: the system card and release record for your company's chosen AI system, with its change classification.
- Divide the workflow into stages, marking bounded tasks you can evaluate and stages that need identified authority.
- Complete the Appendix F system card. Where a field has no evidence yet, say so rather than leave it blank.
- Complete the release record, noting which configuration identifiers your company can produce today and which it cannot.
- Report any evaluation with both denominators and its error pattern; if none exists, state what the protocol would test.
- Define the three change classes for your system with one example each, and name who classifies a change and how disagreement is resolved.
- Name the limitation your company would be most tempted to leave off the card. If you have no AI system yet, use the proposal under consideration or a tool staff already use informally.
YOUR 90-DAY PLAN
Section 17 — The system card and release record for our system.
How this is marked — the rubric
| Workflow divided into bounded tasks and authority-bearing stages, authority named by role | 20 |
| System card complete, with evidence gaps and material limitations stated | 30 |
| Release record identifies configuration, scope, approvals, conditions and rollback | 25 |
| Change classes defined with examples, a named classifier and a disagreement route | 25 |
| Total · pass mark 70 | 100 |
Submitting session 17
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 17, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.Capabilities Covered
Workflow divisionTwo-denominator reportingSource repair before model changeBound approvalSystem cards and release records
The Board · Chapter 29SESSION 18
Portfolio Control and the Board Report
Why This Matters
A board does not govern models. It governs the organisation's collective authority and exposure. This session moves from one release decision to the portfolio, and to the paper that reaches your board each quarter. The book's demand is uncomfortable for anyone used to presenting progress: report counts with their denominators, state what management does not know, and explain the pilot that failed in the same terms as the one that worked. A board paper that is all green can still mean very little. You will draft the board pack section of your plan, and Pop Quiz 4 opens at the end.
25-Minute Module
0:00 – 5:00
From approvals to a portfolio
- Portfolio governance connects individual approvals with shared dependencies, concentration, capability and total exposure. Twenty separate acceptable deployments can still create a material common risk if they depend on the same service, corpus or credential infrastructure.
- Keep a dependency view of vendors, models, clouds, data feeds, retrieval sources, identity services and specialist reviewers. Record which critical processes an outage, update or compromise would affect, and use it in continuity exercises.
- OPERATIONS · THE WORKED CASE
- Operations. A fictional gas processing company has three advisory maintenance models, each approved separately. If all three draw on the same data feed and identity service, one outage or compromise reaches all three. The approvals cannot show this; the dependency view can.
ENTERPRISE · THE PARALLEL- Enterprise. The same fictional company's drafting assistants for contracts, HR and regulatory work share one retrieval corpus. A revision-control defect there is one common risk, not three unrelated issues. The doctrine applies identically; only the dependency differs.
- THE DOCTRINE
The portfolio is governed when management can account for the organisation's collective authority and exposure, not merely recite a list of projects.
Ask which single supplier, if it failed tomorrow, would appear in more than one incident report.
5:00 – 11:00
Counts with denominators
- Report systems discovered, reviewed, operating within current approval and with unresolved evidence gaps, and include discovery coverage. 95% assessed means something different when the remaining 5% contains the most consequential systems.
- Show risk-weighted coverage beside simple counts and explain the weighting. A composite score must not hide unassessed high-consequence uses; the board sees those exceptions directly. Record approval expiry and overdue remediation: a documented system operating outside its current permission is different from an undocumented one, and from a confirmed harmful incident.
Dashboard term Appendix H definition Active systems Operating or available for use in the stated inventory scope Current approval Installed scope and configuration remain within an unexpired authorisation Assessed coverage Number assessed divided by the defined inventory denominator, with discovery coverage stated separately Material control failure A failed control affecting a consequential claim or boundary under the organisation's definitions Realised benefit An observed benefit with a stated comparison and cost basis, distinct from forecast or released capacity - Use consistent definitions between periods, or the trend line measures your vocabulary.
Every percentage on a board slide owes the board its denominator and a list of what is not in it.
11:00 – 17:00
The quarterly paper and delegated authority
- The pack covers material deployments and changes, evidence gaps, control failures, incidents and near misses, unresolved risk acceptances, realised benefit and upcoming decisions. It also states what management does not know.
- Appendix H's paper runs: decision requested; material change; current exposure; evidence state; control performance; incidents; value, with realised results, scenario estimates and costs kept separate; upcoming obligations; management recommendation with alternatives. Technical detail goes in an annex available for challenge, not out of the record.
- Training completions describe activity. Correct handling of a realistic challenge describes capability. A containment drill describes performance in a rehearsed scenario. None alone establishes effectiveness.
Decision Accountable authority Evidence or consultation Approve business purpose Business owner Alternatives and value case Authorise data use Designated data authority Rights, purpose, privacy and security review Accept technical readiness Technical release authority Validation and operational readiness Permit engineering consequence Domain authority Applicable engineering review Permit material external action Authorised business signatory Bound transaction and relevant approvals Accept residual risk Authority under risk delegation Independent challenge and explicit limitations Contain an incident Incident authority Urgency, scope and evidence preservation Reinstate the service Release and risk authorities Remediation evidence and reconciled consequences - The matrix is a proposal. Name actual people and the source of their authority. Combining roles in a smaller enterprise should not erase the distinction between decisions or the need for challenge.
If a board paper does not say what decision it wants, the board may make one you did not intend.
17:00 – 22:00
Limits, monitoring and the adverse result
- Management can limit autonomous spend, external actions, sensitive-data uses and reliance on a single provider, according to risk appetite and business needs. No numerical limit in the book is a universal industry standard. Fund discovery, bounded experiments and production separately, and keep the option to retire a system: revoke access, export records, preserve evidence, stop integrations.
- Appendix I: for each indicator record purpose, population, calculation, sampling, acceptable range, uncertainty, latency, owner and response. For tool calls outside approved destinations, a zero count requires confidence that logging and denial are functioning.
- A control test record compares expected with observed results and names scope, remediation and who can challenge. Review queue analysis compares arrivals, waiting time, peak load and unresolved cases with the approved staffing assumption: a system that safely defers difficult cases can still create harm if the queue cannot be handled.
- Explain a failed pilot by claim, evidence and decision: the model may not improve the target, the workflow may lack capacity, or cost may exceed benefit. Tell the board the cost, the lesson supported by evidence and the resulting action.
- THE DOCTRINE
Do not call every disappointing result a learning success without specifying what changed. Credibility depends on retaining the denominator of unsuccessful work.
A portfolio that has never reported a failure has not yet been honest with its board.
22:00 – 25:00
Before you close the laptop
- Write down the one AI number your company reports upward today, and its missing denominator.
- This session produces Section 18 of your 90-Day AI Governance Plan: the board pack, with counts and denominators, metrics and portfolio limits.
- Pop Quiz 4 — Release Decisions and the Board opens now: five questions on Sessions 16 to 18, closed book. Sit it before Session 19.
Write the paper you would want to receive as the director who must answer for it.
Key Terms
Discovery coverage
How much of the organisation's actual AI use the inventory has searched, stated separately so a high assessed percentage cannot hide what was never found.
Assessed coverage
Systems assessed divided by the defined inventory denominator, reported alongside discovery coverage.
Current approval
A system's installed scope and configuration remain within an unexpired authorisation. A documented system can still be operating outside it.
Delegated authority matrix
The table assigning each distinct AI decision to an accountable authority with the evidence it requires, naming actual people and their source of authority.
Self-Check — Before You Submit
1. A fictional refining company's draft board slide says 95% of AI systems are assessed. The unassessed 5% includes a plant-facing advisory tool and a procurement agent with payment access. What do you require?
- A. Nothing, since 95% is strong coverage
- B. The denominator, discovery coverage and the unassessed high-consequence uses shown as named exceptions
- C. A risk-weighted composite score in place of the percentage
- D. Removal of the figure until the 5% is assessed
B. The denominator, discovery coverage and the unassessed high-consequence uses shown as named exceptions — 95% assessed means something different when the remaining 5% holds the most consequential systems. A composite can hide exactly those uses, and removing the figure denies the board the exposure it governs.
2. In the book's terms, which is evidence of capability rather than activity?
- A. Ninety per cent of staff completed AI training
- B. The board approved the AI policy
- C. The inventory grew this quarter
- D. Reviewers correctly handled a realistic challenge scenario
D. Reviewers correctly handled a realistic challenge scenario — Training completions describe activity; correct handling of a realistic challenge describes capability. A policy approval and a growing inventory are activity records too.
3. A pilot at a fictional oilfield service company is stopped because review cost exceeded observed benefit. The lead proposes omitting it from the quarterly paper or calling it a valuable learning experience. What should the paper say?
- A. The claim, evidence, decision to stop, cost, lesson supported by evidence and resulting action
- B. Nothing, since stopped pilots are not material
- C. That it was a learning success, without detail
- D. That the vendor's product was unsuitable
A. The claim, evidence, decision to stop, cost, lesson supported by evidence and resulting action — The book asks for the adverse result in claim, evidence and decision terms. Omitting it, or calling it a learning success without saying what changed, destroys the denominator of unsuccessful work.
Today’s Assignment
Write Section 18 of your plan: the first quarterly AI board paper your company could honestly submit, with counts and denominators, metrics and portfolio limits.
- Give the counts: discovered, reviewed, within current approval and with unresolved gaps, each with its denominator and your discovery coverage.
- Define your dashboard terms following Appendix H, and name unassessed high-consequence uses as exceptions.
- Draft the paper in the Appendix H structure, including what management does not know, with realised results, scenarios and costs kept separate.
- Propose portfolio limits on autonomous spend, external actions, sensitive-data uses and single-provider reliance tied to your company's risk appetite, and complete the delegated authority matrix with actual names.
- Report one adverse result or stopped effort honestly in claim, evidence and decision terms. If your company has never reported one, say so and say why.
YOUR 90-DAY PLAN
Section 18 — The board pack: counts with denominators, metrics and portfolio limits.
How this is marked — the rubric
| Counts reported with denominators and discovery coverage, high-consequence exceptions named | 30 |
| Paper follows the Appendix H structure and states what management does not know | 25 |
| Portfolio limits tied to risk appetite, and a matrix naming people and their authority | 25 |
| An adverse result reported with claim, evidence, decision, cost and resulting action | 20 |
| Total · pass mark 70 | 100 |
Submitting session 18
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 18, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.ASSESSMENT NOW OPEN
Pop Quiz 4 — Release Decisions and the Board opens now. Sit it before you start the next session — it is how you find out whether the doctrine landed, and the practice papers may be resat as often as you like.
Sit Pop Quiz 4 →Closed book, no notes, and Coach Ritchie switches himself off while you sit it.Capabilities Covered
Portfolio exposureDenominator disciplineBoard paper constructionDelegated authorityReporting adverse results
Culture · Chapters 31 and 32SESSION 19
Capability Maturity, the Culture of Correction and What Remains Unresolved
Why This Matters
A governance programme can hold every template in this book and still fail if people learn that raising a defect costs them. This session is about the culture that keeps the records honest, and the limits no record removes. You will assess maturity by what actually happens rather than by a self-scored questionnaire, decide what each role must be able to do, adapt the book's model acceptable use policy, and write down plainly what your company cannot yet know about its system. That last part is not a weakness in the plan. It is what makes the rest of it believable. Practice Exam 4, the full-syllabus paper, opens at the end.
25-Minute Module
0:00 – 5:00
Maturity you can observe
- A policy can exist without being used. A process can be used without producing reliable evidence. A reliable record can exist without giving anyone authority to act. So the book assesses maturity by observable behaviour, in five stages that are the author's categories, not an external standard.
Stage What it means Visible Consequential uses and owners can be identified within the stated discovery coverage Defined Approval path, classification and evidence requirements are clear Operating Releases and changes follow the process Evidenced Controls and outcomes can be demonstrated Adaptive Findings lead to timely changes, and ineffective uses can be withdrawn - Assess by capability and scope, not one flattering overall score: an organisation can be adaptive in one domain and merely visible in another.
- THE DOCTRINE
An aggregate maturity average can conceal an immature high-consequence use behind well-controlled drafting assistants.
- Assess under ordinary conditions. Take a recent change, an expired approval or an inconvenient incident, and ask what actually happened.
The inconvenient incident will tell you more than last quarter's questionnaire.
5:00 – 11:00
Role literacy and the culture of correction
- Executives interpret the limits of evidence and risk acceptance. Technical teams understand data timing, uncertainty, evaluation and action boundaries. Domain reviewers know what they must verify. General users know approved use, information handling and escalation.
- OPERATIONS · THE WORKED CASE
- Operations. Test with realistic scenarios. In a fictional gas processing company, the engineer's test is whether she spots a unit mismatch or an excluded operating mode, the very conditions that defeat the Session 16 approval.
ENTERPRISE · THE PARALLEL- Enterprise. The purchasing reviewer must notice a changed beneficiary; the manager must distinguish a point estimate from a lower confidence bound; the user must recognise that an unsupported citation does not establish a fact.
- Training completion is evidence of attendance, not capability. Use exercises, sampled decisions and coaching.
- People should be able to report defects without being accused of undermining innovation. Preserve evidence of concern, investigate fairly and explain the action.
- THE DOCTRINE
An organisation that punishes discovery will reduce the visibility of risk rather than the risk itself.
- A challenge should still be specific: which claim is unsupported, which boundary is crossed, which observation changes the decision. Show that negative results can change investment. Link incidents, validation failures, overrides and vendor changes to the control catalogue, and keep known failure examples, distinguishing constructed tests from actual incidents.
Ask who in your company last stopped something on evidence, and what happened to them afterwards.
11:00 – 16:00
The acceptable use policy
- Appendix D is a model policy for adaptation: replace its bracketed roles and references, and obtain review through your normal policy and legal process. It covers employees, contractors and other authorised users, including AI embedded in existing software.
- Services and information. Approval of a service does not authorise every use; register new consequential uses. Do not enter information beyond the approved classification, or upload another project's records merely because the platform accepts them.
- Decisions and verification. Do not delegate a consequential decision or external action without explicit approval, or use an assistant to bypass payment approval, engineering authority, professional review, access controls or document control. Sign-off applies to the final content or transaction. Verify material claims against authoritative records, and never create an inspection record, certification, approval or result that did not occur.
- Conduct and disclosure. Do not present generated content as a verified measurement or professional determination without the required basis and authority, or claim endorsement, certification or qualifications without evidence.
- Incidents and accountability. Report harmful outputs, permission breaches, unauthorised actions and compromised sources, and preserve evidence. Managers must give enough time and information for meaningful review. Investigate non-compliance considering intent, training, system design and actual consequence.
A policy that demands verification without giving reviewers time is written to be broken.
16:00 – 21:00
What remains unresolved, stated honestly
- Serious governance distinguishes a working control from an unsolved research question. An unresolved question can justify a narrower scope, external review, additional research or refusal of the proposed authority.
Unresolved area What the book recommends Evaluation does not bound every capability A benchmark samples tasks; it cannot prove absence of a dangerous capability. State the environment and authority bounded by architecture Reinforcement learning and optimisation Define objective and constraints separately. An average constraint differs from a per-action hard limit, which the action service must enforce Mechanistic understanding A generated reasoning trace is not necessarily faithful to internal computation. Use checkable outputs, sources and action boundaries Broad social and environmental effects A scenario is not a forecast. Avoid a precise carbon or workforce claim unsupported by the actual service and use pattern - The book offers no general solution to safe reinforcement learning or unrestricted autonomous control. It constrains deployment authority to what the engineering and statistical evidence can justify.
- THE DOCTRINE
At release, say what you permit, what evidence supports it, which uncertainty remains and who will intervene. Where you cannot, narrow the claim or authority until you can.
Do not equate eloquent explanation with proven reliability, whether it comes from a model or a vendor.
21:00 – 25:00
Before you close the laptop
- Write down your company's maturity stage for your chosen system, and the recent event that proves it.
- This session produces Section 19 of your 90-Day AI Governance Plan: role literacy, the acceptable use policy and the limits you will state honestly.
- Practice Exam 4 — Full Syllabus opens now: fifteen questions across Sessions 1 to 19, closed book. Sit it before Session 20. Three Final Exam questions will be rebuilt from your own earlier mistakes, so read every returned explanation.
The limit you write down today is the one nobody can accuse you of hiding later.
Key Terms
Maturity stage
One of the author's five observable stages, visible, defined, operating, evidenced and adaptive, assessed by capability and scope rather than as one score.
Role-specific literacy
The capability each role needs, tested with realistic scenarios rather than training completion.
Per-action hard limit
A limit the action service enforces on every individual action, unlike an average constraint that can be met while one transaction is unacceptable.
Honest final permission
A release statement of what is permitted, its evidence, the remaining uncertainty and who will intervene. Where any part cannot be stated, the claim or authority is narrowed.
Self-Check — Before You Submit
1. An agent at a fictional gas marketing company meets its average monthly cost target, but one transaction it executed was unacceptable. Which control was missing?
- A. A per-action hard limit enforced by the action service
- B. A lower average cost target
- C. More training for users who read the monthly report
- D. A longer reasoning trace for each transaction
A. A per-action hard limit enforced by the action service — A policy can satisfy an average constraint and still create an unacceptable single transaction. A tighter average has the same flaw, and a reasoning trace is not necessarily faithful to what the system computed.
2. Which statement about the book's maturity stages is correct?
- A. They are an international standard to certify against
- B. They are the author's categories, assessed by capability and scope, and can differ between domains
- C. They should be averaged into one board score
- D. Approving policies makes an organisation adaptive
B. They are the author's categories, assessed by capability and scope, and can differ between domains — The book calls them the author's categories, not an external standard, and warns that an average can conceal an immature high-consequence use. An approved policy shows definition, not adaptation.
3. At a fictional refining company, a document controller reports that the drafting assistant cited a superseded procedure. The sponsor says she is slowing innovation and should wait for the quarterly review. What do you do?
- A. Agree, since one citation error is not material
- B. Ask her to correct the citation quietly
- C. Preserve the evidence, investigate fairly, explain the action, and check whether the corpus defect affects other assistants
- D. Suspend every AI tool until the quarterly review
C. Preserve the evidence, investigate fairly, explain the action, and check whether the corpus defect affects other assistants — Punishing discovery reduces the visibility of risk, not the risk, and one assistant's revision failure may expose a common corpus defect. Suspending everything is not a specific, proportionate response.
Today’s Assignment
Write Section 19 of your plan: your company's maturity assessment for the chosen system, role literacy tests, an adapted acceptable use policy and the limits you will state honestly.
- Assess the stage your company has reached for the chosen system, citing one recent change, expired approval or incident. Name a domain where you are further behind.
- Write one realistic scenario test each for executives, technical staff, domain reviewers and general users.
- Adapt Appendix D: name the policy owner, intake route, incident route and responders, and the clause your staff would find hardest to follow today.
- Describe honestly how a person who reports a defect is treated in your company today, and one change that would reward correction.
- Write your system's honest final permission, with one unresolved limit and how you will narrow the claim or authority because of it.
YOUR 90-DAY PLAN
Section 19 — Role literacy, the acceptable use policy and the limits we will state honestly.
How this is marked — the rubric
| Maturity stage assessed from a real recent event, with a weaker domain named | 20 |
| Four role-specific scenario tests that test capability rather than completion | 25 |
| Appendix D adapted with named owner, routes and responders, and the hardest clause identified | 25 |
| An honest final permission with an unresolved limit and the resulting narrowing | 30 |
| Total · pass mark 70 | 100 |
Submitting session 19
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 19, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.ASSESSMENT NOW OPEN
Practice Exam 4 — Full Syllabus opens now. Sit it before you start the next session — it is how you find out whether the doctrine landed, and the practice papers may be resat as often as you like.
Sit Practice Exam 4 →Closed book, no notes, and Coach Ritchie switches himself off while you sit it.Capabilities Covered
Observable maturityRole literacy testingRewarding correctionAcceptable use policyHonest limits
The Mandate · Chapter 30 · CapstoneSESSION 20
The First Ninety Days
Why This Matters
Nineteen sessions ago you chose one AI system or proposal in your company and separated what it claims from what it is being permitted to do. Today you sign the plan built around it. This session has three parts: the first ninety days as the book proposes them, the checklist that tells you whether a month is genuinely complete, and the capstone itself, nineteen sections assembled into one signed 90-Day AI Governance Plan. Ninety days will not give your company mature control of every use, and the book does not pretend otherwise. They can give you a working governance path and a small number of deployments you can show are controlled. Then the Final Exam.
25-Minute Module
0:00 – 6:00
Ninety days: a working path, not mature control
- The first ninety days should produce a working governance path and a small number of demonstrably controlled deployments. The sequence is a proposed method, to be adapted to your actual scale and risk.
Period Purpose Deliverables Days 1 to 30 Establish the boundary Mandate, initial inventory with coverage limits, role allocation, urgent-risk decisions, intake process Days 31 to 60 Create evidence and controls Evidence contracts, evaluation protocols, implemented controls, vendor decisions, tested response paths Days 61 to 90 Release narrowly and inspect Evaluations against pre-specified rules; approve, restrict or decline; first board report; independent walkthrough - Month one starts with a written mandate, an accountable executive and operational stop authority. Discover consequential uses first, including embedded features and supplier-supported decisions, and record uncertainty rather than claim completeness. Agree tiers, escalation and approval roles, and make disclosure of existing uses possible without unnecessary obstruction.
- THE DOCTRINE
A policy document alone does not complete the first month.
If your first thirty days end with a policy and no stop authority, you have written about governance rather than established it.
6:00 – 11:00
Inside the second and third months
- Choose one or two representative deployments for detailed walkthroughs. Trace their decision boundary, data and action rights, and contain urgent exposure while preserving continuity.
- OPERATIONS · THE WORKED CASE
- Operations. In a fictional upstream operator, one walkthrough might take an advisory maintenance model: trace what it reads and recommends, and confirm it has no route to change a work order.
ENTERPRISE · THE PARALLEL- Enterprise. The other might take the regulatory team's drafting assistant: trace its corpus and permissions, and whether anything it produces can leave the company without an authorised signatory.
- Days 31 to 60: repair missing data definitions, uncontrolled documents and broad tool permissions before tuning a model around those defects. Implement release records, change classes and outcome-linking. Define review capacity, fallback and containment, then run a review-capability exercise and a containment drill.
- THE DOCTRINE
A failed drill is a finding requiring remediation, not a reason to edit the report into a success.
- Days 61 to 90: start bounded production or shadow trials only where justified. Ask independent assurance to walk through one consequential outcome and one control. Reconcile the business case with observed workload and costs, and do not count estimated capacity release as realised cash merely because the pilot completed.
The second month is where enthusiasm meets the documents. Repair the documents.
11:00 – 16:00
Complete means a reviewer can see it operate
Appendix J Checklist, abridged Month one Mandate with named executive and stop authority; consequential uses with discovery limits; owners and dependencies; urgent unsupported authority contained; tiers agreed; intake and incident routes published Month two Evidence contracts; obligation map reviewed; data and source controls repaired; protocols frozen; action boundaries enforced; release and change records; review capacity tested; containment and fallback rehearsed Month three Evaluations with uncertainty; bounded releases approved or declined; monitoring producing real records; independent walkthrough; value reconciled; board pack with exceptions; next-quarter priorities Quarterly continuation Expand discovery; resolve high-consequence gaps; retest material changes; verify corrective actions; retire unsupported uses; update legal currency and training; preserve lessons from adverse results - Mark an item complete only when the artefact exists and a reviewer can see how it operates. Where a test fails, record the finding and interim restriction.
- Resource choices. A smaller enterprise can combine roles but should preserve challenge for consequential uses. A larger one may need a central policy and platform function with domain-specific approval.
- Avoid a large governance platform before you understand the process it will support; software can automate an unclear approval route. Start with a controlled method and scale the tooling when the need is demonstrated.
A ticked box with no artefact behind it is a statement, not evidence.
16:00 – 21:00
The ninety-day review, and a last word
- On day ninety, ask whether the company can identify its consequential uses, reconstruct an approval, show operating controls, contain a failure and report a limitation honestly. Evidence of those capabilities in the selected scope is a credible beginning, and the next quarter addresses the gaps.
- THE DOCTRINE
A calendar milestone should not become an excuse to declare maturity without operational evidence.
- The privilege of using powerful technology comes with the responsibility to examine it carefully. Ask what the system knows, what the evidence supports, where it can fail and who will act when it does.
- Let the evidence change your mind. Give your people the authority to challenge an attractive result. Build systems that can be questioned, corrected and stopped.
- Corrections and substantive disagreement are welcome. A defensible operating doctrine should improve when better evidence becomes available, including evidence from your company.
Intelligence earns its place in an organisation when leadership can account for its consequences.
21:00 – 25:00
The capstone and the Final Exam
- The capstone is Section 20: Sections 1 to 19 assembled into one signed 90-Day AI Governance Plan with a one-page executive summary on top. It will run well beyond the usual 900 words, and that is expected. The summary stays on its page.
- Two things close the programme, in order. First the capstone, submitted as Session 20. Then the Final Exam, sat on this page: fifteen questions, three rebuilt from your own earlier mistakes, closed book, pass mark 70. The certificate of completion requires the capstone and the Final Exam at 70 or above.
- Send me the plan even if two sections embarrass you. I would rather mark an honest document than a tidy one.
Sign it only when you would defend every permission in it to your own board.
Key Terms
90-Day AI Governance Plan
The signed document assembled from Sections 1 to 19 for one AI system or proposal, with a one-page executive summary and day-30, day-60 and day-90 gates.
Written mandate
The first-month authority naming the accountable executive and operational stop authority.
Ninety-day review
The test of whether the company can identify consequential uses, reconstruct an approval, show operating controls, contain a failure and report a limitation honestly.
Quarterly continuation
The Appendix J work after day ninety: wider discovery, closing high-consequence gaps, retesting changes and retiring unsupported uses.
Self-Check — Before You Submit
1. At a fictional gas processing company on day 52, the containment drill fails: suspension took far longer than planned and queued recommendations still reached the work-order system. The programme manager proposes marking the item complete because the drill took place. What do you do?
- A. Accept it, since the drill was rehearsed as required
- B. Mark it complete and note the delay in an annex
- C. Record a finding with an interim restriction, assign remediation and keep the item open
- D. Drop the drill requirement for the selected scope
C. Record a finding with an interim restriction, assign remediation and keep the item open — Appendix J says not to mark a capability complete because a drill occurred, and a failed drill is a finding requiring remediation. Moving the failure to an annex edits the report into a success.
2. On day 80 a pilot closes and the sponsor wants the board pack to show estimated hours released as a realised cash saving. What does the book require?
- A. Report the hours as cash, since the pilot completed
- B. Reconcile the business case with observed workload and costs, without counting estimated capacity release as realised cash
- C. Report them as cash with an estimate footnote
- D. Leave value out of the pack
B. Reconcile the business case with observed workload and costs, without counting estimated capacity release as realised cash — Chapter 30 says not to count estimated capacity release as realised cash merely because the pilot completed. A footnote does not correct a mislabelled figure, and omitting value withholds a decision input.
3. Which is one of the questions the book asks at the ninety-day review?
- A. Can the company reconstruct an approval
- B. Has every AI use reached the adaptive stage
- C. Has a governance platform been deployed
- D. Has every employee signed the policy
A. Can the company reconstruct an approval — The review asks whether the company can identify uses, reconstruct an approval, show controls, contain a failure and report a limitation honestly. Ninety days cannot deliver mature control everywhere, and a platform or signed policy shows activity.
Today’s Assignment
Assemble Sections 1 to 19 into one signed 90-Day AI Governance Plan for your company, with a one-page executive summary on top.
- Assemble all nineteen sections in order and edit for consistency. Where a later session changed your mind, correct the earlier section and say so in one line.
- Write the executive summary: the first system or decision governed, its claim and counterfactual, owner and approver, evidence contract, risk class, release controls, monitoring and incident route, and board reporting.
- Set the day-30, day-60 and day-90 gates from Appendix J, with the artefact a reviewer would see and the result that would make you restrict or decline.
- Name the accountable executive and stop authority, and state which roles your company will combine and where challenge is preserved.
- Sign and date the plan, and name the commitment you are least confident your company will keep, and why. The plan will run beyond 900 words; the summary may not. If your company has no AI system yet, use the proposal under consideration or a tool staff already use informally.
YOUR 90-DAY PLAN
Section 20 — The assembled, signed 90-Day AI Governance Plan with its executive summary.
How this is marked — the rubric
| All nineteen sections present, in order and consistent, with corrections noted | 25 |
| A one-page executive summary covering every required element for one named system | 25 |
| Day-30, day-60 and day-90 gates with reviewable artefacts and conditions to restrict or decline | 25 |
| Named executive and stop authority, preserved challenge, signature and an honest weakest commitment | 25 |
| Total · pass mark 70 | 100 |
Submitting session 20
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 20, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.FINAL EXAM
Your Final Exam is sat in this session. 15 questions, of which 3 are rebuilt from questions you got wrong on the practice papers. Sit it once your capstone is submitted.
Sit the Final Exam →Closed book, no notes, and Coach Ritchie switches himself off while you sit it.Capabilities Covered
Ninety-day sequencingChecklist disciplineResource choicesPlan assemblyAccountable leadership