THE INTELLIGENT AI GOVERNANCE LEADERSHIP PROGRAMME
Separate the claim from the permission.
20-Session Programme · 25 Minutes Per Day · Free · For leaders in Nigeria’s oil and gas sector
WEEK 2 OF 4 · TECHNICAL EVIDENCE THAT SURVIVES CHALLENGE · SESSIONS 6–10 · WORK AT YOUR OWN PACE
Lifecycle · Chapter 8SESSION 6
Inventory, Data and Lifecycle Control
Why This Matters
Week 1 gave you the claim, the permission, an owner, an evidence contract, a classification and an obligation map. Every one of those assumes you know which AI systems your company is actually using. Ask yourself whether you could list them today. The book is clear that AI can arrive inside software you already own, inside assistants your staff configure themselves, and inside supplier services that IT records as ordinary applications. Today we build the inventory that week 2 depends on, trace the data behind one system from source to decision, and decide which changes count as governance events. This is the least glamorous session in the programme and one of the most consequential. You cannot govern a system you have not found.
25-Minute Module
0:00 – 5:00
Systems in use, not models purchased
- The book is precise: an AI inventory should describe systems in use, not merely purchased models. That includes embedded AI, internally built pipelines, employee-configured assistants and external agents that handle your company's information.
- THE DOCTRINE
Discovery coverage is a separate metric from the number of items found. A long list tells you what was found, not what was missed.
- Give each system a stable record: purpose, owners, users, affected populations, model and vendor dependencies, data sources, action permissions, classification, approved scope and current release. Connect it to contracts, incidents and review dates.
- A spreadsheet can support an early programme if ownership and update discipline are clear.
When you are given an inventory count, ask where the team did not look.
5:00 – 10:00
Discover through several routes
- Compare five routes, because each misses something: procurement and expense records, platform configurations, data flows, user interviews and business-process reviews.
- The book names three ways systems hide. A vendor adds AI to existing software without a new purchase. A user configures an agent inside an approved suite. A business function relies on a supplier's ranking service while IT records it as an ordinary web application.
- Ask what decisions are influenced. Staff may not know whether a supplier uses a neural network, but they can tell you whether the software scores candidates, predicts faults or recommends prices. Technical classification can follow. Do not require perfect terminology before permitting disclosure.
- OPERATIONS · THE WORKED CASE
- Operations. In a fictional gas processing company, a maintenance software update adds a fault-prediction feature. Nothing was bought, so procurement never sees it; the platform configuration review does.
ENTERPRISE · THE PARALLEL- Enterprise. In the same fictional company, a team configures an assistant inside the approved office suite to draft correspondence. Only user interviews and process reviews find it. Same doctrine, different route.
- Report the uncertainty. The team may have reviewed all approved procurement items but only a sample of employee configurations. That is a defined coverage limit, and more useful than an unsupported declaration of enterprise completeness.
Ask your people what the software decides, not what it is.
10:00 – 16:00
Data provenance and timing
- Data lineage must connect a source to a release and a decision. Record who created the data, why, which rights permit its use, how it was transformed and when it became available.
- Timing is essential. A field recorded after an outcome can leak the answer into a retrospective prediction.
- Separate missing values from genuine zeros. The book's example: a zero flow reading may mean no flow, a failed instrument, or a missing feed mapped to zero by a pipeline. Those meanings require different responses. Record unit conventions, timezone handling, duplicate logic and sensor maintenance history where they affect the decision.
- THE DOCTRINE
A work order labelled failure may include preventive work, suspected defects or confirmed breakdowns. Training against that label estimates the recorded administrative outcome, not necessarily the physical phenomenon the sponsor thinks it represents.
Ask for the meaning of one zero in the training data.
16:00 – 21:00
Lifecycle evidence and change as a governance event
Lifecycle stage Evidence to retain Problem definition The decision and the alternatives Data preparation Rights, lineage and quality findings Development Experiment choices and configuration Validation Frozen protocol, test population and results Release Results connected to the installed configuration and permissions Operation Monitored outcomes and changes Retirement Access revoked; records managed under the applicable retention rules - Datasheets for Datasets and Model Cards for Model Reporting propose structured documentation. Their existence does not establish that the statements are correct. A reviewer should verify the statements material to the deployment [S13, S14].
- A new model is an obvious change. A revised system prompt, retrieval index, source policy, tool permission, decision threshold or user population can be equally consequential. Define change classes by their effect on the approved claim and control boundary.
Change, as the book gives it Response Minor editorial change to guidance Ordinary review New capability to send external messages Renewed authority review Model update the vendor calls routine Regression tests may still be required - Record the installed state, so that incident evidence can identify the configuration actually used.
Ask who would know if the vendor changed the model tomorrow.
21:00 – 25:00
Your wrap
- THE DOCTRINE
The inventory becomes valuable when it drives action. A list that is updated only before an audit is an archive, not an operating control.
- Before you close: write down how many routes your company has actually used to find AI, the one data field behind your Session 1 system whose timing or meaning you cannot confirm, and the one change that would require renewed authority review. That is Section 6 of your 90-Day AI Governance Plan.
- Then write your assignment.
An honest coverage limit is a finding; unsupported completeness is a liability.
Key Terms
Discovery coverage
The share of places AI could be in use that the team actually reviewed, reported separately from the number of systems found.
Data lineage
The record connecting a data source to a release and a decision: who created the data, why, under which rights, how it was transformed and when it became available.
Change class
A category of change defined by its effect on the approved claim and control boundary, which sets the review a change requires.
Installed state
The configuration actually in use at a given time, recorded so that incident evidence can identify what the system really was.
Self-Check — Before You Submit
1. After reviewing every procurement record, the technology team reports a complete inventory of 14 AI systems. What do you ask first?
- A. Which of the 14 systems carries the highest risk classification
- B. Which discovery routes were used, and what coverage limit follows
- C. Whether each system has a model card on file
- D. Whether the vendor contracts include AI clauses
B. Which discovery routes were used, and what coverage limit follows — Procurement misses AI added to existing software, staff-configured assistants and supplier services recorded as ordinary applications. Asking about risk first is tempting, but it ranks a list whose completeness is unknown.
2. In the training data for a fictional compressor model, many flow readings are zero. What does the book say about that?
- A. Zero is a valid measurement and should be kept
- B. Zeros should be removed before training
- C. Zeros should be replaced with the average reading
- D. Zero may mean no flow, a failed instrument or a missing feed mapped to zero, and each needs a different response
D. Zero may mean no flow, a failed instrument or a missing feed mapped to zero, and each needs a different response — Missing values must be separated from genuine zeros. Keeping, removing or averaging them are undocumented choices until the meaning is established.
3. A business unit wants its approved drafting assistant to send messages to external parties directly. The vendor calls it a simple setting. What is the correct response?
- A. Treat it as a change requiring renewed authority review before it is enabled
- B. Approve it through ordinary editorial review
- C. Enable it and record it in the next audit update
- D. Ask the vendor to confirm the setting is safe
A. Treat it as a change requiring renewed authority review before it is enabled — The book names a new capability to send external messages as needing renewed authority review: it moves the control boundary. A vendor's assurance is not your company's permission.
Today’s Assignment
Write Section 6 of your plan: your company's AI inventory, the data provenance behind your Session 1 system, and its change triggers.
- List the AI systems in your company found through at least three of the five routes, and state the coverage limit that results.
- Complete the stable record for your Session 1 system, using every field listed in this session, with owners named by role.
- Trace one critical input from source to decision, including when it becomes available, how missing values and zeros are handled, and what the label actually records.
- Define three change classes for the system, each with the review it triggers, and say where the installed state is recorded.
- The honest part: where does your company's current inventory or change control fall short, and who should own the fix?
- No AI system yet? Use a proposal under consideration or a tool staff already use informally. Aim for 400 to 900 words.
YOUR 90-DAY PLAN
Section 6 — Our AI inventory, data provenance and change triggers.
How this is marked — the rubric
| Uses at least three discovery routes and states a specific coverage limit rather than claiming completeness | 25 |
| Completes the stable record and names an accountable owner by role | 20 |
| Traces one input's lineage including availability time, rights, the missing-versus-zero rule and what the label records | 30 |
| Defines three change classes tied to the approved claim and control boundary, and names one real shortfall with an owner | 25 |
| Total · pass mark 70 | 100 |
Submitting session 6
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 6, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.Capabilities Covered
AI inventoryDiscovery coverageData lineageLabel meaningChange control
Technical Evidence · Chapters 9 and 10SESSION 7
The Target, the Metrics and the Cost of Error
Why This Matters
Somebody will soon show you a model that is 94.85% accurate. In the book's example, that model raises 575 alerts to find 80 failures, and a model that never raises an alert scores higher. Today is about the questions that stop a board approving the wrong number: what exactly was the model tested to predict, when was each piece of information available, and what does each kind of error cost your company. I am not asking you to run the calculations. I am asking you to recognise when they have not been run, and to check the few that matter on one page.
25-Minute Module
0:00 – 5:00
Write the target before the score
- A performance measure is meaningful only after the target, prediction time, population and decision horizon are defined. The question will this asset fail can conceal any work order, an unplanned stoppage, a confirmed component fault, or a failure requiring intervention within a specific period.
- Write an estimand: the quantity the analysis intends to estimate. The book's example: the probability of a confirmed unplanned failure within 14 days among equipment operating at the prediction cutoff. Ranking the ten assets most useful to inspect this week is a different objective, not validated with an interchangeable score.
- Report the target and data cutoff beside the result, with sample counts, event counts, exclusions and missingness. A reviewer should not have to infer them from a notebook.
- THE DOCTRINE
An approval based on reliable discrimination for one target cannot justify a different operational claim.
Ask your team to read the estimand aloud. If it changes between meetings, stop there.
5:00 – 10:00
Leakage, validation design and labels
- Leakage is information the model could not have had at prediction time. A diagnosis, repair description or final work-order code predicts well because it was recorded after the outcome. A random split does not remove it.
- Fit preprocessing within the training partition, and keep the final test apart from tuning: repeated inspection converts a test set into development data. One well or equipment unit appearing in both partitions can also leak.
- Use time-based validation for future outcomes and grouped validation for unseen sites, people or assets. The book's illustrative design trains on 18 months, tunes on three and tests once on the next three: teaching choices, not universal requirements.
- Evaluate operating modes separately: a model trained mainly on steady operation may perform poorly during startup; one validated on graduate hiring may not generalise to specialised offshore roles.
- No observed failure is not always a true negative: if maintenance follows an alert, the prevented failure is never seen. Separate observed failure, engineering confirmation, intervention and inferred avoided event, and adjudicate ambiguous labels with domain reviewers.
For every strong feature, ask when it exists in the real workflow.
10:00 – 15:00
A complete illustrative confusion matrix
- If 1% of observations contain a failure, a system that always predicts no failure is 99% accurate and detects none.
- The book's illustrative case, recomputed: 10,000 equipment periods, 100 confirmed failures. All inputs are illustrative.
Actual outcome Alert No alert Total Failure 80 (true positives) 20 (false negatives) 100 No failure 495 (false positives) 9,405 (true negatives) 9,900 Total 575 9,425 10,000 - Precision 80 / 575 = 13.91%. Recall 80 / 100 = 80%. False positive rate 495 / 9,900 = 5%; specificity 95%. Accuracy 9,485 / 10,000 = 94.85%. Appendix M adds F1 = 160 / 675 = 0.237, which ignores true negatives and the business cost of error.
- The model creates 575 alerts to detect 80 failures. Whether that is acceptable depends on review capacity, warning lead time, intervention cost and the consequence of a miss. Accuracy answers none of them.
For a rare event, ask for the four cells.
15:00 – 21:00
Prevalence, thresholds and the cost of error
- Precision depends on how common the event is. Appendix M, recomputed, with precision = s × pi / [s × pi + f × (1 − pi)]:
Assumed prevalence Sensitivity False positive rate Calculated precision 1% 80% 5% 13.91% 5% 80% 5% 45.71% 10% 80% 5% 64.00% - A sensitivity scenario, not a forecast: real shifts can change the error rates too. Saito and Rehmsmeier analyse why precision-recall views can be more informative than ROC for imbalanced data [S15], but no curve removes the need to choose and evaluate the deployment threshold and its workload.
- The simplified rule: act when C_FP(1 − p) < C_FN p. If inspection costs $1,000 and prevents an expected $50,000 loss with certainty, act above p = 0.02. If inspection enables prevention in only half of true failures, the threshold rises to 0.04. It assumes no inspection harm, no capacity constraint and a correctly estimated risk.
- OPERATIONS · THE WORKED CASE
- Operations. In a fictional upstream operator, lowering a pump model's alert threshold raises recall and the inspection queue together. The crew absorbing the alerts is part of the decision.
ENTERPRISE · THE PARALLEL- Enterprise. In a fictional oilfield service company, a model flags supplier invoices. A false positive costs reviewer time and delays payment; a false negative passes a wrong invoice. Same arithmetic, different costs and owner.
- THE DOCTRINE
Management should approve the consequence of the threshold, not only its numerical value. A threshold change can alter workload, harm and fairness even when the model remains unchanged.
Recall alone breeds alarms; precision alone hides misses.
21:00 – 25:00
Scenario is not evidence, and your wrap
- Appendix M's constructed maintenance case: first-year cost $234,500, benefit $15,000 per prevented failure. Ten prevented gives −$84,500; twenty gives $65,500. Break-even is 15.6333, so at least 16. That is not a prediction that 16 will be prevented; the prevention rate is the unresolved causal question.
- Its NPV example, $120,000 invested and $100,000 a year for three years at an assumed 15%, gives approximately $108,322.51. The 15% is a teaching assumption, not a Nigerian market benchmark, and if the $100,000 is priced staff time that cannot be redeployed or removed, the cash-flow reading is unsupported.
- Before you close: write your Session 1 system's estimand, and the cost of each kind of error in your company's terms. That is Section 7 of your 90-Day AI Governance Plan.
A precise figure on an undefined target is still undefined.
Key Terms
Estimand
The exact quantity an analysis intends to estimate, including target, population, prediction time and horizon.
Leakage
Information that would not exist at the moment of prediction in the real workflow, inflating apparent performance.
Precision
The share of alerts that turn out to be real. It falls when the event becomes rarer, even with the model unchanged.
Operating point
The threshold at which a model is deployed, with the workload and error trade-off it produces.
Self-Check — Before You Submit
1. Holding sensitivity at 80% and false positive rate at 5%, what is precision if prevalence is 5%?
- A. 13.91%
- B. 64.00%
- C. 45.71%
- D. 80.00%
C. 45.71% — 0.8 × 0.05 / (0.8 × 0.05 + 0.05 × 0.95) = 0.04 / 0.0875 = 45.71%. 13.91% is the 1% case; the table is a scenario, not a forecast.
2. A vendor shows a fault model with 99% accuracy where 1% of periods contain a failure. What do you ask for first?
- A. The confusion matrix and alert workload at the deployment threshold
- B. The area under the ROC curve
- C. A larger test set to confirm the 99%
- D. The F1 score
A. The confusion matrix and alert workload at the deployment threshold — At 1% prevalence an always-no model scores 99% and detects nothing. The four cells show detections, misses and workload. Area under the curve covers operating points the business may never use.
3. A team labels every alert followed by preventive maintenance a false alarm, since no failure occurred. What is wrong?
- A. Nothing, if the labels were applied consistently
- B. The prevented failure is unobserved; failure, confirmation, intervention and inferred avoided event must be separated
- C. Those cases should all be relabelled as true positives
- D. Those cases should be deleted from the test set
B. The prevented failure is unobserved; failure, confirmation, intervention and inferred avoided event must be separated — Calling prevented cases false alarms punishes useful intervention; calling every intervention a true positive rewards unnecessary work. Deleting them hides the problem.
Today’s Assignment
Write Section 7 of your plan: the target definition for your Session 1 system and the cost of each kind of error in your company.
- Write the estimand in one sentence, and one competing target someone in your company might assume instead.
- List the three strongest inputs and state when each exists in the real workflow. Flag any that could leak the outcome.
- Sketch the four confusion-matrix cells, with labelled assumptions where you lack data, and state the alert workload and who absorbs it.
- Describe the cost of a false positive and a false negative in your company's terms, including non-financial consequences, and name the executive who approves the threshold's consequence.
- The honest part: what performance figure has your company accepted without a target, threshold or workload attached?
- No AI system yet? Use a proposal under consideration or a tool staff already use informally. Aim for 400 to 900 words.
YOUR 90-DAY PLAN
Section 7 — The target definition and the cost of each kind of error for our system.
How this is marked — the rubric
| States a one-sentence estimand with target, population, prediction time and horizon | 25 |
| Checks the timing of the strongest inputs and identifies any leakage risk | 20 |
| Sets out the four confusion-matrix cells with labelled assumptions and the resulting alert workload | 25 |
| Describes both error costs in the company's terms and names the executive who approves the threshold's consequence | 30 |
| Total · pass mark 70 | 100 |
Submitting session 7
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 7, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.Capabilities Covered
Target definitionLeakage checksConfusion matrixPrecision and prevalenceThreshold approval
Technical Evidence · Chapters 11 and 12SESSION 8
Statistical Uncertainty and Calibration
Why This Matters
Yesterday you asked what a number measures. Today you ask how much it can be trusted. A recall of 80% on 100 failures is a narrower claim than it sounds, and a test with no harmful outputs in 300 trials is not proof of safety. Your technical team will present results with confidence. Your job is to ask for the interval, the unit of sampling and the rule that was fixed before the test, and to know that a model can rank well while overstating risk. I will be plain about one thing: you do not need statistical significance to stop a system that has done something dangerous.
25-Minute Module
0:00 – 5:00
A score is a measurement, not a property
- THE DOCTRINE
A performance estimate is not a property engraved on a model. It is a measurement from a sample under a protocol.
- Sample variation, dependence, selection and label quality decide how much confidence it deserves. Report intervals with the estimate and state the unit of sampling.
- Ten thousand sensor readings from one equipment event are not ten thousand independent failure examples.
- If independence is across assets, sites or events, the uncertainty analysis must respect that structure.
Ask how many independent events sit behind the headline number.
5:00 – 11:00
Two calculations a leader can check
- The proportion interval. In the book's illustrative independent sample, the model detects 80 of 100 failures: recall 0.80. The two-sided 95% Wilson interval, recomputed:
Step Value Observed recall p_hat, sample n, z 0.8, 100, 1.96 z² 3.8416 Denominator d = 1 + z²/n 1.038416 Centre c = (p_hat + z²/(2n)) / d approximately 0.7889 Half-width h approximately 0.0778 Interval approximately 0.711 to 0.867 - If the approving rule requires the lower bound to exceed a 78% minimum, an observed 80% provides limited evidence: the lower bound is 71.1%. State the rule before testing. Choosing a point estimate or lower bound after seeing the result is an avoidable source of bias.
- Wilson assumes independent binomial sampling. Clustered failures, repeated observations, uncertain adjudication or a selected sample can break it; a cluster bootstrap may be needed, justified rather than applied automatically.
- Zero observed failures. With no harmful outputs in 300 independent representative trials, the one-sided 95% upper bound is 1 − 0.05^(1/300), approximately 0.9936%. The often-used approximation 3/n gives 1%.
- It applies to the defined test sampling process only. It does not establish safety in an open-ended environment or for hazardous conditions excluded from the test.
When you hear zero failures, ask how many trials, drawn from what.
11:00 – 15:00
Comparing systems, and when to stop
- Testing two systems on the same cases? Keep the paired design: the uncertainty of the difference depends on the cases where outcomes differ.
- Define a minimum practically useful improvement. A tiny difference can become statistically detectable in a large sample without justifying the cost of changing the workflow. Report effect size and uncertainty together.
- Test many metrics, subgroups and thresholds and a favourable result becomes likelier by chance. Exploratory findings can motivate a follow-up test, not a confirmed release claim.
- Checking a pilot repeatedly and stopping when the result looks good changes the error properties of a fixed-sample test. Plan interim reviews with a sequential method, or treat them as operational checks.
- THE DOCTRINE
Safety containment should not wait for statistical significance. One serious permission breach can justify stopping an agent even when the sample is too small for a population-rate estimate.
The right to stop is an authority question, not a statistical one.
15:00 – 21:00
Calibration and uncertainty in the workflow
- Discrimination asks whether the model ranks positive cases above negative ones. Calibration asks whether a stated probability matches what actually happens. If a group of equipment periods is scored 0.8 but only 0.4 produce the event, an expected-loss rule will overstate the benefit of intervening. Guo and colleagues support treating calibration separately from accuracy, without establishing that one procedure works everywhere [S16].
- Ask for reliability plots with sample counts and uncertainty, on data not used to fit the calibrator. The Brier score is not a pure calibration measure. Ovadia and colleagues found uncertainty methods can deteriorate under dataset shift [S17]. Conformal prediction gives marginal coverage under assumptions such as exchangeability, not coverage for every subgroup, and wide intervals can be useless [S18].
- Do not take a vendor's uncertainty label as direct measurement of aleatoric or epistemic uncertainty without examining the method. A low uncertainty score is not proof the input is familiar.
- OPERATIONS · THE WORKED CASE
- Operations. In a fictional gas processing company, a high-uncertainty risk score triggers specialist review or a new measurement, not the same alert as every other score.
ENTERPRISE · THE PARALLEL- Enterprise. In the same company, a drafting assistant abstains on uncertain cases. Measure the share it handles, error among handled cases and the outcome of deferred ones: refusing difficult work flatters reliability.
- THE DOCTRINE
An honest uncertainty estimate supports bounded permission. It does not remove the need for fallback when the estimate itself is unreliable.
If every uncertainty level leads to the same action, the display is decoration.
21:00 – 25:00
Your wrap
- Before you close: write the decision rule your Session 1 system must meet, stated before any test, including whether it applies to the point estimate or the lower bound. Then write what action high uncertainty triggers. That is Section 8 of your 90-Day AI Governance Plan.
- Validation is complete when a reviewer can see the sample, protocol, estimate, uncertainty, exclusions and intended inference.
- Pop Quiz 2 opens now: five questions on Sessions 6 to 8, closed book. Sit it before you start Session 9.
An impressive score without its foundations is hard to defend.
Key Terms
Wilson interval
A range around an observed proportion, such as recall, that shows how precisely a sample of a given size estimates it under stated sampling assumptions.
Unit of sampling
The independent thing actually counted, such as events, assets or sites, which sets how much evidence a sample holds.
Calibration
Agreement between a model's stated probabilities and observed frequencies in a defined population. A model can rank well and still be poorly calibrated.
Coverage of an abstaining system
The proportion of cases a system handles rather than defers, reported with error among handled cases.
Self-Check — Before You Submit
1. A generative assistant produced no harmful outputs in 300 independent representative trials. What can you conclude?
- A. The assistant is safe for deployment
- B. Under the test distribution, a one-sided 95% upper bound of about 0.99% on the harmful-output probability
- C. The harmful-output rate is exactly zero
- D. The deployment harmful-output rate is below 1%
B. Under the test distribution, a one-sided 95% upper bound of about 0.99% on the harmful-output probability — 1 − 0.05^(1/300) ≈ 0.9936%, under the test distribution. Reading it as a deployment rate is the tempting error: excluded hazardous conditions are outside what it says.
2. Your evidence contract requires recall's lower bound to exceed 78%. The model detects 80 of 100 failures and the team reports a pass. What do you do?
- A. Accept the pass, because the point estimate exceeds the requirement
- B. Switch the rule to the point estimate, since the result is close
- C. Approve it with a note to retest later
- D. Return it: the Wilson lower bound is about 71.1%
D. Return it: the Wilson lower bound is about 71.1% — The rule was stated before testing and refers to the lower bound. Changing the rule after seeing the result is the avoidable bias the book warns about.
3. In a 40-run pilot, an agent commits one serious permission breach. The team says the sample is too small to estimate a rate. What is the right decision?
- A. Contain or stop the agent now; containment does not wait for statistical significance
- B. Continue to 300 runs to estimate the rate properly
- C. Apply a Wilson interval before deciding
- D. Record it as an outlier and continue
A. Contain or stop the agent now; containment does not wait for statistical significance — Statistical uncertainty and the authority to contain an observed hazard are different questions. The team is right about the estimate and wrong about the decision.
Today’s Assignment
Write Section 8 of your plan: how uncertain the evidence for your company's system is, and how uncertainty enters the workflow.
- State the unit of sampling behind the main performance figure and how many independent events it rests on.
- Write the decision rule the system must meet before testing: the metric, the threshold, and whether it applies to the point estimate or a lower bound.
- Say whether calibration has been checked on held-out data, and whether the business uses scores as rankings or probabilities.
- Define what high uncertainty triggers (specialist review, a new measurement or abstention), and how deferred cases are absorbed.
- Name who in your company can contain the system on one serious observed hazard.
- The honest part: where has your company accepted a point estimate without an interval, or moved the goalposts after a result? Aim for 400 to 900 words; no AI system yet means using a proposal or a tool staff already use informally.
YOUR 90-DAY PLAN
Section 8 — How uncertain our evidence is and how uncertainty enters the workflow.
How this is marked — the rubric
| Identifies the unit of sampling and the number of independent events behind the main figure | 20 |
| States a pre-specified decision rule that says whether it applies to the point estimate or a lower bound | 30 |
| Links uncertainty to a defined action and a capacity for deferred cases | 25 |
| Names, by role, who can contain the system on an observed hazard, and one real weakness in current practice | 25 |
| Total · pass mark 70 | 100 |
Submitting session 8
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 8, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.ASSESSMENT NOW OPEN
Pop Quiz 2 — Targets, Metrics and Uncertainty opens now. Sit it before you start the next session — it is how you find out whether the doctrine landed, and the practice papers may be resat as often as you like.
Sit Pop Quiz 2 →Closed book, no notes, and Coach Ritchie switches himself off while you sit it.Capabilities Covered
Interval readingZero-event boundsPre-specified rulesCalibrationRight to contain
Technical Evidence · Chapters 13 and 14SESSION 9
Distribution Shift and Fairness
Why This Matters
A model that passed validation last quarter was tested on last quarter's world. Equipment ages, operating modes change, people respond to its recommendations, and the system can change the very data used to judge it. Today has two halves. First, how to tell a drift signal from a verdict, which triggers should make your company review, restrict, recalibrate or suspend, and what happens when data does not arrive. Second, fairness: how one set of predictions can be fair by one measure and unfair by another, and why the honest answer is an explicit trade-off rather than a reassuring dashboard. Both halves end with someone in authority deciding.
25-Minute Module
0:00 – 5:00
Drift is a signal, not a verdict
Kind of change What changes Covariate shift The input distribution Label shift Outcome prevalence, under particular assumptions Concept drift The relationship between inputs and outcomes - Real deployments can show several at once, and observed data may not separate them cleanly.
- THE DOCTRINE
An input-distribution alarm does not establish performance deterioration. A changed sensor range may reflect an approved operating change, and stable inputs can hide a changed relationship.
- Choose indicators tied to a risk hypothesis: missingness, unit changes, range violations, operating-mode mix, scores and delayed outcomes. An arbitrary population-stability-index threshold is not a universal safety limit, and a distance without a response rule creates an alert queue nobody can interpret.
For every drift alarm, ask what it means for the approved claim.
5:00 – 10:00
Feedback changes what you can see
- If teams inspect only the highest-ranked assets, confirmed defects concentrate among alerted cases because those are observed more closely. The model looks better than it is.
- Where ethically and operationally appropriate, use a structured verification sample of unalerted cases or existing inspection records. Never withhold required safety inspection merely to create an experiment.
- Successful intervention prevents the event the model predicts. Record interventions, or retraining can teach the system that useful warning patterns are harmless.
- Revalidation triggers, as the book proposes them: a materially changed operating mode, input definition, population, label process, model, vendor behaviour or action boundary. Each trigger should specify whether the system is reviewed, restricted, recalibrated or suspended.
- Recalibration and retraining are both changes requiring controlled evidence. Keep a stable regression suite and a current operational sample, for different purposes.
Ask how the team learns about failures the model did not flag.
10:00 – 14:00
Safe degradation
- Specify what happens when required data is late or invalid: stop recommending, use a restricted advisory mode, or return the decision to an existing process. Avoid undocumented substitution, such as treating missing flow as zero or using an old document without warning.
- OPERATIONS · THE WORKED CASE
- Operations. In a fictional upstream operator, a pump model's sensor feed fails overnight. The documented response: stop scoring and return to the existing inspection routine.
ENTERPRISE · THE PARALLEL- Enterprise. In the same company, a drafting assistant's source library has not refreshed. It flags the stale source or stops, and the team drafts manually. Identical principle, different dependency.
- Test the fallback for the actual workload. If manual handling cannot absorb the deferred cases, the system needs a capacity plan and a prioritisation rule.
- The technical team detects change, domain specialists interpret it, and the business owner changes authority or resources. The response is incomplete until those connect.
Run the fallback once on a real day. You will learn whether it is a plan or a paragraph.
14:00 – 21:00
Fairness with explicit trade-offs
- Demographic parity compares positive-decision rates across groups; equal opportunity, true-positive rates; equalised odds, true-positive and false-positive rates; predictive parity, precision. They can disagree on the same predictions. Hardt, Price and Srebro formalise equality of opportunity; Kleinberg, Mullainathan and Raghavan show tensions between specified calibration and balance criteria under differing base rates. Neither proves fair treatment impossible or settles legal requirements [S19, S20].
- The book's illustrative comparison, recomputed. Both groups have 1,000 people and the model behaves identically in each:
Assumed figure Group A Group B Meet the qualification criterion 200 100 True-positive rate / false-positive rate 80% / 10% 80% / 10% True positives 160 80 False positives 80 90 Positive decisions 240 170 Precision 66.67% 47.06% - Equal error rates coexist with different selection rates and precision because assumed prevalence differs. That is not evidence the prevalence difference is legitimate: the criterion, access to opportunity and labels may be flawed. A label such as previously hired or previously promoted records past organisational decisions, not future capability.
- Bias can enter anywhere from outreach and labels to thresholds and how managers use outputs. Removing a protected attribute does not remove proxies. Establish the legal basis, access restrictions and purpose before collecting sensitive audit data.
- THE DOCTRINE
Treat missing subgroup evidence as a limitation rather than a pass. Absence of a statistically significant disparity does not establish equality, particularly with limited power.
- Interventions: repair the process, improve labels, collect evidence, change the task, adjust thresholds or withdraw the use. Thresholds are not an automatic fix. Include a route for affected people to seek review.
Ask which fairness measure was chosen, why, and which harm it misses.
21:00 – 25:00
Your wrap
- Before you close: write three revalidation triggers for your Session 1 system, each with its response, and one sentence on what the system does when its key input fails. If the system affects people, add which fairness measure matters and why. That is Section 9 of your 90-Day AI Governance Plan.
- The book ties fairness to the decision, the affected people and the relevant law. Use the affected populations in your Session 6 inventory record to say who that is for your system.
- Then write your assignment.
Shift and fairness are operating issues, not finished by a technical report.
Key Terms
Concept drift
A change in the relationship between inputs and outcomes, which can degrade a model while inputs look unchanged.
Revalidation trigger
A defined change, such as a new operating mode, population or model, that requires review, restriction, recalibration or suspension.
Safe degradation
Documented, tested behaviour when required data is late or invalid, including how work returns to an existing process.
Equalised odds
A fairness criterion requiring equal true-positive and false-positive rates across groups. It can coexist with unequal precision when base rates differ.
Self-Check — Before You Submit
1. A fictional compressor model's input distribution shifts sharply the week after an approved change of operating mode. What should happen?
- A. Suspend the model automatically
- B. Assess it against labelled outcomes and domain review for its significance to the approved claim
- C. Ignore it, because the operating change was approved
- D. Retrain on the new data immediately
B. Assess it against labelled outcomes and domain review for its significance to the approved claim — Drift is a signal, not a verdict. A materially changed operating mode is itself a revalidation trigger, so ignoring it is wrong; retraining at once is a change needing controlled evidence.
2. In the book's subgroup example, both groups have 80% true-positive and 10% false-positive rates. Why is precision 66.67% in Group A but 47.06% in Group B?
- A. The model is biased against Group B
- B. Group B has fewer people
- C. Group B has more false negatives
- D. The assumed share meeting the criterion differs, 200 against 100
D. The assumed share meeting the criterion differs, 200 against 100 — With equal error rates, different prevalence produces different precision. It shows neither that the model is biased nor that the prevalence difference is legitimate, so examine the label process.
3. A vendor's fairness report on a recruitment screening tool shows no significant disparity for a subgroup of 12 applicants. How do you read it?
- A. As a limitation: with so few people, no significant disparity does not establish equality
- B. As a pass for that subgroup
- C. As evidence the tool is fair overall
- D. As a reason to remove the subgroup from reporting
A. As a limitation: with so few people, no significant disparity does not establish equality — Missing or underpowered subgroup evidence is a limitation, not a pass. Report counts and intervals, and treat intersections and small groups with care.
Today’s Assignment
Write Section 9 of your plan: revalidation triggers, safe degradation and the fairness trade-off for your company's system.
- Name the drift indicators you would monitor, the risk hypothesis each connects to, and who interprets and decides.
- Write at least three revalidation triggers from the book's list, each with its response: review, restrict, recalibrate or suspend.
- Describe safe degradation for one critical dependency: what happens when data is late or invalid, which process takes over, and whether it has the capacity.
- State whether the system's observed outcomes are selective, for example only inspected cases, and how you would reduce that without withholding required safety inspection.
- Fairness: name the people the system affects, the measure you would use and why, and the harm it does not capture. If you conclude no people are materially affected, say so and show the evidence.
- The honest part: which of these would fail in your company today? Aim for 400 to 900 words; no AI system yet means using a proposal or a tool staff already use informally.
YOUR 90-DAY PLAN
Section 9 — Our revalidation triggers, safe degradation and fairness trade-off.
How this is marked — the rubric
| Writes three or more triggers, each with a specific response and a named decision-maker by role | 30 |
| Describes a tested or testable fallback with a capacity check for the workload it returns | 25 |
| Addresses selective labelling without proposing to withhold required safety inspection | 20 |
| States a fairness measure, its rationale and the harm it does not capture, or evidences why no people are materially affected | 25 |
| Total · pass mark 70 | 100 |
Submitting session 9
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 9, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.Capabilities Covered
Drift interpretationRevalidation triggersSafe degradationFairness trade-offsSubgroup evidence
Technical Evidence · Chapters 15 and 16SESSION 10
Explanation, Causation and the Claim of Business Improvement
Why This Matters
This is where the week's technical evidence meets the claim your sponsor actually wants to make: that the system will improve the business. An attractive explanation chart does not show what causes failures, and a good prediction test does not show that deploying the system improves outcomes. The book's own illustration shows a pilot that looks like a clear success only because the assisted team received easier work. Today you learn to separate what the prediction says, what the explanation indicates, what the causal evidence supports and what your company permits, and to read the book's Simpson's paradox until you can spot one in a board paper. Then we close Week 2 and you sit Practice Exam 2.
25-Minute Module
0:00 – 5:00
Define the explanation question
- Explanation serves different purposes: debugging, helping an operator examine an output, supporting a decision notice, or testing for an inappropriate signal. Each needs different evidence.
- LIME builds local surrogate explanations; SHAP attributes a model's output to features using a framework connected to Shapley values. Both can reveal model behaviour. Neither automatically identifies the physical or social cause of an outcome [S21, S22].
- Which measured variables contributed to this risk score? is a model-behaviour question. Will changing this control setting reduce failure risk? needs a causal and engineering argument. A high attribution may be causal, a proxy, a consequence or an artefact of data collection. Ask for the explainer, background sample and interpretation on record.
- Test explanation quality: fidelity to the model, stability and sensitivity to meaningful change. Adebayo and colleagues propose sanity checks that test whether explanations respond to model and data randomisation [S23]. Do not treat a language model's description of its own reasoning as a faithful trace of its computation.
Ask which question the explanation is meant to answer. A chart presented without one cannot be judged.
5:00 – 10:00
Causal claims need identification
- Observing that maintained assets have more failures may simply reflect maintenance being assigned to risky assets. It does not mean maintenance causes failure. Association is not intervention.
- State the causal estimand. The book's example: the average effect of a new alert-and-intervention policy on downtime within the eligible asset population. Identify the comparison, confounders, timing, interference and observation process. Pearl and Mackenzie provide an accessible reference for the distinction [S24].
- The book's operating practice: require a written identification argument before accepting a causal benefit claim.
- Suppose a model attributes much of a risk score to discharge temperature. That does not authorise lowering temperature outside the equipment's operating envelope. A domain authority must examine the physical mechanism; the model may be detecting a symptom rather than a lever.
- THE DOCTRINE
A good assurance report separates what the prediction says, what the explanation indicates, what the causal evidence supports and what the organisation permits.
When an explanation chart is followed by a proposal to change a setting, ask for the engineer.
10:00 – 15:00
Prediction is not business improvement
- A retrospective test establishes performance for a defined sample. The deployed intervention includes the model, users, action policy and supporting resources. The book's programme example: new alerts, an engineer's review, spare-parts planning and faster approval. If all four change together, the result belongs to the programme, not the model alone.
- Randomisation can reduce selection bias, but the unit of assignment matters. A trial with 20 sites is not equivalent to 20,000 independent work orders simply because each site contributes many records. Never withhold required safety controls or professional care to run an experiment.
- A before-and-after comparison can be confounded by season, market conditions, asset condition, staffing or another intervention. Difference-in-differences needs a credible parallel-trends argument; matching cannot remove unmeasured confounding automatically. A model that selects high-risk assets for treatment creates selection by design.
Ask what else changed during the pilot, and write the answer into the evidence record.
15:00 – 21:00
An illustrative Simpson's paradox
- The book's assumed counts, recomputed:
Cases AI-assisted workflow Comparison workflow Easy 81 of 90 (90%) 19 of 20 (95%) Difficult 4 of 10 (40%) 40 of 80 (50%) Aggregate 85 of 100 (85%) 59 of 100 (59%) - The assisted workflow is worse within each stratum and better in aggregate, because it received many more easy cases. Case mix reversed the comparison. Stratification does not always solve confounding; the right analysis depends on the causal structure and the decision.
- OPERATIONS · THE WORKED CASE
- Operations. In a fictional gas processing company, a maintenance pilot reports less downtime. Were pilot assets easier, and did spare-parts planning change too?
ENTERPRISE · THE PARALLEL- Enterprise. In the same company, a drafting assistant shows more first-time document approvals. Did the pilot team choose simpler documents? The paradox works identically.
- Measure outcomes and whether the intervention happened. If alerts go unactioned for lack of spare parts, a null result shows the programme failed under those conditions, not that the signal was useless. Unusual pilot attention can flatter results, so record operational intensity. Do not exclude unsuccessful users without a pre-specified rationale: intention-to-treat and per-protocol analyses answer different questions.
Whenever an aggregate comparison looks decisive, ask for the same comparison by difficulty.
21:00 – 25:00
The scale decision, and your wrap
- Expansion needs evidence on performance, operating capacity, cost, control effectiveness and transfer to new contexts, and a plan stating what will be retested and what remains bounded.
- THE DOCTRINE
The most useful experimental record includes the result that would lead management to stop. A programme designed only to justify a predetermined launch cannot establish a credible business improvement claim.
- Before you close: write the business improvement your sponsor claims, the comparison that would test it, and the result that would make you stop. That is Section 10 of your 90-Day AI Governance Plan.
- This closes Week 2. Practice Exam 2 opens now: fifteen questions across Sessions 1 to 10, closed book. Sit it before you start Session 11.
Write the stopping result down before the pilot starts. Afterwards, everyone remembers it differently.
Key Terms
Causal estimand
The precise effect a causal claim intends to estimate, in a defined population.
Identification argument
The written case for why a comparison isolates the effect claimed, naming the comparison, confounders, timing and observation process.
Simpson's paradox
A reversal in which one option is worse within every subgroup but better in aggregate, because the subgroups are mixed in different proportions.
Intention-to-treat
An analysis that keeps everyone offered the programme, including unsuccessful users, to estimate the effect of offering it. Per-protocol analysis answers a different question.
Self-Check — Before You Submit
1. In the book's illustration, the AI-assisted workflow succeeds on 85 of 100 cases against 59 of 100 for the comparison, yet it is worse among both easy and difficult cases. What explains the aggregate result?
- A. The assisted workflow is genuinely better overall
- B. The difference is random noise
- C. The assisted group contains many more easy cases
- D. The difficult cases were mislabelled
C. The assisted group contains many more easy cases — Easy cases are 90 of 100 for the assisted group and 20 of 100 for the comparison. Case mix reversed the comparison. Treating 85% against 59% as proof of benefit is exactly the trap.
2. A SHAP chart shows discharge temperature dominates a fictional compressor's risk score. The plant team proposes lowering it. What do you decide?
- A. Approve, because the attribution identifies the cause
- B. Hold it for the domain authority to examine the physical mechanism and permitted control action
- C. Approve a small reduction as a trial
- D. Ask the data team for a second explainer to confirm
B. Hold it for the domain authority to examine the physical mechanism and permitted control action — Attribution describes model behaviour, not a control lever; the model may be detecting a symptom. A second explainer still answers only a model-behaviour question.
3. A pilot shows no reduction in downtime; many alerts were not acted on because spare parts were unavailable. What has been established?
- A. The predictive signal is useless
- B. The model should be retrained
- C. The pilot should be rerun excluding ignored alerts
- D. The programme as implemented did not produce the claimed result under those conditions
D. The programme as implemented did not produce the claimed result under those conditions — Process measures show the intervention did not happen. That does not prove the signal useless. Excluding ignored alerts without a pre-specified rationale introduces selection.
Today’s Assignment
Write Section 10 of your plan: what would prove the business improvement claimed for your company's system, and what would not.
- State the business improvement claimed, as a causal estimand: the effect, the population and the outcome.
- Describe the full intervention (model, users, action policy and supporting resources) and say which parts the evidence could attribute to the model alone.
- Propose a comparison: randomised where appropriate and permissible, or observational with its assumptions stated. Name two confounders and the unit of assignment.
- Name the process measures that would show whether the intervention actually happened, and the result that would make management stop.
- The honest part: where has your company accepted a before-and-after or an explanation chart as proof of benefit? Aim for 400 to 900 words; no AI system yet means using a proposal or a tool staff already use informally.
YOUR 90-DAY PLAN
Section 10 — What would prove our claimed business improvement, and what would not.
How this is marked — the rubric
| States the claim as a causal estimand with effect, population and outcome | 20 |
| Separates the model from the rest of the intervention and limits attribution accordingly | 20 |
| Proposes a comparison with its unit of assignment, two named confounders and no withheld safety controls | 30 |
| Names process measures and a pre-stated stopping result, and one real example of an accepted weak claim | 30 |
| Total · pass mark 70 | 100 |
Submitting session 10
Write your answer in Word or Google Docs. The first line must read
Write your answer in Word or Google Docs. The first line must read
Session: 10, or the submission is refused. Upload the Word file (.docx), or switch to the Google Doc tab and paste your document’s share link.Marked against the rubric above and returned as a written letter, saved to your progress page. Do not submit until you have passed the self-check above.ASSESSMENT NOW OPEN
Practice Exam 2 — Midpoint opens now. Sit it before you start the next session — it is how you find out whether the doctrine landed, and the practice papers may be resat as often as you like.
Sit Practice Exam 2 →Closed book, no notes, and Coach Ritchie switches himself off while you sit it.Capabilities Covered
Explanation purposeCausal identificationSimpson's paradoxPilot designScale decision