Block A
Cold scenario
The brief arrives live. Fifteen minutes to form a position, then facilitated cold-calling and cross-examination. Escalations drop in as you get comfortable. Discomfort is the mechanism.
Scenarios: AI Engineering Under Real Conditions
Every session, you are put in a room where something has gone wrong and asked what you would do about it. The brief is incomplete. Someone senior is unhappy. There is no obviously correct answer.
Created by Baljeet Dogra
Course Objectives
The other programmes teach you to build systems. This one teaches you to decide when the situation is ambiguous, the data is incomplete, and people are watching. That is what the job actually is, and it is the thing interviews try — usually badly — to test for.
- Attack a situation you do not understand: what to ask first, how to speak without rambling, what would change your mind.
- Isolate a failing stage with evidence before you touch anything.
- Choose the least-bad architecture when you cannot have what you want, and say what you gave up.
- Triage, contain and communicate before root cause — including when the bill went up 40× overnight.
- Threat-model an attack or a harm, and escalate something nobody asked you to look for.
- Disagree upward with evidence, and know when to accept a decision you lost.
How a week works
There are no lectures. Two 90-minute blocks each week. Roughly 48 facilitated scenarios over the programme, drawn from a bank of 150+, plus asynchronous solo scenarios. Cohort capped at 16–20: cold-calling does not work at scale.
Block B
Structured review
Hidden structure revealed, failure modes catalogued, the real outcome discussed. You rewrite your position as a one-page decision memo, peer-reviewed before the next session.
The scenario bank
A scenario course fails the moment it becomes a quiz show with an answer key. Participants are never told which type they are facing.
~40%
Determinate
There is a defensibly correct answer, and most people get it wrong. Builds diagnostic skill.
~40%
Contested
Competent engineers genuinely disagree. The reasoning is what is assessed.
~20%
Trap
The correct answer is “don’t build this,” “do nothing yet,” or “this isn’t an AI problem.” Saying no is the rarest senior skill.
Every scenario uses the same anatomy: a half-page brief as it would actually arrive; facts withheld unless you ask; a hidden structure revealed after the attempt; two or three escalations mid-discussion; a rubric of reasoning moves, not an answer key; common failure modes; and, where it is drawn from a real incident, what actually happened — including when the “correct” answer lost.
16-week curriculum
About 3 hours live each week. Expand a block for the syllabus. Content stays searchable when closed.
01 Reasoning under ambiguity Weeks 1–2
Before any domain content: how to attack a situation you do not understand.
- Clarifying questions as a diagnostic skill — what to ask first and why.
- Structuring an answer out loud, in real time, without rambling.
- Back-of-envelope estimation: traffic, tokens, cost, latency, headcount.
- Stating assumptions explicitly and revising them publicly when they are wrong.
- Answering “what would change your mind?” — the question most engineers cannot answer.
02 Diagnosis Weeks 3–4
Something is broken and nobody knows why.
- Retrieval failures, silent quality drift after a provider model update.
- An agent that succeeds in staging and fails in production.
- Evaluation scores that improved while users got angrier.
- A 99th-percentile latency cliff nobody can reproduce.
Skill: isolate the failing stage with evidence before touching anything.
03 Architecture under constraint Weeks 5–6
Design it, but you cannot have what you want.
- Air-gapped deployment; a hard 200ms budget; a corpus that changes hourly.
- Ten thousand tenants with strict isolation.
- A regulator who requires every output traceable to a source document.
- A total budget of £2,000 a month.
Skill: choosing the least-bad option and articulating what you gave up.
04 Incident and cost crisis Weeks 7–8
It is broken now, and it is expensive. Timed exercises with a facilitator playing an increasingly agitated stakeholder.
- A live outage during a product launch.
- A bill that went up 40× overnight.
- An agent that took a destructive action on production data.
- A prompt injection that exfiltrated another tenant’s documents.
- A model deprecation notice with six weeks’ warning.
Skill: triage, containment, communication, then root cause — in that order. Includes the postmortem and the customer communication.
05 Adversarial and safety Weeks 9–10
Someone is attacking you, or your system is harming someone.
- Indirect injection through an uploaded document; a jailbreak on a public-facing assistant.
- A model giving confidently wrong advice in a domain with real consequences.
- Discovering a training set contains data you had no right to use.
- A support agent that quietly performs worse for one group of users.
Skill: threat modelling, and escalating something you found that nobody asked you to look for.
06 Stakeholders and organisational reality Weeks 11–12
The hardest scenarios in the course, and the ones interviews never test.
- A VP has promised a customer something you cannot build.
- A colleague ships an agent with production database write access.
- A vendor demo shows numbers your team cannot reproduce.
- You are asked to add AI to a product that does not need it.
- Six months of work needs killing, and you championed it.
- Your evaluation says the new model is worse; the executive sponsor says it feels better.
Skill: disagreeing upward with evidence, and knowing when to accept a decision you lost.
07 Industry immersion Weeks 13–14
You choose one vertical and work a concentrated set of that sector’s pressures — regulatory, data, failure-cost and cultural. Each track pairs a domain practitioner with the facilitator.
- Financial services · healthcare · legal · retail and e-commerce
- Industrial and manufacturing · public sector · media and publishing · HR and recruitment
08 Interview simulation Weeks 15–16
Full mock loops under realistic conditions, recorded and reviewed. The panel includes at least one external interviewer you have never met. Written feedback against a published rubric.
- System design — 60 minutes, whiteboard, hostile follow-ups.
- Debugging — a live broken system, think aloud.
- Deep dive — cross-examination on something you actually built.
- Behavioural — conflict, failure, disagreement, a decision you got wrong.
- Reverse round — assessing the interviewer’s organisation, which strong candidates do and most participants have never attempted.
A worked example
Shortened, to show the format. Friday, 16:40.
Brief. A message from the Head of Customer Operations: “The assistant is confidently giving customers wrong refund amounts. Started sometime this week. We’ve had 6 complaints. Can you look before Monday? We can’t turn it off, it’s handling 70% of contacts.”
Withheld unless asked. Nothing was deployed this week — but the model provider silently updated the default endpoint alias on Tuesday. The refund policy was updated on Monday, in a table with merged cells. There is no eval suite; quality is monitored by CSAT, which lags four days. “Confidently wrong” means it is citing a real clause and misreading a number.
Escalations. At 17:30 legal asks whether the wrong amounts were honoured. The CEO asks for a public statement. Your first hypothesis is disproven at 18:15.
A strong response asks about the change window before proposing a fix, triages containment before root cause, separates the two candidate causes with a cheap test, recognises the merged-cell table as a parsing failure not a model failure, and treats the missing eval suite as the actual incident — this went undetected for four days.
Common failure modes: rewriting the prompt first · assuming a model regression because it is fashionable · promising a Monday fix with no diagnosis · treating the parsing bug as the whole story.
Real outcome: both causes were real and compounding. The team fixed the parsing in two hours and spent three weeks building the eval suite they should have had.
How you are assessed
Scored on reasoning quality, not conclusion. A participant reaching a defensible position the facilitator disagrees with scores full marks. A participant reaching the “right” answer by luck does not.
- 30% live scenario performance across the cohort (facilitator and peer rubric).
- 25% decision memos — clarity, honesty about trade-offs, stated assumptions.
- 20% interview simulation round.
- 15% quality of clarifying questions, tracked per scenario — the strongest seniority signal.
- 10% peer review contribution.
Who this is for
Engineers who can already build
You have shipped LLM features. Interviews and Friday outages still feel like a different sport. Target: deciding under incomplete information.
Leads who review AI work
You are in the room when a VP has promised something, or a vendor demo cannot be reproduced. Target: disagreeing upward with evidence.
Prerequisites
- You have built LLM systems — APIs, retrieval, or agents — not only completed tutorials
- You can read a production log and a cost dashboard without translation
- You are willing to be visibly wrong in front of peers. Week one establishes that, or the format produces silence
You do not need advanced depth. This runs well alongside a build programme rather than only after it.
Not a fit if
- You have not yet built an LLM system
- You want lectures and an answer key
- You expect to pass without speaking in the room
Need to choose an architecture first? Take LLM Apps: Architecture by Use Case. Need to learn to ship first? Take Applied GenAI Engineering (ship-first) or Production Generative AI Systems (topic-ordered). This course is the overlay: can you decide when people are watching?
Questions
How is this different from the build programmes?
LLM Apps: Architecture by Use Case asks which architecture, and what will go wrong. Applied GenAI Engineering asks whether you can build it. Production Generative AI Systems asks whether you understand it deeply enough to be accountable. This one asks whether you can decide, under pressure, with incomplete information, when people are watching. It is the overlay that transfers directly into interview performance.
How much time is this?
16 weeks, 48 hours live: two 90-minute blocks each week, plus asynchronous solo scenarios and a weekly decision memo. Cohort capped at 16–20.
Is there a correct answer?
Sometimes. About 40% of the bank is determinate, 40% contested, 20% traps where the right move is not to build. You are never told which. Asking the right withheld question is scored — it is the single most predictive seniority signal in the course.
Ready to sit in the room?
16 weeks, 48 live scenarios, a mock interview loop with someone you have never met. Create an account to enrol in the next cohort.
Enrol now