Captions are on by default. Use the player’s CC control to toggle.
Starter notebook
Scaffolding with TODOs — not a finished solution. Open the notebook in Jupyter, VS Code, or Colab (upload the file). Install from requirements.txt in the same folder.
Folder on this site: /genai-starters/ — treat it as the course repo. Lab 3.3 / 5.2 also include Dockerfile and docker-compose.yml there.
Week 1 · Module 1 · about 3 hours
Lab 1.1 — Tiny language model
Goal: train a very small model to guess the next character in English text, watch it get slightly better, then watch it still produce rubbish. That is enough to understand what ChatGPT is doing — and what the API hides from you.
In plain English
A large language model (LLM) such as ChatGPT is a program that keeps asking: “given the text so far, what token comes next?” A token is a chunk of text (a word, part of a word, or a character). This lab uses the simplest version: one character at a time, on your laptop, with no cloud API. You are not building ChatGPT. You are building a toy so the rest of the course is not abstract.
Think of it as autocomplete. After seeing “Th” many times followed by “e”, the toy should start preferring “e”. After a few minutes of training it will still ramble and repeat. That failure is the point.
Words used in this lab
Dataset / corpus
The text file you train on. A few chapters of a public-domain book is plenty.
Loss
A number that means “how wrong were the guesses?” Lower is better. If loss goes down, the toy is learning something. If it barely moves, the toy is too small or the setup is wrong.
Epoch
One full pass over your text file. Doing 5–20 epochs is enough here. More is not “better ChatGPT”.
Temperature
How random the next-character pick is. Low temperature = safer, more repetitive. High = wilder, more nonsense.
Before you start
Python 3.10+ and a notebook (Jupyter, VS Code, or Google Colab). CPU is fine. No GPU needed.
Install: pip install torch matplotlib (PyTorch). If that is heavy, pip install numpy matplotlib and use a tiny NumPy version — same idea.
Download a short public-domain text, under 1 MB. Example: a single Gutenberg book chapter saved as input.txt.
What to build
A loop that: reads characters → turns each letter into a number → guesses the next number → measures how wrong it was → nudges the numbers inside the network a tiny bit. You do not need to invent the maths from scratch. Use a starter like the one below and fill in the comments.
Shape of the notebook — not a copy-paste solution
# 1. Read input.txt as one long string
# 2. List unique characters (a, b, … space, punctuation)
# 3. Map character → integer and integer → character
# 4. Cut the string into short windows, e.g. 32 characters → predict the next one
# 5. Train a small network (even 1–2 layers) for 5–20 epochs
# 6. Each epoch: print the loss
# 7. After training: start from "The " and sample 200 characters
# once with low temperature, once with high temperature
Steps
Load the text. Print the length and the first 200 characters. If this fails, stop — you do not have a dataset yet.
Turn letters into numbers. If the file has 70 unique characters, each letter is an ID from 0–69. Print five examples, e.g. T → 32.
Train. After each epoch, write down the loss. You want to see it drop (for example 3.2 → 2.1). If it stays flat, the learning rate is wrong or the model is not connected to the data — fix that before generating text.
Generate. Seed with a phrase that appears in the book, then with a phrase that does not. Save both outputs.
Name one failure you actually saw, in ordinary words. Examples: “it repeats the same three words”, “it dumps random punctuation”, “low temperature is boring, high temperature is garbage”. That is the observation. Do not dress it up.
Half-page note answering only this: if you deleted your toy and called the OpenAI or Gemini API instead, which of those failures would you still see in the product, and which would the vendor hide (rate limits, safety filters, a much bigger model)? There is no trick answer — write what you think, then we discuss it in the live session.
What “good enough” looks like
A notebook that trains and prints loss. A screenshot or plot of loss going down.
Two generated snippets (seen prompt vs unseen prompt).
Half a page of notes. You are not expected to derive back-prop or explain attention yet — that is lab 1.2 (tokens and attention).
If the generated text is bad, you still passed. The lab is to watch learning and leftover failure, not to ship a chatbot.
Week 2–3 · Module 1 · about 4 hours
Lab 1.2 — Tokens and attention
Goal: see how a real model chops a sentence into pieces, and why that chopping later breaks document search if you split in the wrong place.
In plain English
Lab 1.1 used one character at a time. ChatGPT does not. It cuts text into tokens — sometimes a word, sometimes a piece of a word, sometimes a number split in half. Two models can chop the same sentence differently. That is why “this prompt is 100 tokens” is not a universal fact.
Attention is the model asking, for each new piece: “which earlier pieces should I look at?” You do not need the matrix algebra. You need to see that some words get ignored, and that later, when we split a PDF into chunks for search, a bad split (table body without its header) is the same kind of mistake.
Words used in this lab
Tokeniser
The program that cuts a string into tokens. GPT models often use tiktoken. Llama/Mistral use a different one. Same sentence, different cuts.
Token
One piece after the cut. Billing and context limits are counted in tokens, not words.
Attention
A map of “how much this word looks at that word” while predicting the next token. A heatmap is enough for this lab.
Chunk
A slice of a document you will later store for search (module 2). If the slice starts in the middle of a table, retrieval will look confident and still be wrong.
Before you start
A notebook. CPU is fine.
Install something that can tokenise: pip install tiktoken. For a second tokeniser, Hugging Face transformers plus any small model tokenizer (for example Llama or Mistral) — or use the tokenizer playground for that model if installing weights is painful.
Optional: an attention visualiser (BertViz, a notebook from the live session, or a tiny transformer demo). A screenshot from a hosted visualiser is acceptable if you say which one.
One PDF page that contains a table with a header row.
Steps
Chop the same sentence two ways. Use: The patient’s ID is 48291-A. Run tiktoken (or the OpenAI tokenizer) and a second, open-model tokenizer. Print the token list and the count for each.
Write what surprised you. Typical surprises: the number splits, the hyphen is its own token, “patient’s” becomes two pieces. One short paragraph is enough.
Look at attention on a sentence of about 40 tokens. You are looking for a picture, not a proof. Note one pair of words that attend strongly and one that barely does.
Mark a bad split. On your PDF table page, draw or describe where a “512-token chunk” would likely cut the table away from its header. That cut is the bug you will hunt in module 2.
Three rules for later. Write three chunking rules you will actually test in lab 2.2. Examples: “never split a table from its header”, “keep a heading with the next paragraph”, “overlap chunks by ~10% so a sentence is not cut in half”.
What “good enough” looks like
A notebook cell or screenshot with two token counts and the token lists.
One attention picture (or a sentence describing what you saw).
Half a page: bad split + three chunking rules.
You are not expected to implement a transformer from scratch. Seeing the cuts and one bad document split is the lab.
Week 4–6 · Module 1 · about 4 hours
Lab 1.3 — Pick a model
Goal: stop saying “this model is the best”. Run the same 20 tasks on three models, write down quality, speed, and cost, and recommend one default with a backup.
In plain English
A foundation model is a ready-made LLM you call (OpenAI, Gemini, Anthropic) or run yourself (Llama, Mistral, and smaller copies). They differ in how much they cost, how slow they are, whether you can send private data, and how well they follow instructions. Marketing pages will not tell you which one is right for your 20 examples.
This lab is a bake-off. Same questions, three models, a spreadsheet. The memo is for a person who signs invoices — short, numbered, no hype.
Words used in this lab
Eval set
A frozen list of inputs with a notion of a good answer. You do not change it mid-bake-off to make a favourite model look better.
Latency
Wall-clock time from send to full reply. Users feel this. “Tokens per second” is optional extra.
Token cost
What you pay for input + output tokens (or GPU time if you self-host). Include retries — a cheap model that you call three times is not cheap.
Rubric
A 1–5 scoring guide written before you look at outputs. Example: 5 = valid JSON and correct fields; 1 = wrong format or invented facts.
Before you start
API keys for at least one hosted model. For the open model, a hosted small instruct model is fine (Groq, Together, Ollama on your laptop).
A spreadsheet or notebook table with columns: id, input, model, quality 1–5, seconds, input tokens, output tokens, notes.
Pick one task you actually care about: support reply, extract JSON, or summarise a policy. Stick to it.
Steps
Write 20 items and the rubric. Include a few nasty ones (missing fields, long input, the model should refuse). Freeze the list.
Pick three models: one closed API (GPT/Gemini/Claude), one open 7–8B-class instruct model, and one cheaper or smaller option. Write their names and prices in the sheet.
Run all 20 × 3. Same prompt template. Log quality, time, tokens. If a call fails, log the retry — do not silently skip.
Cost per 1,000 good answers. Count only scores you would ship (say 4+). If a model needed retries, those tokens still count.
One-page memo: recommended default, fallback if the default is down or too slow, and one hard “do not use this model when…”. No “best in class” without a number from your table.
What “good enough” looks like
The spreadsheet with 60 rows (or 20 rows × 3 model columns) filled in.
A one-page memo a manager could act on.
If the cheap model wins on your task, that is a valid result. The lab is the comparison, not picking a famous name.
Week 7–8 · Module 2 · about 4 hours
Lab 2.1 — Let the model use tools
Goal: stop the model inventing an order status. Give it two functions it can call, loop until it has an answer or should stop, and log every way it can fail.
In plain English
A chat model only predicts text. If you need live data (search your docs, look up an order), the program must: offer named tools → the model replies “call this tool with these arguments” → your code runs the function → you send the result back → repeat. That cycle is a tool loop. One lucky JSON blob in a screenshot is not a loop.
You will also see the model invent a tool name, send broken JSON, or follow a user who says “ignore your rules”. Those are the failures to log, not to hide.
Words used in this lab
Tool / function calling
You declare functions (name, arguments). The model chooses one instead of writing a fake answer.
Schema
The shape of the arguments (e.g. order_id must be a string). If the model sends garbage, you reject it — you do not run the function.
Tool loop
Call → run → feed result back, up to a max number of steps (here: 4). Then you must stop even if it is still asking.
Jailbreak
A user message trying to override your instructions (“ignore previous rules”). Your test set should include a few. A clean refusal is a pass.
Before you start
An API that supports tools (OpenAI-compatible is easiest), or a small local model with a tool format you parse yourself.
Two stub functions are enough — they can return fake data. You are testing the loop, not a real warehouse.
What to build
Shape of the loop — not a copy-paste solution
# tools: search_docs(query), get_order_status(order_id)
# for each user message:
# send messages + tool list to the model
# if the model calls a tool:
# if name unknown or JSON invalid → log "schema" / "hallucinated tool", tell the model, retry
# else run stub, append result, loop (max 4)
# if the model answers in text → return it
# if it should refuse (out of scope / jailbreak) → short refusal, no tools
# log: success | retry | schema | timeout | hallucinated tool | policy refuse
Steps
Stub two tools:search_docs and get_order_status. Hard-coded return values are fine.
Write the loop with a max of 4 steps, JSON validation, and a path for “I will not call that tool”.
Thirty fake user messages: 10 that should work, 10 that are not your product’s job, 10 that are broken (missing id, nonsense).
Log a failure class for every turn. Keep the list small: schema, timeout, invented tool name, policy refuse, success.
Add a safety instruction that should block “ignore previous instructions”. Show one blocked example in your notes.
What “good enough” looks like
Three traces you can paste: a success, a retry after bad JSON, a clean refusal.
A table or log file for the 30 turns.
The stubs can be dumb. The loop and the logs are the product of this lab.
Week 9–10 · Module 2 · about 5 hours
Lab 2.2 — Search your documents
Goal: put 200+ pages in a search index, try three ways of slicing them, and measure which slicing actually finds the right page for 25 questions.
In plain English
RAG means Retrieval-Augmented Generation: look up relevant passages first, then ask the model to answer using those passages. Without retrieval, the model guesses from memory. With retrieval, it can cite your handbook — if the right paragraph was stored and found.
Slicing (chunking) is the unglamorous part. Too small: the model gets fragments. Too big: search matches the wrong section. Overlap means the end of one slice repeats at the start of the next so a sentence is not cut in half. You will pick a winner with a number, not a vibe.
Words used in this lab
Corpus
The documents you index. A public docs site, a handbook PDF set, or a wiki export. About 200 pages is the target, not 2.
Chunk / overlap
Chunk = one stored slice. Overlap = shared text between neighbours (about 10–15% of the chunk size).
Vector search
Turn each chunk into a list of numbers (an embedding) and find nearest neighbours to the question. Chroma, FAISS, or a hosted vector DB are all fine.
Recall@5
For each question you already know the right passage. If that passage appears in the top 5 search hits, it counts as a hit. Average over 25 questions.
Before you start
A corpus you are allowed to use. If you only have 80 pages, say so and still compare three chunk sizes — do not fake 200.
Reuse the three chunking rules you wrote in lab 1.2 if they still make sense.
pip install an embeddings model + a local store (sentence-transformers + chroma/faiss) or a hosted embedding API.
Steps
Load the corpus and print page or file count. Give every chunk an id you can write down (file + chunk number).
Build three indexes: chunk sizes about 256, 512, and 1024 tokens, each with 10–15% overlap. Same embedding model for all three so the comparison is fair.
Write 25 questions and, for each, the gold chunk id (the passage that actually answers it). If you cannot find a gold passage, the question does not belong in the set.
Retrieve top-5 for every question on every index. Compute recall@5. Put the three numbers in a table.
Keep the winner as the baseline for lab 2.3. Write two sentences: why you think it won (shorter chunks matched headings? longer chunks kept tables together?).
What “good enough” looks like
A table of recall@5 for three chunk sizes.
The 25 questions + gold ids in a file.
A short note on why the winner won.
A low score is still a valid lab if the measurement is honest. You will try to raise it in 2.3.
Week 11–12 · Module 2 · Portfolio 1
Lab 2.3 — Better search + a score
Goal: add keyword search next to meaning search, rerank the combined results, make the model cite passages, and leave behind a test you can rerun after every change.
In plain English
Vector search is good at “this means roughly the same”. It is weak at exact tokens: error codes, SKUs, names. Keyword search (BM25 is the usual name) is the opposite. Hybrid means you run both and combine the ranked lists. A reranker then looks at the top 20 and picks a better top 5 — a second, slower model that only scores “does this passage match this question?”.
Then you ask the LLM to answer using those 5 passages and to quote them. Faithfulness means: the answer should not invent facts that are not in the passages. You will also break retrieval on purpose (typo, table question) so you know how it fails.
Words used in this lab
BM25 / keyword search
Old-fashioned word matching with smarter weighting. Catches exact codes that embeddings miss.
Hybrid / fuse
Combine two ranked lists (e.g. Reciprocal Rank Fusion). You do not need a custom formula — a library default is fine if you name it.
Rerank
Re-score a shortlist. Top-20 in, top-5 out.
Faithfulness
Did the answer stay inside the retrieved text? A frozen rubric or a RAGAS-style score is fine. Freeze the method before you tune.
Before you start
Lab 2.2’s winning index and the same 25 questions. Do not secretly change gold ids to improve the score.
A keyword index on the same chunks (whoosh, elasticsearch, or BM25 in a Python library).
Steps
Add keyword search and fuse it with your dense results. Record recall@5 vs lab 2.2’s baseline.
Rerank fused top-20 down to top-5. Record recall@5 again. If rerank hurts, say so — do not delete the row.
Answer with citations. Each answer should point at chunk ids. Score faithfulness and relevance with a rubric you write down and freeze (or RAGAS if you already know it).
Break it on purpose: a typo query and a question whose answer lives in a table. Screenshot or paste the miss. One paragraph on why.
Package Portfolio 1: notebook or small API, an eval script, a README that says exactly how to rerun the 25 questions.
What “good enough” looks like
Before/after eval table after one retrieval change (hybrid, rerank, or a chunking tweak).
README a classmate could follow without you in the room.
Portfolio 1 is “search that you can measure”, not a pretty chat theme.
Week 13–14 · Module 3 · about 3 hours
Lab 3.1 — Prompt, search, or train?
Goal: decide how to improve a product without defaulting to “we should fine-tune”. Score three briefs. At least one must be “do not train a custom model”.
In plain English
There are three common levers. Prompting — change instructions and examples. RAG — give the model documents at question time (module 2). Fine-tuning — extra training so the model’s weights change (expensive, needs data, goes stale). PEFT (next lab) is a cheaper way to fine-tune, not a reason to always fine-tune.
A knowledge-heavy bot that must cite a changing handbook is usually RAG. A “always output this XML shape” job might be prompting or a small tune. A tool-heavy workflow is often tools + prompts, not a new model. Write the defence a budget holder would actually accept.
Words used in this lab
Fine-tune
Train on extra examples so behaviour changes even without retrieved docs. Needs a dataset and a plan when the world changes.
PEFT
Parameter-efficient fine-tuning: train a small add-on instead of all weights. Lab 3.2 is the hands-on version. Here you only decide whether you need it.
Drift
How often the facts or policy change. High drift argues against baking facts into weights.
Latency / GPU budget
Can you host a tuned model at all? If the SLA is 2 seconds on a laptop, a large tune is a non-starter.
Steps
Three short product briefs (half a page each is plenty): one knowledge-heavy, one style/format-heavy, one tool-heavy. Invent them from a domain you know, or use a real work problem with names removed.
Score each on: how much labelled data you have, how fast facts change, latency budget, need for citations, GPU/money budget. A 1–5 per factor is enough.
Assign a path: prompt only, RAG, fine-tune (including PEFT), or a hybrid. At least one brief must be “do not fine-tune” with a reason.
One-page defence for the “do not fine-tune” choice — the page you would send with a “no” to a vendor demo.
What “good enough” looks like
Three scored briefs in the repo.
The one-page “do not fine-tune” note.
There is no extra credit for choosing fine-tune on all three. The skill is saying no.
Week 15–16 · Module 3 · about 6 hours
Lab 3.2 — Light fine-tuning
Goal: teach a small open model a narrow job by training a small add-on (not the whole network). Try two add-on sizes. Check it did not forget how to answer ordinary questions.
In plain English
Full fine-tuning rewrites most of the model. That needs serious GPUs. LoRA freezes the original model and trains a thin “adapter” of extra numbers. QLoRA is LoRA plus compressing the frozen model so it fits on a smaller GPU (Colab, a rented 24 GB box, etc.). Rank is “how thick is the adapter”: 8 vs 16 is a typical comparison — bigger is not automatically better.
You will hold out 15% of your examples so you are not grading the model on sentences it just memorised. Catastrophic forgetting means it got good at your format and worse at general chat — you will check 10 ordinary prompts on purpose.
Words used in this lab
LoRA / QLoRA
LoRA = small adapter. QLoRA = that adapter plus a memory-saving (quantised) base model. Use a tutorial notebook from the live session if you have not done this before — do not invent the trainer from scratch.
Rank
Adapter thickness. You will train two ranks (e.g. 8 and 16) and compare quality vs GPU hours.
Holdout
Examples you never train on. Eval only here for the “did we learn the job?” number.
Catastrophic forgetting
The tuned model becomes worse at general questions. Ten prompts from outside your dataset are the check.
Before you start
A GPU: Colab, a cloud notebook, or a workstation. CPU-only QLoRA is usually too slow for this lab — say so if you are blocked and use the smallest model that will finish.
200–1,000 instruction pairs: domain Q&A or “rewrite this as JSON”. Quality beats a scraped mess.
Reuse the 1–5 rubric from lab 1.3 so scores mean the same thing.
Steps
Prepare the data. Shuffle, hold out 15%, sanity-check five examples by eye.
Train two QLoRA (or LoRA) runs at two ranks. Log GPU hours (or Colab runtime) for each.
Eval the untouched base model and both adapters on the holdout with the same rubric.
Forgetting check: 10 general prompts on base vs winner. Note if the winner got ruder, shorter, or worse.
Pick a rank in two sentences: quality vs hours vs forgetting. That choice feeds lab 3.3.
What “good enough” looks like
An eval table: base vs rank A vs rank B.
GPU-hour note and a rank justification.
Notes from the 10 general prompts.
If neither adapter beats the base model, that is a result — write why (data too noisy, too few steps) instead of hiding the table.
Week 17–18 · Module 3 · Portfolio 2
Lab 3.3 — Serve the tuned model
Goal: put the winning adapter behind a small API, switch it off with one setting, and prove the holdout still works through that API.
In plain English
A notebook that “works on my GPU” is not a product. Serving means: another program sends HTTP, gets text back. You need a switch for “use the original model” vs “use the tuned one”, because the tuned one will sometimes be worse in production and you must roll back without redeploying a mystery.
If loading two models at once is too heavy, two configs or an env var that picks weights is enough. The README must show a classmate how to call it.
Words used in this lab
Adapter export
Save the LoRA/QLoRA files (and a note of the base model name). Lab 3.2’s winner.
Baseline
The untuned model. Your off switch should land here.
Rollback
How you disable the adapter in one config change (env var, flag, or config file) — written so an on-call engineer could do it.
Smoke test
A short list of calls that must work after start-up. Here: 20 holdout items through the API, not a full retraining.
Steps
Export the winning adapter from 3.2. Note the base model id in the README.
Serve with FastAPI (or equivalent). One POST that takes a prompt and returns text.
Add a switch such as model=baseline|tuned (query param or header). Default should be obvious in the README.
Smoke-test 20 holdout items through the API. Paste a tiny results table next to lab 3.2’s notebook numbers — they should not wildly disagree.
Write the rollback plan in the README: the one change, and how you know you are on baseline (a log line or a /health field is nice, not required).
What “good enough” looks like
Portfolio 2 README with curl (or HTTP) examples for baseline and tuned.
Eval table from the API smoke test.
Rollback paragraph.
A stub that returns canned text is not this lab. Loading a small model on CPU is allowed if GPU serving is out of reach — document the hardware.
Week 19–21 · Module 4 · about 5 hours
Lab 4.1 — Documents and pictures
Goal: take photos or scans of pages, pull fields into JSON, score what you got wrong — then turn a short spec into a labelled diagram and list how the picture-model lied.
In plain English
A vision-language model (VLM) is an LLM that can look at an image. Combined with OCR (reading printed letters as text), you can turn a form photo into structured fields. Layout and handwriting still break this. You will measure field-by-field, not “it looked pretty good”.
The second half is generation: from a text spec, produce a diagram (an image model, or Mermaid rendered to PNG/SVG). Models invent boxes and labels that were never in the spec. Catching that is the point.
Words used in this lab
OCR
Optical character recognition — pixels to text. Tesseract, a cloud OCR, or the VLM itself if it reads the page. Say which you used.
VLM
Vision-language model: GPT-4o-class, Gemini, or an open VLM. It maps image + prompt → text/JSON.
Schema
The JSON shape you want (name, date, id, …). Validate it. Extra keys or missing required fields are errors.
Hallucination (here)
A field or diagram label that is not on the page / not in the spec. Three types you actually saw, not a textbook list.
Before you start
10 pages you are allowed to photograph. Redact real IDs, names, and account numbers before they leave your machine. Fake forms are better than leaking data.
A JSON schema written first (even 6 fields).
A 10-line architecture spec for the diagram half (boxes and arrows in words).
Steps
Extract to schema for all 10 pages. Mix of OCR+VLM is fine. Save raw JSON.
Score field-level accuracy (correct / total fields). Note layout misses and handwriting misses separately if you have both.
Generate a diagram from the spec. Image model or Mermaid → render — pick one and stick to it.
List three hallucination types you observed (wrong number, extra box, confident blank). For each, one sentence on how you would catch it in production (schema check, human review, compare to OCR text).
What “good enough” looks like
Sample JSON for at least two pages, plus the accuracy table.
The diagram file and the spec next to it.
Three hallucination notes.
Low accuracy on handwriting is expected. Measuring it is the lab.
Week 22–24 · Module 4 · Portfolio 3
Lab 4.2 — Several agents
Goal: split a 4-step business job across a planner, workers that call tools, and a supervisor. Save a run that worked and a run that failed, retried, then fell back.
In plain English
An “agent” here is not magic. It is a loop: look at state → choose a next action (often a tool from lab 2.1) → update state. Several agents means several roles. A planner breaks the user request into steps. Workers do one kind of work (search, write, call an API). A supervisor checks the result and says “done”, “retry”, or “stop”. LangGraph (or similar) is a library that stores this as a graph of nodes and edges so you can debug it.
You must show a trace: a log of who did what, in order. A happy path is not enough. Time out a tool on purpose and show retry then fallback.
Words used in this lab
Graph
Nodes (roles) and arrows (what happens next). Draw it in the README so a human can follow without reading all the Python.
Trace
Timestamped log: node name, input summary, output summary, errors. LangSmith, a JSONL file, or print statements you save.
Supervisor
The node that is allowed to halt the run or send work back. Without it, workers loop forever.
Fallback
What you do when a tool times out after retries — e.g. return search snippets and a “could not complete” flag, not a fake success.
Before you start
Reuse tools from 2.1 and retrieval from 2.3 if they still exist. Do not rebuild RAG from zero unless you have to.
LangGraph, CrewAI, or a tiny state machine you write — name the library. A pile of unconnected scripts is not a graph.
Steps
Pick a 4+ step workflow in words first: intake → retrieve → decide → write-back (or similar). One paragraph.
Implement planner, workers, supervisor. Max steps / recursion limit on, or it will run away.
Turn on tracing. Save one happy-path file and one file where a tool times out, retries, then fallback.
Draw the graph in the README (Mermaid is enough). Label the fallback arrow.
What “good enough” looks like
Portfolio 3 runs locally (README says how).
Two saved traces + the diagram.
If the supervisor is just an if-statement, say so. Honesty beats a fake “multi-agent platform” screenshot.
Week 25–26 · Module 5 · about 4 hours
Lab 5.1 — Timeouts and fallbacks
Goal: decide how slow is too slow, hammer last module’s agent, stop waiting on dead tools, and when things are on fire return a simpler search-only answer plus a one-page “what we do at 3am” note.
In plain English
Demos assume every API is healthy. Production does not. An SLO is a promise you chose (“95% of requests finish in under 8 seconds, fewer than 2% hard errors”). A timeout is “we stop waiting”. A circuit breaker is “this tool has failed too often, stop calling it for a while”. Degraded mode is “the fancy agent loop is off; just search and show snippets” — worse answers, but the product still responds.
A runbook is the page an engineer reads during an incident: symptoms, the switch to flip, who to ping. You will write one even if the “incident” is you killing a stub.
Words used in this lab
SLO
Service level objective — the numbers you will plot. Pick something you can actually measure on a laptop.
p95 latency
95% of requests were faster than this. One slow request should not hide in an average.
Circuit breaker
After N failures in a window, fail fast without calling the tool. After a cooldown, try again.
Runbook
One page: how we know it is broken, the degraded switch, how we know it recovered.
Steps
Write an SLO (example: p95 < 8s, error rate < 2%). Put it at the top of the notes.
Drive ~100 requests. On a laptop, sequential with a small delay is fine — say so. Record latency and errors.
Timeouts per tool plus a breaker after N failures (pick N, write it down).
When the breaker opens, return hybrid-search snippets from lab 2.3 (or a stub) without the agent loop. Label the response as degraded.
One-page runbook.
What “good enough” looks like
A table or plot of latency under that load.
The runbook.
One captured degraded response.
You do not need Kubernetes. You need numbers and an off-ramp.
Week 27–28 · Module 5 · Portfolio 4
Lab 5.2 — Put it in a container
Goal: a classmate can run your API with Docker, a health URL says when it is ready, and you publish a latency number at a request rate you actually ran.
In plain English
A container packages the app plus its system libraries so “works on my machine” becomes docker compose up. A health endpoint is a boring URL that returns 200 only when the process can actually answer (model loaded, or stub loaded — not merely “the web server started”). Metrics are counters you can scrape: how long requests took, how many tokens you spent. Prometheus text format is a simple file format for that; you do not need a full Grafana stack.
QPS is queries per second. Pick a rate your laptop can sustain. p95 you already met in 5.1 — measure it again through the container. Mention how real GPU serving (vLLM and friends) would differ, even if you used a stub locally.
Words used in this lab
Container / Compose
Docker image + a compose file that starts the API (and maybe a vector DB). The README command must work on a clean checkout.
/health
Returns 200 only when ready. 503 while the model is loading. Load balancers use this.
QPS
Requests per second you used in the baseline. If you ran 2 QPS for 2 minutes, write that — do not invent 1000.
vLLM-style serving
A specialised server for LLM tokens. You may not run it. You must still write 5–10 lines: what it is for, and what you used instead.
Steps
Dockerfile + compose wrapping the API from 3.3 or 4.2 (the one you will keep).
GET /health is 200 only when ready.
Export latency and token-cost counters (Prometheus text on /metrics is enough).
Measure p95 at a QPS you really ran. Put instance size (RAM, CPU, GPU or “CPU stub”) in the README.
Serving note: vLLM (or equivalent) vs what you actually ran.
What “good enough” looks like
README: docker compose up and how to hit /health and one predict URL.
p95 + QPS written down.
If Docker cannot load your full model, containerise the API with a stub and say so. A compose file that fails on first run is not done.
Week 29–30 · Module 5 · about 4 hours
Lab 5.3 — Safety, privacy, spend
Goal: block secret-looking input, hide emails and account numbers before they hit the model or the logs, park shaky answers for a human, cap tokens, and write a one-page “who is allowed to change this” pack.
In plain English
Guardrails are checks around the model, not vibes in the prompt. PII is personal data (emails, account numbers). Redact means replace with a placeholder before the LLM or the log file sees it. A human queue is a list of answers you do not auto-send — a JSON file of tickets is enough. A spend cap is a hard stop: this request or this day has used too many tokens, return a short error instead of another paid call.
Governance is boring on purpose: where data flows, how long you keep logs, who can edit the system prompt. One page, not a legal novel.
Words used in this lab
Guardrail / filter
Code that inspects input or output. Reuse jailbreak cases from lab 2.1. Regex for API keys is a start; say what you do not catch.
PII redaction
Strip or mask emails and account-like numbers before the model and before logs. Show a before/after pair.
Human queue
If confidence is low (your own score, or “breaker open”, or filter unsure), write a ticket instead of answering the user as if it were final.
Token cap
Max tokens per request and per day (in-memory counter is fine for the lab). Hitting the cap returns a clear error, not a hang.
Steps
Input filter: secrets patterns + the jailbreak probes from 2.1.
Redact emails and account numbers before model and logs.
Low-confidence → queue (a JSON file is fine). Show one item in the queue.
Hard caps per request and per day. Demo a blocked request and a capped request.
Governance pack (one page): data flow, retention, who approves a prompt change.
What “good enough” looks like
The one-page pack.
Evidence of a blocked request and a capped request (log or screenshot).
Before/after redaction.
This will not pass a real security audit. It is the skeleton so you know what “responsible” means in code, not in a slide.
Week 31–32 · Capstone
Lab 5.4 — Industry capstone
Goal: one real-looking problem in a domain you pick, the thinnest system that hits a metric you wrote down first, and a 12-minute demo. Reuse earlier labs. Do not start a new toy from zero.
In plain English
This is the portfolio piece. Healthcare, banking, manufacturing, or a sanitized brief from your job. You are not building a company. You are showing you can: state the problem, draw the pieces (data, models, tools, humans, logs), implement the smallest path that moves the metric, measure it, and explain what you cut.
If RAG was the heart of the problem, your 2.3 index should show up. If tools were the heart, 2.1 and 4.2 should show up. Starting over “because the capstone should be clean” is how people run out of time.
Words used in this lab
Success metric
A number you chose before coding (e.g. 15/20 eval items score ≥4, p95 < 8s). If you cannot measure it, it is not a metric.
Thinnest path
The least machinery that can hit the metric. Extra agents and extra models are a smell unless the metric requires them.
Eval set
20+ items, frozen, with a rubric. Same spirit as lab 1.3.
Steps
One-page problem statement and success metrics — date it. Write this before you code.