Run
Training job
Config, data pin, checkpoints, a resume after a crash.
Custom LLM Training
Data, tokenisers, continued training, and the ops tax — enough to know when you should not train, and how to when you must.
Created by Baljeet Dogra
Course Objectives
- Build a data pipeline that is legal, deduplicated and split honestly.
- Run a continued-training or instruction-tuning job you can resume.
- Evaluate for regressions, not only the new skill.
- Write the cost of owning the model for two years.
What you will produce
Memo
Own vs not
Two-year cost, legal, evals — a recommendation a director can read.
6 weeks curriculum
Expand a part for the syllabus. Content stays searchable when closed.
01 Data Weeks 1–2
Sources, licences, PII, dedupe, mixture. Garbage in is a legal problem too.
- Licences
- Dedupe
- PII
- Mixtures
02 Tokenisers and jobs Week 3
When to retokenise. Job config, checkpointing, loss curves.
- Tokenisers
- Jobs
- Checkpoints
- Loss
03 Train Weeks 4–5
Continued pre-training vs instruction tuning. Small scale, real discipline.
- CPT
- SFT
- Hyperparams
- Eval
04 Own it Week 6
Regressions, deprecation, the memo. Capstone recommendation.
- Regressions
- Cost
- Deprecation
- Capstone
Who this is for
ML engineers considering ownership
A vendor model is not good enough, or legal says so. Prove it.
Platform leads
You will be asked “should we train”. You need a memo, not a vibe.
Prerequisites
- Deep Learning Specialisation or equivalent
- GPU budget for small runs
- You understand LoRA already — or take Model Optimisation & Fine-tuning first
Not a fit if
- Training GPT-4 at home
- People who have never fine-tuned
Related: Model Optimisation & Fine-tuning · Deep Learning Specialisation
Questions
How big a model?
Small enough to finish. The discipline transfers; the cluster does not.
RLHF?
Discussed. Not the lab. Preference data is a specialist follow-on.
Ready to start?
Six weeks to a training job and an own-versus-not memo. Create an account to enrol.
Enrol now