LLM Apps: Types, Architecture by Use Case, Challenges and Risks

Before any architecture, the properties that break conventional engineering instincts.

• Non-determinism as a first-class design constraint: testing, debugging, support, reproducibility and blame.
• Silent failure. Conventional systems mostly fail loudly. LLM systems return fluent, well-formatted, confident wrong answers at a steady background rate.
• The prompt is untested code in production — usually not in version control, not reviewed, and not owned.
• Capability is not the constraint. Cost, latency, trust, integration and permissions are. Demos test capability, which is why demos mislead.
• “It worked when I tried it” is not evidence. Introduced early so every later module can be held to it.
• Variable cost per unit of work — an unfamiliar model for teams used to fixed-cost servers.

Lab 1: one prompt, 50 runs at temperature 0, two model versions. Write up the variance. Most learners’ first genuine surprise.

Bounded input, bounded output, deterministic surroundings. The most under-used and most reliable family.

• 1. Classification and routing. Input normalisation → constrained-output call → confidence handling → deterministic action → sampled human review. Taught first because it is the cheapest, most measurable, most deployable pattern — and the one most often skipped. Dominant failure: distribution shift in the input, silently. Cost driver: volume × input length.
• 2. Structured extraction and document processing. Ingest → layout/OCR → chunk or page targeting → schema-constrained extraction → validation → exception queue. The exception queue is the architecture. Dominant failure: confident extraction from the wrong region. Risk class: data accuracy with downstream financial or legal effect.
• 3. Transformation and summarisation. Source → length/scope control → generation → faithfulness check → provenance. Dominant failure: omission rather than fabrication, which is much harder to detect and much easier to get sued over.

Labs 2–3: build a classifier with an exception queue and measure it. Then extraction with a validation layer; deliberately feed it malformed documents.

Answers must come from a corpus, not the weights.

• 4. RAG question answering. Nine failure surfaces, not a pipeline diagram: retrieval miss vs noise, stale index, chunk boundary damage, conflicting sources, lost-in-the-middle, citation that doesn’t support the claim, permission bleed through a shared index. Wrong when the answer lives in a database, the corpus fits in context, the question is a calculation, or nobody will own curating a contradictory corpus.
• 5. Semantic search and discovery. The human still reads the source, which moves the consequence profile substantially. Often the correct de-risked substitute for archetype 4.
• 6. Text-to-query analytics. Schema and semantic layer → query generation → static validation → read-only sandboxed execution → result explanation → query shown to the user. Dominant failure: a syntactically valid query that answers a subtly different question. Never hide the generated query, never grant write access.

Lab 4: diagnose a broken RAG system from traces alone. Pre-built. Nine seeded faults.

• 7. Conversational assistant. Session state, history compaction, tool or retrieval access, safety and scope boundaries, escalation. Dominant failure: scope creep by the user into territory the system was never evaluated on.
• 8. Embedded copilot and inline generation. Tight context from the current work, low-latency model tier, suggestion never commitment, accept/reject telemetry. Latency budget in hundreds of milliseconds dictates model, prompt size and caching. The accept/reject signal is the best evaluation data in the catalogue and is routinely not captured.
• 9. Generative content production at scale. Brief → generation → brand and factual gate → human approval → publishing. Dominant risks are not technical: IP provenance, near-duplicates, search-engine treatment, reputational exposure. Includes where this pattern destroys value rather than creating it.

• 10. Deterministic workflow with LLM steps. An explicit graph or state machine where the LLM occupies specific nodes, with typed contracts, retries, idempotency and a human gate on side effects. The argument of the course: this is the correct architecture for the large majority of tasks that teams currently label “agentic”. Reliability compounds multiplicatively across steps; this pattern is where you control the multiplier.
• 11. Single agent with tools. Narrow tool contracts, reasoning loop, step and time budget, termination, sandbox, traces, reversibility. Failures as first-class content: non-termination, loop repetition, silent tool failure treated as a result, cascading error, budget exhaustion, confused deputy. Wrong when the task path is knowable in advance, the actions are irreversible, the tool surface is wide, or nobody will fund tracing and evaluation.
• 12. Multi-agent and long-horizon systems. Taught with a deliberately sceptical framing: cost, latency and debugging multiply faster than capability in most business contexts. The module’s job is to make you able to say no with reasons, and to build it properly when the answer is yes.

Labs 5–6: convert an “agent” into a deterministic workflow and compare reliability and cost. Then build a tool-using agent, budget-bound and sandbox it.

• 13. LLM-as-judge and evaluation systems. Golden dataset → rubric → judge model → calibration against human labels → drift monitoring. A real production system whose chief failure is a judge that agrees with your own model’s style rather than its correctness.
• 14. Gateway, routing and platform layer. Unified interface, routing by task and cost, caching, quota, logging and PII, fallback chain, spend controls. Turns vendor risk from an existential dependency into a configuration change. Usually built too late, after three teams have hardcoded three different providers.

The decisions that recur in every archetype and are usually made by accident.

• Context engineering as a budget problem: what earns a place in the window, and what to evict first.
• Memory: session, short-term summary, long-term store, and the retrieval of memory as its own failure surface.
• State and durability: where the truth lives when a multi-step run dies halfway.
• Model selection as architecture: capability tiers, routing, cascade, open-weights vs API, and what changes when you cannot send the data out of your estate.
• When fine-tuning is genuinely the right answer, and the maintenance liability you take on.

Lab 7: red-team a peer’s system with indirect prompt injection.

• Latency: streaming, perceived vs actual, the p99 problem, speculative and cached paths.
• Cost engineering: token accounting, caching tiers, batch vs interactive, cost per resolved task rather than per call, a unit-economics model before the feature ships.
• Observability: tracing, replay, prompt and version pinning. Without a trace store your system is undebuggable by construction.
• Human-in-the-loop: where the gate goes, how to make review cheap, automation complacency, presenting uncertainty and provenance without the interface lying by omission.
• Rollout: shadow mode, canaries, evaluation gates in CI, reversibility.

Lab 8: unit economics model for one archetype at three traffic levels.

A register, not a warning list. Each risk is paired with the architectural control that mitigates it, because a risk with no control is just anxiety.

• Correctness: fabrication, omission, stale grounding, retrieval miss, format breakage, non-termination, cascading error, over-refusal, quality regression on a provider version bump.
• Security: direct and indirect prompt injection, tool abuse and confused deputy, exfiltration through rendered output and URL construction, permission bleed in shared indexes, third-party tool servers, treating model output as untrusted input.
• Data protection: UK GDPR in practice — lawful basis, purpose limitation where prompts become training data, DSAR and erasure against embeddings and prompt logs, retention, international transfer, DPIA triggers that actually apply.
• Regulatory and liability: EU AI Act risk tiering for a system you are designing now. Sector overlays, with financial services treated in depth. Who is accountable for a generated output.
• Commercial and operational: vendor concentration, price and rate-limit changes mid-contract, non-determinism inside support, evaluation theatre, shadow AI, the cost tail from the first month of real traffic.
• Organisational: who owns the prompt, who owns the corpus, who is on call, what happens when the model is confidently wrong to a customer at 2am.

Lab 9: complete a risk register with mapped controls for a real use case.

• Decision procedure: use case → axes → candidate archetypes → elimination on constraints → simplest survivor.
• The “simpler alternative rejected” discipline.
• Architecture decision records for AI systems, including fields conventional ADRs lack: evaluation basis, cost model, failure envelope, reversibility.
• Writing for the approver: security team, DPO, board, sceptical senior engineer.
• Capstone defences.

Lab 10: 20-minute defence against structured challenge.