Large Language Models tutorial

Learn what large language models are, how they read text, and how they write replies.

Course contents

LLM Basics — What a large language model is, the one job it does (guess the next word), what it is good at, what it cannot do, and how it learned from text.
Generation Flow — The full path from the prompt you type to the reply you read: chat messages, tokens, the model, picking a token, the loop, stopping, and streaming.
Model Limits — The real limits of an LLM: the context window, output length, knowledge cutoff, model size and memory, speed, and cost, with small calculations for each.

Tokens — What a token is, why LLMs read tokens instead of letters or words, token IDs and the vocabulary, and why token counts matter.
Tokenization — How a tokenizer learns its pieces with byte pair encoding (BPE), how it splits new words, and how special tokens mark the chat format.
Token Explorer — Use a real LLM tokenizer in your browser to see how text, spaces, numbers, code and other languages split into tokens, and work with token counts.

Embeddings — How a model turns each token into a list of numbers that carries meaning, how similarity between vectors is measured, and how real embeddings group similar text.
Attention — How attention lets each token look back at earlier tokens and decide which ones matter, with scores, softmax weights, mixing, the causal mask and attention heads.
Model Training — How an LLM is made: pretraining on next-token guesses, loss and learning by small steps, fine-tuning on instruction examples, and learning from human preferences.

Token Prediction — Run a real language model in your browser and see its real next-token guesses, how context changes them, and how sure the model is.
Logits — What logits are, how softmax turns them into chances, why only the gaps between logits matter, log-probabilities, and nudging a token with logit bias.
Generation Loop — Watch a real model write text one token at a time, see the choices at every step, how the loop stops, why greedy decoding can repeat, and how streaming works.

Determinism — Why an LLM can give a different answer each time, how greedy decoding and a fixed seed make it repeat, and when repeatable output matters.
Temperature — How temperature reshapes the model's chances before a token is picked: low temperature is sharp and safe, high temperature is flat and creative.
Top-K — How top-k sampling keeps only the k most likely tokens, cuts off the long tail of unlikely ones, and how it works together with temperature.
Top-P — How top-p (nucleus) sampling keeps the smallest set of tokens whose chances add up to p, why it adapts to how sure the model is, and how all the settings work together.

Hallucinations — Why LLMs make up facts that sound right, how to spot low-confidence tokens, and how to reduce hallucinations with given context, permission to say I don't know, and checks.
JSON Validation — How to get JSON from an LLM and check it: parse it, check required keys and allowed values, clean common wrappers, and find the temperature where JSON starts to break.
LLM APIs — How code talks to an LLM: the chat messages format, request settings, streaming replies, token usage and cost, errors and retries, with WebLLM and cloud services.