Inside LLMs in Large Language Models

Lessons in this section

1. Embeddings

   How a model turns each token into a list of numbers that carries meaning, how similarity between vectors is measured, and how real embeddings group similar text.

2. Attention

   How attention lets each token look back at earlier tokens and decide which ones matter, with scores, softmax weights, mixing, the causal mask and attention heads.

3. Model Training

   How an LLM is made: pretraining on next-token guesses, loss and learning by small steps, fine-tuning on instruction examples, and learning from human preferences.