A Neural Network Playground
Daniel Smilkov and Shan Carter, TensorFlow · interactive
A small neural network that trains in your browser. Add layers and neurons, change the learning rate, and watch the boundary it draws bend to fit the data.
Machine Learning Explainers
Other people's pages that show how these ideas work. The Machine Learning Lineage says what each idea is and what it built on. These explainers and visualizations let you watch the ideas run. Each one lists the Lineage ideas it covers.
Neural network basics
What a network is and how it learns.
3Blue1Brown
Grant Sanderson · video
Animated maths lessons. The neural networks series starts with what a network is, moves through gradient descent and backpropagation, and ends with how attention and transformers work.
How transformers work
The design behind today's language models, step by step.
Transformer Explainer
Polo Club of Data Science, Georgia Tech · interactive
GPT-2 running live in your browser. Type your own text and follow it through embedding and attention to the probabilities for the next word.
LLM Visualization
Brendan Bycroft · interactive 3D
A 3D model of a small GPT with every number in view. A guided walkthrough follows a tiny network as it sorts six letters, one step at a time.
The Illustrated Transformer
Jay Alammar · article
The drawn walkthrough many people learned the transformer from. It starts with a translation example and builds up through self-attention, multiple heads and positional encoding to the full encoder and decoder.
Variations on the design
How recent models rework the parts.
A Visual Guide to Attention Variants in Modern LLMs
Sebastian Raschka · article
Drawings of how attention has been reworked in recent models: multi-head, grouped-query, sliding-window and multi-head latent attention, then sparse attention and hybrid designs.
A Visual Guide to Mixture of Experts (MoE)
Maarten Grootendorst · article
More than 50 illustrations of how a model splits its work among specialist sub-networks, with a router choosing a few of them for each token. Covers language models and vision models.
Running a model
What happens after training, when a model answers a request.
LLM Inference Handbook
Modular · handbook
An animated visualizer follows one prompt through the tokenizer, scheduling and prefill to the streamed answer. Later chapters cover the metrics that matter, planning a deployment and making inference faster.
Looking inside
Tools for seeing what a model has learned.
Neuronpedia
Neuronpedia · tool
An open-source platform for interpretability, the study of what happens inside a model. Browse the features that sparse autoencoders find in real models, see the text that sets each one off, and steer a model with them.
Embedding Atlas
Apple · tool
A tool for exploring large sets of embeddings, where each item is placed as a point so that similar items sit close together. Search, filter and zoom across the whole map in your browser.