Alice's Adventures in a Differentiable Wonderland, Volume I: A Tour of the Land
The same text, told at every level, in every language and look shown here. Pick, then listen or download.
The same player the product gives for your own book: the voice, the drawings as they are drawn, the text.
A modern textbook on how neural networks really work, from tensors and gradients to transformers. Chapter 6 shows how a program finds its own gradients: automatic differentiation, the reverse walk behind every trained model.
This is a sample: 1 of 13 chapters.
The book assumes only one thing about each primitive: that we know its local rates, collected in two tables called Jacobians. The input Jacobian says how each output number moves when each input number is nudged, so it has one row per output and one column per input.
The second way is the one you learned at school. Take a small function: a times the sine of a number, plus b times the number times its sine.
The book writes out the rate of the final score with respect to the weights of one step, somewhere in the middle of the chain. It is a long product. First comes a row of ones, which stands for the final sum: every number of the last result counts once in the score.
So a backward step is just two calls. Call the weight vector-Jacobian product with the adjoint, and out come the rates of the score with respect to that step's weights. Call the input vector-Jacobian product with the same adjoint, and out comes the new adjoint, ready to be carried one step back.
In a functional library such as JAX, everything is a function, and so is differentiation. A vector-Jacobian product becomes a function that makes functions.
The more careful answer uses the fan you just drew. For a function that curves upwards, a convex function, a subgradient at a point is the slope of a line through that point which stays below the whole function everywhere.
Examples for: a curious adult with school maths who uses AI apps every day; add one modern instance per idea, such as how a chat assistant or a phone's photo app is trained
The lectures retell the English text of arXiv 2404.17625 (version 3, 2025) by Simone Scardapane, published under an open licence that allows retelling and translation. The retelling, the drawings and the voice are made by AI and checked against the book by a second model. How AI is used.
The lectures adapt the original and are released under the same licence, free to share and adapt with credit to the author and a link to this page: CC BY-SA 4.0
Upload a textbook or any difficult book. The first five minutes are free.
Upload a book