Alice's Adventures in a Differentiable Wonderland, Volume I: A Tour of the Land
The same text, told at every level, in every language and look shown here. Pick, then listen or download.
The same player the product gives for your own book: the voice, the drawings as they are drawn, the text.
A modern textbook on how neural networks really work, from tensors and gradients to transformers. Chapter 6 shows how a program finds its own gradients: automatic differentiation, the reverse walk behind every trained model.
This is a sample: 1 of 13 chapters.
The only thing assumed about each primitive is that we know how to compute its partial derivatives with respect to its two arguments. The first is the input Jacobian, ∂ₓ[f(𝐱, 𝐰)] ∼ (c′, c): how each output moves when each input is nudged.
Now the third way, starting from the chain rule. Take two primitives in a row, 𝐡 = f₁(𝐱), 𝐲 = f₂(𝐡). The chain rule of Jacobians says ∂ₓ𝐲 = ∂_𝐡𝐲 · ∂ₓ𝐡.
Take the calculator chain from the last part again: x is 2, multiplied by w₁ equal to 3, then by w₂ equal to 5. Run it forward and write down what you see: 6 after the first step, 30 after the second. Now start backwards with h̃ equal to 1.
The weight VJP turns out to be an outer product, with no rank-3 tensor anywhere: vjp_f,𝐰(𝐯) = 𝐯𝐱ᵀ. An outer product of two vectors is the table whose entry in row i and column j is the i-th entry of the first times the j-th entry of the second.
Now the sigmoid, the S-shaped curve σ(s) that was popular in the past as a soft version of the step function. Its derivative is σ′(s) = σ(s)(1 − σ(s)). In words: the slope is the value times what is left to reach 1.
The full story is trickier. For functions that are not convex, a subgradient need not exist, and one turns to generalizations that relax the definition to a small neighbourhood of x, such as the Clarke subdifferential.
Examples for: a curious adult with school maths who uses AI apps every day; add one modern instance per idea, such as how a chat assistant or a phone's photo app is trained
The lectures retell the English text of arXiv 2404.17625 (version 3, 2025) by Simone Scardapane, published under an open licence that allows retelling and translation. The retelling, the drawings and the voice are made by AI and checked against the book by a second model. How AI is used.
The lectures adapt the original and are released under the same licence, free to share and adapt with credit to the author and a link to this page: CC BY-SA 4.0
Upload a textbook or any difficult book. The first five minutes are free.
Upload a book