Библиотека · CC BY-SA 4.0

Приключения Алисы в Странедифференцируемых чудес

Alice's Adventures in a Differentiable Wonderland, Volume I: A Tour of the Land

Иллюстрация к книге

Выберите версию

Один и тот же текст на каждом уровне, на всех языках и во всех оформлениях, что здесь есть. Выберите, потом слушайте или скачивайте.

Слушать и смотреть

Тот же плеер, который продукт даёт для вашей книги: голос, рисунки по мере того, как они рисуются, текст.

Современный учебник о том, как на самом деле устроены нейросети: от тензоров и градиентов до трансформеров. Первые три главы закладывают основу: о чём книга, линейная алгебра и анализ, на которых работает сеть, градиентный спуск, что такое набор данных и функция потерь.

Это пробная часть: 3 из 13 (главы).

Из лекций

Three blocks for the parts of the book with their chapters, followed by a website and lab notebooks.
About this book

The Book's Three Parts

The third part of the book, Down the Rabbit Hole, continues our exploration of differentiable models by considering alternative designs for sets, most importantly attention layers and transformer models in Chapters 10 and 11, graphs in Chapter 12, and finally recurrent layers for temporal sequences in Chapter 13.

Two arrows from the origin and their sum as the diagonal of a parallelogram; the length of an arrow is its norm.
License, and the mathematical preliminaries

Adding Vectors, Measuring Length

Vectors possess their own algebra, which we call a vector space, in the sense that any two vectors 𝐱 and 𝐲 of the same shape can be linearly combined, 𝐳 = a𝐱 + b𝐲, to provide a third vector, with zᵢ = axᵢ + byᵢ.

The parabola f(x) = x² − 1.5x with two tangent lines, one falling where the slope is negative and one rising where it is positive; then the three rules of derivatives listed.
Einstein's notation, and derivatives

Slopes of x² − 1.5x

Geometrically, the derivative can be understood as the slope of the tangent passing through a point, or equivalently as the best first-order approximation of the function itself at that point, as shown in Figure F.2.1.

A convex bowl with a straight chord between two points above it; the curve stays below the chord. Beside it a bumpy curve where a chord crosses it.
Convergence of gradient descent

Convex: Below Every Chord

We can generalize this idea with the concept of convexity of a function. There are many possible definitions of convexity; we choose the one below for simplicity of exposition.

A large grid of grey cells for a model's parameters with only a thin strip of extra cells highlighted; below, a dataset where only some rows have labels.
Variants of supervised learning

Tuning Few Parameters

Fine-tuning is made especially easy by the presence of large open-source repositories online.

The MAP formula split into a likelihood term and a regularizer term; with a flat prior the regularizer becomes a constant and only maximum likelihood remains.
Maximum likelihood, and Bayesian learning

MAP: Likelihood Plus Regularizer

The overall beauty of this setup is marred by the fact that the posterior is in general impossible to compute in closed form, except for very specific choices of prior and likelihood. Lacking this, one is forced to approximate solutions, either by Markov chain Monte Carlo or by variational inference.

Как это сделано

Примеры для: a curious adult with school maths who uses AI apps every day; add one modern instance per idea, such as how a chat assistant or a phone's photo app is trained

Источник

Лекции пересказывают с английского оригинала книгу arXiv 2404.17625 (version 3, 2025), автор — Симоне Скардапане; книга вышла под открытой лицензией, которая разрешает пересказ и перевод. Пересказ, рисунки и голос сделаны ИИ и сверены с книгой второй моделью. Как используется ИИ.

Лекции — переработка оригинала и распространяются под той же лицензией: их можно свободно распространять и перерабатывать с указанием автора и ссылкой на эту страницу: CC BY-SA 4.0

Ваша трудная книга — так же

Загрузите учебник или любую сложную книгу. Первые пять минут бесплатно.

Загрузить книгу