07
7 minute interactive lab
Gradient descent
Which way should every weight move?
YOU’LL LEAVE KNOWINGTraining is repeated, local improvement of the weights.
THE EXPERIMENT
Find the bottom without seeing the map.
Move the controls. Watch the internal state respond. The explanation follows the behavior—not the other way around.
Runs the real small-scale equationThe displayed values are calculated from the formula shown in this lesson.
OPTIMIZERWalk downhill
This step size makes useful progress without much overshoot.
w = 3.80minimum loss
current loss7.840
Training is repeated, local improvement of the weights.
The gradient is the local slope of the loss. It tells the optimizer which direction increases error, so the weight moves the other way. The learning rate controls how far it trusts that local clue.
IN DEVELOPER TERMS
weight -= learning_rate × gradientGO DEEPER
Check the model against the source.
Connectionism optimizes for intuition, then points you to the rigorous treatment.
Who gets blamed for a wrong answer?