Lesson 20 — Gradient descent

Gradient descent with two weights

Now the model has two weights: pred = w * x + b, where b is the bias (the intercept). Training updates both. The gradient for w is error × x; the gradient for b is just error. Going through the data example by example and updating after each one is called stochastic gradient descent (SGD). One pass over all data is an epoch.

miss = (k * h + c) - p
k -= rate * miss * h
c -= rate * miss

Why it matters: real models have millions of weights, but each one is updated exactly like this.