Lesson 18 — Gradient over a whole dataset

The gradient over a whole dataset

In Lesson 11 the model learned from a single example. Real training uses many. The loss is MSE over all examples, and its gradient for w is the average of 2 × error × x. One update that uses every example is a batch gradient step.

g = sum(2 * (k * h - p) * h for h, p in zip(hours, pay)) / len(hours)
k = k - rate * g

Watch the loss: if training works, it should drop every step.

Why it matters: "loss going down" is the main thing ML engineers watch while a model trains. If it goes up, the learning rate is usually too big.