Lesson 13 — Train / test split

Test on data the model has never seen

A model can simply memorise its training data. To find out whether it really learned, we hide part of the data: the training set is used for learning, the test set only for the final check. A common split is 80% / 20%. Python slicing makes this easy: data[:cut] is everything before position cut, data[cut:] is everything from cut on.

data = [1, 2, 3, 4, 5]
data[:4], data[4:]   # ([1, 2, 3, 4], [5])

Why it matters: a model that scores 100% on data it trained on and 50% on new data is useless. Only the test score tells the truth.