Python Model Evaluation

In the last lesson, Training a Model, you trained a model and it predicted the first five flowers correctly. So it looks good, right? But “looks good on five” is not proof. A model can ace a handful and fail the rest. Before you ever trust a model, you need a real number that says how well it does, and you need to know the traps that make a good-looking number lie. That is what evaluation is, and this lesson teaches it.

🤔 Why eyeballing predictions is not enough

Last time we printed five predictions and they matched. That felt convincing. But it proves almost nothing.

Think about it:

• Five out of thirty is a tiny sample. The model could be wrong on the other twenty-five.
• “They look right” is a feeling, not a measurement. You cannot compare two models with a feeling.
• Some mistakes matter more than others. Missing a disease is far worse than a false alarm, and a single glance hides that.

So we need numbers. Evaluation means measuring how well a model does with real scores, on the test set it never trained on. A number lets you trust the model, compare it to others, and know when it needs work.

🎯 Accuracy: the first score

The simplest measure is accuracy: out of all the test examples, what fraction did the model get right? If it got 28 out of 30 correct, that is 28/30, about 0.93, or 93%.

scikit-learn computes it for you with accuracy_score. You give it the real answers and the model’s predictions.

This trains the model and measures its accuracy on the test set:

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier
from sklearn.metrics import accuracy_score

data = load_iris()
X_train, X_test, y_train, y_test = train_test_split(
    data.data, data.target, test_size=0.2, random_state=42
)

model = DecisionTreeClassifier(random_state=42)
model.fit(X_train, y_train)
predictions = model.predict(X_test)

score = accuracy_score(y_test, predictions)
print("Accuracy:", score)

Accuracy: 1.0

So accuracy_score(y_test, predictions) compares the real answers to the predictions and returns the fraction correct. Here it is 1.0, which means 100%, every one of the 30 test flowers was classified right. The iris dataset is famously easy, so a perfect score is normal here. On harder, real-world data you would see something like 0.85 or 0.90, and that is fine.

📊 The confusion matrix: where the mistakes are

Accuracy gives you one number, but it hides where the model went wrong. The confusion matrix is a small table that shows exactly which types got confused for which. It is the difference between “you got 2 wrong” and “you mixed up versicolor and virginica”.

This builds the confusion matrix for our predictions:

from sklearn.metrics import confusion_matrix

matrix = confusion_matrix(y_test, predictions)
print(matrix)

[[10  0  0]
 [ 0  9  0]
 [ 0  0 11]]

Here is how to read it. Each row is the real flower type, and each column is what the model predicted. The numbers on the diagonal (top-left to bottom-right) are the correct predictions. Anything off the diagonal is a mistake.

• Row 1: 10 flowers were really type 0, and all 10 were predicted as type 0. Perfect.
• The diagonal holds 10, 9, 11, all correct, and every off-diagonal spot is 0, so there were no mistakes.

If the model had confused two types, you would see a number off the diagonal. For example, a 2 in row 2, column 3 would mean “two type-1 flowers were wrongly predicted as type 2”. That tells you exactly which pairs the model struggles with, which a single accuracy number never could.

⚖️ The trap of overfitting

Here is the most important warning in machine learning. A model can score perfectly on the data it trained on and still be useless. This problem is called overfitting.

Overfitting means the model memorized the training data instead of learning the general pattern. Picture a student who memorizes the answers to last year’s exam word for word. They score 100% on that exact paper, but give them a new question and they are lost. They learned the answers, not the subject.

How you spot it:

• The model scores very high on the training data but much lower on the test data. That gap is the warning sign.
• A model that learned the real pattern scores similarly on both. A model that memorized scores great on training and poorly on the test.

This checks for overfitting by comparing the two scores:

train_score = model.score(X_train, y_train)
test_score = model.score(X_test, y_test)

print("Training accuracy:", train_score)
print("Test accuracy:", test_score)

Training accuracy: 1.0
Test accuracy: 1.0

Here both scores are high and close together, so this model is healthy, not overfitting. But if you saw training 1.0 and test 0.65, that big gap would scream “it memorized the training data and does not really understand”. That is why we always check the test score, never just the training score.

🌍 Common evaluation tools

Here are the measures you will reach for, beyond plain accuracy.

⚠️ Common Mistakes

A few evaluation traps:

• Judging a model on its training accuracy. Of course it does well on data it studied. Always report the test score.
• Trusting accuracy alone on uneven data. If 95% of emails are not spam, a lazy model that says “not spam” every time scores 95% while catching zero spam. The confusion matrix exposes this; accuracy alone hides it.
• Ignoring a big gap between training and test scores. That gap is the clearest sign of overfitting and should never be brushed aside.

✅ Best Practices

Habits for honest evaluation:

• Always measure on the test set, the data the model never trained on.
• Look at both the training and test scores together. A large gap means overfitting.
• Use the confusion matrix, not just accuracy, so you see where the mistakes actually are.
• For uneven data, where one class is rare, lean on the classification report instead of trusting accuracy alone.

🧩 What You’ve Learned

✅ Evaluation means measuring a model with real numbers on the test set, not by eyeballing a few predictions.

✅ Accuracy is the fraction of predictions that were correct, from accuracy_score(y_test, predictions).

✅ The confusion matrix is a table showing which classes got confused for which, with correct predictions on the diagonal.

✅ Overfitting is when a model memorizes the training data: high training score, low test score, a big gap between them.

✅ Accuracy alone can lie on uneven data, so use the confusion matrix and the classification report too.

🚀 What’s Next?

You can now train a model and judge it honestly with real scores. The simple models you have used learn from rows and columns of numbers. But some problems, like recognizing faces or understanding speech, need something more powerful. Next we meet deep learning and the neural networks behind it.

Introduction to Deep Learning