Confusion Matrix Explained
A practical guide to confusion matrices, including TP, TN, FP, FN, classification errors, normalized matrices, multiclass classification, and model evaluation.
A confusion matrix is one of the most useful tools for understanding the performance of a classification model. Instead of reducing model performance to a single number, it shows how predictions are distributed across correct and incorrect outcomes.
The confusion matrix is built around four fundamental concepts: true positives, true negatives, false positives, and false negatives. These values can then be used to calculate metrics such as accuracy, precision, recall, specificity, and F1 score.
Understanding a confusion matrix is important because two models can have similar accuracy while making very different types of mistakes. Looking at the individual prediction outcomes reveals which errors the model is actually making.
What Is a Confusion Matrix?
A confusion matrix is a table that compares the actual classes of observations with the classes predicted by a classification model. Each cell contains the number of observations that fall into a particular actual-predicted combination.
For binary classification, there are two possible classes, usually called positive and negative. This produces a 2 × 2 matrix containing four possible outcomes.
| Actual / Predicted | Positive | Negative |
|---|---|---|
| Positive | True Positive (TP) | False Negative (FN) |
| Negative | False Positive (FP) | True Negative (TN) |
The word true or false describes whether the prediction was correct. Positive or negative describes the class that was predicted or observed.
True Positive (TP)
A true positive occurs when the model predicts the positive class and the actual class is also positive.
For example, if a fraud detection model predicts that a transaction is fraudulent and the transaction really is fraudulent, the prediction is a true positive.
- Predicted: positive
- Actual: positive
- Prediction: correct
True Negative (TN)
A true negative occurs when the model predicts the negative class and the actual class is also negative.
For example, if a spam classifier predicts that an email is legitimate and the email really is legitimate, the prediction is a true negative.
- Predicted: negative
- Actual: negative
- Prediction: correct
False Positive (FP)
A false positive occurs when the model predicts the positive class even though the actual class is negative. It is a false alarm.
For example, if a spam filter marks a legitimate email as spam, the result is a false positive.
- Predicted: positive
- Actual: negative
- Prediction: incorrect
False Negative (FN)
A false negative occurs when the model predicts the negative class even though the actual class is positive. It is a missed positive case.
For example, if a security system fails to identify an actual malicious event, the result is a false negative.
- Predicted: negative
- Actual: positive
- Prediction: incorrect
The Four Outcomes at a Glance
| Outcome | Actual class | Predicted class | Meaning |
|---|---|---|---|
| TP | Positive | Positive | Correctly detected positive case |
| TN | Negative | Negative | Correctly detected negative case |
| FP | Negative | Positive | Incorrect positive prediction |
| FN | Positive | Negative | Missed positive case |
These four values form the basis for many classification metrics. Once they are known, the model's error pattern becomes much easier to analyze.
A Practical Example
Imagine a binary classifier that detects fraudulent transactions. Suppose the model is evaluated on 1,000 transactions. There are 100 fraudulent transactions and 900 legitimate transactions.
| Actual / Predicted | Fraud | Legitimate |
|---|---|---|
| Fraud | 80 | 20 |
| Legitimate | 30 | 870 |
In this example, the model correctly identifies 80 fraudulent transactions, so there are 80 true positives. It misses 20 fraudulent transactions, producing 20 false negatives.
The model also incorrectly identifies 30 legitimate transactions as fraudulent, producing 30 false positives. The remaining 870 legitimate transactions are correctly classified as negative, producing 870 true negatives.
What Does a Good Confusion Matrix Look Like?
For a binary classifier, a strong model generally has large values on the main diagonal of the confusion matrix and relatively small values in the off-diagonal cells.
The main diagonal represents correct predictions: true positives and true negatives. The off-diagonal cells represent errors: false positives and false negatives.
However, there is no universal requirement that both types of errors must be equally small. In some applications, a model may intentionally accept more false positives to reduce false negatives, or vice versa.
Confusion Matrix and Accuracy
Accuracy measures the proportion of all predictions that are correct. In terms of the confusion matrix, correct predictions are the true positives and true negatives.
Accuracy is useful when the classes are reasonably balanced and false positives and false negatives have similar importance.
However, accuracy can hide serious problems in imbalanced datasets. A model can correctly classify the majority class almost every time while failing to detect the minority class.
Confusion Matrix and Precision
Precision measures the reliability of positive predictions. It compares true positives with all cases that the model predicted as positive.
The confusion matrix makes it easy to understand why precision changes. If false positives increase while true positives remain constant, precision decreases.
This makes the confusion matrix particularly useful when false alarms are a major concern.
Confusion Matrix and Recall
Recall measures how many actual positive cases the model successfully identifies. It compares true positives with all actual positive cases.
If false negatives increase while true positives remain constant, recall decreases. The confusion matrix therefore provides a direct way to see why a model has poor recall.
For example, a medical screening model may have high recall if it detects nearly all positive cases, even if it produces additional false positives that require further investigation.
Confusion Matrix and Specificity
Specificity measures how effectively a model identifies negative cases. It focuses on true negatives and false positives.
High specificity means that the model rarely classifies negative cases as positive. This can be important when false alarms create significant costs or unnecessary work.
Confusion Matrix and F1 Score
The F1 score combines precision and recall into a single metric. Because both precision and recall are derived from the confusion matrix, the matrix provides the underlying information required to calculate F1.
F1 can be useful when both false positives and false negatives matter. However, the confusion matrix remains valuable because it shows the actual distribution of errors rather than hiding them behind one summary score.
Normalized Confusion Matrix
A standard confusion matrix contains raw counts. A normalized confusion matrix converts those counts into proportions or percentages, making it easier to compare datasets or classes with different sizes.
For example, suppose one class contains 10,000 observations and another contains only 500. Raw counts may make the larger class dominate the visual representation. Normalization can show the proportion of each class that was correctly or incorrectly classified.
Normalization can be performed by rows, columns, or across the entire matrix. Row normalization is particularly useful for understanding how actual classes are distributed among predictions.
Binary vs Multiclass Confusion Matrices
Binary classification has two classes and therefore produces a 2 × 2 confusion matrix. Multiclass classification has more than two classes, producing a larger matrix.
For example, a model that classifies images into cats, dogs, and birds produces a 3 × 3 matrix. Each row represents an actual class and each column represents a predicted class.
| Actual / Predicted | Cat | Dog | Bird |
|---|---|---|---|
| Cat | TP for Cat | Cat → Dog | Cat → Bird |
| Dog | Dog → Cat | TP for Dog | Dog → Bird |
| Bird | Bird → Cat | Bird → Dog | TP for Bird |
The diagonal cells represent correct predictions for each class. Off-diagonal cells show which classes are being confused with one another.
How to Read a Multiclass Confusion Matrix
A multiclass confusion matrix can reveal problems that an overall accuracy score would hide. For example, a model might classify cats and dogs very well but frequently confuse birds with another class.
To analyze a particular class, treat that class as the positive class and all other classes as negative. This allows per-class precision, recall, specificity, and F1 scores to be calculated.
- Inspect the diagonal to see which classes are recognized correctly.
- Inspect each row to see where actual examples of a class are being sent.
- Inspect each column to see what types of observations are being predicted as a particular class.
- Look for pairs of classes that are frequently confused.
- Compare per-class metrics instead of relying only on aggregate performance.
Rows vs Columns: Why Orientation Matters
Confusion matrices are commonly displayed with actual classes on rows and predicted classes on columns, but some libraries or visualizations use the opposite orientation.
This distinction matters when interpreting individual cells. Before analyzing a confusion matrix, check which axis represents actual labels and which represents predicted labels.
Confusion Matrix for Imbalanced Classification
Confusion matrices are especially useful for imbalanced classification because they show the number of minority-class cases that were missed and the number of majority-class cases that were incorrectly flagged.
Consider a dataset with 99,000 legitimate transactions and 1,000 fraudulent transactions. A model that predicts everything as legitimate can achieve 99% accuracy, but its confusion matrix would reveal that every fraudulent transaction is a false negative.
This example demonstrates why accuracy without the underlying prediction counts can provide a misleading impression of model quality.
Using the Confusion Matrix to Diagnose Errors
One of the biggest advantages of a confusion matrix is that it supports error diagnosis. Instead of asking only whether the model is good or bad, you can ask what type of mistake is occurring.
| Observed problem | Possible interpretation |
|---|---|
| Many false positives | The model may be too aggressive in predicting the positive class |
| Many false negatives | The model may be too conservative or insufficiently sensitive |
| One class has many errors | The features may not distinguish that class well |
| Two classes are frequently confused | Those classes may have overlapping characteristics |
| Training matrix is much better than test matrix | The model may be overfitting |
Changing the Classification Threshold
Many classification models produce probabilities or scores rather than directly outputting a final class. A threshold is then used to decide whether an observation should be classified as positive.
Changing the threshold changes the confusion matrix. Lowering the threshold usually causes more observations to be classified as positive. This can increase true positives but may also increase false positives.
Raising the threshold usually makes positive predictions more conservative. This can reduce false positives but may increase false negatives.
| Threshold change | Typical effect |
|---|---|
| Lower threshold | More positive predictions, often higher recall and more false positives |
| Higher threshold | Fewer positive predictions, often higher precision and more false negatives |
Confusion Matrix and Model Comparison
When comparing classification models, confusion matrices can reveal differences that aggregate metrics hide. Two models may have the same accuracy but very different error distributions.
For example, Model A might have more false positives and fewer false negatives, while Model B has fewer false positives but misses many more positive cases. Which model is better depends on the application.
For a security detection system, Model A might be preferable if missing threats is expensive. For an automated system where every false alarm requires significant manual work, Model B might be preferable.
Confusion Matrix vs Classification Report
A classification report usually summarizes metrics such as precision, recall, F1 score, and support for each class. A confusion matrix instead shows the actual counts of prediction outcomes.
| Tool | Main purpose |
|---|---|
| Confusion matrix | Understand which predictions are correct and which classes are being confused |
| Classification report | Summarize precision, recall, F1, and class support |
| Accuracy | Measure overall proportion of correct predictions |
| Precision-recall curve | Analyze the precision-recall trade-off across thresholds |
These tools complement each other. A classification report provides convenient summary metrics, while the confusion matrix helps explain where those metrics come from.
Common Mistakes When Using Confusion Matrices
- Assuming rows and columns always have the same orientation.
- Looking only at the diagonal and ignoring the types of errors.
- Using raw counts without considering class imbalance.
- Interpreting false positives as universally worse than false negatives.
- Comparing matrices built from different datasets without normalization or context.
- Relying only on the confusion matrix without considering the appropriate evaluation metrics.
- Ignoring changes in the matrix after modifying the classification threshold.
- Evaluating only training predictions and assuming they represent generalization performance.
Best Practices
- Define the positive class clearly before interpreting TP, FP, FN, and TN.
- Always check which axis represents actual and predicted classes.
- Inspect false positives and false negatives separately.
- Use the confusion matrix alongside precision, recall, F1, and other relevant metrics.
- Use normalized matrices when class sizes differ significantly.
- Analyze each class separately in multiclass classification.
- Evaluate the matrix on validation or test data rather than only training data.
- Compare confusion matrices at relevant classification thresholds.
- Connect model errors to the real-world cost of incorrect predictions.
- Use confusion matrices to identify specific failure patterns that can guide model improvement.
Frequently Asked Questions
What is a confusion matrix used for?
A confusion matrix is used to analyze classification model predictions by showing true positives, true negatives, false positives, and false negatives. It helps identify the types of mistakes a model makes and provides the basis for several evaluation metrics.
What are TP, TN, FP, and FN?
TP means the model correctly predicted a positive case. TN means it correctly predicted a negative case. FP means it incorrectly predicted a positive case for a negative example. FN means it incorrectly predicted a negative case for a positive example.
Why is a confusion matrix better than accuracy alone?
Accuracy summarizes all predictions into one value and can hide class-specific problems. A confusion matrix shows the individual types of correct and incorrect predictions, making it easier to detect false-positive or false-negative problems.
Can a confusion matrix be used for multiclass classification?
Yes. A multiclass confusion matrix contains one row and one column for every class. The diagonal represents correct predictions, while off-diagonal cells show which classes are being confused with one another.
What does a good confusion matrix look like?
For a well-performing classifier, most observations are usually concentrated on the diagonal, representing correct predictions. However, the acceptable balance between false positives and false negatives depends on the application's requirements.
Conclusion
A confusion matrix provides a detailed view of how a classification model makes predictions. Its four fundamental outcomes—true positives, true negatives, false positives, and false negatives—describe not only whether predictions are correct, but also what kind of mistakes the model makes.
The matrix is especially valuable for imbalanced datasets, threshold analysis, multiclass classification, and model comparison. It also provides the foundation for metrics such as precision, recall, specificity, and F1 score.
Rather than relying on a single accuracy value, use the confusion matrix to understand the model's error profile and connect those errors to the actual requirements of the application. This makes classification evaluation more transparent and provides useful information for improving the model.