F1 Score Explained

The F1 score is one of the most commonly used metrics for evaluating classification models. It combines precision and recall into a single score, making it useful when you want to evaluate how well a model identifies positive cases while also limiting incorrect predictions.

Unlike accuracy, the F1 score can provide a more informative view when classes are imbalanced or when both false positives and false negatives matter. It is especially common in tasks such as spam detection, fraud detection, information retrieval, medical classification, and text classification.

What Is the F1 Score?

The F1 score is the harmonic mean of precision and recall. It produces a value between 0 and 1, where a higher score generally indicates better classification performance.

Precision measures how many of the examples predicted as positive were actually positive. Recall measures how many of the actual positive examples the model successfully identified. The F1 score combines these two measurements into one metric.

The key idea is that a model needs both reasonably high precision and reasonably high recall to achieve a high F1 score. A very high value for one metric cannot completely compensate for a very low value for the other.

F1 Score Formula

The F1 score is calculated as the harmonic mean of precision and recall:

F1 = 2 × (Precision × Recall) / (Precision + Recall)

The formula can also be written using true positives, false positives, and false negatives:

F1 = 2TP / (2TP + FP + FN)

True negatives are not included directly in the F1 formula. This is one reason the metric is particularly useful when the positive class is the main focus and the number of negative examples is much larger.

Why Does the F1 Score Use a Harmonic Mean?

The harmonic mean is useful because it penalizes situations where precision and recall are very different. A simple arithmetic average could produce a deceptively high result when one metric is excellent and the other is poor.

For example, suppose a model has 0.9 precision and 0.1 recall. Their arithmetic average is 0.5, which might make the model appear moderately effective. The F1 score, however, is approximately 0.18. This better reflects the fact that the model performs poorly on one of the two important dimensions.

Precision, Recall, and F1 Score

To understand F1 properly, it is important to understand the two metrics it combines.

Consider a spam classifier. If the model labels legitimate emails as spam, those predictions are false positives and reduce precision. If the model allows spam emails through, those are false negatives and reduce recall. The F1 score summarizes the trade-off between these two types of errors.

F1 Score and the Confusion Matrix

The F1 score can be calculated directly from a binary classification confusion matrix. A confusion matrix contains four important values: true positives, true negatives, false positives, and false negatives.

For F1, the most important values are TP, FP, and FN. True negatives do not appear in the formula because F1 focuses on the positive class.

How to Calculate F1 Score Step by Step

Suppose a binary classification model produces the following results:

First calculate precision. The model made 100 positive predictions, of which 80 were correct.

Precision = TP / (TP + FP)
Precision = 80 / (80 + 20)
Precision = 0.80

Next calculate recall. There were 90 actual positive examples, and the model correctly identified 80 of them.

Recall = TP / (TP + FN)
Recall = 80 / (80 + 10)
Recall ≈ 0.889

Finally, use precision and recall to calculate the F1 score.

F1 = 2 × (0.80 × 0.889) / (0.80 + 0.889)
F1 ≈ 0.842

The resulting F1 score is approximately 0.84, or 84.2% when expressed as a percentage.

Interpreting F1 Score Values

The F1 score ranges from 0 to 1. A score of 1 means that both precision and recall are perfect. A score of 0 occurs when the model has no useful performance on the positive class under the relevant calculation.

F1 Score vs Accuracy

Accuracy measures the proportion of all predictions that are correct. It is calculated as the number of correct predictions divided by the total number of predictions.

Accuracy = (TP + TN) / (TP + TN + FP + FN)

Accuracy can become misleading when the classes are heavily imbalanced. For example, imagine a fraud detection dataset where only 1% of transactions are fraudulent. A model that predicts every transaction as legitimate could achieve approximately 99% accuracy while detecting no fraud at all.

The F1 score focuses on precision and recall for the positive class, so it can expose this type of failure more clearly.

F1 Score vs Precision and Recall

F1 should not automatically replace precision or recall. The three metrics answer different questions.

If false positives are especially expensive, precision may be more important. For example, an email security system may want to avoid incorrectly blocking legitimate messages. If false negatives are especially dangerous, recall may matter more. A medical screening system may prioritize finding as many potentially positive cases as possible.

F1 is most useful when both types of errors matter and you want one metric that represents their balance.

When Should You Use the F1 Score?

F1 is a good candidate when the positive class is important, the dataset may be imbalanced, and both false positives and false negatives matter.

• Spam and unwanted-message detection
• Fraud and anomaly classification
• Information retrieval and search
• Text classification
• Named entity recognition
• Document classification
• Binary classification with imbalanced classes
• Tasks where precision and recall need to be considered together

It is less appropriate when precision and recall have very different business or safety costs. In those cases, optimizing a single F1 score may hide an important trade-off.

When Should You Not Use F1 Alone?

F1 is only one view of model performance. A model can have a respectable F1 score while still behaving poorly in a way that matters for the application.

• When false positives and false negatives have very different costs
• When true negatives are important to the evaluation
• When the actual class distribution needs to be represented explicitly
• When probability calibration matters
• When ranking quality is more important than classification quality
• When different classes have substantially different importance

For example, a fraud detection model might need very high recall even if precision decreases. Optimizing F1 alone could choose a threshold that is not appropriate for the real-world cost of missing fraud.

F1 Score and Classification Thresholds

Many classification models produce a probability or confidence score rather than an immediate class label. A threshold is then used to decide whether an example should be classified as positive.

Changing the threshold changes the number of predicted positives. As a result, precision and recall usually change as well, which means the F1 score can change too.

For example, lowering a positive-class threshold may cause the model to identify more positive cases. This can increase recall but also produce more false positives, reducing precision. Raising the threshold can have the opposite effect.

Maximum F1 Does Not Always Mean Best Model

A common mistake is to select the model with the highest F1 score automatically. In practice, the best model depends on the actual objective.

Suppose two fraud detection models have similar F1 scores. Model A has higher recall and lower precision, while Model B has higher precision and lower recall. If missing a fraudulent transaction is much more expensive than reviewing a legitimate transaction, Model A may be preferable even if the F1 scores are nearly identical.

F1 is therefore better viewed as an evaluation metric rather than a universal optimization target.

F1 Score for Imbalanced Datasets

Class imbalance occurs when one class contains substantially more examples than another. Examples include fraud detection, rare disease classification, and some forms of anomaly detection.

In such cases, accuracy can be dominated by the majority class. F1 can provide a more useful perspective because it evaluates precision and recall for the positive class without directly counting true negatives.

However, F1 does not magically solve class imbalance. You still need to examine the class distribution and understand which errors the model makes. Precision, recall, the confusion matrix, and potentially other metrics should be considered alongside F1.

Macro, Micro, and Weighted F1

For multiclass classification, there is not always one obvious positive class. F1 can therefore be calculated separately for each class and then aggregated using different averaging strategies.

Macro F1 is particularly useful when performance on minority classes should not be hidden by a large majority class. Weighted F1 can be easier to interpret when class frequencies are important. Micro F1 gives greater influence to classes with more examples.

Binary vs Multiclass F1

In binary classification, F1 is commonly reported for the positive class. In multiclass classification, the metric needs an averaging strategy such as macro, micro, or weighted averaging.

For example, a three-class classifier might achieve very different F1 scores for class A, class B, and class C. Reporting only one averaged score could hide the fact that the model performs poorly on one particular class.

F1 Score vs F-Beta Score

The F1 score gives precision and recall equal importance. Sometimes that is not appropriate. The F-beta score extends the same idea by allowing you to give more weight to either precision or recall.

When beta is greater than 1, recall receives more weight. When beta is less than 1, precision receives more weight. F1 is simply the special case where beta equals 1.

This makes F-beta useful when the costs of false positives and false negatives are asymmetric.

Advantages of the F1 Score

• Combines precision and recall into a single metric
• Penalizes a large imbalance between precision and recall
• Can be more informative than accuracy for imbalanced classification
• Is easy to calculate and compare
• Is widely used across machine learning and information retrieval
• Works naturally with confusion-matrix-based evaluation

Limitations of the F1 Score

• Does not directly consider true negatives
• Does not reflect the different costs of false positives and false negatives
• Can hide differences between precision and recall
• Does not evaluate probability calibration
• Can be difficult to interpret without knowing the class distribution
• A single average can hide poor performance on individual classes

Common Mistakes When Using F1

• Using F1 without checking precision and recall separately
• Comparing F1 scores produced with different evaluation datasets
• Ignoring class imbalance
• Choosing the highest F1 without considering the real objective
• Reporting weighted or micro F1 without explaining the averaging method
• Treating an F1 score as universally good or bad
• Optimizing the test set repeatedly to improve the reported score

F1 Score Example in Python

In Python, libraries such as scikit-learn can calculate the F1 score directly from the true labels and model predictions.

from sklearn.metrics import f1_score

actual = [1, 1, 1, 0, 0, 0]
predicted = [1, 1, 0, 0, 0, 1]

score = f1_score(actual, predicted)
print(score)

For multiclass problems, the averaging strategy can be specified explicitly.

score = f1_score(actual, predicted, average="macro")

When reporting a model's F1 score, always specify whether the task is binary or multiclass and, for multiclass classification, which averaging method was used.

F1 Score Best Practices

• Report precision, recall, and F1 together when possible
• Use a validation set to choose model parameters and classification thresholds
• Keep the final test set separate for unbiased evaluation
• Check the confusion matrix to understand the types of errors
• For multiclass tasks, inspect per-class metrics
• Choose the averaging strategy deliberately
• Consider F-beta when precision and recall have different importance
• Compare the metric against a meaningful baseline
• Evaluate the metric on data that represents the real deployment environment

Frequently Asked Questions

Conclusion

The F1 score is a classification metric that combines precision and recall into a single number. Its harmonic-mean formulation makes it sensitive to poor performance in either metric, which is why it is often useful for imbalanced classification problems.

F1 should not be treated as a universal measure of model quality. The best evaluation strategy depends on the problem and the costs of different errors. In practice, F1 is most useful when interpreted together with precision, recall, the confusion matrix, class distribution, and other metrics relevant to the application.