Classification problems sit at the center of many real-world AI applications. These models map data inputs (features) and transform that data into probabilities that represent the likelihood a row is classified as each of the discrete labels (labels).
Whether you are:
- Flagging a financial transaction as fraudulent or safe
- Deciding if a CT scan shows signs of pneumonia
- Predicting whether a customer will churn next month
- Ranking the top products a shopper is likely to buy
Classification turns raw data into structured decisions. The real power isn’t just in picking the most likely label. It’s in using the probability scores themselves to tune thresholds, balance tradeoffs, rank results, and achieve different outcomes depending on the task.
Binary classification
Classification models don’t just output a label, they output a probability. For example, flagging a transaction that has a 0.91 probability of being fraudulent.
By default, we might assign the label “fraud” if the score is greater than or equal to 0.5. However, thresholds can be tuned depending on the business need.
Lower threshold (for example 0.3):
- More predictions are labeled as fraud
- More fraud cases are caught (fewer false negatives)
- More safe transactions are incorrectly flagged (more false positives)
Higher threshold (for example 0.8):
- Only the riskiest transactions are flagged
- Fewer safe transactions are incorrectly flagged (fewer false positives)
- More fraudulent transactions slip through (more false negatives)
Adjusting thresholds doesn’t assume that the predictions follow any specific distribution. What matters is calibration and whether the model’s predicted probabilities are close to actual real-world frequencies.
Where error rate is a conceptual indicator of how many false outcomes of each type occur at different thresholds. For example, a data scientist could implement the following thresholds for automated decision-making and categorization of predicted labels where high confidence in either direction is required.
Multi-class classification
When there are multiple possible labels, models usually output a full probability distribution across those labels. In a multi-class setting (where only one label is correct), the outputs are produced with a softmax function and may sum to 1. For example:
- Cat: 0.10
- Dog: 0.70
- Rabbit: 0.20
In a multi-label setting (where more than one label can be true), each label is scored independently with a sigmoid function. In this case, the probabilities do not need to sum to 1 because multiple classes may apply at once.
The default approach in multi-class problems is argmax, which selects the class with the highest probability. In the example above, the model would predict Dog.
But there are variations of argmax that may be useful in practice:
- Hard argmax selects the single class with the maximum score. This is the most common approach across classification tasks.
- Top-k argmax returns the top k predictions instead of only one. For example, both Dog and Rabbit might be returned. This is useful in recommendation systems.
- Soft variants sample from the distribution instead of always choosing the maximum. This is sometimes used in reinforcement learning or generative models where controlled randomness is valuable.
Beyond labels: using probabilities for business value
The predicted label is only half the story. The probabilities themselves open much richer possibilities:
- Ranking allows products, ads, or job applicants to be sorted by probability, enabling personalized recommendations.
- Prioritization supports use cases like healthcare, where doctors may need a risk score to prioritize patients.
- Decision support moves beyond binary outcomes. Instead of just approving or denying a loan, probability scores can inform tiers of human review or escalation paths.
In other words, the raw prediction powers automation, while the probabilities power intelligence. These scores can be used in many different applications and layered in different ways to achieve the outcomes that matter most to the business.
Ranking Diagram:
Items → Model → Probabilities
———————————-
Item A → 0.85
Item B → 0.60
Item C → 0.25
Stack Rank → A > B > C
Classification metrics
Adjusting thresholds changes the balance between false positives and false negatives, so it’s important to measure how the model performs.
- Precision tells us how many of the items predicted as positive were actually correct.
- Recall tells us how many of the true positives the model successfully identified.
- The F1 score combines precision and recall into a single number to capture their balance.
To evaluate models across all possible thresholds, we often use the ROC curve and its AUC (Area Under the Curve), which summarize the overall ability of a model to separate classes. These metrics ensure that threshold choices are tied to business goals, such as minimizing false alarms or catching as many true cases as possible.
Takeaway
Classification models give us more than labels, they give us scores we can act on. By tuning thresholds, adjusting decision rules, and leveraging probability distributions, we can build systems that are ccurate and aligned to business goals. This could mean catching fraud, predicting failure, or making the right recommendation at the right time.
