STATISTICS GUIDE · CLASSIFICATION
When to use logistic regression
Logistic regression estimates the chance of a category, such as “pass” or “not pass.” This guide explains the idea, shows a small Python example, and explains when another model may fit better.
Quick answer: use logistic regression when the result you want to predict is a category. It first estimates a probability. A rule you choose then turns that probability into a class label.
What does logistic regression mean?
Logistic regression is a model for predicting a category. Despite the word “regression,” it is used as a classifier in machine learning. For a yes-or-no question, it estimates the probability that the answer is yes. For example, it could estimate the chance that a student passes based on study hours.
The model combines its input values into a score, then applies a logistic, or S-shaped, function. The result stays between 0 and 1, so it can be read as a probability estimate. In scikit-learn, predict_proba() returns class probabilities; predict() returns a label using the model’s decision rule. The default binary cutoff is 0.5, but a project may need a different cutoff.
Example: estimate the chance of passing
Here, hours is the input and passed is the target: 1 means “passed,” and 0 means “did not pass.” These eight rows are only a teaching example. They are too small to support a real-world claim about students.
import numpy as np
from sklearn.linear_model import LogisticRegression
hours = np.array([1, 2, 3, 4, 5, 6, 7, 8]).reshape(-1, 1)
passed = np.array([0, 0, 0, 1, 0, 1, 1, 1])
model = LogisticRegression()
model.fit(hours, passed)
new_hours = np.array([[4.5], [6.5]])
probability_pass = model.predict_proba(new_hours)[:, 1]
predicted_class = model.predict(new_hours)
for h, p, label in zip(new_hours[:, 0], probability_pass, predicted_class):
print(f"{h:.1f} hours: probability={p:.3f}, class={label}")
The probability values can help rank cases from lower to higher estimated chance. The predicted class depends on the cutoff. A probability is not a promise, and the code does not show how well the model will work on new students.
When should you use logistic regression?
- Your target is a category. Examples include fraud/not fraud, churn/stay, or pass/fail.
- You need probabilities. A probability can support a later decision, such as reviewing only high-risk cases.
- You want a model that is relatively easy to inspect. Its coefficients describe changes in log-odds, though that phrase needs care when explaining results.
- You can check the result on held-out data. Use measures that fit your task, such as precision, recall, a confusion matrix, or log loss.
For a continuous target, such as a test score or house price, begin with a regression model for numeric outcomes. If the relationship is strongly curved, compare suitable transformations or nonlinear models. If the categories have more than two possible values, choose a multiclass setup and check how your software handles it.
Limits that matter
Logistic regression does not prove that an input causes an outcome. A high accuracy score can also hide a bad model when one class is much more common than another. Check class balance, inspect mistakes, and test the model on data that was not used to fit it.
Probability estimates can be poorly calibrated. If decisions depend on the probability itself, compare predicted risks with actual rates and consider calibration methods. Choose the cutoff based on the cost of false positives and false negatives, rather than treating 0.5 as automatically correct.
Frequently asked questions
Is logistic regression a classification or regression method?
In machine-learning libraries such as scikit-learn, it is a classification method. It estimates probabilities for class labels. The name comes from the statistical model used to link inputs to those probabilities.
Does logistic regression always use a 0.5 threshold?
No. Some software uses 0.5 by default for binary classes, but you can choose another cutoff. The useful cutoff depends on the cost of each kind of mistake and how reliable the probabilities are.
Can logistic regression predict more than two classes?
Yes. Many implementations support multiclass logistic regression. Check the library’s settings and class order, then evaluate each class rather than relying only on one overall accuracy number.
Can I use logistic regression when my target is a number?
Use it when the target represents categories. For a continuous number such as revenue or temperature, use a model designed for numeric outcomes unless you have a clear reason to turn that number into categories.
Sources and related guides
- scikit-learn: Logistic regression — probability estimates, binary classes and the default decision threshold.
- scikit-learn: LogisticRegression API — fitting, predicting labels and returning probabilities.