STATISTICS GUIDE · MODEL CHOICE

Types of linear regression in machine learning

“Regression” can mean a family of models that predict numbers. This guide compares common types, shows how their prediction shapes differ, and clears up why logistic regression is usually a classification method.

Beginner-friendly model guideDocs checked October 30, 2026

Quick answer: start with linear regression for a roughly straight numeric relationship. Try polynomial or other nonlinear models when the pattern bends. Use regularization when a linear model is too sensitive to its training data. Compare models on the same held-out cases.

An infographic comparing simple and multiple linear regression, polynomial regression, ridge and lasso, tree regression, and logistic classification. — **Infographic: model families**The word “regression” is used broadly. The target type and model assumptions still matter.

Illustrative prediction shapes: a straight line, a curved polynomial, and a piecewise-constant tree. — **Prediction shape matters**These curves are illustrative, not fitted to a dataset. They show why model families can behave differently.

Common types of regression models

Model	What changes	Useful starting point
Simple linear regression	One input predicts one numeric outcome with a straight-line relationship.	A small, easy-to-explain baseline.
Multiple linear regression	Several inputs predict one numeric outcome.	When more than one measured factor may help explain the outcome.
Polynomial regression	Adds powers such as x² so a linear model can follow a curve.	A curved pattern that is still smooth and manageable.
Ridge, Lasso, Elastic Net	Add a penalty that can shrink coefficient values; Lasso can also set some to zero.	Many or correlated input features, with tuning checked on held-out data.
Decision tree regression	Splits rows into groups and predicts a value for each final group.	Nonlinear patterns and interactions; control tree size to reduce overfitting.
Logistic regression	Estimates class probabilities rather than a continuous number.	Binary or multiclass classification, despite “regression” in its name.

Polynomial regression is often grouped with linear models because the model is linear in its coefficients. For example, it can fit y = b₀ + b₁x + b₂x². The curve bends as x changes, while the unknown coefficients still enter as a weighted sum. Ridge and Elastic Net add penalties that discourage large coefficients; Lasso can also shrink some coefficients to zero.

Example: predicting a number that levels off

Imagine predicting a quiz score from practice time. Scores may rise quickly at first and then level off. A straight line may be too simple across the full range. A polynomial curve may represent the bend; a decision tree may use a few flat steps. These shapes are hypotheses, so check which one predicts new data best.

A practical comparison is to keep some rows aside before training. Fit each candidate on the same training rows, then compare an error measure such as RMSE or MAE on the held-out test rows. For classification, use classification measures instead. Do not compare a model’s training score with another model’s test score.

RegressionTester has calculators for linear regression, polynomial regression and multiple linear regression. They help explore fitted equations; they do not replace a careful holdout evaluation for every use case.

A simple model-selection checklist

Check the target. Is it a number, a yes/no label, or one of several classes?
Plot the data. Look for a straight pattern, a curve, groups, outliers or changing spread.
Fit a simple baseline. Keep the result interpretable and record the data split.
Compare realistic alternatives. Tune model complexity using training/validation data, then report final performance on untouched test data.
Check the cost of mistakes. A slightly more accurate model may be harder to explain or maintain.

Frequently asked questions

What are the main types of linear regression in machine learning?

Common forms include simple and multiple linear regression, polynomial regression, Ridge, Lasso and Elastic Net. They differ by number or type of input terms and by whether a penalty is added to the fit.

Is polynomial regression still linear regression?

It is linear in the fitted coefficients, even though its curve can bend with x. Adding x² or x³ creates new input features; the coefficients are still combined linearly.

Is logistic regression used to predict a number?

Usually it predicts class probabilities, such as the chance of yes or no. It is grouped with classifiers in common machine-learning libraries.

Which regression model is best?

There is no best model for every dataset. Compare candidates on the same data split, use a metric that matches the task, inspect errors and consider whether people need to understand the model.

Sources and related guides

scikit-learn: Linear models — ordinary least squares, Ridge, Lasso, Bayesian methods and logistic classification.
scikit-learn: Decision trees — tree regression behavior, step-like predictions and complexity controls.