The least squares regression line is the exact line of best fit. Instead of drawing by eye, you calculate the slope and intercept from the data, so everyone gets the same answer.
What “least squares” means
For each data point, the vertical distance from the point to the line is called a residual. The least squares line is the line that makes the total of the squared residuals as small as possible. Squaring stops points above and below the line cancelling out, and penalises points that are far away.
The formulas
- x̄ and ȳ are the means of the x and y values.
- Sxy = Σ(x − x̄)(y − ȳ), the sum of the products of the deviations.
- Sxx = Σ(x − x̄)², the sum of the squared x deviations.
- Slope: m = Sxy ÷ Sxx
- Intercept: c = ȳ − m·x̄, because the line passes through the mean point (x̄, ȳ).
Worked example
Find the least squares line for the five points (1, 2), (2, 4), (3, 5), (4, 4) and (5, 5).
Step 1: the means. x̄ = (1 + 2 + 3 + 4 + 5) ÷ 5 = 3 and ȳ = (2 + 4 + 5 + 4 + 5) ÷ 5 = 4.
Step 2: a table of deviations.
| x | y | x − x̄ | y − ȳ | (x − x̄)(y − ȳ) | (x − x̄)² |
|---|---|---|---|---|---|
| 1 | 2 | −2 | −2 | 4 | 4 |
| 2 | 4 | −1 | 0 | 0 | 1 |
| 3 | 5 | 0 | 1 | 0 | 0 |
| 4 | 4 | 1 | 0 | 0 | 1 |
| 5 | 5 | 2 | 1 | 2 | 4 |
| Totals | Sxy = 6 | Sxx = 10 | |||
Step 3: the slope. m = Sxy ÷ Sxx = 6 ÷ 10 = 0.6.
Step 4: the intercept. c = ȳ − m·x̄ = 4 − 0.6 × 3 = 2.2.
The least squares line is y = 0.6x + 2.2.
Check this example in the Line of Best Fit calculator →
The correlation coefficient
With one more sum, Syy = Σ(y − ȳ)², you can find r:
r = Sxy ÷ √(Sxx × Syy)
In the example, the y deviations are −2, 0, 1, 0, 1, so Syy = 4 + 0 + 1 + 0 + 1 = 6. Then r = 6 ÷ √(10 × 6) = 6 ÷ 7.746 ≈ 0.775, a strong positive correlation. See Scatter graphs and correlation for how to describe r.
What the slope and intercept mean
Example: car values
For ten used cars, the least squares line for value (in £1000s) against age (in years) is y = −1.53x + 19.1.
Slope −1.53: on average, a car loses about £1,530 of value each year.
Intercept 19.1: the line predicts a value of about £19,100 at age 0, when new. The data only covers ages 1 to 10, though, so this is an extrapolation and only a rough estimate.
See the car data in the Line of Best Fit calculator →
Always say what the slope means in the units of the question, such as “marks per hour” or “pounds per year”. That is usually what an exam question is asking for.
Tips
- Check that the line passes through (x̄, ȳ). If it doesn’t, there is a mistake in c.
- Sxx can’t be negative, because it is a sum of squares. Sxy can be negative, and then the slope is negative.
- The regression line of y on x is for predicting y from x. Predicting x from y strictly needs a different line, although for strongly correlated data the two are close.