Least-Squares Regression Line
In short: Least-squares regression line is the straight line that predicts y from x by making the sum of the squared vertical distances from the data points to the line as small as possible.
AP StatisticsStatisticsAP Statistics Unit 5
What is Least-Squares Regression Line?
The least-squares regression line is the best-fitting straight line through a scatterplot, in a specific sense: no other line has a smaller sum of squared residuals. A residual is the vertical gap between an observed point and the line.
It is written \(\hat{y} = a + bx\). The slope is \(b = r\,\frac{s_y}{s_x}\), the intercept is \(a = \bar{y} – b\bar{x}\), and the line always passes through the point \((\bar{x}, \bar{y})\). The slope says how much the predicted \(y\) changes for each one-unit increase in \(x\); the intercept is the prediction at \(x = 0\), which may not be meaningful.
How to spot it
- A scatterplot with a linear trend is given and a prediction for a new x is requested from the least-squares regression line.
- Summary statistics, means, standard deviations and r, are provided and the equation of a line is wanted.
- A question asks to interpret the slope or intercept in context.
- Computer output lists “Constant” and a coefficient for the explanatory variable.
The method
- Compute the slope \(b = r\,\frac{s_y}{s_x}\), or \(b = \frac{\sum (x – \bar{x})(y – \bar{y})}{\sum (x – \bar{x})^2}\).
- Find the intercept \(a = \bar{y} – b\bar{x}\).
- Write the least-squares regression line \(\hat{y} = a + bx\), and use it to predict, staying within the range of the observed \(x\) values.
- Check residuals: \(y – \hat{y}\) should add to zero and show no curved pattern.
Least-Squares Regression Line: solved example
Problem
Five students record hours studied \(x\) and quiz score \(y\): (1, 55), (2, 62), (3, 70), (4, 74), (5, 84). Find the least-squares line, predict the score for 3.5 hours, and find the residual at \(x = 4\).
Solution
\(\bar{x} = 3\) and \(\bar{y} = 69\). Then \(\sum (x – 3)(y – 69) = 28 + 7 + 0 + 5 + 30 = 70\) and \(\sum (x – 3)^2 = 10\), so \(b = 7\). Intercept: \(a = 69 – 7(3) = 48\).
The line is \(\hat{y} = 48 + 7x\). At 3.5 hours: \(48 + 24.5 = 72.5\).
At \(x = 4\) the prediction is 76 and the actual score is 74, so the residual is \(74 – 76 = -2\).
Try it
Question 1
Interpret the slope 7 in the quiz example.
Show the answer
For each additional hour studied, the predicted quiz score increases by about 7 points.
Question 2
A line has \(r = 0.8\), \(s_x = 5\), \(s_y = 15\), \(\bar{x} = 20\), \(\bar{y} = 100\). Find the equation.
Show the answer
\(b = 0.8 \times \frac{15}{5} = 2.4\) and \(a = 100 – 2.4(20) = 52\), so \(\hat{y} = 52 + 2.4x\).
Least-Squares Regression Line vs. Residual
The least-squares regression line is the model, one formula for all predictions. A residual is the leftover for one particular observation: actual value minus predicted value. The line is chosen so that the residuals are as small as possible overall, in the sense of squared size.
Keep going
Lessons
Worksheets and quizzes
- Least-Squares Regression and Residuals – Worksheet
- Least-Squares Regression and Residuals – Quiz
- Scatterplots and Correlation – Worksheet
- Scatterplots and Correlation – Quiz
Related words
Whole dictionary: A–Z · AP Statistics · Statistics