How least squares regression works
Scatter your points on a graph and imagine sliding a ruler around until it looks like the best straight line through them. “Looks best” needs a definition, and least squares picks a specific one: for each point, measure the vertical gap between the point and the line (how far off the line’s prediction of y is), square it, and add all the squares up. The least squares line is the unique line that makes this total as small as possible. Why vertical gaps? Because the line is a prediction machine — you feed it x, it predicts y — so the error that matters is the error in y. Why squared? Two honest reasons: squaring stops positive and negative misses from canceling (a line threading between wildly scattered points shouldn’t score as perfect), and it punishes one big miss more than several small ones, which drags the line toward not-being-terribly-wrong anywhere. A bonus reason mathematicians love: squares make the minimization solvable with one clean formula — no searching required.
Paste your pairs in any reasonable format — 1,2 3,5 4,7, one pair per line, or semicolon-separated — and you get the fitted line ŷ = mx + b, the correlation coefficient r, and r².
The formula
Here x̄ and ȳ are the means of the x and y values, the sums run over all n points, m is the slope, and b is the y-intercept. The numerator measures how x and y move together (their co-variation); the denominator measures how much x varies on its own. The intercept formula encodes a lovely fact: the least squares line always passes exactly through the point of means (x̄, ȳ), the center of gravity of your data. The correlation is r = Σ(x − x̄)(y − ȳ) ⁄ √(Σ(x − x̄)² · Σ(y − ȳ)²), always between −1 and 1.
Worked example
Points: (1, 2), (2, 3), (3, 5), (4, 6). The means are x̄ = 2.5 and ȳ = 4. The deviations from the means give Σ(x − x̄)(y − ȳ) = (−1.5)(−2) + (−0.5)(−1) + (0.5)(1) + (1.5)(2) = 3 + 0.5 + 0.5 + 3 = 7, and Σ(x − x̄)² = 2.25 + 0.25 + 0.25 + 2.25 = 5. So m = 7⁄5 = 1.4 and b = 4 − 1.4 × 2.5 = 0.5: the line is ŷ = 1.4x + 0.5. With Σ(y − ȳ)² = 4 + 1 + 1 + 4 = 10, the correlation is r = 7⁄√50 ≈ 0.98995 and r² ≈ 0.98. Paste 1,2 2,3 3,5 4,6 above and you’ll get exactly these numbers.
r versus r² (and the sentence every statistics teacher repeats)
The two numbers answer different questions. r tells you the direction and tightness of the linear relationship: +1 is a perfect uphill line, −1 a perfect downhill line, 0 no linear trend at all. r² tells you the share of the variation in y that the line accounts for: r = 0.9 sounds impressive until you notice r² = 0.81, meaning 19% of the variation in y has nothing to do with your line. Since squaring discards the sign, r² alone can’t tell you whether the trend goes up or down — report both. And the mandatory warning, which is mandatory because it is genuinely easy to forget: correlation is not causation — a strong r says x and y move together, not that x makes y move (ice cream sales and drowning deaths correlate beautifully; the cause of both is summer). One practical caution deserves equal billing: least squares is sensitive to outliers, because squaring amplifies big misses. A single typo’d point can swing the whole line, so eyeball your data before you trust the fit.