==============================================================================================
RAHUL'S ML BLOG -- notes on machine learning, worked out by hand est. 2026
==============================================================================================
home | about | archive | glossary | contact
----------------------------------------------------------------------------------------------
CHAPTER 2 . GRADING A GUESSER . PART 1 OF 2
Two Rulers for One Guess: MSE and R^2
============================================================================================
We have built a guesser.
Now comes the question that quietly decides everything.
Beginners skip right past it.
Once the line is drawn -- WAS THE LINE ANY GOOD?
A guess you cannot grade is a guess you cannot trust.
First, what is the guesser?
It is the straight-stick rule: a "dial" (a number you can turn up or down) sits on each
column of the sheet.
For one row, you multiply each column's value by its dial.
Then you add those products.
Then you add one fixed extra number called the nudge.
That sum is the guess.
The sheet here is cars, not houses.
Same rule, new sheet.
Two different rulers will grade the very same guesses.
Here is the twist that makes the post worth reading.
They hand back two different verdicts, and BOTH are right.
One tells you how big the misses are.
The other tells you whether you have beaten the village idiot.
The problem, drawn once before the words:
car 1: guess = 20 mpg, real = 18 mpg, miss = +2 <- too high by 2
car 2: guess = 16 mpg, real = 15 mpg, miss = +1
car 3: guess = 12 mpg, real = 14 mpg, miss = -2 <- too low by 2
ruler 1 (MSE): how big are the misses?
ruler 2 (R^2): are you better than the single dumbest guess (the average)?
dumb baseline: ignore everything, guess the average MPG every time
good machine: misses LESS than the dumb baseline
R^2 = 0 means "as dumb as the baseline"
R^2 = 1 means "perfect"
R^2 < 0 means "worse than the baseline -- something has gone badly wrong"
SAME STICK, NEW SHEET
displacement horsepower weight acceleration | MPG <- right answer
-----------------------------------------------------------
307 130 3504 12.0 | 18
350 165 3693 11.5 | 15
... | ...
one row = one car | the answer
guess = d1*disp + d2*horse + d3*weight + d4*accel + nudge
Pick four columns, guess the fifth (MPG, miles per gallon).
The machinery is the straight-stick rule again.
A dial on each of the four columns, multiply, add, plus the fixed nudge.
The dials get set by shrinking the squared leftover.
A "leftover" is the gap between the right answer and the guess.
"Squared" means each gap times itself, which kills the minus sign.
So setting the dials means picking the dials that make those squared gaps as small as
they go.
One round of multiply-and-add per guess (a "dot-product").
Nothing new to build.
The new work is grading.
RULER ONE: MSE -- JUST THE SIZE OF THE MISS
Start with a mistake-score called RMSE.
RMSE stands for root mean squared error.
It is built in four moves, in order: take the miss, square it, average the squares, then
take the square root.
MSE drops that last move:
miss -> square -> mean -> root = RMSE
miss -> square -> mean -> STOP = MSE
Stare at the name MSE and it gives itself away:
M S E
| | +- Error the miss (y - guess)
| +--- Squared kill the sign
+----- Mean average them
No R. No Root.
So take the miss, square it, average -- and stop there.
Leave off the square root (writing ** 0.5 in code raises to the power one-half, which is
a square root).
MSE = (1/n) sum_i (y_i - yhat_i)^2
In that line: y_i is the right answer for car i.
yhat_i is the guess for car i.
(y_i - yhat_i) is the miss for car i.
You square each miss, add them all up (that is the sum), then divide by n, the count of
cars.
>> NOTE: WHAT MSE IS, EXACTLY
MSE is RMSE squared.
RMSE is root mean squared error -- the same miss-square-mean then a square root.
So MSE lives in SQUARED answer-units -- "miles per gallon, squared," which no one can
picture.
That is the price of dropping the root.
What you buy: MSE is the exact thing the straight-stick rule shrinks when it sets the
dials, so it is the rule's own home ruler.
RMSE is just MSE pulled back into plain units for a human to read.
Same ordering of which fit is better -- the square root only ever rises as its input
rises, so it never swaps the order.
Every piece of that recipe -- the square, the mean, the stop-at-one-number -- has to
earn its place. So make each one earn it, starting with the square.
WHY SQUARE AT ALL -- RAW MISSES VOTE EACH OTHER OUT
Try to grade WITHOUT the square. The three cars from the top of the page:
car 1: guess 20 mpg, real 18 -> miss = 20 - 18 = +2 (too high by 2)
car 2: guess 16 mpg, real 15 -> miss = 16 - 15 = +1 (too high by 1)
car 3: guess 12 mpg, real 14 -> miss = 12 - 14 = -2 (too low by 2)
Add the raw misses: (+2) + (+1) + (-2) = +1.
The report reads "one mpg wrong, in total" -- about a machine that missed every
single car. The high misses and the low misses CANCELLED. Push it to the edge: a
guesser that shouts 100 too high on half the fleet and 100 too low on the other
half sums to zero -- a perfect grade for a useless machine. Raw misses carry
signs, and signed misses vote each other out.
So kill the sign before adding. Square each miss:
(+2)^2 + (+1)^2 + (-2)^2 = 4 + 1 + 4 = 9
Nine units of misery. Nothing cancelled; every car's miss is in the bill.
A guesser misses +3 on one car and -3 on another. Its raw-sum grade, and
its squared grade?
check your slate: raw (+3) + (-3) = 0, a perfect score for two misses --
the lie. squared: 9 + 9 = 18. The square keeps both crimes on the books.
WHICH MEANS ANY SIGN-KILLER WOULD DO -- SO WHY THE SQUARE AND NOT THE BARE SIZE?
Dropping the minus outright also kills the sign: the bare sizes |+2|, |+1|, |-2|
add to 2 + 1 + 2 = 5, positive, no cancelling. Two honest candidates, so the
square must beat the bare size somewhere else. It does: in how the penalty
PUSHES as one miss shrinks.
Take one miss, call it m, and nudge it by a hair h. The squared penalty moves by
(m + h)^2 - m^2 = m^2 + 2mh + h^2 - m^2 = 2mh + h^2
Per unit of hair that is 2m + h, and as the hair shrinks toward nothing the push
settles at exactly 2m. Check it at m = 4: penalty 4^2 = 16; shrink the miss to
3.9 and the penalty is 15.21; the 0.1 of shrink paid back 0.79, a push of 7.9
per unit -- and 2m = 2 x 4 = 8, matching (the small gap is the +h just derived).
Read what push = 2m means:
miss m = 4 -> push 8 a big miss gets shoved HARD
miss m = 0.1 -> push 0.2 a small miss gets a gentle touch
miss m = 0 -> push 0 AT the right answer, the grading goes quiet
The bare size |m| pushes differently: 1 per unit when m is above zero, 1 per
unit (the other way) when m is below -- the same shove for a hair as for a
mile -- and at m = 0 the push jumps straight from -1 to +1 with no quiet point
in between. The square's push scales with the crime and settles to zero exactly
at the answer. That is why the straight-stick rule shrinks SQUARED leftovers
when it sets its dials, and why the ruler that grades it squares too.
THE SQUARES ARE IN -- NOW WHY DIVIDE BY THE COUNT?
Skip the divide and the ruler grades the paperwork, not the machine. Imagine a
guesser that misses EVERY car by exactly 1 mpg -- same skill on every row. On a
5-car sheet: five squared misses of 1, total 5. On a 400-car sheet: total 400.
Same machine, same skill, and the no-divide ruler calls the second job eighty
times worse. The sum grows with the SHEET. Divide by the count and both sheets
read exactly 1.0 -- "one squared mpg of miss per car" -- a per-car grade that
follows the guesser, not the size of the pile it was handed.
AND WHY CRUSH IT ALL TO ONE NUMBER?
Because a grade exists to be COMPARED. Turn a dial, re-grade, keep the turn if
the grade fell -- that loop is how the dials get set, and it needs a number that
can FALL. Keep the whole column of squared misses instead and the comparison
dies: after a dial turn, car 3's square drops while car 7's rises -- better or
worse? A column can only shuffle; one number can fall. The mean is the crush
that turns a sheet of misses into a verdict a dial can act on.
Count the clerk-strokes for MSE on a whole sheet of 400 cars (made-up count).
Per car: 1 subtraction to get the miss, plus 1 squaring = 2 strokes.
So 400 cars x 2 = 800 strokes.
Then 399 additions to add the 400 squares into one sum.
Then 1 division by the count to get the mean.
That totals 800 + 399 + 1 = 1,200 strokes.
One clerk clears it in an hour; the room, before the coffee cools.
PROBLEM MSE CAN'T SOLVE
"the guess was off by 9."
good? bad? ...compared to WHAT?
A raw miss-size has no scale of its own.
Off by 9 mpg is dreadful for a car.
But off by 9 feet is wonderful for an archer firing across a valley.
So the number alone won't say good or bad.
Which means you need something to measure it AGAINST.
That is the second ruler.
RULER TWO: R^2 -- A RACE AGAINST A FOOL
IN HAND: a sheet of cars, a stick that guesses mpg from four columns, and one ruler.
That ruler is MSE: miss -> square -> mean -> stop.
MSE sizes the misses but cannot say whether the size is good.
This section adds the second ruler: a race against an opponent.
R^2 is the name of this second ruler (read "R squared").
A score needs something to be scored against.
So R^2 picks the dumbest honest opponent imaginable and makes the stick race it.
Meet the fool.
He has not looked at a single column.
He does not even know the cars have engines.
All he has is a pile of past answers and no clues at all:
answers: 2 4 9 3 7
The fool must shout one number for every car, no matter what.
What is his safest shout?
The AVERAGE of the answers.
Add them: 2+4+9+3+7 = 25.
Divide by the count, 5: 25/5 = 5.
So his shout is 5.
Not because the fool is clever (he is not).
But because the average sits at the very bottom of the squared-miss bowl.
Any other single number he could pick misses more.
Check the miss-squares for two shouts.
Shout 5: the misses are 2-5, 4-5, 9-5, 3-5, 7-5 = -3, -1, 4, -2, 2; squared they are
9, 1, 16, 4, 4; sum = 9 + 1 + 16 + 4 + 4 = 34.
Shout 6: the misses are 2-6, 4-6, 9-6, 3-6, 7-6 = -4, -2, 3, -3, 1; squared they are
16, 4, 9, 9, 1; sum = 16 + 4 + 9 + 9 + 1 = 39, which is worse.
shout 5 -> miss^2 = 9 + 1 + 16 + 4 + 4 = 34
shout 6 -> miss^2 = 16 + 4 + 9 + 9 + 1 = 39 (worse)
WHY THE AVERAGE IS THE FLOOR
Shout some number c.
The total squared miss as c varies is f(c) = sum (y_i - c)^2.
To find the bottom, set the slope to zero.
The slope is f'(c) = -2 sum (y_i - c), and we set it = 0.
Divide both sides by -2: sum (y_i - c) = 0.
That sum is sum y_i minus n copies of c, where n is the count.
So sum y_i - n*c = 0.
Solve for c: c = (1/n) sum y_i, which is the average, exactly.
The curve is a bowl (its second slope, 2n, is greater than zero), so that point is
the lowest, not the highest.
Therefore the fool's best constant shout is always the mean.
A concrete check, by pencil.
Try four different constant guesses against the same answers (2, 4, 9, 3, 7):
guess c misses^2 total squared miss
------------------------------------------------------------------------
4 (2-4)^2+(4-4)^2+(9-4)^2+(3-4)^2+(7-4)^2 = 4+0+25+1+9 = 39
5 (avg) (2-5)^2+(4-5)^2+(9-5)^2+(3-5)^2+(7-5)^2 = 9+1+16+4+4 = 34 <- smallest
6 (2-6)^2+(4-6)^2+(9-6)^2+(3-6)^2+(7-6)^2 =16+4+9+9+1 = 39
7 (2-7)^2+(4-7)^2+(9-7)^2+(3-7)^2+(7-7)^2 =25+9+4+16+0 = 54
The average (5) gives the smallest total. Any other number misses more.
Same answers (2, 4, 9, 3, 7). Try the constant shout c = 3 and see where it
lands against the average's 34.
check your slate: (2-3)^2 = 1; (4-3)^2 = 1; (9-3)^2 = 36; (3-3)^2 = 0;
(7-3)^2 = 16; total = 1 + 1 + 36 + 0 + 16 = 54. Worse than the average's
34 -- the bowl rises on both sides of 5.
Call the fool's total squared miss the TOTAL WOBBLE.
For these answers the fool shouts 5, and his total wobble is 9 + 1 + 16 + 4 + 4 = 34.
Now race the fool against the stick:
truth: 2 4 9 3 7
FOOL (says 5): miss^2 = 9, 1, 16, 4, 4 -> 34 <- total wobble
STICK (uses cols): says 2.5, 4.5, 8, 3.5, 6.5
miss^2 = .25,.25,1,.25,.25 -> 2 <- leftover
(stick's misses, worked: 2-2.5 = -.5, 4-4.5 = -.5, 9-8 = +1, 3-3.5 = -.5,
7-6.5 = +.5; squares .25, .25, 1, .25, .25; sum = 2.)
The stick's total squared miss is .25 + .25 + 1 + .25 + .25 = 2.
Call that the LEFTOVER.
LEFTOVER IS THE BAD PART, NOT THE GOOD PART
It is tempting to read "leftover = 2" as the stick's score.
It is the opposite.
Leftover is the misery STILL LEFT after the stick tried -- the small bad bit it could
not explain.
The GOOD part is what the stick ATE: total wobble minus leftover.
Here that is 34 - 2 = 32.
total wobble #################### 34 (the fool's whole job)
stick ATE ################## 32 = 34 - 2 (the good part)
leftover # 2 (the bad bit still left)
R^2 is the slice the stick ate, as a fraction of the whole bar.
Total wobble (the fool's total squared miss) = 34.
Leftover (the stick's total squared miss) = 2.
Ate = 34 - 2 = 32.
So:
R^2 = (total wobble - leftover) / total wobble = 32 / 34 ~= 0.94
DIVIDE BY THE WHOLE BAR, NOT THE COUNT
The divider is the WHOLE wobble (34), not the row count (5).
R^2 asks "what fraction of the WHOLE wobble did you eat?"
That is a slice of a bar, not an average.
The count sits on top and bottom and cancels.
Which is the very reason R^2 carries no units.
A clumsier stick (made-up) guesses 3, 5, 8, 2, 6 for the same truths
(2, 4, 9, 3, 7). The fool's total wobble is still 9 + 1 + 16 + 4 + 4 = 34.
Work this stick's leftover and its R^2.
check your slate: misses 2-3 = -1, 4-5 = -1, 9-8 = +1, 3-2 = +1, 7-6 = +1;
squares 1, 1, 1, 1, 1; leftover = 5. Ate = 34 - 5 = 29. R^2 = 29/34 ~ 0.85.
A bigger leftover means a smaller slice eaten -- this stick grades below the
first one's 32/34 ~ 0.94.
The scale falls right out:
R^2 = 1.0 ate everything, leftover zero -- a perfect line
R^2 = 0.0 ate nothing -- no better than the fool
R^2 < 0 WORSE than the fool -- the line actively hurts
It can never top 1, because you cannot eat more than the whole bar.
But it CAN dip below 0.
On a sheet the stick has never seen, it sometimes does.
GUESS ONCE, GRADE TWICE
IN HAND: two rulers for the same misses.
MSE: miss -> square -> mean -> stop.
R^2: the slice eaten.
On the five cars the fool's total wobble is 9+1+16+4+4 = 34.
The stick's leftover is .25+.25+1+.25+.25 = 2.
So R^2 = (34 - 2)/34 = 32/34 ~ 0.94.
This section adds the discipline: guess once, then hold both rulers to the same guesses.
set the dials -> guess the sealed pile -> guesses (computed ONCE)
|
+----------+----------+
v v
MSE -> a SIZE R^2 -> a SLICE
The two rulers do not need two sets of guesses.
You guess once, then hold both rulers against the same misses.
MSE says how big the misses are.
R^2 says how much better than a fool you did.
The few lines that do this -- one little helper that guesses a single time and holds up
both rulers -- are waiting at the end of the post.
The clerk bill agrees.
Guessing is the dear part.
One guess on four columns = 4 multiplications + 4 additions = 8 strokes.
So 400 cars (made-up count) cost 400 x 8 = 3,200 strokes.
MSE after that is the 1,200 strokes counted earlier (800 to subtract-and-square, 399 to
add, 1 to divide).
R^2 adds the fool's wobble (near 1,600 strokes) and a 2-stroke slice.
So both rulers together cost less than guessing once.
And one discipline holds.
The "sealed pile" is a batch of rows the stick was kept away from while its dials were
set.
Open that sealed pile ONCE, to grade -- never to choose.
Grading on rows the stick never saw is the whole job of the hidden pile.
THREE THINGS WORTH STATING PRECISELY
IN HAND: one set of guesses off the sealed pile, graded twice.
MSE for the size of the misses.
R^2 for the slice of the fool's wobble eaten.
This section adds the fine print: the facts about the two rulers that are easiest to get
wrong.
>> NOTE: ON TRAINING THE BAR SPLITS CLEANLY -- ON TEST IT NEED NOT
The "training pile" is the rows the dials were set on.
The "test pile" is the sealed rows the stick never saw.
With a nudge column, the straight-stick rule forces its leftovers to sit at
right-angles to its guesses.
So on the TRAINING pile the bar splits exactly: total wobble = ate + leftover.
(Textbook names: TSS = ESS + RSS. TSS is total sum of squares, ESS is explained sum
of squares, RSS is residual sum of squares.)
There R^2 lands between 0 and 1.
On the TEST pile no such right-angle holds.
So the leftover can be larger than the wobble.
Which means R^2 = 1 - RSS/TSS can fall below zero.
A negative test R^2 is not a bug -- it is the line doing worse than shouting the
average.
>> NOTE: ONE COLUMN -- R^2 IS JUST CORRELATION, SQUARED
Correlation, written r, is a number from -1 to +1 saying how tightly two columns rise
and fall together.
Fit a straight stick on a single column and R^2 equals that correlation squared.
That is literally why it wears the name "R squared."
With several columns it widens to the squared correlation between the answers and the
stick's guesses.
R^2 ALWAYS RISES WHEN YOU ADD A COLUMN -- ON THE TRAINING PILE
Hand the stick one more column and training R^2 never drops.
Even if the column is pure noise.
The old fit is still available, so the stick can only do at least as well.
Therefore training R^2 is a poor judge of whether a column earns its place.
Use the TEST R^2 instead, or the adjusted R^2 that docks you for each extra dial.
WHERE THE MISS-SIZE COMES FROM (STIFF vs JUMPY)
Average a guesser's test MSE over many possible training piles and it splits three
ways: E[MSE] = bias^2 + variance + noise.
(E[MSE] means the average MSE over those many piles.)
Bias is the line being too stiff to follow the true shape.
Variance is the line being too jumpy, swinging with each pile it sees.
Noise is the part no rule can ever remove.
A "stiff vs jumpy" dial -- a stiff guesser ignores wiggles, a jumpy one chases every
wiggle -- is exactly this bias-vs-variance trade, seen through MSE.
One honest IOU, named out loud: that three-way split is STATED here, not derived.
The proof expands (truth - guess)^2 around the average guess and shows the cross
terms die -- average-of-averages algebra this post does not carry.
Until the blog works it by pencil, treat the split as a named promise, not an earned
fact.
What IS earned here: the three words mean real things you can already see -- a flat
line on curvy truth (stiff), a wild line that changes with every re-deal (jumpy), and
the scatter no line removes (noise).
1. MSE = miss -> square -> mean -> stop. RMSE without the root, in squared
answer-units, and the very thing the stick shrinks.
2. A raw miss-size means nothing alone -- "off by 9" needs something to measure
against.
3. R^2 races the stick against a fool who only ever shouts the average (the bottom of
the squared-miss bowl).
4. R^2 = (total wobble - leftover) / total wobble = 1 - RSS/TSS. No units. Tops out at
1; can go below 0 on unseen rows.
5. Guess once, grade twice: the same guesses feed both rulers.
A few places these two rulers bite, each a belief that feels right.
MSE looks like it should end in a root. It does not -- the tool already squares and
means:
miss -> square -> mean -> STOP MSE
miss -> square -> mean -> sqrt RMSE (the R is the root)
Call mean_squared_error and then ** 0.5 and you have built RMSE by accident.
"Leftover = 2, so 2 is the score" runs backwards. The leftover is the misery STILL
there after the stick tried; the good part is what it ATE:
total wobble = good part (eaten) + leftover (still missed)
R^2's bottom is the whole wobble, not the row count:
WRONG: (total wobble - leftover) / 5 (5 rows)
RIGHT: (total wobble - leftover) / total wobble
Dividing by the whole bar is what cancels the units -- which is why R^2 has none.
Add a noise column, watch R^2 climb, call the fit better? Training pile only:
training R^2: add any column -> can only hold or rise (old fit still a candidate)
test R^2: add noise -> usually drops
Judge on the test R^2, or adjusted R^2.
R^2 below 0 looks like a bug; on unseen rows it is honest:
train: leftover <= wobble guaranteed -> R^2 in [0, 1]
test: leftover CAN exceed wobble -> R^2 = 1 - RSS/TSS < 0
Below zero just means the line did worse than shouting the flat average.
Grade either ruler on the training pile and the number flatters you:
train pile --> used to set the dials --> grade here? no, you studied these rows
sealed test pile --> never touched --> grade here: yes
Both MSE and R^2 read the sealed test pile; scoring on the rows you fitted gives a
false-low mistake.
Hunting the strongest drag, -0.04 against -0.007:
-0.007 ----|--- 0
-0.04 -----| further below zero = SMALLER = most negative = strongest
min(dials, key=dials.get) hands back that column's name.
Last, -0.007 per pound and -0.04 per horsepower cannot be ranked as they stand --
each dial wears its own column's units:
-0.007 per POUND (a tiny step) -0.04 per HORSEPOWER (a big step)
Put every column on one shared ruler (StandardScaler) first, and the comparison is
honest.
Nothing above needed a computer -- only pencils, clerks, and patience.
This last section is for the day you meet one: the same moves, spoken in Python.
First the five cars by hand, one line each (no loop) -- the same truths and stick-guesses
from the worked example, so you can run it and watch 34, 2, and 0.94 print back:
FIVE CARS: (TRUTH, STICK'S GUESS), WRITTEN OUT ONE PER LINE
y1, g1 = 2, 2.5
y2, g2 = 4, 4.5
y3, g3 = 9, 8.0
y4, g4 = 3, 3.5
y5, g5 = 7, 6.5
The five cars, truth vs the stick's guess, the miss, and the squared miss:
car truth guess miss miss^2
1 2 2.5 -0.5 0.25
2 4 4.5 -0.5 0.25
3 9 8.0 1.0 1.00
4 3 3.5 -0.5 0.25
5 7 6.5 0.5 0.25
------
sum of squared misses (the leftover) = 2.00
RULER ONE -- MSE = MISS -> SQUARE -> MEAN -> STOP
mse = ((y1-g1)**2 + (y2-g2)**2 + (y3-g3)**2 + (y4-g4)**2 + (y5-g5)**2) / 5
print("MSE:", mse) # 0.4 (.25+.25+1+.25+.25 = 2, over 5)
RULER TWO -- R^2 = RACE THE STICK AGAINST THE ALWAYS-AVERAGE FOOL
fool = (2 + 4 + 9 + 3 + 7) / 5 # 5.0 (the fool's one safe shout)
total_wobble = ((2-fool)**2 + (4-fool)**2 + (9-fool)**2 + (3-fool)**2 + (7-fool)**2) # 34
leftover = ((y1-g1)**2 + (y2-g2)**2 + (y3-g3)**2 + (y4-g4)**2 + (y5-g5)**2) # 2
r2 = (total_wobble - leftover) / total_wobble
print("R^2:", round(r2, 2)) # 0.94 = (34 - 2) / 34 = 32/34
And the why-checks from the pencil half -- the cancellation lie, the two
sign-killers, the square's push, and sum-versus-mean -- each settled by a line
of the same arithmetic:
WHY SQUARE: RAW MISSES VOTE EACH OTHER OUT (the three cars from the top)
raw_sum = (+2) + (+1) + (-2) # 2 + 1 - 2 = 1 <- "one mpg wrong" about three misses
print("raw sum:", raw_sum) # 1 (the lie)
sq_sum = (+2)**2 + (+1)**2 + (-2)**2 # 4 + 1 + 4 = 9 every miss on the bill
print("squared sum:", sq_sum) # 9
bare_sum = abs(+2) + abs(+1) + abs(-2) # 2 + 1 + 2 = 5 the other sign-killer
print("bare-size sum:", bare_sum) # 5
THE SQUARE'S PUSH AT m = 4: SHRINK THE MISS BY 0.1, MEASURE THE PAYBACK
push = (4.0**2 - 3.9**2) / 0.1 # (16 - 15.21) / 0.1 = 7.9 ~ 2m = 8
print("push at m=4:", round(push, 4)) # 7.9
THE +3 / -3 DRILL
print("raw:", (+3) + (-3)) # 0 a perfect grade for two misses
print("squared:", 3**2 + (-3)**2) # 18 both crimes on the books
SUM GRADES THE SHEET, MEAN GRADES THE MACHINE (every car missed by exactly 1)
sum_5 = 1 + 1 + 1 + 1 + 1 # 5 five squared misses of 1
sum_400 = 400 * 1 # 400 four hundred of them, one product
mean_5 = sum_5 / 5 # 1.0
mean_400 = sum_400 / 400 # 1.0 same machine, same grade
print("sums:", sum_5, sum_400) # 5 400 <- the sum grew 80x for no new skill
print("means:", mean_5, mean_400) # 1.0 1.0 <- the mean held still
No loop: five cars, five lines, every miss visible. On a real sheet of hundreds of cars
you would not hand-type each row -- there the toolbox does the same two rulers in two
lines. It is the same arithmetic as above, just run over every row for you:
>> NEW TO PYTHON? One new piece here:
return (a, b) -- hand back two values at once, bundled as a pair (a "tuple")
from sklearn.metrics import mean_squared_error, r2_score
def evaluate_guesser(model, X_test, y_test):
guesses = model.predict(X_test) # guess ONCE
mse = round(mean_squared_error(y_test, guesses), 3) # ruler 1: a size (our mse)
r2 = round(r2_score(y_test, guesses), 3) # ruler 2: a slice (our r2)
return (mse, r2) # truth first, guesses second
Plain term used above Standard label
------------------------------------- ----------------------------------------
size of miss, squared and averaged mean squared error (MSE)
...with the root put back root mean squared error (RMSE)
the always-average fool the mean / baseline predictor
total wobble total sum of squares (TSS)
leftover residual sum of squares (RSS)
what the stick ate explained sum of squares (ESS)
slice of wobble eaten coefficient of determination (R^2)
too stiff vs too jumpy bias-variance
docked for each extra dial adjusted R^2
----------------------------------------------------------------------------------------------
IN THIS CHAPTER (Chapter 2 -- Grading a Guesser):
Part 1 (this post) .
Part 2 -- Reading the Dials
Previous chapter: Chapter 1 -- Predicting House Prices
<- Back to all posts
----------------------------------------------------------------------------------------------
home . source on GitHub
==============================================================================================