==============================================================================================
RAHUL'S ML BLOG -- notes on machine learning, worked out by hand est. 2026
==============================================================================================
home | about | archive | glossary | contact
----------------------------------------------------------------------------------------------
CHAPTER 2 . GRADING A GUESSER . PART 2 OF 2
Reading the Dials: What the Coefficients Say
============================================================================================
First, what is a dial?
The guesser here is the straight-stick rule.
It puts one number on each column of the sheet, and that number is the dial.
To guess one row, you multiply each column's value by its dial, add the products, then
add one fixed extra number called the nudge.
So far the dials have been the machine's private business -- numbers it set for itself
to make good guesses.
But those dials are not just machinery; they are a message.
Each one is the rule quietly telling you how its column pushes the answer around.
Read them and the black box starts talking.
This post asks one innocent-looking question of a sheet of cars.
Which column drags miles per gallon (MPG) down the hardest?
It uses that question to spring two snags that catch almost everyone.
One is a trick of arithmetic.
The other is a trick of units, and it is the kind of mistake that ends up in published
papers.
Dial A reads +5. So column A shoves the answer up, hard -- right? Not so fast.
Suppose columns A and B move almost identically, near-duplicates. The machine can
pin them against each other:
dial_A = +5 --.
>-- on every row, +5*A - 4*B ~ a small steady amount
dial_B = -4 --' the SUM is stable; each dial ALONE is arbitrary
Read dial A by itself and you announce "+5, a huge positive push." Read both and
the honest answer is "cannot say" -- the two mean something only together. Nudge
the data a hair and the pair could flip to +9 and -8 with the same result.
So never read a dial in isolation. Put every column on the same ruler (same
spread) before comparing dials, or use a leash (regularisation) to untangle the
duplicated columns first. Then a dial's size means what you think it means.
ONE DIAL PER COLUMN
guess = d1*disp + d2*horse + d3*weight + d4*accel + nudge
+--+--+
turn this dial up by 1 unit of its column,
the guess moves by d1 -- everything else held still
A POSITIVE dial pushes the answer up as its column grows.
A NEGATIVE dial drags the answer down as its column grows.
The size of the dial says how hard.
So "drags mpg down hardest" means the MOST NEGATIVE dial.
Easy -- except the rule hands the dials to you with the names torn off.
BARE ROW PROBLEM
model.coef_ -> [ -0.01, -0.04, -0.007, +0.12 ] numbers only, no names
X.columns -> [ disp, horse, weight, accel ] names, same order
Two rows, same order, one missing its labels.
"Zip" means pair them up position by position: first name to first dial, second to
second, and so on.
Zip the names onto the dials and you have a shelf you can read:
zip -> disp : -0.01
horse : -0.04
weight: -0.007
accel : +0.12
MOST NEGATIVE MEANS MIN, NOT MAX
number line -- furthest BELOW zero is the strongest drag:
-0.04 -0.01 -0.007 +0.12
horse disp weight accel
<-- most negative most positive -->
-0.04 IS SMALLER THAN -0.007
The catch: the digit 4 LOOKS bigger than the digit 7.
So the eye wants -0.04 to be the larger drag and reaches for it.
But further below zero is the SMALLER value.
So "most negative" is the MINIMUM, not the maximum.
Here the strongest drag is horse at -0.04.
So zip the names on, then grab the name sitting on the smallest dial.
This is a plain winner-hunt: scan the shelf, keep the smallest.
The two lines that do it are at the end of the post.
IN HAND: a bare row of dials [-0.01, -0.04, -0.007, +0.12] with the names zipped
back on (disp, horse, weight, accel).
The most-negative hunt crowns horse (-0.04) the hardest drag.
This section shows that crown is counterfeit until the columns share a ruler.
DIALS WEAR THEIR COLUMN'S UNITS
Here is the deeper catch, the one the easy version walks straight into.
A dial is "answer-units per ONE unit of its column."
And every column is measured on its own ruler, with its own idea of how big "one unit"
is:
weight dial = mpg per 1 POUND (pounds run 1500 ... 5000)
horse dial = mpg per 1 HORSEPOWER (horsepower runs 45 ... 230)
a step of "1 pound" is tiny; a step of "1 horsepower" is large.
so a small weight-dial and a large horse-dial may carry
the SAME real punch -- the numbers just wear different rulers.
RAW DIALS ARE NOT COMPARABLE ACROSS COLUMNS
Comparing -0.007 per pound against -0.04 per horsepower is like comparing prices in
two different currencies without the exchange rate.
The bare dial that LOOKS smallest can be the weakest or the strongest real drag.
It depends only on how wide its column runs.
So "which drags hardest" is not honestly answerable from the raw dials alone.
The fix is the same-ruler trick.
"Spread" here means how widely a column's values scatter (its standard deviation).
Replace each value x with (x - column average) / column spread.
So every column ends at zero average and a spread of one BEFORE setting the dials.
Now every dial means "mpg per one-spread step of its column" -- one shared currency.
And the most-negative one is the honest strongest drag.
(The code that does this is at the end of the post.)
>> NOTE: THIS DOESN'T CONTRADICT "OLS NEEDS NO SCALING"
OLS (ordinary least squares) is the name for the straight-stick rule that sets its
dials by shrinking the squared leftover.
Its GUESSES and its scores (MSE, the size of the misses; R^2, the slice of the
answer's wobble it explains) do not care about scaling.
Rescale a column and its dial rescales inversely, leaving the answer untouched.
So scaling changes nothing about how well it fits.
It only changes whether the dials are COMPARABLE TO EACH OTHER as a measure of pull.
Same-ruler is for the reading, not the fitting.
AND DON'T OVER-TRUST A SINGLE DIAL
"HOLDING THE OTHERS FIXED" CAN BE A FICTION
A dial means "move this column by one, hold the rest still."
But displacement, horsepower and weight rise together -- heavy cars tend to have big
engines.
When columns move as a pack (the name for this is collinearity), the rule cannot
cleanly tell their pulls apart.
It can shuffle the credit between their dials.
So a single dial can swing wildly, even flip sign, with a tiny change in the pile.
Read individual dials as a story to check, not a verdict to trust.
Your physics hunch -- heavier car, worse mileage -- is a guess to test.
The dials are evidence, and tangled columns make that evidence shaky.
A concrete dial comparison, by pencil, before and after same-ruler.
Suppose 2 columns: weight (pounds, runs 2000-5000) and horsepower (runs 45-230):
raw dials (each in its own unit):
weight dial = -0.007 mpg per POUND
horse dial = -0.04 mpg per HORSEPOWER
At a glance, the digit 4 makes -0.04 look 6x stronger than -0.007. But is it?
The dial says "per ONE unit of its column." A step of "1 pound" is tiny;
a step of "1 horsepower" is large. To compare, rescale both columns so
each has a spread of 1, then refit:
after same-ruler (every column rescaled to spread 1):
weight dial = -3.2 mpg per ONE-SPREAD of weight
horse dial = -2.1 mpg per ONE-SPREAD of horsepower
Now the comparison is fair.
Weight (-3.2) is further below zero than horse (-2.1).
So weight pulls harder per equal-sized step of each column.
The raw dials had reversed the ranking.
The weight column's spread is about 1000 pounds.
One-spread step of weight (about 1000 pounds) changes the guess by -3.2 mpg.
One-spread step of horsepower changes it by -2.1 mpg.
Same currency, honest comparison.
A weight dial reads -0.005 mpg per pound (made-up), and weight's spread is 800
pounds. Convert it to the shared currency: mpg per one-spread step.
check your slate: the per-spread dial = raw dial x column spread.
So -0.005 x 800 = -4.0.
One typical-sized step in weight (800 pounds) drops the guess by 4 mpg.
That is the honest pull, now comparable to any other column's per-spread dial.
PENCIL IT
shelf (already on a shared ruler):
disp : -0.01 horse : -0.04 weight: -0.007 accel : +0.12
most negative? scan for the lowest:
-0.04 <- horse (further below zero than -0.01 or -0.007)
strongest_drag = "horse"
accel (+0.12) is the only one PUSHING mpg up.
A different shelf (made-up), already on a shared ruler: cyl +0.05, disp -0.02,
horse -0.06, accel +0.09. Name the strongest drag.
check your slate: scan for the lowest -- +0.05, -0.02, -0.06, +0.09.
The smallest (furthest below zero) is -0.06 = horse.
Most-negative is a MIN.
The digit 6 looks small, but -0.06 sits lower than -0.02, so horse drags hardest.
Finding that strongest drag is the cheapest move in the whole blog.
With d dials you need d - 1 comparisons -- just 3 for these four.
No room of clerks; one clerk crowns the winner before the kettle boils.
1. Each column keeps one dial: + pushes the answer up, - drags it down; size is
strength.
2. The rule hands back a bare row of dials -- zip the column names on to read them.
3. "Most negative / drags hardest" is the MINIMUM dial, not the maximum;
min(d, key=d.get) returns its name.
4. Raw dials wear each column's own units -- not comparable until you put the columns
on one shared ruler first.
5. When columns move together, a single dial is shaky; treat it as evidence, not proof.
A few places this bites, each a thing that looks right.
model.coef_ comes back as a bare row of numbers with the names torn off:
model.coef_ -> [-0.01, -0.04, -0.007, 0.02] which number is which column?
Zip the names back on so each dial carries its label:
dials = dict(zip(X.columns, model.coef_))
-> {'disp': -0.01, 'horse': -0.04, 'weight': -0.007, 'accel': 0.02}
For the strongest downward drag, -0.04 against -0.007:
-0.007 ----|--- 0
-0.04 -----| further below zero = SMALLER = strongest drag down
-0.04 wins, even though "4" feels smaller than "7". Use min(...), not max(...).
Plain min on a shelf of {name: value} grabs the wrong thing:
d = {'accel': 0.02, 'horse': -0.04, 'disp': -0.01}
min(d) -> 'accel' (smallest NAME, alphabetically)
min(d, key=d.get) -> 'horse' (name whose VALUE is smallest)
You want the name with the smallest value, not the first name in the alphabet.
Two raw dials cannot be ranked head to head when their columns wear different units:
-0.007 per POUND (one pound is a tiny step)
-0.04 per HORSEPOWER (one horsepower is a big step)
Put every column on the same ruler (StandardScaler) first; then each dial reads in
the same per-spread unit and the comparison is honest.
"I scaled the columns, so the guesses change" -- they do not. Rescale a column and
its dial rescales inversely, cancelling out:
column x10 -> dial /10 -> product unchanged
guesses, MSE, R^2: identical before and after
Scaling changes only whether dials are COMPARABLE, never the fit itself.
Last, a dial reads as "the effect of this column, holding the others fixed" -- but
when columns move together you can never truly hold them fixed:
disp, horse, weight all rise together with car size
-> the fit cannot split their pulls cleanly
-> one dial can swing wildly, or even flip sign
Read dials as evidence, not proof.
Nothing above needed a computer -- only pencils, clerks, and patience.
This last section is for the day you meet one: the same moves, spoken in Python.
First the reading by hand -- the four dials from the worked example written out, the
most-negative hunt, and the per-spread conversion, no loop:
BARE ROW OF DIALS, ONE PER LINE, WITH THEIR COLUMN NAMES
disp_dial = -0.01
horse_dial = -0.04
weight_dial = -0.007
accel_dial = +0.12
ZIP NAMES ONTO DIALS, THEN "DRAGS HARDEST" = THE MINIMUM (MOST NEGATIVE), NOT THE
DIGIT THAT LOOKS BIGGEST
dials = {"disp": disp_dial, "horse": horse_dial, "weight": weight_dial, "accel": accel_dial}
strongest_drag = min(dials, key=dials.get) # 'horse' at -0.04
print(strongest_drag, dials[strongest_drag]) # horse -0.04
RAW DIALS WEAR EACH COLUMN'S UNITS; CONVERT TO THE SHARED CURRENCY:
PER-SPREAD DIAL = RAW DIAL * COLUMN SPREAD
weight_per_spread = -0.005 * 800 # -4.0 mpg per one-spread (800 lb) step
print(weight_per_spread) # -4.0
Four dials, one winner, every comparison visible. On a real fitted model you do not type
the dials -- they come back from the stick as a bare row (model.coef_), so you zip the
column names on the same way. The two snippets below do exactly the hand-steps above, once
raw and once after putting every column on a shared ruler so the comparison is honest:
>> NEW TO PYTHON? Each named once:
dict(zip(a, b)) -- pair two rows into a labelled shelf: name -> value
min(d, key=d.get) -- the name (key) whose value is smallest, not the smallest
name; plain min(d) would sort the names alphabetically
m[-1] -- the last step of a pipeline (here, the fitted stick)
BARE DIALS, EACH IN ITS OWN COLUMN'S UNITS -- NOT COMPARABLE ACROSS COLUMNS
dials = dict(zip(X.columns, model.coef_))
strongest_drag = min(dials, key=dials.get) # KEY of the smallest VALUE -> a name
SAME-RULER FIRST, THEN READ THE DIALS -- NOW AN HONEST COMPARISON
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
m = make_pipeline(StandardScaler(), LinearRegression())
m.fit(X_train, y_train)
dials = dict(zip(X.columns, m[-1].coef_)) # now in one shared ruler
strongest_drag = min(dials, key=dials.get) # an honest comparison
>> NOTE: IT HANDS BACK A NAME, NOT A NUMBER
min(dials, key=dials.get) walks the shelf, scores each name by its dial, and returns
the NAME with the lowest score -- not the dial itself. Plain min(dials) would instead
compare the names alphabetically and hand back "accel" -- the wrong question answered
confidently.
Plain term used above Standard label
----------------------------------- ------------------------------------------
dial on a column coefficient / weight
the fixed nudge intercept
the bare row of dials model.coef_
put columns on one shared ruler standardisation
dials after same-ruler standardised (beta) coefficients
columns moving as a pack collinearity / multicollinearity
hold the others fixed ceteris paribus / partial effect
----------------------------------------------------------------------------------------------
IN THIS CHAPTER (Chapter 2 -- Grading a Guesser):
Part 1 -- Two Rulers for One Guess .
Part 2 (this post)
Next chapter: Chapter 3 -- Sorting Into Bins
<- Back to all posts
----------------------------------------------------------------------------------------------
home . source on GitHub
==============================================================================================