==============================================================================================
RAHUL'S ML BLOG -- notes on machine learning, worked out by hand est. 2026
==============================================================================================
home | about | archive | glossary | contact
----------------------------------------------------------------------------------------------
CHAPTER 19 . LEARNING BY COPYING . PART 2 OF 5
The Copying Machine by Pencil
============================================================================================
On the desk sit 24,208 question-answer pairs, cut from a diary of a human playing a
block-pushing game 206 times. Each pair: a QUESTION of five numbers (hand x, hand y,
block x, block y, block twist -- the whole table at one moment) and an ANSWER of
sixteen numbers (the human's next eight target points, two numbers each, sealed
together). Something must learn to turn any five such numbers into the sixteen the
human would have produced. Build it with a pencil --
down to a number about the real machine that the pencil can call before training
even starts.
RAW NUMBERS CHOKE SMALL DIALS, SO RE-RULE EVERY COLUMN FIRST
The real first question in the diary reads [222.0, 97.0, 223.0, 381.6, 3.0] -- table
positions in the hundreds next to a twist under seven. The machine below is a web of
DIALS (adjustable multipliers), and fresh dials are born small, near +-0.1. Feed 222.0
through a small dial and the output swings in the tens; feed the twist 3.0 through its
neighbour and it barely whispers. The loud column drowns the quiet one before learning
begins, and every dial-turn computed later would be sized for the shout, not the
whisper.
So every column is RE-RULED before anything else: subtract the column's middle, divide
by its spread (its typical distance from the middle). Both are measured once, over the
whole diary. Rounded to one decimal for pencil work, the five question columns:
column: hand x hand y T x T y twist
middle: 229.1 293.3 246.5 272.4 1.8
spread: 101.9 96.5 63.9 65.9 1.8
Re-rule the real first question, one column at a time (results to two decimals):
hand x: (222.0 - 229.1) / 101.9 = -7.1 / 101.9 = -0.07
hand y: ( 97.0 - 293.3) / 96.5 = -196.3 / 96.5 = -2.03
T x : (223.0 - 246.5) / 63.9 = -23.5 / 63.9 = -0.37
T y : (381.6 - 272.4) / 65.9 = 109.2 / 65.9 = 1.66
twist : ( 3.0 - 1.8) / 1.8 = 1.2 / 1.8 = 0.67
[222.0, 97.0, 223.0, 381.6, 3.0] --re-rule--> [-0.07, -2.03, -0.37, 1.66, 0.67]
Now every column speaks at the same volume: a value of 1 means "one typical distance
above the middle", whichever column it lives in. The ANSWER columns get the same
treatment with their own rulers -- target-x middle 228.2 spread 101.6, target-y middle
294.0 spread 96.0 -- so the human's first move [233, 71] becomes
(233 - 228.2) / 101.6 = 0.05 (71 - 294.0) / 96.0 = -2.32.
Two guards close the section. The machine will THINK entirely in re-ruled units, so
every answer it prints must be DE-RULED on the way out (multiply by the spread, add
back the middle) before the table can use it. And a spread of exactly zero (a column
that never varies) would divide by zero, so each spread is floored at 0.000001.
Re-rule a block height of 338.3 (T y column: middle 272.4, spread 65.9).
CHECK: (338.3 - 272.4) / 65.9 = 65.9 / 65.9 = 1.0 -- exactly one spread above.
NOW THE MACHINE: A WEB THAT WIDENS, MIXES, AND NARROWS
question answer
5 numbers --> [ 256 dials-and-adders ] --> squash -->
[ 256 dials-and-adders ] --> squash -->
[ 256 dials-and-adders ] --> squash --> [ 16 outputs ]
narrow neck -> wide belly (three times) -> narrow spout
The machine is a web of adders. One UNIT owns one dial per incoming number plus one
free-standing nudge: it multiplies each input by its dial, adds everything plus the
nudge, and passes the sum on. Between layers sits a SQUASH: any negative sum becomes
0, any positive sum passes untouched. Without the squash, stacked layers of multiply-
and-add collapse into one layer of multiply-and-add -- one straight ruler, however
deep the stack; the squash's kink is what lets depth buy new shapes.
Why 5 wide, then 256, then 16? Five numbers in and sixteen out are forced by the job.
The wide middle is capacity: each of the 256 units mixes ALL five question numbers
into one feature, the next layer mixes those features, and the spout re-mixes 256
features down to the sixteen answer numbers. Count what that costs, box by box --
each box charges (inputs x units) dials plus (units) nudges:
box 1: 5 x 256 + 256 = 1,280 + 256 = 1,536
box 2: 256 x 256 + 256 = 65,536 + 256 = 65,792
box 3: 256 x 256 + 256 = 65,536 + 256 = 65,792
box 4: 256 x 16 + 16 = 4,096 + 16 = 4,112
total = 137,232
137,232 adjustable numbers, and NO table of answers anywhere: the same dials serve
every one of the 24,208 questions. Everything the machine will ever know about
pushing lives in how those dials are set.
One re-boxing detail: the spout prints a flat row of 16, but the answer is eight
moves of two numbers. Same sixteen numbers, re-boxed 16 -> 8 x 2 -- the numbers never
change, only the container -- so guess and recorded answer line up number for number.
Count the dials-and-nudges of a small web: 3 inputs -> 4 units -> 2
outputs.
CHECK: 3 x 4 + 4 = 16 ; 4 x 2 + 2 = 10 ; total 26.
A TINY TWIN BY PENCIL, BECAUSE 137,232 DIALS HIDE THE ARITHMETIC
Shrink the machine until every wire is visible: 2 question numbers -> 3 middle units
-> 2 answer numbers (a one-move envelope). Take a toy re-ruled question x = [1.0,
-2.0] and set the dials by hand:
middle unit m0: dials [ 0.5, -0.1 ] nudge 0.1
middle unit m1: dials [ 0.2, 0.3 ] nudge 0.0
middle unit m2: dials [-0.4, -0.1 ] nudge 0.3
out unit o0: dials [ 1.0, 0.5, -1.0 ] nudge 0.0
out unit o1: dials [ 0.5, -0.2, 2.0 ] nudge 0.1
Forward, every stroke written. Each middle: dial1 x input1 + dial2 x input2 + nudge:
m0 = 0.5 x 1.0 + (-0.1) x (-2.0) + 0.1 = 0.5 + 0.2 + 0.1 = 0.8
m1 = 0.2 x 1.0 + 0.3 x (-2.0) + 0.0 = 0.2 - 0.6 = -0.4
m2 = -0.4 x 1.0 + (-0.1) x (-2.0) + 0.3 = -0.4 + 0.2 + 0.3 = 0.1
Squash each (negative -> 0):
m0 = 0.8 m1 = 0 (squashed dead) m2 = 0.1
Then the outputs read the three squashed middles:
o0 = 1.0 x 0.8 + 0.5 x 0 + (-1.0) x 0.1 + 0.0 = 0.8 - 0.1 = 0.7
o1 = 0.5 x 0.8 + (-0.2) x 0 + 2.0 x 0.1 + 0.1 = 0.4 + 0.2 + 0.1 = 0.7
The tiny twin answers [0.7, 0.7]. Seventeen dials-and-nudges (2x3+3 = 9 and 3x2+2 = 8)
did all of it, and one middle died at the squash -- its whole row of wires going out
carries nothing this pass.
HOW WRONG WAS IT? SUBTRACT, SQUARE, AVERAGE
Say the human's recorded answer for this question, re-ruled, was [0.1, -2.3]. The
METER grades the guess in three strokes: subtract each pair, square each gap (a miss
of -3 must count as badly as +3, and squaring also punishes one big miss more than
two small ones), then average:
miss 1: 0.7 - 0.1 = 0.6 square: 0.36
miss 2: 0.7 - (-2.3) = 3.0 square: 9.00
meter = (0.36 + 9.00) / 2 = 9.36 / 2 = 4.68
One number, 4.68, for the whole answer. On the real machine the meter melts far more
than two: it grades 128 pairs at once (a HANDFUL -- more on that below), each pair
sixteen numbers, so one reading averages 128 x 8 x 2 = 2,048 squared misses. Watch
the melt on a 2 x 2 answer so no number hides:
truth [ 1.0 2.0 ] guess [ 1.0 2.0 ] misses [ 0.0 0.0 ]
[ 3.0 4.0 ] [ 2.0 5.0 ] [-1.0 1.0 ]
squares [ 0 0 ] sum = 0 + 0 + 1 + 1 = 2 meter = 2 / 4 = 0.5
[ 1 1 ]
Every miss is in there; none is reported separately. The meter is a melter: many
numbers in, ONE out.
Guess [1.0, 2.0] against truth [2.0, 0.0]. Meter?
CHECK: misses -1.0 and 2.0 ; squares 1.0 and 4.0 ; meter = 5.0 / 2 = 2.5.
LEARNING IS A DIAL-TURN, AND THE WIGGLE TEST PRICES EACH DIAL
The meter read 4.68. Learning = turning each dial a little so the next reading is
smaller. Which way, and how much, one dial at a time? Ask the dial directly: WIGGLE
it and watch the meter.
Take o0's first dial (1.0, the wire from middle m0 to output o0) and nudge it up by
0.001. Only o0 changes, by 0.001 x m0 = 0.001 x 0.8 = 0.0008:
o0: 0.7 -> 0.7008
miss1: 0.6 -> 0.6008 square: 0.6008 x 0.6008 = 0.36096064
meter: (0.36096064 + 9.00) / 2 = 4.68048032
The meter rose 0.00048032 for a wiggle of 0.001, so this dial's PULL is
pull = 0.00048032 / 0.001 = 0.48032 ~ 0.480
(The clean product miss x middle = 0.6 x 0.8 = 0.48 is the exact pull; the extra
0.00032 is the wiggle's own square and shrinks as the wiggle shrinks.) A positive
pull means "turning this dial UP makes things WORSE", so turn it down, a small step
against the pull:
new dial = 1.0 - 0.1 x 0.48 = 1.0 - 0.048 = 0.952
Now the dial the squash hid: wiggle o0's SECOND dial (0.5, the wire from the squashed-dead
m1). The output changes by 0.001 x m1 = 0.001 x 0 = 0. The meter does not move. Pull
= 0. A dead middle hides every wire that leaves it -- this pass teaches those dials
nothing, and only a different question that wakes m1 can reach them.
The real machine does exactly this pricing for all 137,232 dials at every meal (by a
bookkeeping shortcut that computes every pull in one backward sweep rather than
137,232 separate wiggles -- the shortcut changes the speed, not the numbers), then
steps each dial against its pull. The step sizes are managed by a tuner named Adam --
a per-dial step-size picker; this page turns dials with a plain fixed step because
the pull, not the tuner, is the idea.
FEEDING SCHEDULE, AT REAL SCALE
Nobody feeds the machine one pair at a time, and nobody feeds it the whole diary at
once. The schedule between:
one meal = a HANDFUL of 128 pairs, drawn at random from the 24,208
one meal -> one meter reading (2,048 misses melted) -> one turn of all dials
one READ = enough meals to pass the whole diary once
= 24,208 / 128 = 189 handfuls (the leftover 16 pairs are skipped;
a random 16 miss out each read, different ones next read)
the course = 400 full reads = 189 x 400 = 75,600 dial-turns
report card: every 10,000 turns the dials freeze and the machine plays 100 fresh
games for a score -- 7 report cards across the course.
Random handfuls matter: the diary is stored game by game, and 128 neighbours from one
game would all say nearly the same thing. A random handful mixes early game 12 with
late game 187, and every meal pulls the dials toward copying EVERYWHERE at once.
AND THE FIRST METER READING IS PREDICTABLE BY PENCIL
Here is the promised number. Before training, what should the very first meter
reading be? The dials are newborn -- small and random -- so the machine's sixteen
outputs hover near 0. The recorded answers were re-ruled: every answer column has
middle 0 and spread 1, so a typical recorded number sits about 1 away from 0. Each
squared miss therefore averages about 1 squared = 1, and the melt of 2,048 such
squares reads about
meter ~ 1.0 before any learning.
The real machine's actual first reading, measured on one real run: 0.9885. The
pencil called it. And the next readings show learning biting -- meter 0.568 by turn
20, 0.188 by turn 40, 0.128 by turn 60 (readings jump meal to meal; each grades a
different random handful). Copying works: the machine drives its miss down and, by
the later report cards, pushes the block like the diary does -- on every question
whose recorded answers agree. That last clause is a crack, and the next page widens
it into the chapter's real problem.
A MACHINE THAT SEES NO TABLE, A METER THAT MELTS, A BADNESS I COULD PREDICT
"The machine sees the table." It sees five numbers, and at raw scale even those
poison it: 222.0 through a newborn +-0.1 dial shouts in the tens while the twist 0.67
whispers, so every pull computed downstream is sized for the shout. Re-ruled, -0.07
and -2.03 and 0.67 speak at one volume. Food prep is not cosmetics; it is what makes
one shared step size sane for every dial.
"A flat print of 16 cannot be graded against an answer shaped 8 by 2." I hunted for a
conversion formula, but the sixteen numbers ARE the eight pairs -- print slots 1 and
2 are move 1, slots 3 and 4 are move 2, and so on. Re-boxing changes the container
and touches no number, and after it the subtraction lines up pair by pair.
"The meter should hand back sixteen verdicts, one per number." It melts them: 128
pairs x 8 moves x 2 numbers = 2,048 squared misses in, ONE number out. I watched the
2 x 2 example -- misses 0, 0, -1, 1 became squares 0, 0, 1, 1, sum 2, meter 0.5 --
and stopped hunting for the per-number report inside a number built to not contain
one. The pulls, not the meter, carry the per-dial detail.
"Somebody must tell the machine which game each question came from." Nobody does. A
handful of 128 arrives with early-game and late-game questions shuffled together, on
purpose: neighbours from one game would whisper the same lesson 128 times, and the
dials -- shared by every question there is -- would lurch toward one game's habits.
The machine never picks its food, and the shuffle is the point.
"A newborn machine starts at some unknowable badness." I computed it before training:
re-ruled answers sit about 1 spread from 0, newborn dials print about 0, so each
squared miss is about 1 and the melt reads about 1.0. The run's first real reading
was 0.9885. When a number is predictable before the experiment, the machine has
stopped being magic -- that is the whole point of the pencil.
SEAM. Pencil ends here; below, the same numbers in Python.
No loops, no functions -- the re-ruling, the tiny twin's forward pass, the meter, one
wiggle-priced dial-turn, and the real machine's schedule arithmetic. Count the twin's
forward pass: 12 multiplies, 10 adds, 3 squashes; the meter adds 2 subtracts, 2
squares, 1 average.
# --- re-rule the real first question (two of the five columns) ---
hand_x = (222.0 - 229.1) / 101.9 # -7.1 / 101.9 = -0.07 (2 decimals)
hand_y = ( 97.0 - 293.3) / 96.5 # -196.3 / 96.5 = -2.03
# answer columns use their own rulers:
move_x = (233.0 - 228.2) / 101.6 # 4.8 / 101.6 = 0.05
move_y = ( 71.0 - 294.0) / 96.0 # -223.0 / 96.0 = -2.32
# --- the tiny twin: question [1.0, -2.0], dials as set above ---
x1, x2 = 1.0, -2.0
m0 = 0.5 * x1 + -0.1 * x2 + 0.1 # 0.5 + 0.2 + 0.1 = 0.8
m1 = 0.2 * x1 + 0.3 * x2 + 0.0 # 0.2 - 0.6 = -0.4
m2 = -0.4 * x1 + -0.1 * x2 + 0.3 # -0.4 + 0.2 + 0.3 = 0.1
m0 = max(0.0, m0) # 0.8 (squash: negatives die)
m1 = max(0.0, m1) # 0 (dead this pass)
m2 = max(0.0, m2) # 0.1
o0 = 1.0 * m0 + 0.5 * m1 + -1.0 * m2 + 0.0 # 0.8 - 0.1 = 0.7
o1 = 0.5 * m0 + -0.2 * m1 + 2.0 * m2 + 0.1 # 0.4 + 0.2 + 0.1 = 0.7
# --- the meter against the recorded answer [0.1, -2.3] ---
miss1 = o0 - 0.1 # 0.6
miss2 = o1 - -2.3 # 3.0
meter = (miss1**2 + miss2**2) / 2 # (0.36 + 9.00) / 2 = 4.68
# --- wiggle-price one dial (o0's wire from m0), then turn it ---
o0_w = 1.001 * m0 + 0.5 * m1 + -1.0 * m2 # 0.7008
meter_w = ((o0_w - 0.1)**2 + miss2**2) / 2 # 4.68048032
pull = (meter_w - meter) / 0.001 # 0.48032 ~ 0.48 (= miss1 * m0)
dial = 1.0 - 0.1 * 0.48 # 0.952: step against the pull
# the dial from dead m1 prices to 0: wiggling it moves nothing this pass
# --- the real machine's schedule, arithmetic only ---
dials = 5*256+256 + 256*256+256 + 256*256+256 + 256*16+16 # 137232
handfuls = 24208 // 128 # 189 per full read (16 pairs sit out)
turns = 189 * 400 # 75600 dial-turns over the course
melted = 128 * 8 * 2 # 2048 misses per meter reading
print(o0, o1, meter) # 0.7 0.7 4.68
----------------------------------------------------------------------------------------------
IN THIS CHAPTER (Chapter 19 -- Learning by Copying):
Part 1 -- The Diary and the Eight Envelopes: Learning by Copying
Part 2 (this post) .
Part 3 -- One Question, Two Right Answers: Where Copying Breaks
Part 4 -- Teaching the Wind: Flow Matching by Pencil
Part 5 -- Riding the Wind: From Noise to an Answer
<- Back to all posts
----------------------------------------------------------------------------------------------
home . source on GitHub
==============================================================================================