==============================================================================================
RAHUL'S ML BLOG -- notes on machine learning, worked out by hand est. 2026
==============================================================================================
home | about | archive | glossary | contact
----------------------------------------------------------------------------------------------
CHAPTER 20 . THE PUSH-T MACHINE . PART 1 OF 3
From Five to Twenty-Two: Why Flow Matching Needs a Clock
============================================================================================
Here is the problem. A robot arm must push a T-shaped block into a goal
strip on a table. A human already solved this task two hundred and six
times. Every move the human ever made was recorded. Can a machine read
those recordings and learn to copy the human well enough to push the
block home in a game it has never seen?
The answer is: almost. The "almost" fails in one specific place, in a way
that is mathematically guaranteed. This post shows what that place is, why
no amount of extra training can fix it, and what the exact fix demands.
The fix forces the machine's input to grow from 5 wires to 22. This post
works out why.
-------
THE TABLE
+--------------------------+
| ___ |
| | | |
| |_T_| <- block |
| |
| (*) <- hand |
| |
|====== GOAL STRIP ========|
+--------------------------+
The hand slides anywhere on the table. Push the T into the goal strip
at the bottom. A round hand (the robot's end-effector) is the only thing
that can touch the block. The human controlled where the hand moved each
tick. Those control decisions are what the machine will copy.
At every tick, the table gives the machine five numbers:
[ T_x | T_y | hand_x | hand_y | hand_speed ]
0 1 2 3 4
T_x and T_y: where the block is. hand_x and hand_y: where the hand is.
hand_speed: how fast the hand is moving. Those five numbers describe
everything visible at this moment.
The machine must send back two numbers: a target point (target_x,
target_y) telling the game engine where to slide the hand next.
WHAT THE MACHINE SENDS BACK EACH TICK
[ target_x | target_y ]
Two numbers in, two numbers out per tick. That is the task.
-------
PLAN EIGHT TICKS AT ONCE
Replanning every single tick means the machine must decide 20 times per
second. Any delay between observation and decision leaves the hand blind.
The solution: let the machine plan eight ticks at once. The machine sends
back eight move targets instead of one. The game uses them one at a time,
and after eight ticks it asks the machine for the next eight.
ONE PLAN: EIGHT MOVES
move 0: [ target_x_0, target_y_0 ]
move 1: [ target_x_1, target_y_1 ]
...
move 7: [ target_x_7, target_y_7 ]
That is 8 x 2 = 16 numbers per plan. The machine reads five numbers and
must print sixteen. The recordings are re-cut to match: every recorded
tick t becomes one question (five situation numbers at tick t) paired
with one answer (the eight recorded moves starting at tick t). The last
seven ticks of each game cannot start a plan -- no eight moves remain --
so they are dropped. The 206 games holding 25,650 recorded ticks yield
24,208 valid question-answer pairs after this cut.
-------
WHAT ONE ADDER-BOX DOES
Before building the machine, understand the one piece it is made from.
The piece is called an adder-box (or neuron or unit). It works like this:
ONE ADDER-BOX: THREE WIRES IN, ONE NUMBER OUT
wire_0 = 0.5
wire_1 = 0.3 each wire has one dial
wire_2 = 0.7
dial_0 = 1.2 dial_1 = -0.5 dial_2 = 0.8
output = wire_0 * dial_0 + wire_1 * dial_1 + wire_2 * dial_2
= 0.5 * 1.2 + 0.3 * (-0.5) + 0.7 * 0.8
= 0.6 + (-0.15) + 0.56
= 1.01
That is the entire computation. Multiply each incoming wire value by that
wire's dial, sum everything, get one number out. Nothing more.
A dial is just a number stored inside the box. Training will change these
dials. They start as small random numbers near zero. After training they
hold numbers that make the box useful.
The box has one dial per incoming wire. Three wires in means three dials.
Twenty-two wires in would mean twenty-two dials per box.
-------
A LAYER IS A ROW OF BOXES
A layer is many adder-boxes sitting side by side, ALL reading the SAME
incoming wires, each producing its own output number.
ONE LAYER: 2 BOXES, 3 WIRES IN
wire_0 = 0.5 ---+---> box_0: dials [1.2, -0.5, 0.8] -> output 1.01
wire_1 = 0.3 ---+---> box_1: dials [0.3, 0.9, 0.2] -> output 0.38
wire_2 = 0.7 ---+
Box_0 applies its own set of dials to the three wires: 0.5*1.2 + 0.3*(-0.5) +
0.7*0.8 = 0.6 - 0.15 + 0.56 = 1.01.
Box_1 applies a DIFFERENT set of dials to the same three wires:
0.5*0.3 + 0.3*0.9 + 0.7*0.2 = 0.15 + 0.27 + 0.14 = 0.56.
Wait -- let me recheck box_1: 0.5*0.3 + 0.3*0.9 + 0.7*0.2 = 0.15 + 0.27 + 0.14 = 0.56.
Every box in a layer reads the same wires but uses its own dial set. Two
boxes, two outputs. 256 boxes, 256 outputs.
Stacking layers: the outputs of one layer become the input wires of the
next. The first layer reads the five original situation numbers and
produces 256 numbers. The second layer reads those 256 and produces 256
more. The last layer reads the final 256 and produces 16: the plan.
-------
WHY THE RELU GATE IS NOT OPTIONAL
Stack two layers without anything between them. Layer 1 computes:
hidden = input * W1
Layer 2 computes:
output = hidden * W2 = (input * W1) * W2 = input * (W1 * W2)
W1 * W2 is just another single matrix. Two stacked layers collapse into
one layer with dials W1*W2. Add a third layer: still collapses. No matter
how many layers you stack without anything between them, the whole thing
is equivalent to one layer. Depth buys nothing.
The gate that prevents this collapse is the ReLU: applied between layers,
it clips every negative number to zero.
RELU: A GATE FOR EACH NUMBER
input number -> if > 0: pass it through unchanged
if <= 0: output zero
0.31 --> 0.31 (positive, passes through)
0.27 --> 0.27 (positive, passes through)
-0.40 --> 0.00 (negative, clipped to zero)
Why does this break the collapse? Because (input * W1) after ReLU is
no longer a linear function of input. Some of the hidden values were
clipped to zero. Which ones were clipped depends on the input itself.
That makes the overall function non-linear, and two non-linear layers
cannot be collapsed into one.
With ReLU between layers, depth buys expressive power. Without it,
depth buys nothing. That is why ReLU appears between every pair of layers
in the machine.
-------
THE FIVE-WIRE COPYING MACHINE
Now the architecture makes sense. Five wires in, three hidden layers of
256 boxes each, sixteen wires out:
5 -> [256 boxes + ReLU] -> [256 boxes + ReLU]
-> [256 boxes + ReLU] -> [16 boxes] -> 16 outputs
Count the dials:
- Layer 1: 5 wires feeding 256 boxes = 5 * 256 = 1,280 dials, plus 256 biases = 1,536
- Layer 2: 256 wires feeding 256 boxes = 256 * 256 = 65,536 plus 256 biases = 65,792
- Layer 3: same = 65,792
- Layer 4: 256 wires feeding 16 boxes = 256 * 16 = 4,096 plus 16 biases = 4,112
- Total: 1,536 + 65,792 + 65,792 + 4,112 = 137,232 dials
All 137,232 dials start as random numbers near zero. Every one of them is
turned a tiny amount on each training pass.
-------
HOW WRONG IS THE GUESS
Every question-answer pair from the diary provides one lesson. The machine
reads the five situation numbers, runs them through all 137,232 dials, and
prints sixteen plan numbers. The true sixteen numbers (the human's actual
moves) are also known. The ruler measures how wrong the machine is:
THE MSE RULER
L = (1/16) * sum over k from 0 to 15 of (guess_k - truth_k)^2
Subtract the guess from the truth at each of the 16 positions. Square
each gap. Average all 16 squared gaps. That is L, the Mean Squared Error.
Why square? Two reasons. First, squaring makes positive and negative gaps
both count as positive (a guess of 0.5 is just as wrong as a guess of -0.5
when the truth is zero). Second, squaring penalises large gaps much more
than small ones: a gap of 2 costs 4; a gap of 1 costs 1; a gap of 0.1
costs only 0.01. The machine learns to avoid large gaps.
If every guess exactly matches the truth, every gap is zero and L = 0.
If every guess is 1.0 away from the truth, L = 1.0. Training pushes L
toward zero by turning the dials.
-------
THE FORK
Suppose the T sits near the centre and the hand is to its left. To push
the T into the goal strip, the human can swing the hand ABOVE the T first,
or BELOW it first. Both paths work. The human chose above on some days,
below on others. The diary holds both.
THE FORK: ONE SITUATION, TWO VALID ANSWERS
[ above path: plan_A ]
/
state s --
\
[ below path: plan_B ]
Both plan_A and plan_B are in the diary. Neither is wrong. When training,
the machine sees state s with the above-plan answer, and sees state s again
with the below-plan answer. It must learn one sixteen-number output for
state s. What does it learn?
-------
THE MATH THAT FORCES THE MACHINE TO PRINT GARBAGE
Pick one output position -- say the y-component of the first move. The
above path has it as Y_A = +0.3 (move up first). The below path has it as
Y_B = -0.3 (move down first). Each appears equally often in the diary.
Call the machine's printed number c. The training ruler grades c against
both answers:
L(c) = (1/2) * (c - Y_A)^2 + (1/2) * (c - Y_B)^2
To find the c that makes L smallest, differentiate with respect to c and
set the result to zero:
dL/dc = (c - Y_A) + (c - Y_B) = 2c - Y_A - Y_B
set equal to zero:
2c - Y_A - Y_B = 0
c = (Y_A + Y_B) / 2
With Y_A = +0.3 and Y_B = -0.3:
c = (0.3 + (-0.3)) / 2 = 0 / 2 = 0.0
The c that minimises the MSE ruler is the average. The machine has no
choice: the average is the provably correct answer to the question the
ruler is asking. The average of "move up 0.3" and "move down 0.3" is
"move straight: 0.0." The hand slides straight into the side of the block.
The block does not move.
This is not a training bug. More epochs, more dials, a bigger machine:
all find the same minimum, because the RULER forces it. The failure is in
the question shape: "given one state, print one answer" cannot be answered
honestly when one state has two correct answers.
-------
STOP PRINTING ANSWERS. PRINT WIND INSTEAD.
If the machine printed the average of two good answers and got a bad one,
the fix is: stop asking it to print the answer. Ask it instead to print
a direction -- the wind that blows from wherever it currently stands
toward the correct answer.
NOISE, BLEND, ANSWER: A STRAIGHT LINE
noise (tau=0.0) blend M answer (tau=1.0)
* -------> M --------> *
| |
pure random the human's recorded
16 numbers 16 moves
Start from pure random noise. Draw a straight line to the recorded answer.
Every point along that line is a blend. tau is how far along: tau=0 is
all noise, tau=1 is the answer, tau=0.4 is 40% of the way.
The blend formula: call the noise N (16 random numbers) and the answer A
(16 recorded moves). The blend at tau=0.4 is:
M = (1 - 0.4) * N + 0.4 * A = 0.6 * N + 0.4 * A
Concrete: N = [0.10, 0.80], A = [0.50, 0.60], tau = 0.4 (showing 2 of
the 16 numbers):
M_0 = 0.6 * 0.10 + 0.4 * 0.50 = 0.06 + 0.20 = 0.26
M_1 = 0.6 * 0.80 + 0.4 * 0.60 = 0.48 + 0.24 = 0.72
The true wind -- the direction from noise to answer -- is:
wind = A - N = [0.50 - 0.10, 0.60 - 0.80] = [0.40, -0.20]
The wind is constant along the line (the line is straight, so its slope
does not change). If you start at blend M and walk the remaining
(1 - tau) = 0.6 of the way along the wind, you land exactly on A:
M_0 + (1 - tau) * wind_0 = 0.26 + 0.6 * 0.40 = 0.26 + 0.24 = 0.50 = A_0
M_1 + (1 - tau) * wind_1 = 0.72 + 0.6 * (-0.20) = 0.72 - 0.12 = 0.60 = A_1
This cures the fork because two different noise draws start at two
different blend points. One noise placed you on the above-path line; the
other on the below-path line. Each line has its own wind pointing to its
own answer. No averaging occurs because the machine is asked "from THIS
blend point, which way?" -- and two blend points are in two different
places.
-------
WHY FIVE WIRES IS NOT ENOUGH FOR THE WIND MACHINE
The wind machine's job: read the situation and print the wind. The
situation is five numbers. Can the machine manage with only those five?
No. Consider two blend points that happen to land at the same position in
16-dimensional space. One is at tau = 0.1 (just 10% of the way from noise
to answer -- mostly noise, large remaining distance). The other is at
tau = 0.9 (90% of the way -- mostly answer, small remaining distance).
SAME BLEND POSITION, DIFFERENT STAGE
tau=0.1: answer
*--------[M?]------------------------------->*
^ big wind still needed
tau=0.9: answer
*-------------------------------------------[M?]->*
^ small finishing nudge
At tau=0.1, the machine is far from the answer. The wind should be large
to carry it there. At tau=0.9, the machine is nearly done. The wind
should be small so it does not overshoot.
If the machine sees only the five situation numbers and the 16-number
blend position, it cannot tell which case it is in. Both blend points
happen to be at the same coordinates but at different stages. The machine
averages the big and small winds and gets both wrong.
With tau, the machine sees the complete picture. The three inputs it needs:
1. The five situation numbers (what is happening on the table)
2. The 16-number blend point M (where are you in the 16D space right now)
3. The tau dial (how far along the walk are you)
Total: 5 + 16 + 1 = 22 wires in.
THE 22-WIRE INPUT ROW
[s0 s1 s2 s3 s4 | M0 M1 M2 ... M15 | tau]
---- five ---- ----- sixteen ------ one
5 + 16 + 1 = 22
-------
THE 22-WIRE MACHINE
Same structure as the five-wire machine, but the first layer now connects
22 wires to 256 boxes instead of 5:
22 -> [256 boxes + ReLU] -> [256 boxes + ReLU]
-> [256 boxes + ReLU] -> [16 boxes] -> 16 outputs (winds)
Dial count:
- Layer 1: 22 * 256 = 5,632 plus 256 biases = 5,888
- Layers 2 and 3: unchanged at 65,792 each
- Layer 4: unchanged at 4,112
- Total: 5,888 + 65,792 + 65,792 + 4,112 = 141,584 dials
Every training lesson: pick a recorded 16-number answer from the diary,
draw 16 random noise numbers, draw a random tau, blend them into M, feed
[situation, M_flat, tau] into the machine as 22 input numbers, compare
the machine's 16 output numbers to the true wind (answer minus noise) with
the MSE ruler, turn all 141,584 dials. Part 2 shows how each dial is
turned and by how much.
-------
WHERE I ACTUALLY GOT STUCK (SO YOU DON'T HAVE TO)
Wrong belief: tau alone is enough. If the machine knows how far along the
walk it is, it can figure out the blend position from the noise and the
answer.
The number that broke it: at any tau, two different noise draws produce
two different blend points at the same tau. The machine has no access to
which noise was drawn -- the noise is drawn fresh at serving time without
a recorded match. So the blend position M cannot be reconstructed from
tau alone. M must be handed to the machine explicitly as 16 separate
numbers. The input is 22, not 6.
The rule to carry: the machine needs all three inputs. State: what the
table looks like. M: where the current blend sits in 16D space. tau: how
far along the walk. Drop any one of them and the machine goes blind.
-------
ONE BREATH
A Push-T table, a T-block, a round hand, five situation numbers at each
tick, 24,208 recorded question-answer pairs. One adder-box multiplies
each incoming wire by a dial and sums: one number out. A layer is 256
such boxes reading the same wires, each with its own dial set. ReLU
between layers clips negative pre-activations to zero -- without it,
stacked layers collapse to one. The five-wire copying machine (five in,
three layers of 256, sixteen out, 137,232 dials) learns to print plans
directly but fails at the fork: when one state has two valid answers,
differentiating the MSE ruler and setting to zero gives c = (Y_A+Y_B)/2,
the average, which is the third path nobody ever chose. The wind machine
prints a direction from the current blend toward the answer instead of
the answer itself, curing the fork because two noise draws start two
separate walks. But the direction depends on WHERE the blend sits (16
numbers) and HOW FAR along the walk it is (tau = 1 number) as well as
the table situation (5 numbers): 5 + 16 + 1 = 22 wires in, 141,584 dials.
SEAM. Pencil ends here; below, the same numbers in Python.
-------
```python
# --------------------------------------------------------------------------
# PART 1: One adder-box (the fundamental unit)
# 3 wires in, 3 dials, 1 number out.
# --------------------------------------------------------------------------
wire_0 = 0.5 ; dial_0 = 1.2 # first wire and its dial
wire_1 = 0.3 ; dial_1 = -0.5 # second wire and its dial
wire_2 = 0.7 ; dial_2 = 0.8 # third wire and its dial
box_output = wire_0*dial_0 + wire_1*dial_1 + wire_2*dial_2 # multiply each, sum
# = 0.5*1.2 + 0.3*(-0.5) + 0.7*0.8 = 0.6 - 0.15 + 0.56
print("one box output:", round(box_output, 4)) # 1.01
# --------------------------------------------------------------------------
# PART 2: Blend formula M = (1 - tau)*noise + tau*answer
# Using 2 numbers; the real machine blends 16-number chunks.
# --------------------------------------------------------------------------
noise_0 = 0.10 ; noise_1 = 0.80 # two noise numbers drawn from a bell curve
answer_0 = 0.50 ; answer_1 = 0.60 # two numbers from one recorded human move
tau = 0.4 # 40% of the way from noise to answer
M_0 = (1 - tau) * noise_0 + tau * answer_0 # 0.6*0.10 + 0.4*0.50 = 0.06+0.20 = 0.26
M_1 = (1 - tau) * noise_1 + tau * answer_1 # 0.6*0.80 + 0.4*0.60 = 0.48+0.24 = 0.72
print("blend M:", round(M_0, 4), round(M_1, 4)) # 0.26 0.72
# --------------------------------------------------------------------------
# PART 3: True wind = answer - noise
# --------------------------------------------------------------------------
wind_0 = answer_0 - noise_0 # 0.50 - 0.10 = 0.40
wind_1 = answer_1 - noise_1 # 0.60 - 0.80 = -0.20
print("wind:", round(wind_0, 4), round(wind_1, 4)) # 0.40 -0.20
# --------------------------------------------------------------------------
# PART 4: Walk the remaining (1-tau) of the wind from M; must land on answer.
# --------------------------------------------------------------------------
remaining = 1 - tau # 1 - 0.4 = 0.6
land_0 = M_0 + remaining * wind_0 # 0.26 + 0.6*0.40 = 0.26+0.24 = 0.50
land_1 = M_1 + remaining * wind_1 # 0.72 + 0.6*(-0.20) = 0.72-0.12 = 0.60
print("landing:", round(land_0, 4), round(land_1, 4)) # 0.50 0.60 == answer
# --------------------------------------------------------------------------
# PART 5: MSE averaging failure -- the math that forces the machine to print
# the mean of two conflicting answers.
# --------------------------------------------------------------------------
Y_A = 0.3 # y-component of the "above" plan: move up 0.3
Y_B = -0.3 # y-component of the "below" plan: move down 0.3
# The MSE-minimising single answer c = (Y_A + Y_B) / 2
c_star = (Y_A + Y_B) / 2 # (0.3 + (-0.3)) / 2 = 0.0
print("MSE best single answer:", c_star) # 0.0 -- straight, into the block
# Confirm c_star loses less than any other choice
loss_c_star = 0.5*(c_star - Y_A)**2 + 0.5*(c_star - Y_B)**2 # 0.5*0.09+0.5*0.09
loss_at_Y_A = 0.5*(Y_A - Y_A)**2 + 0.5*(Y_A - Y_B)**2 # 0 + 0.5*0.36
loss_at_mid = 0.5*(0.15 - Y_A)**2 + 0.5*(0.15 - Y_B)**2 # 0.5*0.0225+0.5*0.2025
print("loss at mean (0.0) :", round(loss_c_star, 4)) # 0.09 -- minimum
print("loss at Y_A (0.3) :", round(loss_at_Y_A, 4)) # 0.18 -- worse
print("loss at 0.15 (halfway):", round(loss_at_mid, 4)) # 0.1125 -- also worse
# --------------------------------------------------------------------------
# PART 6: The 22-wire input row -- count the wires.
# --------------------------------------------------------------------------
s0=0.32; s1=0.55; s2=0.18; s3=0.73; s4=0.02 # five situation numbers (normalised)
# 16 blend numbers (real is 8 moves x 2 coordinates, here listed flat)
M = [0.26, 0.72, 0.31, 0.65, 0.28, 0.70, 0.25, 0.67,
0.22, 0.64, 0.19, 0.61, 0.16, 0.58, 0.13, 0.55] # 16 numbers (8 blend pairs)
tau_val = 0.4 # one number: how far along the walk
row = [s0, s1, s2, s3, s4] + M + [tau_val] # glue side by side
print("input row width:", len(row)) # 22 = 5 + 16 + 1
print("first 5 (state) :", row[:5]) # [0.32, 0.55, 0.18, 0.73, 0.02]
print("last 1 (tau) :", row[-1]) # 0.4
```
Running this code prints:
one box output: 1.01
blend M: 0.26 0.72
wind: 0.4 -0.2
landing: 0.5 0.6
MSE best single answer: 0.0
loss at mean (0.0) : 0.09
loss at Y_A (0.3) : 0.18
loss at 0.15 (halfway): 0.1125
input row width: 22
first 5 (state) : [0.32, 0.55, 0.18, 0.73, 0.02]
last 1 (tau) : 0.4
The one adder-box outputs 1.01. The blend sits at [0.26, 0.72]. Walking
the remaining 60% of the wind from there lands exactly back on [0.50,
0.60] -- the original answer. The MSE minimiser is 0.0 (the average),
which loses less than any other single answer, including one of the true
answers (0.18 vs 0.09). The input row is exactly 22 numbers wide.
-------
>> NOTE: STANDARD JARGON
adder-box / neuron = one unit: multiply each input by a dial, sum, output one number
dial / weight = one multiplier stored in a box; training changes these numbers
layer = a row of adder-boxes all reading the same input wires
ReLU = max(x, 0): clips negative pre-activations to zero
MSE = Mean Squared Error: average of squared gaps between guess and truth
action chunk = the 8-move plan the machine prints (8 * 2 = 16 numbers)
blend / M = (1-tau)*noise + tau*answer: a point on the line from noise to answer
wind / velocity = answer - noise: the constant direction from noise toward answer
tau = progress dial: 0 is pure noise, 1 is pure answer
22-wire input = cat([state(5), blend_flat(16), tau(1)]) = one row fed to the machine