April 2026 Puzzle 1(a)

Max-of-5 mechanistic analysis

A compact index for the static reports generated while reverse-engineering the one-layer attention-only transformer.

Report 1

[ANS] Attention Patterns

Shows that Head 3's [ANS] attention is max-selective on the sampled inputs, while Head 1 mostly attends to special tokens.

Open attention report Report 2

Head 3 OV Circuit

Tests whether Head 3 copies the selected max digit into the output logits. The result is not a simple diagonal copy circuit.

Open OV report Report 3

Component Logit Attribution

Decomposes final [ANS] logits into the residual stream and each attention head, then attributes the winning answer margin.

Open attribution report Report 4

QK Digit Ranking

Measures raw pre-softmax QK scores from [ANS] to digit keys. Head 3 has the cleanest monotonic score ramp from digit 0 to digit 9.

Open QK report
PDF attention report Attention metrics CSV OV diagonal summary CSV QK metrics CSV OV metadata JSON Attribution metadata JSON