Card: learning-curve figures (eff, losses), 64M final @180k row

.gitattributes

@@ -37,3 +37,5 @@ board_overlay_v5.png filter=lfs diff=lfs merge=lfs -text
 charts/scaling_v5_latarnik.png filter=lfs diff=lfs merge=lfs -text
 progress_v5.png filter=lfs diff=lfs merge=lfs -text
 decontam_verify/fwe_kept_sample.jsonl filter=lfs diff=lfs merge=lfs -text
+figures/fig1_learning_curves_web.png filter=lfs diff=lfs merge=lfs -text
+figures/fig2_loss_curves_web.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -171,6 +171,7 @@ Two final models are training now with a recipe fixed before the runs started. N
 | 64M final (14×576) | 120,000 (15.8%) | 45.58 | 75.15 | 2.4593 | 74.53 | training in progress; not a result |
 | 64M final (14×576) | 140,000 (18.4%) | 44.70 | 74.74 | 2.4501 | 74.11 | training in progress; not a result |
 | 64M final (14×576) | 160,000 (21.1%) | 45.79 | 73.81 | 2.4416 | 74.19 | training in progress; not a result |
+| 64M final (14×576) | 180,000 (23.7%) | 44.99 | 73.72 | 2.4344 | 73.90 | training in progress; not a result |
 | 128M final (16×768) | 20,000 (2.6%) | 43.90 | 73.91 | 2.5383 | 71.34 | training in progress; not a result |
 | 128M final (16×768) | 40,000 (5.3%) | 46.68 | 74.81 | 2.4612 | 72.77 | training in progress; not a result |
 | 128M final (16×768) | 60,000 (7.9%) | 47.43 | 75.24 | 2.4151 | 73.28 | training in progress; not a result |
@@ -193,6 +194,16 @@ Reference, same protocol: the v1 models at the end of their runs (step 400,000)
 
 At step 40,000 the final 64M run was +0.99 eff ahead of v1 at the same step (ARC-Easy +2.02, BLiMP +0.85, byte_ppl −0.0016) and the final 128M run +0.81 eff (ARC-Easy +0.85, BLiMP +1.60, byte_ppl +0.0035). At step 80,000 the 64M runs are level (73.75 vs 73.76: ARC-Easy +1.60, BLiMP −1.72, byte_ppl −0.0145), and the final 128M run is +0.79 eff ahead of its v1 (73.83 vs 73.04: ARC-Easy +0.21, BLiMP +2.04, byte_ppl −0.0137). At step 160,000 the final 64M run is 0.31 eff behind v1 at the same step (74.19 vs 74.50: ARC-Easy +0.80, BLiMP −1.71, byte_ppl −0.0020). At step 120,000 the final 128M run is +1.02 eff ahead of its v1 (74.61 vs 73.59: ARC-Easy +0.80, BLiMP +2.32, byte_ppl +0.0132); for v1 128M this is the last checkpoint before its resume. Read this with three caveats: each is a single measurement (one ARC-Easy point is about one standard error at n = 2,376); the v1 pool was not cleaned of the test-set overlaps listed under *Benchmark contamination check*, while the final pool was; and v1 used a shorter cosine schedule (400k steps; at step 160,000 its learning rate was at about 69% of peak versus 91% for the final run), so from roughly 100k steps on the same-step comparison increasingly favours v1.
 
+**Learning curves** (training in progress; not a result; final 64M to step 180,000, 128M to step 120,000):
+
+![Figure 1: eff against tokens seen for the v1 and final runs, 64M and 128M](figures/fig1_learning_curves_web.png)
+
+*Figure 1.* eff (canonical board protocol) against tokens seen, for v1 (400k-step cosine) and the final runs (760k-step cosine, in progress). Lines are fits within the measured range only, with no extrapolation; the band is ±2σ (seed and evaluation noise). The v1 128M points up to its resume (40k, 80k and 120k, grey) are not fitted.
+
+![Figure 2: training loss and WikiText-2 bits per byte against tokens seen](figures/fig2_loss_curves_web.png)
+
+*Figure 2.* (a) Training loss (5k-step centred mean), final runs up to their last evaluated step. Training losses are on each run's own pool; v1 sampled windows with replacement and, on its shorter schedule, had a lower learning rate at the same step; both lower its training loss, so (a) compares shape, not level. From about 3B tokens the final 64M run has a higher training loss than v1; v1 128M has no logged loss before its resume at 120k. (b) Bits per byte on the WikiText-2 test without the 7 near-copies found in the v1 pool (55 articles), the same text for all runs; the remaining articles overlap any pool by at most ~9% of their 13-grams. On these 55 articles the final and v1 runs are within ±0.01 bits per byte at the same step.
+
 ## Data-attribution experiments (64M, 2026-09-25)
 
 Question: **does changing the training data in the late phase of training move the efficiency score?** Method: two identical 64M models continue from the same flagship checkpoint (step 320k) with the same recipe, seed and number of steps (80k, to step 400k); **they differ only in the data**. Both are evaluated at 340k / 360k / 380k / 400k with the board protocol and a paired bootstrap over items. Each checkpoint folder has its own README with the full data specification.