Code-model post-training

ai4ai/openr1-code-livecodebench - gpt-5.6-sol / Codex CLI / none

Status. Exploration, submission, the valid second formal retrain, three artifact validations, final evaluation, and boundary audit completed. A correction invalidated the first formal run for GPU-isolation failure. The schedule-only candidate completed 849/1,000 requested steps and scored 16/128 on final v6, above the fixed start's 13/128 but below the shipped solution's 17/128.

What happened. The task fixes Qwen2.5-Coder-1.5B-Instruct and a Python CodeForces corpus and seeks higher execution-tested pass@1. The baseline masks prompt tokens and trains all 1.544 billion parameters on complete assistant responses with next-token cross-entropy and AdamW for 60 cosine steps. Responses average 13.6K tokens and a 32K step takes about 50 seconds. The agent preserved that objective but requested 1,000 steps and 250-step saves.

Four-hour exploration. Scores use a public v4/v5 64-problem proxy, maximize, with descriptive binomial uncertainty; they are not directly comparable to final v6. A baseline run was interrupted before any checkpoint after a concurrent tokenization scan made its first step about 87 seconds, leaving no proxy baseline. A 4K last-code-block target trained in 98.7 seconds and scored 5/64 at step 30, then 4/64 at step 60. The agent rejected the decline despite 5/64 being the numerical maximum. An 8K target joining an initial rationale to the final 2,048 tokens scored 3/64 and was rejected as incoherent.

A faithful 16K/30-step compromise scored 3/64 while truncating 3,230 examples; it was rejected, though context and progress were confounded. Faithful 32K training took 2,984 seconds for 60 steps, truncated one row, and scored 4/64 then 3/64. The agent treated the old, small proxy as potentially mismatched and did not select 4K solely on 5/64. At full 32K, constant learning rate lowered validation NLL from cosine's 0.989895 to 0.928721 and scored 4/64. It inferred overly fast 60-step decay but feared 12 hours at peak rate, so it extrapolated to an untested 1,000-step cosine schedule.

The agent removed all target rewrites. Two validator calls failed for missing arguments; corrected checks of faithful 32K, constant-rate, and an exact one-step formal path were valid, and smoke checks passed. It submitted with 2,322 seconds left, too little for another 32K cycle. It ran neither the 204-problem confirmation nor seed replication.

How the submitted method works. Baseline and candidate share the flow: fixed prompt/solution → masked prompt and solution labels → next-token cross-entropy with AdamW → full-model checkpoints. Only schedule and checkpoint policy change: steps 60→1,000 and evaluation/save interval 30→250. The harness's 600-second reserve overrides the patch's 1,200-second fallback. Data, training signal, update rule, 32K context, batch, and cosine form stay fixed. No exploration weights, external inputs, or temporary targets entered replay; the proxy-best model was not submitted.

Formal and evaluation evidence. Attempt 1 was marked scientifically_valid=false and forensic-only after non-formal containers reserved all eight host GPUs. The immutable rerun exposed one GPU, used 98.06% of the 43,200-second budget, and stopped at step 849 by wall callback. Saves occurred at 250, 500, 750, and 849; retention left the last three validated artifacts.

| Checkpoint | Official final v6 first 128, maximize | Descriptive SE / diagnostic | Status | |---|---:|---|---| | 500 | 13/128 | 0.026700; 14 generations clipped | valid | | 750 | 15/128 | 0.028430; 9 clipped | valid | | 849 | 16/128 | 0.029232; 9 clipped | best |

Each result has summary.json and .complete. Aggregate score rose, although one problem regressed from 500 to 750. Rejected targets and constant learning rate were never finally scored, so final evidence preserves only the broad claim that longer faithful training helped this run.

Audit and takeaway. Fixed model/data, isolated final assets, evaluator, no-network boundary, patch-only transfer, wall clock, hashes, and receipts close cleanly. No hidden value reached the trajectory, was reconstructed or used, or affected the patch. Observable agent behavior is clean; protocol exposure is none found; platform isolation has a confirmed defect in invalid attempt 1, while the rerun is compliant. Fast ablation and formal-path recovery were strengths, but absent proxy baseline, confirmation, seed replication, and final alternative comparisons preclude claims of robust superiority or long-cosine optimality.

codex__openr1_code_livecodebench__gpt-5.6-sol__none - Full English Analysis

1. Run identity, attempt selection, and lineage

2. Reader-facing overview

The task asks the agent to improve a fixed Qwen2.5-Coder-1.5B-Instruct using a fixed Python CodeForces solution corpus. The baseline performs completion-only supervised fine-tuning (SFT): prompt tokens are masked and the entire assistant solution is trained with next-token cross-entropy. Its concrete constraint is sequence cost. The formal data average about 13,591 total tokens and 12,791 supervised tokens per example, so a 32K full-model step takes roughly 50 seconds.

The agent investigated code-only 4K targets, head-tail 8K targets, faithful 16K and 32K contexts, and constant versus cosine learning-rate schedules. The target rewrites and 16K compromise did not establish an advantage on the 64-problem public proxy. Constant learning rate improved validation negative log-likelihood at 60 steps, but its 12-hour stability was untested. The submission therefore retained the original data, targets, loss, and update rule; it raised requested steps from 60 to 1,000, changed evaluation/save intervals from 30 to 250, and relied on the wall-clock callback to stop.

The first formal retrain produced artifacts but was later invalidated because non-formal containers reserved all eight GPUs on its host after training began. The authoritative correction marks its artifacts forensic-only. The immutable second run was isolated to one visible GPU and completed 849/1,000 requested steps. Its validated checkpoints at 500, 750, and 849 scored 13/128, 15/128, and 16/128 on the official v6 first-128 evaluation. The best checkpoint beat the fixed start's 13/128 by three problems but trailed the shipped 60-step solution's 17/128 by one.

Observable agent behavior is clean, and no final problem contents were exposed, reconstructed, or carried into the patch. The main scientific weaknesses are the missing public-proxy baseline, no 204-problem confirmation or seed replication, and no final evaluation of the rejected methods. Thus the evidence supports improvement over the fixed start in this one formal run, not robust superiority to the shipped baseline or optimality of the exploration choice.

3. Task, baseline, and evaluation contract

3.1 Task and fixed boundaries

``text Starting artifact / model: pinned Qwen/Qwen2.5-Coder-1.5B-Instruct. Available training data and assets: a fixed Python projection of CodeForces reasoning solutions, 8,133 rows deterministically split into 8,005 train and 128 validation rows; public LiveCodeBench v4/v5 is available only during exploration. Agent-editable surface: solution training source, hyperparameters, schedule, engineering, and checkpoint policy; the submitted artifact is a source patch, not exploration weights. Fixed or forbidden components: starting model, corpus, final v6 rows, fresh formal start, one-GPU and wall-clock limits; formal retraining mounts neither public/final rows nor exploration outputs. Proxy evaluator: livecodebench_public_pass_at_1, maximize; public v4/v5 health split, n=64, greedy one-sample generation, 2,048 new-token cap; binomial descriptive standard error. A disjoint n=204 confirmation split exists but was not run. Final evaluator: livecodebench_v6_pass_at_1_first128, maximize; fixed hash-ordered first 128 v6 problems, greedy one-sample generation, 2,048 new-token cap; binomial descriptive standard error. Artifact contract: publish and simultaneously retain at most three valid checkpoints, then choose the best valid artifact; an official final result requires summary.json plus the corresponding .complete receipt. ``

Both evaluators execute extracted code against official tests, but they use different release splits and sample counts. Their scores are not directly comparable. The reported standard errors describe problem-level Bernoulli dispersion only; they do not cover training-seed variance or paired problem transitions.

3.2 How the baseline works

``text [fixed model, problem prompt, and one complete assistant solution] -> [chat-template encoding; prompt labels masked and assistant tokens retained] -> [the fixed assistant text supplies next-token targets] -> [cross-entropy / next-token NLL optimized with AdamW and a cosine learning rate] -> [all 1,543,714,304 model parameters change and are exported as checkpoints] ``

The baseline naively truncates a full chat at 32,768 tokens, trains in bfloat16 with per-device batch 3 and gradient accumulation 6 (effective batch 18), and uses learning rate 1e-5, warmup ratio 0.03, minimum cosine ratio 0.1, and gradient clipping 0.2. It requests 60 steps, evaluates and saves every 30, and retains at most three checkpoints. Validation NLL is diagnostic rather than the code-execution objective. The agent diagnosed long rationales as the concrete bottleneck: they make 32K full-model training slow while evaluation allows only 2,048 generated tokens. This motivated both target compression and longer scheduling experiments.

4. Four-hour exploration and decision process

The run lasted from about 09:44 to 13:05 UTC on 2026-08-10. The early phase inspected the task, corpus, baseline, and evaluators and measured 32K speed. The middle phase trained and scored the target transformations and context-length alternatives. The final phase established a faithful 32K control, tested learning-rate scheduling, restored a clean formal code path, validated it, and submitted with about 2,322 seconds left—too little for another approximately 50-minute 32K train-plus-evaluate cycle.

U-01 - Long-sequence diagnosis and interrupted control

Motivation and hypothesis. The agent wanted a faithful 32K/60-step control and a direct account of where baseline time was spent. Corpus inspection found very long assistant responses and 362 rows marked as ending for length, supporting its hypothesis that rationale tokens consumed disproportionate compute.

Concrete change and experimental setup. It launched the unchanged 32K baseline while concurrently running a CPU tokenization statistics scan.

Observed result. Preprocessing took about 4.5 minutes and the first training step took about 87 seconds under contention. The agent stopped the scan and then the training process with exit code 130 before any checkpoint, summary, or proxy score existed because it projected roughly 86 minutes to completion.

Agent interpretation. It recognized that the concurrent scan worsened the first step but concluded that faithful 32K training remained expensive even without that contention, so shorter targets were worth testing first.

Report assessment and confounds. This was an interrupted and contaminated timing probe, not a performance baseline. A public-proxy baseline is therefore not available. The run still supplied a useful engineering signal about sequence cost.

Decision and consequence. The agent abandoned this control and moved to target compression.