Code-model post-training

openr1_code_livecodebench - gpt-5.6-sol / Codex / high

Status. Exploration, explicit submission, formal retraining, frozen validation of three artifacts, three official final evaluations, and the boundary audit all completed. The best formal checkpoint was step 852 at 16/128 = 0.125 on hidden LiveCodeBench v6 first-128.

What happened. The task fixes Qwen2.5-Coder-1.5B-Instruct and a CodeForces rationale corpus. The baseline masks prompt tokens and uses the entire existing rationale-plus-code response as the next-token cross-entropy target for full-model supervised fine-tuning. The agent diagnosed that 60 steps schedule only 1,080 sample slots for 8,005 rows, while median responses are roughly 46,902 characters. It submitted the same training signal with a lower learning rate, a longer wall-clock schedule, and progress-diverse exports; the result beat the task-provided fixed-start reference of 13/128 but not the shipped baseline's 17/128.

Four-hour exploration. The common proxy was public v4/v5 pass@1, maximized on a 64-problem health tier and a disjoint 204-problem confirmation tier; standard errors are descriptive across problems, not seed uncertainty. A completion control at step 30 took about 24:42 and scored 3/64 (stderr 0.02642) with perfect extraction; it was manually stopped after that comparable checkpoint. Code-only supervision extracted the last syntax-valid Python fence and shortened context: 60 steps took 71 seconds but scored 2/64, and 480 steps lowered validation loss further yet fell to 0/64, so the agent rejected it. A concise target retaining an original rationale prefix plus valid code scored 2/64 at both steps 240 and 480 and was also rejected. Length grouping modestly reduced step-30 time to about 22:48 but tied 3/64, so it was not adopted. A completed 60-step completion run remained 3/64; on confirmation, step 30 scored 14/204 and step 60 scored 13/204. The agent interpreted this weak nonmonotonicity as a reason to retain early and late artifacts. Two one-step smoke tests verified formal defaults and milestone copying; an initially malformed validator call was corrected successfully. It submitted with 1,383 seconds remaining and no active GPU job, judging that no new train-score loop would finish.

How the submitted method works. Baseline and formal flows are the same: fixed prompt and full original response → mask prompt labels → response-token cross-entropy → full-parameter AdamW update → full causal-model export. Formal replay used target_mode=completion; optional code-only and concise functions present in the patch were inactive. The executed changes were hyperparameter, schedule, engineering, and checkpoint-policy changes—not an objective or update-rule change: learning rate moved from \(10^{-5}\) to \(3\times10^{-6}\), warmup became ten fixed steps, 2,000 steps were requested, and checkpoints 150, 400, and the wall-clock endpoint were exported. No explored weights carried over; replay started from the fixed model and data.

Formal and evaluation evidence. Formal execution used 42,379.595 of 43,200 seconds and stopped by its wall-clock guard at 852 completed—not 2,000 requested—steps. All three artifacts loaded as 1,543,714,304-parameter models. Each final result has both summary.json and .complete: step 150 scored 14/128 = 0.109375 (stderr 0.02759; 125/128 extracted), step 400 scored 13/128 = 0.1015625 (0.02670; 124/128), and step 852 scored 16/128 = 0.125 (0.02923; 123/128). Correctness was nonmonotonic while outputs lengthened and extraction declined. The public depth proxy slightly favored the early model, whereas hidden v6 favored the endpoint; proxy evidence supported preserving full-response supervision but did not predict the best depth. Rejected targets were not final-scored and cannot be ranked on v6.

Audit and takeaway. Explore mounted fixed model/data plus public v4/v5; formal mounted only fixed start/data and the matching-hash patch; hidden v6 appeared only during scoring. No network input, external example or weight, final-test query, evaluator modification, or explore-checkpoint continuation was found. Explore and formal each exposed one GPU, the assigned formal GPU was idle at dispatch, and wall-clock limits were respected. Stale per-attempt status: running fields and conflicting low-level explore exit codes are control-record defects, but job status, queue, manifests, hashes, outcomes, and completion receipts agree. Observable agent behavior is clean; no protocol-boundary exposure was found; platform, scheduling, and resource isolation were compliant. The agent showed strong negative-ablation and artifact-risk management, but one seed, weak proxy selection, and failure to beat the shipped baseline remain the main limits.

codex__openr1_code_livecodebench__gpt-5.6-sol__high - Full English Analysis

1. Run identity, attempt selection, and lineage

The explore lifecycle says agent_exit_state: completed, termination_reason: agent_explicit_submit, and raw agent exit 137. The per-attempt control file instead records exit 127 and retains stale status: running. The job-level status and queue are terminal_behavior, while the nonempty patch, submission receipt, lifecycle receipt, and .explore.complete agree on the submitted hash. Thus the low-level exit-code discrepancy is inconclusive, but it does not make submission ambiguous. The formal per-attempt control record has a similar stale status: running; the job status, queue, formal manifest, outcome.json, .retrain.complete, and .formal.complete all establish successful formal completion. The formal manifest's final_scoring: false marks that run as retrain-and-validation only; the later summaries and complete markers under its expected final root record the separate scoring stage, so these records do not conflict. There are no correction receipts or extra gate-only attempts for this exact configuration.

2. Reader-facing overview

The task asks an agent to improve a fixed Qwen2.5-Coder-1.5B-Instruct model using only a fixed CodeForces reasoning corpus. The shipped baseline performs completion-only supervised fine-tuning (SFT): prompt tokens are masked, and the model learns the entire existing rationale-plus-code response. Its 60 requested optimizer steps represent only 1,080 sample slots against 8,005 training rows, while the very long responses make each step expensive.

The agent reproduced a short completion control, then asked whether code-only targets, concise retained rationales, or length-grouped batches could use the budget better. Code-only and concise targets trained much faster but reached only 0–2 correct out of 64 public health problems, versus 3/64 for completion training. Length grouping gave a modest timing gain and the same 3/64. A larger 204-problem confirmation comparison gave 14/204 at step 30 and 13/204 at step 60. The submission therefore preserved the completion signal but lowered the learning rate, requested a much longer wall-clock run, and engineered early, middle, and endpoint exports. The executed objective and parameter-update rule did not change.

Formal replay completed 852 of 2,000 requested steps before its wall-clock guard stopped training. Checkpoints 150, 400, and 852 all passed frozen loading. Official LiveCodeBench v6 first-128 scores were respectively 14/128, 13/128, and 16/128; checkpoint 852 was best at 0.125 with descriptive standard error 0.02923. On the same final protocol, this is three problems above the task-provided fixed-start reference of 13/128 but one below the shipped-baseline reference of 17/128. The run therefore produced valid improvement over the fixed start, but did not improve the shipped solution.

The trajectory shows useful corpus diagnosis, negative ablation, confirmation, and artifact engineering. Its central scientific limitation is weak model-selection evidence: one training seed, coarse public proxies, and a documented distribution shift to final v6. The boundary audit found no hidden-final access, external data or weights, explore-artifact carryover, or resource violation; observable agent behavior is classified clean.

3. Task, baseline, and evaluation contract

3.1 Task and fixed boundaries

``text Starting artifact / model: Qwen/Qwen2.5-Coder-1.5B-Instruct at fixed revision pinned private revision Available training data and assets: open-r1/codeforces-cots at revision pinned private revision; 8,133 Python-projection rows deterministically split into 8,005 train and 128 validation rows; public LiveCodeBench v4/v5 is explore-only Agent-editable surface: solution training source, target transformations derived from allowed rows, row selection/reweighting/packing, hyperparameters, stopping logic, and checkpoint policy Fixed or forbidden components: starting model and data assets, formal entry point bash editable workspace, frozen evaluators, and hidden v6 data; no external examples or weights, evaluator-prompt training, network lookup, or final-test probing Proxy evaluator: livecodebench_public_pass_at_1 / maximize / public v4-v5; health_64 at offset 0, n=64, and disjoint confirmation_204 at offset 64, n=204 / descriptive binomial stderr Final evaluator: livecodebench_v6_pass_at_1_first128 / maximize / v6-only, first 128 after fixed hash ordering / n=128 / descriptive binomial stderr Artifact contract: fresh formal retraining from the fixed model and data; publish at most three checkpoint-<progress> model artifacts; frozen-load validation, then best valid result among up to three final evaluations ``

Pass@1 is the proportion of problems solved when the model greedily generates one program per prompt and the official tests execute it. Both proxy tiers use public v4/v5 and are mutually disjoint. Final scoring mounts hidden v6 only inside the scoring phase and evaluates its fixed first 128. Generation is otherwise similar—one deterministic generation, at most 2,048 new tokens, official extraction, and official execution—but the releases and problem populations differ. Proxy and final scores therefore are not directly subtractable or same-distribution replications.

Every reported standard error is descriptive across binary problem outcomes. It does not measure training-seed variance or paired model differences. The asset lock also reports zero rank correlation for the health proxy on maintainer reference models, further limiting its use for model ordering. The task deliberately discloses aggregate references under the final protocol: fixed start 13/128 (0.1015625) and shipped baseline 17/128 (0.1328125). Those aggregates reveal no final problem content and were not rerun by this trajectory.

3.2 How the baseline works

``text [fixed model, one CodeForces prompt, and its existing full rationale-plus-code response] -> [tokenizer builds a chat sequence and truncates it at 32,768 tokens] -> [prompt labels are set to -100; response tokens themselves supply the targets] -> [next-token cross-entropy, or completion negative log-likelihood, is optimized with AdamW and cosine learning-rate decay] -> [all 1,543,714,304 Qwen parameters change, and a full Hugging Face causal-language-model artifact is exported] ``

Negative log-likelihood (NLL) here measures probability assigned to held-out existing completions; lower is better for imitation, but it is not program-execution accuracy. The baseline uses seed 42, split seed 20260727, learning rate \(1\times10^{-5}\), 60 optimizer steps, per-device batch 3, gradient accumulation 6, and effective batch 18. It uses a 0.03 warmup ratio, cosine decay to a 0.1 minimum-rate fraction, zero weight decay, gradient clipping at 0.2, bfloat16 full-model training, gradient checkpointing, and scaled dot-product attention. It evaluates and saves every 30 steps and retains at most three checkpoints simultaneously.

The agent identified two linked bottlenecks. Sixty steps schedule only 1,080 sample slots, roughly 13% of the 8,005-row training set. The corpus is also unusually verbose: response characters have a median near 46,902 and a 95th percentile near 86,999; 7,889/8,133 rows contain think tags and 7,797 contain code fences, while the last code fence begins around 97% through a typical response. It also counted 7,771 normally stopped rows, 362 length-ended rows, 336 without code fences, and 22 whose last fenced program did not parse. This diagnosis motivated shorter-target experiments rather than an invented new data source.

4. Four-hour exploration and decision process

The explore run used 13,056 seconds, or 3:37:36. The opening stage established boundaries, measured the corpus, and produced a step-30 completion control. The middle stage tested code-only targets, longer exposure to that target, a concise-rationale target, and length grouping. The later stage completed a 60-step completion run, compared steps 30 and 60 on the larger confirmation tier, and then implemented and smoke-tested wall-clock stopping, milestone export, and frozen artifact loading. The agent submitted with 1,383 seconds remaining and no active GPU work.

U-01 - What actually limits the baseline?

Motivation and hypothesis. The agent suspected that the baseline both underexposed the corpus and spent most computation imitating long rationales. Before changing targets, it needed a local completion anchor and evidence that answer extraction was not the immediate failure mode.

Concrete change and experimental setup. It kept the baseline completion target, 32,768-token context, \(1\times10^{-5}\) learning rate, effective batch 18, and seed 42. Because the first optimizer step took about 86 seconds, it manually interrupted the nominal 60-step control after creating checkpoint 30 and entering step 31. It scored checkpoint 30 on the canonical 64-problem public health tier.

Observed result. Reaching the step-30 save took about 24:42. Validation completion NLL was 1.001141, and the frozen validator could load the artifact. The health result was 3/64 = 0.046875, descriptive stderr 0.02642; extraction was 64/64, mean generation length was 215.5 tokens, and no output hit the token cap.

Agent interpretation. The agent treated this as a low but usable anchor. Complete extraction suggested formatting was not the dominant problem, while step cost and low data coverage remained actionable.

Report assessment and confounds. The run did not complete its requested 60 steps, so it is not a full baseline timing result. Checkpoint 30 is nevertheless directly comparable to later step-30 ablations. One seed and three solved problems provide very coarse resolution.

Decision and consequence. It retained completion training as a fallback and tested shorter supervision targets next.