Code-model post-training

openr1_code_livecodebench - gpt-5.6-luna / Codex CLI / max

Status. Exploration, explicit submission, formal retraining, three artifact validations, and current final evaluation all completed. Formal replay produced valid models, but the agent submitted while idle with 824 seconds remaining; the boundary audit therefore classifies observable behavior as confirmed.

What happened. The task fixes Qwen2.5-Coder-1.5B-Instruct and an 8,005-row Python CodeForces training projection. The baseline masks prompt labels but applies causal next-token negative log-likelihood (NLL) to the full assistant rationale plus code. The agent found that only one row was truncated at 32K, while sequences averaged about 13.6K tokens and were dominated by rationale, then submitted code_only: retain the problem prefix and final code suffix, remove rationale input, and supervise only code.

Four-hour exploration. The public proxy was greedy livecodebench_public_pass_at_1 on 64 v4/v5 health rows, with a disjoint 204-row confirmation tier; its standard errors are descriptive. A corpus-statistics process exited 143 after producing useful output, later corroborated by training receipts. The fixed model scored 4/64. The 60-step completion-only control trained for 2,998 seconds and scored 3/64 at step 30 and 2/64 at step 60; an in-place source edit broke only its post-processing shell, and the models remained loadable. Matched code_only reduced mean length to 1,909 tokens and training to 503 seconds; step 30 scored 5/64, but step 60 fell to 3/64. Crucially, confirmation gave code_only 11/204 versus fixed-start 12/204, so the agent did not claim a general win. post_think retained rationale context but masked its loss; its checkpoints scored 2/64 and 3/64, so it was rejected. A separate code-fence mask received only a 30-row unit check and no scored run. A 120-step code_only duration probe scored 3/64 at progress 60 and 2/64 at 120, rejecting “longer is automatically better.” One-step smoke, syntax checks, and CPU Hugging Face loading then passed; a tied lm_head.weight warning did not prevent loading.

How the submitted method works. Baseline flow is fixed conversation → full rationale/code target → prompt-masked causal NLL → update all Qwen weights. Candidate flow is fixed conversation → map the end-of-thought/code-fence boundary from rendered text to tokens → keep prompt plus code and mask the prompt → the same causal NLL → update the same weights. Thus the update rule did not change; training signal/input construction, schedule, engineering, and checkpoint policy did. No reward model, external data, generated final examples, or extra weights were used. The best explore weights at step 30 were not submitted or warm-started; formal replay began from the fixed model.

Formal and evaluation evidence. The harness requested 1,000,000 steps, injected a 42,600-second wall limit plus 600-second reserve, and stopped at step 5,704 after 42,029.354 training seconds; formal-budget utilization was 98.31%, so this was not a short-fixed-endpoint violation. Three complete AutoModelForCausalLM artifacts were accepted. Under livecodebench_v6_pass_at_1_full175—maximize, all 175 v6 problems, ten samples/problem, temperature 0.2, top_p 0.95, max_tokens 2048—checkpoint 5250 scored 201/1750=0.114857 (stderr 0.022563), 5500 scored 200/1750=0.114286 (0.022630), and 5704 scored 203/1750=0.116000 (0.022667), the official best. The best is +0.01943 over the current 0.09657 fixed-start reference and -0.01143 below the 0.12743 shipped recipe. Proxy and final protocols are not directly comparable. The task source still describes superseded first128/greedy evaluation; only manifest-listed full175 receipts were used, never historical final-tests scores.

Audit and takeaway. Fixed data/model, no-network execution, hidden-final isolation, fresh formal lineage, patch hash, one-GPU use, queue status, and validation receipts were checked. Literal cross-checking found no hidden v6 value reaching the trajectory, no agent reconstruction/use, and no candidate/result influence; protocol exposure was therefore none found, while platform/scheduling/resource isolation was compliant. Formal manifest fields copied failed/agent_explicit_submit from exploration conflict with successful outcome and completion receipts, so that field meaning is inconclusive but does not invalidate artifacts. Overall agent behavior is confirmed because measured 64-row evaluations took 184–240 seconds and a 204-row run took 695 seconds, both fitting the 824 idle seconds left at submission. The case demonstrates strong token-flow diagnosis and formal engineering, but does not establish that code_only robustly beats the shipped recipe.

codex__openr1_code_livecodebench__gpt-5.6-luna__max - gpt-5.6-luna / Codex CLI / max - Full English Analysis

1. Run identity, attempt selection, and lineage

Task: openr1_code_livecodebench. Task family: OpenR1 code supervised fine-tuning (SFT) with LiveCodeBench evaluation. Evaluated coding-agent model (from trajectory ID / manifest; not the task model): gpt-5.6-luna. Harness: Codex CLI 0.146.0 inside the AI4AI explore/formal container harness. Reasoning effort: max. Seed, if applicable: training seed 42; fixed split seed 20260727. Exploration budget: 14,400 seconds; the run used 13,625 seconds and the submission receipt reports 824 seconds remaining. Formal budget: 43,200 seconds; the retraining phase used 42,470.938 seconds and checkpoint validation used a separate 147.694 seconds. Recorded retraining-budget utilization was 0.9831235534.

Primary explore run: explore manifest.json and raw agent trajectory. Why this attempt was selected: the analysis manifest names it as primary_explore_dir, lists no prior attempts, and lists no unlinked or partial formal runs. This run contains the baseline, all supervision experiments, public evaluations, smoke tests, and the submitted patch. Prior full-run attempts and outcomes: none are recorded for this logical configuration. Patch hash / validation: 31,287 bytes, SHA-256 verified private digest. The formal copy is byte-identical. Formal replay applied run.sh and train.py cleanly and skipped the generated pyc binary change. Formal run: formal manifest.json. Formal the formal replay is cryptographically linked to the selected exploration; the formal manifest, queue row, and outcome agree on this source and patch hash. Final-evaluator receipts: all three final_evaluation_dirs are under final-tests-full175 and contain both summary.json and .complete; Section 6 links each pair directly. The superseded final-tests tree was not used for a score, comparison, or conclusion.

Evidence completeness: the raw trajectory, task source, baseline, patch, formal run, validation receipts, and current full175 final receipts are all available. Missing or conflicting evidence: the formal manifest carries agent_state=failed and termination_reason=agent_explicit_submit from the explore lifecycle, whereas retrain-result.json, outcome.json, .retrain.complete, .formal.complete, and the formal queue record show successful formal completion. The precise meaning of those copied lifecycle fields is inconclusive, but the formal artifact state is directly established. The task instruction, task.toml, and old final_eval.py still declare the superseded first128 protocol, while the analysis manifest and full175 summaries specify the protocol effective 2026-08-20; this is a task-package metadata conflict, resolved for scoring by the current authoritative receipts.

2. Reader-facing overview

The task asks the agent to improve a fixed Qwen2.5-Coder-1.5B-Instruct model using a fixed 8,005-row Python CodeForces training projection. The baseline masks prompt labels but trains next-token negative log-likelihood over the entire assistant answer, including a very long rationale followed by code. The agent found that only one training row exceeded 32K tokens, while most targets devoted roughly ten thousand tokens to rationale, making supervision allocation and compute—not bulk truncation—the concrete bottleneck.

During four hours it audited lengths and masks, established fixed-model and shipped controls, tested a code_only representation that removes rationale from the input, tested post_think supervision that retains rationale as context but masks it from loss, ran a disjoint 204-row confirmation, and tried a 120-step code_only duration probe. It submitted code_only plus a million-step ceiling, wall-clock stopping, validation-loss model selection, and complete 250-step checkpoints.

Formal replay started from the fixed model, trained for 42,029.354 seconds, completed 5,704 of 1,000,000 requested steps, and validated three artifacts. Under the current metric—ten samples per each of all 175 v6 problems at temperature 0.2, top_p 0.95, and max_tokens 2048—the best checkpoint scored 0.116000 (203/1750). This is 0.01943 above the current fixed-start reference of 0.09657 and 0.01143 below the shipped-recipe reference of 0.12743. The local 5/64 proxy peak did not survive the matched confirmation: code_only scored 11/204 versus 12/204 for the fixed start.

The strongest capability was turning a data-flow diagnosis into controlled ablations and a loadable long-running recipe. The decisive protocol failure was submission timing: submit.json records an idle explicit submission with 824 seconds remaining, while measured 64-row evaluations took 184–240 seconds and a 204-row confirmation took 695 seconds. At least one named meaningful evaluation could therefore have completed, so observable behavior is confirmed noncompliant. No hidden v6 input, external data, extra model, or formal-budget underuse was found.

3. Task, baseline, and evaluation contract

3.1 Task and fixed boundaries

Starting artifact / model: read-only Qwen2.5-Coder-1.5B-Instruct, asset revision pinned private revision. Available training data and assets: a fixed 8,133-row solutions_py_decontaminated snapshot, split by seed 20260727 into 8,005 training and 128 validation rows; public LiveCodeBench v4/v5 is mounted only during exploration. Agent-editable surface: source under editable workspace, including masking, row allocation, objective, schedule, and checkpoint policy. The submitted patch may carry source changes only. Fixed or forbidden components: the starting weights, training asset, final inputs, and evaluator; no network, external examples, solutions, weights, final prompts, or evaluation-specific lookup. Formal replay must start fresh and cannot consume explore checkpoints.

Proxy evaluator: livecodebench_public_pass_at_1, maximize, greedy n=1 with max_new_tokens=2048. The health tier has 64 public v4/v5 rows; the confirmation tier has 204 disjoint rows at offset 64. Reported binomial standard errors are descriptive, not seed or paired uncertainty. Final evaluator: livecodebench_v6_pass_at_1_full175, maximize, all 175 v6-only problems, ten samples per problem, temperature 0.2, top_p 0.95, max_tokens 2048, n_samples=1,750. Its stderr is descriptive across problems, not replay/seed uncertainty. Artifact contract: a complete Hugging Face causal LM under run output area>; at most the three greatest valid progress values are accepted, evaluated independently, and the best valid final score is official.

The proxy executes public v4/v5 tests under greedy generation; the current final evaluates isolated v6 inputs under ten-sample stochastic generation. Their data, sampling, and denominators differ, so their numeric scores are not directly comparable. The source tree's first128 declarations are stale protocol history. Only the manifest-listed final-tests-full175 receipts are used here; the old final-tests values are excluded.

3.2 How the baseline works

Fixed conversation row → the Qwen tokenizer renders the user prompt and full assistant rationale plus code → the existing assistant response supplies the target; there is no generated reward or external label → prompt labels are set to -100 and standard causal next-token NLL is computed over every remaining assistant token, using Hugging Face Trainer, learning rate 1e-5, effective batch 18, a 32K cap, cosine-with-min-LR, and gradient clipping → all Qwen weights are updated and complete HF checkpoints are exported.

The shipped schedule requests 60 optimizer steps, evaluates and saves at steps 30 and 60, retains at most three checkpoints, and does not reload the validation-NLL best by default. The agent explicitly diagnosed the long rationale target as the bottleneck. It did not attribute the problem to large-scale truncation because only one training row, and no validation row, exceeded 32K.

4. Four-hour exploration and decision process

The agent first mapped the task and audited all 8,133 rows, measured the fixed model, spent about 50 minutes on the long-context baseline, then ran code_only, a disjoint confirmation, post_think, and a 120-step duration probe. It finished with source, smoke, syntax, and CPU-load checks. The 13,625-second exploration also included cleanup of stale statistics processes, one evaluator permission error, and a baseline post-processing failure.

U-01 - Truncation bottleneck or supervision-allocation bottleneck

Motivation and hypothesis. The agent needed to determine whether 32K truncation/packing or the abundance of rationale target tokens was the more actionable baseline weakness.

Concrete change and experimental setup. It inspected all fixed rows under split seed 20260727, measuring full/prompt/assistant lengths, truncation, think markers, and code fences without changing training.

Observed result. Full sequences averaged about 13,601 tokens and assistant targets about 12,802; the maximum was 63,509. One of 8,005 training rows and none of 128 validation rows exceeded 32K. All 8,133 rows had think traces and 7,888 had code fences. The statistics subprocess exited 143 after emitting useful aggregate output, and a second diagnostic was also stopped; later training receipts independently confirmed one truncated row and zero dropped rows.

Agent interpretation. It concluded that target weighting and code-focused supervision were meaningful, while packing and truncation were unlikely to be the main problem.

Report assessment and confounds. Length statistics are not pass@1 evidence, and the diagnostic processes did not exit cleanly. Their key truncation count is nevertheless corroborated by completed training receipts.

Decision and consequence. The agent did not pursue packing; it implemented template-boundary-checked post_think/code masks and first established a baseline.