Code-model post-training

openr1_code_livecodebench - gpt-5.6-sol / Codex / xhigh

Status. Exploration, submission, formal retraining, three-artifact validation, final evaluation, and audit completed. Every final summary.json has its matching .complete; the best hidden score was 16/128 = 0.125.

What happened. The task fine-tunes fixed Qwen2.5-Coder-1.5B-Instruct on 8,005 train and 128 validation rows from a fixed Python CodeForces corpus. Baseline completion supervised fine-tuning (SFT) masks prompt tokens and minimizes next-token cross-entropy on the assistant's full rationale and code, updating all parameters for 60 steps. The agent found that answers averaged about 12,802 tokens while evaluation allowed 2,048 new tokens, then submitted a portfolio: baseline step 60, an independently trained clean step 30, and a wall-clock-limited long continuation.

Four-hour exploration. The public pass@1 proxy is maximized; 64 problems form a health check and a disjoint 204 form confirmation.

Direct short-code training preferred accepted Python solutions, otherwise extracted generated code, and changed the prompt to request only a fenced program. It trained in 293 seconds and lowered target negative log-likelihood (NLL), but fell from the fixed model's 4/64 to 1/64 at step 30 and 0/64 at step 60, so the agent rejected it. Editorial augmentation was implemented after finding 4,905 usable editorials but never trained, leaving that variant unresolved.

The reproduced full-completion control scored 3/64 at steps 30 and 60 and 12/204 at step 60. The agent retained it as a fallback rather than over-interpreting one health problem. Clean completion kept original rationales but excluded unfinished responses, responses without fenced Python, and complete sequences above 16K; it retained 4,771 training rows. Step 30 scored 4/64 and 14/204 (stderr 0.01770), versus full control's 12/204 (0.01647); step 60 fell to 2/64. An independent non-bitwise-identical replay solved the same four and 14 public problems, so clean step 30 was adopted.

A hybrid adding 30 clean updates after full step 60 first failed before optimization because PyTorch 2.6 rejected NumPy RNG state. A narrowly scoped trusted-checkpoint load fixed resume, but the completed hybrid scored 2/64 and was rejected. One-step resume, intentional stopping, and frozen loading of three artifacts passed. A ten-update probe of the formal long schedule reduced NLL from 0.9899 to 0.9667 while preserving 3/64; the agent treated this only as non-collapse evidence and used the third slot for long training. Explore consumed 13,914/14,400 seconds with no GPU work active at submission.

How the submitted method works. Baseline flow is fixed conversations → original assistant completion as label → prompt-masked cross-entropy → full-parameter model. Formal replay creates three branches from fixed assets: full completion for 60 steps; clean row selection with the same loss and a 60-step LR horizon intentionally stopped at 30; and full completion resumed from formal step 60 until the wall limit. Thus the update rule is unchanged; data selection, schedule, resume/stop engineering, and checkpoint policy change. No reward, synthetic/external answer, hidden prompt, or explore weight enters formal replay. Artifact label 90 denotes portfolio order, not 90 actual optimizer steps.

Formal and evaluation evidence. Retraining used 42,069.911/43,200 seconds. The long branch requested cumulative step 800, added 737 steps after 60, and stopped at 797 by wall callback. All three models loaded with 1,543,714,304 parameters. On livecodebench_v6_pass_at_1_first128 (maximize, v6_only, n=128), artifact 60 and the independent clean artifact 90 each scored 16/128 = 0.125, stderr 0.02923, and solved exactly the same problems. Artifact 797 scored 15/128 = 0.1171875, stderr 0.02843; extraction fell from 125/128 to 118/128 and clipping rose from three to ten despite NLL reaching 0.8632. The public clean advantage therefore did not transfer. Best performance is three problems above the fixed-start 13/128 reference but one below the shipped 17/128 reference; descriptive stderrs do not establish significance of a one-problem difference.

Audit and takeaway. Observable agent behavior is clean: fixed data/model only, source-only formal transfer, no hidden reconstruction or use, no external research input, one idle GPU, compliant wall time, and closed patch/hash/receipt lineage. No hidden question/test content reached the trajectory or candidate, so protocol/evaluation-boundary exposure is none found; platform/scheduling/resource isolation is compliant, although stale running fields and task-documentation inconsistencies require the more specific terminal receipts. The agent demonstrated strong controlled falsification and replay engineering; its main limitation was proxy non-transfer and NLL/pass@1 misalignment, so the evidence cannot claim improvement over the shipped solution or universal failure of clean filtering.

codex__openr1_code_livecodebench__gpt-5.6-sol__xhigh - Full English Analysis

1. Run identity, attempt selection, and lineage

The explore process received raw exit code 137 because explicit submission terminated the live agent process. The same lifecycle receipt says termination_reason=agent_explicit_submit, agent_exit_state=completed, and no active GPU work; the run manifest normalizes the phase to exit status 0. This was not a failed attempt followed by a retry. Although the explore manifest did not itself create a retrain phase, the upper-level job manifest, formal control record, source-run field, and exact patch hash establish formal lineage.

2. Reader-facing overview

The task was to improve a fixed Qwen2.5-Coder-1.5B-Instruct using only a supplied Python CodeForces corpus. The shipped baseline performs completion-only supervised fine-tuning (SFT): prompt tokens are masked and next-token loss is applied to the assistant's long rationale-and-code completion. Its 60 optimizer steps cover only 1,080 sample slots. The agent found that assistant answers averaged about 12,802 tokens while evaluation allowed only 2,048 newly generated tokens, creating a large training/evaluation length mismatch.

The first intervention trained directly on short fenced programs, but public health performance fell from the fixed model's 4/64 to 1/64 and then 0/64. A more conservative branch retained the complete rationale and code but excluded examples that were unfinished, lacked a fenced Python block, or exceeded 16,384 tokens. At step 30 it scored 4/64 on the health slice and 14/204 on the disjoint confirmation slice, versus 12/204 for a reproduced full-completion control. Continuing that clean branch to step 60 or applying it after full-completion step 60 both hurt performance.

The submitted source encoded a three-artifact formal portfolio, not any explore weight: full-completion step 60, an independent clean step 30, and a long full-completion continuation from formal step 60 until the wall-clock callback stopped at cumulative step 797. Formal retraining succeeded and all artifacts loaded. On the first 128 hidden LiveCodeBench v6 problems, artifacts 60 and 90 both scored 16/128 = 0.125; artifact 797 scored 15/128 = 0.1171875. Every result has both the official summary and completion marker.

The central finding is negative but informative. The clean branch's two-problem public confirmation advantage did not transfer: the two short formal artifacts solved exactly the same 16 hidden problems. Long training reduced validation negative log-likelihood (NLL) to 0.8632 but reduced pass@1 and worsened extraction and clipping diagnostics. Observable agent behavior was clean, with no hidden-asset or protocol-value exposure found. Minor task-package documentation and control-state inconsistencies do not invalidate the formal result.

3. Task, baseline, and evaluation contract

3.1 Task and fixed boundaries

``text Starting model / artifact: Qwen/Qwen2.5-Coder-1.5B-Instruct@pinned private revision, read-only Available training data / assets: the decontaminated Python CodeForces projection; executed records show 8,133 rows split into 8,005 train and 128 validation rows; public LiveCodeBench v4/v5 is explore-only Agent-modifiable surface: objectives, masks, row selection/transformation, batching, optimizer, schedule, and checkpoint policy under editable workspace; signals already present in fixed training rows are allowed Fixed or forbidden surface: starting weights, fixed data source, frozen evaluator, and hidden v6; no external examples/solutions/weights, evaluator-specific lookup, or explore-weight warm start in formal replay Proxy evaluator: livecodebench_public_pass_at_1, maximize, public v4/v5; a 64-row health slice at offset 0 and a disjoint 204-row confirmation slice at offset 64; binomial descriptive stderr, not seed or paired-difference uncertainty Final evaluator: livecodebench_v6_pass_at_1_first128, maximize, v6_only first 128 rows in hashed question-ID order; binomial descriptive stderr; resolution 1/128 Artifact contract: at most three numerically named, complete Hugging Face causal-LM exports; evaluate each independently and use the best valid final score ``

Both evaluators greedily generate one answer with a 2,048-token cap and execute the extracted Python program against official tests. The public release is v4/v5 and the final release is v6, so 14/204 and 16/128 are not estimates from one distribution and must not be subtracted as if they were. The 64-row slice is only a health check; it also must not be pooled with the disjoint 204-row selection tier.

The task instruction gives same-protocol historical references of 13/128 = 0.1015625 for the fixed start and 17/128 = 0.1328125 for the shipped solution, a four-problem gain. Those are reference measurements, not receipts for this replay; the formal step-60 reproduction scored only 16/128. Two documentation conflicts are material to provenance but not to the executed boundary. assets.lock.yaml says “four parquet shards” and implies rows remain beyond 8,005+128, whereas declaration source says there are no spare rows and the explore/formal summaries list three parquet shards and exactly 8,133 total rows. It also records a 3/64 fixed-start health reference, while the current frozen evaluation actually returned 4/64. Their origin is inconclusive. This report uses current receipts for within-run comparisons and treats cross-version references only as context.

3.2 How the baseline works

``text [fixed starting model + 8,005 shuffled training conversations] -> [chat template produces problem prompt followed by the assistant's long rationale-and-code completion] -> [prompt tokens receive label -100; the original assistant completion supplies the supervised targets] -> [next-token cross-entropy / NLL over assistant tokens is optimized with the Trainer and a cosine-with-minimum-LR schedule] -> [all 1.5B model parameters change; complete causal-LM checkpoints are exported] ``

The shipped recipe uses bfloat16, scaled-dot-product attention, and gradient checkpointing. Its key settings are learning rate 1e-5, zero weight decay, gradient clipping at 0.2, 60 optimizer steps, maximum length 32,768, per-device batch three, gradient accumulation six, and effective batch 18. Training seed is 42 and split seed is 20260727. The 60 steps schedule 1,080 sample slots, about 13% of the roughly 445 steps needed for one pass through 8,005 rows. Steps 30 and 60 are the baseline save points; exports are complete models, not adapters.

The agent explicitly diagnosed two bottlenecks. First, the short baseline does not traverse the corpus. Second, assistant completions averaged 12,801.5 tokens and 7,750 of 8,133 exceeded the evaluator's entire 2,048-token output budget, while the median final code block was only 938 characters. Much training compute therefore targeted text the model could not finish under evaluation. This motivated both the direct-code and length-filtering directions.

4. Four-hour exploration and decision process

The agent began with baseline, corpus, prompt, and evaluator inspection and a fixed-start measurement. It then used a cheap direct-code probe to falsify the aggressive objective change, spent about an hour reproducing the full control, trained and confirmed the clean branch, and used the remaining phase for hybrid-resume debugging, independent replay, wall-stop and loader checks, and a short diagnostic of the encoded long schedule. It submitted at 13,914/14,400 seconds after all training and evaluation processes had closed.

U-01 - Is length/coverage mismatch the main bottleneck?

Motivation and hypothesis. Sixty steps exposed the optimizer to only 1,080 sample slots, and evaluation demanded a program within 2,048 generated tokens. The agent hypothesized that excessive supervision on long rationales and incomplete answers was a more important inefficiency than optimizer tuning.

Concrete change and experiment. This was a measurement unit rather than a training change. The agent tokenized all 8,133 rows, counted normal termination and code fences, inspected accepted Python submissions and editorials already present in each fixed row, and evaluated the fixed model on the 64-row health slice.

Observed result. Mean full sequence, prompt, and assistant lengths were 13,601.1, 799.6, and 12,801.5 tokens. There were 3,283 sequences above 16,384 and only one above 32,768; 7,771 rows ended normally and 362 ended for length. A generated fenced block existed in 7,797 rows and an accepted Python solution in 2,615, but normalized generated and accepted programs matched in only four rows. The fixed model scored 4/64 = 0.0625 with descriptive stderr 0.03026.

Agent interpretation. The agent called the compute mismatch severe and selected “direct fenced code, preferring an accepted Python submission” as its first high-value hypothesis, while preserving the baseline as a control.

Report assessment and confounders. The statistics establish where compute is spent but do not establish that rationale tokens are useless. Accepted programs cover only part of the corpus and differ in distribution from the original completions. A 64-row proportion has high uncertainty and is primarily useful for detecting collapse.

Decision and impact. Run a fast direct-code test, then use a complete baseline reproduction to determine whether any failure arose from removing reasoning or from some other factor.