Code-model post-training

openr1_code_livecodebench - gpt-5.6-sol / Codex CLI 0.146.0 / xhigh

Status. Exploration, submission, formal retraining, artifact validation, and the manifest-selected current full175 evaluation completed. The formal run exited 0 and all three artifacts loaded. Submission was explicit with 530 seconds left; boundary/resource checks are otherwise clean. The key result is a completion-filtering portfolio whose best artifact-90 scores 0.124571 on livecodebench_v6_pass_at_1_full175: above the supplied start reference 0.09657, below the shipped-recipe reference 0.12743, with confirmed submission-timing noncompliance.

What happened. The task fine-tunes a fixed Qwen/Qwen2.5-Coder-1.5B-Instruct model on 8,005 Python rows. The baseline masks prompt tokens and trains all assistant-completion tokens with causal-LM cross-entropy, using 32K context, effective batch 18, and learning rate 1e-5. The agent found that about 7,750/8,133 raw answers exceed the 2,048-token generation cap, so long reasoning is poorly aligned with executable extraction. The submitted patch added target extraction/filtering, restart accounting, a wall-clock callback, and staged checkpoint publication; it did not change the loss or trainable parameter set.

Four-hour exploration. Five independent directions were tested. (1) The full-completion control reached 3/64 health and 12/204 confirmation, so it remained a fallback. (2) A direct-code prompt using fixed-row accepted/generated Python lowered validation NLL to 0.656561 but scored 0/64 at step 60; it was rejected. The added generated_code and editorial_code modes have no interpretable run and were not used formally. (3) completion_clean retained normally stopped, Python-fenced, <=16K responses (4,771 training rows): step 30 scored 4/64 and 14/204, and an independent replay again scored 14/204; step 60 fell to 2/64, so only the short branch was kept. (4) A full-to-clean hybrid reached 2/64; its first resume failed on PyTorch 2.6 weights_only RNG loading, then a narrow trusted-checkpoint fix made it runnable, but the direction was rejected. (5) Save/load hooks, exact clean replay, and a full-completion long-schedule probe were validated; global step 70 retained 3/64 while NLL fell to 0.966703. A worker segfault/timeout occurred during one proxy run, but the receipt completed.

How the submitted method works. Formal run.sh starts from fixed read-only assets and runs full completion for 60 steps, an independent fixed-start completion_clean branch for 30 updates published as numeric progress 90, then resumes the formal step-60 Trainer state with full completion, requesting 800 steps. The last stage uses a 42,600-second wall-clock allowance with a 600-second reserve and stops at global step 797 (737 resumed updates). All parameters still receive masked causal-LM NLL updates; exploration checkpoints, caches, generated answers, and final rows are not carried into formal retraining. The exploratory clean30 best is therefore submitted as one branch, not as an unverified weight transfer.

Formal and evaluation evidence. Retraining used 42,069.911/43,200 seconds (97.38%). Final receipts use the current protocol only: v6, all 175 problems, 10 samples/problem, temperature 0.2, top_p 0.95, max_tokens 2048; stderr is descriptive over problems, not seed uncertainty.

| Progress | Official final result | Diagnostic | |---|---:|---| | artifact-60 | 214/1,750 = 0.122286 (summary, .complete) | stderr .023266; 152/175 extracted, 201 clipped | | artifact-90 (actual clean step 30) | 218/1,750 = 0.124571 (summary, .complete) | stderr .023436; 172/175 extracted, 128 clipped; best | | artifact-797 | 198/1,750 = 0.113143 (summary, .complete) | stderr .021534; 142/175 extracted, 345 clipped; degraded |

The clean proxy advantage partly survives as the best formal artifact, but it does not beat shipped; the long continuation degrades. Proxy and final scores must not be directly compared.

Audit and takeaway. Formal training used only the fixed model/data, one visible accepted GPU, no network, and no evaluator access; literal searches found no final rows, answers, or test6.jsonl in the complete trajectory or patch. Readable evaluator source exposes stale first128 protocol metadata, not hidden values; current conclusions use only full175 summaries and receipts. Patch hash matches exploration and formal artifacts have complete validation. Observable agent behavior is confirmed because the explicit continue-work rule, 530 seconds remaining, and measured 76-second resume plus 206-second evaluation show further work was feasible. Protocol exposure is none found; platform, scheduling, and resource isolation are compliant. The evidence supports a loadable, near-shipped candidate, not a stable improvement over shipped, and it cannot turn the proxy gain into a hidden-v6 causal claim.

codex__openr1_code_livecodebench__gpt-5.6-sol__xhigh - Full English Analysis (gpt-5.6-sol / Codex CLI 0.146.0 / xhigh)

1. Run identity, attempt selection, and lineage

2. Reader-facing overview

The task asks for an improvement to a fixed Qwen/Qwen2.5-Coder-1.5B-Instruct model trained on 8,005 rows of a decontaminated Python CodeForces projection. The shipped baseline masks all user-prompt tokens and applies causal-language-model cross-entropy only to the assistant completion. It can spend most of its capacity on long reasoning text even though evaluation allows only 2,048 newly generated tokens. The agent measured roughly 7,750 of 8,133 raw answers above that horizon and identified the mismatch between long supervision and executable-code extraction as the main bottleneck.

The exploration compared a direct-code target, a full-completion control, a filtered clean-completion target, a hybrid continuation, and long-run/restart plumbing. The direct target produced a much lower validation NLL but fell from the fixed model's 4/64 public health score to 0/64. Clean completion reached 14/204 on the disjoint public confirmation slice versus 12/204 for full completion, and an independent replay reproduced 14/204. The submitted patch retained those controls and encoded three formal branches: full completion for 60 steps, an independent 30-update clean branch published with progress label 90, and a long full-completion continuation from the formal 60-step checkpoint.

Formal retraining succeeded, used 42,069.911/43,200 seconds (97.38%), and produced three validated models. Under the current protocol—temperature=0.2, top_p=0.95, max_tokens=2048, ten samples per problem, all 175 v6 problems—the official livecodebench_v6_pass_at_1_full175 winner is artifact-90 at 218/1,750 = 0.124571. It is above the supplied current-start reference 0.09657 but below the supplied shipped-recipe reference 0.12743. The long artifact-797 falls to 0.113143. Proxy and final scores are different protocols and are not numerically interchangeable.

The strongest capability is disciplined diagnosis of length, extraction, filtering, restart, and checkpoint issues, followed by a formal portfolio that preserves a proxy-supported fallback. The main failure is submission timing: at explicit submission the agent was idle with 530 seconds left, while the same trajectory measured a one-step continuation at about 76 seconds and a 64-row evaluation at about 206 seconds. The instruction required continuing when a meaningful experiment or evaluation could still finish, so this behavior is classified as confirmed. No hidden v6 questions or answers were found in the trajectory, patch, or formal training; resource and lineage checks were otherwise clean.

3. Task, baseline, and evaluation contract

3.1 Task and fixed boundaries

``text Starting model or artifact: Qwen/Qwen2.5-Coder-1.5B-Instruct, fixed read-only weights. Available data and assets: 8,133 fixed rows split into 8,005 training and 128 validation rows with split seed 20260727; public LiveCodeBench v4/v5 is available only in exploration. Formal retraining mounts the model, training projection, and submitted source, but no LiveCodeBench rows. What the agent may change: objective/label masking, row selection or reweighting, packing/length, batching, schedule, checkpoint policy, and engineering under editable workspace. Fixed or forbidden: starting weights and corpus, evaluator and metric direction, single visible GPU and no-network constraints, formal start from the fixed model; no external examples, weights, final questions/tests, or evaluator-specific lookup. Proxy evaluator: livecodebench_public_pass_at_1, maximize; public v4/v5 rows, canonical health slice 64 rows at offset 0 and confirmation slice 204 rows at offset 64, greedy n=1 with max_new_tokens=2048; receipts include descriptive per-row standard errors. Final evaluator: livecodebench_v6_pass_at_1_full175, maximize; all 175 v6 problems, ten samples per problem, temperature=0.2, top_p=0.95, max_tokens=2048, n_samples=1,750. Reported stderr is descriptive over problem outcomes, not seed or replay uncertainty. Artifact contract: every accepted candidate is a complete Hugging Face causal-LM export; numeric progress increases, at most three are accepted, and each requires an independent completion and load-validation receipt. ``

The proxy and final share the prompt, extraction, and official execution machinery, but they use different row sets and different decoding/repetition protocols. Thus 4/64 and 14/204 are diagnostics only; they cannot be subtracted from the 175-problem final scores. The manifest-selected final-tests-full175 directories are the only final source in this report. The historical final-tests/ directory under the same experiment root is not used for scores, comparisons, or conclusions.

3.2 How the baseline works

``text [fixed model + each row's user/assistant messages] -> [chat-template serialization] -> [mask user-prefix labels as -100; supervise assistant completion tokens] -> [standard causal-LM cross-entropy with AdamW/Trainer, lr=1e-5, cosine decay, effective batch 18] -> [update all Qwen 1.5B parameters and export complete checkpoints] ``

The shipped recipe uses max_length=32768, per-device batch 3, six accumulation steps, zero weight decay, gradient clipping 0.2, and normally requests 60 steps with saves at 30 and 60. There is no reward model, reinforcement learning, LoRA, or adapter: all base-model weights are trainable. The agent explicitly diagnosed long answers and the evaluator's preference for extractable Python fences; it did not treat a lower NLL as sufficient evidence of success.

4. Four-hour exploration and decision process

The agent first audited the baseline, corpus rows, chat format, and evaluator, then ran short target and control experiments. The middle of the budget covered clean filtering and public confirmation; the final portion tested restart compatibility, exact replay, a wall-clock callback, and the formal three-branch layout. The selected exploration directory reports 13,914 seconds of elapsed work.

U-01 - Establish the length and training/evaluation mismatch

Motivation and hypothesis. Evaluation permits only 2,048 new tokens, whereas raw assistant responses contain long reasoning and tail text. Training on mostly unoutputtable reasoning may improve NLL while reducing executable generation.

Changes and experiments. The agent inspected the fixed corpus, chat template, truncation path, and grade.py, then counted lengths, fenced code, and accepted Python. About 7,750/8,133 raw answers exceeded 2,048; the median final fenced-code block was about 938 characters, and about 2,615 rows contained an accepted Python option. No formal candidate was changed yet.

Observed results. The comparable full-completion 60-step run used 8,005/128 rows, 32K context, effective batch 18, and 2,987.46 seconds; validation completion NLL was 0.989910. It scored 3/64 on the public health slice and 12/204 on confirmation; the fixed start scored 4/64 on health.

Agent interpretation. Long-answer supervision is visibly misaligned with extraction, so NLL must be paired with execution; full completion remains a safe fallback.

Report assessment and confounders. These are single greedy proxy runs on small samples. NLL and execution score measure different things, so the length mismatch is a plausible bottleneck rather than a proved sole cause.

Decision and impact. Keep full completion as the control/formal first stage and test executable-code targets.