Molecular graph diffusion

digress_qm9_graph_diffusion - gpt-5.6-sol / Codex CLI 0.146.0 / medium

Status. Exploration, explicit submission, formal retraining, validation of three retained artifacts, and all three final evaluations completed with traceable lineage. The boundary audit found a confirmed early-submission breach, but no data, model, evaluator, or formal-replay contamination.

What happened. The task was to improve discrete molecular-graph diffusion on fixed hydrogen-free QM9. The baseline corrupts categorical atom and bond labels, asks a nine-layer graph Transformer to reconstruct the clean graph, and updates roughly 8.2 million parameters with node cross-entropy plus five times edge cross-entropy. The agent identified experimental overhead—one epoch took about 211 seconds but the full validation/sampling wrapper about 551—and submitted the same model and objective with a larger, cosine-decayed learning rate and cheaper training diagnostics. Best official real-test NLL fell from the shipped 69.57 to 65.52.

Four-hour exploration. The first long probe was interrupted during costly initial validation; a one-epoch baseline reached validation-alias NLL 77.88 and exported weights, but its zero-sample final diagnostic divided by zero. Batch 1,024 took 492.6 seconds per epoch and reached 79.88, refuting the throughput hypothesis, so 512 was retained. FP16 first failed on autocast-unsafe binary cross-entropy; after parameter-free reporting metrics were removed it was still slower and was stopped, while FP32 without those metrics reached 76.43. That early, non-submitted checkpoint's frozen proxy—validation-boundary validity × uniqueness × novelty, maximize, 2,000 generated molecules—was 0.41675 and took 686.75 seconds; no uncertainty was reported and sampling was effectively unseeded. Constant 4e-4 reached 71.25 in five epochs and 68.88 in twenty. TF32 changed runtime by only about 1% and did not isolate a score gain, so it was treated as neutral but safe. Constant 8e-4 was unstable and only reached 70.28 after fifteen epochs. Cosine 4e-4 to 4e-5 reached 68.87 in fifteen epochs, matching the twenty-epoch result, while 3e-4 to 3e-5 reached 70.14 and was rejected. A final one-batch smoke test exited cleanly and its 533-tensor artifact was frozen-loader valid; however, the submitted 500-epoch decay extrapolated the explored fifteen-epoch schedule.

How the submitted method works. Fixed training graphs still supply clean atom and bond targets; marginal categorical diffusion corrupts them at a random time, the same Transformer predicts clean categories, and AdamW/AMSGrad minimizes the unchanged weighted cross-entropy while updating all parameters. The patch raises the initial rate from 2e-4 to 4e-4, cosine-decays it to 4e-5 over 500 epochs, enables high matmul precision, and removes non-training metrics, training samples, sanity validation, frequent validation, and post-fit testing. This is hyperparameter, schedule, and engineering tuning: data, labels, loss, update rule, architecture, and three-artifact retention are unchanged. No generated rewards, synthetic labels, external models, reasoning traces, or exploration weights entered formal replay.

Formal and evaluation evidence. The fresh-start formal retrain phase used seed 42, batch 512, and one GPU for 41,781 seconds—96.72% of the 12-hour budget—before the injected internal timer stopped it near epoch 531; it requested 1,000 epochs rather than a short fixed endpoint. Eleven checkpoints were cumulatively published, but only progress 449, 499, and 531 were retained and all loaded successfully. Their official NLLs, minimize on the real-test split with row count and uncertainty not available, were 66.70, 67.28, and 65.52. Each also had 10,000-molecule diagnostics; the validity–uniqueness–novelty products were 0.3205, 0.3184, and 0.3148. Progress 531 was best, improving NLL by 4.05 or 5.82%, although the non-monotonic trend supports retaining alternatives. NLL improvement survived final evaluation; the candidate's frozen proxy did not receive a same-protocol repeat, and its best novelty 0.3287 was below the shipped 0.5190, so broad molecular-quality improvement is unsupported.

Audit and takeaway. Literal cross-checking first identified evaluator-only real-test inputs and their fingerprints/sizes, then searched the complete trajectory including tool returns, all commands, and the patch; no hidden value was exposed, reconstructed, used, or incorporated. Fixed source/data, evaluator integrity, external inputs, fresh formal lineage, single-idle-GPU isolation, hash, progress, and paired receipts were clean; protocol exposure was none found, and platform/scheduling/resource isolation was compliant. Observable agent behavior was confirmed because it submitted idle with 786 seconds left despite a task rule to continue meaningful work and a measured 686.75-second frozen evaluation that could have tested the actual candidate and left time to read the summary. This weakens exploration completeness, not official result validity; the case demonstrates effective profiling and recipe hardening but cannot establish statistical significance or uniform generative-quality gains.

codex__digress_qm9_graph_diffusion__gpt-5.6-sol__medium - Full English Analysis

1. Run identity, attempt selection, and lineage

The primary provenance records are the raw trajectory, explore manifest, submission receipt, lifecycle record, and explore completion receipt.

2. Reader-facing overview

The task asks the agent to improve DiGress, a discrete graph-diffusion model, on fixed hydrogen-free QM9 data. The baseline corrupts categorical atom and bond labels and trains a graph Transformer to reconstruct the clean graph with node cross-entropy plus five times edge cross-entropy. Its immediate limitation was experimental throughput: one epoch took about 211 seconds, while initial validation, final validation, and molecular sampling brought an end-to-end probe to about 551 seconds.

The agent tested batch size, mixed precision, parameter-free reporting metrics, TF32, learning-rate magnitude, and cosine decay. It retained the same model, data, and loss, raised AdamW's initial learning rate from 2e-4 to 4e-4, decayed it to 4e-5 over 500 epochs, enabled the high float32 matmul setting, and removed expensive diagnostics and post-fit sampling that do not provide training signal. A 15-epoch cosine probe reached validation NLL 68.87, essentially matching 68.88 from 20 epochs at constant 4e-4; this was hyperparameter, schedule, and engineering tuning rather than a new generation algorithm.

Formal replay trained from the fixed random start for 41,781 seconds and stopped under the injected wall-clock timer near epoch 531. All three retained artifacts validated. Their official test NLLs, where lower is better, were 66.70, 67.28, and 65.52 at progress 449, 499, and 531; progress 531 was best and improved on the shipped 69.57 result by 4.05, or 5.82%. The only frozen proxy run used an early one-epoch, non-submitted checkpoint and scored 0.41675 on 2,000 generated molecules, so the submitted candidate's proxy result was never measured under that protocol.

The agent showed effective profiling, negative-result use, and long-run recipe hardening. Its important protocol failure was submission timing: the explicit receipt shows 786 seconds left, no active work, and an idle GPU, while this trajectory had already measured the same frozen proxy evaluation at 686.75 seconds on the same model topology. Running it on the actual best candidate was a specific meaningful evaluation that could have completed with time to read the compact summary. Observable agent behavior is therefore confirmed, although the early stop did not contaminate the formal artifacts or official scores; formal budget use was compliant.

3. Task, baseline, and evaluation contract

3.1 Task and fixed boundaries

``text Starting artifact / model: DiGress graph-Transformer source at the pinned revision; formal replay starts from fresh random weights rather than exploration weights Available training data and assets: fixed qm9_no_h train/validation assets; proc_test_no_h.pt in explore/retrain is a byte-identical compatibility alias of validation Agent-editable surface: workspace training source, hyperparameters, schedule, and engineering implementation, producing model.ckpt Fixed or forbidden components: real test tensor/raw test files are final-evaluator-only; evaluator modification, external data/models, network-sourced inputs, explore-weight carryover, and more than one GPU are forbidden Proxy evaluator: validity_uniqueness_novelty product, maximize, validation boundary, 2,000 generated molecules, no stderr/CI and effectively unseeded sampling; it also reports validation NLL, minimize Final evaluator: upstream NLL, minimize, hidden real-test split, row count not available; it separately reports diagnostics over 10,000 generated molecules, with no stderr/CI and effectively unseeded sampling Artifact contract: a complete model.ckpt loadable by the frozen loader; retain at most three numerically named formal checkpoints concurrently, validate and score every retained checkpoint ``

These boundaries are established by the task instruction, task declaration, frozen proxy evaluator, and final evaluator. The proxy product measures generative diagnostics on the validation boundary, whereas the final headline is an NLL estimate on the hidden real-test split. The metrics, splits, and sample accounting differ, so their scores are not directly comparable. The final diagnostic count of 10,000 molecules is not the row count used for test NLL.

3.2 How the baseline works

``text [hydrogen-free QM9 atom categories, bond categories, and graph structure] -> [marginal categorical transitions corrupt a randomly selected diffusion time; a 9-layer graph Transformer predicts the clean node and edge categories] -> [clean training graphs directly supply atom and bond targets; there is no external reward or synthetic label] -> [node cross-entropy + 5 × edge cross-entropy, optimized by AdamW with AMSGrad at fixed 2e-4] -> [all roughly 8.2 million parameters change and model.ckpt is published by training progress] ``

The baseline uses 500 diffusion steps, batch size 512, seed 42, weight decay 1e-12, gradient clipping at 1, and requests 1,000 epochs. It publishes every 50 epochs and retains the latest three checkpoints. Although one experiment YAML names batch size 1,024, the outer wrapper's last override makes 512 the executed baseline. Validation NLL is a diffusion variational-bound estimate; generation reverses all 500 categorical noising steps. The agent did not claim an architectural or objective defect. Its first useful diagnosis was operational: one training epoch took 211.2 seconds, but surrounding validation and sampling expanded the run to 550.72 seconds, leaving little four-hour capacity for both search and robust evaluation.

4. Four-hour exploration and decision process

Exploration ran from 2026-08-10 14:23:30 UTC to 18:11:18 UTC, about 3 hours 47 minutes 48 seconds. The agent first established a runnable baseline and separated training from evaluation overhead, then tested batch size, precision and metrics, TF32 and learning rates. The last phase compared constant and cosine schedules and hardened export/loadability. Environment checks and polling served those questions; the final approximately 13 minutes were idle rather than used to evaluate the actual candidate.

U-01 - Establishing the baseline and separating overhead

Motivation and hypothesis. An initial 20-epoch probe spent too long in initial validation. The agent needed per-epoch versus wrapper-overhead measurements before selecting experiments that could finish within four hours.

Concrete change and experimental setup. It terminated the unproductive 20-epoch probe, ran one baseline epoch at batch 512 and fixed 2e-4, and requested zero final samples in an attempt to skip sampling.

Observed result. Training itself took 211.2 seconds; the full run took 550.72 seconds and reached validation-alias NLL 77.88, close to the task's one-epoch reference 78.05. It exported 533 tensors but exited 1 because the empty-sample path divided by zero during final diagnostics. The initial 20-epoch run had no reliable summary.

Agent interpretation. The agent concluded that the model learned, but initial/final validation and sampling consumed much of the budget, and zero samples exposed a terminal wrapper defect.

Report assessment and confounds. NLL 77.88 is a validation-alias diagnostic, not hidden-test performance. The training evidence and artifact are interpretable, but the explicit terminal failure means this was not a successful end-to-end baseline run.

Decision and consequence. Later probes shortened training and removed avoidable evaluation paths; clean exit and frozen loading became separate submission requirements.