Public trajectory release
Trajectory Explorer
Looking for what each task contains? See the Task Gallery.
Browse every (task, model, harness, effort) configuration. Scores are task-local; open a case for the baseline, candidate algorithm, proxy evidence, formal replay, all retained checkpoints, qualifications, and audit record.
This is a redacted English release, not a raw log dump. “Protocol exposure” records what the benchmark made visible; it does not imply the agent used that information or changed the result. Those dimensions remain separate.
290 of 290 configurations
Claude Opus 5
Claude Code · high effort
Each update draws eight prompts and random latents, samples current SD1.5+LoRA with 20-step DDIM, guidance 5.0, and eta=0, detaches the first 19 steps, and backpropagates negative mean aesthetic reward through the last step, VAE, and…
Claude Opus 5
Claude Code · low effort
The baseline flow is prompt/latent to 50-step image, fixed aesthetic reward, normalized advantage, clipped PPO update, then LoRA. The candidate uses 20-step deterministic DDIM, detaches the first 19 steps, minimizes negative mean aesthetic…
Claude Opus 5
Claude Code · max effort
Baseline flow is prompt → complete sampled trajectory → fixed final-image reward → clipped policy-gradient loss → LoRA update. Candidate flow is prompt and latent → 20-step denoising with earlier steps gradient-free → one late clean-latent…
Claude Opus 5
Claude Code · medium effort
Prompts still come from the fixed 45-item simple-animal distribution. Frozen SD1.5 generates with scoring-matched 20-step DDIM and guidance 5; the first 19 steps are detached, while the last step backpropagates through the frozen VAE,…
Claude Opus 5
Claude Code · xhigh effort
Baseline flow is prompt and noise → image → scalar aesthetic reward → normalized advantage and PPO loss → attention LoRA. Candidate flow uses deterministic 20-step DDIM, runs the first 18 steps without gradients, differentiates the last…
Claude Sonnet 5
Claude Code · high effort
The flow remains prompt to image, frozen aesthetic reward, normalized advantage, clipped PPO update, then LoRA output. Reward, data, 50-step sampling, loss, learning rate, seed, and trainable parameters are unchanged; the changes are…
Claude Sonnet 5
Claude Code · low effort
The training signal and PPO update are unchanged. The patch adds a fused attention kernel, plaintext reward logging, wall-clock-driven long training, and ten-epoch checkpointing with three retained artifacts. An accidental bytecode…
Claude Sonnet 5
Claude Code · max effort
Candidate flow remains: fixed model and prompts - 20-step image generation - fixed aesthetic reward and per-prompt advantages - unchanged clipped PPO/DDPO update - LoRA weights. Changes are sampling, hyperparameters, schedule, engineering,…
Claude Sonnet 5
Claude Code · medium effort
Both baseline and candidate follow prompt → online DDIM image trajectory → fixed aesthetic reward → prompt-conditioned advantage normalization → clipped PPO → LoRA update. The candidate changes engineering, schedule, checkpoint policy, and…
Claude Sonnet 5
Claude Code · xhigh effort
The reward, advantage calculation, PPO objective, prompt source, and LoRA-only update rule were unchanged. The candidate changed sampling, learning rate, requested schedule, logging, and checkpoint policy: request 100,000 epochs, save each…
GPT-5.6 Luna
Codex · high effort
Both recipes follow prompt → 50-step image trajectory → fixed aesthetic reward → reward normalization and clipped advantage → clipped policy objective → LoRA update. Reward, prompts, sampling batch, learning rate, trainable parameters, and…
GPT-5.6 Luna
Codex · low effort
A prompt produces stochastic diffusion trajectories; the fixed aesthetic model supplies image rewards; per-prompt normalized, clipped advantages enter the PPO loss; AdamW updates U-Net attention LoRA while the base model, VAE, and text…
GPT-5.6 Luna
Codex · max effort
Baseline flow is prompt and current LoRA → 50-step image/latent trajectory → frozen aesthetic reward → per-prompt-normalized advantage → PPO-clipped AdamW update of LoRA. Candidate flow is identical except for 20-step trajectories, 300…
GPT-5.6 Luna
Codex · medium effort
Starting from the fixed model, the LoRA UNet generates 20-step DDIM trajectories; the frozen aesthetic network supplies online rewards; the candidate replaces each gathered batch with fractional ranks, then applies the original per-prompt…
GPT-5.6 Luna
Codex · none effort
Fixed prompts and random latents produce 16 image trajectories with 25 diffusion denoising steps; the unchanged frozen scorer supplies online aesthetic rewards; prompt-history-normalized advantages feed the unchanged PPO/AdamW update; only…
GPT-5.6 Luna
Codex · xhigh effort
Each formal epoch samples 32 animal prompts, generates 50-step diffusion trajectories, obtains frozen aesthetic rewards, converts them to prompt-history z-score advantages, and uses AdamW at 3×10^-4 for one clipped-PPO inner pass over LoRA…
GPT-5.6 Sol
Codex · high effort
Every branch starts from fixed SD1.5, samples the supplied 45 prompts, generates 32 images per epoch with 20-step DDIM, and applies frozen aesthetic rewards, original normalization, AdamW, and PPO-style DDPO to rank-4 attention LoRA for 26…
GPT-5.6 Sol
Codex · low effort
Each restart still begins from the fixed model, samples online images, obtains frozen aesthetic rewards, applies the original normalized-advantage PPO/AdamW update, and trains one LoRA for 13 epochs. The new driver planned seeds 43–53,…
GPT-5.6 Sol
Codex · max effort
Stage one keeps clipped DDPO but uses 50-step rollouts, batch 16, fused attention, and 40% transition training for twelve epochs. Stage two draws balanced prompts and new latents, minimizes negative mean aesthetic reward, and updates only…
GPT-5.6 Sol
Codex · medium effort
Fixed prompts and noise produce 32 online images through frozen SD 1.5 plus rank-4 LoRA and 20 DDIM steps. The frozen aesthetic model supplies reward; the original normalized-advantage PPO loss trains LoRA on 10 randomly selected…
GPT-5.6 Sol
Codex · none effort
Each epoch samples simple-animal prompts, generates complete 50-step trajectories, obtains frozen aesthetic rewards, normalizes them per prompt, and takes two AdamW updates of LoRA with the unchanged PPO surrogate. The patch is schedule,…
GPT-5.6 Sol
Codex · xhigh effort
Baseline flow is animal prompt → 50-step stochastic DDIM trajectory → fixed CLIP/aesthetic reward and per-prompt standardized advantage → PPO over all 50 transitions → rank-4 LoRA update. The candidate keeps the prompt source, reward,…
GPT-5.6 Terra
Codex · high effort
Baseline and candidate both sample prompts and latents, generate on-policy images, obtain rewards from the same frozen predictor, normalize advantages per prompt, and use the same PPO objective and AdamW to update only attention LoRA. The…
GPT-5.6 Terra
Codex · low effort
Baseline and candidate share one flow: prompt → SD v1.5+LoRA 50-step image/transition probabilities → fixed aesthetic reward → normalized, clipped advantage → PPO surrogate → AdamW LoRA update. Data, training signal, objective, sampling,…
GPT-5.6 Terra
Codex · max effort
Baseline and candidate share “animal prompt and noise → 50-step trajectory → frozen aesthetic reward and normalized advantage → clipped DDPO/PPO loss → rank-4 UNet LoRA.” The candidate uses one 32-image sampling batch and batches of 16,…
GPT-5.6 Terra
Codex · medium effort
Formal replay still samples simple_animals, generates images through 50 denoising steps, converts frozen aesthetic scores into per-prompt normalized advantages, and optimizes one inner pass of the clipped DDPO/PPO objective with AdamW at…
GPT-5.6 Terra
Codex · none effort
The effective candidate equals the baseline: prompt and noise produce a 50-step diffusion trajectory; a fixed local scorer produces each reward; per-prompt normalization and clipped advantages feed the existing PPO loss and AdamW; only…
GPT-5.6 Terra
Codex · xhigh effort
Baseline flow is prompt/noise → 50-step stochastic denoising and image → fixed aesthetic reward → normalized advantage and clipped PPO/AdamW → rank-4 LoRA. The candidate changes training to 20 steps, keeps 3e-4 through epoch 12,…
Kimi K3
Claude Code · max effort
Baseline flow is prompt → stochastic diffusion trajectory → aesthetic reward → clipped PPO advantage loss → LoRA update. Candidate flow is prompt plus fresh latent → 20-step deterministic DDIM → fixed differentiable aesthetic score →…
Claude Opus 5
Claude Code · high effort
Baseline and candidate share the same flow: fixed QM9 graph → categorical diffusion noise → graph-Transformer clean-category predictions → original categories as targets → cross-entropy/AdamW updates. The candidate adds four-worker…
Claude Opus 5
Claude Code · low effort
The learning flow remains fixed graph → noisy categories → clean-category predictions → true-category cross-entropy → all network weights. No external data, synthetic label, or new active objective was added. Engineering changes use eight…
Claude Opus 5
Claude Code · max effort
Fixed training graphs are noised and reconstructed by the unchanged nine-layer, T=500 model; clean graph categories remain the only targets, and the original loss and AdamW gradient rule update the same parameters. The candidate instead…
Claude Opus 5
Claude Code · medium effort
The submitted 12-layer Transformer still predicts clean atom/bond labels from noised fixed-training graphs and uses the original cross-entropy plus AdamW update. No external data, generated labels, rewards, or exploratory weights enter…
Claude Opus 5
Claude Code · xhigh effort
Fixed training graphs still supply clean atom/bond targets after marginal categorical noising; the same nine-layer Transformer predicts class distributions, and the same cross-entropy/AdamW rule updates all model parameters. The candidate…
Claude Sonnet 5
Claude Code · high effort
Both baseline and candidate follow graph input → categorical diffusion noise → graph-Transformer clean-class predictions → original atom/bond labels → cross-entropy and AdamW updates to the same 8.2M parameters. The candidate changes…
Claude Sonnet 5
Claude Code · low effort
Training data, corruption, labels, cross-entropy, AdamW, model parameters, and sampler are unchanged. Only the learning-rate coefficient follows a 40-epoch cosine decay to one-thousandth of the base rate and then restarts; saves occur…
Claude Sonnet 5
Claude Code · max effort
Baseline flow is graph categories to marginal noise to nine-layer Graph Transformer clean-category targets to atom-plus-5×bond cross-entropy, with AdamW updating all model weights and a 500-step reverse sampler producing molecules.…
Claude Sonnet 5
Claude Code · medium effort
The baseline takes fixed training graphs through discrete corruption and Transformer prediction, obtains targets from original atom/bond labels, applies weighted cross-entropy and AdamW, and reverses 500 diffusion steps at generation. The…
Claude Sonnet 5
Claude Code · xhigh effort
Fixed training graphs are noised; the unchanged Transformer predicts clean atoms/bonds; original categories provide targets; unchanged cross-entropy and AdamW update the full model. The patch only changes schedule, engineering, and…
GPT-5.6 Luna
Codex · high effort
Fixed training graphs are categorically noised at one of 200 times; the unchanged nine-layer Transformer predicts clean atom/bond targets; unchanged cross-entropy and AdamW update the same weights. Changes are hyperparameter, schedule,…
GPT-5.6 Luna
Codex · low effort
There is no candidate method. A zero-byte patch implies the unchanged baseline flow: fixed graph → marginal discrete corruption → graph-Transformer clean-category prediction → weighted node/edge cross-entropy → AdamW update → 500…
GPT-5.6 Luna
Codex · max effort
The candidate permutes batch strata, jitters within each, and maps them to times 0–500, approximately covering the horizon before the unchanged cosine/marginal corruption. The unchanged Transformer predicts clean categories; unchanged…
GPT-5.6 Luna
Codex · medium effort
Both versions take a fixed molecular graph, sample one diffusion time per batch, apply marginal-category noise, predict clean atom and bond categories, and update the same Transformer with AdamW. The candidate expands the time grid and…
GPT-5.6 Luna
Codex · none effort
Both methods follow fixed graph → categorical corruption → graph-Transformer prediction → clean node/edge labels → weighted cross-entropy/AdamW → updated full model. Only schedule, diagnostic sampling, and checkpoint policy changed; the…
GPT-5.6 Luna
Codex · xhigh effort
Fixed training graphs supply atom/bond targets; a 200-step marginal process makes noisy graphs; a 12-layer Transformer predicts clean categories; unchanged node and five-times edge cross-entropy update all parameters with AdamW at 2e-4,…
GPT-5.6 Sol
Codex · high effort
A seeded random time corrupts each fixed training graph; the unchanged full-feature Transformer predicts the original four atom and five edge classes, and the original graph supplies labels. AdamW still updates all parameters, but the edge…
GPT-5.6 Sol
Codex · low effort
Input corruption, clean-graph targets, loss, AdamW update rule, batch size 512, seed 42, model, and 500-step diffusion all remained unchanged. The patch changed only learning rate, schedule, engineering, and checkpoint policy: 3e-4, TF32,…
GPT-5.6 Sol
Codex · max effort
The candidate trains on all 117,776 allowed graphs, applies the same marginal corruption, predicts the original labels, and updates all approximately 8.2 million parameters with AdamW. The loss family and discrete-denoising update…
GPT-5.6 Sol
Codex · medium effort
Fixed training graphs still supply clean atom and bond targets; marginal categorical diffusion corrupts them at a random time, the same Transformer predicts clean categories, and AdamW/AMSGrad minimizes the unchanged weighted cross-entropy…
GPT-5.6 Sol
Codex · none effort
Fixed training graphs are noised at sampled diffusion times; the unchanged Transformer predicts original atom/bond labels; unchanged cross-entropy and AdamW update the same parameters. Changes are schedule/engineering/checkpoint policy,…
GPT-5.6 Sol
Codex · xhigh effort
Baseline flow is fixed graph → marginal corruption → clean-category prediction → original-category targets → weighted cross-entropy/AdamW → updated Transformer. Candidate corruption instead approaches uniform categories; labels, loss,…
GPT-5.6 Terra
Codex · high effort
Fixed training graphs are corrupted at a random diffusion time; an eleven-layer Transformer predicts original node and edge labels; unchanged weighted cross-entropy and AdamW at 2e-4 update online parameters; a warm-started 0.999 EMA…
GPT-5.6 Terra
Codex · low effort
Training data, diffusion, network, loss, optimizer, seed 42, batch size 512, learning rate 0.0002, requested 1,000 epochs, and three-artifact retention all remain unchanged. The sole source change is the SAVE_INTERVAL default in run.sh,…
GPT-5.6 Terra
Codex · max effort
Formal replay starts the same nine-layer Transformer on standard train graphs; clean atom/bond classes remain targets, and the same loss and AdamW update roughly 8.2 million parameters. Seed 42 now initializes model and workers.…
GPT-5.6 Terra
Codex · medium effort
Fixed training graphs still supply atom and bond labels; the same marginal corruption, Transformer, node cross-entropy plus five-times edge cross-entropy, AdamW/AMSGrad, and all approximately 8.2 million trainable parameters remain.…
GPT-5.6 Terra
Codex · none effort
Baseline flow is clean QM9 graph → marginal categorical corruption on a 500-step cosine schedule → graph-Transformer atom/bond predictions → atom cross-entropy plus five times bond cross-entropy → AdamW updates all weights; generation…
GPT-5.6 Terra
Codex · xhigh effort
Fixed clean graphs remain the labels; the unchanged nine-layer Transformer predicts atoms and bonds from marginally corrupted graphs, and AdamW still updates all approximately 8.2 million parameters at learning rate 2e-4. Only the bond…
Kimi K3
Claude Code · max effort
The labels remain the original clean training graphs, the weighted cross-entropy and AdamW update rule remain unchanged, and no external or synthetic data is added. Changes are hyperparameter, schedule, engineering, and checkpoint policy:…
Claude Opus 5
Claude Code · high effort
From 61,135 pinned rows, formal replay retained 53,748 margin-qualified pairs and deterministically derived 28,941 constraint pairs from the same prompts/responses; local checkers supplied labels without external models, evaluation rows,…
Claude Opus 5
Claude Code · low effort
Fixed chosen/rejected pairs feed the same beta=0.01 sigmoid DPO loss; paged AdamW updates only r=128 LoRA at 5e-6. The candidate kept 4-bit loading, length 1024, 16 pairs per update, all 61,135 rows, and seed 42. It changed 772 steps to…
Claude Opus 5
Codex · max effort
The candidate drops pairs with empty terminal content or score gap below 0.5, leaving 53,674/61,135. The fixed policy and adapter-disabled reference score both responses up to 1,536 tokens; the original preference still drives beta-0.01…
Claude Opus 5
Claude Code · medium effort
Baseline flow is ordinary chosen/rejected pairs → policy/reference relative scores → sigmoid DPO → LoRA update. The candidate keeps 40,000 ordinary pairs, transforms allowed UltraFeedback prompts/chosen answers into satisfying/violating…
Claude Opus 5
Claude Code · xhigh effort
Both baseline and candidate follow: fixed policy and preference pairs → current/reference log-probabilities → original chosen-over-rejected labels → sigmoid DPO with beta=0.01 and paged AdamW → rank-128 LoRA adapter. The candidate leaves…
Claude Sonnet 5
Claude Code · high effort
For each fixed preference pair, the policy and frozen reference score both completions. Pair ordering drives beta=0.01 sigmoid DPO, while chosen tokens also drive a 0.5-weight supervised loss; AdamW updates only the rank-128 QLoRA adapter.…
Claude Sonnet 5
Claude Code · low effort
Baseline and candidate both follow: fixed prompt and answer pair - policy/reference token log-probabilities - dataset's chosen-over-rejected label - sigmoid DPO - LoRA-only update and adapter export. There are no generated rewards, labels,…
Claude Sonnet 5
Claude Code · max effort
Baseline flow is preference pair → policy/reference chosen-versus-rejected log probabilities → sigmoid DPO → LoRA update. The candidate uses the same pair and frozen reference, then optimizes DPO plus token negative log likelihood on the…
Claude Sonnet 5
Claude Code · medium effort
Fixed chosen/rejected pairs feed the same beta-0.01 sigmoid DPO loss; paged AdamW updates only rank-128 LoRA while the quantized backbone remains frozen. There are no generated labels, rewards, synthetic data, or external tools. The patch…
Claude Sonnet 5
Claude Code · xhigh effort
Baseline flow is fixed pairs → active-policy and frozen-reference probabilities → sigmoid DPO → LoRA update. The candidate drops ties and adds 0.2-weight chosen likelihood, so relative preference and absolute generation train the same…
GPT-5.6 Luna
Codex · high effort
Baseline DPO compares chosen/rejected log-probability gaps from the LoRA policy with gaps from the same model with its adapter disabled, applies a sigmoid loss with β=0.01, and updates only LoRA. The candidate leaves that update rule…
GPT-5.6 Luna
Codex · low effort
Both recipes take fixed chosen/rejected pairs, compute their log probabilities under a LoRA policy and the same policy with LoRA disabled as reference, then optimize sigmoid DPO and export only LoRA weights. The candidate merely rescales…
GPT-5.6 Luna
Codex · max effort
Baseline flow is fixed pairs → policy/reference sequence probabilities → pairwise sigmoid DPO → LoRA update. Candidate flow uses the same computation and data but adds 0.1 times chosen-token negative log-likelihood to DPO at weight 1.0; no…
GPT-5.6 Luna
Codex · medium effort
For each fixed preference pair, the policy and frozen reference produce chosen/rejected log probabilities; their relative margin feeds beta=0.01 sigmoid DPO, and paged AdamW updates rank-128 LoRA on a frozen backbone. No outside reward,…
GPT-5.6 Luna
Codex · none effort
Baseline and candidate both follow: fixed preferred/rejected pair → adapter-enabled policy and adapter-disabled frozen-reference log probabilities → beta=0.01 sigmoid DPO loss → paged_adamw_32bit updates to LoRA only. Formal replay used…
GPT-5.6 Luna
Codex · xhigh effort
The current LoRA policy and frozen reference score each chosen/rejected pair. Pair labels supply the unchanged sigmoid-DPO signal at beta=0.01; chosen completion tokens additionally supply cross-entropy targets, and the summed loss uses…
GPT-5.6 Sol
Codex · high effort
Baseline and candidate use fixed chosen/rejected labels, current and frozen-reference log probabilities, a beta=0.01 sigmoid DPO loss, and the same rank-128 LoRA parameters. The candidate changes the objective by downweighting only…
GPT-5.6 Sol
Codex · low effort
Baseline and candidate both flow from fixed chosen/rejected conversations to policy/reference log probabilities, pairwise DPO signal, the same sigmoid loss and paged AdamW optimizer, and updated LoRA weights. There are no synthetic labels,…
GPT-5.6 Sol
Codex · max effort
Fixed UltraFeedback pairs are shuffled with seed 42 and rows with chosen-minus-rejected score below 0.5 are removed. Pair ordering then feeds the unchanged beta-0.01 sigmoid DPO policy/reference log-ratio; only the same LoRA parameters…
GPT-5.6 Sol
Codex · medium effort
Baseline flow: pair - policy/reference response log probabilities - binary preference - sigmoid DPO - LoRA update. Candidate flow: the same, except unequal-length tail tokens receive zero weight before the DPO log-ratio difference is…
GPT-5.6 Sol
Codex · none effort
Baseline flow is fixed preference pair - current LoRA and adapter-disabled reference log-probabilities - original chosen/rejected label - sigmoid DPO loss - LoRA-only update. The candidate merely prepends a filter when MIN_SCORE_GAP0; it…
GPT-5.6 Sol
Codex · xhigh effort
Baseline flow is fixed response pairs → policy and frozen-reference response probabilities → existing preferred/rejected ordering → sigmoid DPO with \(\beta=0.01\) → LoRA adapter. The candidate first removes zero-margin pairs using…
GPT-5.6 Terra
Codex · high effort
For each fixed preference pair, the LoRA policy and the reference obtained by disabling LoRA compute chosen/rejected probabilities; the unchanged sigmoid DPO loss, paged AdamW, and cosine schedule update only LoRA. The candidate changes…
GPT-5.6 Terra
Codex · low effort
Baseline flow is fixed chosen/rejected pairs - current and adapter-disabled reference log probabilities - dataset preference direction - sigmoid DPO loss - LoRA-only optimization. Candidate flow first discards rows whose asset score gap is…
GPT-5.6 Terra
Codex · max effort
Formal replay freshly loads the fixed policy and all preference pairs. LoRA policy probabilities and cached adapter-disabled reference probabilities enter the same beta=0.01 sigmoid DPO loss; only rank-128 LoRA over attention and MLP…
GPT-5.6 Terra
Codex · medium effort
Baseline flow is pinned preferred/rejected pairs - policy and frozen-reference log probabilities - dataset preference label - beta=0.01 sigmoid DPO - LoRA updates. The candidate first admits only pairs whose provided chosen-minus-rejected…
GPT-5.6 Terra
Codex · none effort
Baseline and candidate both follow: fixed preferred/rejected pair → policy/reference log probabilities → chosen-over-rejected DPO signal at beta 0.01 → sigmoid DPO optimized by paged AdamW → rank-128 LoRA adapter. Data, batch 16, 5e-6…
GPT-5.6 Terra
Codex · xhigh effort
Baseline flow is fixed preference pair → policy/reference response probabilities → stored preferred/rejected target → sigmoid DPO → LoRA adapter. The formal candidate repeats that unchanged computation for twelve independent starts,…
Kimi K3
Claude Code · max effort
The baseline converts each chosen/rejected label into a sigmoid-DPO ranking loss and updates only LoRA. The candidate keeps the fixed start, full dataset, seed 42, length 1,024, global pair batch 16, learning rate 5e-6, and LoRA structure,…
Claude Opus 5
Claude Code · high effort
Baseline flow is fixed ingredients → no target → uniform streaming average → one state dict. Candidate flow is proxy-label ingredient ranking → top-K mean softmax teacher → cross-entropy/Adam → 72 sum-one coefficients, negatives allowed →…
Claude Opus 5
Claude Code · low effort
The candidate forms a complete state dict as alpha-weighted visual and text ingredients, with coefficients summing to one and signs unrestricted. Frozen CLIP proxy cross-entropy supplies Adam gradients; subtracting the mean gradient…
Claude Opus 5
Codex · max effort
Branch A ranks ingredients by full-proxy accuracy and keeps an equal-weight addition only when accuracy strictly improves. B bootstraps proxy rows, repeats weighted-accuracy greedy search, and averages coefficient vectors; C uses the same…
Claude Opus 5
Claude Code · medium effort
Baseline flow is ingredients → no signal → equal coefficients → one state. Candidate flow is proxy and ingredients → repeated five-fold fitting of 12 rules → labels provide accuracy/cross-entropy or in-task teacher logits provide KL…
Claude Opus 5
Claude Code · xhigh effort
Baseline flow is frozen states → label-free uniform average → one state dictionary. Candidate flow is the same states plus true proxy labels → fast candidate correctness vectors → repeated fit/held rule comparisons → all-row refit of three…
Claude Sonnet 5
Claude Code · high effort
Baseline flow is fixed states → equal coefficients → normalized tensorwise sum → model.pt, with no labels, loss, optimizer, or gradients. The candidate repeatedly splits 1,000 classes into 500 search and 500 holdout classes, reranks…
Claude Sonnet 5
Claude Code · low effort
Baseline flow: 72 fixed weights → uniform tensor average → no label signal, loss, or optimizer → averaged state dictionary. Candidate flow: fixed weights plus labeled proxy → single-model and half-constrained greedy candidates plus…
Claude Sonnet 5
Claude Code · max effort
It caches each ingredient's features on two fixed folds, scores uniform, all singles, and all pairs, then repeatedly hill-climbs from randomized starts and coordinate-refines positive weights for small soups. Fixed labels supply fold…
Claude Sonnet 5
Claude Code · medium effort
Baseline flow was fixed ingredients → equal coefficients → direct parameter average → one state dictionary. Candidate flow was cached ingredient predictions → random A/B split → select A's best single and fit greedy on A → on B select a…
Claude Sonnet 5
Claude Code · xhigh effort
Baseline flow: fixed ingredients - proxy predictions - top-1 selection - uniform/greedy weighted average - state dict (the exported model weights). Candidate flow: rank all ingredients and publish best-single (progress 1) and full strict…
GPT-5.6 Luna
Codex · high effort
The baseline maps 72 state dicts directly to their equal average and uses no label signal. The candidate vectorizes ingredient logits; with seed 42 it cycles random 200/400/600/800-class fitting subsets and performs 25 Adam updates at…
GPT-5.6 Luna
Codex · low effort
It scores all 72 single models on the allowed proxy, ranks them, starts from the best, and considers each remaining model once. Each proposal is an equal-weight average of the current selected set plus that ingredient; it is retained only…
GPT-5.6 Luna
Codex · max effort
It scores all ingredients with fixed proxy labels, caches the top 24 plus pinned models, evaluates exploration recipes, and enumerates equal 2–4-way combinations among the top 18. Source then defines seed-sampled positive mixtures of width…
GPT-5.6 Luna
Codex · medium effort
The baseline streams all 72 ingredients into one equal average. The candidate first ranks ingredients using accuracy from the fixed proxy labels, then exports an emphasized top-four soup, the best single ingredient, and a soup that adds…
GPT-5.6 Luna
Codex · none effort
Baseline flow: fixed states → unconditional equal average → one exported state. Candidate flow: publish uniform; use fixed proxy labels/top-1 as a discrete signal to rank all ingredients and strict-greedily accept equal-average…
GPT-5.6 Luna
Codex · xhigh effort
Baseline flow is fixed weights → equal tensor average → one model. Candidate flow is fixed weights and proxy → construct singleton/sparse-soup predictions → use true proxy labels for top-1 selection → search pair grids, top-six triple…
GPT-5.6 Sol
Codex · high effort
Baseline flow is fixed states to label-free uniform tensor averaging to one model. Candidate flow is fixed states and true proxy labels to cached candidate predictions, pooled accuracy plus a four-partition gate, deterministic coefficient…
GPT-5.6 Sol
Codex · low effort
It immediately publishes 69/44, 69/44/63, and 69/44/59 soups, caches all ingredient logits, ranks singles, searches a 71×9 anchor-pair grid, then loops over global sparse and local affine perturbations. Proxy top-1 supplies selection;…
GPT-5.6 Sol
Codex · max effort
A float32 GPU bank supports gradient-free coordinate searches on three train/validation splits: paired images, 500 versus 500 classes, and a joint 600-class/image split. Training rows choose moves, held-out rows choose endpoints, and…
GPT-5.6 Sol
Codex · medium effort
Baseline flow is ingredients, no selection signal, equal tensor average, one artifact. The candidate uses permitted proxy labels for signed coordinate proposals, accepting lower cross-entropy while accuracy remains at least 0.700, then…
GPT-5.6 Sol
Codex · none effort
Progress 100 equally blends models 69/44; progress 200 equally blends the six greedy models; progress 300 uses those six with model_63 weighted 2.5×. Exploration proxy accuracy supplied the fixed choices. Formal replay only normalizes…
GPT-5.6 Sol
Codex · xhigh effort
The baseline averages 72 frozen states without labels, a loss, or updates. The candidate caches fixed proxy inputs, ranks ingredients, runs top-k, line, greedy, pair, triad, lower-ranked, and broad-pair families, then uses remaining wall…
GPT-5.6 Terra
Codex · high effort
Baseline flow is fixed states to equal average to one state dictionary, with no optimization signal. Candidate flow caches the same proxy, fixes offsets 0/1 as selection/holdout, constructs the exploration calibration, then…
GPT-5.6 Terra
Codex · low effort
Baseline flow: 72 fixed state dicts → fixed 1/72 weights → tensor average → one model. Candidate flow: the same ingredients plus proxy labels → rank by top-1 → greedily accept only improving equal-weight merges → export complete…
GPT-5.6 Terra
Codex · max effort
Baseline: fixed states → equal tensor sum → complete state, with no label, loss, optimizer, or gradient. Candidate: fixed states and provided proxy labels → deterministic resident-CLIP predictions → top-k, greedy, anchored, uniform-blend,…
GPT-5.6 Terra
Codex · medium effort
The baseline streams each fixed state dict once and applies coefficient 1/72. The candidate first scores that uniform state, then uses proxy accuracy from the fixed CLIP forward pass as its selection signal; it greedily changes only a…
GPT-5.6 Terra
Codex · none effort
No method was submitted. The baseline flow is fixed states to equal linear average to complete model.pt, with no reward, label, loss, optimizer, or trainable parameter. The temporary exploration flow would instead use proxy accuracy as the…
GPT-5.6 Terra
Codex · xhigh effort
Baseline flow: frozen state dictionaries to 1/72 tensor averaging to one exported state dictionary. Candidate flow: proxy labels provide allowed top-1 selection counts; strict greedy and half-guarded affine interpolation choose…
Kimi K3
Claude Code · max effort
Baseline flow: fixed weights → equal tensor average → one state dictionary. Candidate flow: fixed weights and proxy predictions → true-label accuracy/cross-entropy or ensemble-probability targets → greedy/Caruana/top-k search or…
Claude Opus 5
Claude Code · high effort
Fixed rows become boxed-answer prompts; the student generates 32×4 online solutions per step; the fixed teacher supplies top-16 targets; unchanged k1 distillation, AdamW, and learning rate 1e-6 update all student weights. These are prompt,…
Claude Opus 5
Claude Code · low effort
Baseline flow is prompt to on-policy student responses, teacher score for sampled tokens, clipped reverse-KL/PPO update, then full-student export. Candidate flow keeps the same student sampling but obtains the teacher's top-16 token…
Claude Opus 5
Claude Code · max effort
Fixed prompts enter the student; it samples four responses at temperature 1.0; the fixed teacher supplies token-level probability targets; the unchanged k1private filesystem location loss and AdamW update all actor weights. The candidate…
Claude Opus 5
Claude Code · medium effort
A fixed training prompt enters the current student, which samples four responses; the fixed teacher supplies top-16 probabilities along those student samples; sampled-token k1 mismatch becomes the policy-gradient-style distillation signal;…
Claude Opus 5
Claude Code · xhigh effort
Baseline flow is prompt → student on-policy response → fixed teacher probability for the sampled token → one-sample REINFORCE-style reverse-KL estimate → all student weights. Candidate flow is prompt → student response → teacher top-16…
Claude Sonnet 5
Claude Code · high effort
The current student still generates four responses online; the fixed teacher scores exactly those tokens; negative reverse-KL costs become token advantages in the PPO surrogate; FSDP updates all approximately 1.777 billion student…
Claude Sonnet 5
Claude Code · low effort
Each formal step samples four shuffled prompts and generates two student solutions per prompt. The frozen teacher scores those online trajectories token by token; the unchanged k1 distillation objective makes two clipped PPO update passes…
Claude Sonnet 5
Claude Code · max effort
Shuffled fixed data enters the unchanged online-sampling pipeline; the frozen teacher supplies token-probability targets, k1 and the 0.20/0.28 ratio clip update only the student, and checkpoints export full weights. There are no outside…
Claude Sonnet 5
Claude Code · medium effort
Baseline flow is fixed problems → student samples → fixed teacher scores sampled tokens → reverse-KL advantage → PPO-style AdamW update of all student weights. The candidate only uses a seed-1 data permutation and 10,240-token responses.…
Claude Sonnet 5
Claude Code · xhigh effort
The candidate samples two prompts from a reproducible seed-42 permutation, generates four student rollouts per prompt, obtains live frozen-teacher probabilities on sampled tokens, negates the clamped k1 KL into token advantages, and…
GPT-5.6 Luna
Codex · high effort
Baseline and candidate share the same flow: fixed prompt - current student samples responses - fixed teacher provides online token targets - k1 reverse-KL advantage drives AdamW policy-gradient updates - full student model is exported.…
GPT-5.6 Luna
Codex · low effort
Intended baseline and candidate flows are identical: fixed prompts lead to student rollouts; the mounted teacher supplies sampled-token probabilities; clamped negative k1 estimates become vanilla policy-gradient advantages; a constant 1e-6…
GPT-5.6 Luna
Codex · max effort
Baseline and candidate both sample eight answers, obtain fixed-teacher probabilities on those exact tokens, optimize the unchanged k1 distillation loss with AdamW at 1e-6, and update all roughly 1.777 billion student parameters. The patch…
GPT-5.6 Luna
Codex · medium effort
Formal replay still uses student responses as on-policy inputs, teacher token probabilities as targets, and the unchanged K1 policy-gradient objective. Models, data, sampling, reward, optimizer settings other than learning rate, requested…
GPT-5.6 Luna
Codex · none effort
Fixed prompts enter the student; the student samples four responses; the fixed teacher supplies token probabilities on those fresh samples; reverse-KL-derived policy-gradient updates change all student weights; complete weights are…
GPT-5.6 Luna
Codex · xhigh effort
Fixed prompt → student samples four online answers at 0.7 → fixed teacher at 1.0 scores the sampled tokens → negative k1 signal becomes the policy-gradient advantage → constant-1e-6 AdamW updates every student parameter. Teacher targets…
GPT-5.6 Sol
Codex · high effort
In both baseline and candidate, the current student samples answers, the fixed teacher assigns probabilities only to sampled student tokens, and unchanged k1 reverse-KL with constant-1e-6 AdamW updates every student parameter. The patch…
GPT-5.6 Sol
Codex · low effort
The student still generates eight responses online, the fixed teacher supplies token-level k1 distillation targets, the same PPO-shaped loss and AdamW update all student weights, and no task reward, external data, or pre-existing rationale…
GPT-5.6 Sol
Codex · max effort
Four fixed-data prompts produce two on-policy student responses each. The frozen teacher supplies log probabilities at student-sampled tokens; student-minus-teacher K1 values are negated, detached, and used as token-level policy-gradient…
GPT-5.6 Sol
Codex · medium effort
Fixed prompts go to the current student, which samples four responses; the fixed teacher supplies token probabilities on those responses; unchanged k1 reverse KL becomes a policy-gradient signal; AdamW updates the full student. Only…
GPT-5.6 Sol
Codex · none effort
Ordered prompts feed four current-student samples; the fixed teacher supplies token log probabilities; a clipped one-sample reverse-KL (k1) value becomes advantage in a one-epoch PPO-style loss; AdamW updates all 1.777B student parameters.…
GPT-5.6 Sol
Codex · xhigh effort
Baseline flow is student on-policy samples → fixed-teacher token scores → K1 distillation policy gradient → all student weights. The candidate additionally compares each sampled answer with the fixed training answer using the DAPO rule…
GPT-5.6 Terra
Codex · high effort
The intended flow remained fixed prompts to online student samples, online teacher token targets, K1 reverse-KL/PPO loss, and AdamW updates to the full student only. No answer label, pre-existing rationale, synthetic data, or external tool…
GPT-5.6 Terra
Codex · low effort
There is no submitted method. The best exploration artifact still used prompt → student sample → teacher probability on sampled tokens → detached negative k1 signal → policy-gradient/AdamW update of all student weights. Forward KL and k3…
GPT-5.6 Terra
Codex · max effort
The student still generates four online responses. The teacher now supplies top-16 next-token probabilities, which form a directly backpropagated forward-KL loss; each row’s supplied answer is scored online by the bundled rule grader and…
GPT-5.6 Terra
Codex · medium effort
Baseline flow is fixed prompts → four online student answers → frozen-teacher probabilities on sampled tokens → clamped k1 reverse-KL estimate and policy-gradient-style AdamW update → a fully updated student model. The candidate uses the…
GPT-5.6 Terra
Codex · none effort
No method was submitted. The empty patch is not an “unchanged baseline candidate,” because formal intake required a nonempty verifiable patch and rejected it. The only launched flow remained: fixed prompts → four current-student samples →…
GPT-5.6 Terra
Codex · xhigh effort
Baseline flow is student on-policy responses - teacher probabilities for sampled tokens - clipped log-ratio advantage - PPO-style full-student update. The candidate keeps the same responses but asks the fixed teacher for its 16 most…
Kimi K3
Claude Code · max effort
Baseline is student states → teacher score for the sampled token → K1 advantage → policy-gradient update. Candidate is student states → online teacher top-16 distribution → direct truncated forward KL → constant-rate AdamW full-parameter…
Claude Opus 5
Claude Code · high effort
Baseline flow is fixed conversations → existing assistant completions as labels → prompt masking → completion NLL → full-weight update. The candidate keeps that signal and update rule. Round r shuffles with 42+7919r, selects 18 times its…
Claude Opus 5
Claude Code · low effort
Phase A keeps the baseline and labels actual steps 30 and 60 as checkpoint-1000/1100. Phase B orders 4,061 row-testable training problems easiest-first, samples eight temperature-0.8 answers per problem from the current sampler, executes…
Claude Opus 5
Claude Code · max effort
Baseline flow is fixed problem/full teacher answer → teacher-token labels → prompt-masked cross-entropy/AdamW → full-weight model. The candidate executes teacher code, removes testable failures, then retains original-prompt full targets or…
Claude Opus 5
Claude Code · medium effort
Stage 1 trains the faithful control. Each later round samples four candidates on 1,280 fixed training prompts from the current sampler, executes them on those prompts' own tests, and accumulates short passing answers. Odd rounds use…
Claude Opus 5
Claude Code · xhigh effort
Baseline flow is fixed long answers → masked-prompt completion likelihood → AdamW updates to all weights. The candidate first trains 60 steps on 1,080 existing long reasoning answers and exports; it then trains up to six epochs on short…
Claude Sonnet 5
Claude Code · high effort
The gate executes only existing fixed-corpus answers against examples embedded in the same rows; it creates no reward, answer, synthetic data, or external input. Retained assistant completions still provide the labels, completion NLL and…
Claude Sonnet 5
Claude Code · low effort
Both recipes tokenize fixed conversations, mask prompt labels, use the fixed assistant completion as the sole target, minimize full-model next-token NLL, and export full weights. The candidate keeps that objective and update rule…
Claude Sonnet 5
Claude Code · max effort
Fixed rows are split, filtered, tokenized with the original chat template, and prompt-masked; full-parameter Trainer updates still optimize completion-only causal NLL. The changes are data selection, length-grouped sampling, elapsed-time…
Claude Sonnet 5
Codex · medium effort
Baseline flow is fixed messages → mask prompt targets → use existing assistant tokens as labels → minimize completion NLL → update all weights. The candidate keeps that flow but batches similar lengths and schedules LR by elapsed…
Claude Sonnet 5
Claude Code · xhigh effort
Baseline and formal candidate share one flow: fixed problem plus original rationale/code → Qwen next-token probabilities → original assistant tokens as labels with prompt masking → full-parameter completion negative log-likelihood →…
GPT-5.6 Luna
Codex · high effort
Fixed problem/solution chats are encoded, prompt labels are masked, supplied solution tokens provide targets, and AdamW backpropagates answer cross-entropy through all Qwen parameters. No generated reward, synthetic data, external model,…
GPT-5.6 Luna
Codex · low effort
In both recipes, fixed prompts and stored assistant responses enter Qwen; prompt tokens are ignored as labels, response tokens supply supervised targets, completion cross-entropy drives gradient descent, and all approximately 1.544 billion…
GPT-5.6 Luna
Codex · max effort
Baseline flow is fixed prompt and rationale-plus-code answer, prompt-label masking, cross-entropy on all assistant tokens, then full-parameter Trainer updates. The candidate locates the assistant start, <private filesystem location, and…
GPT-5.6 Luna
Codex · medium effort
Baseline flow is fixed conversation → all completion labels → cross-entropy → full-parameter update. Candidate flow is the same conversation → locate its last fenced block → mask every other label → the same cross-entropy and…
GPT-5.6 Luna
Codex · none effort
Baseline and candidate both map the fixed prompt and assistant completion to model token probabilities, use the completion as the label, minimize completion-only NLL, and update every weight. No reward, synthetic label, external data, or…
GPT-5.6 Luna
Codex · xhigh effort
Baseline and active candidate share one flow: fixed prompt plus existing rationale/code → Qwen token predictions → prompt masking and all answer tokens as labels → full-parameter cross-entropy optimization → complete Hugging Face export.…
GPT-5.6 Sol
Codex · high effort
Baseline and formal flows are the same: fixed prompt and full original response → mask prompt labels → response-token cross-entropy → full-parameter AdamW update → full causal-model export. Formal replay used target_mode=completion;…
GPT-5.6 Sol
Codex · low effort
Both baseline and candidate use fixed prompt/solution pairs, fixed solution-token labels, completion-only NLL, AdamW, and full-parameter updates; no reward model, synthetic data, external model, or exploration weight is used. The candidate…
GPT-5.6 Sol
Codex · max effort
Both branches independently reload the fixed model and use the same 8,005 original long-reasoning training rows; assistant tokens remain labels, prompts remain masked, and AdamW minimizes the same next-token cross-entropy over all…
GPT-5.6 Sol
Codex · medium effort
Formal replay reads 8,005 fixed training examples, uses the original assistant completion as the label, masks prompt tokens, and updates all model parameters with the unchanged next-token loss. Only hyperparameters, schedule, engineering,…
GPT-5.6 Sol
Codex · none effort
Baseline and candidate share the flow: fixed prompt/solution → masked prompt and solution labels → next-token cross-entropy with AdamW → full-model checkpoints. Only schedule and checkpoint policy change: steps 60→1,000 and evaluation/save…
GPT-5.6 Sol
Codex · xhigh effort
Baseline flow is fixed conversations → original assistant completion as label → prompt-masked cross-entropy → full-parameter model. Formal replay creates three branches from fixed assets: full completion for 60 steps; clean row selection…
GPT-5.6 Terra
Codex · high effort
Fixed prompts and existing full answers are encoded to 8,192 tokens; prompt labels are masked; answer tokens provide cross-entropy targets; full-model optimization updates every parameter. Pre-existing rationale remains supervised when not…
GPT-5.6 Terra
Codex · low effort
Baseline flow is fixed conversation → model next-token predictions → complete assistant response as labels → completion cross-entropy → all weights and a full model change. The candidate inserts a “last fenced Python block” transform…
GPT-5.6 Terra
Codex · max effort
Fixed prompts and full supplied completions are tokenized to 32,768 tokens; prompt positions are masked, completion-token cross-entropy backpropagates through every model parameter, and complete weights are published every 100 steps.…
GPT-5.6 Terra
Codex · medium effort
Both baseline and candidate follow: fixed prompt and teacher answer - Qwen token prediction with prompt labels masked - teacher tokens as the sole target - completion NLL and gradient optimization - all 1.54B weights updated into a…
GPT-5.6 Terra
Codex · none effort
Baseline and candidate share the same flow: fixed conversation → Qwen token predictions → existing assistant tokens as labels with prompt tokens masked → cross-entropy and AdamW → updated full-model checkpoint. No generated reward,…
GPT-5.6 Terra
Codex · xhigh effort
Baseline and candidate share this flow: fixed problem and completion - corpus assistant tokens as labels with prompt masking - token cross-entropy - full-parameter updates - complete model export. The candidate changes only…
Kimi K3
Claude Code · max effort
Baseline flow: fixed conversation → full assistant-answer labels → prompt-masked token cross-entropy → all weights updated. Candidate flow: require normal termination and fenced code, remove the existing think segment, preserve the…
Claude Opus 5
Claude Code · high effort
Baseline flow is fixed questions and reference model → NPO plus retain NLL → paged AdamW updates all weights → full model. The candidate runs seven fresh-start NPO arm types, diagnoses train-role Extraction/probability/ROUGE, and accepts…
Claude Opus 5
Claude Code · low effort
Phase A scores forget answers under current and frozen models, combines NPO with retain-answer NLL and anchor-to-current token KL, and updates all 1.236B current parameters with AdamW. Repair weakens forgetting and strengthens retention;…
Claude Opus 5
Claude Code · max effort
Baseline flow is allowed forget/retain answers → current/reference likelihoods → NPO ratio signal plus retain labels → preference loss plus retain NLL → full-weight update. Candidate flow is the same allowed data plus frozen anchor →…
Claude Opus 5
Claude Code · medium effort
Baseline flow is fixed rows and frozen start - current/reference likelihoods and retained labels - NPO plus retain NLL - full-model update. The candidate restarts each stage - applies NPO, length-normalized SimNPO, or uniform-target…
Claude Opus 5
Claude Code · xhigh effort
Baseline flow is fixed QA/anchor → current and frozen models score forget answers → loser-style NPO plus retain-answer cross-entropy → update all current weights. ANPO retains those targets, optionally adds forward KL(anchor || current) on…
Claude Sonnet 5
Claude Code · high effort
Fixed forget10/retain90 answers feed the current model; a frozen start model supplies forget-reference probabilities. Beta-0.1 NPO forget loss and true-answer retain cross-entropy are combined with gamma=1, alpha=6, then paged AdamW 32-bit…
Claude Sonnet 5
Claude Code · low effort
Both baseline and candidate follow: fixed forget/retain text → current/reference answer likelihoods → NPO forget signal plus retain-answer NLL → full-model optimization → standalone export. The candidate leaves data, sampling, β=0.1, loss…
Claude Sonnet 5
Claude Code · max effort
Fixed forget answers and randomly paired train-role retain answers enter the current model; a frozen copy of the start supplies the relative-likelihood target for NPO, while original retain answers supply NLL targets. Their weighted sum…
Claude Sonnet 5
Claude Code · medium effort
Forget answers pass through current and frozen-start models; their NLL difference supplies NPO pressure, while current-model retain90 NLL supplies preservation supervision. Paged AdamW updates all current-model parameters under the same…
Claude Sonnet 5
Codex · xhigh effort
Fixed forget10/retain90 batches enter the current and frozen-reference models; relative forget-answer likelihood supplies NPO's signal and original retain answers supply supervision. The candidate optimizes beta-0.1 NPO plus coefficient-3…
GPT-5.6 Luna
Codex · high effort
Both baseline and candidate pair each anchored forget10 row with a random retain90 row. Current and frozen-anchor likelihoods create the NPO forgetting target; original retain answers create the NLL retention target; all Llama weights are…
GPT-5.6 Luna
Codex · low effort
Baseline and candidate both flow from fixed forget/retain rows, through current and frozen-reference likelihoods, to NPO forget loss plus equally weighted retain NLL, then paged AdamW updates of all current-model weights. Data sampling,…
GPT-5.6 Luna
Codex · max effort
Baseline flow is paired forget/retain answers - current/reference likelihoods - NPO logistic forget loss plus retain NLL - full-model update. Candidate flow is the same pairs - current layer-7 answer activations and frozen retain…
GPT-5.6 Luna
Codex · medium effort
Baseline forget supervision is a current-versus-reference NPO ratio; candidate SimNPO applies a smooth sigmoid loss to each current-model forget answer's mean NLL, then adds alpha=2 retain-answer NLL. Formal settings were beta=0.1,…
GPT-5.6 Luna
Codex · none effort
Each forget row is paired with a retain row. Current model and frozen anchor provide the relative NPO forget signal; the original retain answer supplies NLL. Loss is gamma times forget loss plus alpha times retain loss, and paged AdamW…
GPT-5.6 Luna
Codex · xhigh effort
Baseline flow is forget-anchored pairing - current/reference forget likelihood ratio plus original retain labels - NPO forget loss plus alpha-1 retain NLL - full-model AdamW update. Candidate flow is full retain90 traversal with random…
GPT-5.6 Sol
Codex · high effort
Every branch independently restarts from the fixed anchor, randomly pairs forget with retain rows, and trains all weights for 120 actual steps using unchanged reference-relative NPO plus retain NLL. Single-GPU AdamW replaces ZeRO-3; the…
GPT-5.6 Sol
Codex · low effort
For each forget record, current and frozen-reference models compute answer NLL, while a randomly paired retain answer supplies cross-entropy. The candidate minimizes unchanged NPO plus eight times retain loss, updates all approximately…
GPT-5.6 Sol
Codex · max effort
Baseline flow is forget/retain answers plus frozen start → reference-relative NPO forget loss and retain NLL → full-weight update. Candidate flow is a retain90-length-anchored pass plus sampled forget10 → the trainable model's per-answer…
GPT-5.6 Sol
Codex · medium effort
Original QA targets supply both losses: a current-versus-frozen-start likelihood ratio yields NPO forget loss, and retain-answer cross-entropy yields retain loss; their sum updates all 1.236 billion parameters, without generated labels,…
GPT-5.6 Sol
Codex · none effort
Baseline flow is paired forget/retain questions → current and frozen-start token probabilities → sequence NPO plus supervised retain NLL → full-weight optimization. Candidate flow keeps the data, sampling, optimizer, 1.5e-5 learning rate,…
GPT-5.6 Sol
Codex · xhigh effort
Baseline flow is fixed answers → current/reference likelihoods → NPO forget loss plus labeled retain cross-entropy → full-model update. Candidate flow is retain-anchored paired data → current-probability-weighted WGA forgetting → retain…
GPT-5.6 Terra
Codex · high effort
Fixed forget10/retain90 answers and the frozen start supply all targets. Retain90 anchors each batch; current-versus-reference forget likelihood enters unchanged beta=0.1 NPO, retain NLL is weighted by 2, and paged AdamW updates the full…
GPT-5.6 Terra
Codex · low effort
Current and frozen-start models compute forget-answer losses; the current model also computes retain loss. A beta-0.1 NPO log-sigmoid term suppresses forget-answer likelihood, an equally weighted retain term preserves allowed answers, and…
GPT-5.6 Terra
Codex · max effort
Baseline and candidate share the flow “forget/retain questions → current and frozen-reference likelihoods → NPO forget loss plus retain NLL → full-model update.” The candidate iterates retain rows, randomly pairs forget rows, and minimizes…
GPT-5.6 Terra
Codex · medium effort
Baseline NPO combines current-versus-reference forget likelihood with retain-answer NLL. SimNPO instead computes each forget answer's mean NLL, applies strong pressure below delta=5, and saturates above it; equal-weight retain NLL…
GPT-5.6 Terra
Codex · none effort
The current model and frozen anchor produce a forget-likelihood difference; fixed retain answers supply retention targets; beta=0.1 NPO loss plus retain NLL is optimized with paged AdamW over approximately 1.236 billion parameters. The…
GPT-5.6 Terra
Codex · xhigh effort
Each of 400 forget examples is paired with a random example from 3,600 retain rows. The current and frozen-start models compute sequence NLL for the forget answer; a beta=0.1 logistic NPO loss pushes current forget NLL above the reference,…
Kimi K3
Claude Code · max effort
Forget rows pair with sampled retain rows. Current/reference models supply the NPO signal; retain answers supply NLL supervision; AdamW uses batch 16, warmup, and cosine decay. A six-branch grid covers strength, freezing, SimNPO, and…
Claude Opus 5
Claude Code · high effort
The baseline flow is fixed dense model plus C4 activations, followed by OWL/Wanda selection and direct zeroing, producing one untrained sparse model. The candidate samples 1,024 C4 windows, uses lambda=0.04 for layer targets, and applies…
Claude Opus 5
Claude Code · low effort
Baseline flow is C4 - Wanda/OWL importance - per-row zeros - unchanged survivors. Candidate flow is 512 seed-0 C4 windows - damped module-global OBS plus blockwise reconstruction - packed C4 read by sparse student and fixed dense teacher -…
Claude Opus 5
Claude Code · max effort
OWL allocates layer sparsity; SparseGPT/OBS freezes a mask and damped CG reconstructs each layer with the dense layer on sparse inputs as target. Masked layer-wise Adam updates surviving weights, biases, and LayerNorm. A fixed dense…
Claude Opus 5
Codex · medium effort
The baseline converts C4 activations into OWL budgets and deletes low Wanda-score weights. The candidate uses 128 blocks for allocation and all 512 for damped second moments, deletes the exact target count by approximate OBS saliency, and…
Claude Opus 5
Claude Code · xhigh effort
Baseline flow is dense model plus C4 activations → OWL layer budgets and Wanda ranking → direct zeros and export. Candidate stage 1 instead uses activation second moments for SparseGPT/OBS masking and closed-form survivor updates. In stage…
Claude Sonnet 5
Claude Code · high effort
Baseline flow: dense OPT and C4 activations - OWL sparsity allocation - Wanda ranking and direct zeroing - one-shot sparse model. Candidate flow: the same fixed start and 128 seed-0 C4 windows - uniform SparseGPT estimates an activation…
Claude Sonnet 5
Claude Code · low effort
The patch first builds the same nonuniform OWL/Wanda mask and publishes it as progress 1, then attempts wall-clock fine-tuning with AdamW, a warmup-plus-cosine schedule, random 2,048-token C4 windows, online dense-teacher signals, and…
Claude Sonnet 5
Claude Code · max effort
SparseGPT uses 512 C4 sequences to estimate layerwise curvature, uniformly removes about 70% of weights, and compensates retained weights; it immediately publishes a pruning fallback. Next-token cross-entropy and AdamW update decoder…
Claude Sonnet 5
Claude Code · medium effort
Each trial starts from the fixed dense model and samples 512 C4-train sequences. SparseGPT forms a local Hessian approximation, removes 70% of each linear layer in 128-column blocks, and adjusts retained weights to reduce…
Claude Sonnet 5
Claude Code · xhigh effort
Baseline flow is fixed weights plus C4 activations, followed by OWL layer targets, Wanda ranking, direct zeroing, and full-model export. The candidate keeps the fixed inputs and OWL allocation but accumulates an approximate Hessian from…
GPT-5.6 Luna
Codex · high effort
Baseline flow is fixed C4 activations to Wanda scores, OWL layer allocation, then per-row zeroing. The candidate prunes with lambda 0.11 and multiplier 10, then uses 128 fixed-C4 sequences as causal-language-model labels; standard shifting…
GPT-5.6 Luna
Codex · low effort
Baseline construction turns fixed C4 activations and dense weights into an OWL/Wanda importance mask; it uses no labels, reward, loss, or backpropagation, and proxy perplexity only selects candidates. The patch generalizes buffer capacity…
GPT-5.6 Luna
Codex · max effort
For each seed 0/1/2, the recipe reloads dense OPT, samples 768 C4 sequences, accumulates a damped approximate Hessian, removes about 70% of weights in 128-column blocks, and propagates inverse-Hessian error corrections before exporting a…
GPT-5.6 Luna
Codex · medium effort
Baseline flow is fixed C4 inputs → Wanda activation importance and OWL layer allocation → rowwise deletion → full sparse model. Candidate flow keeps OWL allocation, accumulates second-order input statistics for every linear module, then…
GPT-5.6 Luna
Codex · none effort
The dense OPT-6.7B model is calibrated on fixed C4 sequences. For each decoder linear layer, the code accumulates mean-squared input activations and assigns each weight the Wanda score abs(weight) × sqrt(input statistic). OWL uses…
GPT-5.6 Luna
Codex · xhigh effort
The baseline flow is C4 activations to Wanda ranking to OWL allocation to one-shot zeroing. The candidate uses 512 C4 sequences for SparseGPT's curvature estimate, zeros 70% uniformly with error compensation, then uses 256 C4 sequences for…
GPT-5.6 Sol
Codex · high effort
Baseline C4 activations feed Wanda/OWL scores, which directly determine zeros without optimization. The candidate instead runs uniform 70% SparseGPT on 1,024 seed-0 C4 blocks, using approximate second-order information to compensate…
GPT-5.6 Sol
Codex · low effort
The baseline statically zeroed low Wanda/OWL scores. The candidate used 1,024 C4 sequences to estimate each layer's activation Hessian, pruned 70% of weights by blockwise second-order reconstruction cost, and compensated survivors. It then…
GPT-5.6 Sol
Codex · max effort
Formal replay starts from the mounted dense model, collects 1,024 C4 sequences, and applies uniform SparseGPT with percdamp=0.01 and block size 128. It saves progress 1, derives an immutable mask, reloads the fixed dense teacher, and…
GPT-5.6 Sol
Codex · medium effort
SparseGPT estimates each linear module's input curvature from 1,024 seed-0 C4 sequences and compensates surviving weights as deletions are applied. Recovery samples 2,048-token C4 windows: an online fixed dense model supplies final hidden…
GPT-5.6 Sol
Codex · none effort
Baseline ranking directly zeroed weights and left survivors unchanged. The candidate accumulated each linear module's input Gram matrix from 1,024 fixed C4 sequences, added damping equal to 0.5 times its mean diagonal, selected about 70%…
GPT-5.6 Sol
Codex · xhigh effort
Baseline flow is dense OPT plus C4 activations to Wanda/OWL hard masking to one untrained sparse model. The candidate uses 2,048 C4 windows to estimate SparseGPT curvature, deletes 70% per row blockwise while compensating retained weights,…
GPT-5.6 Terra
Codex · high effort
The baseline converts C4 activations into Wanda importance and directly zeros weights. The candidate estimates second-order statistics from 256 C4 windows, removes weights blockwise while compensating survivors, then samples fresh C4: the…
GPT-5.6 Terra
Codex · low effort
Baseline flow: fixed weights and C4 activations produce Wanda saliency; OWL reallocates layer density; low-saliency weights are zeroed; a complete model is exported. Candidate flow: the same inputs build an input-covariance/Hessian…
GPT-5.6 Terra
Codex · max effort
SparseGPT uses second-order C4 statistics to prune 70% while compensating retained weights. On new C4 spans, fixed dense OPT supplies temperature-1 token targets; the student uses pure teacher cross-entropy, AdamW, cosine decay, masked…
GPT-5.6 Terra
Codex · medium effort
Baseline flow: C4 inputs → channel activation statistics → Wanda scores plus OWL layer allocation → direct zeroing → one sparse model. Candidate flow: 256 C4 sequences for one seed → per-layer activation Hessian approximation → SparseGPT…
GPT-5.6 Terra
Codex · none effort
No method was submitted. Baseline flow is C4 windows - dense OPT activations - Wanda importance |weight| sqrt(channel activation) - OWL layer allocation and lowest-score zeroing - full sparse model. The attempted flow retained the same…
GPT-5.6 Terra
Codex · xhigh effort
Baseline flow is C4 activations → Wanda/OWL importance and layer allocation → one-shot zero mask → sparse model. Candidate flow is 1,024 C4 calibration windows → SparseGPT second-order mask and retained-weight compensation → online…
Kimi K3
Claude Code · max effort
Baseline C4 activations produce OWL/Wanda scores and direct zeros. The candidate forms a seed-0 mask from 512 sequences, then a sparse student and frozen allowed dense teacher consume packed C4. Fused AdamW minimizes 0.5 next-token CE plus…
Claude Opus 5
Claude Code · high effort
The baseline flows from generated board, to policy rollouts, to sparse solve reward and group-relative advantage, to GRPO, to merged full weights. The candidate generates fresh training boards; BFS supplies shortest actions and mechanical…
Claude Opus 5
Claude Code · low effort
Candidate flow is fresh in-container 6×6 one-box board → BFS shortest path and rule-synthesized rationale → optional 0.3 recoverable deviation and relabeling → assistant-token-only cross entropy with full-parameter AdamW → atomic Hugging…
Claude Opus 5
Claude Code · max effort
Training still uses fresh online boards, sparse frozen-environment rewards, grouped advantages, and the unchanged clipped GRPO update over all Qwen weights. A separate 192-board validation bank runs every ten steps: near-best states are…
Claude Opus 5
Claude Code · medium effort
Baseline flow is generated board and policy rollouts, then environment reward, GRPO clipped loss, and updated Qwen weights. Candidate flow first generates each board in-run, solves it with BFS, optionally perturbs the trajectory,…
Claude Opus 5
Claude Code · xhigh effort
The baseline maps online boards through policy trajectories and environment/format rewards to within-board GRPO advantages and clipped updates of all weights. The candidate adds distance reduction to reward, strengthens malformed-output…
Claude Sonnet 5
Claude Code · high effort
Baseline flow is online board → policy trajectories → frozen-environment reward → normalized, clipped GRPO loss → full-weight update. The candidate prepends fresh board → in-container BFS action label and locally templated rationale →…
Claude Sonnet 5
Claude Code · low effort
Online Sokoban boards still provide all rewards and trajectories. The candidate retains normalized, clipped GRPO and AdamW, adds a reference-policy low_var_kl term (coefficient 0.02), and raises entropy from 0.001 to 0.004; full Qwen…
Claude Sonnet 5
Claude Code · max effort
The baseline flow is online board → current-policy trajectories → frozen-environment return and within-board advantage → clipped GRPO/PPO update → changed full-model weights. The candidate keeps that flow and adds public validation as a…
Claude Sonnet 5
Claude Code · medium effort
The baseline flow is board and model sample → environment/format reward → same-board GRPO advantage and clipped loss → full-model update. The candidate still samples normally, but after reset a wall-time scheduler selects environments;…
Claude Sonnet 5
Codex · xhigh effort
Formal replay still generates boards online, samples multi-turn trajectories, derives normalized group advantages from frozen-environment rewards, and applies the unchanged GRPO loss to all 3.086 billion parameters. Only schedule,…
GPT-5.6 Luna
Codex · high effort
Online boards produce actions and gym rewards, which are variance-filtered, group-normalized, and optimized with unchanged GRPO/AdamW on the full actor. The candidate adds a per-step board-potential difference; data, decoding, optimizer,…
GPT-5.6 Luna
Codex · low effort
The flow remained online boards and rollouts → frozen environment reward → reward-variance filtering → normalized clipped GRPO loss → AdamW full-model update → merged Hugging Face artifacts. Training signal, data generation, sampling,…
GPT-5.6 Luna
Codex · max effort
Baseline flow is: generated board, current-policy trajectories, sparse environment/format reward, within-board normalized GRPO advantage, clipped policy optimization, full-policy weights. Candidate flow is identical except AdamW starts at…
GPT-5.6 Luna
Codex · medium effort
Baseline flow is online board → self-generated model trajectories → fixed-environment solve reward plus format penalty → group-normalized, top-p-filtered clipped policy loss → full Qwen weight update. The candidate keeps that reward,…
GPT-5.6 Luna
Codex · none effort
Baseline flow is online one-box board, multi-turn policy sampling, frozen-environment return plus format penalty, same-board group normalization, then clipped-PPO full-weight optimization. The candidate intended to replace 35% of inputs…
GPT-5.6 Luna
Codex · xhigh effort
Fresh formal boards and trajectories feed native Sokoban returns into standardized GRPO advantages; reward-variance softmax top-p filtering at 0.9 selects rollouts, and seq-mean-token-mean loss with AdamW, learning rate 1e-6, and entropy…
GPT-5.6 Sol
Codex · high effort
Baseline flow is policy rollouts, environment success reward, then clipped GRPO updating the actor. Candidate flow is freshly generated boards, local breadth-first search for one-to-ten-action shortest paths, evaluator-format examples with…
GPT-5.6 Sol
Codex · low effort
Each round requests 32,768 fresh 6×6, one-box boards. Local breadth-first search finds a shortest path; unsolved or greater-than-10-step boards are discarded, while intermediate states receive the next one or two actions plus a fixed…
GPT-5.6 Sol
Codex · max effort
Unlike baseline reward-based GRPO, the candidate generates boards online, uses breadth-first search to label the next one or two shortest-path actions at every reachable state, and optimizes weighted response cross-entropy. Direction words…
GPT-5.6 Sol
Codex · medium effort
Baseline flow: generated board → sampled multi-turn policy actions → environment reward → filtered clipped GRPO loss → policy update and merged model. Candidate flow: generated 6×6 one-box board → BFS shortest path, split into two-action…
GPT-5.6 Sol
Codex · none effort
Formal replay still generated fresh boards, sampled policy trajectories, used frozen-environment success reward and format penalties, applied the original softmax top-p variance filter and normalized GRPO loss, and updated the full model.…
GPT-5.6 Sol
Codex · xhigh effort
Baseline flow is board → policy rollouts → sparse solve reward → clipped group-relative objective → full-policy update. Candidate flow is fresh online board → bundled BFS shortest path → online two-action and templated short-rationale…
GPT-5.6 Terra
Codex · high effort
On each training reset, breadth-first search computes shortest remaining distance over the player and box state. Each native step reward becomes native reward plus 0.20 (previous distance - current distance), capped at depth 32. The…
GPT-5.6 Terra
Codex · low effort
Its flow remained online board generation, policy rollouts, environment reward and format penalty, reward-variance group filtering, normalized GRPO advantages, clipped full-policy optimization, then sharded save and Hugging Face merge.…
GPT-5.6 Terra
Codex · max effort
The packaged generator creates sequential boards with rotation/mirror augmentation; BFS finds a legal solution of at most ten actions and formats evaluator-compatible multi-turn examples with at most two actions per turn. Prompt tokens are…
GPT-5.6 Terra
Codex · medium effort
The candidate still samples online boards and grouped trajectories, obtains action, box-position, completion, and formatting rewards from the frozen environment, normalizes group-relative policy optimization advantages, filters by reward…
GPT-5.6 Terra
Codex · none effort
Online board to policy trajectories to live environment signals to the unchanged clipped GRPO objective remains the full update flow; all Qwen weights are still trained. Only schedule and checkpoint policy changed: formal orchestration…
GPT-5.6 Terra
Codex · xhigh effort
Baseline flow is fresh board → policy rollouts → environment reward → GRPO advantages and clipped loss → updated full weights. The candidate adds an online breadth-first-search target: raw reward plus 0.5 × clip(distance_before −…
Kimi K3
Claude Code · max effort
Online boards feed trajectories; the unchanged environment supplies rewards plus -0.3 for malformed turns; unchanged group-relative advantages, variance filtering, and clipped GRPO update the full policy. No external labels,…
Claude Opus 5
Claude Code · high effort
Each member starts from the fixed backbone, LoRA seed 42, and zero scalar head; all 8,192 pairs are left-truncated to 1,024, then LoRA and head train for 128 steps with microbatch 8, accumulation 8, 1e-5, and seed 1001 upward. Labels and…
Claude Opus 5
Claude Code · low effort
The last 512 training pairs are internal validation; the first 7,680 train. Each independent cycle restarts fixed Mistral with a new seed and zero head, length-groups examples, and runs 360 Bradley–Terry steps plus…
Claude Opus 5
Claude Code · max effort
Each member shares the fixed backbone and initialization, sees all 8,192 pairs once in a different order, and updates rank-64 LoRA plus the scalar head with unchanged Bradley–Terry loss. Four members become one rank-256 adapter by stacking…
Claude Opus 5
Claude Code · medium effort
Baseline flow is preference pair → frozen backbone/LoRA/head scores → supplied chosen-over-rejected target → Bradley–Terry update → rank-128 adapter and head. The candidate cycles five duration/learning-rate recipes across data/dropout…
Claude Opus 5
Claude Code · xhigh effort
Each member left-truncates all fixed pairs to 1,024 tokens and minimizes pair-weighted -log sigmoid(r_chosen-r_rejected) while training the same LoRA modules and scalar head. Labels, backbone, and core pairwise update form remain;…
Claude Sonnet 5
Claude Code · high effort
Fixed pair → shared Mistral/LoRA/head scores both responses → the chosen-over-rejected label supplies the target → unchanged Bradley–Terry loss and AdamW update LoRA plus the head. The candidate changes only learning-rate timing,…
Claude Sonnet 5
Claude Code · low effort
Every seed reloads the fixed base and a fresh scalar head, scores both answers, and uses the unchanged preference ordering and Bradley–Terry loss. Paged AdamW 32-bit, learning rate 5e-6, four warmup steps, cosine decay, LoRA targets, and…
Claude Sonnet 5
Claude Code · max effort
A salted row hash assigns 7,782 pairs to training and 410 to internal validation. Each seed reloads the fixed base, fresh head, and fresh LoRA factors, then uses the unchanged loss and 252-step recipe. Full-holdout tie-adjusted accuracy…
Claude Sonnet 5
Claude Code · medium effort
Baseline flow: preference pair → two scalar scores → original preference label → Bradley–Terry loss and AdamW → LoRA plus scalar-head updates. Candidate flow: make a fixed 7,680/512 split → repeat that unchanged training rule from fresh…
Claude Sonnet 5
Claude Code · xhigh effort
Each member starts fresh from the pinned base with seed 42+m, fresh LoRA and scalar head, a fresh optimizer, and the same 8,192 pairs. Its supervision and Bradley-Terry update rule are unchanged. After training, LoRA is merged into the…
GPT-5.6 Luna
Codex · high effort
Both methods tokenize the fixed pairs, emit two scalar rewards, use the original preference as the target, and update the same rank-128 LoRA and scalar head with paged AdamW. The candidate changes the loss to -log…
GPT-5.6 Luna
Codex · low effort
Fixed chat formatting and 4,096-token left truncation turn each chosen/rejected pair into two conversations. The frozen Mistral backbone, zero-dropout LoRA modules, and new scalar head score both; dataset preference order supplies labels,…
GPT-5.6 Luna
Codex · max effort
Each fixed pair is rendered with the same template, passed through frozen Mistral plus LoRA, and mapped to two scalar rewards. The chosen rejected order supplies the Bradley–Terry target; logistic loss updates LoRA and the scalar head.…
GPT-5.6 Luna
Codex · medium effort
Fixed pairs are chat-formatted and left-truncated at 4096 tokens; the model emits chosen/rejected scalars; fixed labels define the ordering; BT loss -log sigmoid(r_chosen-r_rejected) is optimized with paged 32-bit AdamW; 335,548,416…
GPT-5.6 Luna
Codex · none effort
Baseline and candidate both render and truncate each pair, assign two scalar rewards, use the preferred/rejected label in the same pairwise logistic loss, and update only LoRA plus the scalar head with AdamW. The candidate only changes…
GPT-5.6 Luna
Codex · xhigh effort
Both baseline and candidate map each chosen/rejected pair through the fixed Mistral backbone, use the original preference as the label, minimize unsmoothed Bradley–Terry loss, and update only LoRA plus the scalar head. Active candidate…
GPT-5.6 Sol
Codex · high effort
Each preference pair yields chosen and rejected scalar rewards; the unchanged loss is -log sigmoid(r_chosen-r_rejected), and only LoRA and the scalar head are optimized. No labels, synthetic data, external model, or reasoning traces were…
GPT-5.6 Sol
Codex · low effort
Baseline flow is preference pair → two scalar rewards from the same model → original chosen/rejected label → unsmoothed Bradley–Terry loss and paged AdamW → updated LoRA and scalar head. The candidate keeps that signal, data, objective,…
GPT-5.6 Sol
Codex · max effort
Baseline flow is fixed pairs → frozen Mistral plus scalar scores → chosen-is-better labels → hard pairwise logistic loss → LoRA and scalar-head updates. The anchor keeps that signal and update rule, changing only within-update ordering and…
GPT-5.6 Sol
Codex · medium effort
Baseline flow is pair → two scalar rewards → fixed chosen/rejected target → Bradley–Terry loss → LoRA and scalar-head updates. The candidate changes truncation, batching, grouped sampling, scheduling, engineering, and checkpoint policy,…
GPT-5.6 Sol
Codex · none effort
Baseline and candidate take a fixed chosen/rejected pair, produce two scalar rewards, apply its label through unchanged BT loss and AdamW, and update LoRA plus score head. The candidate shortens truncation from 4,096 to 1,024, changes…
GPT-5.6 Sol
Codex · xhigh effort
Baseline flow is fixed preference pair → two scalar rewards → chosen/rejected label → Bradley–Terry loss → LoRA/head updates. The candidate preserves that target, loss, optimizer, 4,096-token context, and effective batch. It trains a…
GPT-5.6 Terra
Codex · high effort
Fixed pairs are chat-formatted and left-truncated to 1,024; fixed Mistral plus LoRA/head produces two rewards; the fixed chosen-over-rejected label drives pure BT with temperature 1, margin 0, global batch 64, 5e-6 cosine learning rate,…
GPT-5.6 Terra
Codex · low effort
The baseline maps each fixed chosen/rejected pair to two scalar scores and optimizes their difference with Bradley–Terry loss while updating LoRA and the scalar head. The candidate keeps the same preference signal, data ordering, loss,…
GPT-5.6 Terra
Codex · max effort
Baseline flow is preference pair → two scalar rewards → dataset preference label → exact Bradley–Terry loss → updated LoRA and scalar head. The candidate first hash-holds out 410 pairs, trains each seed 42–56 on 7,782 pairs for 189…
GPT-5.6 Terra
Codex · medium effort
Both baseline and candidate map each answer through the fixed backbone and scalar head, use the dataset preference as the label, minimize the same pairwise loss, and update only LoRA plus the head. The candidate changes only schedule and…
GPT-5.6 Terra
Codex · none effort
Both recipes encode each fixed pair, produce two Mistral scalar scores, derive the target order from the fixed preference label, and update LoRA plus the scalar head with the same Bradley–Terry loss and AdamW configuration. The candidate…
GPT-5.6 Terra
Codex · xhigh effort
The full Mistral model scores chat-templated pairs; fixed chosen/rejected labels feed unchanged BT and paged AdamW; all parameters and the scalar head update. No generated labels, synthetic data, external model, rationale, or explore…
Kimi K3
Claude Code · max effort
Baseline flow is fixed pair → two scalar rewards → chosen/rejected label → Bradley-Terry optimization → LoRA and scalar head. The candidate keeps that update, uses 2,048-token length-grouped batches, and trains five center seeds plus…
| Task | System | Harness | Effort | Outcome | Best score |
|---|---|---|---|---|---|
| DDPO | Claude Opus 5 | Claude Code | high | Completed | 8.4185 |
| DDPO | Claude Opus 5 | Claude Code | low | Completed | 12.134 |
| DDPO | Claude Opus 5 | Claude Code | max | Completed | 17.668 |
| DDPO | Claude Opus 5 | Claude Code | medium | Completed | 8.9804 |
| DDPO | Claude Opus 5 | Claude Code | xhigh | Completed | 14.446 |
| DDPO | Claude Sonnet 5 | Claude Code | high | Completed | 5.8233 |
| DDPO | Claude Sonnet 5 | Claude Code | low | Completed | 6.1309 |
| DDPO | Claude Sonnet 5 | Claude Code | max | Completed | 6.1536 |
| DDPO | Claude Sonnet 5 | Claude Code | medium | Completed | 5.8642 |
| DDPO | Claude Sonnet 5 | Claude Code | xhigh | Completed | 4.4878 |
| DDPO | GPT-5.6 Luna | Codex | high | Completed | 5.6178 |
| DDPO | GPT-5.6 Luna | Codex | low | Completed | 5.5413 |
| DDPO | GPT-5.6 Luna | Codex | max | Completed | 6.5327 |
| DDPO | GPT-5.6 Luna | Codex | medium | Completed | 5.5574 |
| DDPO | GPT-5.6 Luna | Codex | none | Completed | 5.1101 |
| DDPO | GPT-5.6 Luna | Codex | xhigh | Completed | 5.7388 |
| DDPO | GPT-5.6 Sol | Codex | high | Completed | 5.7702 |
| DDPO | GPT-5.6 Sol | Codex | low | Terminal | Unavailable |
| DDPO | GPT-5.6 Sol | Codex | max | Completed | 8.982 |
| DDPO | GPT-5.6 Sol | Codex | medium | Completed | 6.8344 |
| DDPO | GPT-5.6 Sol | Codex | none | Completed | 5.822 |
| DDPO | GPT-5.6 Sol | Codex | xhigh | Completed | 5.9235 |
| DDPO | GPT-5.6 Terra | Codex | high | Completed | 6.6246 |
| DDPO | GPT-5.6 Terra | Codex | low | Completed | 5.5413 |
| DDPO | GPT-5.6 Terra | Codex | max | Completed | 6.0074 |
| DDPO | GPT-5.6 Terra | Codex | medium | Completed | 5.8522 |
| DDPO | GPT-5.6 Terra | Codex | none | Completed | 5.5413 |
| DDPO | GPT-5.6 Terra | Codex | xhigh | Completed | 5.9558 |
| DDPO | Kimi K3 | Claude Code | max | Completed | 9.0433 |
| DiGress | Claude Opus 5 | Claude Code | high | Completed | 65.660 |
| DiGress | Claude Opus 5 | Claude Code | low | Completed | 65.380 |
| DiGress | Claude Opus 5 | Claude Code | max | Completed | 64.850 |
| DiGress | Claude Opus 5 | Claude Code | medium | Completed | 65.840 |
| DiGress | Claude Opus 5 | Claude Code | xhigh | Completed | 65.790 |
| DiGress | Claude Sonnet 5 | Claude Code | high | Completed | 66.470 |
| DiGress | Claude Sonnet 5 | Claude Code | low | Completed | 66.320 |
| DiGress | Claude Sonnet 5 | Claude Code | max | Completed | 66.170 |
| DiGress | Claude Sonnet 5 | Claude Code | medium | Completed | 65.360 |
| DiGress | Claude Sonnet 5 | Claude Code | xhigh | Completed | 66.640 |
| DiGress | GPT-5.6 Luna | Codex | high | Completed | 67.390 |
| DiGress | GPT-5.6 Luna | Codex | low | Source unavailable | Unavailable |
| DiGress | GPT-5.6 Luna | Codex | max | Completed | 65.300 |
| DiGress | GPT-5.6 Luna | Codex | medium | Completed | 66.170 |
| DiGress | GPT-5.6 Luna | Codex | none | Terminal | Unavailable |
| DiGress | GPT-5.6 Luna | Codex | xhigh | Completed | 68.310 |
| DiGress | GPT-5.6 Sol | Codex | high | Completed | 66.030 |
| DiGress | GPT-5.6 Sol | Codex | low | Completed | 66.880 |
| DiGress | GPT-5.6 Sol | Codex | max | Completed | 65.940 |
| DiGress | GPT-5.6 Sol | Codex | medium | Completed | 65.520 |
| DiGress | GPT-5.6 Sol | Codex | none | Completed | 67.010 |
| DiGress | GPT-5.6 Sol | Codex | xhigh | Completed | 63.720 |
| DiGress | GPT-5.6 Terra | Codex | high | Terminal | Unavailable |
| DiGress | GPT-5.6 Terra | Codex | low | Completed | 65.350 |
| DiGress | GPT-5.6 Terra | Codex | max | Completed | 69.880 |
| DiGress | GPT-5.6 Terra | Codex | medium | Completed | 64.330 |
| DiGress | GPT-5.6 Terra | Codex | none | Source unavailable | Unavailable |
| DiGress | GPT-5.6 Terra | Codex | xhigh | Completed | 65.620 |
| DiGress | Kimi K3 | Claude Code | max | Completed | 65.240 |
| DPO | Claude Opus 5 | Claude Code | high | Completed | 0.615 |
| DPO | Claude Opus 5 | Claude Code | low | Completed | 0.4673 |
| DPO | Claude Opus 5 | Codex | max | Completed | 0.477 |
| DPO | Claude Opus 5 | Claude Code | medium | Completed | 0.6223 |
| DPO | Claude Opus 5 | Claude Code | xhigh | Completed | 0.4213 |
| DPO | Claude Sonnet 5 | Claude Code | high | Completed | 0.54 |
| DPO | Claude Sonnet 5 | Claude Code | low | Completed | 0.4044 |
| DPO | Claude Sonnet 5 | Claude Code | max | Completed | 0.5424 |
| DPO | Claude Sonnet 5 | Claude Code | medium | Completed | 0.3995 |
| DPO | Claude Sonnet 5 | Claude Code | xhigh | Completed | 0.5061 |
| DPO | GPT-5.6 Luna | Codex | high | Completed | 0.4019 |
| DPO | GPT-5.6 Luna | Codex | low | Completed | 0.4722 |
| DPO | GPT-5.6 Luna | Codex | max | Completed | 0.5327 |
| DPO | GPT-5.6 Luna | Codex | medium | Completed | 0.4915 |
| DPO | GPT-5.6 Luna | Codex | none | Completed | 0.4068 |
| DPO | GPT-5.6 Luna | Codex | xhigh | Completed | 0.5642 |
| DPO | GPT-5.6 Sol | Codex | high | Completed | 0.4576 |
| DPO | GPT-5.6 Sol | Codex | low | Completed | 0.431 |
| DPO | GPT-5.6 Sol | Codex | max | Completed | 0.4358 |
| DPO | GPT-5.6 Sol | Codex | medium | Completed | 0.4939 |
| DPO | GPT-5.6 Sol | Codex | none | Completed | 0.3995 |
| DPO | GPT-5.6 Sol | Codex | xhigh | Completed | 0.4818 |
| DPO | GPT-5.6 Terra | Codex | high | Completed | 0.4939 |
| DPO | GPT-5.6 Terra | Codex | low | Completed | 0.4213 |
| DPO | GPT-5.6 Terra | Codex | max | Completed | 0.46 |
| DPO | GPT-5.6 Terra | Codex | medium | Completed | 0.4479 |
| DPO | GPT-5.6 Terra | Codex | none | Completed | 0.4431 |
| DPO | GPT-5.6 Terra | Codex | xhigh | Completed | 0.4383 |
| DPO | Kimi K3 | Claude Code | max | Completed | 0.5424 |
| Model Soup | Claude Opus 5 | Claude Code | high | Completed | 0.6935 |
| Model Soup | Claude Opus 5 | Claude Code | low | Completed | 0.6961 |
| Model Soup | Claude Opus 5 | Codex | max | Completed | 0.6936 |
| Model Soup | Claude Opus 5 | Claude Code | medium | Completed | 0.6959 |
| Model Soup | Claude Opus 5 | Claude Code | xhigh | Completed | 0.6959 |
| Model Soup | Claude Sonnet 5 | Claude Code | high | Completed | 0.6942 |
| Model Soup | Claude Sonnet 5 | Claude Code | low | Completed | 0.6983 |
| Model Soup | Claude Sonnet 5 | Claude Code | max | Completed | 0.6985 |
| Model Soup | Claude Sonnet 5 | Claude Code | medium | Completed | 0.6927 |
| Model Soup | Claude Sonnet 5 | Claude Code | xhigh | Completed | 0.6936 |
| Model Soup | GPT-5.6 Luna | Codex | high | Completed | 0.6943 |
| Model Soup | GPT-5.6 Luna | Codex | low | Completed | 0.6936 |
| Model Soup | GPT-5.6 Luna | Codex | max | Completed | 0.6941 |
| Model Soup | GPT-5.6 Luna | Codex | medium | Completed | 0.6936 |
| Model Soup | GPT-5.6 Luna | Codex | none | Completed | 0.6937 |
| Model Soup | GPT-5.6 Luna | Codex | xhigh | Completed | 0.6924 |
| Model Soup | GPT-5.6 Sol | Codex | high | Completed | 0.6972 |
| Model Soup | GPT-5.6 Sol | Codex | low | Completed | 0.6941 |
| Model Soup | GPT-5.6 Sol | Codex | max | Completed | 0.7006 |
| Model Soup | GPT-5.6 Sol | Codex | medium | Completed | 0.695 |
| Model Soup | GPT-5.6 Sol | Codex | none | Completed | 0.694 |
| Model Soup | GPT-5.6 Sol | Codex | xhigh | Completed | 0.6918 |
| Model Soup | GPT-5.6 Terra | Codex | high | Completed | 0.6937 |
| Model Soup | GPT-5.6 Terra | Codex | low | Completed | 0.6936 |
| Model Soup | GPT-5.6 Terra | Codex | max | Completed | 0.6936 |
| Model Soup | GPT-5.6 Terra | Codex | medium | Completed | 0.6942 |
| Model Soup | GPT-5.6 Terra | Codex | none | Source unavailable | Unavailable |
| Model Soup | GPT-5.6 Terra | Codex | xhigh | Completed | 0.6938 |
| Model Soup | Kimi K3 | Claude Code | max | Completed | 0.6935 |
| OPD | Claude Opus 5 | Claude Code | high | Completed | 0.449 |
| OPD | Claude Opus 5 | Claude Code | low | Completed | 0.4214 |
| OPD | Claude Opus 5 | Claude Code | max | Completed | 0.3922 |
| OPD | Claude Opus 5 | Claude Code | medium | Completed | 0.4323 |
| OPD | Claude Opus 5 | Claude Code | xhigh | Completed | 0.4401 |
| OPD | Claude Sonnet 5 | Claude Code | high | Completed | 0.4266 |
| OPD | Claude Sonnet 5 | Claude Code | low | Completed | 0.4318 |
| OPD | Claude Sonnet 5 | Claude Code | max | Completed | 0.4276 |
| OPD | Claude Sonnet 5 | Claude Code | medium | Completed | 0.4266 |
| OPD | Claude Sonnet 5 | Claude Code | xhigh | Completed | 0.4214 |
| OPD | GPT-5.6 Luna | Codex | high | Completed | 0.4245 |
| OPD | GPT-5.6 Luna | Codex | low | Source unavailable | Unavailable |
| OPD | GPT-5.6 Luna | Codex | max | Completed | 0.4333 |
| OPD | GPT-5.6 Luna | Codex | medium | Completed | 0.4375 |
| OPD | GPT-5.6 Luna | Codex | none | Completed | 0.4323 |
| OPD | GPT-5.6 Luna | Codex | xhigh | Completed | 0.4266 |
| OPD | GPT-5.6 Sol | Codex | high | Completed | 0.4245 |
| OPD | GPT-5.6 Sol | Codex | low | Completed | 0.437 |
| OPD | GPT-5.6 Sol | Codex | max | Completed | 0.4292 |
| OPD | GPT-5.6 Sol | Codex | medium | Completed | 0.4302 |
| OPD | GPT-5.6 Sol | Codex | none | Completed | 0.4255 |
| OPD | GPT-5.6 Sol | Codex | xhigh | Completed | 0.4177 |
| OPD | GPT-5.6 Terra | Codex | high | Source unavailable | Unavailable |
| OPD | GPT-5.6 Terra | Codex | low | Source unavailable | Unavailable |
| OPD | GPT-5.6 Terra | Codex | max | Completed | 0.4349 |
| OPD | GPT-5.6 Terra | Codex | medium | Completed | 0.4427 |
| OPD | GPT-5.6 Terra | Codex | none | Source unavailable | Unavailable |
| OPD | GPT-5.6 Terra | Codex | xhigh | Completed | 0.4328 |
| OPD | Kimi K3 | Claude Code | max | Completed | 0.4052 |
| OpenR1 | Claude Opus 5 | Claude Code | high | Completed | 0.125 |
| OpenR1 | Claude Opus 5 | Claude Code | low | Completed | 0.1328 |
| OpenR1 | Claude Opus 5 | Claude Code | max | Completed | 0.1094 |
| OpenR1 | Claude Opus 5 | Claude Code | medium | Completed | 0.125 |
| OpenR1 | Claude Opus 5 | Claude Code | xhigh | Completed | 0.125 |
| OpenR1 | Claude Sonnet 5 | Claude Code | high | Completed | 0.125 |
| OpenR1 | Claude Sonnet 5 | Claude Code | low | Completed | 0.125 |
| OpenR1 | Claude Sonnet 5 | Claude Code | max | Completed | 0.1172 |
| OpenR1 | Claude Sonnet 5 | Codex | medium | Recipe failure | Unavailable |
| OpenR1 | Claude Sonnet 5 | Claude Code | xhigh | Completed | 0.1172 |
| OpenR1 | GPT-5.6 Luna | Codex | high | Completed | 0.1172 |
| OpenR1 | GPT-5.6 Luna | Codex | low | Completed | 0.0859 |
| OpenR1 | GPT-5.6 Luna | Codex | max | Completed | 0.1094 |
| OpenR1 | GPT-5.6 Luna | Codex | medium | Completed | 0.1016 |
| OpenR1 | GPT-5.6 Luna | Codex | none | Completed | 0.1094 |
| OpenR1 | GPT-5.6 Luna | Codex | xhigh | Completed | 0.1094 |
| OpenR1 | GPT-5.6 Sol | Codex | high | Completed | 0.125 |
| OpenR1 | GPT-5.6 Sol | Codex | low | Completed | 0.1172 |
| OpenR1 | GPT-5.6 Sol | Codex | max | Completed | 0.1172 |
| OpenR1 | GPT-5.6 Sol | Codex | medium | Completed | 0.1328 |
| OpenR1 | GPT-5.6 Sol | Codex | none | Completed | 0.125 |
| OpenR1 | GPT-5.6 Sol | Codex | xhigh | Completed | 0.125 |
| OpenR1 | GPT-5.6 Terra | Codex | high | Completed | 0.1016 |
| OpenR1 | GPT-5.6 Terra | Codex | low | Completed | 0.1172 |
| OpenR1 | GPT-5.6 Terra | Codex | max | Completed | 0.1172 |
| OpenR1 | GPT-5.6 Terra | Codex | medium | Completed | 0.1172 |
| OpenR1 | GPT-5.6 Terra | Codex | none | Completed | 0.1016 |
| OpenR1 | GPT-5.6 Terra | Codex | xhigh | Completed | 0.125 |
| OpenR1 | Kimi K3 | Claude Code | max | Completed | 0.0938 |
| NPO | Claude Opus 5 | Claude Code | high | Completed | 1 |
| NPO | Claude Opus 5 | Claude Code | low | Completed | 0.996 |
| NPO | Claude Opus 5 | Claude Code | max | Completed | 1 |
| NPO | Claude Opus 5 | Claude Code | medium | Completed | 1 |
| NPO | Claude Opus 5 | Claude Code | xhigh | Completed | 1 |
| NPO | Claude Sonnet 5 | Claude Code | high | Completed | 0.7286 |
| NPO | Claude Sonnet 5 | Claude Code | low | Completed | 0.9685 |
| NPO | Claude Sonnet 5 | Claude Code | max | Completed | 0.9436 |
| NPO | Claude Sonnet 5 | Claude Code | medium | Completed | 0.969 |
| NPO | Claude Sonnet 5 | Codex | xhigh | Completed | 0.9628 |
| NPO | GPT-5.6 Luna | Codex | high | Completed | 0.988 |
| NPO | GPT-5.6 Luna | Codex | low | Terminal | Unavailable |
| NPO | GPT-5.6 Luna | Codex | max | Completed | 0.9275 |
| NPO | GPT-5.6 Luna | Codex | medium | Completed | 0.7373 |
| NPO | GPT-5.6 Luna | Codex | none | Completed | 0.9359 |
| NPO | GPT-5.6 Luna | Codex | xhigh | Completed | 0.951 |
| NPO | GPT-5.6 Sol | Codex | high | Completed | 0.9659 |
| NPO | GPT-5.6 Sol | Codex | low | Completed | 0.7422 |
| NPO | GPT-5.6 Sol | Codex | max | Completed | 0.9827 |
| NPO | GPT-5.6 Sol | Codex | medium | Completed | 0.9929 |
| NPO | GPT-5.6 Sol | Codex | none | Completed | 0.9772 |
| NPO | GPT-5.6 Sol | Codex | xhigh | Completed | 1 |
| NPO | GPT-5.6 Terra | Codex | high | Completed | 0.9922 |
| NPO | GPT-5.6 Terra | Codex | low | Completed | 0.9521 |
| NPO | GPT-5.6 Terra | Codex | max | Completed | 0.9538 |
| NPO | GPT-5.6 Terra | Codex | medium | Completed | 0.9045 |
| NPO | GPT-5.6 Terra | Codex | none | Completed | 0.7104 |
| NPO | GPT-5.6 Terra | Codex | xhigh | Completed | 0.9228 |
| NPO | Kimi K3 | Claude Code | max | Completed | 0.9815 |
| OWL | Claude Opus 5 | Claude Code | high | Completed | 13.246 |
| OWL | Claude Opus 5 | Claude Code | low | Completed | 12.962 |
| OWL | Claude Opus 5 | Claude Code | max | Completed | 13.041 |
| OWL | Claude Opus 5 | Codex | medium | Completed | 13.383 |
| OWL | Claude Opus 5 | Claude Code | xhigh | Completed | 13.307 |
| OWL | Claude Sonnet 5 | Claude Code | high | Completed | 21.678 |
| OWL | Claude Sonnet 5 | Claude Code | low | Completed | 53.997 |
| OWL | Claude Sonnet 5 | Claude Code | max | Completed | 14.255 |
| OWL | Claude Sonnet 5 | Claude Code | medium | Completed | 19.741 |
| OWL | Claude Sonnet 5 | Claude Code | xhigh | Completed | 21.239 |
| OWL | GPT-5.6 Luna | Codex | high | Completed | 29.242 |
| OWL | GPT-5.6 Luna | Codex | low | Completed | 50.996 |
| OWL | GPT-5.6 Luna | Codex | max | Completed | 19.673 |
| OWL | GPT-5.6 Luna | Codex | medium | Completed | -24.310 |
| OWL | GPT-5.6 Luna | Codex | none | Completed | 42.898 |
| OWL | GPT-5.6 Luna | Codex | xhigh | Completed | 16.162 |
| OWL | GPT-5.6 Sol | Codex | high | Completed | 13.253 |
| OWL | GPT-5.6 Sol | Codex | low | Completed | 14.898 |
| OWL | GPT-5.6 Sol | Codex | max | Completed | 17.405 |
| OWL | GPT-5.6 Sol | Codex | medium | Completed | 15.754 |
| OWL | GPT-5.6 Sol | Codex | none | Completed | 20.850 |
| OWL | GPT-5.6 Sol | Codex | xhigh | Completed | 13.850 |
| OWL | GPT-5.6 Terra | Codex | high | Completed | 15.236 |
| OWL | GPT-5.6 Terra | Codex | low | Completed | 20.370 |
| OWL | GPT-5.6 Terra | Codex | max | Completed | 15.155 |
| OWL | GPT-5.6 Terra | Codex | medium | Completed | 20.176 |
| OWL | GPT-5.6 Terra | Codex | none | Source unavailable | Unavailable |
| OWL | GPT-5.6 Terra | Codex | xhigh | Completed | 16.153 |
| OWL | Kimi K3 | Claude Code | max | Completed | 14.257 |
| RAGEN | Claude Opus 5 | Claude Code | high | Completed | 1 |
| RAGEN | Claude Opus 5 | Claude Code | low | Completed | 1 |
| RAGEN | Claude Opus 5 | Claude Code | max | Completed | 0.2344 |
| RAGEN | Claude Opus 5 | Claude Code | medium | Completed | 1 |
| RAGEN | Claude Opus 5 | Claude Code | xhigh | Completed | 0.2109 |
| RAGEN | Claude Sonnet 5 | Claude Code | high | Completed | 0.8945 |
| RAGEN | Claude Sonnet 5 | Claude Code | low | Completed | 0.043 |
| RAGEN | Claude Sonnet 5 | Claude Code | max | Completed | 0.2324 |
| RAGEN | Claude Sonnet 5 | Claude Code | medium | Completed | 0 |
| RAGEN | Claude Sonnet 5 | Codex | xhigh | Completed | 0.2402 |
| RAGEN | GPT-5.6 Luna | Codex | high | Completed | 0.125 |
| RAGEN | GPT-5.6 Luna | Codex | low | Completed | 0.1543 |
| RAGEN | GPT-5.6 Luna | Codex | max | Terminal | Unavailable |
| RAGEN | GPT-5.6 Luna | Codex | medium | Terminal | Unavailable |
| RAGEN | GPT-5.6 Luna | Codex | none | Terminal | Unavailable |
| RAGEN | GPT-5.6 Luna | Codex | xhigh | Terminal | Unavailable |
| RAGEN | GPT-5.6 Sol | Codex | high | Completed | 1 |
| RAGEN | GPT-5.6 Sol | Codex | low | Completed | 0.8828 |
| RAGEN | GPT-5.6 Sol | Codex | max | Completed | 1 |
| RAGEN | GPT-5.6 Sol | Codex | medium | Terminal | Unavailable |
| RAGEN | GPT-5.6 Sol | Codex | none | Completed | 0.0918 |
| RAGEN | GPT-5.6 Sol | Codex | xhigh | Completed | 0.5488 |
| RAGEN | GPT-5.6 Terra | Codex | high | Completed | 0.0859 |
| RAGEN | GPT-5.6 Terra | Codex | low | Terminal | Unavailable |
| RAGEN | GPT-5.6 Terra | Codex | max | Completed | 0.998 |
| RAGEN | GPT-5.6 Terra | Codex | medium | Completed | 0.1836 |
| RAGEN | GPT-5.6 Terra | Codex | none | Completed | 0.2207 |
| RAGEN | GPT-5.6 Terra | Codex | xhigh | Completed | 0.2656 |
| RAGEN | Kimi K3 | Claude Code | max | Completed | 0.1855 |
| BTRM | Claude Opus 5 | Claude Code | high | Completed | 71.948 |
| BTRM | Claude Opus 5 | Claude Code | low | Completed | 71.636 |
| BTRM | Claude Opus 5 | Claude Code | max | Completed | 75.750 |
| BTRM | Claude Opus 5 | Claude Code | medium | Completed | 77.093 |
| BTRM | Claude Opus 5 | Claude Code | xhigh | Completed | 64.351 |
| BTRM | Claude Sonnet 5 | Claude Code | high | Completed | 74.535 |
| BTRM | Claude Sonnet 5 | Claude Code | low | Completed | 69.266 |
| BTRM | Claude Sonnet 5 | Claude Code | max | Completed | 74.889 |
| BTRM | Claude Sonnet 5 | Claude Code | medium | Completed | 71.490 |
| BTRM | Claude Sonnet 5 | Claude Code | xhigh | Completed | 38.325 |
| BTRM | GPT-5.6 Luna | Codex | high | Completed | 73.975 |
| BTRM | GPT-5.6 Luna | Codex | low | Completed | 74.495 |
| BTRM | GPT-5.6 Luna | Codex | max | Completed | 74.362 |
| BTRM | GPT-5.6 Luna | Codex | medium | Completed | 75.837 |
| BTRM | GPT-5.6 Luna | Codex | none | Completed | 75.339 |
| BTRM | GPT-5.6 Luna | Codex | xhigh | Completed | 75.431 |
| BTRM | GPT-5.6 Sol | Codex | high | Completed | 73.987 |
| BTRM | GPT-5.6 Sol | Codex | low | Completed | 73.951 |
| BTRM | GPT-5.6 Sol | Codex | max | Completed | 73.885 |
| BTRM | GPT-5.6 Sol | Codex | medium | Completed | 73.686 |
| BTRM | GPT-5.6 Sol | Codex | none | Completed | 75.168 |
| BTRM | GPT-5.6 Sol | Codex | xhigh | Completed | 73.843 |
| BTRM | GPT-5.6 Terra | Codex | high | Completed | 75.416 |
| BTRM | GPT-5.6 Terra | Codex | low | Completed | 72.919 |
| BTRM | GPT-5.6 Terra | Codex | max | Completed | 66.934 |
| BTRM | GPT-5.6 Terra | Codex | medium | Completed | 74.085 |
| BTRM | GPT-5.6 Terra | Codex | none | Completed | 75.210 |
| BTRM | GPT-5.6 Terra | Codex | xhigh | Completed | 70.890 |
| BTRM | Kimi K3 | Claude Code | max | Completed | 74.121 |