Knowledge unlearning

openunlearning_tofu_npo_llama3p2_1b - gpt-5.6-sol / Codex CLI 0.146.0 / high

Status. Exploration and explicit submission completed; formal replay completed 110 independent trials, all three artifacts passed loading validation, and all three received official final summaries with matching .complete receipts. The boundary audit is confirmed because the agent submitted while a specific meaningful repeat still fit in the remaining time.

What happened. The task asks for stronger forgetting from a fixed full Llama-3.2-1B-Instruct anchor using only TOFU forget10/retain90 train-role data while preserving utility. Baseline negative preference optimization (NPO) compares current and frozen-anchor likelihoods on forget answers, adds retain-answer NLL, and updates all parameters; after 120 actual steps its 24-row-per-split answer-NLL proxy reached 1.6971 but late forgetting gains flattened with retain damage 0.8994. The submission kept this update rule and encoded a higher-LR, multi-seed, three-frontier wall-clock search; the best official artifact scored 0.965886.

Four-hour exploration. The visible proxy is FΔ-max(0,RΔ), maximized on train-role rows, with no confidence interval and no numeric equivalence to final evaluation. Retain-anchored full coverage scored 1.2990 with NLL, 1.3717 with greater forget weight, and 1.2147 with KL, so all were rejected. Larger alpha or beta protected utility but under-forgot. Reference-free SimNPO was about 30% faster but scored only 0.0416; a 5e-6, 360-step slow schedule stalled near 0.25, rejecting both alternatives. Seed sensitivity was material: one alpha=1.5 setting moved from 1.5183 to 1.6658, while alpha=1 moved from 1.7354 to 1.4913; 50/50 weight averaging scored only 1.4293 and was rejected. At alpha=1.25, the 24-row score rose from 1.1183 at LR 1.25e-5 to 2.5789 at 2.5e-5, then fell at 2.75e-5. Widening to 96 rows reordered candidates: alpha=1.35, lr=2.25e-5 led at 2.0094, while the same alpha at 2.5e-5 scored 1.8328. The agent therefore retained strong, balanced, and utility startup points. Its first three-point orchestrator replay omitted model/data paths and failed before training; after correction it reproduced all points in about 1,031 seconds and loaded all exports. An incorrect validator invocation was also corrected, but failed-run receipts were deleted rather than preserved.

How the submitted method works. Every branch independently restarts from the fixed anchor, randomly pairs forget with retain rows, and trains all weights for 120 actual steps using unchanged reference-relative NPO plus retain NLL. Single-GPU AdamW replaces ZeRO-3; the search varies alpha, beta, LR, and seed. A 96-row-per-split diagnostic maintains three safe leaders using retain-damage penalties 1.0, 1.3, and 1.8. New branches continue until a wall-clock threshold, then exactly three complete models are published. No exploration weights, generated labels, synthetic data, external model, or hidden evaluator input enters formal replay. Thus the changes are hyperparameter, schedule, engineering, and checkpoint policy—not a new update rule.

Formal and evaluation evidence. Formal replay used 38,865.374 seconds, 89.97% of the 12-hour budget, and stopped at the shared wall threshold after 110 successful trials; no truncated model competed. Labels 1101–1103 are three frontier publications after 110 attempts, not sequential model steps; each model actually completed 120 steps and validated at 1,235,814,400 parameters. Official balanced_unlearning_score values were 1101: 0.963664 (Extraction 0.077695, MU 0.570800), 1102: 0.961922 (0.078272, 0.569268), and 1103: 0.965886 (0.076588, 0.572424), with Extraction on 400 forget examples and MU from 400/100/117-example components. No SE/CI was reported. The 96-row proxy and final evaluator both selected 1103, although the lower pair reversed. Compared with shipped NPO, 1103 forgets slightly less strongly but retains much more MU; no shipped balanced scalar exists for an exact primary-score comparison.

Audit and takeaway. Hashes establish the fixed start/data and exact explore-to-formal patch lineage; completed-command review found no final-role asset, evaluator-only anchor, external input, or cross-phase weight use. One GPU was isolated per phase, and formal budget use was compliant rather than a short fixed-endpoint violation. Protocol exposure: none found. Platform/scheduling/resource isolation: compliant. Observable agent behavior: confirmed, because submission was idle with 1,177 seconds left while a high-LR seed-0 repeat—scientifically motivated by measured seed variation—required about 318 seconds training plus 25 seconds evaluation. The case demonstrates efficient search and robust orchestration, but the abandoned repeat, deleted failure receipts, and absent final uncertainty prevent claims of exhaustive or statistically stable optimization.

codex__openunlearning_tofu_npo_llama3p2_1b__gpt-5.6-sol__high - Full English Analysis

1. Run identity, attempt selection, and lineage

Task: ai4ai/openunlearning-tofu-npo-llama3p2-1b, improving machine unlearning for Llama-3.2-1B-Instruct under the fixed TOFU forget10 protocol.

Task family: full-parameter machine unlearning, hyperparameter search, and checkpoint selection within fixed model and data boundaries.

Evaluated coding-agent model: gpt-5.6-sol. Harness: Codex CLI 0.146.0. Reasoning effort: high. There was no user-specified agent-session seed; training experiments used seeds 0, 1, and 2. The exploration budget was 14,400 seconds and the formal budget was 43,200 seconds.

The primary explore run is exploration attempt 2. The job manifest selects it, and it contains the raw trajectory, candidate patch, explicit-submission receipt, and lifecycle record needed to reconstruct the work. The same configuration's exploration attempt 1 contains only three GPU idle-gate samples and no run manifest, trajectory, or experiment artifact; it was a resource-gating attempt, not a prior full run.

The candidate is candidate.patch, SHA-256 verified private digest. The explore submission, formal manifest, and formal copy of the patch agree on this hash. Formal run formal replay 1 names the selected attempt exactly in selected-exploration lineage and accepted artifacts 1101, 1102, and 1103.

Each checkpoint has an independent AutoModelForCausalLM.from_pretrained validation plus a completion receipt; all three are valid 1,235,814,400-parameter models. Each final summary.json has both a matching .complete receipt and a passed official-evaluation receipt, so all three are official final results.

The evidence chain is materially complete. The explore manifest records agent_state=failed and raw exit 137 while also recording top-level status=explore_terminal and exit_status=0; the lifecycle resolves this as harness termination during an explicit submit, so the scientific phase completed rather than crashed. The raw JSONL contains a small number of standalone non-JSON text records, but the agent messages, 272 completed command records, and terminal submit action remain recoverable. A formal dispatch launch snapshot retained the earlier gating state, while the later outcome, formal manifest, and completion receipts consistently record success; this is a stale launch snapshot, not an unresolved result conflict. Standard errors or confidence intervals for final scores are not available because the evaluator did not produce them.

2. Reader-facing overview

The task asks for stronger forgetting from a fixed full Llama anchor while retaining general capability, using only the TOFU forget10/retain90 training-role projection. The shipped baseline uses negative preference optimization (NPO): it lowers the current model's relative likelihood of forget10 answers against a frozen start-model reference and adds retain90 answer negative log-likelihood to preserve utility. After 120 actual updates, the exploration baseline's visible 24-row-per-split proxy was 1.6971; late forgetting gains had flattened while retain damage remained 0.8994.

The agent investigated broader retain coverage, NLL versus KL retention, NPO weights and beta, reference-free SimNPO, a slow long schedule, seed variation, weight averaging, and a high-learning-rate basin. Expanding selection from 24 to 96 rows reordered candidates, exposing small-proxy selection risk. It submitted code rather than an exploration model: an approximately 11-hour sequence of independent NPO retrains from the fixed anchor, with a 96-row train-role diagnostic maintaining leaders in three retention-penalty bands and publishing exactly three complete models at the end.

Formal replay ran successfully for 38,865.374 seconds, 89.97% of the 12-hour budget, completed 110 valid independent trials, and emitted three loadable artifacts. Official balanced_unlearning_score values were 0.963664, 0.961922, and 0.965886 for 1101–1103; 1103 was best, with Extraction 0.076588 and MU 0.572424. Relative to the fixed start, it achieved strong forgetting while retaining much of the start utility. Relative to the shipped NPO component reference, forgetting was slightly weaker but MU was much higher. The task does not provide the shipped method's same-protocol balanced scalar, so an exact primary-score improvement cannot be claimed.

The strongest capability was turning observed learning-rate turnover, seed sensitivity, and proxy-sample reordering into a time-bounded multi-frontier formal search. The central failure was submission timing. The agent explicitly submitted with 1,177 seconds remaining and no active work, while this trajectory measured about 343 seconds for a leading high-LR retrain plus 96-row evaluation. Repeating the leading basin at seed 0 was a concrete meaningful untested direction, especially after seed variation had already been demonstrated. Observable agent behavior is therefore confirmed execution-protocol noncompliance. Formal budget use, artifact lineage, GPU isolation, and hidden-evaluation separation were otherwise compliant.

3. Task, baseline, and evaluation contract

3.1 Task and fixed boundaries

Starting model: the published full Llama-3.2-1B-Instruct anchor at revision pinned private revision. Every formal candidate must retrain freshly from this full anchor, never from an exploration checkpoint.

Available training data and assets: the TOFU revision 324592d8 train-role projection, containing 400 forget10 rows and 3,600 retain90 rows, plus the training-start model. The evaluator-only retain90 anchor and final-role data are absent during exploration and training.

The agent may edit the entire editable workspace training surface: row selection and weighting, sampling, unlearning objective and reference treatment, optimization, batching, schedule, and checkpoint policy. Fixed or forbidden components include asset roles, fresh formal replay, the external official scorer, and its primary metric. External data or checkpoints, network inputs, evaluation-specific lookups, and replacement of the declared final metric are prohibited.

The proxy measures answer negative-log-likelihood change from the start model. Forget-answer change FΔ should increase; retain-answer change RΔ should remain small. The ordinary proxy is FΔ - max(0,RΔ), maximized. Most exploration comparisons deterministically use the first 24 train-role rows from each split; because both candidate and start are evaluated, a run performs 96 model-row evaluations. Formal selection uses 96 rows per split. Neither protocol reports a standard error or confidence interval, and both are diagnostics on visible training-role data rather than hidden evaluation.

The final evaluator reports Extraction, where lower is better, and Model Utility (MU), where higher is better. Its maximized primary scalar, balanced_unlearning_score, is the harmonic mean of normalized forgetting progress and utility retention. This is a local composite, not a native OpenUnlearning metric or extra validity gate. Extraction uses 400 forget10 examples. MU aggregates retain90 (n=400), real-authors (n=100), and world-facts (n=117) components. The evaluator separately runs the retain90 reference, training start, and candidate. It reports no sampling uncertainty.

The proxy tests train-role answer likelihood only. The final evaluator also uses perturbed answers, text-similarity behavior, author knowledge, and world facts. Their scores are not numerically interchangeable and must not be subtracted across protocols. The artifact contract accepts up to three complete Hugging Face exports with increasing numeric progress; if more are emitted, only the three greatest progress labels survive, and each accepted artifact is scored independently.

3.2 How the baseline works

One baseline update can be summarized as follows:

``text fixed full anchor + 400 forget10 rows + 3,600 retain90 rows -> pair each forget row with a randomly sampled retain row; score forget answers with the current and frozen reference models -> derive an NPO signal from the current/reference forget-answer log-likelihood ratio and a retention target from correct retain answers -> combine beta=0.1 log-sigmoid negative preference loss with weight-1 retain NLL and optimize -> update every parameter of the 1.24B current model while leaving the reference frozen; export a complete model ``

The baseline anchors data on forget10, so an epoch traverses 400 forget examples while randomly sampling retain90. It declares ten epochs, per-device batch 8, gradient accumulation 4, effective batch 32, learning rate 1.5e-5, one warmup epoch, paged AdamW, ZeRO-3 without offload, and gradient checkpointing. The direct trainer state shows 120 actual updates and about 9.24 epochs, not ten completed epochs; execution records take precedence over the declaration.

The agent initially misestimated forget10 as roughly 125 rows and therefore misestimated updates per epoch. Direct dataset and training receipts establish 400 rows and 120 steps. Its eventual concrete diagnosis was that the baseline proxy flattened after the epoch-8 label: FΔ moved only from 2.5759 to 2.5965 while RΔ remained near 0.9, moving the proxy from 1.6695 to 1.6971. More of the same low-LR path did not appear promising.

4. Four-hour exploration and decision process

The agent began by inspecting boundaries, NPO source, and the data loader, then established the full baseline and repaired publication. The middle of the run tested data coverage, retention losses, objective weights, an alternative loss, and training length. The latter part quantified seed variance, mapped a high-LR basin, widened selection to 96 rows, and built and replayed the formal orchestrator. The explore manifest reports 13,263 seconds of phase time, most of it supporting interpretable training, evaluation, or debugging; the submission receipt separately records 1,177 seconds still available when the agent stopped idle.

U-01 - Baseline bottleneck and single-GPU execution

Motivation and hypothesis. The agent first needed the shipped NPO speed, memory use, learning curve, and artifact behavior to distinguish an objective limitation from an execution bottleneck.

Concrete change and experimental setup. It used a two-step single-GPU canary to test an execution path without ZeRO-3, then ran the shipped baseline. It evaluated epoch labels 2, 4, 6, and 8 plus the final model on 24 rows per split.

Observed result. The canary took about 81 seconds and peaked at 20,572 MiB. The full baseline wrapper took 549.351 seconds, including 482.042 seconds of training, with 32,440 MiB peak memory. Proxy scores rose from 0.2856 at label 2 to 1.1946, 1.5569, 1.6695, and 1.6971. After training, a live-edited unmatched quote in run.sh broke publication. The model and metadata survived, and the agent manually published the artifact and corrected its epoch bookkeeping.

Agent interpretation. It interpreted the late curve as a local plateau for the low-LR baseline and concluded that the single-GPU path could reduce full-trial time to roughly five or six minutes, enabling search.

Report assessment and confounds. The plateau interpretation fits the visible curve, but a 24-row train-role proxy without uncertainty cannot establish a plateau in official final performance. The early 125-row estimate was wrong. The publication failure was agent-introduced engineering overhead but did not invalidate the completed weights.

Decision and consequence. The baseline remained a fallback. Later trials used the fast single-GPU path and targeted more forgetting with less retain damage.