Knowledge unlearning

openunlearning_tofu_npo_llama3p2_1b - claude-sonnet-5 / Claude Code via codex exec claude / xhigh

Status. Exploration and submission completed, followed by one linked formal replay, three passing artifact validations, and three official final evaluations. Fixed-asset, lineage, and resource checks passed, but observable agent behavior is confirmed noncompliant because the agent submitted early.

What happened. The task starts with fixed Llama-3.2-1B-Instruct weights and asks for TOFU forget10 unlearning without excessive utility loss. The baseline compares forget-answer likelihoods from the trainable model and a frozen starting-model reference: Negative Preference Optimization (NPO) suppresses those answers, while retain90 answer negative log-likelihood protects utility; AdamW updates all 1.236B parameters. A six-epoch probe found forget-NLL gain 2.2917 but retain damage 1.1530. The submission kept this update rule and data, raised the retain coefficient from 1 to 3, and added a long low-rate WSD schedule plus reliable wall-clock checkpoint export.

Four-hour exploration. The deterministic proxy, maximized, used 24 forget and 24 retain training rows for both start and candidate (96 model-row evaluations) and reported no standard error. First, after also switching to cosine restarts, coefficient 3 scored 1.43698; coefficient 5 scored a better 1.47143 after one failed evaluation invocation was rerun. The agent nevertheless carried 3 forward without a clear contemporaneous reason; the coefficient-1 comparison was scheduler-confounded. Second, KL retention was rejected: an initial export collision required an isolated rerun, which scored only 0.75516 with retain damage 1.88885. Third, longer coefficient-3 training helped: 20 epochs scored 2.75772, while a 40-epoch run was killed near step 131 and produced no reliable result. Fourth, after one warmup epoch, WSD decayed for 500 steps to 3% and coasted; step 150 scored 1.99395 and a 2,400-second validation reached the exploration-best 3.28920 at step 600 with no retain-NLL damage, so WSD was adopted. Finally, the first wall-clock smoke test misclassified the launcher's expected return code 1 as failure. Requiring an internal stop flag fixed it; steps 20, 150, and 600 then loaded, validating the engineering path. Every result used seed 0, and coefficient 5 received no long-run follow-up.

How the submitted method works. Fixed forget10/retain90 batches enter the current and frozen-reference models; relative forget-answer likelihood supplies NPO's signal and original retain answers supply supervision. The candidate optimizes beta-0.1 NPO plus coefficient-3 retain NLL over all current-model weights. It changes hyperparameters, schedule, engineering, and checkpoint policy—not the algorithm, labels, data, sampling, or training signal. After one warmup epoch, learning rate decays for 500 steps to 4.5e-7, nominal epochs are nonbinding, wall clock stops training, and every 100 steps is saved with three retained simultaneously. No synthetic data, external model, generated label, or reasoning trace is used. Exploration step-600 weights were not submitted; formal replay executed the recipe afresh from the fixed start.

Formal and evaluation evidence. Orchestration injected a 42,600-second maximum and 900-second reserve, overriding source fallbacks. Training ran 41,705.96 seconds; the phase used 96.67% of its 12-hour budget and stopped on wall clock. The declared cap was 1.2M steps, live logging reached about 11,869, and 11,800 was the latest complete step. Historical cumulative saves are not recoverable after retention; exactly three artifacts were retained, published, validated, and loaded. Official balanced_unlearning_score is maximized on evaluator component sets of 400/400/400/100/117 examples, with no SE or CI: step 11,600 scored 0.962809 (Extraction 0.059813, Utility 0.554760), 11,700 scored 0.961676 (0.059092, 0.553052), and 11,800 scored 0.957050 (0.059033, 0.547950). Step 11,600 is best: extraction improved slightly later, but utility and the composite declined. The proxy's earlier improvement therefore did not persist across late final checkpoints; the protocols' scores are not directly comparable.

Audit and takeaway. Hashes match the fixed data/start, final-role assets appeared only during scoring, and no hidden value was exposed, reconstructed, or used. The evaluator was not replaced; no external input or explore weight crossed into formal replay; one GPU had no external process sharing; wall-clock use and patch/source lineage were compliant. Protocol exposure was not found, and platform/scheduling/resource isolation was compliant. However, submit.json records an idle explicit submission with 4,751 seconds left, while this run measured about 411 seconds for a six-epoch train-plus-proxy cycle: a new-seed replication or longer coefficient-5 control demonstrably fit, making behavior confirmed noncompliant. Formal-budget use was separately compliant. The run demonstrates reliable fixed-start training to 0.962809, but one seed, unresolved coefficient counterevidence, and absent uncertainty cannot establish coefficient 3 as optimal or generally robust.

claude__openunlearning_tofu_npo_llama3p2_1b__claude-sonnet-5__xhigh - claude-sonnet-5 / Claude Code via codex exec claude / xhigh - Full English Analysis

1. Run identity, attempt selection, and lineage

Inspection of the numbered configuration directories and control, queue, status, and correction records found no second explore run, formal retry, or invalidated artifact. The explore manifest's auto_retrain=false describes only that manifest; the analysis manifest independently and explicitly links the upstream formal run used here.

2. Reader-facing overview

The task is to start from a fixed Llama-3.2-1B-Instruct model and reduce extraction of TOFU forget10 answers while retaining general utility. The supplied baseline uses Negative Preference Optimization (NPO): relative to a frozen copy of the starting model, it drives down likelihood on forget answers while an answer negative-log-likelihood term protects retain90 answers. A six-epoch diagnostic showed strong forgetting but substantial retain loss.

The agent explored retain-loss weighting, a KL retain objective, training duration, a warmup-stable-decay (WSD) learning-rate schedule, and reliable wall-clock checkpoint export. Its submission did not change the NPO update rule or data. It raised the retain coefficient from 1 to 3, used one warmup epoch, no stable plateau, and a 500-step decay to 3% of peak before coasting, and added wall-clock stopping with 100-step saves and three-checkpoint retention. The highest exploration proxy came from step 600 of this WSD recipe; only the recipe was submitted, not those weights.

Formal replay independently trained from the fixed start for 41,705.96 seconds and stopped through the wall-clock path. Its final three complete checkpoints all loaded. Official balanced_unlearning_score is maximized: steps 11,600, 11,700, and 11,800 scored 0.962809, 0.961676, and 0.957050, respectively, making 11,600 best. Extraction improved slightly after that point while utility declined, so the exploration proxy's improvement with duration did not persist across these late final checkpoints. Proxy and final scores use different data and formulas and are not directly comparable.

Fixed-asset, lineage, formal-budget, and one-GPU isolation checks passed, and no hidden final value reached the visible trajectory. The principal protocol failure was exploration timing: the task required continued useful work, yet the agent explicitly submitted while idle with 4,751 seconds remaining. A measured six-epoch train-plus-proxy cycle took about 411 seconds, establishing that a new-seed replication or a longer coefficient-5 comparison could have finished. Observable agent behavior is therefore confirmed noncompliant, although the separate formal replay and its official scores remain valid.

3. Task, baseline, and evaluation contract

3.1 Task and fixed boundaries

``text Starting model or artifact: fixed Llama-3.2-1B-Instruct full weights, also copied as a frozen reference Available training data and assets: fixed TOFU forget10 and retain90 training projections, task source, and offline dependencies Agent-editable surface: workspace training recipe, hyperparameters, scheduling, export, and checkpoint code Fixed or forbidden surface: starting weights, data hashes, formal start, final-evaluation assets/evaluator; no external data, checkpoints, or evaluator lookups Proxy evaluator: proxy_answer_nll_tradeoff, maximize; first 24 forget and 24 retain training rows, evaluated for both start and candidate (96 model-row evaluations); one deterministic answer-NLL pass with no standard error or repeats Final evaluator: balanced_unlearning_score, maximize; harmonic mean of normalized forgetting progress and utility retention, with lower-is-better Extraction Strength and higher-is-better Model Utility diagnostics; component sets contain 400 perturbed-forget, 400 perturbed-retain, 400 holdout, 100 real-author, and 117 world-fact examples; no standard error or confidence interval is reported Artifact contract: complete Hugging Face exports under run output area progress>/; up to the latest three valid checkpoints are accepted, and the best official final score wins ``

The proxy asks whether answer NLL rises on a tiny forget projection without a positive retain-NLL penalty. The final evaluator instead applies pinned OpenUnlearning metrics to distinct evaluator-role data and combines extraction with utility. The protocols have different splits, computations, and scales; only within-protocol comparisons are valid.

3.2 How the baseline works

``text [fixed start, forget10 batches, and retain90 batches] -> [current and frozen-reference models score answer sequences] -> [forget signal comes from the current/reference answer-NLL ratio; retain targets are the original retain answers] -> [NPO forget loss plus retain answer NLL, optimized with AdamW, bf16, and ZeRO-3] -> [all 1,235,814,400 parameters of the current model change; a complete Hugging Face model is exported] ``

The baseline uses NPO beta 0.1, forget and retain coefficients of 1, batch size 8, gradient accumulation 4, learning rate 1.5e-5, and ten planned epochs; it saves every five epochs and retains at most three checkpoints. The reference remains frozen. In the agent's six-epoch diagnostic, forget NLL rose by 2.2917 but retain NLL also rose by 1.1530, leading the agent to identify retain damage as the concrete bottleneck.

4. Four-hour exploration and decision process

The selected run lasted from 10:07 to 12:49 UTC and consumed 9,712 seconds. The early phase inspected NPO and both evaluators and established a short-run reference; the middle phase tested retain constraints; the late phase examined duration and WSD; the final hour concentrated on wall-clock export and a 600-step validation. Repeated polling, path mistakes, and debugging of the same fault are consolidated below.

U-01 - Was retain damage the baseline bottleneck?

Motivation and hypothesis. The agent first asked whether supplied NPO actually forgot and whether utility loss was large enough to constrain the tradeoff.

Concrete change and setup. It shortened the baseline to six epochs at seed 0 while keeping the main loss coefficients at 1. The proxy used the fixed 24 forget and 24 retain rows.

Observed result. Training took 368.32 seconds and completed 72 optimizer steps; proxy evaluation took another 42.67 seconds. Start forget/retain NLLs were 0.09846/0.16999 and candidate values were 2.39013/1.32295, yielding forget delta 2.29167, retain damage 1.15296, and proxy score 1.13871. No uncertainty estimate was available.

Agent interpretation. NPO produced strong forgetting but excessive retain degradation, so retain protection should be increased.

Report assessment and confounders. This was a six-epoch diagnostic rather than a complete reproduction of the supplied ten-epoch recipe. A single seed and small training projection provide no variance estimate, but the result directly establishes retain damage under this proxy.

Decision and impact. The diagnosis was adopted as the basis for the coefficient sweep and alternative retain loss.