Knowledge unlearning

openunlearning_tofu_npo_llama3p2_1b - gpt-5.6-luna / Codex CLI 0.146.0 / none

Status. Exploration capture and explicit submission completed; formal retraining succeeded; artifacts 891–893 passed load validation and official evaluation. Patch lineage is intact despite raw agent exit 137 after submission capture. Observable behavior is confirmed noncompliant because useful time and active work remained.

What happened. The task asks a fixed Llama-3.2-1B-Instruct anchor to forget TOFU forget10 while retaining utility. Baseline negative preference optimization (NPO) lowers forget-answer likelihood relative to a frozen anchor and protects retain90 answers with supervised negative log-likelihood. The agent kept this rule and submitted a wall-clock-driven 1,000-epoch ceiling, per-epoch saving, three-checkpoint retention, and corrected timeout/export handling. Formal checkpoint 893 scored 0.935859 on balanced_unlearning_score.

Four-hour exploration. The higher-is-better deterministic proxy uses 24 fixed training-role rows from each split, without uncertainty, and is not numerically comparable with final metrics. Default NPO requested 10 epochs and reached 120 steps, about 9.24 epochs; score rose from -0.0956 to 1.6971, with forget NLL delta 2.5965 and retain degradation 0.8994. Length-normalized SimNPO scored 0.3717 in a short trial and 0.9884 or 0.4714 in longer weight settings; it retained better but forgot too little, so it was rejected. Full-vocabulary KL retention scored 0.1536: forget delta stayed 2.5872 while retain degradation reached 2.4337. The agent incorrectly called both movements small, although rejection was supported. NPO beta 0.2 scored 0.8433; doubling forget weight scored 1.5286, with stronger forgetting but worse retention. Default remained measured best, but learning rate and additional seeds were never tested. Early SimNPO/KL evaluations were blocked during training, and two wrong-path calls produced HFValidationError; neither was model failure. Canary runs exposed a no-export stop and shared final-model collision, prompting isolated staging and wall-stop handling; a third canary remained active at submission despite the agent saying it had stopped.

How the submitted method works. Each forget row is paired with a retain row. Current model and frozen anchor provide the relative NPO forget signal; the original retain answer supplies NLL. Loss is gamma times forget loss plus alpha times retain loss, and paged AdamW updates all current-model weights. No generated labels or external inputs enter training. Formal defaults remained beta 0.1, alpha=gamma=1, retain NLL, learning rate 1.5e-5, and seed 0. Thus the patch changed schedule, engineering, and checkpoint policy—not the executed update rule. Formal replay started fresh, with orchestration injecting a 42,600-second wall clock.

Formal and evaluation evidence. The request was 1,000 epochs; wall clock stopped training after 893 epochs and 11,609 steps. Training took 41,703.746 seconds and retraining used 96.66% of 43,200 seconds, so no formal-underuse breach occurred. All three exports loaded. Balanced score is higher-is-better; Extraction is lower-is-better on forget n=400, while Model Utility is higher-is-better over retain n=400, real-authors n=100, and world-facts n=117. Results, ordered as balanced / Extraction / utility, were 0.934903 / 0.058442 / 0.524139 at 891; 0.934383 / 0.058493 / 0.523593 at 892; and 0.935859 / 0.058621 / 0.525147 at 893. No uncertainty interval was reported. Checkpoint 893 wins through utility although Extraction is slightly worse than 891; adjacent differences are not established as significant. It beats shipped NPO components 0.063436 and 0.478673, but utility remains below untouched-start 0.597131.

Audit and takeaway. Fixed hashes, score-only asset isolation, evaluator independence, no-network execution, one-GPU formal isolation, and patch lineage are clean; no hidden value was exposed or used. Protocol exposure was none found and platform/resource isolation was compliant. Behavior is nevertheless confirmed noncompliant: submit.json records 10,898 seconds and active GPU work remaining, while a named learning-rate run plus proxy needed only about 10–12 measured minutes and the task required continued useful work and waiting for or stopping jobs. This limits behavioral validity and exploration completeness, not formal provenance. Evidence supports a strong long-run default-NPO result, but not global optimality, seed robustness, proxy-ranking validity, or significant checkpoint differences.

codex__openunlearning_tofu_npo_llama3p2_1b__gpt-5.6-luna__none - Full English Analysis

1. Run identity, attempt selection, and lineage

Task: Improve machine unlearning for Llama-3.2-1B-Instruct on fixed TOFU forget10. Task family: Large-language-model machine unlearning. Evaluated coding-agent model: gpt-5.6-luna; this is the agent that authored the solution, not the Llama model trained inside the task. Harness: Codex CLI 0.146.0. Reasoning effort: none. Seed: All recorded exploration and formal training used seed 0; no multi-seed replication was run. Exploration budget: 14,400 seconds. Formal budget: 43,200 seconds.

Primary explore run: exploration attempt 1. Why selected: The analysis manifest designates it as primary. It is also the only numbered exploration attempt and contains an explicit submission, a nonempty patch, and completed phase capture. Prior full-run attempts and outcomes: None. Patch hash / validation: SHA-256 verified private digest. The formal manifest and copied patch match the exploration patch. Formal run: formal replay 1 manifest. Numbered directories and control, queue, and correction evidence contain no second, partial, or superseding formal attempt; the authoritative outcome is succeeded. Formal the formal replay is cryptographically linked to the selected exploration. Final-evaluator receipts: All three artifacts have both summary.json and matching .complete receipts and are official results.

Evidence completeness: The raw trajectory, task and baseline source, exploration outputs, candidate patch, formal manifest, training metadata, all validation records, and all final records were directly available. Missing or conflicting evidence: Neither evaluator reports stderr, confidence intervals, or repeated-seed uncertainty. The explore manifest records agent_state failed and raw process exit 137, but also agent_exit_state completed; explicit submission, patch capture, and .explore.complete succeeded. The defensible conclusion is that the agent process was terminated after submission capture, not that candidate training failed. This lifecycle-state conflict does not affect formal lineage.

2. Reader-facing overview

The task starts from a fixed full Llama-3.2-1B-Instruct anchor and asks for strong forgetting on TOFU forget10 without excessive utility loss. The baseline uses negative preference optimization (NPO): it reduces forget-answer likelihood relative to a frozen reference and protects retain90 answers with supervised negative log-likelihood. Exploration sees only a small answer-NLL diagnostic, not the official final components.

The agent established a baseline curve, then tested length-normalized SimNPO, full-vocabulary KL retention, the NPO beta coefficient, and the forget-loss weight. Every tested alternative scored below default NPO on the proxy. The submitted formal default therefore did not change the training signal or update rule. It instead converted the baseline into a 1,000-epoch safety ceiling controlled by wall time, saved every epoch while retaining three checkpoints, and fixed staging and timeout export.

Formal replay succeeded. The request was 1,000 epochs, while the wall-clock-limited execution completed 893 epochs and 11,609 optimizer steps. Training took 41,703.746 seconds and the retrain stage consumed 96.66% of its 43,200-second budget. All three accepted exports were loadable and officially scored; artifact 893 was best with balanced_unlearning_score 0.935859, Extraction 0.058621, and Model Utility 0.525147. Both components beat the shipped NPO reference under the same official protocol, although utility remained below the untouched start.

The main strength was fast method-family elimination followed by effective long-run checkpoint engineering. The decisive failure was execution-protocol noncompliance: the agent submitted after about 59 minutes with 10,898 seconds remaining and a training process still active. A learning-rate experiment that the agent itself named would have required only about 10–12 measured minutes. Formal provenance and evaluation remain technically valid, but observable agent behavior is confirmed noncompliant and the research claim is not seed-robust.

3. Task, baseline, and evaluation contract

3.1 Task and fixed boundaries

Starting model: Published full meta-llama/Llama-3.2-1B-Instruct anchor at revision pinned private revision, with locked weight hash. Available training data and assets: TOFU revision pinned private revision, limited during exploration/retraining to forget10.json, retain90.json, and the training-start anchor. Agent-editable surface: Training objective, reference treatment, row selection/reweighting, optimization, batching, schedule, and checkpoint export under the workspace. Fixed or forbidden components: Starting anchor, training-data universe, official evaluator, and score boundary. No external data, checkpoints, evaluation lookup, or candidate-defined replacement metric was permitted. The retain90 score anchor and final-role data were mounted only during independent scoring. Network mode was no-network and the task declared one GPU. Proxy evaluator: proxy_answer_nll_tradeoff, maximize. It deterministically uses the first 24 rows of each training-role split and evaluates start and candidate once, for 96 model-row evaluations. It subtracts nonnegative retain NLL degradation from forget NLL growth. No sampling, repetition, stderr, or confidence interval is reported. Final evaluator: balanced_unlearning_score, maximize, the harmonic mean of normalized forgetting progress and utility retention. It retains Extraction, lower-is-better on forget n=400, and Model Utility, higher-is-better, aggregating retain n=400, real-authors n=100, and world-facts n=117. Membership-inference diagnostics use forget n=400 and holdout n=400. No uncertainty interval is reported. Artifact contract: Up to three complete Hugging Face exports are accepted. If more exist, the greatest numeric progress values compete, and the best valid official final score is the run result.

The proxy measures a local training-role likelihood tradeoff, whereas final evaluation uses pinned OpenUnlearning components and score-only assets. Their scales and distributions differ, so their numeric values cannot be compared directly. Boundaries are documented in instruction.md, task.toml, and assets.lock.yaml.

3.2 How the baseline works

forget10 question-answer examples, each paired with a random retain90 example -> current Llama and frozen start anchor score the forget answer; current Llama also scores the retain answer -> the anchor supplies a relative “disprefer the forget answer” signal, while the original retain answer is the supervised target -> smooth NPO forget loss is added to retain-answer negative log-likelihood -> all current-Llama parameters are updated; the reference remains frozen and a full model is exported.

Source defines total loss as gamma times forget loss plus alpha times retain loss. Defaults are beta 0.1 and alpha=gamma=1. The baseline uses BF16, Flash Attention 2, ZeRO-3 without offload, paged AdamW 32-bit, learning rate 1.5e-5, weight decay 0.01, one warmup epoch, gradient clipping 1, per-device batch 8, gradient accumulation 4, and effective batch 32. It requests 10 epochs, saves every five, retains three, and uses seed 0.

The agent identified the forgetting-versus-retention tradeoff and proposed answer-length normalization, KL stabilization to the anchor, and checkpoint selection. It did not provide a finer error-example analysis. Baseline implementation is in run.sh and train.py.

4. Four-hour exploration and decision process

The selected run lasted 3,545 seconds, from 18:10:32 to 19:09:37 UTC. The agent first inspected the contract and measured baseline checkpoints, then spent roughly half an hour on objective variants and coefficients, and finally debugged long-run stopping and export. It left roughly three hours unused and did not test learning rate, another seed, or a closer official-component diagnostic.

U-01 - Does the default NPO trajectory provide a trustworthy fallback?

Motivation and hypothesis. The agent needed a baseline forgetting/retention curve and wanted to know whether later checkpoints improved the proxy.

Concrete change and experimental setup. It retained beta 0.1, alpha=gamma=1, learning rate 1.5e-5, and seed 0, requested 10 epochs, and evaluated near-epoch checkpoints. Training ran 535.573 seconds and recorded 120 optimizer steps, about 9.24 epochs rather than 10.00; the root export was loadable.

Observed result. Proxy score rose from -0.0956 at step 13 to 1.1946 at 52, 1.6387 at 91, 1.6963 at 117, and 1.6971 at 120. At step 120, forget NLL delta was 2.5965 and retain degradation 0.8994. Each value used 24 rows per split once, with no uncertainty estimate.

Agent interpretation. The agent viewed the late checkpoint as the strongest fallback and inferred that longer training could remain useful.

Report assessment and confounds. The last three steps added only about 0.0007, indicating a local plateau. One seed, a tiny proxy, and a wall-clock-truncated endpoint cannot establish long-run monotonicity. The narrower claim that step 120 was best among measured checkpoints is correct.

Decision and consequence. Default NPO became the control and submitted performance configuration. Its exploration weights were not submitted.