SEAL makes the preparation of training material a learning target

SEAL makes the usefulness of training material something a model can learn. It generates candidate text from a supplied passage, uses that text to update model parameters, and tests the adapted model's answers. The selected candidate then becomes a target for training the generation policy. Answers provide the selection feedback; the chosen training text supplies the target for the outer update. Nested loops, §3.1; selection and outer training, Appendix B.2

This retrospective, dated 14 October 2026, examines Self-Adapting Language Models, arXiv:2506.10943v1, dated 12 June 2025. The numerical results are the authors' reported results; this article provides no independent replication results. Pinned v1 edition

One passage, five alternative learning trials

The editorial diagram starts with a fictional passage: “Morning Library is closed Tuesday, and open Wednesday.” Its expanded candidate says, “Visitors cannot enter on Tuesday.” The passage, candidate and question are teaching examples, not SQuAD records. No experimental score or winning candidate is shown.

The paper calls this generated material a self-edit. In the main implications-prompt setting, it can contain restatements, inferences and logical consequences of the passage. Its role is concrete: the original passage and generated material enter the inner supervised fine-tuning step, or SFT. The reported setup adapts parameters through low-rank LoRA adapters and computes causal language-modeling loss over the generated statements. This is where the model learns from the prepared text. Self-edit format and inner update, §3.2; passage-plus-generated-material setup, §4.2

The next step tests what that training enabled. The adapted model answers questions about the passage with the passage absent from its answering context. In the teaching example, the question is whether visitors can enter on Tuesday and the reference answer is “No”; the diagram supplies no actual model answer. “No context” applies at answer time. The passage was available earlier for generating material and adapting parameters. Figure 2, §3.2; answer-time setting, Appendix B.1

The four stacked documents are four other candidates for the same passage. Each undergoes its own adaptation and evaluation. During outer training, each candidate is trained and evaluated over three random seeds; the results are averaged, and the generation with the highest mean is retained for that passage. The five branches compare alternative ways to learn the same material. They do not represent five successive updates accumulating knowledge in one model state. Candidate trials and selection, Appendix B.2; parameter updates, Algorithm 1

The diagram's teal and blue distinguish inner adaptation from generation and outer training. Its box sizes and arrow lengths express the relationships between steps, without encoding accuracy, data quantity, compute or time.

Why the feedback leads to a second SFT

Each of the two outer training rounds takes 50 passage–question–answer triples from the SQuAD training set and generates five candidates per passage. Selecting one candidate for each passage produces 50 training pairs: the passage prompt is the input, and its selected self-edit is the completion. SFT on those pairs teaches the generation policy to produce the selected material. The evaluation answers have already done their job by supplying the selection feedback. Two rounds, candidate selection and 50 prompt-completion pairs, Appendix B.2

This distinction explains why the paper calls the outer procedure reinforcement learning even though its update uses SFT. Its ReST-EM implementation samples candidate generations, selects them using downstream performance, and reinforces the selected text through SFT. The reward determines which material becomes a generation target. ReST-EM and selected-edit training objective, §3.1

Algorithm 1 keeps the candidate-adapted parameters and the outer policy update distinct: the adapted state supplies the evaluation score, while outer training uses the selected text. It does not import the winning trial's adapted weights into the generator. Generation and learning are two roles of the same model in this implementation. Algorithm 1 and shared-model implementation, §3.1

The selection criterion is usefulness after training. It does not independently verify every generated implication. Likewise, averaging three seeds supports candidate selection under the grading protocol; it does not independently validate the grader. Neither distinction is a finding that the material or grader is wrong.

Which comparisons show the gain

The knowledge-incorporation experiments use the Qwen2.5-7B base model and SQuAD v1.1. Evaluation covers 200 held-out passages and 974 questions. Answers are automatically graded by gpt-4.1-2025-04-14, with greedy decoding for the grader. “Single passage” means one passage's material per isolated adaptation; the evaluation itself spans many passages. Model and dataset, Appendix B.1; grading and evaluation counts, Appendix B.4; isolated edits, §5

Training on the passage alone scores 33.5%. Adding implications generated by the base model, before training the generation policy with RL, scores 39.7%. After two outer rounds, the main implications condition reaches 47.0%. The main results use tuned hyperparameters. “Before RL” describes the generator's training history: the answering model still receives an inner parameter update. Baseline definitions and Table 2, §4.2

Appendix C compares different material-generation prompts. Its rewrite condition starts at 49.4% and reaches 55.6% after two rounds. Keeping each prompt with its own starting point makes the policy-training comparison visible. Prompt conditions and Table 6, Appendix C

The authors report higher accuracy after policy training in both conditions. These are arithmetic differences between reported scores, without a statistical-significance claim. The full 33.5%→47.0% increase includes a change in the adaptation input when generated material is added, so it cannot all be assigned to RL. The rewrite prompt's higher starting score also precedes policy training; a comparison across prompts changes another experimental condition.

One adaptation, a pooled batch, and a sequence

The 47.0% result concerns isolated passage adaptation. Incorporating a larger batch and preserving knowledge through later updates ask different questions.

For pooled continued pretraining, the researchers sample five generations per passage and aggregate the material across 200 passages. This procedure combines generations, whereas outer training retains one candidate per context. SEAL reports 43.8% after the single pooled run. That score concerns batch incorporation; it does not measure retention after 200 successive updates. Comparing 43.8% with the isolated 47.0% result also changes the adaptation regime and does not isolate why the pooled score is lower. Pooled protocol and result, §4.2 and Appendix B.3

The sequential experiment measures retention directly. Each new test passage triggers an edit, and all previously seen tasks are retested after every update. The training objective does not explicitly optimize retention, but the experiment does measure it. The authors report gradual decline on earlier tasks as edits accumulate, alongside the ability to perform multiple updates without complete collapse. Sequential protocol and findings, §5

The feedback loop also relies on passages arriving with reference questions and answers. The authors identify that requirement as a barrier to scaling the reported RL training to unlabeled corpora. Having the model generate evaluation questions is a proposed extension. Context-dependent evaluation, §5

SEAL's demonstrated contribution is a connection between material preparation and measured learning: generated text changes parameters, the resulting answers select text, and that selection trains future generation. The gains support more effective adaptation in the reported setting. The sequential results show why preserving earlier knowledge remains a separate problem. Reliable indefinite accumulation, open-ended recursive self-improvement and unbounded acceleration remain unestablished by these experiments.