Meta FAIR Self-Improving Memory Hits 56.0 on Continual Learning Bench
Sentiment: Neutral ⚖️Meta FAIR has introduced Self-Improving Memory, a method for AI agents that manage their own memory.
The model first discovers its own memory instructions.
Previous in Artificial Intelligence
It then trains those skills into its weights.
Co-authors include researchers from the University of Washington and Johns Hopkins University.
TL;DR Method: The base is Kimi-K2.6, a 1T-parameter MoE with 32B active and 256K context.
Distillation adds a LoRA adapter (rank 64) on attention projections.
Runs on: Training hardware is not disclosed.
Kimi-K2.6 serving needs a multi-GPU node; Vast.ai recommends 8x H200.
Native INT4 weights exist.
Performance: Self-found memory prompts average 56.0 on 3 tasks, versus 44.6 for the best human-written ones.
Best: Codebase, 64.3 with optimized prompts, +20.6 over the best hand-written harness.
Worst: Spectrum distillation adds only 2.9 points (46.5 vs 43.6 for SFT on generic runs).
What is Self-Improving Memory?
Self-Improving Memory is a 2-stage method from Meta FAIR that teaches an agent model to manage file-based memory.
The agent works through a stream of subtasks.
Each subtask starts with an empty context.
Only a persistent memory directory, edited with file tools, carries over.
Two prompts control that memory.
OFFLOAD updates memory after a subtask.
LOAD retrieves notes before the next one.
Generic versions leave the strategy to the model.



