Allen Institute for AI (AI2) Researchers introduce SERA, Mushy Verified Environment friendly Repository Brokers, as a coding agent household that goals to match a lot bigger closed methods utilizing solely supervised coaching and artificial trajectories.
What’s SERA?
SERA is the primary launch in AI2’s Open Coding Brokers collection. The flagship mannequin, SERA-32B, is constructed on the Qwen 3 32B structure and is skilled as a repository degree coding agent.
On SWE bench Verified at 32K context, SERA-32B reaches 49.5 p.c resolve fee. At 64K context it reaches 54.2 p.c. These numbers place it in the identical efficiency band as open weight methods reminiscent of Devstral-Small-2 with 24B parameters and GLM-4.5 Air with 110B parameters, whereas SERA stays totally open in code, information, and weights.
The collection contains 4 fashions at present, SERA-8B, SERA-8B GA, SERA-32B, and SERA-32B GA. All are launched on Hugging Face below an Apache 2.0 license.
Mushy Verified Era
The coaching pipeline depends on Mushy Verified Era, SVG. SVG produces agent trajectories that appear like real looking developer workflows, then makes use of patch settlement between two rollouts as a tender sign of correctness.
The method is:
- First rollout: A perform is sampled from an actual repository. The instructor mannequin, GLM-4.6 within the SERA-32B setup, receives a bug fashion or change description and operates with instruments to view information, edit code, and run instructions. It produces a trajectory T1 and a patch P1.
- Artificial pull request: The system converts the trajectory right into a pull request like description. This textual content summarizes intent and key edits in a format much like actual pull requests.
- Second rollout: The instructor begins once more from the unique repository, however now it solely sees the pull request description and the instruments. It produces a brand new trajectory T2 and patch P2 that tries to implement the described change.
- Mushy verification: The patches P1 and P2 are in contrast line by line. A recall rating r is computed because the fraction of modified strains in P1 that seem in P2. When r equals 1 the trajectory is tough verified. For intermediate values, the pattern is tender verified.
The important thing end result from the ablation research is that strict verification just isn’t required. When fashions are skilled on T2 trajectories with completely different thresholds on r, even r equals 0, efficiency on SWE bench Verified is analogous at a set pattern depend. This implies that real looking multi step traces, even when noisy, are invaluable supervision for coding brokers.
Information scale, coaching, and price
SVG is utilized to 121 Python repositories derived from the SWE-smith corpus. Throughout GLM-4.5 Air and GLM-4.6 instructor runs, the complete SERA datasets comprise greater than 200,000 trajectories from each rollouts, making this one of many largest open coding agent datasets.
SERA-32B is skilled on a subset of 25,000 T2 trajectories from the Sera-4.6-Lite T2 dataset. Coaching makes use of normal supervised fantastic tuning with Axolotl on Qwen-3-32B for 3 epochs, studying fee 1e-5, weight decay 0.01, and most sequence size 32,768 tokens.
Many trajectories are longer than the context restrict. The analysis workforce outline a truncation ratio, the fraction of steps that match into 32K tokens. They then desire trajectories that already match, and for the remainder they choose slices with excessive truncation ratio. This ordered truncation technique clearly outperforms random truncation after they evaluate SWE bench Verified scores.
The reported compute price range for SERA-32B, together with information era and coaching, is about 40 GPU days. Utilizing a scaling regulation over dataset dimension and efficiency, the analysis workforce estimated that the SVG strategy is round 26 occasions cheaper than reinforcement studying primarily based methods reminiscent of SkyRL-Agent and 57 occasions cheaper than earlier artificial information pipelines reminiscent of SWE-smith for reaching comparable SWE-bench scores.
Repository specialization
A central use case is adapting an agent to a particular repository. The analysis workforce research this on three main SWE-bench Verified tasks, Django, SymPy, and Sphinx.
For every repository, SVG generates on the order of 46,000 to 54,000 trajectories. Attributable to compute limits, the specialization experiments prepare on 8,000 trajectories per repository, mixing 3,000 tender verified T2 trajectories with 5,000 filtered T1 trajectories.
At 32K context, these specialised college students match or barely outperform the GLM-4.5-Air instructor, and in addition evaluate nicely with Devstral-Small-2 on these repository subsets. For Django, a specialised scholar reaches 52.23 p.c resolve fee versus 51.20 p.c for GLM-4.5-Air. For SymPy, the specialised mannequin reaches 51.11 p.c versus 48.89 p.c for GLM-4.5-Air.
Key Takeaways
- SERA turns coding brokers right into a supervised studying downside: SERA-32B is skilled with normal supervised fantastic tuning on artificial trajectories from GLM-4.6, with no reinforcement studying loop and no dependency on repository take a look at suites.
- Mushy Verified Era removes the necessity for assessments: SVG makes use of two rollouts and patch overlap between P1 and P2 to compute a tender verification rating, and the analysis workforce present that even unverified or weakly verified trajectories can prepare efficient coding brokers.
- Massive, real looking agent dataset from actual repositories: The pipeline applies SVG to 121 Python tasks from the SWE smith corpus, producing greater than 200,000 trajectories and creating one of many largest open datasets for coding brokers.
- Environment friendly coaching with specific price and scaling evaluation: SERA-32B trains on 25,000 T2 trajectories and the scaling research reveals that SVG is about 26 occasions cheaper than SkyRL-Agent and 57 occasions cheaper than SWE-smith at comparable SWE bench Verified efficiency.
Try the Paper, Repo and Mannequin Weights. Additionally, be happy to comply with us on Twitter and don’t overlook to affix our 100k+ ML SubReddit and Subscribe to our Publication. Wait! are you on telegram? now you’ll be able to be a part of us on telegram as nicely.
