Sample Page Title

The generative AI race has lengthy been a recreation of ‘greater is healthier.’ However because the trade hits the bounds of energy consumption and reminiscence bottlenecks, the dialog is shifting from uncooked parameter counts to architectural effectivity. Liquid AI crew is main this cost with the discharge of LFM2-24B-A2B, a 24-billion parameter mannequin that redefines what we must always count on from edge-capable AI.

The ‘A2B’ Structure: A 1:3 Ratio for Effectivity

The ‘A2B’ within the mannequin’s title stands for Consideration-to-Base. In a conventional Transformer, each layer makes use of Softmax Consideration, which scales quadratically (O(N²)) with sequence size. This results in large KV (Key-Worth) caches that devour VRAM.

Liquid AI crew bypasses this by utilizing a hybrid construction. The ‘Base‘ layers are environment friendly gated quick convolution blocks, whereas the ‘Consideration‘ layers make the most of Grouped Question Consideration (GQA).

Within the LFM2-24B-A2B configuration, the mannequin makes use of a 1:3 ratio:

• Whole Layers: 40
• Convolution Blocks: 30
• Consideration Blocks: 10

By interspersing a small variety of GQA blocks with a majority of gated convolution layers, the mannequin retains the high-resolution retrieval and reasoning of a Transformer whereas sustaining the quick prefill and low reminiscence footprint of a linear-complexity mannequin.

Sparse MoE: 24B Intelligence on a 2B Price range

Crucial factor of LFM2-24B-A2B is its Combination of Specialists (MoE) design. Whereas the mannequin accommodates 24 billion parameters, it solely prompts 2.3 billion parameters per token.

It is a game-changer for deployment. As a result of the lively parameter path is so lean, the mannequin can match into 32GB of RAM. This implies it may well run regionally on high-end shopper laptops, desktops with built-in GPUs (iGPUs), and devoted NPUs with no need a data-center-grade A100. It successfully supplies the data density of a 24B mannequin with the inference pace and vitality effectivity of a 2B mannequin.

Benchmarks: Punching Up

Liquid AI crew experiences that the LFM2 household follows a predictable, log-linear scaling habits. Regardless of its smaller lively parameter rely, the 24B-A2B mannequin persistently outperforms bigger rivals.

• Logic and Reasoning: In assessments like GSM8K and MATH-500, it rivals dense fashions twice its dimension.
• Throughput: When benchmarked on a single NVIDIA H100 utilizing vLLM, it reached 26.8K complete tokens per second at 1,024 concurrent requests, considerably outpacing Snowflake’s gpt-oss-20b and Qwen3-30B-A3B.
• Lengthy Context: The mannequin incorporates a 32k token context window, optimized for privacy-sensitive RAG (Retrieval-Augmented Technology) pipelines and native doc evaluation.

Technical Cheat Sheet

Key Takeaways

• Hybrid ‘A2B’ Structure: The mannequin makes use of a 1:3 ratio of Grouped Question Consideration (GQA) to Gated Brief Convolutions. By using linear-complexity ‘Base’ layers for 30 out of 40 layers, the mannequin achieves a lot quicker prefill and decode speeds with a considerably diminished reminiscence footprint in comparison with conventional all-attention Transformers.
• Sparse MoE Effectivity: Regardless of having 24 billion complete parameters, the mannequin solely prompts 2.3 billion parameters per token. This ‘Sparse Combination of Specialists’ design permits it to ship the reasoning depth of a giant mannequin whereas sustaining the inference latency and vitality effectivity of a 2B-parameter mannequin.
• True Edge Functionality: Optimized by way of hardware-in-the-loop structure search, the mannequin is designed to slot in 32GB of RAM. This makes it totally deployable on consumer-grade {hardware}, together with laptops with built-in GPUs and NPUs, with out requiring costly data-center infrastructure.
• State-of-the-Artwork Efficiency: LFM2-24B-A2B outperforms bigger opponents like Qwen3-30B-A3B and Snowflake gpt-oss-20b in throughput. Benchmarks present it hits roughly 26.8K tokens per second on a single H100, exhibiting near-linear scaling and excessive effectivity in long-context duties as much as its 32k token window.

Try the Technical particulars and Mannequin weights. Additionally, be happy to observe us on Twitter and don’t overlook to affix our 120k+ ML SubReddit and Subscribe to our Publication. Wait! are you on telegram? now you possibly can be part of us on telegram as nicely.