Benchmarking Pace, Scale, and Value Effectivity

This weblog submit focuses on new options and enhancements. For a complete checklist, together with bug fixes, please see the launch notes.

GPT-OSS-120B: Benchmarking Pace, Scale, and Value Effectivity

Synthetic Evaluation has benchmarked Clarifai’s Compute Orchestration with the GPT-OSS-120B mannequin—one of the vital superior open-source massive language fashions obtainable right now. The outcomes underscore Clarifai as one of many prime {hardware} and GPU-agnostic engines for AI workloads the place velocity, flexibility, effectivity and reliability matter most. 

What the benchmark reveals (P50, final 72h; single question, 1k-token immediate):

• Excessive throughput: 313 output tokens per second—among the many very quickest measured on this configuration.

• Low latency: 0.27s time-to-first-token (TTFT), so responses start streaming nearly immediately.

• Compelling worth/efficiency: Positioned within the benchmark’s “most engaging quadrant” (excessive velocity + low worth).

Pricing that scales:

Clarifai affords GPT-OSS-120B at $0.09 per 1M enter tokens and $0.36 per 1M output tokens. Synthetic Evaluation shows a blended worth (3:1 enter:output) of simply $0.16 per 1M tokens, inserting Clarifai considerably beneath the $0.26–$0.28 cluster of opponents whereas matching or exceeding their efficiency.

Beneath is a comparability of output velocity versus worth throughout main suppliers for GPT-OSS-120B. Clarifai stands out within the “most engaging quadrant,” combining excessive throughput with aggressive pricing.

Output Pace vs. Value

This chart compares latency (time to first token) in opposition to output velocity. Clarifai demonstrates one of many lowest latencies whereas sustaining top-tier throughput—inserting it among the many best-in-class suppliers.

Latency vs. Output Pace

Why GPT-OSS-120B Issues

As one of many main open-source “GPT-OSS” fashions, GPT-OSS-120B represents the rising demand for clear, community-driven alternate options to closed-source LLMs. Operating a mannequin of this scale requires infrastructure that may not solely ship excessive velocity and low latency, but additionally preserve prices below management at manufacturing scale. That’s precisely the place Clarifai’s Compute Orchestraction makes a distinction.

Why This Benchmark Issues

These outcomes are greater than numbers—they present how Clarifai has engineered each layer of the stack to optimize GPU utilization. With CO, a number of fashions can run on the identical GPUs, workloads scale elastically, and enterprises can squeeze extra worth out of each accelerator. The payoff is quick, dependable, and cost-efficient inference that may assist each experimentation and large-scale deployment.

Test the complete benchmarks on Synthetic Evaluation right here. 

Right here’s a fast demo of the way to entry the GPT-OSS-120B mannequin within the Playground.

Native Runners

Native Runners allow you to develop and run fashions by yourself {hardware}—laptops, workstations, edge containers—whereas making them callable by means of Clarifai’s cloud API. Clarifai handles the general public URL, routing, and authentication; your mannequin executes regionally and your knowledge stays in your machine. It behaves like another Clarifai‑hosted mannequin.

Why groups use Native Runners

• Construct the place your knowledge and instruments stay. Preserve fashions near native recordsdata, inside databases, and OS‑degree utilities.

• No customized networking. Begin a runner and get a public URL—no port‑forwarding or reverse proxies.

• Use your personal compute. Convey your GPUs and customized setups; the platform nonetheless supplies the API, workflows, and governance round them.

New: Ollama Toolkit (now within the CLI)

We’ve added an Ollama Toolkit to the Clarifai CLI so you may initialize an Ollama‑backed mannequin listing in a single command (and select any mannequin from the Ollama library). It pairs completely with Native Runners—obtain, run, and expose an Ollama mannequin through a public API with a minimal setup.

The CLI helps --toolkit ollama plus flags like --model-name, --port, and --context-length, making it trivial to focus on particular Ollama fashions.

Instance workflow: run Gemma 3 270M or GPT‑OSS- 20B regionally and serve it by means of a public API

1. Decide a mannequin in Ollama.

   • Gemma 3 270M (tiny, quick; 32K context): gemma3:270m. 

   • GPT‑OSS 20B (OpenAI open‑weight, optimized for native use): gpt-oss:20b. 

2. Initialize the mission with the Ollama Toolkit.
   Use the command above, swapping --model-name to your choose (e.g., gpt-oss:20b). This can create a brand new mannequin listing construction that’s suitable with the Clarifai platform. You possibly can customise or optimize the generated mannequin by modifying the 1/mannequin.py file as wanted.

3. Begin your Native Runner.
   From the mannequin listing:

    

   The runner registers with Clarifai and exposes your native mannequin through a public URL; the CLI prints a prepared‑to‑run shopper snippet. 

4. Name it like every Clarifai mannequin.
   For instance (Python SDK):

   Behind the scenes, the API name is routed to your machine; outcomes return to the caller over Clarifai’s safe management aircraft.

    

Deep dive: We printed a step‑by‑step information that walks by means of operating Ollama fashions regionally and exposing them with Native Runners. Test it out right here.

Strive it on the Developer Plan

You can begin without cost, or use the Developer Plan—$1/month for the primary 12 months—which incorporates as much as 5 Native Runners and limitless runner hours.

Take a look at the complete instance and setup information within the documentation right here.

Billing

We’ve made billing extra clear and versatile with this launch. Month-to-month spending limits have been launched: $100 for Developer and Important plans, and $500 for the Skilled plan. For those who want increased limits, you may attain out to our group.

We’ve additionally added a brand new bank card pre-authorization course of. A brief cost is utilized to confirm card validity and obtainable funds — $50 for Developer, $100 for Important, and $500 for Skilled plans. The quantity is routinely refunded inside seven days, guaranteeing a seamless verification expertise.

Management Middle

• The Management Middle will get much more versatile and informative with this replace. Now you can resize charts to half their authentic dimension on the configure web page, making side-by-side comparisons smoother and layouts extra manageable.
• Charts are smarter too: the Saved Inputs Value chart now appropriately reveals the common price for the chosen interval, whereas longer date ranges routinely show weekly aggregated knowledge for simpler readability. Empty charts show significant messages as an alternative of zeros, so that you at all times know when knowledge isn’t obtainable.
• We’ve additionally added cross-links between compute price and utilization charts, making it easy to navigate between these views and get a whole image of your AI infrastructure.

Further Adjustments 

• Python SDK: Fastened Native Runner CLI command, up to date protocol and gRPC variations, built-in secrets and techniques, corrected num_threads defaults, added stream_options validation, prevented downloading authentic checkpoints, improved mannequin add and deployment, and added consumer affirmation to stop Dockerfile overwrite throughout uploads.
  Test all SDK updates right here.
• Platform Updates: Added a public useful resource filter to shortly view Group-shared assets, improved Playground error messaging for streaming limits, and prolonged login session length for Google and GitHub SSO customers to seven days.
  Discover all platform modifications right here.

Prepared to start out constructing?

With Native Runners, now you can serve fashions, MCP servers, or brokers straight from your personal {hardware} with out importing mannequin weights or managing infrastructure. It’s the quickest method to take a look at, iterate, and securely run fashions out of your laptop computer, workstation, or on-prem server. You possibly can learn the documentation to get began, or take a look at the weblog to see the way to run Ollama fashions regionally and expose them through a public API.