Sakana AI Bets on Nvidia's Nemotron to Prove Orchestrated Small Models Can Match Frontier LLMs

Sakana AI is folding Nvidia's open-source Nemotron models into Fugu, its language model orchestrator, in a move designed to test a contrarian thesis: that a well-coordinated ensemble of smaller models can hold its own against monolithic frontier systems from OpenAI, Google, and Anthropic.

Fugu doesn't function like a conventional single model. Instead, it dynamically routes tasks across multiple language models, selecting whichever one is best suited to a given query rather than relying on one generalist system to handle everything. Adding Nemotron gives it new tools to work with, chief among them Nemotron 3 Nano Omni, a multimodal model capable of processing text, images, video, and audio, and Nemotron 3 Ultra, a sparse model built with roughly 550 billion total parameters but only about 55 billion active at inference time. Nvidia's Nemotron lineup has built a reputation for strong performance in coding, tool calling, and instruction following, three areas that matter directly for how well an orchestrator like Fugu can delegate work.

Sakana AI has already run internal benchmarks suggesting the approach has legs. According to the company, its Fugu Ultra configuration performed comparably to Anthropic's Claude 3.5 and Google's Gemini Preview, results that, if they hold up under outside scrutiny, would lend real weight to the idea that intelligently combined smaller models can rival the output of a single massive one at a fraction of the underlying compute footprint.

That claim comes with a caveat. Independent evaluators who tested earlier versions of Fugu flagged concerns about latency and operating cost, the kind of overhead that orchestration systems tend to introduce when they have to route, evaluate, and sometimes chain calls across several models rather than querying one. Whether Nemotron's addition helps offset those costs or compounds them is an open question Sakana AI hasn't yet addressed in detail.

The company has also declined to specify when the Nemotron-equipped version of Fugu will actually ship, saying only that the integration is planned for an upcoming release. That leaves the more interesting technical questions, how routing decisions get made in practice, what the real-world latency looks like once Nemotron 3 Ultra's sparse architecture is in the loop, and whether the Claude- and Gemini-level benchmark results survive contact with independent testing, still unanswered.

For Nvidia, the partnership extends Nemotron's reach beyond direct API use into a third-party orchestration layer, a validation point for its open-weight strategy. For Sakana AI, it's a chance to make the case that orchestration, not scale alone, is a viable path to frontier-level performance, provided the speed and cost concerns that dogged earlier Fugu releases don't resurface.