NVIDIA's Nemotron 3.5 Lightning Lands on AWS SageMaker JumpStart

NVIDIA's Nemotron 3.5 Lightning model is now available through Amazon SageMaker JumpStart, giving AWS customers direct access to a model built specifically for high-throughput enterprise agents. The listing marks another step in NVIDIA's push to place its Nemotron family inside mainstream cloud deployment pipelines.

The model is distilled from the larger Nemotron 3 Ultra, and it uses a hybrid Mixture-of-Experts design with 30 billion total parameters but only 3 billion active per forward pass. That sparsity is the core of its efficiency pitch: fewer parameters engaged per token means faster inference without matching the compute cost of a dense model of similar scale.

NVIDIA claims Nemotron 3.5 Lightning reaches up to 4x the throughput of comparable models, hitting roughly 410 tokens per second. It also reports 30% faster task completion in enterprise workloads, a gain the company attributes to the model's architecture rather than raw parameter count.

Built for Long-Running Agents

Context length is where the model makes its more technical claim. Using DFlash speculative decoding, Nemotron 3.5 Lightning can process up to 1 million tokens of context, a scale aimed squarely at persistent, long-running AI agents that need to retain extensive state across sessions.

That focus on agents rather than chat is deliberate. NVIDIA positions the model for high-throughput enterprise automation, the kind of workload where an agent might be running continuously, executing tool calls, and maintaining memory over hours or days rather than answering isolated prompts.

Notably, Nemotron 3.5 Lightning is fully open-trained on open datasets. That openness matters for enterprises with compliance or data-governance requirements, since it lets organizations post-train the model on their own tools, internal workflows, and policy constraints rather than working around a closed, vendor-controlled base model.

Why the SageMaker Route Matters

Availability through SageMaker JumpStart lowers the integration barrier for AWS-native teams considerably. Rather than standing up custom inference infrastructure, developers can deploy Nemotron 3.5 Lightning with the same JumpStart tooling used for other foundation models on the platform.

The combination of a sparse MoE architecture, million-token context handling, and open training data suggests NVIDIA is chasing a specific niche: enterprises that want agentic AI at scale without the latency and cost penalties of dense frontier models. Whether that efficiency claim holds up under independent benchmarking outside NVIDIA's own figures remains to be tested by early adopters on SageMaker.