A routine fraud alert does not need a frontier reasoning model to judge whether it merits a human’s attention. It needs a fast, cheap, repeatable decision — the kind of high-volume step that eats most of an agent’s token budget. NVIDIA and AWS are betting that a single specialized model, not a giant general one, should handle that work, and they have made it far easier to deploy AWS launch post.
A 30B model that runs like a 3B one
The model is Nemotron 3.5 Lightning, a 30-billion-parameter Mixture-of-Experts system that activates only 3 billion parameters per forward pass AWS launch post. According to the AWS launch post, NVIDIA reports the model delivers up to 4x higher throughput and up to 30% faster task completion on high-volume agentic workloads. Its hybrid design, 1M-token context window, and speculative decoding are tuned for long, multi-turn agent sessions rather than one-shot answers.
The goal is cheaper repetition, not smarter reasoning.
Venu Kanamatareddy noted, in the AWS launch post he authored, that teams can deploy Lightning through JumpStart “without configuring the serving infrastructure yourself” AWS launch post. That removes the largest frictional cost of running a custom model: standing up and maintaining the serving stack.
NeMo Switchyard routes the busywork
NeMo Switchyard is the other half of the story NVIDIA announcement. NVIDIA calls Lightning the highest-efficiency model in its class for long-running agentic AI, and released Switchyard as an open-source routing library that sends each agent step to the model best suited to it — a small specialized model for classification and extraction, a frontier model for planning. Enterprises can wire it to their own mix of open, proprietary, and NVIDIA models without rewriting their agent code.
The small-model bet, and its catch
The efficiency case is real, but it is not free. Routing every step to the right model adds its own operational overhead, and a specialized model still cannot plan a multi-stage workflow the way a frontier model can. The trade-off is deliberate: pay less per high-frequency call, and reserve the expensive model for the steps that actually need it.
Nemotron 3.5 Lightning is open and customizable, NVIDIA says, so teams can post-train it on their own tools and keep the resulting weights NVIDIA announcement. That control matters for organizations that do not want agent logic locked inside a single vendor’s hosted API. Ubuntu users can also pull the model with a single command, as our guide to running Nemotron 3.5 Lightning on Ubuntu shows zBrandco guide.
A model that runs on a single GPU, holds a long session in memory, and can be owned and modified by the team using it changes what “an agent” has to cost. The interesting question is no longer whether small models can share the load, but how many workloads were quietly over-provisioned for years.
