Skip to content
Neural Daily

ToolsNews

Modal: Nvidia Vera CPU doubles agent sandbox throughput

Modal will add Nvidia's Arm-based Vera CPU to power its Sandboxes product. In Modal's own tests, Vera delivered up to twice the task throughput of an equivalent x86 Intel GNR machine under load. Modal has until now run only on x86.

By: Mikola Máté Patrik · · 3 min read

Rows of green glass cubes, the cover image of Modal's NVIDIA Vera Sandboxes blog post
Image: Modal · Press kit · Official blog cover image · Source

Why the CPU matters here

Sandboxes are isolated environments where AI agents run the code they write. Modal's blog explains that these workloads are hard to predict (Modal). A sandbox may sit mostly idle waiting on an LLM, then spike to 10x its baseline CPU use during an npm install or an expensive grep. Hundreds of them run on each host, so one tenant's spike can slow down another's.

Every second of CPU time also costs more than the CPU itself. For background agents, faster runs mean faster answers and lower costs. In reinforcement learning, expensive GPUs often sit idle while sandboxed rollouts finish.

What Modal measured

Modal built its own benchmark, Agentless Agent Bench. It runs the oracle solutions and verifiers of 50 Terminal-Bench tasks without network calls in concurrent Docker containers. The tasks model the CPU work agents typically do: compilation, file grepping, git operations, emulation.

Run one at a time on an idle machine, Vera was often faster than chips from Intel, AMD and AWS Graviton, but not always, and not by much.

Modal benchmark chart of serial task latency for NVIDIA Vera, Intel GNR, AWS Graviton 4 and 5 and AMD Zen 5
Modal · Press kit · Official benchmark chart from the blog post · Source

Source: Modal. Serial task latency in seconds on selected tasks: NVIDIA Vera, Intel GNR, AWS Graviton 4 and 5, AMD Zen 5.

Under load, the picture changed. Modal reports up to double the task throughput on Vera compared with an equivalent x86 GNR machine, while per-task latency stayed equal or lower. Rival chips hit throughput plateaus while Vera kept improving. Modal credits Vera's high memory bandwidth.

What comes next

Supporting Arm is a big change for Modal, requiring changes across its infrastructure and image build system. The Next Web notes that Modal did not say when Vera-powered sandboxes will reach customers (The Next Web). According to the outlet, Sandboxes bring in $300 million a year, and Modal raised $750 million at a $15.75 billion valuation last week.

Why it matters

These are Modal's own results and have not been independently checked. If they hold, the CPU may be as much of a bottleneck as the GPU for agent workloads. For teams running many parallel agents or RL rollouts, throughput under dense packing is the metric to watch, not single-task latency.

    Produced with the help of AI, reviewed by an editor. How we make our articles

    Reviewed and edited by the Neural Daily desk. About

    Related articles