NVIDIA’s VFlash speeds robot trajectory decoding 12x, cuts prediction errors by 61%

NVIDIA researchers have developed a new visual planning method that accelerates robot trajectory decoding by up to 12 times, while reducing 3D trajectory prediction errors by as much as 61% in specific tests.

Called VFlash, the method allows vision-language models (VLMs) to generate complete robot trajectories and other spatial predictions in a single pass, rather than producing coordinates one at a time.

In robot manipulation tests, VFlash also cut overall inference latency by about 80% on NVIDIA’s Jetson Thor and DGX Spark platforms, potentially helping robots respond faster to changing environments.

The research addresses a limitation in conventional VLMs, which often generate spatial predictions one coordinate at a time. As robots need to predict longer trajectories, track more objects and process increasingly detailed spatial information, this sequential approach can increase processing time.

VFlash instead uses the known structure of visual predictions to generate an entire geometric object or trajectory in one forward pass.

In tests involving robot manipulation, the researchers reported that VFlash reduced inference latency from 3,552.7 milliseconds to 703.3 milliseconds on NVIDIA’s Jetson Thor platform. This represents an approximately 80% reduction compared with conventional autoregressive decoding.

On DGX Spark hardware, latency fell from 3,356.5 milliseconds to 649.9 milliseconds.

Model predicts complete trajectories

VFlash works by identifying the structure of a requested visual prediction before generating its coordinates. For example, a bounding box has a fixed coordinate layout, while a trajectory can contain a variable number of two-dimensional and three-dimensional points.

The model uses a lightweight diffusion-based drafter to predict these coordinates jointly in a single forward pass. Unlike conventional speculative decoding, which checks generated tokens against a target model, VFlash can accept structurally valid visual predictions without requiring token-by-token agreement.

The researchers evaluated VFlash across several two-dimensional and three-dimensional robot trajectory prediction tasks. The method reduced prediction errors in multiple tests compared with the underlying autoregressive approach.

In three-dimensional trajectory tests, it lowered average displacement error by as much as 61%, depending on the dataset and training configuration.

The researchers also tested VFlash on 3D point-motionforecasting, where it predicted the future movement of objects across video frames. Preliminary single-rollout measurements showed approximately 25 times faster generation than autoregressive decoding.

Faster predictions aid robots

The researchers also investigated whether faster visual predictions could improve the performance of downstream robot-control systems.

They connected VFlash-generated trajectories to a world action model evaluated on MolmoSpaces, a benchmark for roboticmanipulation. The model achieved an average success rate of 70.2% across four task suites, compared with 62.6% for the baseline system.

The largest improvements appeared in pick-and-place and object-closing tasks, where success increased by 11.6 and 12.8 percentage points, respectively.

Beyond robotics, the researchers tested VFlash for autonomous driving using NAVSIM and Waymo datasets. The model generated driving trajectories with competitive benchmark performance, while reinforcement learning further improved its planning results.

The findings suggest that generating detailed visual plans more efficiently could help physical AI systems respond faster to changing environments while maintaining spatial accuracy.

The research, titled VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding, was published by NVIDIA researchers and collaborators in November 2026.

Source: https://interestingengineering.com/ai-robotics/nvidia-vflash-robot-planning-speed-trajectory-accuracy