Amazon’s TorchNeuron lets PyTorch code run natively on Trainium chips, taking aim at Nvidia’s software moat
Amazon is releasing TorchNeuron, a free open-source tool that lets developers run PyTorch code on its Trainium AI chips with a single configuration change and no rewriting, in a bid to erode a key advantage behind Nvidia’s dominance.
This story was prepared automatically by software.
Amazon is introducing TorchNeuron, a tool that lets its custom Trainium AI chips run code written in PyTorch, the most popular AI programming framework, without modification. As Reuters reported, the move is aimed at eroding the software advantage that underpins Nvidia’s dominance of the AI computing market.
One configuration change
PyTorch, the open-source framework stewarded by Meta, is used by most AI developers; Amazon says 85% of AI researchers and developers write their code in it. Until now, moving to new hardware typically meant rewriting and revalidating code that had taken months to perfect. Nvidia has spent more than a decade building CUDA, the software that makes PyTorch code run well on its chips.
TorchNeuron registers Trainium as a standard PyTorch device, so, according to the PyTorch blog, pointing a model at .to('neuron') is enough to run it on Trainium. It supports eager execution, torch.compile, distributed training, the PyTorch profiler and custom kernels through the Neuron Kernel Interface, on Trainium2 and Trainium3. It is being released as open source under the Apache 2.0 licence, with a public beta going live at PyTorch Conference North America on October 20.
Early users report savings
“This has been about meeting customers where they’re at,” Colin Brace, vice president of software at Amazon’s Annapurna Labs chip unit, told Reuters. He said more than 30 early customers, spanning robotics, world models and voice AI, are already using the software, and that switching to Amazon’s chips can bring steep cost savings.
In one example cited on the PyTorch blog, speech AI company Deepgram said Trainium processed about 60% more input audio seconds per dollar than its existing GPU baselines in its benchmarks. Hugging Face said TorchNeuron makes Trainium a native backend for its Transformers and Diffusers libraries.
The chip race moves to software
Reuters noted that other cloud providers with their own AI chips, such as Google, are also working to lure developers away from Nvidia with better PyTorch tools. As AI labs look beyond Nvidia to meet their computing needs, software compatibility is becoming a key battleground in the chip market.
Sources: Reuters (via U.S. News), Amazon, PyTorch. Image: Amazon.
Related articles
