Nvidia’s dominance in AI isn’t only about its chips. It’s also about software: years of work making sure the tools developers already use run best on Nvidia hardware. On Monday, October 19, Amazon Web Services took direct aim at that advantage with a new tool called TorchNeuron.

The pitch is simple. If you’ve built an AI model with PyTorch, you can now run it on Amazon’s own Trainium chips without rewriting your code. Here’s what that means and why it could matter well beyond Amazon’s cloud.

What TorchNeuron does

PyTorch is the free, open-source toolkit that a huge share of AI developers use to build, train and run models. Amazon says about 85% of AI researchers and developers write their code in it. Facebook originally created PyTorch, and since 2022 it has lived under the Linux Foundation.

TorchNeuron is a native PyTorch “backend” for Trainium. According to Amazon’s announcement, existing PyTorch code can run on Trainium with a single configuration change and no rewrite. It plugs into the standard mechanism PyTorch already uses to let new hardware register itself, so the usual tools for testing, debugging and scaling models keep working.

A few key details from Amazon:

  • Supported chips: Trainium2 and Trainium3.
  • Open source: TorchNeuron is being released under the Apache 2.0 license at PyTorch Conference North America on October 20.
  • Scaling: It works with TorchTitan, PyTorch’s native framework for distributed training, including Fully Sharded Data Parallelism.
  • Testing: Amazon says it tests TorchNeuron against hundreds of thousands of AI models on every build.
  • Ecosystem: Hugging Face has enabled native Trainium support in its popular tools, and Amazon has made a major contribution to vLLM, an open-source tool for speeding up inference.

In a side-by-side demo, Amazon says engineers ran the same training script on a conventional chip and on Trainium, changing only the setting that picks the hardware, and the model learned the same things at the same rate.

Why this is a shot at Nvidia

As TechCrunch explains, Nvidia has spent years making sure its CUDA software supports PyTorch apps seamlessly. The result is that Nvidia hardware is deeply embedded in PyTorch, essentially by default. Switching to another chip has often meant months of rewriting and revalidating work, which is a powerful reason to stay put.

TorchNeuron tries to remove that friction for Trainium. And Amazon is going a step further: it says it is contributing changes upstream to PyTorch itself, working with the community to remove code that ties the framework to specific chips. The stated goal is to make PyTorch more hardware-agnostic for everyone, not just Trainium users.

“The real story is removing the lock-in that kept developers from having a real choice at all,” Colin Brace, VP of Technology Software at AWS Annapurna Labs, told TechCrunch.

Amazon isn’t the first to try this. TechCrunch notes PyTorch users can already port apps to AMD GPUs or Apple’s chips with a configuration change. Amazon’s argument is that Trainium comes with the rest of AWS attached, so startups don’t have to piece together compute from one provider and everything else from another.

Who’s already using it

Amazon named several early adopters across very different fields. Voice AI company Cartesia ran its standard inference pipeline on Trainium with minimal changes to its PyTorch code. The Lieber Institute for Brain Development is using it to explore AI approaches in drug discovery research. Brace also pointed TechCrunch to Deepgram, an AI voice company, which he said is processing 70% more audio per dollar after moving to Trainium.

Those are Amazon’s own figures and customer examples, so it’s fair to wait for independent benchmarks before drawing big conclusions about price and performance.

What it means for the rest of us

Most people will never type a line of PyTorch. But the economics of AI chips shape the cost of the apps and services built on top of them. If developers can move models between chips without a painful rewrite, chipmakers have to compete harder on price and performance, and that pressure can eventually show up as cheaper or faster AI features.

There are open questions too. TechCrunch points out that part of PyTorch’s appeal is hiding hardware headaches, and pulling Nvidia-specific code out of the framework could either give developers more choice or create new configuration hassles. We’ll find out as the open-source release lands and developers try it on real workloads.

The bottom line

TorchNeuron is Amazon’s clearest attempt yet to loosen Nvidia’s grip by going after software, not just silicon. With the code going open source at PyTorch Conference and changes headed upstream into PyTorch itself, the next few months will show whether “one config change” holds up in the real world, and whether the AI chip race is about to get a lot more competitive.

Image: Free computer server room image via Rawpixel, CC0 1.0