Amazon’s AI chips are considerably cheaper than NVIDIA’s GPUs. And these are the chips Anthropic makes use of to coach (and possibly run) Claude.
Amazon’s Trainium chip has been making headlines recently. Amazon Net Providers (AWS) has been aggressively pitching this chip to clients, providing server leases with computing energy akin to NVIDIA’s H100 GPUs however at a steep low cost (posts on X recommend it’s only a quarter of what NVIDIA expenses).
It’s a daring transfer, and there’s a mixture of technique, economics, and market dynamics driving it.
NVIDIA’s H100 GPUs are, as you would possibly know, the go-to for AI coaching, powering every little thing from giant language fashions to picture/video era. However they arrive with a hefty price ticket. Final I checked, working an EC2 P5.48xlarge occasion prices round $98.32/hour (on-demand pricing). Certain, it’s a beast of a setup, with 8 H100 GPUs, 640 GB of VRAM, and a couple of TB of RAM, however that value nonetheless stings.
In addition to, they’re at all times brief on provide.
The availability scarcity is well-documented. TSMC’s 4nm manufacturing course of simply can’t sustain with the loopy demand for H100s. And it’s not for lack of effort. Yields (the proportion of chips that truly work) take a severe hit…
