AI Inference Startups Are Quietly Buying Their Own Chips

AI Summary

AI inference startups that resell cloud computing capacity are beginning to buy their own chips as they struggle to secure enough compute for customers. The shift could change their business economics, while reliance on smaller neocloud providers raises operational risk, underscored by a reported hack of Fireworks.

The fast-growing crop of startups that lease computing capacity from cloud firms and then resell it, without spending a nickel on their own AI chips, are at a crossroads. Finding it harder to line up enough computing capacity to serve their customers, some are starting to buy their own chips, which is likely to drastically change the economics of their business. The difficulty in getting enough compute also has pushed some of these startups to look for computational resources from dozens of smaller neoclouds, which tend to have more available capacity than larger hyperscalers such as Microsoft Azure, Amazon Web Services and Google Cloud Platform. News this week of a hack of Fireworks, one of the best known of these startups, highlights the risk of that approach.