Does Generative AI Need a New Business Model?

Generative AI is expensive. Training a large language model costs tens or hundreds of millions of dollars. OpenAI is said to be spending $12 billion on Nvidia chips next year—equal to roughly 40% of Apple’s entire R&D spend. Big Tech’s investment in generative AI startups is somewhere around $20 billion. Clearly, those “in-the-know,” think there is a massive opportunity to get a return on these kinds of capital expenditures.

Here’s the issue: the current business model doesn’t make sense because increasing usage conflicts with profits. 

There are three major costs involved in generative AI: 

1. Infrastructure: data centers, servers, GPU chips, etc
2. Training: operating costs to train models
3. Inference: operating costs to generate content for users

The first two costs—infrastructure and training—are knowable by the companies building generative AI. They can predict the capital investment and operating expenses to train a model based on the size of the model being trained. There’s clearly some margin of error because training isn’t an error-free process, but it’s logical that estimates are within an acceptable margin of error. If these were the only costs, one could create a pricing model that would provide a decent return on capital. 

Inference costs, however, I would argue are mostly unknowable today. In a recent article in the Wall Street Journal, someone in-the-know said that Microsoft’s cost-to-serve a GitHub Copilot user in the first three months of 2023 averaged more than $30 per month, while the most active users cost $80 per month. It seems fair to assume that Microsoft currently loses money on GitHub Copilot, since it charges $10 per month versus an average cost of $30 per month. But what if usage grows and the average cost increases to $50 or $80 or more? As a former Wall Street analyst who covered Microsoft, these numbers give me heartburn.

Microsoft isn’t alone. OpenAI, Google, Anthropic, and others are selling generative AI services for a fixed fee per month. The problem is that the cost-to-serve is not fixed as well—the cost varies based on usage. As usage of generative AI increases, the cost-to-serve increases without any increase in revenue. 

There is good reason to believe that costs will continue to rise:

• As generative AI systems become more useful, usage will increase.
• As generative AI systems become multi-modal, users will do more with more modes—text, images, audio, and video—increasing the number of prompts and responses.
• As generative AI technology is embedded throughout existing software products, users will use them more. In fact, it’s likely that the entire goal of vertically integrated cloud companies is to create frictionless, habituated, usage where AI is embedded in the workflow.

Generative AI companies’ goal is to create highly intelligent systems that will be tightly integrated with users’ lives. What happens to costs if users use these systems for hours per day? Character.ai claims that its users chat with its avatars for more than two hours per day on average. How many inferences are made in those two hours and how much is Character.ai losing on its $9.99 per month subscription? Is there a business model to support this usage?

The reality is that we don’t know. If the Wall Street Journal’s source is correct, the current model of charging users $10 to $20 per month for generative AI is unsustainable. But there are very few consumers who can afford to pay something like $100 per month, limiting the total B2C market size. 

Businesses will need stronger evidence than exists today of value add—through either labor amplification or elimination—to pay significantly more. There is a good case for that value with highly paid knowledge workers, but it’s less clear about various other business use cases that are closer to the edge of the business. Microsoft clearly believes that businesses will pay more, announcing that Microsoft Office Copilot will be priced at $30 per user per month—a significant step up from the starting price of $5 per month for Office. But will that $30 per month cover the cost-to-serve workers who might chat with Copilot for hours every day? 

One option to increasing price is to limit usage. Currently, OpenAI and others limit the number of prompts per day, capping cost. We have experienced ChatGPT shortening responses by saying “and so on” which, to us, appears to be designed to reduce the response length and cost. These tactics are unsustainable, though, if OpenAI wants ChatGPT to become an essential tool. Usage of essential workflow tools can’t be metered or gated like a more static system like storage. If you become reliant on working with a generative AI system, you need it to work—now. You can’t rely on the tool if you have a deadline today, but you’ve hit your prompt limit. 

The variable cost of generative AI seems fundamentally different from other internet services. Search, e-commerce, publishing, and social media all benefit from variable revenue that offsets variable costs by charging per transaction, click, or view. All of these industries have been able to lose money in the early days and “make it up on volume” long-term. With its current business model, however, generative AI will lose more with volume. Improving products and increasing usage is, today, in conflict with company profits. 

It’s possible that generative AI companies will explore variable revenue sources. But the most obvious—advertising—feels in direct conflict with user value. Displaying ads based on a user prompt might seriously erode trust. And incorporating ad copy in a response would seriously erode usefulness. When I ask for a picture of a truck, how will I feel if Ford has paid for that truck to be an F-150? 

Losing money in the early days of generative AI is expected. And capping usage is acceptable as people are still in experiment mode. But the long-term vision of generally intelligent systems that will become essential digital partners requires always-on availability and acceptable affordability. Cracking this business model challenge seems as important as creating the next great model.