ASIC: The Economics of Custom Chips

ASIC / Custom chips are not a new thing.

Broadcom mentioned a $10 billion order from the fourth "mysterious customer" (OpenAI) at its earnings call, which has elevated the discussion of "GPU vs ASIC" to a new level.

Taking the opportunity over the weekend, I reorganized the economics of ASICs and some narrative changes in the chip industry. The article borrowed an analysis framework from TD Cowen, which is well-written.

ROI Framework for Self-Developed Chips

1/ There is no real ASIC vs GPU; to put it bluntly, everyone is an "ASIC";

• In traditional views, GPUs sacrifice performance for flexibility, while ASICs are optimized for specific tasks and are more efficient; this framework is no longer as applicable because the most advanced GPUs have become highly specialized at the chip architecture level, with most computing power dedicated to AI acceleration, particularly matrix multiplication, which has little relation to graphics processing;

• In large language models (LLMs), over 90% of the computation is matrix multiplication, and these chips are essentially ASICs born for the same specific application (AI acceleration). (For example, NV's Tensor Core / Google's Matrix Multiply Unit / Amazon's Tensor Engine); the yellow part in the image below.

• To some extent, everyone is an "ASIC" in AI;

2/ The essence of distinguishing ASIC vs GPU is not the technical route, but the business model. From another perspective, the main differences are:

• A. Merchant chips produced by chip manufacturers and sold to many people.
• B. Custom chips produced by cloud providers for their own use. This boundary has begun to blur since Google previously announced selling TPUs.

In simple terms, it’s about which route is more "cost-effective" + has long-term strategic significance.

Since we are starting from the perspective of "money," analyzing with an ROI framework may be more intuitive. Simplifying things, there are only two situations: "buy" vs "build";

3/ The temptation vs cost of "self-development"

• Commercial chips have very high profits (I don't need to elaborate on this; everyone knows. Just look at Nvidia's $4 trillion market value); 80% of chip costs are converted into Nvidia's profits

• Self-research is certainly aimed at capturing this portion of profit, but custom chips are highly "binary." They either succeed or fail; there is no "middle state" where my self-researched chip, although poor, can still run to some extent.

4/ Why do custom chips have the saying "failure means death"? TD Cowen provides a "bold statement" that as long as your self-researched chip cannot achieve 50% of the performance of NVIDIA's most advanced chips, then it is all electronic waste, and even if it has been taped out, it should not go into mass production.

Several basic assumptions (the assumptions are a bit long, so they are processed in gray),

• Understand this chip investment using the concept of "AI factory" as used by Huang; the "product" of this factory is the "tokens" generated through API calls.

• There exists a public price for "inference services" in the market; OpenAI, Anthropic, and Google all provide their LLM services in the form of APIs, and this price is usually billed as "$/1M tokens."

• Hardware performance determines the production speed of the "product"; the performance of an AI chip can be directly reflected in how many tokens it can generate per second while processing a model, i.e., throughput (measured in tokens/second). The stronger the chip's performance, the faster the token generation speed.

• In other words, revenue/second = (price/token) × (throughput, tokens/second)

With the above assumptions,

• Performance = computing power = revenue, the core competitiveness of the AI factory; a chip with double the performance of its competitor can generate double the revenue in the same amount of time.

• The current API pricing in the market is actually based on a hidden common foundation: the vast majority of them run on NVIDIA's GPUs. This makes NVIDIA's performance the "gold standard" for market pricing (others can only "swallow" this standard).

• In other words, your revenue does not solely depend on the absolute performance of your own chip, but rather on the relative performance of your chip versus NVIDIA's chip (because NVIDIA defines the revenue standard);

End of assumptions, straight to the conclusion. (TD Cowen's report has a very detailed calculation, but due to space limitations, I will jump directly to the conclusion.)

• The IRR for buying Huang's chips is very high, around 24% (there's no need to take this number too seriously, it's mainly to explain the decision-making thought process). "The more you buy, the more you save";

• If the self-developed products from cloud manufacturers can achieve 70% performance of NVIDIA chips, then the IRR can reach 35% (the profits from NVIDIA are saved, converted into cost savings, and ultimately transformed into IRR);

• If the self-developed products can achieve 50% performance of NVIDIA chips, then the IRR is roughly the same, reaching the critical point. At this point, you can still push for mass production because it has strategic significance.

• However, if your self-developed products can only achieve 30% performance of NVIDIA chips, then the IRR drops sharply (this is not linear, as many costs like electricity and data centers are fixed costs), and you should decisively abandon self-development; even if you have tape-out, do not proceed to mass production. In this case, the IRR may even fall below your company's cost of capital/WACC; (in the context of OpenAI, they are using VC money, and their WACC is certainly much higher than Google's);

Analyzing Upcoming Industry Changes with a Framework

Let's assume the above hypotheses and framework provide some insights, and take a look at what is happening in the industry now.

1/ Google's TPU has just reached the 50% critical point, so we see that this quarter TPU is directly ramping up production (driven by the demand for Gemini). This is also why things on the TPU chain have been so hot recently.

2/ The OpenAI orders announced by Broadcom may have also surpassed the 50% performance threshold, so management mentioned that orders have already been placed. (We previously mentioned in our public account that OpenAI has also poached many people from Google's TPU team);

• However, this "mass production" still has an uncertainty, which is that you need to ensure that when you go into mass production, your chips can still achieve 50% performance of NVIDIA (because NVIDIA has not stagnated, but is continuously enhancing their chip capabilities)
• This decision of whether to "mass produce" or not does not depend on Broadcom or OpenAI, but rather on the "relative capabilities" and "relative iteration speeds" of both chips at the time of mass production.

3/ OpenAI, having lost Microsoft as a backer, faces greater financial pressure. It may need to take a different approach to reduce costs by relying on custom chips? This is a report from TI over the weekend.

• The prediction from three months ago indicated lower capital consumption.
• The current prediction shows greater capital consumption, then suddenly turns positive. It looks like a story of "capital investment generating returns"... using ASICs to paint a big picture for investors?

4/ After all, OpenAI wants to do everything, including cloud computing.

5/ Looking at the revenue side, there have been no adjustments to revenue recently. What has been adjusted is the long-term revenue... the strategy of painting a big picture has its traces.

6/ Finally, here’s a chart from UBS showing the ASIC roadmap and costs for 2026.


Source: 180K, Original title: "ASIC / Custom Chip Economics (September 7)"

Risk Warning and Disclaimer

The market has risks, and investment requires caution. This article does not constitute personal investment advice and does not take into account the specific investment goals, financial conditions, or needs of individual users. Users should consider whether any opinions, views, or conclusions in this article are suitable for their specific circumstances. Investment based on this is at one's own risk