Why Is AI Driving Up the Cost of Computers and Game Consoles?

The AI industry is now heavily investing in computer memory, directly collaborating with manufacturers to develop chips worth billions. These chips are the same ones found in smartphones, laptops, and gaming consoles. This could either drive prices up significantly or cause shortages, hindering production.

What drives AI’s need for memory?

AI models are tremendously large, consisting of grids filled with billions or trillions of parameters (values stored in memory) that undergo complex and repetitive calculations. This process forms the basis of how large language models process input and generate output.

Transferring this expansive data between affordable yet slower hard drives (often referred to as storage) and the processor results in a significant bottleneck. To mitigate this, a considerable amount of faster RAM (commonly termed computer memory) is utilized.

Additionally, the models created by AI companies operate at a grand scale. This necessitates computers capable of managing hundreds, thousands, or even millions of iterations of these models to cater to numerous users simultaneously.

The growing need for handling compute-intensive activities, scaling to accommodate a large user base, and minimizing limitations on expansion through virtually limitless investments results in an unquenchable thirst for hardware. Competing with firms that produce millions of laptops annually is increasingly challenging.

Why can’t chip manufacturers increase output?

It’s more complex than it appears. Semiconductor factories face production capacity limits, and establishing a new facility demands substantial investment and often spans several years.

Additionally, there are indications that manufacturers may not wish for the current scarcity to subside. Reports from Korean media suggest that Samsung Electronics and SK Hynix dominate chip production, collectively accounting for roughly 70 percent. Averse to augmenting supply, they risk having new chip factories remain underutilized during a downturn in the AI sector.

With current demand flourishing, Samsung is in a position to: raise prices as much as 60%. Why would they disrupt this momentum? For instance, a 32-gigabyte chip sold by Samsung for $149 in September is priced at $239 by November.

Have shortages like this been experienced before?

Indeed. The surge in AI has compelled firms to aggressively accumulate graphics processing unit (GPU) chips to construct extensive data centers for training and running increasingly larger models. This persistent demand has driven Nvidia’s stock price up from $13 at the beginning of 2021 to over $200 recently.

The year 2021 also witnessed widespread chip shortages across the board, triggered by a combination of the global pandemic, trade disputes, natural disasters, and extreme weather events. This disruption impacted the production of items ranging from pickup trucks to microwave ovens.

That same year experienced storage shortages as a new cryptocurrency known as Chia, which depends on storage space rather than raw computing power, gained rapid popularity.

In summary, technological advancements are outpacing developments in global supply chains.

When could this shortage end?

Not in the immediate future. OpenAI has entered into contracts with Samsung and SK Hynix that will likely dictate delivery timelines, possibly consuming 40% of global memory supply. However, this represents just one AI entity; Microsoft, Google, ByteDance, and others are similarly seeking to acquire as many chips as possible.

The resolution of this shortage may hinge on whether the anticipated AI downturn, frequently mentioned by economists and industry leaders, actually materializes, potentially leading to a surplus. However, this scenario poses risks of severe financial repercussions.

Should such a downturn not occur, projections suggest it may not settle until 2028, when new factories from smaller firms begin to contribute, allowing supply and demand to reach some semblance of balance.

Some experts indicate that this prolonged shortage could become a broader manufacturing challenge. Sanchit Vir Gogia, an industry analyst at Greyhound Research, noted to Reuters, “Memory shortages have evolved from a component-level issue to a macroeconomic concern.”