Amazon develops dedicated cooling equipment to tackle the high energy consumption challenges of GPUs in the AI era
Amazon's cloud computing division has developed specialized hardware for cooling the next generation of NVIDIA GPUs to address the high energy consumption challenges of GPUs in the AI era. The new device, "row heat exchanger," can be inserted into existing and newly built data centers to address the shortcomings of traditional cooling methods. Customers can use this service through AWS's P6e computing instances, in conjunction with NVIDIA's high-density computing hardware, to support the training and operation of large AI models
According to Zhitong Finance APP, Amazon (AMZN.US) announced on Wednesday that its cloud computing division has developed specialized hardware for cooling the next-generation NVIDIA (NVDA.US) graphics processing units (GPUs) — these GPUs are widely used for AI-related computing tasks. NVIDIA's GPUs provide powerful momentum for the explosion of generative AI, but they consume a tremendous amount of energy. This means that companies using these processors must be equipped with additional devices for cooling.
Amazon had considered building data centers capable of widely deploying liquid cooling systems to fully leverage the performance of these high-power NVIDIA GPUs. However, Dave Brown, Vice President of Computing and Machine Learning Services at Amazon Web Services (AWS), stated that the process takes too long, and the available equipment on the market does not meet the demand. Dave Brown said, "They either take up too much floor space in the data center or significantly increase water usage. While some of these solutions may work in small-scale scenarios for other service providers, they simply do not have enough liquid cooling capacity to support our scale."
As a result, Amazon engineers conceived and developed the In-Row Heat Exchanger (IRHX), a device that can be inserted into existing and newly built data centers. The previous generation of NVIDIA chips was adequately cooled using traditional air cooling methods.
Dave Brown stated that customers can now use this AWS service through a computing instance called P6e. These new systems work in conjunction with NVIDIA-designed high-density computing hardware. NVIDIA's GB200 NVL72 installs 72 NVIDIA Blackwell GPUs in a single rack and works collaboratively through interconnection to train and run large AI models.
Amazon has previously launched various self-developed infrastructure hardware. The company has developed custom chips for general computing and AI, and designed its own storage servers and network routers. By running its own hardware, Amazon reduces its reliance on third-party suppliers, which helps improve profitability. In the first quarter, AWS delivered its best operating profit margin since 2014, and this business unit also contributed the majority of Amazon's net profit
