While the world continues to watch AI and robotics technology unfold at breakneck speed, data centers, which play a central role in the development and adoption of AI tech as a whole, are receiving increased scrutiny both in terms of power requirements and how data centers should operate as a whole when they are built at scale.
NVIDIA Networking SVP Kevin Deierling said the development of AI and computing underwent three major parallel transformations, resulting in an explosion in the number of tokens generated, the watt-per-token price, and the processing power required to hadle them.
In response, the company presented a reimagining of the data center into what it called the AI Factory, which, through a concerted effort to optimize models and upgrade its architectures and chips, would help drive down tokens per watt and tokens per dollar, according to Deierling.
CTO of Embedded and Critical Power at Flex, Paul Deamer, highlights the industry’s need to embrace how data centers are built to sustain deployment at a pace that keeps up with tech development, noting that data centers’ foremost challenge is energy availability.
The one-megawatt rack, soon to be the industry’s new norm, would bring about changes, such as rack designs needing to be redesigned to accommodate greater cooling and increased rack weights, which would affect floor loading and the overall facility layout, Deamer said.
Deamer also pointed out the reduction in time expected to establish data centers, noting that deployment time for AI data centers, which used to be measured in years, has now contracted to mere months.
Ultimately, the industry needs to move away from designing single components and start designing them as integrated systems directly at the design stage, rather than integrating them later, Deamer said.
Super Micro Chief Business Officer Vik Malyala said the company has noticed the shift in scale in terms of data center building, and that the scale and its price point hurt users, adding, “AI needs to be cheaper. It needs to be democratized.”
Malyala went on to note that a data center takes one to two years to complete if all goes according to plan, and that between the initiation of construction and its completion, the tech the center is supposed to house may have already evolved in a way that makes the center outdated.
The company’s Building Blocks solution took into consideration all the complexity and interconnectedness of systems housed in data centers, from the number of sockets required by CPUs of various manufacturers, to how the machines are cooled, and how they are installed, and even battery packs, and packages them into suitable solutions, whether one needs them by the unit, or a row, or in racks.
Schneider Electric explored how to maximize power utilization and distribution in data centers so they can be built at scale and operate efficiently, issuing a white paper that investigates the viability of the “80-degree direct current (DC)” problem and ensures that future AI data centers can still operate efficiently at scale, and proposed the centralized power distribution model as a solution to maximizie energy efficiency at data cetners.
