Power consumption of supercomputers

For the first time, the TOP500 list is also providing power consumption values for many of the computing systems and it will continue tracking them in consistent manner. As “name-plate” power ratings can be several times higher than actual consumed power levels, we decided not to report name-plate or peak-power ratings at all and to report measured values only.

Measurements:

For consistency, we asked system manufacturers and owners to measure power consumption while running the Linpack benchmark. Either the complete system or part of the system could be measured. If only part of a system was measured, it had to include all essential hardware such as shared fans, power supplies in enclosures or racks. Components which depend heavily on the machine-room environment such as non-essential disks, water-cooling jackets around air-cooled racks, UPS systems, and similar parts should be excluded from measurements. Their power consumption is a reflection of the environment a computer system is used in and not of the computer system itself. Measurements reported took place on nodes, blade-enclosures, system racks, or full systems. These data were then scaled linearly to the full system.

Power Metrics:

Power efficiency is a popular metric used to compare different technologies. It can be used for this purpose as long as systems of similar size are compared. Power efficiency, however, is not useful for ranking individual systems. Due to their basic nature, efficiencies or densities carry no information about the “size” of an object and therefore cannot be used to rank system by size as done in the TOP500. To further increase the potential for misinterpretation, the ratio of Linpack performance over power consumption will always rank smaller system of a certain type higher than larger systems of the same type, giving the false and misleading impression that smaller systems are more useful for supercomputing than larger systems.

We therefore decided to list (at this point) only absolute power consumption of systems in the TOP500 itself. We are currently considering alternative approaches for ranking, which will include multiple system features such as performance, power consumption, and memory size.

Results:

General Power Levels:

First results about the general power consumption values reported include:

• Average Power consumption of a TOP10 system is 1.32 Mwatt and average power efficiency is 248 Mflop/s/Watt.
• Average Power consumption of a TOP50 system is 908 Kwatt and average power efficiency is 193 Mflop/s/Watt.
• Average Power consumption of a TOP500 system is 257 kwatt and average power efficiency is 122 Mflop/s/Watt.

One possible explanation for the decreasing efficiency with rank is that only newer systems and technologies with better efficiencies can be found toward the top of the list. Toward the end of the list a mixture of newer and older technologies lowers the average efficiency level.

Power Efficiencies of Technologies:

Power efficiency values of different systems in the TOP500 are influenced by a variety of factors such as power consumption, Linpack efficiency, parallel scaling behavior, and size of system measured. With these restrictions in mind, we can analyze the collected data and find in general that:

• Most energy efficient supercomputers are based on:
  • IBM QS22 Cell processor blades up to 488 Mflop/s/Watt,
  • IBM BlueGene/P systems up to 371 Mflop/s/Watt
• Intel Harpertown quad-core blades are catching up fast:
  • IBM BladeCenter HS21with low-power processors (L5420) up to 265 Mflop/s/Watt
  • SGI Altix ICE 8200EX Xeon nodes (E5472) with high efficient Linpack up to 240 Mflop/s/Watt
  • Hewlett-Packard Cluster Platform 3000 BL2x220 with double density blades up to 227 Mflop/s/Watt
• These systems are already ahead of BlueGene/L (up to 210 Mflop/s/Watt).