JAXA's Supercomputer Boasts 'World's Highest Execution Efficiency'

Apr 6, 2009
Tomohisa Takei, Nikkei Electronics
The main system of the new supercomputer revealed by JAXA. The groups of computers are the parts surrounded by the dark gray walls. They are set up in 2 x 3 rows.
The main system of the new supercomputer revealed by JAXA. The groups of computers are the parts surrounded by the dark gray walls. They are set up in 2 x 3 rows.
[Click to enlarge image]
The "FX1" servers are stacked on the shelves on both sides of the pathway.
The "FX1" servers are stacked on the shelves on both sides of the pathway.
[Click to enlarge image]
The new supercomputer was installed at JAXA's Chofu Aerospace Center. This is a model of the building where the supercomputer is located.
The new supercomputer was installed at JAXA's Chofu Aerospace Center. This is a model of the building where the supercomputer is located.
[Click to enlarge image]
Fujitsu's scientific computation server, FX1
Fujitsu's scientific computation server, FX1
[Click to enlarge image]
The supercomputer features Fujitsu's microprocessor "SPARC64VII," which integrates four SPARC cores.
The supercomputer features Fujitsu's microprocessor "SPARC64VII," which integrates four SPARC cores.
[Click to enlarge image]
The wiring of the FX1, which forms the M system. The black cables are the InfiniBand cables used to connect nodes, the red cables are for node management and the green cables are used to control the FX1, which consists of four nodes.
The wiring of the FX1, which forms the M system. The black cables are the InfiniBand cables used to connect nodes, the red cables are for node management and the green cables are used to control the FX1, which consists of four nodes.
[Click to enlarge image]
NEC's "SX-9" incorporated for executing applications optimized for vector calculators
NEC's "SX-9" incorporated for executing applications optimized for vector calculators
[Click to enlarge image]

The Japan Aerospace Exploration Agency (JAXA) disclosed its new supercomputer, which started full-scale operation April 1, 2009.

The new supercomputer's main system, the "M system," uses 3,008 microprocessors, each of which integrates four CPU cores, and achieved 110.6TFLOPS based on a Linpack benchmark. This is equivalent to the 17th computing power in the world, according to the latest list of "Top500" global supercomputer rankings (announced in the fall of 2008).

In consideration of the theoretical peak performance (120TFLOPS) of the supercomputer, its execution efficiency is 91.19%, the world's highest efficiency, outperforming the previous top efficiency of 90.78% in the latest TOP500 list, said Kozo Fujii, director of JAXA's Engineering Digital Innovation Center. Fujitsu Ltd supplied the system.

The new supercomputer consists of five systems. And the M system, which achieved 110.6TFLOPS, uses Fujitsu's "FX1" scientific computation server. The FX1 houses four sets of computing nodes featuring the quad-core 2.5GHz "SPARC64VII" processor in a 5U rack case. The computing nodes are connected via DDR InfiniBand. The total capacity of the M system's main memory is 94 Tbytes.

In addition to the M system, the supercomputer includes the "P system," which is used for specific projects, the "V system," which uses applications optimized for vector computers, the "A system," a compute server for general applications and a "storage system."

Like the M system, the P system uses the FX1. But its number of nodes was reduced to 384. Its theoretical peak performance is 15TFLOPS, and the total capacity of its main memory is 6 Tbytes.

The V system employed the "SX-9," NEC Corp's vector type supercomputer equipped with 48 microprocessors. The storage system's disc and tape units have a capacity of 1 and 10 Pbytes, respectively.

Systems suited for JAXA's purposes

Commenting on JAXA's new supercomputer, Fujii said, "We focused on choosing systems suited for our purposes rather than pursuing superficial levels of performance." JAXA evaluated the systems based on execution speeds when fluid simulation and other software that it actually uses are being run, as well as the system's cost and reliability.

"It's not that we desired this composition," said Yuichi Matsuo, leader of the Research Area at JAXA's Engineering Digital Innovation Center. "The computer consists of these systems as a result of comparisons based on the actual software."

"We achieved the 91.19% execution efficiency through our efforts to build up hardware and software aimed at effectively using the computing units," said Aiichiro Inoue, president of the Next Generation Technical Computing Unit at Fujitsu.

For example, Fujitsu included a hardware mechanism to synchronize threads and a function to reduce the competition in the 6-Mbyte secondary cache shared by four cores in its SPARC64VII microprocessor. The balance of transmission speed between nodes and the microprocessor's computing speed, as well as the memory bandwidth, is important for preventing calculations from being interrupted.

With the FX1, Fujitsu effectively made the most of each microprocessor's computing performance by mounting two memory controllers to secure a bandwidth of up to 40 Gbytes per second, and by employing the InfiniBand interface for connecting nodes, it said.