The QPACE supercomputer, which features Xilinx Virtex-5 LX110T FPGA devices, is used in the study of Quantum Chromodynamics. The main simulation process used to model Quantum Chromodynamics is known as Lattice QCD and is only possible using high-powered, highly parallel supercomputers. Xilinx Virtex-5 LX110T FPGAs were selected to provide core networking technology in QPACE.
Lattice QCD algorithms typically use relatively small messages, so network latency has a major impact on the efficiency. Custom network technologies, while delivering enough bandwidth, often introduce unacceptable latencies, in the region of 10 microseconds. Using FPGA technology the team achieved a cell-to-cell latency of just 3 microseconds, with a latency as low as 0.5 microseconds achieved for the optimized design of the 3-dimensional Torus network implemented in the Virtex-5 FPGA. The Torus network is able to sustain 2.5GHz without bit errors on hundreds of node cards, with single node cards successfully tested at 3GHz.
The QPACE team used Xilinx Virtex-5 LX110T FPGAs to implement the network processors (NWP), which interface between the processing elements and the interconnection networks, managing all network traffic for optimal performance. RocketIO transceivers on the Virtex-5 FPGA were also instrumental in implementing the Rambus FlexIO interface.
Using a custom-design approach has made QPACE one of the most power efficient supercomputers ever developed, with a peak performance in single/double precision of 26/56TFlops and an average power consumption of 29kW per rack.
The QPACE supercomputer uses node cards, comprising a PowerXCell 8 that is an enhanced version of the Cell Broadband Engine Architecture developed by Sony, Toshiba and IBM and first seen in the PlayStation3, integrated alongside 8 Synergistic Processing Elements and a Power Processing Element. The maximum number of node cards in a rack is 256 and a typical system comprises four racks, or 1024 node cards connected through three types of networks.
More info: Xilinx