By: TOP500 Team
At ISC14, over 18 months ago, Intel began talking about their next-generation fabric technology, Intel® Omni-Path Architecture (Intel® OPA). Their new fabric for High Performance Computing (HPC) and low-latency scale-out systems is part of the Intel® Scalable Systems Framework (Intel® SSF), a new design approach for scalable, flexible HPC systems.
By February of 2016, Intel OPA enjoyed a commanding interest from the community and is making an impact on HPC systems being installed at the Pittsburgh Supercomputing Center (PSC) and the National Nuclear Security Administration’s three national labs—Lawrence Livermore, Sandia, and Los Alamos national laboratories. In addition to Intel OPA, CTS-1 integrates the latest generation of Intel® Xeon® processors. The Tri-Labs CTS procurement runs for three years.
According to Barry Davis, general manager of the HPC compute and networking group, Intel has seen an incredible market response that is greater than what they anticipated when they first started talking about the technology. “Intel OPA has far exceeded our initial expectations, on many fronts,” Davis said.
Leading Institutions Add Intel OPA to their Infrastructure
The Bridges cluster at PSC was delivered early this year. The system is unique, in that it is designed for converged workloads, combining traditional HPC simulation projects with visualization, machine learning, and high performance data analytics. The IT community has only recently been talking about the converged data center. Bridges is the first installation of a system with Intel technologies that covers the gamut of these computing workloads. “The democratization of HPC and convergence of disparate workloads is truly a phenomenon that IT is just beginning to experience,” added Davis. “We’re very excited that Bridges uses Intel OPA to support this next wave of data center infrastructure modernization.”
The first group of nodes in Bridges was up and running for the MIDAS MISSION Public Health Hackathon that took place on February 25 and 26 at PSC. Teams of developers ran their visualization workloads, which drew on large healthcare databases, on HPE server nodes with Intel OPA as the fabric. Shawn Brown, Director of Public Health Applications at PSC, felt it was “important for the organizers of the MIDAS MISSION Public Health Hackathon to make available Intel Omni-Path Architecture to the participants, so that they had the best technology on the market at their disposal to create top notch visualization applications.”
Jay DePasse, a Hackathon participant and member of Team vzt said, "The MIDAS MISSION Public Health Hackathon was a great experience, and having access to resources like Bridges and its extremely high-speed Intel Omni-Path network was very helpful in producing data driven visualizations."
Switch Technology that Makes a Difference in Performance
Unique to fabric technologies, the Intel OPA 48-port switch is enabling integrators to build systems with larger node counts interconnected at full switch bandwidth than can be done with the smaller switches of alternative fabric technologies, according to Sid Mair, Penguin Computing’s Senior VP of their Federal Systems Division. Penguin expects the fewer switches to lead to lower costs and complexity for building out new systems. “We have fewer cables, less parts, and a faster fabric. That becomes a very cost-effective, high-performance solution compared to the alternative,” said Mair.
Additionally, in collaboration with Intel, Penguin Computing’s work with Intel OPA on their Tundra Extreme Scale series platform has resulted in a 48-port Intel OPA switch designed for the high-density Open Compute Project (OCP) architecture and its 12-volt power rail. According to Mair, that enables a new level of density that can accommodate not only the compute nodes, but the switches as well. “Now you have incredible scalability in a much, much smaller space. We took the OCP architecture and essentially increased the density by more than 50 percent for HPC applications. We use less floor space, less power, less cooling—a 12-volt Intel OPA option creates a significant benefit for OCP-based clusters.”
Momentum into the Future
The latest announcement on Intel OPA installations is University of Colorado gearing up for a cluster with both Intel OPA and Intel® Xeon Phi™ processors code name Knights Landing. So, Intel OPA has had a strong beginning since its release late last year.
According to Davis, part of the success in driving Intel OPA efforts is due to the company’s strong engagement with their worldwide OEM channel of some 75 partners. “We’ve made tremendous progress, enabling them to launch significant efforts with Intel OPA in 2016. We even worked with some partners to build benchmarking clusters in order to help them along with their customers understand the value of the new fabric.” Davis added that through those clusters they continue to help establish and prove the fabric’s performance and price/performance value in HPC. “Those efforts are helping drive Intel OPA momentum for 2016.”
“We’re very excited about bringing this new architecture to the HPC marketplace to uniquely fulfill the demanding needs of supercomputing and the converged data center,” commented Davis. “And, we’re incredibly pleased at the momentum of Intel OPA market reaction, both with OEMs and end users.”
Learn more about Intel Omni-Path Architecture.