The Industrial Usage of HPC Systems takes off

1996 was again a year full of changes for the High Performance Computing (HPC) community. The shake out of vendors culminated in Silicon Graphics (SGI) buying Cray Research. A few month later died the father of ``Supercomputing'' Seymour Cray after a car accident. New systems like the T3E entered the market place quite impressive and there is a new number one on the TOP500. It is again a Japanese system, a special version of Hitachi's SR2201 massively parallel (MPP) system with 2048 processors. However the most important changes took place behind the title-pages of newspapers and was not easy to follow. During this year the industrial usage of HPC systems in general and of MPP systems specifically gained a lot of momentum. The U.S. is leading this trend very strongly with already 38% of all systems installed at commercial customers. Many of these system are used for non traditional applications in finance or for data mining. We will discuss in this paper the different developments based on the TOP500 lists of supercomputer sites available since June 1993 [1] and which, for the first time, provide a reliable base for a well-founded analysis of the high-performance computing field. Reports about the situation in previous years have been published before [2, 3, 4].

Last year we speculated at this time that we will see a new system at the top position of the TOP500 during this year [5]. We had indeed for both issues of TOP500 a new number one. In June 1996 it was a model of Hitachi's SR2201 system with 1024 processors installed at the Computing Center of the University of Tokyo. But this system of a new generation which is described in detail in one article of this special issue [6] kept this position only for one list. In November 1996 a customized version of the same architecture with 2048 processors is the new number one. Also an additional processor added to the Numerical Wind Tunnel (NWT) -- the former leader of the TOP500\ for 4 issues shifted it's performance ahead of the 1024 SR2201 system which is now number three. Due to the late arrival of the T3E and the big ASCI machines the TOP500 is leaded by three Japanese machines all installed in Japan as well.

But this cannot be taken as a sign of leadership in the field of HPC. Looking at all the 500 systems in the TOP500 we see a different picture with the USA as clear leader as producer and consumer of HPC systems. In 1996 it also became evident that the industrial usage of HPC systems was taking off in the USA. Europe and especially Japan are lagging quite behind in this very important aspect of HPC.

High end workstations with symmetrical multiprocessor design (SMP) which have been quite successful since they entered the market in 1994 are now starting to fall off the TOP500 list. This is due to their limited architectural scalability which limits the performance level they can achieve. SGI introduced as follow up its first distributed memory system ORIGIN 2000. At the same time Cray which now belongs to SGI started to deliver the T3E system. The T3E shows up in the list with already 23 systems and 746 accumulated GFlop/s.

Looking at the computing power of the individual machines present in the TOP500 and the evolution of the total market size, we plot the performance of the systems at positions 1, 100 and 500 in the list as well as the total accumulated performance. In Fig. 1 the curves of position 100 and 500 show on the average an increase of a factor of two within one year. The curves for position 1 and for the accumulated performance however show only a factor of 1.8 increase per year.

Figure 1: The performance over time as it can be seen in the TOP500.

Geographical Distribution

Looking at the TOP500 systems installed we see a quite stable distribution over time in Fig. 2. The upward trend of systems in the US still goes on while it changed in Japan and we see a small upward trend now. Even as Japan is behind in the number of installed SMP systems the new generation of Japanese parallel supercomputer is showing up in the TOP500 now.

Figure 2: The geographical distribution of the system counts over time.

Looking at the total of installed performance in Fig. 3, contrary to the number of systems seen in Fig. 2, Japan is again well ahead of Europe as it was in the last years. This reflects the fact that during the last years several very powerful systems of the latest Japanese supercomputer generations were installed in Japan. These Fujitsu VPP500, NEC SX4 and Hitachi SR2201 systems all make it to the TOP500. Taking a closer look at the strong increase of the installed performance in the US during the last year, we find that Cray Research installed 535 Gflop/s, IBM 336 Gflop/s, HP/Convex 54 Gflop/s and SGI gained only 2 Gflop/s. The share of all other vendors together went down by 6 Gflop/s. In Europe Cray took a big jump from 281 Gflop/s to 782 Gflop/s while in Japan Hitachi gained the most installed performance and is now second with 775 Gflop/s behind Fujitsu with 911 Gflop/s. In case of Hitachi however the first two of their systems already accumulate 589 Gflop/s.

Figure 3: The geographical distribution of the performance over time.

If we not only look at where the systems are installed but where they are manufactured, we see in Table 1 that almost all systems located in the US were built in the US. In Japan, too, the majority of systems come from Japanese manufacturers, but the share of US manufacturers is much higher than vice versa. The European vendors lost five systems compared to 1995 and are far from dominating their home market. Most of the systems installed in Europe are coming from US vendors. The share of Japanese systems is slightly lower compared to their world-wide average. Looking at the installed performance on Table 2, we get a similar picture with an even stronger dominance of the US and Japanese vendors on their home market.

Table 1: Geographical distribution where systems are installed and where they are manufactured.

Table 2: Geographical distribution of the accumulated performance showing where it is installed and where it is manufactured.

Market Shares of Vendors

The shake out of the HPC manufacturers culminated 1996 in SGI buying Cray Research. This merger created a strong new market leader in the HPC arena. Together they are dominating the market with a total share of 44% of the installed systems. However, this is only slightly more than Cray Research had on its own (41%) when we started the TOP500\ in June 1993. In Fig. 4 we see that Cray Research by itself has gained back the pole position from SGI with which it switch positions if we look at the situation in June 1996. Most of the raise of Cray is due to the 23 early T3E installations in the list.IBM is close second to Cray Research with 25% of systems installed. SGI/Cray and IBM hold together 2/3 of the market. The three Japanese companies Fujitsu, NEC and Hitachi have together 72 (14%) systems in the list. Looking at the changes in the accumulated performances of the different vendors in Fig. 5, we see that the installed performance of Cray made a big jump due to the T3E. The strong increase of the Japanese vendors and IBM is continuing.

Figure 4: The market share of the most important vendors over time.

Figure 5: The market share in performance of the most important vendors over time.

Architectural Changes

The big increase in the number of installed symmetrical multiprocessor workstations (SMP) in 1995 was the dominating effect with respect to computer architecture. In 1996 SMPs are already on their way out of the TOP500 again while the number of MPP systems is still raising. This reflects the product announcement of single companies like SGI. They introduced the Origin 2000 series (6 system on the list) which is an MPP system as follow up to their very successful SMP series PowerChallenge. The share of parallel vector processors (PVP) remained stable at a level slightly above 20%. MPP systems are the clearly dominating class of systems in the TOP500 with 2/3 of all systems belonging to this class.

Figure 6: The evolution of the architectures as it can be seen in the TOP500.

In our very first report [2] Japan was very much behind with the number of installed MPP systems in 1993. This began to change in 1994 [3]. The number of installed MPP systems in Japan is with 58% now only a little behind the world wide average of 64%. But like in the previous years almost no SMP systems have been installed in Japan.

Table 3: Average system size for the different classes of systems.

Looking at the average performance of a system in the different classes for the different regions we see in Table 3 that the MPP systems installed in Japan are quite powerful. Most of them are build in Japan and are based on system architectures with distributed memory and nodes with vector capabilities. This type of architecture is still not able to enter the US market, but is already entering the European market. The average system size in Japan is now measured in Gflop/s more than twice as high than in the US or in Europe. Compared to 1995 the European installations have substantially gained in average size when the average size was 5.8 Gflop/s compared to 13.1 Gflop/s now.

Technological Changes

Let us now try to analyze the technology used for the processors. With respect to the chip technology we find that the number of systems based on ECL chip technology is steadily decreasing from 332 in mid 1993 to now 79 by the end of 1996. During the same time the number of systems using proprietary processors with custom chips decreased from 59 to 35 in late 1995 and raised again to 60 by November 1996. This increase is due to the number of vector processors build with CMOS technology. It does not reflect any increasing use of proprietary CMOS-RISC processors. 342 of the systems in the current list are built by using `off-the-shelf' processors.

Figure 7: The usage of different node technologies as can be seen in the TOP500. We count for this figure the following systems as CMOS off-the-shelf: Convex SPP, IBM SP1/2, SGI.

In Fig. 7 we see that the number of systems with nodes binary-compatible to workstation systems is keeping its high share with now 50%. This class of systems includes the ones from Silicon Graphics, the Convex SPP and the IBM SP1 and SP2. This high market share of systems with such a node design shows the advantage of using standard workstation nodes keeping the design costs low. Also all available software for the workstations can immediately be used on the parallel systems, at least on a single processor. This seems to be a big advantage for selling systems to industrial users as can be seen in Table 4. 75% of all system installed at industrial customers are using systems build out of workstation compatible nodes. This includes systems with shared memory like the SGI PowerChallenge and systems with distributed memory like the IBM SP2.

Table 4: Number of systems with different node technologies for the different application areas.

Application Areas

Looking at the different application areas in Fig. 8 and Fig. 9 we see an increasing share for 1996 with finally 30% of installed systems and 14.8% of the installed performance after the decreasing share of industrial installations during the last years. If you look at the TOP500 in more detail you see that only IBM with 53%, SGI with 38% and HP/Convex with 32% have an over proportional share of industrial installations in their customer base. This is a very strong indication which advantage binary compatible nodes might have in the HPC market. IBM is leader in the industrial market place with 67 systems (45%) installed even ahead of the team SGI/Cray with 58 systems (39%). Convex has 7 industrial installations (5%) and all other vendors share 11% in the industrial market place.

Figure 8: The distribution of systems on the different application areas over time.

Figure 9: The distribution of performance on the different application areas over time.

In Table 5 we see the geographical distribution of the systems installed. It is evident that the USA are the clear leader in the industrial usage of HPC technology.

Table 5: Number of systems in USA/Canada, Europe, Japan for the different application areas.

Conclusions

From the present eight releases of the TOP500 we see:

• For positions in the range of 100--500 the performance of the individual systems is increasing by a factor of 2 every year while the total installed performance is increasing by a factor of 1.8 every year.
• The new number one for both releases of the TOP500 in 1996 have been Japanese systems and not the announced systems from US manufacturers.
• The US and Japanese vendors are dominating their home markets, while European manufacturers are playing no role at all even not in Europe.
• The shake out of the HPC manufacturers culminated in SGI buying Cray Research.
• SGI/Cray and IBM are leading the list with respect to the number of installed systems and with respect to installed performance.
• MPP systems are the dominating architecture, while the number of SMP systems started to go down in the TOP500.
• The number of ECL based systems is strongly decreasing all the time, and by the end of 1996 about 84% of the systems in the TOP500 were built with CMOS technology.
• In the TOP500 a strong trend to nodes being binary-compatible to major workstation families can be seen since 1995.
• Vendors using such "off-the-shelf" nodes (IBM, SGI and Convex) are in the position to sell an over proportional number of systems to industrial customers .
• IBM is leader in the industrial market place with 67 systems installed even ahead of the team SGI/Cray with 58 systems.
• The USA are the clear leader in the industrial usage of HPC technology.

With the TOP500 project going into its fifth year, many trends and evolutions of the HPC market could be made quite transparent. This has proven the TOP500 to be a valuable tool. Some of the trends mentioned can surely be stated and anticipated without the TOP500\ while many others are certainly surprising and could not be visualized without it.

References

1

J. J. Dongarra, H. W. Meuer and E. Strohmaier, TOP500 Supercomputer Sites. Technical Report 33, 34, 38, 40,, 41, 42, 47, 48 University of Mannheim, Germany, June 1993, November 1993, June 1994, November 1994, June 1995, November 1995, June 1996, November 1996

2

J. J. Dongarra, H. W. Meuer and E. Strohmaier, eds. TOP500 Report 1993, University of Mannheim, 1994

3

J. J. Dongarra, H. W. Meuer and E. Strohmaier, eds. TOP500 Report 1994, SUPERCOMPUTER 60/61, volume 11, number 2/3, June 1995

4

J. J. Dongarra, H. W. Meuer and E. Strohmaier, eds.
TOP500 Report 1995, SUPERCOMPUTER , volume 12, number 1, January 1996

5

H. W. Meuer and E. Strohmaier, New Building Blocks for HPCN in 1995, SUPERCOMPUTER , volume 12, number 1, pages 6-15, January 1996

6

E. Schnepf Developments in Japan, this issue

Hans W. Meuer
Computing Center, University of Mannheim, Germany
email: meuer@r.uni-mannheim.de
Erich Strohmaier
Computer Science Department, University of Tennessee, Knoxville, USA
email: erich@cs.utk.edu