The Gartner Group has implemented a new methodology for measuring IBM Corp and plug compatible mainframe performance and it is said to be based on four principles. The first is to establish performance of the base processor to estimate the mid-point of new processors; secondly, estimate the performance of the uniprocessor; thirdly, estimate MP ratios to establish a base for comparison with other systems; and lastly, establish that each additional engine will provide a lower performance than the preceding one. Gartner acknowledges that its figures are not based on measurements, but on calculations, and they allow 5% either side for error. As systems are measured in customer installations, the figures will be adjusted accordingly. The new methodology establishes Hitachi Ltd’s six-way mainframe at 214 MIPS and the five-way at 184 MIPS, but under the former system, the six-way would execute at 230 MIPS. The uniprocessor is more powerful than anticipated, estimated at 45 MIPS, and the MP factor of the EX620 is 1.75 times the EX310. Under the old system, the uniprocessor would have been rated at 45.5 MIPS with an MP factor for the EX620 of 1.74 times the EX310. Observers have been expecting IBM to reposition its top-end ES/9021 820, 860 and 900 machines and the figures in the table indicate that there have been changes. They are rated at 159 MIPS, 193 MIPS and 225 MIPS respectively, and under the old methodology, they were 178 MIPS, 211 MIPS and 263 MIPS.

Vendor Model CPUs MIPS Vendor Model CPUs MIPS

Amdahl 5995-8650 8 310 Hitachi EX520 5 184 Amdahl 5995-6650 6 248 IBM 9021-820 4 159 IBM 9021-900 6 225 Hitachi EX420 4 152 Hitachi EX620 6 214 Amdahl 5995-3550 3 143 IBM 9021-860 5 193 Hitachi EX310 3 122 Amdahl 5995-4550 4 184 IBM 3090-720 6 117

When IBM announced the machines, observers rated the 820 around 150 MIPS and the 900 at 214 MIPS. If IBM announces seven and eight way machines, they will be rated at 255 MIPS and 282 MIPS, while Hitachi’s would come in at 243 MIPS and 268 MIPS, both lower than Amdahl Corp’s 8650M, rated at 310 MIPS, 350 old MIPS. Gartner’s former methodology was to take a processor’s mid-point as announced by vendors and then multiply that by the XA MIPS established for a rated and well researched model. The next step was to apply an assumed ESA factor to the XA rating and arrive at the ESA MIPS rating. For example, a four-engine processor running MVS/ESA has an 11% uplift compared with the same processor running MVS/XA. This methodology provided accessible numbers that users and the press could use for comparing vendor claims, but as workloads and processor design has become more complicated, the estimates became less valid. The margin for error was acceptable when measuring two processor systems, but measurements for six and eight engine systems are flawed when the uniprocessor is rounded off in this way. Another complication is that vendors stopped announcing XA measurements and estimates. Also, different releases of MVS/ESA produce different results, and published results often seemed to be in conflict. The latest generation of mainframes saw Gartner’s MIPS ratings reach astronomic heights. Gartner applied its ESA uplift and mid-point methodology, but IBM, Hitachi and Amdahl had already included ESA effects in their claims, and in effect, Gartner doubled the effects of ESA.