Celebrate a new benchmark

My Column in Financial Express today (27th June 2003) carries an abridged
version of my earlier post

Celebrate A New Performance Standard

On May 20, 2003, Hewlett-Packard posted a TPC-C benchmark of more than 700,000
transactions per minute, the second highest ever performance in transaction
processing. What is amazing about this is that the event also marks a
single-digit price performance ($9/tpmC) at this level of performance.
Transaction Processing Council, an independent organisation, has been
developing industry benchmarks by conducting measurements and publishing them
regularly, for more than a decade. These benchmarks attempt to characterise
different scenarios. The most important benchmark, the TPC-C benchmark,
characterises online transaction processing (OLTP). This benchmark includes a
peak performance measure, represented by transactions per minute of TPC-C
(tpmC), as well as a price performance measure represented by price/tpmC
(typically in dollars).

Currently, TPC-C measures the number of new order transactions per minute that
can be generated by a computer system while performing five concurrent
transactions of entering an order, monitoring stock levels, delivering an
order, recording payment, and checking the order status. The transaction
involves generic transactions of a firm with 100,000 catalog items, 3,000
customers spread over 10 sales districts, with an order response time of five
seconds or less. That is, a 50,000 tpmC represents an ability to generate
50,000 new orders. For computing costs, the cost of basic hardware, operating
system, database management system (DBMS) and TP monitor for a 24X7 operation
are included, and maintenance costs for three years.

Just as car racing indirectly influences automotive performance, TPC-C ratings
over the years have pushed the envelope of high-end transaction processing
through a combination of high-performance servers, operating systems and DBMS;
high-speed storage, and networking infrastructure. For example, in September
1992, when the TPC-C benchmark was first published, the top performer had 54
tpmC and a price performance of $188,562/tpmC. For a while, Sun Sparc 8-way
processor and HP 9000 H70 servers — with nearly 1,000 tpmC and
$1,000/tpmC — were considered the best. In 1998, the industry had moved
to a new peak of 52,871 tpmC and price performance of $135. What is exciting
about last month’s result is a peak performance of 707,112 and price
performance of $9.13/tpmC.

Interestingly, the results show some interesting characteristics: o Mainframes
are not dead. Yet, transaction processing is not for mainframes alone. Of the
total of 120 published benchmarks as of June 2003, there were 23 from IBM,
eight from Bull, seven from NEC, seven from Unisys, six from Fujitsu and six
from Fujitsu-Siemens, all mainframe manufacturers. There were 41 from HP, 19
from Dell, and one from Sun; there was even one each from Network Appliance
and Rack server, and none of them are mainframe guys!

* With HP bagging 41 and IBM bagging 23 of the 120 benchmarks, 50 per cent of
* the high-end OLTP market is held by the Top Two — a clear case of
* consolidation.

* Of the 120 benchmarks, Microsoft SQL Server DBMS product accounts for 86
* (Oracle and IBM DB2 getting 18 and three respectively). MS Windows accounts
* for 90 (IBM AIX, HP-UX, Sun Solaris, and Compaq True OS accounting for 12,
* five, four & four respectively). Within months of the launch, Windows Server
* 2003 has 38 out of 90 benchmarks, demonstrating the maturity of Microsoft
* products.

* The fastest price performance benchmark from HP is based on Itanium 2
* microprocessor-based system, thus disproving the myth that commodity
* processors are not ready for OLTP.

* The lonely Red Hat Linux-based benchmark with $17.32/tpmC is nearly twice as
* expensive as compared to the best-performing HP $9.13/tpmC (that too with a
* peak performance of 707,132 tpmC compared to Linux performance of 138,362).
* So, Linux does not always save money.

* Perhaps the most interesting feature is a sure win for commodity products:
* Intel Itanium, Microsoft Windows Server 200 and Microsoft SQL Server
* products. They were dubbed ‘not ready for prime time’. Surely
* and steadily, their time has come.

It may not be celebration time for some vendors. Vendors like HP, Intel &
Microsoft may celebrate, about which I am not too interested. What is truly
interesting is that the end customer has something real to celebrate about
— a whopping 700,000+ tpmC at $9/tpmC!

The author is Director of the Indian Institute of Information Technology,
Bangalore. Views expressed herein are personal. He can be reached at

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: