Why we can't measure IT efficiency

Going beyond PUE
To expand the scope of data centre metrics beyond PUE and capture more of the data centre activities the generally accepted direction of progress has been to move up the data centre stack and try to capture the “IT work” or “productivity” delivered by the IT equipment or software platforms. A range of proposals have been made of greatly varying complexity, measurement overhead and validity.

Despite much effort there has been no significant progress in finding portable, measurable and effective metrics for anything above PUE. This is because of a series of underlying issues with attempting to measure the output of the IT equipment or application software which have not been addressed. Until these issues are solved IT work and data centre productivity metrics will continue to be hard to measure, harder to explain and not useful for comparison between data centres.

A number of metrics have been proposed which attempt to get around this problem by generating an arbitrary “efficiency” benchmark for the IT equipment and then measuring how much of the “output” of the benchmark is used. Unfortunately this simply moves the problem from one place to another and does nothing to address the underlying problems of accuracy, relevance and comparability.

Until a general solution to this underlying problem is found any further attempts at IT efficiency or data centre productivity metrics will be as subjective and user dependent as agreeing on the single set of ingredients and assembly method for the “perfect” sandwich.

Measuring work
To develop IT work or productivity metrics it is necessary to measure the work or output delivered by the IT equipment or software. Unfortunately all existing methods for this are quite subjective and two different operators are unlikely to agree on precisely how to measure it. Worse, two different platforms belonging to the same operator and hosted in the same data centre may not be measurable in the same way.

As an example of this problem let’s drill down through the IT equipment to what should be a simple measurement, a single hard disk drive. We will try to measure the "work" of this hard disk drive in three different applications.

In the first case the hard disk is being used as a backup device for user data. In this application the disk is inactive and hopefully powered off for most of the day, only working for a short period where the backup data is transferred. In this case our work measurements are the total amount of data we can store on the disk and on how reliably that data is stored, measured by the MTTDL  of the disk or disk system.

As a second case we might connect the disk to a streaming media server. In this application the primary copy of the content is held elsewhere and we are not concerned by the MTTDL. Whilst we are interested in the storage capacity the primary constraint to the media server will be the sustained data transfer rate of the disk as once this is reached the server cannot deliver any more streams. So for this application our work measurement is mostly SDTR with an extra, small weighting for capacity.

In a third case we could use the disk in a database server. Here the reliability of the disk becomes important to us again but our primary concern becomes not how quickly the disk can transfer long contiguous sex cam live sex cam free sex cam chat sections of data but how quickly it can read and write small sections of data all over the disk surface in response to database reads and writes. In this application it is common to only use part of the disk surface as the major constraint is the seek time and more disks are added to obtain IOPS  performance well before capacity is reached. In this case our work measurement is mostly IOPS with some weighting for reliability.

Using measured work to determine efficiency
The underlying issue is that given the multiple dimensions of “work” or “productivity” from our IT devices we must somehow compact our multiple measurements of each work dimension into a single value to compare with the consumed energy (kWh).

Measuring the perfect sandwich
Unfortunately, as the example of a single hard disk shows, to measure IT efficiency by comparing the measured work with the consumed energy we have to generate a series of fudge factors which we apply to each dimension of measurement in order to combine them. The problem is that different applications of IT devices and different operators place different values on each of these dimensions and there is no universally “correct” set of fudge factors any more than there is a universally agreed “perfect” way of making a sandwich.

For a more detailed coverage of this and other issues with data centre metrics please see the white paper Future of data centre metrics.

(1) MTTDL - Mean Time to Data Loss

(2) IOPS - Input output Operations per Second