The Need for Operational Definitions of Metrics

While coming up with operational metrics one must not forget to begin by first clarifying and defining what they mean. This is because there is a good chance that there may be ambiguity regarding what the metric really means. To better understand how to come up with operational definitions, the following procedure must be followed.

Same Vocabulary: The same vocabulary must be followed all across the organization. This ensures consistency in what is being communicated by people from different facilities or locations.

Consider an automobile dealer with 4 branches. Three branches state that they deliver the car 2 days after the customer placing the order while one branch states that it takes 3 days to deliver the car. One may be led to believe that this dealership has a broken process. On closer inspection it is possible that a difference in vocabulary is the real cause.

It is possible that the three dealership call the day the customer placed the order as day 0 while the fourth one called it as day one. Therefore the time taken by both of them to deliver the car was same i.e. 72 hours. It is just that the vocabulary used was different. Such inconsistency can mislead the management and make them take wrong decisions. Hence while defining the metrics the relevant vocabulary must be defined as well.

How Data Is Collected ? There is also a possibility that there could be discrepancy in how different dealerships collect data. Let’s assume that one used random sampling to collect data while the other used stratified sampling. The results are obviously not comparable because the methods used are not the same. Hence data collection methods must be specified with no ambiguity.

How Calculations are Done ? Different departments are also likely to do calculations in different ways. This is also likely to create discrepancy in the value of the metrics being used. One department may round off the decimals while others may not and in some case it may produce significant difference in the values. To prevent this standard ways of calculating must be prescribed. It possible the data collection and calculation must be automated.

Develop the Range: Metrics do not always fall in the same line. They have approximately the same value, however not exactly the same value. Hence the range within which they normally fall in must be designed. This is usually done by creating control charts. If the metrics lie within the given range, then the process if working fine, if it is not then someone senior is alerted.

Classify what Happens if Variables go Beyond Range: Even when variable go out of range there are classifications. If the variable crosses the upper limit, it means one thing and if it crosses the lower limit, it means another. Also the magnitude by which it crosses the upper and lower limits can lead to the variables being classified separately.

Metrics will only be effective if the management has complete control over how they are created. The numbers should be comparable and mean exactly what they are supposed to mean.