Measure what matters

Measure What Matters (Part II)

August 11, 2013

• Views19,819 Views
• Like207
• Comments60

inShare1,690

The only way to look into the future is use theories since conclusive data is only available about the past."

— Clayton Christensen, et al., Seeing What’s Next

When the late Milton Friedman taught his graduate courses in economics at the University of Chicago, he used to ask what his students came to call his two terrifying questions:

• How do you know?
• So what?

In terms of constructing a theory, these questions are profound; the first making us observe the world, the second forcing us to say what the effect is. To answer the latter, we need theory—a statement of cause and effect.

Philosophers of science, such as Karl Popper and Thomas Kuhn, have explained how to build a theory, which is done in a cyclical pattern as follows:

• Observation
• Categorization
• Prediction
• Confirmation

This is why all scientific theories are formulated in a manner exposing them to being disproved. Scientists always ask, “What would it take to admit your theory is wrong?” If you answer nothing, you are arguing an assertion—a matter of faith—not reason. If you cannot be wrong, you cannot be right either.

This is why scientists are constantly looking for anomalies. It is an iterative process, a never-ending quest of improvement, since only a better theory can replace an inferior theory.

Nassim Taleb, author of The Black Swan, describes this as:

Substractive epistemology: the sucker thinks Truth is search for knowledge; the nonsucker knows Truth is search for ignorance.”

This is what the Scottish philosopher David Hume meant when he wrote “Knowledge is only ignorance postponed.”

Pantometry vs. Theory

Unlike management accountants and auditors, who tend to focus on lagging indicators—such as a business’s financial statements—economists developed not only lagging indicators, but also leading and coincident indicators.

• Leading indicators anticipate the direction in which the economy is headed.
• Coincident indicators provide information about the current status of the economy.
• Lagging indicators change months after a downturn or upturn in the economy.

This is not to claim economists can predict the future; far from it. There is a tremendous amount of history that supports the observation that no one can predict the future. That said, the indicators have no doubt expanded our knowledge of how an economy operates, and may even provide a clue as to where it is heading, but they are still a compendium of averages, and averages can be very misleading—on average, everyone in the world has one testicle. Nonetheless, the indicators can be useful.

Your individual FICO—Fair Isaac Corporation—score is comprised of five weighted components that have been tested to have predictive capabilities about your future credit worthiness:

• Payment history
• Amounts owed
• Length of credit history
• New credit
• Types of credit used

Notice the FICO score is not simply a measurement. It certainly contains measurements, but it is driven by a theory of which characteristics can predict future behavior. This is the essential difference between measuring for the sake of measuring and a measure that is enlightened by a theory.

The social networking website Facebook discovered that the best predictor of whether or not members would contribute to the site was if they saw their friends contributing content, which is why most sites list updates on what their friends are doing online.

This is a critical distinction being made between a key performance indicator and a key predictive indicator.

A performance indicator is merely a measurement, such as the number of patents filed, or new revenue, but lacks a falsifiable theory.

A predictive indicator, by contrast, is a measurement supported by a theory, which can be tested and refined, in order to explain, prescribe, or predict behavior.

Utilizing Einstein’s method of a gedanken—thought experiment—is a good place to start.

A Gedanken

Engage in this thought experiment. You are the CEO of Continental (now United) Airlines: Which leading indicators would you want to look at on a daily—or even hourly or shorter—basis to determine whether or not Continental was fulfilling its mission of flying passengers around the world profitability?

It is relatively easy to develop lagging indicators, such as profit, revenue per passenger mile, cost per passenger mile, repeat customer bookings, frequent flyer miles earned, and so on. But they are lagging indicators, all of the employees would not be able to influence those results on a day-to-day basis. How would the baggage handler’s behavior change as a result of learning last month’s load factor? We need some canaries in the airline that are leading indicators of performance.

You could certainly develop coincident indicators, by tracking in real time all of the lagging indicators mentioned; and no doubt the airlines do this internally to some extent. But that still does not necessarily help the pilots, flight crews, baggage handlers, or food service caterers fulfill the goals and objectives of the airline on an hour-by-hour timeline.

What are needed are leading indicators—theories—that have some predictive power; in other words, they predict the financial results of Continental by predicting future customer behavior.

In his book, From Worst to First, Gordon Bethune details how he was able to turn around the failed airline (which had filed for Chapter 7 bankruptcy twice in the preceding decade) between February 1994 and 1997, turning it into one of the best and most profitable airlines in the sky.

It is a remarkable story, and it illustrates the importance of utilizing leading key predictive indicators (KPIs) to focus the entire organization on its purpose and mission. Bethune basically tracked three leading KPIs:

• On-time arrival
• Lost luggage
• Customer complaints

When Bethune became CEO, Continental ranked dead last in all of these indicators, which are also measured by the Department of Transportation (known as the “Triple Crown Criteria”). Bethune analyzed the problems—and there were many—and discovered the culture of the airline was focused on driving down cost per available seat mile (the standard measure of cost in the airline industry).

It cut costs at every opportunity, by packing the planes with more seats, reducing the food and drink portions, paying its people poorly, and so forth. It believed its mission was to cut costs; but as Bethune constantly pointed out,

We aren’t in business to save money, we are in business to put out a good product….You can make a pizza so cheap nobody wants to eat it. And you can make an airline so cheap nobody wants to fly it.”

There is an enormous difference between controlling costs and being cheap. A company must make investments that deliver value to the customer.

What makes the three indicators just listed leading is that they measure success the same way the customer does. And that is critical because, ultimately, the success of any business is a result of loyal customers who return.

None of the three indicators would ever show up on a financial statement, but, as the airlines have learned over the years—by testing the theory—they have a predictive correlation with profits.

Some organizations seem to believe the more they measure, the more will get done. If we get what we measure, isn’t it time to measure what really matters rather than merely measuring for the sake of measuring?

More importantly, if we get what we measure, isn’t it time to start measuring what we want to become?

Featured on:
• Your Career