(Due to the liability culture of the United States, neither the company nor individual is named in the below story. In addition, because the below story occurred years ago, the company may not exist anymore. In addition, the individual may no longer be public about anything. I heard this story at a technical event and confirmed the story at the time, but have no interest in following the company or individual outside of the one time event of confirming the story.)

Story: Manipulating Measures

A few years back, coders became obsessed with a popular coding website to learn code. The company featured a daily behavioral streak to encourage consistent learning. Coders bragged about their behavioral streaks and did everything they could to keep their streak alive.

One coder had a baby during the coder's streak. The coder could not code for several days because the coder needed to put 100% of their attention toward their newborn. The coder reached out to the company, upset that their streak ended, and asked for the company to edit their streak even though it had been disrupted. The company posted at the time that they had edited the streak of this coder to keep it consistent, even though this did not reflect the coder's actual activity.

Misunderstanding the Measure

I love this short story because it highlights people who don't understand what a measure means. In fact, neither the company nor the individual in the story understand a behavioral streak. They use a tool without putting the tool into its proper context.

A measurement serves as an audit. If a company or individual "hack" or "change" a measure to the opposite of what its measuring, then the measure ceases to have value. A behavioral streak reflects consistent behavior. If a person's behavior was not consistent, then the behavioral streak was broken.

Imagine a construction worker using a tape measure to ensure that a ceiling is 12 feet high. A tape measure is a measurement tool that is not impacted by our thoughts or feelings (independent). The customer who will buy the house wants a minimum 12 foot high ceiling because they happen to be an extremely tall person. But the constructor worker identifies a problem - they built a ceiling that is only 9 feet high. Recognizing the problem, the construction worker creates his own tape measure that counts 1, 2, 3, 4, 5, 6, 7, 8, 12 feet. Now the ceiling is 12 feet high, aligning with what the customer wants.

Every person reading this with basic mathematical understanding may laugh at this absurd analogy. Yet this is precisely what the coding company in the story did and what the coder wanted.

If a person stops coding for two days because they had baby, then the behavioral streak ended.

In Poor Charlie's Almanack by, Charlie Munger discusses how measures can help in some cases. But Charlie cautions that measures can cause people to focus on the measure and miss the meaning of the measure. We observe this when people try to manipulate a measure, which makes the measure meaningless.

I share this caution because we all feel tempted to manipulate measures, especially if doing so saves us time and energy. I can help clients with research. I can also help clients get exceptional results over time from the research. But I cannot prevent clients from trying to hack or manipulate measures to look good, when the results say the opposite.

If you simply want to tell yourself that you're succeeding when you're not, then no product or service can help you.

In Summary

A measure must always be independent of you, what you think, and how you feel.

If a measure shows that you failed, then you failed. Own it and improve. Likewise, if a measure shows a disruption because you experienced the most exciting event that a person can have, then it means that you just experienced the most exciting event that a person can have. Celebrate that moment.

On a business level, the business advice of "what's measured improves" is only true if the measure remains independent of the business and the people involved.

Research: Measures