The Metrics Meta-Game

As the stewards of good data practice within a company, we data professionals are often asked to help set metrics, [and] there’s a whole meta-game around the process that we’re playing(...)

This topic—designing metrics that are less susceptible to "gaming"—is one that I've actually never seen covered before. And it's a very good one: often, it's actually the most important issue in setting metrics that will be used to assess an organization's (and thereby multiple individuals') performance. Why? Goodhart's Law, of course:

“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”

This is an excellent read. Highly recommended.


Want to receive more content like this in your inbox?