Are you teaching to the test?

Your metric needs an objective.

Or you will manage to the metric without knowing why.

This is a fundamental part of the work I do with data teams. We don’t just define success metrics - we define the purpose that metric holds.

What started as “Stakeholder Satisfaction” morphs into optimize the stakeholder survey to get higher numbers, which morphs into only sending the survey to people most likely to respond positively, which morphs into your team performing strictly to improve the survey results (a la “teach to the test”).

Without a metric objective, you can’t identify what negative gaming behaviors are.

Without a metric objective, you can’t tell if you’ve drifted from the original purpose.

Without a metric objective, your team will optimize a number rather than a result.

And you forget the point of the metric. At one moment in time, you believed your data team existed to serve the business teams and make them tremendously happy with data.

Your survey may or may not be serving that purpose anymore.

Jeff Bezos, when reflecting on an customer metric important customer metric at Amazon said this: “I don’t really care about this metric. I care about customer happiness. That metric is only useful to me so long as it reflects how happy customers are”

I’m here,

Sawyer