What if you were an evil data scientist?


This is one of the most unusual posts I've come across in the 5+ years I've curated the Roundup. I'll let the author introduce it:

I work in data, and one thing that I think everyone in data knows but never really thinks to explain to non-data people is I could make the numbers say almost anything I want them to and the only thing keeping the world safe is my sense of ethics. (twitter)

The author goes through a bunch of different scenarios of how a self-interested data scientist could take fairly small but nefarious actions to forward their own career. My take: I 100% agree that there are things in this vein that are possible and that certainly happen every day.

But what I believe is the much-more-common situation is that motivated reasoning creeps its way into the analytical process. There's typically no moment in time when someone decides to do something to subvert the truth in their own best interests. Instead, there are a series of small decisions that must be made about how to account for reality in a given model, and it's not always super-clear what "correct" looks like. Assuming you're human, you'll have an incredibly hard time preventing your own personal interests from clouding your judgment, and the compounding of many seemingly-inconsequential decisions can end up being quite large. The effect is similar, and the tactics are identical, but evil intent is not required.

Why link this?

  1. If you're in a large org, it's not crazy to actually watch for self-serving behaviors like these.
  2. Even if it's just you, it's a great reminder of your own power and to seriously check your own motivated reasoning.


Want to receive more content like this in your inbox?