Michael Kaminsky is blowing up the current notion of an A/B test:
Traditional A/B testing rests on a fundamentally flawed premise. Most of the time, version A will be better for some subgroups, and version B will be better for others. Choosing either A or B is inherently inferior to choosing a targeted mix of A and B.
This is a fairly basic statement—it seems obviously true upon immediate consideration. It turns out to have rather significant impacts, though, if taken seriously. That is what the rest of the post explores.