Scaling Airbnb’s Experimentation Platform

I love so much of the writing and thinking that comes out of the Airbnb data team and this post does not disappoint. This post is trivially about technology, but the real story here is about organizational behavior.

Airbnb has around 500 experiments running at any given point in time and they monitor 50k metrics for those experiments. They invest heavily in the infrastructure and tooling required to deliver this capability organization-wide, and they have tight processes around the workflow. One example:

The Data Engineering team certifies around 50% of metrics. These metrics have an SLA guarantee and changes to their definitions are closely audited.

That is just really cool. I frequently get asked about books on data organizational behavior and, honestly, I don't love any of the ones I've seen. I hope the folks on the data team at Airbnb write one.


Want to receive more content like this in your inbox?