Using Deep Learning at Scale in Twitter’s Timelines

Building and deploying timeline algorithms has to be one of the most challenging tasks in all of modern software engineering: complex recommender algorithms deployed at the very large scale with high criticality.

About a month ago, Linkedin published a behind-the-scenes look at the engineering of their feed. This post from Twitter is similar, and it's even more fascinating: while Linkedin relies on a human-in-the-loop strategy and has a much more defined idea of what "quality" content looks like, Twitter's problem is honestly a harder one to solve. Its scale is bigger, its content variety is wider, and timeliness is even more important. This post goes surprisingly deep on how the Twitter engineering team thought about the problem and what they actually did to to move to their new, deep-learning-based timeline. 

Highly recommended. 


Want to receive more content like this in your inbox?