Big Data Processing at Spotify

labs.spotify.com

This is an awesome history of data processing at Spotify, from batch jobs written in Python and Luigi to streaming jobs written in Google Cloud Dataflow and their own tool called Scio. They've done lots of thinking about stream processing and have some great commentary on it.

The Spotify team is executing at the frontier of what is currently achievable; this story isn't just about them, it's also a story of the evolution of an industry.

Read more...
Linkedin

Want to receive more content like this in your inbox?