Building a Data Pipeline with Airflow

Holy crap. I've linked to Mark's stuff before, and this article doesn't disappoint. In it, he walks through the complete process of setting up Airflow (now an Apache project) using a simple example of grabbing foreign exchange rates from an API, storing them in Postgres, and then caching them in Redis. It's not simple to get Airflow up and running, but this article gives you everything you need.

Linkedin Revue

Want to receive more content like this in your inbox?