Functional Data Engineering — A Modern Paradigm for Batch Data Processing

medium.com

I must apologize: this article was originally posted back in January and somehow I missed it. I came across it this past week and think it's the single best post I've ever read on the design of batch data processing pipelines. Of course, it's written by the original author of Airflow, so he knows a thing or two about the topic.

I have so much to say about this post, but Maxime says all of it better. Better to just read the post yourself.

Read more...
Linkedin

Want to receive more content like this in your inbox?