Python vs SQL: Comparison for Data Pipelines

towardsdatascience.com

I now work under the paradigm of “Do not move data to code, move code to your data”. Python moves your data to the code while SQL acts on it in place.

Yep. This is the reason to express ETL workloads in SQL, and it's one that many folks still don't understand. The author saw a 14x performance improvement moving some simple data transformations from Pandas to Postgres... that number would've been much bigger on a real analytical database. Use Python when it's actually needed.

Read more...
Linkedin

Want to receive more content like this in your inbox?