Presto Infrastructure at Lyft - Lyft Engineering

eng.lyft.com

Early in 2017 we started exploring Presto for OLAP use cases and we realized the potential of this amazing query engine.

...and thus begins a love story for the ages ;) The Lyft team has truly made some huge investments in the overall platform and has a serious Presto infrastructure stood up internally (roughly ~ 1/3 the aggregate memory of Pinterest's infrastructure).

The thing I always wonder about companies using Presto, though, is their comparison set. Lyft migrated their query workloads from both Hive and Redshift and found Presto to be a better choice than either. My guess is that they didn't evaluate either Snowflake or Bigquery, though, given that both platforms were significantly less mature in 2017 when their original migration was in flight. I still haven't seen a heads-up comparison of Presto vs. either of these more modern analytic databases.

It's articles like this where the Presto team announces a 10x improvement in its UNNEST operation that actually make me believe that it's meaningfully behind; unnesting has been almost surprisingly fast on Snowflake for years.

Read more...
Linkedin

Want to receive more content like this in your inbox?