Search Framework With High Performance, Availability, and Scalability

A few years ago Pinterest released their own Search Framework (Manas) in order to support their growth.

As they use Manas for the majority of their products, they made it a high priority matter to improve its indexing latency.

Manas already supports incremental indexing, which is able to provide indexing latency within the order of tens of minutes. 

Michael Mi wrote a super insightful story about the approach they took to solve the problem. It contains tons of details on the architecture, failover and auto-scaling, error recovery, etc.