Uber built Manifold, a model-agnostic visualization tool for ML performance diagnosis and model debugging, to optimize our model iteration process.
Optimizing ML models is hard. It requires a data scientist to hold quite a lot in their brain at once—product designers would say it has high "cognitive load". High cognitive load tasks are not uncommon for technical fields that are in their nacency (as ML is), but as these fields mature it becomes important to reduce the cognitive load in order to broaden the potential user base.
This is starting to increasingly be a focus in ML. Google's heavy focus on AutoML is one approach, and providing better tooling to model builders to do their own tuning (what Uber is doing with Manifold) is another.
I'm personally very interested in the "make ML accessible" trend and think we're still in the mainframe phase—big, centralized, inaccessible (except to the high priests).Read more...