Using Machine Learning to Explore Neural Network Architecture

Designing neural networks is hard: it takes lots of time from lots of highly trained researchers. Which, of course, is why Google is trying to automate it:

In our approach (which we call "AutoML"), a controller neural net can propose a “child” model architecture, which can then be trained and evaluated for quality on a particular task. That feedback is then used to inform the controller how to improve its proposals for the next round. We repeat this process thousands of times — generating new architectures, testing them, and giving that feedback to the controller to learn from.

This blog post is short and sweet and very accessible. My favorite part is the diagrams of the human and machine-designed networks: it's impressive how similar the topologies are, but also interesting the small and non-obvious ways that the machine-designed networks improve over the human ones.

Also a good read on this topic: this week Airbnb had a good writeup of their usage of automated machine learning. Airbnb seems to think of the technique as more of a workflow enhancement, saving data scientists time, rather than an area of basic research allowing them to push forward the boundary of algorithm development. Their approach is probably more day-to-day useful (if less groundbreaking). 


Want to receive more content like this in your inbox?