From Local Machine to Dask Cluster with Terraform

Learn how you can take local code that does grid search with the Scikit-Learn package to a cluster of AWS (EC2) nodes with Terraform.

Data scientists and the data science blogosphere have made a lot of hay out of Docker. And with good reason: Docker is definitely an important tool in the data science tool belt. But I'm always surprised that Terraform doesn't make more of an appearance in these conversations.

Terraform has become a mainstay of the devops community, and is a part of the infrastructure-as-code trend. With it, you can describe the configuration of a set of cloud resources, run "apply" and your infrastructure (complete with associated containers) is spun up. 

We use Terraform internally very heavily; I expect its usage within the data science community to grow. This is an awesome walkthrough of how one data science team used it to solve a real problem.


Want to receive more content like this in your inbox?