Most of the development/experimentation I was doing with scikit-learn’s machine learning algorithms was on my local development machine. But eventually I needed to do some heavy duty model training / cross validation, which would take weeks on my local machine. So I decided to make use of one of the cheaper compute optimized EC2 instances that AWS offers.
Unfortunately I had some trouble getting scikit-learn to install on a stock Amazon’s EC2 Linux, but I figured it out eventually. I’m sure others will run into this, so I thought I’d write about it.
Note: you can of course get an EC2 community image or an image from the EC2 marketplace that already has Anaconda or scikit-learn and tools installed. This guide is for installing it on a stock Amazon EC2 Linux instance, in case you already have an instance setup you want to use.
In order to get scikit-learn to work, you’ll need to have pandas, scipy and numpy installed too. Fortunately Amazon EC2 Linux comes with python 2.7 already installed, so you don’t need to worry about that.
Start by ssh’ing into your box. Drop into rootshell with the following command (if you’re going to be typing “sudo” before every single command, might as well be root by default anyway, right?)
sudo su
First you need to install some development tools, since you will literally be compiling some libraries in a bit. Run the following commands:
yum groupinstall ‘Development Tools’
yum install python-devel
Next you’ll install the ATLAS and LAPACK libraries, which are needed by numpy and scipy:
yum install atlas-sse3-devel lapack-devel
Now you’re ready to install first all the necessary python libraries and finally scikit-learn:
pip install numpy
pip install scipy
pip install pandas
pip install scikit-learn
Congratulations. You now have scikit-learn installed on the EC2 Linux box!