intro-to-scikit-learn-1-Basics.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to scikit-learn\n",
    "_Adapted from notebook by [Andreas Mueller](http://amueller.github.io/)._\n",
    "\n",
    "Let us look at how to use the scikit-learn fit/predict API in practice. First, let us get some data to play with. We will use the [Digit Dataset](http://scikit-learn.org/stable/auto_examples/datasets/plot_digits_last_image.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_digits\n",
    "digits = load_digits()\n",
    "digits.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "digits.images.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print(digits.images[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "plt.matshow(digits.images[0], cmap=plt.cm.Greys)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "digits.data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "digits.target.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "digits.target"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Data is always a numpy array (or sparse matrix) of shape (n_samples, n_features)**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split the data to get going"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.cross_validation import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(digits.data,\n",
    "                                                    digits.target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Really Simple API\n",
    "\n",
    "0) Import your model class. For this example, we'll use [Linear Support Vector Classification](http://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html). [More on SVM at Jose's class on Jan 25!]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.svm import LinearSVC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1) Instantiate an object and set the parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "svm = LinearSVC(C=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2) Fit the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "svm.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3) Apply / evaluate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "print(svm.predict(X_train))\n",
    "print(y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "svm.score(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "svm.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## And again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "rf = RandomForestClassifier(n_estimators=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "rf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "rf.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visual comparison of classifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pylab as pl\n",
    "from matplotlib.colors import ListedColormap\n",
    "from sklearn.cross_validation import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.datasets import make_moons, make_circles, make_classification\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier, AdaBoostClassifier\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.discriminant_analysis import LinearDiscriminantAnalysis\n",
    "from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis\n",
    "\n",
    "h = .02  # step size in the mesh\n",
    "\n",
    "names = [\"Nearest Neighbors\", \"Linear SVM\", \"RBF SVM\", \"Decision Tree\",\n",
    "         \"Random Forest\", \"AdaBoost\", \"Naive Bayes\", \"Lin Discr Analysis\", \"Quad Discr Analysis\"]\n",
    "classifiers = [\n",
    "    KNeighborsClassifier(3),\n",
    "    SVC(kernel=\"linear\", C=0.025),\n",
    "    SVC(gamma=2, C=1),\n",
    "    DecisionTreeClassifier(max_depth=5),\n",
    "    RandomForestClassifier(max_depth=5, n_estimators=10, max_features=1),\n",
    "    AdaBoostClassifier(),\n",
    "    GaussianNB(),\n",
    "    LinearDiscriminantAnalysis(),\n",
    "    QuadraticDiscriminantAnalysis()]\n",
    "\n",
    "X, y = make_classification(n_features=2, n_redundant=0, n_informative=2,\n",
    "                           random_state=1, n_clusters_per_class=1)\n",
    "rng = np.random.RandomState(2)\n",
    "X += 2 * rng.uniform(size=X.shape)\n",
    "linearly_separable = (X, y)\n",
    "\n",
    "datasets = [make_moons(noise=0.3, random_state=0),\n",
    "            make_circles(noise=0.2, factor=0.5, random_state=1),\n",
    "            linearly_separable\n",
    "            ]\n",
    "\n",
    "figure = pl.figure(figsize=(21, 7))\n",
    "i = 1\n",
    "# iterate over datasets\n",
    "for ds in datasets:\n",
    "    # preprocess dataset, split into training and test part\n",
    "    X, y = ds\n",
    "    X = StandardScaler().fit_transform(X)\n",
    "    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.4)\n",
    "\n",
    "    x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5\n",
    "    y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5\n",
    "    xx, yy = np.meshgrid(np.arange(x_min, x_max, h),\n",
    "                         np.arange(y_min, y_max, h))\n",
    "\n",
    "    # just plot the dataset first\n",
    "    cm = pl.cm.RdBu\n",
    "    cm_bright = ListedColormap(['#FF0000', '#0000FF'])\n",
    "    ax = pl.subplot(len(datasets), len(classifiers) + 1, i)\n",
    "    # Plot the training points\n",
    "    ax.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=cm_bright)\n",
    "    # and testing points\n",
    "    ax.scatter(X_test[:, 0], X_test[:, 1], c=y_test, cmap=cm_bright, alpha=0.6)\n",
    "    ax.set_xlim(xx.min(), xx.max())\n",
    "    ax.set_ylim(yy.min(), yy.max())\n",
    "    ax.set_xticks(())\n",
    "    ax.set_yticks(())\n",
    "    i += 1\n",
    "\n",
    "    # iterate over classifiers\n",
    "    for name, clf in zip(names, classifiers):\n",
    "        ax = pl.subplot(len(datasets), len(classifiers) + 1, i)\n",
    "        clf.fit(X_train, y_train)\n",
    "        score = clf.score(X_test, y_test)\n",
    "\n",
    "        # Plot the decision boundary. For that, we will assign a color to each\n",
    "        # point in the mesh [x_min, m_max]x[y_min, y_max].\n",
    "        if hasattr(clf, \"decision_function\"):\n",
    "            Z = clf.decision_function(np.c_[xx.ravel(), yy.ravel()])\n",
    "        else:\n",
    "            Z = clf.predict_proba(np.c_[xx.ravel(), yy.ravel()])[:, 1]\n",
    "\n",
    "        # Put the result into a color plot\n",
    "        Z = Z.reshape(xx.shape)\n",
    "        ax.contourf(xx, yy, Z, cmap=cm, alpha=.8)\n",
    "\n",
    "        # Plot also the training points\n",
    "        ax.scatter(X_train[:, 0], X_train[:, 1], c=y_train, cmap=cm_bright)\n",
    "        # and testing points\n",
    "        ax.scatter(X_test[:, 0], X_test[:, 1], c=y_test, cmap=cm_bright,\n",
    "                   alpha=0.6)\n",
    "\n",
    "        ax.set_xlim(xx.min(), xx.max())\n",
    "        ax.set_ylim(yy.min(), yy.max())\n",
    "        ax.set_xticks(())\n",
    "        ax.set_yticks(())\n",
    "        ax.set_title(name)\n",
    "        ax.text(xx.max() - .3, yy.min() + .3, ('%.2f' % score).lstrip('0'),\n",
    "                size=15, horizontalalignment='right')\n",
    "        i += 1\n",
    "\n",
    "figure.subplots_adjust(left=.02, right=.98)\n",
    "figure.savefig('ClassifierComparison.png')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}