01 - Course overview.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Foundations of Machine Learning\n",
    "## Course overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Learning objectives:\n",
    "\n",
    "- Build predictive models from training data\n",
    "- Correctly evaluate predictive models\n",
    "- Analyze and compare the performance of different models\n",
    "- Reason about the mathematical foundations of data mining techniques\n",
    "- Recognize when a predictive model is under/overfitting\n",
    "- Combine the above with dimension-reduction techniques\n",
    "- Visualize and explore data using embeddings and clustering"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Lecturers:\n",
    "\n",
    "- Joaquin Vanschoren (j.vanschoren@tue.nl) MF 7.104a\n",
    "- Vlado Menkovski (v.menkovski@tue.nl) MF 7.097b\n",
    "- Anne Driemel (a.driemel@tue.nl) MF 7.073"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Contact hours:\n",
    "\n",
    "- Mondays, 10:45 - 12:30: Plenary Lectures (Flux 1.02)\n",
    "- Thursdays, 13:45 - 15:30: Tutorials and Feedback (Flux 1.02)\n",
    "    - These are NOT labs\n",
    "- Thursdays, 15:45 - 17:30: Plenary Lectures (Flux 1.02)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Course materials: \n",
    "  \n",
    "- See Canvas for:\n",
    "    - Syllabus\n",
    "    - Announcements, discussions\n",
    "    - Assignments, grades\n",
    "- Lecture materials (Notebooks+PDFs) on GitHub\n",
    "    - https://github.com/joaquinvanschoren/ML-course\n",
    "    - The README contains pointers to relevant books"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Evaluation:\n",
    "\n",
    "- No exam, only assignments (4 team assignments, 1 individual)\n",
    "- Preliminary(!) overview:\n",
    "\n",
    "    - 1: Linear Models, Model selection, Ensembles (15 points)\n",
    "        - Released Feb 8 , Deadline Mar 1\n",
    "    - 2: Kernel methods and Bayesian Inference (15 points)\n",
    "        - Released Mar 1, Deadline Mar 15\n",
    "    - 3: Dimensionality reduction, Embeddings (15 points)\n",
    "        - Released Mar 15, Deadline Mar 29\n",
    "    - 4: Deep learning (15 points)\n",
    "        - Released Mar 29, Deadline Apr 12\n",
    "    - Individual assignment: Data challenge (30 points)\n",
    "        - Released Mar 22, Deadline Apr 20"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- Team assignments in teams of 2 students\n",
    "    - Free choice, form groups on Canvas before Feb 9th!\n",
    "        - Can be the same for all 4 team assignments\n",
    "        - You can use Canvas Discussions or Chat to find teammates\n",
    "    - You're allowed to find a new teammate (yourselves) after each assignment\n",
    "- Passing grade: \n",
    "    - 6/10 over all assignments\n",
    "    - 5/10 on the individual assignment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Course contents\n",
    "May still change!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Contents: Week 1\n",
    "* Machine learning concepts\n",
    "* Build first models with `scikit-learn`\n",
    "* k-Nearest Neighbors\n",
    "* Tutorials:\n",
    "    - Linear Algebra\n",
    "    - Data Analysis with Python\n",
    "* Linear Models\n",
    "    - Linear regression, ridge regression, lasso\n",
    "    - Logistic regression, linear SVMs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Contents: Week 2 (after spring break)\n",
    "* Model evaluation and selection\n",
    "    - Overfitting, cross-validation, ROC analysis, Bias-Variance analysis\n",
    "    - Hyperparameter optimization\n",
    "* Tutorials:\n",
    "    - Data Analysis with Python (continued)\n",
    "    - Feature engineering\n",
    "* Ensemble learning\n",
    "    - Decision trees\n",
    "    - Bagging, (Gradient) Boosting, Stacking"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Contents: Week 3\n",
    "* Kernel methods\n",
    "    - Support Vector Machines, Kernelization\n",
    "* Tutorials:\n",
    "    - Feature engineering (continued)\n",
    "* Bayesian Learning\n",
    "    - Naive Bayes\n",
    "    - Gaussian processes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Contents: Week 4\n",
    "* Constructing pipelines\n",
    "    - Preprocessing, feature engineering, learning\n",
    "    - Building machine learning systems\n",
    "* Tutorials:\n",
    "    - Analysing images\n",
    "* Dimensionality reduction 1\n",
    "    - PCA, MDS, Isomap"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Contents: Week 5\n",
    "* Dimensionality reduction 2\n",
    "    - Random projections, Locality-sensitive hashing\n",
    "* Locality-sensitive hashing\n",
    "    - Jaccard Similarity, MinHashing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Contents: Week 6\n",
    "* Clustering\n",
    "    - Lloyd's algorithm, kMeans++, Gonzales' algorithm\n",
    "* Introduction to Deep Learning\n",
    "    - Artificial neurons, gradient descent, backprop"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Contents: Week 7\n",
    "* Multilayer Perceptron\n",
    "* Convolutional Neural Networks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Contents: Week 8\n",
    "* Recurrent Neural Networks\n",
    "* Q&A"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}