KNN-classification-PS.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "deletable": false,
    "editable": false
   },
   "source": [
    "Copyright 2020 Andrew M. Olney and made available under [CC BY-SA](https://creativecommons.org/licenses/by-sa/4.0) for text and [Apache-2.0](http://www.apache.org/licenses/LICENSE-2.0) for code.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# KNN classification: Problem solving"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this session, you'll work through a complete example using a new dataset, `binary`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the dataframe\n",
    "\n",
    "The `binary.csv` dataset contains 4 variables:\n",
    "\n",
    "| Variable    | Type    | Description           |\n",
    "|:-------------|:---------|:-----------------------|\n",
    "| admit | Nominal   | the admittance status (0=not admitted, 1=admitted) |\n",
    "| gre  | Ratio   | the student's GRE score  |\n",
    "| gpa | Ratio   | the student's GPA |\n",
    "| rank  | Ordinal   | rank of the institution (1=highest to 4=lowest prestige)  |\n",
    "\n",
    "\n",
    "Start by importing `pandas`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"/%IFN5#t2uAm}E`8:KV:\">pd</variable></variables><block type=\"importAs_Python\" id=\"_@5-r*j4E`}d?=DXLinf\" x=\"16\" y=\"10\"><field name=\"libraryName\">pandas</field><field name=\"libraryAlias\" id=\"/%IFN5#t2uAm}E`8:KV:\">pd</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load a dataframe with `binary.csv` and display the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>admit</th>\n",
       "      <th>gre</th>\n",
       "      <th>gpa</th>\n",
       "      <th>rank</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>380</td>\n",
       "      <td>3.61</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>660</td>\n",
       "      <td>3.67</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>800</td>\n",
       "      <td>4.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>640</td>\n",
       "      <td>3.19</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>520</td>\n",
       "      <td>2.93</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>395</th>\n",
       "      <td>0</td>\n",
       "      <td>620</td>\n",
       "      <td>4.00</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>396</th>\n",
       "      <td>0</td>\n",
       "      <td>560</td>\n",
       "      <td>3.04</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>397</th>\n",
       "      <td>0</td>\n",
       "      <td>460</td>\n",
       "      <td>2.63</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>398</th>\n",
       "      <td>0</td>\n",
       "      <td>700</td>\n",
       "      <td>3.65</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>399</th>\n",
       "      <td>0</td>\n",
       "      <td>600</td>\n",
       "      <td>3.89</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>400 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     admit  gre   gpa  rank\n",
       "0        0  380  3.61     3\n",
       "1        1  660  3.67     3\n",
       "2        1  800  4.00     1\n",
       "3        1  640  3.19     4\n",
       "4        0  520  2.93     4\n",
       "..     ...  ...   ...   ...\n",
       "395      0  620  4.00     2\n",
       "396      0  560  3.04     3\n",
       "397      0  460  2.63     2\n",
       "398      0  700  3.65     2\n",
       "399      0  600  3.89     3\n",
       "\n",
       "[400 rows x 4 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataframe = pd.read_csv('datasets/binary.csv')\n",
    "\n",
    "dataframe\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"o69xDDO)6.5D2/bJqST?\">dataframe</variable><variable id=\"/%IFN5#t2uAm}E`8:KV:\">pd</variable></variables><block type=\"variables_set\" id=\"gM*jw`FfIR3)8=g0iEB7\" x=\"11\" y=\"186\"><field name=\"VAR\" id=\"o69xDDO)6.5D2/bJqST?\">dataframe</field><value name=\"VALUE\"><block type=\"varDoMethod_Python\" id=\"ny0sjvqTnn2B]K2za7Li\"><field name=\"VAR\" id=\"/%IFN5#t2uAm}E`8:KV:\">pd</field><field name=\"MEMBER\">read_csv</field><data>pd:read_csv</data><value name=\"INPUT\"><block type=\"text\" id=\"dfrpI5b@DHr+DQ:|@vpv\"><field name=\"TEXT\">datasets/binary.csv</field></block></value></block></value></block><block type=\"variables_get\" id=\"dn{+Q#DO%lN;G_tFGJ#B\" x=\"8\" y=\"304\"><field name=\"VAR\" id=\"o69xDDO)6.5D2/bJqST?\">dataframe</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare the train/test data\n",
    "\n",
    "To train the classifiers, you need to split the dataframe into training data and testing data.\n",
    "\n",
    "Start by creating a dataframe `Y` that just has `admit` in it, and then display `Y` so you can be sure it worked."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>admit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>395</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>396</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>397</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>398</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>399</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>400 rows × 1 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     admit\n",
       "0        0\n",
       "1        1\n",
       "2        1\n",
       "3        1\n",
       "4        0\n",
       "..     ...\n",
       "395      0\n",
       "396      0\n",
       "397      0\n",
       "398      0\n",
       "399      0\n",
       "\n",
       "[400 rows x 1 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y = dataframe[['admit']]\n",
    "\n",
    "Y\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"@83Gxqb{/h|%%9Yr?2q!\">Y</variable><variable id=\"o69xDDO)6.5D2/bJqST?\">dataframe</variable></variables><block type=\"variables_set\" id=\"46~F|Y~BiXd|Vok0}p,p\" x=\"33\" y=\"69\"><field name=\"VAR\" id=\"@83Gxqb{/h|%%9Yr?2q!\">Y</field><value name=\"VALUE\"><block type=\"indexer_Python\" id=\"p,##xUo8Nj3`vpMVBjr8\"><field name=\"VAR\" id=\"o69xDDO)6.5D2/bJqST?\">dataframe</field><value name=\"INDEX\"><block type=\"lists_create_with\" id=\"a.wA7Xdx%GbQ)t#XDKvV\"><mutation items=\"1\"></mutation><value name=\"ADD0\"><block type=\"text\" id=\"aZG|GLqavO~,RXBOcuJ+\"><field name=\"TEXT\">admit</field></block></value></block></value></block></value></block><block type=\"variables_get\" id=\"W]bu*LAe{2;n#?+7WQ-[\" x=\"30\" y=\"219\"><field name=\"VAR\" id=\"@83Gxqb{/h|%%9Yr?2q!\">Y</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next do the same thing for `X` using the other columns in the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>gre</th>\n",
       "      <th>gpa</th>\n",
       "      <th>rank</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>380</td>\n",
       "      <td>3.61</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>660</td>\n",
       "      <td>3.67</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>800</td>\n",
       "      <td>4.00</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>640</td>\n",
       "      <td>3.19</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>520</td>\n",
       "      <td>2.93</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>395</th>\n",
       "      <td>620</td>\n",
       "      <td>4.00</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>396</th>\n",
       "      <td>560</td>\n",
       "      <td>3.04</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>397</th>\n",
       "      <td>460</td>\n",
       "      <td>2.63</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>398</th>\n",
       "      <td>700</td>\n",
       "      <td>3.65</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>399</th>\n",
       "      <td>600</td>\n",
       "      <td>3.89</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>400 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     gre   gpa  rank\n",
       "0    380  3.61     3\n",
       "1    660  3.67     3\n",
       "2    800  4.00     1\n",
       "3    640  3.19     4\n",
       "4    520  2.93     4\n",
       "..   ...   ...   ...\n",
       "395  620  4.00     2\n",
       "396  560  3.04     3\n",
       "397  460  2.63     2\n",
       "398  700  3.65     2\n",
       "399  600  3.89     3\n",
       "\n",
       "[400 rows x 3 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = dataframe[['gre', 'gpa', 'rank']]\n",
    "\n",
    "X\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"py,.kr!f9i=.v7I5_d+^\">X</variable><variable id=\"o69xDDO)6.5D2/bJqST?\">dataframe</variable></variables><block type=\"variables_set\" id=\"46~F|Y~BiXd|Vok0}p,p\" x=\"33\" y=\"69\"><field name=\"VAR\" id=\"py,.kr!f9i=.v7I5_d+^\">X</field><value name=\"VALUE\"><block type=\"indexer_Python\" id=\"p,##xUo8Nj3`vpMVBjr8\"><field name=\"VAR\" id=\"o69xDDO)6.5D2/bJqST?\">dataframe</field><value name=\"INDEX\"><block type=\"lists_create_with\" id=\"a.wA7Xdx%GbQ)t#XDKvV\"><mutation items=\"3\"></mutation><value name=\"ADD0\"><block type=\"text\" id=\"aZG|GLqavO~,RXBOcuJ+\"><field name=\"TEXT\">gre</field></block></value><value name=\"ADD1\"><block type=\"text\" id=\"b/mW9(0w,rS.ny;Y(]5A\"><field name=\"TEXT\">gpa</field></block></value><value name=\"ADD2\"><block type=\"text\" id=\"4yc?!4pnBI1/rSBQ41!t\"><field name=\"TEXT\">rank</field></block></value></block></value></block></value></block><block type=\"variables_get\" id=\"W]bu*LAe{2;n#?+7WQ-[\" x=\"30\" y=\"219\"><field name=\"VAR\" id=\"py,.kr!f9i=.v7I5_d+^\">X</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To split the data into training and testing data, import `model_selection`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn.model_selection as model_selection\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"uASGz64Zb$AOvQyV4pRj\">model_selection</variable></variables><block type=\"importAs_Python\" id=\"sN1YO5FEzpHyxb31@j,Z\" x=\"16\" y=\"10\"><field name=\"libraryName\">sklearn.model_selection</field><field name=\"libraryAlias\" id=\"uASGz64Zb$AOvQyV4pRj\">model_selection</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now split the data into training and testing data, using `test_size` at one of the following 3 values depending on your birthday:\n",
    "\n",
    "If your birthday is in:\n",
    "\n",
    "- Jan, Feb, Mar, Apr, use `0.2` \n",
    "- May, Jun, Jul, Aug, use `0.4`\n",
    "- Sep, Oct, Nov, Dec, use `0.6`\n",
    "\n",
    "So depending on your birthday, we'll use 20, 40, or 60% of the data for testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "splits = model_selection.train_test_split(X, Y, test_size=0.2)\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"_ut$e0PL4OMi4o1MXTpw\">splits</variable><variable id=\"uASGz64Zb$AOvQyV4pRj\">model_selection</variable><variable id=\"py,.kr!f9i=.v7I5_d+^\">X</variable><variable id=\"@83Gxqb{/h|%%9Yr?2q!\">Y</variable></variables><block type=\"variables_set\" id=\"oTGRJ#{R!U^we@Bl@pkT\" x=\"31\" y=\"224\"><field name=\"VAR\" id=\"_ut$e0PL4OMi4o1MXTpw\">splits</field><value name=\"VALUE\"><block type=\"varDoMethod_Python\" id=\"f?j@ker(a#hJv;Nh)IGX\"><field name=\"VAR\" id=\"uASGz64Zb$AOvQyV4pRj\">model_selection</field><field name=\"MEMBER\">train_test_split</field><data>model_selection:train_test_split</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"er6r2]}|nA;1;}VsM5I7\"><mutation items=\"3\"></mutation><value name=\"ADD0\"><block type=\"variables_get\" id=\".mm}`*H4)i%Eq5z={e-$\"><field name=\"VAR\" id=\"py,.kr!f9i=.v7I5_d+^\">X</field></block></value><value name=\"ADD1\"><block type=\"variables_get\" id=\"I3dOV;CPBf^~E%BvgthZ\"><field name=\"VAR\" id=\"@83Gxqb{/h|%%9Yr?2q!\">Y</field></block></value><value name=\"ADD2\"><block type=\"dummyOutputCodeBlock_Python\" id=\"@Hg?ib/!8fH$;f3pWJy2\"><field name=\"CODE\">test_size=0.2</field></block></value></block></value></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## KNN\n",
    "\n",
    "First import `neighbors`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn.neighbors as neighbors\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"~%[Y}a{Syr+LrQ[I8?d(\">neighbors</variable></variables><block type=\"importAs_Python\" id=\"ayw$B{(evWs,ynm/fP%g\" x=\"123\" y=\"313\"><field name=\"libraryName\">sklearn.neighbors</field><field name=\"libraryAlias\" id=\"~%[Y}a{Syr+LrQ[I8?d(\">neighbors</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next define the KNN model, e.g. using `create`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "knn = neighbors.KNeighborsClassifier(n_neighbors=5)\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"DOr#Z1+,_{KG18zR#G)4\">knn</variable><variable id=\"~%[Y}a{Syr+LrQ[I8?d(\">neighbors</variable></variables><block type=\"variables_set\" id=\"`z.|tb|jX$Y2a5;iN{pz\" x=\"-7\" y=\"283\"><field name=\"VAR\" id=\"DOr#Z1+,_{KG18zR#G)4\">knn</field><value name=\"VALUE\"><block type=\"varCreateObject_Python\" id=\"9i7]OHHOnhf2~MjH^n+y\"><field name=\"VAR\" id=\"~%[Y}a{Syr+LrQ[I8?d(\">neighbors</field><field name=\"MEMBER\">KNeighborsClassifier</field><data>neighbors:KNeighborsClassifier</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"`.s=9G`sj;R8,?u+OR9Y\"><mutation items=\"1\"></mutation><value name=\"ADD0\"><block type=\"dummyOutputCodeBlock_Python\" id=\"R*)t!8POzxw3}@cy,{:]\"><field name=\"CODE\">n_neighbors=5</field></block></value></block></value></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now train KNN and do the predictions in one cell.\n",
    "Save the predictions in the variable `knnPredictions`, and then show the predictions to make sure it worked."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0,\n",
       "       1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,\n",
       "       0, 0, 1, 0, 1, 0, 1, 0, 1, 1, 0, 0, 1, 0, 0, 0, 1, 1, 0, 1, 0, 0,\n",
       "       0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0])"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "knn.fit(splits[0], np.ravel(splits[2]))\n",
    "\n",
    "knnPredictions = knn.predict(splits[1])\n",
    "\n",
    "knnPredictions\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"DOr#Z1+,_{KG18zR#G)4\">knn</variable><variable id=\"vM0j[9C:2`MqdV2+),oH\">knnPredictions</variable><variable id=\"Zhzp)s*VL?V@ES3(j:*b\">np</variable><variable id=\"_ut$e0PL4OMi4o1MXTpw\">splits</variable></variables><block type=\"varDoMethod_Python\" id=\"Z)$q-zn.KTC{+^l-wH6u\" x=\"259\" y=\"169\"><field name=\"VAR\" id=\"DOr#Z1+,_{KG18zR#G)4\">knn</field><field name=\"MEMBER\">fit</field><data>knn:fit</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"e_B;36VOJ^lH70V=aWY}\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"lists_getIndex\" id=\"C#,#1*rEm+]qEx?L1x[L\"><mutation statement=\"false\" at=\"true\"></mutation><field name=\"MODE\">GET</field><field name=\"WHERE\">FROM_START</field><value name=\"VALUE\"><block type=\"variables_get\" id=\"b_Sz{9#d7d=ystO|k?l_\"><field name=\"VAR\" id=\"_ut$e0PL4OMi4o1MXTpw\">splits</field></block></value><value name=\"AT\"><block type=\"math_number\" id=\":s{r1~S,,@.CSh#9`$R;\"><field name=\"NUM\">1</field></block></value></block></value><value name=\"ADD1\"><block type=\"varDoMethod_Python\" id=\"zYBlZ,!^P^%Ib5R@Ch.H\"><field name=\"VAR\" id=\"Zhzp)s*VL?V@ES3(j:*b\">np</field><field name=\"MEMBER\">ravel</field><data>np:ravel</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"9s({WSn={~Ink.5O+6Cc\"><mutation items=\"1\"></mutation><value name=\"ADD0\"><block type=\"lists_getIndex\" id=\"^)UBt0jM;BnGmWeG7pw*\"><mutation statement=\"false\" at=\"true\"></mutation><field name=\"MODE\">GET</field><field name=\"WHERE\">FROM_START</field><value name=\"VALUE\"><block type=\"variables_get\" id=\"4Vo!*g]qQ=D}XtD2i39/\"><field name=\"VAR\" id=\"_ut$e0PL4OMi4o1MXTpw\">splits</field></block></value><value name=\"AT\"><block type=\"math_number\" id=\"RysCD3.C27sBxztz(T}2\"><field name=\"NUM\">3</field></block></value></block></value></block></value></block></value></block></value></block><block type=\"variables_set\" id=\"qn$V)Zc+x1,^]#ZWBu0;\" x=\"259\" y=\"295\"><field name=\"VAR\" id=\"vM0j[9C:2`MqdV2+),oH\">knnPredictions</field><value name=\"VALUE\"><block type=\"varDoMethod_Python\" id=\"rq%)Cuo$S!Rv]]83[(OQ\"><field name=\"VAR\" id=\"DOr#Z1+,_{KG18zR#G)4\">knn</field><field name=\"MEMBER\">predict</field><data>knn:predict</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"g!3rY4P1(W4]{2oU]dDb\"><mutation items=\"1\"></mutation><value name=\"ADD0\"><block type=\"lists_getIndex\" id=\"70_kQu*_lQfPL$)[E*]S\"><mutation statement=\"false\" at=\"true\"></mutation><field name=\"MODE\">GET</field><field name=\"WHERE\">FROM_START</field><value name=\"VALUE\"><block type=\"variables_get\" id=\"f8W4Kf.xz`3No$qQ@|16\"><field name=\"VAR\" id=\"_ut$e0PL4OMi4o1MXTpw\">splits</field></block></value><value name=\"AT\"><block type=\"math_number\" id=\"}:Kw6eCPs%6nW8FY_E!N\"><field name=\"NUM\">2</field></block></value></block></value></block></value></block></value></block><block type=\"variables_get\" id=\"l~QDDPzM@1QK#7dp~N~H\" x=\"264\" y=\"396\"><field name=\"VAR\" id=\"vM0j[9C:2`MqdV2+),oH\">knnPredictions</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You should see a mix of `1` and `0` in the predictions. \n",
    "\n",
    "**QUESTION:**\n",
    "\n",
    "Do you think `0` or `1` is more common in this dataset?\n",
    "What could you do with the dataframe to check?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ANSWER: (click here to edit)**\n",
    "\n",
    "*`0` looks more common. An easy way to check would be to use `describe` on the dataframe.*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classifier evaluation\n",
    "\n",
    "To see if the model is any good, do some evaluations.\n",
    "\n",
    "First import `metrics`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sklearn.metrics as metrics\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"C8HQ^p):qDkQA/nhxe{{\">metrics</variable></variables><block type=\"importAs_Python\" id=\"|1fEPj_#9:@0Qa@y|1F.\" x=\"135\" y=\"207\"><field name=\"libraryName\">sklearn.metrics</field><field name=\"libraryAlias\" id=\"C8HQ^p):qDkQA/nhxe{{\">metrics</field></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Accuracy\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And calculate the KNN accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7125"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.accuracy_score(splits[3], knnPredictions)\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"C8HQ^p):qDkQA/nhxe{{\">metrics</variable><variable id=\"vM0j[9C:2`MqdV2+),oH\">knnPredictions</variable><variable id=\"_ut$e0PL4OMi4o1MXTpw\">splits</variable></variables><block type=\"varDoMethod_Python\" id=\"l,28Y;cf6Q.r-rTBWP4}\" x=\"8\" y=\"188\"><field name=\"VAR\" id=\"C8HQ^p):qDkQA/nhxe{{\">metrics</field><field name=\"MEMBER\">accuracy_score</field><data>metrics:accuracy_score</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"`V^tbb51h?lxzH^Z,L0+\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"lists_getIndex\" id=\"6Nb!QUCs+zcSbgi2=8Jb\"><mutation statement=\"false\" at=\"true\"></mutation><field name=\"MODE\">GET</field><field name=\"WHERE\">FROM_START</field><value name=\"VALUE\"><block type=\"variables_get\" id=\"0SesC-O%M@-rqWgnWJ[q\"><field name=\"VAR\" id=\"_ut$e0PL4OMi4o1MXTpw\">splits</field></block></value><value name=\"AT\"><block type=\"math_number\" id=\"ZQBTZmCAMzn[;:irh:{@\"><field name=\"NUM\">4</field></block></value></block></value><value name=\"ADD1\"><block type=\"variables_get\" id=\":5r(/A;QY|,.0/gzcHi.\"><field name=\"VAR\" id=\"vM0j[9C:2`MqdV2+),oH\">knnPredictions</field></block></value></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**QUESTION:**\n",
    "\n",
    "What does the accuracy tell you about the errors the model is making?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ANSWER: (click here to edit)**\n",
    "\n",
    "*Accuracy doesn't tell you about a specific type of error; you'd need to use a confusion matrix or other metric to get that information*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**QUESTION:**\n",
    "\n",
    "What do you expect to happen to accuracy as you increase `test_size`?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ANSWER: (click here to edit)**\n",
    "\n",
    "*Accuracy should go down as test size increases, because putting more data into test means less data for training, and KNN needs as much training data as possible*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Recall/Precision per class\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the KNN `classification_report`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.77      0.84      0.81        57\n",
      "           1       0.50      0.39      0.44        23\n",
      "\n",
      "    accuracy                           0.71        80\n",
      "   macro avg       0.64      0.62      0.62        80\n",
      "weighted avg       0.70      0.71      0.70        80\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(metrics.classification_report(splits[3], knnPredictions))\n",
    "\n",
    "#<xml xmlns=\"https://developers.google.com/blockly/xml\"><variables><variable id=\"C8HQ^p):qDkQA/nhxe{{\">metrics</variable><variable id=\"vM0j[9C:2`MqdV2+),oH\">knnPredictions</variable><variable id=\"_ut$e0PL4OMi4o1MXTpw\">splits</variable></variables><block type=\"text_print\" id=\"Sx0uF}9IfzkucZiyR^1:\" x=\"196\" y=\"360\"><value name=\"TEXT\"><shadow type=\"text\" id=\"a=J{E!^,yE0MlS-rEKnT\"><field name=\"TEXT\">abc</field></shadow><block type=\"varDoMethod_Python\" id=\"l,28Y;cf6Q.r-rTBWP4}\"><field name=\"VAR\" id=\"C8HQ^p):qDkQA/nhxe{{\">metrics</field><field name=\"MEMBER\">classification_report</field><data>metrics:classification_report</data><value name=\"INPUT\"><block type=\"lists_create_with\" id=\"`V^tbb51h?lxzH^Z,L0+\"><mutation items=\"2\"></mutation><value name=\"ADD0\"><block type=\"lists_getIndex\" id=\"6Nb!QUCs+zcSbgi2=8Jb\"><mutation statement=\"false\" at=\"true\"></mutation><field name=\"MODE\">GET</field><field name=\"WHERE\">FROM_START</field><value name=\"VALUE\"><block type=\"variables_get\" id=\"0SesC-O%M@-rqWgnWJ[q\"><field name=\"VAR\" id=\"_ut$e0PL4OMi4o1MXTpw\">splits</field></block></value><value name=\"AT\"><block type=\"math_number\" id=\"ZQBTZmCAMzn[;:irh:{@\"><field name=\"NUM\">4</field></block></value></block></value><value name=\"ADD1\"><block type=\"variables_get\" id=\":5r(/A;QY|,.0/gzcHi.\"><field name=\"VAR\" id=\"vM0j[9C:2`MqdV2+),oH\">knnPredictions</field></block></value></block></value></block></value></block></xml>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**QUESTION:**\n",
    "\n",
    "Is there a particular class that KNN does worse on?\n",
    "Why do you think that might be?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**ANSWER: (click here to edit)**\n",
    "\n",
    "*About 69% of the `admit`s are `0`, so the classifier has a harder time with the `1`s because they are less common. This is called imbalanced classes and is problem often seen in the real world.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "xpython",
   "language": "python",
   "name": "xpython"
  },
  "language_info": {
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}