tutorials/explainers/LIME/lime_tabular_penguin.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img width=\"150\" alt=\"Logo_ER10\" src=\"https://user-images.githubusercontent.com/3244249/151994514-b584b984-a148-4ade-80ee-0f88b0aefa45.png\">\n",
    "\n",
    "### Model Interpretation using LIME for penguin dataset classifier\n",
    "This notebook demonstrates the use of DIANNA with the LIME tabular method on the penguins dataset.\n",
    "\n",
    "LIME (Local Interpretable Model-agnostic Explanations) is an explainable-AI method that aims to create an interpretable model that locally represents the classifier. For more details see the [LIME paper](https://arxiv.org/abs/1602.04938)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Colab setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "running_in_colab = 'google.colab' in str(get_ipython())\n",
    "if running_in_colab:\n",
    "    # install dianna\n",
    "    !python3 -m pip install dianna[notebooks]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 0 - Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import dianna\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "from sklearn.model_selection import train_test_split\n",
    "from dianna.utils.onnx_runner import SimpleModelRunner\n",
    "from dianna.utils.downloader import download"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1 - Loading the data\n",
    "Load penguins dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "penguins = sns.load_dataset('penguins')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prepare the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>bill_length_mm</th>\n",
       "      <th>bill_depth_mm</th>\n",
       "      <th>flipper_length_mm</th>\n",
       "      <th>body_mass_g</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39.1</td>\n",
       "      <td>18.7</td>\n",
       "      <td>181.0</td>\n",
       "      <td>3750.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>39.5</td>\n",
       "      <td>17.4</td>\n",
       "      <td>186.0</td>\n",
       "      <td>3800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>40.3</td>\n",
       "      <td>18.0</td>\n",
       "      <td>195.0</td>\n",
       "      <td>3250.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>36.7</td>\n",
       "      <td>19.3</td>\n",
       "      <td>193.0</td>\n",
       "      <td>3450.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>39.3</td>\n",
       "      <td>20.6</td>\n",
       "      <td>190.0</td>\n",
       "      <td>3650.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>338</th>\n",
       "      <td>47.2</td>\n",
       "      <td>13.7</td>\n",
       "      <td>214.0</td>\n",
       "      <td>4925.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>340</th>\n",
       "      <td>46.8</td>\n",
       "      <td>14.3</td>\n",
       "      <td>215.0</td>\n",
       "      <td>4850.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>341</th>\n",
       "      <td>50.4</td>\n",
       "      <td>15.7</td>\n",
       "      <td>222.0</td>\n",
       "      <td>5750.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>342</th>\n",
       "      <td>45.2</td>\n",
       "      <td>14.8</td>\n",
       "      <td>212.0</td>\n",
       "      <td>5200.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>343</th>\n",
       "      <td>49.9</td>\n",
       "      <td>16.1</td>\n",
       "      <td>213.0</td>\n",
       "      <td>5400.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>342 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     bill_length_mm  bill_depth_mm  flipper_length_mm  body_mass_g\n",
       "0              39.1           18.7              181.0       3750.0\n",
       "1              39.5           17.4              186.0       3800.0\n",
       "2              40.3           18.0              195.0       3250.0\n",
       "4              36.7           19.3              193.0       3450.0\n",
       "5              39.3           20.6              190.0       3650.0\n",
       "..              ...            ...                ...          ...\n",
       "338            47.2           13.7              214.0       4925.0\n",
       "340            46.8           14.3              215.0       4850.0\n",
       "341            50.4           15.7              222.0       5750.0\n",
       "342            45.2           14.8              212.0       5200.0\n",
       "343            49.9           16.1              213.0       5400.0\n",
       "\n",
       "[342 rows x 4 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Remove categorial columns and NaN values\n",
    "penguins_filtered = penguins.drop(columns=['island', 'sex']).dropna()\n",
    "\n",
    "# Get the species\n",
    "species = penguins['species'].unique()\n",
    "\n",
    "# Extract inputs and target\n",
    "input_features = penguins_filtered.drop(columns=['species'])\n",
    "target = pd.get_dummies(penguins_filtered['species'])\n",
    "\n",
    "# Let's explore the features of the dataset\n",
    "input_features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data-set currently has four features that were used to train the model: bill length, bill depth, flipper length, and body mass. These features were used to classify the different species."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Training, validation, and test data split."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(input_features, target, test_size=0.2,\n",
    "                                                    random_state=0, shuffle=True, stratify=target)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get an instance to explain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get an instance from test data\n",
    "data_instance = X_test.iloc[10].to_numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2 - Loading ONNX model\n",
    "DIANNA supports ONNX models. Here we demonstrate the use of LIME explainer for tabular data with a pre-trained ONNX model, which is a MLP classifier for the penguins dataset. <br>\n",
    "\n",
    "The model is trained following this notebook: <br>\n",
    "https://github.com/dianna-ai/dianna-exploration/blob/main/example_data/model_generation/penguin_species/generate_model.ipynb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Gentoo'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load onnx model and check the prediction with it\n",
    "model_path = download('penguin_model.onnx', 'model')\n",
    "loaded_model = SimpleModelRunner(model_path)\n",
    "predictions = loaded_model(data_instance.reshape(1,-1).astype(np.float32))\n",
    "species[np.argmax(predictions)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A runner function is created to prepare data for the ONNX inference session."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import onnxruntime as ort\n",
    "\n",
    "def run_model(data):\n",
    "    # get ONNX predictions\n",
    "    sess = ort.InferenceSession(model_path)\n",
    "    input_name = sess.get_inputs()[0].name\n",
    "    output_name = sess.get_outputs()[0].name\n",
    "\n",
    "    onnx_input = {input_name: data.astype(np.float32)}\n",
    "    pred_onnx = sess.run([output_name], onnx_input)[0]\n",
    "    \n",
    "    return pred_onnx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3 - Applying LIME with DIANNA\n",
    "The simplest way to run DIANNA on image data is with `dianna.explain_tabular`.\n",
    "\n",
    "DIANNA requires input in numpy format, so the input data is converted into a numpy array.\n",
    "\n",
    "Note that the training data is also required since LIME needs it to generate proper perturbation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "explanation = dianna.explain_tabular(run_model, input_tabular=data_instance, method='lime',\n",
    "                                     mode ='classification', training_data = X_train.to_numpy(),\n",
    "                                     feature_names=input_features.columns, class_names=species)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4 - Visualization\n",
    "The output can be visualized with the DIANNA built-in visualization function. It shows the importance of each feature contributing to the prediction.\n",
    "\n",
    "The prediction is \"Gentoo\", so let's visualize the feature importance scores for \"Gentoo\".\n",
    "\n",
    "It can be noticed that the body mass feature has the biggest weight in the prediction."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAc8AAAEGCAYAAAADnx5XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbCElEQVR4nO3dfZRlVX3m8e8jTSQoIEqLmoiliAjIi6HAwaC20ajxLaIoitE4Gg3KYMTlrHGWictEnWDMDHFUQEJIqygayURAR1CQt6BAF/LSNIgoaCAypvMCgi8t0L/54+yO10511T1N37q3qr+ftWpx7r777vO7t5t+ap9z7j6pKiRJ0vAeMO4CJElabAxPSZJ6MjwlSerJ8JQkqSfDU5KknpaNuwAtjF122aWmpqbGXYYkLRpXXnnlP1fV8tmeMzy3ElNTU8zMzIy7DElaNJJ8b1PPedhWkqSeDE9JknoyPCVJ6snwlCSpJ8NTkqSeDE9JknoyPCVJ6snwlCSpJxdJ0PglP9/2/rKSFgFnnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLU06IJzyRTSa6bpf2UJHu37e8m2aVt391j7JVJDt/MulYkeeqWGEuStDgsG3cB91dV/d6YS1gB3A18bcx1SJIWyKKZeTbLknwqyQ1JzkiyfZILk0z3GSSdjyS5Mcl5wMMHnjswyUVJrkxybpJHtvYLk3woydVJrktycJIp4Cjg2Nb+tDbM05N8LcnNc81C26z1oiRntr7HJXl1kiuSrE6ye+u3MsmJSS5r/VYkObV9DivnGP9NSWaSzKxdu7bPRyRJmsNiC889gROqai/gh8BbNnOcw9pYewOvBZ4KkGRb4MPA4VV1IHAq8P6B121fVQe0/Z5aVd8FTgKOr6oDquqS1u+RwKHAC4Hj5qllf7oA3gt4DfCEqjoYOAU4ZqDfzsAhwLHAWcDxwD7AvkkOmG3gqjq5qqaranr58uXzlCFJGtZiO2x7a1Vd2rZPA966meM8HTi9qu4Dvp/kq619T+BJwFeSAGwD3D7wutMBquriJDsmecgmxv98Va0Hrk+y6zy1rKqq2wGSfAf4cmtfDTxzoN/ZVVVJVgM/qKrV7TVrgCng6nn2I0naQhZbeNY8j++vAGuq6pD7uf91G405l8G+6wcer+cX/3zWzdJntn6SpBFbbIdtd0uyIdiOBP5+M8e5GDgiyTbtnOaGGd6NwPIN+0iybZJ9Bl53RGs/FLizqu4E7gJ22Mw6JEmL0GILzxuBo5PcQHcO8MTNHOfvgJuA64FPAF8HqKqfAYcDH0hyDd2h0KcOvO6nSa6iO8/5htZ2NnDYRhcMSZKWsFRt6SOfS1OSC4F3VNXMuGvZHNPT0zUzM6GlZ+DItn8fJU2IJFdW1azf5lhsM09JksZuSV9okmRf4JMbNa+rqqf0HauqVkxCHZKk8VvS4dm+znGAdUiStiQP20qS1JPhKUlST4anJEk9GZ6SJPVkeEqS1JPhKUlST4anJEk9GZ6SJPVkeEqS1JPhKUlST4anJEk9Lem1bbVIeBsySYuMM09JknoyPCVJ6snwlCSpJ8NTkqSeDE9JknoyPCVJ6snwlCSpJ8NTkqSeDE9JknpyhSHNKxnt+C4wJGmxceYpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST2NLDyTvDXJDUn+MclHWttRSV47qn3OU893k+yyhcecSnLkwOPXbXivkqSla9kIx34L8Oz2Mw1QVSeNcH8kCZCqWj/K/QyYAo4EPr1A+5MkTYCRzDyTnAQ8DvgSsPNA+3uSvKNtX5jkQ0muTnJdkoMH+nwyydeT3JTkjQOv/69JViW5Nskft7apJDcm+QRwHfDoIer7nSRXtH1/LMk2rf3uJO9Pck2Sy5Ls2tp3b49XJ3lfkrvbUMcBT2vjHNvaHpXknFb7n81Tx91JPphkTZLzkhzcPpebk7y49Xldks8n+UqbPf+XJG9PclWr6aFzjP+mJDNJZtauXTvfxyJJGtJIwrOqjgK+DzwT+Lc5um5fVQfQzVJPHWjfD/gN4BDg3UkeleQ5wB7AwcABwIFJnt767wGcUFX7VNX35qotyV7AEcCvt33fB7y6Pf0g4LKq2h+4GNgQ3B8CPlRV+wK3DQz3TuCSqjqgqo5vbQe08fcFjkgyV5g/CPhqVe0D3AW8D/hN4DDgTwb6PQl4KXAQ8H7gx1X1ZODrwCYPg1fVyVU1XVXTy5cvn6MMSVIfozxsO4zTAarq4iQ7JnlIaz+zqn4C/CTJBXSBeSjwHOCq1ufBdKH5D8D3quqyIff5LOBAYFV3lJdfBv6pPfcz4Att+0q6IIMuxF/Stj8N/Pkc459fVXcCJLkeeAxw6yb6/gw4p22vBtZV1T1JVtMdEt7ggqq6C7gryZ3A2QOv2W+OWiRJIzDu8KxNPJ6tPcCfVtXHBp9IMgX8qMc+A3y8qv77LM/dU1Ub9n0fm/f5rBvYnm+Mwf2t3/DaqlqfZPB1g2OuH3i8fjNrlCTdD+P+qsoRAEkOBe7cMGMDfjvJdkkeBqwAVgHnAq9P8uD2ml9J8vDN2Of5wOEbXpvkoUkeM89rLgNe1rZfOdB+F7DDZtQgSVrExj1r+WmSq4BtgdcPtF8LXADsAry3qr4PfL+dr/x6O9x6N/A7dLO7oVXV9Un+EPhykgcA9wBHA3OdK30bcFqSd9EdZt0Q8tcC9yW5BljJ3Od3JUlLRH5+1HCBd5xcCLyjqmY2an8PcHdVzXVecUEl2R74SVVVklcCr6qq3x53XX1MT0/XzMzM/B1n0f2uMjpj+isoSXNKcmVVTc/23FAzzyS7A7dV1bokK+guUvlEVd2xpYqccAcCH2nfI72DX5wlS5K2MsMetv1bYDrJ44GTgTPprjp9/ubuuKpWbKL9PZs7JkCSy4EHbtT8mqpavbljVtUlwP6TVJMkaXyGDc/1VXVvksOAD1fVh9u5yolTVU8Zdw0bm8SaJEmbb9irbe9J8irgd/n59yC3HU1JkiRNtmHD8z/TLRTw/qq6JcljgU+OrixJkibXUIdt29c7/huwW3t8C/CBURYmSdKkGmrmmeRFwNW0peSSHJDkrBHWJUnSxBr2sO176NaXvQOgqq6mu2uKJElbnaEvGBpYOm+DhbpnpiRJE2XYr6qsSXIksE2SPYC3Al8bXVmSJE2uYWeexwD70N3N49N0a7u+bUQ1SZI00eadeSbZBvhiVT0TeNfoS5IkabLNO/OsqvuA9Ul2WoB6JEmaeMOe87wbWJ3kKwzceLqq3jqSqiRJmmDDhuf/aT/aCnnLMEn6RcOuMPTxURciSdJiMez9PG8B/sP8o6pcKEGStNUZ9rDt4J20twNeDjx0y5cjSdLkG+p7nlX1LwM//1hVfwG8YLSlSZI0mYY9bPtrAw8fQDcTHXbWKknSkjJsAP7Pge17gVuAV2z5ciRJmnzDhucbqurmwYZ2Q2xJkrY6w65te8aQbZIkLXlzzjyTPJFuQfidkrx04Kkd6a66lSRpqzPfYds9gRcCDwFeNNB+F/DGEdWkrU2yMPtxqSRJW8ic4VlVZwJnJjmkqr6+QDVJkjTRhr1g6KokR9Mdwv33w7VV9fqRVCVJ0gQb9oKhTwKPAJ4LXAT8Kt2hW0mStjrDhufjq+qPgB+1ReJfADxldGVJkjS5hg3Pe9p/70jyJGAn4OGjKUmSpMk27DnPk5PsDPwRcBbwYODdI6tKkqQJNuz9PE9pmxcB3oZMkrRVG+qwbZJdk/xVki+1x3snecNoS5MkaTINe85zJXAu8Kj2+FvA20ZQjyRJE2/Y8Nylqv4GWA9QVfcC942sKkmSJtiw4fmjJA8DCiDJfwLuHFlVkiRNsGGvtn073VW2uye5FFgOHD6yqiRJmmDz3VVlt6r6h6r6RpJn0C0UH+DGqrpnrtdKkrRUzXfY9vMD25+tqjVVdZ3BKUnams0XnoP3ivL7nZIkMX941ia2JUnaas13wdD+SX5INwP95bZNe1xVteNIq5MkaQLNdzPsbRaqEEmSFothv+c5Fkmmklw3S/spSfZu299NskvbvrvvWFugxhVJnjrweGUSv8YjSUvYsN/znChV9XvjrmHACuBu4GtjrkOStEAmeubZLEvyqSQ3JDkjyfZJLkwyvbkDJtkmyQeTrEpybZLfb+0r2thnJPlm22/ac89vbVcm+d9JvpBkCjgKODbJ1Ume1nbx9CRfS3LzXLPQtr+LkpzZ+h6X5NVJrkiyOsnurd/KJCcmuaz1W5Hk1PaZrJxj/DclmUkys3bt2s39uCRJG1kM4bkncEJV7QX8EHjLFhjzDcCdVXUQcBDwxiSPbc89mW7R+73pvp7z60m2Az4G/FZVHUi3whJV9V3gJOD4qjqgqi5pYzwSOBR4IXDcPLXsTxfAewGvAZ5QVQcDpwDHDPTbGTgEOJZutafjgX2AfZMcMNvAVXVyVU1X1fTy5cvn+0wkSUNaDOF5a1Vd2rZPowul++s5wGuTXA1cDjwM2KM9d0VV3VZV64GrgSngicDNVXVL63P6PON/vqrWV9X1wK7z9F1VVbdX1TrgO8CXW/vqtu8Nzq6qau0/qKrVrcY1G/WTJI3YYjjnufH3S7fE900DHFNV5/5CY7ICWDfQdB+b9xkNjpFN9vqPfdcPPF6/0b7XzdJntn6SpBFbDDPP3ZIc0raPBP5+C4x5LvDmJNsCJHlCkgfN0f9G4HHtHCfAEQPP3QXssAVqkiQtEoshPG8Ejk5yA915vxO3wJinANcD32hfX/kYc8zequondOdaz0lyJV1gbrgl29nAYRtdMCRJWsLSnUbTfJI8uKrublfffhS4qaqOH3ddw5qenq6ZmZlxlzG7zHdkewvx77qkHpJcWVWzfrNjMcw8J8Ub2wVGa4Cd6GarkqSt0JK70CTJvsAnN2peV1VPuT/jtlnmZs00R1WTJGk8llx4VtVq4IBx1zFoEmuSJG0+D9tKktST4SlJUk+GpyRJPRmekiT1ZHhKktST4SlJUk+GpyRJPRmekiT1ZHhKktST4SlJUk+GpyRJPS25tW21CHmrMEmLjDNPSZJ6MjwlSerJ8JQkqSfDU5KkngxPSZJ6MjwlSerJ8JQkqSfDU5KkngxPSZJ6coUhjV8y7gokLVUjWsHMmackST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZnpIk9bTowzPJVJLrNvO1K5J8YUvXJEla2hZ9eEqStNCWSnguS/KpJDckOSPJ9kmeleSqJKuTnJrkgQBJnpfkm0m+Aby0tT0gyU1Jlg88/vaGxxtLsjLJiUkuS3Jzm8Ge2va/cqDfiUlmkqxJ8scD7ccluT7JtUn+vLW9PMl1Sa5JcvGm3mh7b3/TXv93SS5PMr2Jvm9q+59Zu3Zt/09VkjSrpRKeewInVNVewA+BtwMrgSOqal9gGfDmJNsBfwm8CDgQeARAVa0HTgNe3cZ7NnBNVc2VODsDhwDHAmcBxwP7APsmOaD1eVdVTQP7Ac9Isl+ShwGHAftU1X7A+1rfdwPPrar9gRfPsd+3AP9WVXsDf9Tex6yq6uSqmq6q6eXLZ/09QJK0GZZKeN5aVZe27dOAZwG3VNW3WtvHgacDT2ztN1VVtb4bnAq8tm2/HvjrefZ5dhtjNfCDqlrdQngNMNX6vKLNcK+iC9a9gTuBnwJ/leSlwI9b30uBlUneCGwzx34PBT4DUFXXAdfOU6ckaQtbKuFZGz2+o/cAVbcCP0jyG8DBwJfmecm69t/1A9sbHi9L8ljgHcCz2gzzi8B2VXVvG/8M4IXAOW3/RwF/CDwauLLNUCVJE2iphOduSQ5p20cCM8BUkse3ttcAFwHfbO27t/ZXbTTOKXSz0c9V1X33s6YdgR8BdybZFfgtgCQPBnaqqv9Ld8h3/9a+e1VdXlXvBtbShehsLgVe0V6zN7Dv/axTktTTsnEXsIXcCByd5FTgeuCtwGXA55IsA1YBJ1XVuiRvAr6Y5MfAJcAOA+OcRXe4dr5DtvOqqmuSXEUX2LfShR5tf2e286+hOz8L8MEke7S284FrNjH0CcDHk1zfxl5DdyhYkrRA0p22E0C7avX4qnrauGvZlCTbANtW1U/bDPo8YM+q+tlcr5uenq6ZmZkFqbG3ZNwVSFqq7kfGJbmyXfT5HyyVmef9luSdwJv5+RW3k2p74IIk29LNUt8yX3BKkrYsw7OpquOA4wbbkrwLePlGXT9XVe8fdT1Jngt8YKPmW6rqMGDW34QkSQvD8JxDC8mRB+Um9n0ucO449i1JmttSudpWkqQFY3hKktST4SlJUk+GpyRJPRmekiT1ZHhKktST4SlJUk+GpyRJPRmekiT1ZHhKktST4SlJUk+ubavx87Z4khYZZ56SJPVkeEqS1JPhKUlST4anJEk9GZ6SJPVkeEqS1JPhKUlST4anJEk9GZ6SJPWUcnWXrUKStcD3gF2Afx5zORubxJpgMuuypuFNYl2TWBNMZl2TUNNjqmr5bE8YnluZJDNVNT3uOgZNYk0wmXVZ0/Amsa5JrAkms65JrGmQh20lSerJ8JQkqSfDc+tz8rgLmMUk1gSTWZc1DW8S65rEmmAy65rEmv6d5zwlSerJmackST0ZnpIk9WR4LnFJHprkK0luav/deRP9zklyR5IvjLCW5yW5Mcm3k7xzlucfmOSz7fnLk0yNqpYeNT09yTeS3Jvk8FHX06Outye5Psm1Sc5P8pgJqOmoJKuTXJ3k75PsPeqahqlroN/LklSSkX/9YYjP6nVJ1rbP6uokvzfumlqfV7S/V2uSfHrUNQ1TV5LjBz6nbyW5YyHqmldV+bOEf4A/A97Ztt8JfGAT/Z4FvAj4wojq2Ab4DvA44JeAa4C9N+rzFuCktv1K4LMj/myGqWkK2A/4BHD4Av2ZDVPXM4Ht2/abJ+Sz2nFg+8XAOZPwWbV+OwAXA5cB0+OuCXgd8JGF+PvUo6Y9gKuAndvjh09CXRv1PwY4daE+t7l+nHkufb8NfLxtfxx4yWydqup84K4R1nEw8O2qurmqfgZ8ptU2aLDWM4BnJck4a6qq71bVtcD6EdaxOXVdUFU/bg8vA351Amr64cDDBwELcTXiMH+vAN4LfAD46QTVtJCGqemNwEer6t8AquqfJqSuQa8CTl+AuuZleC59u1bV7W37/wG7jqmOXwFuHXh8W2ubtU9V3QvcCTxszDWNQ9+63gB8aaQVDVlTkqOTfIfuiMdbR1zTUHUl+TXg0VX1xQWoZ6iampe1w+5nJHn0BNT0BOAJSS5NclmS5424pmHrAqCdmngs8NUFqGtey8ZdgO6/JOcBj5jlqXcNPqiqSuJ3k5aQJL8DTAPPGHctAFX1UeCjSY4E/hD43XHWk+QBwP+iO0w6Sc4GTq+qdUl+n+6Iy2+MuaZldIduV9Adybg4yb5Vdcc4ixrwSuCMqrpv3IWA4bkkVNWzN/Vckh8keWRV3Z7kkcBCHIqZzT8Cg79d/2prm63PbUmWATsB/zLmmsZhqLqSPJvuF6RnVNW6SahpwGeAE0daUWe+unYAngRc2M4APAI4K8mLq2pmTDVRVYN/r0+hm6mP0jB/frcBl1fVPcAtSb5FF6arxlzXBq8Ejh5hLb142HbpO4uf//b/u8CZY6pjFbBHkscm+SW6/xHO2qjPYK2HA1+tdpXAGGsah3nrSvJk4GPAixfo3NQwNe0x8PAFwE3jrquq7qyqXapqqqqm6M4PjzI4560JoP0iu8GLgRtGWM9QNQGfp5t1kmQXusO4N09AXSR5IrAz8PUR1zO8cV+x5M9of+jOGZ5P9w/ZecBDW/s0cMpAv0uAtcBP6H4Dfe4Iank+8C26q+ve1dr+hO4fM4DtgM8B3wauAB63AJ/PfDUd1D6PH9HNgtcs0J/bfHWdB/wAuLr9nDUBNX0IWNPquQDYZxI+q436XsiIr7Yd8rP60/ZZXdM+qydOQE2hO8R9PbAaeOWk/PkB7wGOW4h6hv1xeT5JknrysK0kST0ZnpIk9WR4SpLUk+EpSVJPhqckST0ZntISleTuBd7fVFtZSFryDE9J91tbEWoKWBLh2d6PtEmGp7TEJVmR5KIkZya5OclxSV6d5Ip2/83dW7+VSU5KMtPum/jC1r5dkr9ufa9K8szW/rokZyX5Kt1CHMcBT2v3XTy2zUQvSXc/1G8keepAPRe2BdG/meRTG+6ek+SgJF9Lck2rb4ck2yT5YJJVbSH135/lPT4oyRfb665LcsQc4w31ftqYp7bXXZVk3HdG0QTxtytp67A/sBfwr3RLrp1SVQcn+QO6eyS+rfWbortN1O7ABUkeT7eeaFXVvm2ZtC8neULr/2vAflX1r0lWAO+oqg2huz3wm1X107Z03+l0K1sBPBnYB/g+cCnw60muAD4LHFFVq5LsSLfi1RuAO6vqoCQPBC5N8uWqumXg/T0P+H5VvaDte6e23Nts4/3BkO/nf9AtEfn6JA8BrkhyXlX9aDP/DLSEOPOUtg6rqur26haQ/w7w5da+mi4wN/ibqlpfVTfRhewTgUOB0wCq6pvA9+jWPQX4SlX96yb2uS3wl0lW0y27uPfAc1dU1W1VtZ5uOb8pYE/g9qpa1fb1w+puTfcc4LVJrgYup1tycnAd3Q3v4zeTfCDJ06rqzjnGG/b9PAd4Z9vvhXTLR+62ifeqrYwzT2nrMHjXlfUDj9fzi/8ObLxe53zrd841CzuWbv3d/el+UR+8EfVgPfcx979FAY6pqnM31aGqvpXuvp3PB96X5Hzg7+apfTaD7yfAy6rqxs0YR0ucM09Jg16e5AHtPOjjgBvpbhrwaoB2eHO31r6xu+huAbbBTnQzv/XAa4Bt5tn3jcAjkxzU9rVDu3DnXODNSbbdUEOSBw2+MMmjgB9X1WnAB+kOv25qvGHfz7nAMQPnY588T/3aijjzlDToH+juaLMjcFQ7X3kCcGI7/Hov8LrqbuK88WuvBe5Lcg2wEjgB+NskrwXOYe5ZKlX1s3ahz4eT/DLd+cln093vcgr4RguytcBLNnr5vsAHk6wH7gHePMd4w76f9wJ/AVyb7qbatwAvnOs9aOvhXVUkAd3VtsAXquqMcdciTToP20qS1JMzT0mSenLmKUlST4anJEk9GZ6SJPVkeEqS1JPhKUlST/8fxS/3hHk/fiAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from dianna.visualization import plot_tabular\n",
    "\n",
    "# get the scores for the target class\n",
    "explanation = explanation[np.argmax(predictions)]\n",
    "\n",
    "_ = plot_tabular(explanation, X_test.columns, num_features=10)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}