Starbucks_Capstone_Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Starbucks Capstone Challenge\n",
    "## 1. Definition\n",
    "### Introduction\n",
    "\n",
    "This data set contains simulated data that mimics customer behavior on the Starbucks rewards mobile app. Once every few days, Starbucks sends out an offer to users of the mobile app. An offer can be merely an advertisement for a drink or an actual offer such as a discount or BOGO (buy one get one free). Some users might not receive any offer during certain weeks. \n",
    "\n",
    "Not all users receive the same offer, and that is the challenge to solve with this data set.\n",
    "\n",
    "Your task is to combine transaction, demographic and offer data to determine which demographic groups respond best to which offer type. This data set is a simplified version of the real Starbucks app because the underlying simulator only has one product whereas Starbucks actually sells dozens of products.\n",
    "\n",
    "Every offer has a validity period before the offer expires. As an example, a BOGO offer might be valid for only 5 days. You'll see in the data set that informational offers have a validity period even though these ads are merely providing information about a product; for example, if an informational offer has 7 days of validity, you can assume the customer is feeling the influence of the offer for 7 days after receiving the advertisement.\n",
    "\n",
    "You'll be given transactional data showing user purchases made on the app including the timestamp of purchase and the amount of money spent on a purchase. This transactional data also has a record for each offer that a user receives as well as a record for when a user actually views the offer. There are also records for when a user completes an offer. \n",
    "\n",
    "Keep in mind as well that someone using the app might make a purchase through the app without having received an offer or seen an offer.\n",
    "\n",
    "### Example\n",
    "\n",
    "To give an example, a user could receive a discount offer buy 10 dollars get 2 off on Monday. The offer is valid for 10 days from receipt. If the customer accumulates at least 10 dollars in purchases during the validity period, the customer completes the offer.\n",
    "\n",
    "However, there are a few things to watch out for in this data set. Customers do not opt into the offers that they receive; in other words, a user can receive an offer, never actually view the offer, and still complete the offer. For example, a user might receive the \"buy 10 dollars get 2 dollars off offer\", but the user never opens the offer during the 10 day validity period. The customer spends 15 dollars during those ten days. There will be an offer completion record in the data set; however, the customer was not influenced by the offer because the customer never viewed the offer.\n",
    "\n",
    "### Cleaning\n",
    "\n",
    "This makes data cleaning especially important and tricky.\n",
    "\n",
    "You'll also want to take into account that some demographic groups will make purchases even if they don't receive an offer. From a business perspective, if a customer is going to make a 10 dollar purchase without an offer anyway, you wouldn't want to send a buy 10 dollars get 2 dollars off offer. You'll want to try to assess what a certain demographic group will buy when not receiving any offers.\n",
    "\n",
    "### Final Advice\n",
    "\n",
    "Because this is a capstone project, you are free to analyze the data any way you see fit. For example, you could build a machine learning model that predicts how much someone will spend based on demographics and offer type. Or you could build a model that predicts whether or not someone will respond to an offer. Or, you don't need to build a machine learning model at all. You could develop a set of heuristics that determine what offer you should send to each customer (i.e., 75 percent of women customers who were 35 years old responded to offer A vs 40 percent from the same demographic to offer B, so send offer A)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Sets\n",
    "\n",
    "The data is contained in three files:\n",
    "\n",
    "* portfolio.json - containing offer ids and meta data about each offer (duration, type, etc.)\n",
    "* profile.json - demographic data for each customer\n",
    "* transcript.json - records for transactions, offers received, offers viewed, and offers completed\n",
    "\n",
    "Here is the schema and explanation of each variable in the files:\n",
    "\n",
    "**portfolio.json**\n",
    "* id (string) - offer id\n",
    "* offer_type (string) - type of offer ie BOGO, discount, informational\n",
    "* difficulty (int) - minimum required spend to complete an offer\n",
    "* reward (int) - reward given for completing an offer\n",
    "* duration (int) - time for offer to be open, in days\n",
    "* channels (list of strings)\n",
    "\n",
    "**profile.json**\n",
    "* age (int) - age of the customer \n",
    "* became_member_on (int) - date when customer created an app account\n",
    "* gender (str) - gender of the customer (note some entries contain 'O' for other rather than M or F)\n",
    "* id (str) - customer id\n",
    "* income (float) - customer's income\n",
    "\n",
    "**transcript.json**\n",
    "* event (str) - record description (ie transaction, offer received, offer viewed, etc.)\n",
    "* person (str) - customer id\n",
    "* time (int) - time in hours since start of test. The data begins at time t=0\n",
    "* value - (dict of strings) - either an offer id or transaction amount depending on the record\n",
    "\n",
    "**Note:** If you are using the workspace, you will need to go to the terminal and run the command `conda update pandas` before reading in the files. This is because the version of pandas in the workspace cannot read in the transcript.json file correctly, but the newest version of pandas can. You can access the termnal from the orange icon in the top left of this notebook.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem Statement\n",
    "\n",
    "The problem is simple: we want to make better purchasing offers to Starbucks' customers. For this, we can use customer's past behaviour to find patterns and try to be more assertive. As given by the Udacity's Starbucks Project Overview, the basic task is to use the data to identify which groups of people are most responsive to each type of offer, and how best to present each type of offer. In other words, this is a classification problem where the model takes user behaviour data as input and produces a group as output (either previously defined or not).\n",
    "\n",
    "This has been one of the [most used](https://towardsdatascience.com/how-to-predict-the-success-of-your-marketing-campaign-579fbb153a97) applications of machine learning in the industry, since it provides you with means to save money spent on marketing campaigns by directing content to users who are more likely to convert based on a multitude of characteristics. \n",
    "\n",
    "### Metrics\n",
    "To evaluate the trained models, we'll compare the models based on it's F1-Score. This is a widely used metric to evaluate classification problems. [Aditya Mishra defines](https://towardsdatascience.com/metrics-to-evaluate-your-machine-learning-algorithm-f10ba6e38234) it as follows:\n",
    "> F1 Score is the Harmonic Mean between precision and recall. The range for F1 Score is [0, 1]. It tells you how precise your classifier is (how many instances it classifies correctly), as well as how robust it is (it does not miss a significant number of instances).\n",
    "\n",
    "Since the F1-Score is [a weighted average of the precision and\n",
    "recall metrics](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html), it works well to evaluate datasets that aren’t very well balanced."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import math\n",
    "import json\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn import metrics\n",
    "from sklearn.preprocessing import MultiLabelBinarizer, LabelEncoder\n",
    "from sklearn.model_selection import train_test_split, cross_val_score\n",
    "from sklearn.dummy import DummyClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn import svm\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from datetime import datetime\n",
    "\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# read in the json files\n",
    "portfolio = pd.read_json('data/portfolio.json', orient = 'records', lines = True)\n",
    "profile = pd.read_json('data/profile.json', orient = 'records', lines = True)\n",
    "transcript = pd.read_json('data/transcript.json', orient = 'records', lines = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `portfolio` dataset preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10, 6)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "portfolio.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "channels      object\n",
       "difficulty     int64\n",
       "duration       int64\n",
       "id            object\n",
       "offer_type    object\n",
       "reward         int64\n",
       "dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "portfolio.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>channels</th>\n",
       "      <th>difficulty</th>\n",
       "      <th>duration</th>\n",
       "      <th>id</th>\n",
       "      <th>offer_type</th>\n",
       "      <th>reward</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[email, mobile, social]</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>ae264e3637204a6fb9bb56bc8210ddfd</td>\n",
       "      <td>bogo</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[web, email, mobile, social]</td>\n",
       "      <td>10</td>\n",
       "      <td>5</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>bogo</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[web, email, mobile]</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>informational</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[web, email, mobile]</td>\n",
       "      <td>5</td>\n",
       "      <td>7</td>\n",
       "      <td>9b98b8c7a33c4b65b9aebfe6a799e6d9</td>\n",
       "      <td>bogo</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[web, email]</td>\n",
       "      <td>20</td>\n",
       "      <td>10</td>\n",
       "      <td>0b1e1539f2cc45b7b9fa7c272da2e1d7</td>\n",
       "      <td>discount</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>[web, email, mobile, social]</td>\n",
       "      <td>7</td>\n",
       "      <td>7</td>\n",
       "      <td>2298d6c36e964ae4a3e7e9706d1fb8c2</td>\n",
       "      <td>discount</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>[web, email, mobile, social]</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>discount</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>[email, mobile, social]</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>informational</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>[web, email, mobile, social]</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>bogo</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>[web, email, mobile]</td>\n",
       "      <td>10</td>\n",
       "      <td>7</td>\n",
       "      <td>2906b810c7d4411798c6938adc9daaa5</td>\n",
       "      <td>discount</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                       channels  difficulty  duration  \\\n",
       "0       [email, mobile, social]          10         7   \n",
       "1  [web, email, mobile, social]          10         5   \n",
       "2          [web, email, mobile]           0         4   \n",
       "3          [web, email, mobile]           5         7   \n",
       "4                  [web, email]          20        10   \n",
       "5  [web, email, mobile, social]           7         7   \n",
       "6  [web, email, mobile, social]          10        10   \n",
       "7       [email, mobile, social]           0         3   \n",
       "8  [web, email, mobile, social]           5         5   \n",
       "9          [web, email, mobile]          10         7   \n",
       "\n",
       "                                 id     offer_type  reward  \n",
       "0  ae264e3637204a6fb9bb56bc8210ddfd           bogo      10  \n",
       "1  4d5c57ea9a6940dd891ad53e9dbe8da0           bogo      10  \n",
       "2  3f207df678b143eea3cee63160fa8bed  informational       0  \n",
       "3  9b98b8c7a33c4b65b9aebfe6a799e6d9           bogo       5  \n",
       "4  0b1e1539f2cc45b7b9fa7c272da2e1d7       discount       5  \n",
       "5  2298d6c36e964ae4a3e7e9706d1fb8c2       discount       3  \n",
       "6  fafdcd668e3743c1bb461111dcafc2a4       discount       2  \n",
       "7  5a8bc65990b245e5a138643cd4eb9837  informational       0  \n",
       "8  f19421c1d4aa40978ebb69ca19b0e20d           bogo       5  \n",
       "9  2906b810c7d4411798c6938adc9daaa5       discount       2  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "portfolio.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see some variables that could use some preprocessing, such as `channels` which is a list of values. We'll use [Scikit-learn's MultiLabelBinarizer](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.MultiLabelBinarizer.html) to transform it.\n",
    "\n",
    "Then we'll create a copy of the original `portfolio` dataset with the new encoded columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "mlb = MultiLabelBinarizer()\n",
    "encoded_channels = mlb.fit_transform(portfolio['channels'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "encoded_channels_df = pd.DataFrame(encoded_channels,\n",
    "                                   columns = mlb.classes_,\n",
    "                                   index = portfolio['channels'].index)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "processed_portfolio = portfolio.copy()\n",
    "\n",
    "processed_portfolio = processed_portfolio.join(encoded_channels_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   difficulty  duration                                id     offer_type  \\\n",
      "0          10         7  ae264e3637204a6fb9bb56bc8210ddfd           bogo   \n",
      "1          10         5  4d5c57ea9a6940dd891ad53e9dbe8da0           bogo   \n",
      "2           0         4  3f207df678b143eea3cee63160fa8bed  informational   \n",
      "3           5         7  9b98b8c7a33c4b65b9aebfe6a799e6d9           bogo   \n",
      "4          20        10  0b1e1539f2cc45b7b9fa7c272da2e1d7       discount   \n",
      "5           7         7  2298d6c36e964ae4a3e7e9706d1fb8c2       discount   \n",
      "6          10        10  fafdcd668e3743c1bb461111dcafc2a4       discount   \n",
      "7           0         3  5a8bc65990b245e5a138643cd4eb9837  informational   \n",
      "8           5         5  f19421c1d4aa40978ebb69ca19b0e20d           bogo   \n",
      "9          10         7  2906b810c7d4411798c6938adc9daaa5       discount   \n",
      "\n",
      "   reward  email  mobile  social  web  \n",
      "0      10      1       1       1    0  \n",
      "1      10      1       1       1    1  \n",
      "2       0      1       1       0    1  \n",
      "3       5      1       1       0    1  \n",
      "4       5      1       0       0    1  \n",
      "5       3      1       1       1    1  \n",
      "6       2      1       1       1    1  \n",
      "7       0      1       1       1    0  \n",
      "8       5      1       1       1    1  \n",
      "9       2      1       1       0    1  \n"
     ]
    }
   ],
   "source": [
    "processed_portfolio = processed_portfolio.drop('channels', axis = 1)\n",
    "print(processed_portfolio)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "difficulty    0\n",
       "duration      0\n",
       "id            0\n",
       "offer_type    0\n",
       "reward        0\n",
       "email         0\n",
       "mobile        0\n",
       "social        0\n",
       "web           0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "processed_portfolio.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "No empty cells, looks like this dataset is good enough for now.\n",
    "\n",
    "### `profile` dataset preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17000, 5)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "profile.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age                   int64\n",
       "became_member_on      int64\n",
       "gender               object\n",
       "id                   object\n",
       "income              float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "profile.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>became_member_on</th>\n",
       "      <th>gender</th>\n",
       "      <th>id</th>\n",
       "      <th>income</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>118</td>\n",
       "      <td>20170212</td>\n",
       "      <td>None</td>\n",
       "      <td>68be06ca386d4c31939f3a4f0e3dd783</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>55</td>\n",
       "      <td>20170715</td>\n",
       "      <td>F</td>\n",
       "      <td>0610b486422d4921ae7d2bf64640c50b</td>\n",
       "      <td>112000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>118</td>\n",
       "      <td>20180712</td>\n",
       "      <td>None</td>\n",
       "      <td>38fe809add3b4fcf9315a9694bb96ff5</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>75</td>\n",
       "      <td>20170509</td>\n",
       "      <td>F</td>\n",
       "      <td>78afa995795e4d85b5d9ceeca43f5fef</td>\n",
       "      <td>100000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>118</td>\n",
       "      <td>20170804</td>\n",
       "      <td>None</td>\n",
       "      <td>a03223e636434f42ac4c3df47e8bac43</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>68</td>\n",
       "      <td>20180426</td>\n",
       "      <td>M</td>\n",
       "      <td>e2127556f4f64592b11af22de27a7932</td>\n",
       "      <td>70000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>118</td>\n",
       "      <td>20170925</td>\n",
       "      <td>None</td>\n",
       "      <td>8ec6ce2a7e7949b1bf142def7d0e0586</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>118</td>\n",
       "      <td>20171002</td>\n",
       "      <td>None</td>\n",
       "      <td>68617ca6246f4fbc85e91a2a49552598</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>65</td>\n",
       "      <td>20180209</td>\n",
       "      <td>M</td>\n",
       "      <td>389bc3fa690240e798340f5a15918d5c</td>\n",
       "      <td>53000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>118</td>\n",
       "      <td>20161122</td>\n",
       "      <td>None</td>\n",
       "      <td>8974fc5686fe429db53ddde067b88302</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age  became_member_on gender                                id    income\n",
       "0  118          20170212   None  68be06ca386d4c31939f3a4f0e3dd783       NaN\n",
       "1   55          20170715      F  0610b486422d4921ae7d2bf64640c50b  112000.0\n",
       "2  118          20180712   None  38fe809add3b4fcf9315a9694bb96ff5       NaN\n",
       "3   75          20170509      F  78afa995795e4d85b5d9ceeca43f5fef  100000.0\n",
       "4  118          20170804   None  a03223e636434f42ac4c3df47e8bac43       NaN\n",
       "5   68          20180426      M  e2127556f4f64592b11af22de27a7932   70000.0\n",
       "6  118          20170925   None  8ec6ce2a7e7949b1bf142def7d0e0586       NaN\n",
       "7  118          20171002   None  68617ca6246f4fbc85e91a2a49552598       NaN\n",
       "8   65          20180209      M  389bc3fa690240e798340f5a15918d5c   53000.0\n",
       "9  118          20161122   None  8974fc5686fe429db53ddde067b88302       NaN"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "profile.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `profile` dataset has some interesting characteristics, such as:\n",
    "* `NaN` values for the `income` column.\n",
    "* `None` value for the `gender`column (which is ok, since there are more genders than M, F, but this might need to be treated since we have `O` value - that probably means 'others').\n",
    "* `became_member_on` has dates that are treated as numbers.\n",
    "* `age` has the value `118` repeated several times. Since it's fairly unlikely for the customers to be 118 years-old, I'm assuming this is also some kind of previous data preprocessing and won't change it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([None, 'F', 'M', 'O'], dtype=object)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "profile['gender'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age                    0\n",
       "became_member_on       0\n",
       "gender              2175\n",
       "id                     0\n",
       "income              2175\n",
       "dtype: int64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "profile.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `None` value for `gender` is not understood as a value, so it needs treatment. We'll fill the `None` values with `Unknown`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "processed_profile = profile.copy()\n",
    "\n",
    "processed_profile['gender'].fillna('Unknown', inplace = True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the `income` column, let's see what's the data distribution without the `NaN`s:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxoAAAFNCAYAAACHVBk0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd3zedb3//8cr48reoxlNulJKJy2dTBnKEgSUPUQEcaCeczzn6/h9PeoR8YB+j6hHUIqggDIqKiCCDCkUKG3pgO6RpknaJm3S7D3fvz+uTyGEpE3bq/lkPO+3W25c12e8P6/rMqbX83qPjznnEBERERERCaUwvwsQEREREZGRR0FDRERERERCTkFDRERERERCTkFDRERERERCTkFDRERERERCTkFDRERERERCTkFDRGQUMLNGM5vodx3DiZldb2YvHWL/WWa2ZzBrEhEZThQ0RGTUM7NiM/u433UcLTNzZlbQa9sPzOwPB5875+Kdc0WHaWdEf3A2s21mdlWP56d5713vbY1mFuGc+6Nz7rwe+z7yPh/HWl8zs1sPsX+8V8/fe23/g5n94Aiu0WpmeT22fdzMio+2bhGRnhQ0RERkUJhZuM8lLAM+1uP5mcDWPrYtd851DmZhx2CRmZ12DOc3Af8ZqmJERHpS0BAR6cHMPmdmb5rZ/zOzGjPbZWYX9tifama/M7Myb//TPfZ9wcwKzazazJ41s5we+5yZfcXMdphZg5ndYWaTzOxtM6s3syVmFuhx/MVm9q6Z1ZrZcjObdYyv6/1v483sIjPb7NWx18z+w8zigBeAHO8b/UYzyzGzKDP7ufd6y7zHUT3a/aaZlXv7bu11nd+b2a/N7HkzawLONrNPmtk67zXv7vnte49v6W/29tWY2ZfMbL6Zrffei18dw9uwjGCQOOgM4O4+ti3z6vmcmb3pPV7m7X/Pe2+u7lH3v5tZhfc+3Nxje5KZPWJmlWZWYmbfNbMwb9+Hepx6vPYIM7vTq+NX3rUO9Zp/Avyorx1mlmJmz3nXr/Eej+112C+Ba/vrqTGzb3m/Iw0W7BE69xC1iIh8iIKGiMhHLQS2AekEP8g9aGbm7XsUiAWmA5nAPQBmdg7w38BVQDZQAjzRq90LgLnAIuCbwGLgeiAPmAFc67V1MvAQ8EUgDbgfeLbnB/xj9CDwRedcgnfdV51zTcCFQJk3zCreOVcG/F+v3tnAScAC4LtenRcA3wA+DhTw4Z6Bg64D7gQSgDcJfoP+WSAZ+CTwZTO7rNc5C4HJwNXAz70aPk7wPb/KzPq6zkC8Dkz3wmIYMA94Ekjuse1UvKDRk3PuYBg5yXtvnvSeZwFJQC5wC3CvmaV4+/7X2zeR4HvzWeBmDsM593+BN4Cvetf66iEOvxc4wfoe+hcG/A4YB+QDLUDv0LIXeAD4Qe+TzWwK8FVgvve7cj5QfLj6RUQOUtAQEfmoEufcA865LuBhgsFhjJllE/ww/iXnXI1zrsM597p3zvXAQ865tc65NuA7wClmNr5Hu3c75+qdc5uAjcBLzrki51wdwd6EOd5xXwDud86tdM51OeceBtoIfuDvz1rvG/9aM6sFvn2IYzuAaWaW6L2OtYc49nrgh865CudcJfBfwI3evquA3znnNjnnmr19vT3jnHvLOdftnGt1zr3mnNvgPV8PPM5HA8od3rEvEQwmj3vX30vwA/gcjoJzrhQoJdhbcBKwwznXArzVY1s0sPIImu0g+P50OOeeBxqBKRYcJnY18B3nXINzrhj4Hz5470KllWCQ+0ivhnOuyjn3Z+dcs3OuwTuur5D238AlZja91/YuIIrg70qkc67YObczxPWLyAimoCEi8lH7Dj7wPkADxBPseah2ztX0cU4OwV6Mg+c1AlUEv+k+aH+Pxy19PI/3Ho8D/r1XcMjzrtGfk51zyQd/gLsOcexngIuAEjN73cxOOcSxH3pd3uOcHvt299jX83Gf28xsoZkt9Ybz1AFfIthz1NNA36cP6THkq9HM8vt5PQeHT51JMLRAsKfl4LaVXlAcqKpe8zmavfrSgQAffe96/j6EygMEg/AlPTeaWayZ3e8N26on+NqTrddcGS9A/gr4Ya/thcC/EuztqDCzJ6zHcEARkcNR0BARGbjdQKqZJfexr4xgQADAm/OQRnBoytFc586ewcE5F+uce/yoqu7FOfeOc+5SgkO/ngaWHNzVx+Efel0Eh+CUeY/LgZ5j/vP4qN5tPgY8C+Q555KA3wD2kbOOQo8hX/Fe70VfDgaNM/ggaLzRY9tHhk0dpQMEezt6v3cHfx+aCA7BOyir1/l9/W/RJ+dcB8HepDv48Hv578AUYKFzLpEP5qL09X7/FDib4NC+nm0/5pw73XsdjuCcFhGRAVHQEBEZIOdcOcEhTvd5E20jzezgh7fHgJvNbLY3l+LHBL8dLz6KSz0AfMn79t/MLM6bRJ1wrK/BzAIWvD9EkvcBtZ7gEBkI9hykmVlSj1MeB75rZhlmlg58Dzg4iXkJwdc81cxivX2Hk0CwV6jVzBYQnMMxmJYRHHr1MYJDpgA2ABMIftA+VNDYT3C+xWF5w+6WAHeaWYKZjSM4n+Xge/cucKaZ5Xvv93eO9lqeRwkOc7qgx7YEgj1AtWaWCnz/EPXWEhza9c2D28xsipmd4/0+t3ptdfXThIjIRyhoiIgcmRsJflO9FaggOLQE59w/CS4T+meC3/RPAq45mgs451YTnKfxK6AGKAQ+d4x193QjUOwNp/kScIN33a0Eg0WRN2Qrh+DY/9XAeoIfyNd623DOvUBw1aKlXo1ve+0faujRV4AfmlkDwWCy5BDHhpxzbjvB/93KvQ/XOOe6gVVAIrD8EKf/AHjYe2+uOsRxB32NYM9FEcHhWY8RnOSPc+5lghPR1wNrgOd6nfsL4ApvtahfDuB1dREMEqk9Nv8ciCHYu7IC+MdhmvkFHw4SUQSH4B0gOJwwE/j/DleLiMhB5tyAe2dFRET6ZWZTCU5yjxpG96EQEZHjRD0aIiJy1Mzscm84VgrB8ft/U8gQERFQ0BARkWPzRaAS2Elw2M2X/S1HRESGCg2dEhERERGRkFOPhoiIiIiIhJyChoiIiIiIhFyE3wX4IT093Y0fP97vMkREREREhrU1a9YccM5l9LVvVAaN8ePHs3r1ar/LEBEREREZ1syspL99GjolIiIiIiIhp6AhIiIiIiIhp6AhIiIiIiIhp6AhIiIiIiIhp6AhIiIiIiIhp6AhIiIiIiIhp6AhIiIiIiIhp6AhIiIiIiIhp6AhIiIiIiIhp6AhIiIiIiIhp6AhIiIiIiIhF+F3ASIiMrQ9trL0qM+9bmF+CCsREZHhRD0aIiIiIiIScgoaIiIiIiIScgoaIiIiIiIScgoaIiIiIiIScgoaIiIiIiIScgoaIiIiIiIScgoaIiIiIiIScgoaIiIiIiIScgoaIiIiIiIScgoaIiIiIiISchF+FyAicrQeW1l61OdetzA/hJWIiIhIb+rREBERERGRkFPQEBERERGRkFPQEBERERGRkFPQEBERERGRkFPQEBERERGRkFPQEBERERGRkNPytiJyzLTMrIiIiPSmHg0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQm5AQUNM7vAzLaZWaGZfbuP/VFm9qS3f6WZje+x7zve9m1mdv7h2jSzCV4bO7w2A972b5jZZjNbb2b/NLNxPc65yTt+h5nddHRvhYiIiIiIhMphg4aZhQP3AhcC04BrzWxar8NuAWqccwXAPcDd3rnTgGuA6cAFwH1mFn6YNu8G7nHOTQZqvLYB1gHznHOzgKeAn3jXSAW+DywEFgDfN7OUI30jREREREQkdAbSo7EAKHTOFTnn2oEngEt7HXMp8LD3+CngXDMzb/sTzrk259wuoNBrr882vXPO8drAa/MyAOfcUudcs7d9BTDWe3w+8LJzrto5VwO8TDDUiIiIiIiITwYSNHKB3T2e7/G29XmMc64TqAPSDnFuf9vTgFqvjf6uBcFejheOoD4RERERERlEEQM4xvrY5gZ4TH/b+wo4hzr+gwuZ3QDMAz52BPVhZrcBtwHk5+f3cYqIiIiIiITKQILGHiCvx/OxQFk/x+wxswggCag+zLl9bT8AJJtZhNer8aFrmdnHgf8LfMw519bj2mf1auu13i/CObcYWAwwb968jwQREZGBemxl6VGfe91CfdEhIiKjw0CGTr0DTPZWgwoQnNz9bK9jngUOrvZ0BfCqc85526/xVqWaAEwGVvXXpnfOUq8NvDafATCzOcD9wKeccxU9rv0icJ6ZpXiTwM/ztomIiIiIiE8O26PhnOs0s68S/PAeDjzknNtkZj8EVjvnngUeBB41s0KCPRnXeOduMrMlwGagE7jdOdcF0Feb3iW/BTxhZj8iuNLUg972nwLxwJ+Cc8Ypdc59yjlXbWZ3EAwvAD90zlUfw3siIiIiIiLHaCBDp3DOPQ8832vb93o8bgWu7OfcO4E7B9Kmt72I4KpUvbd//BD1PQQ81P8rEBERERGRwaQ7g4uIiIiISMgpaIiIiIiISMgpaIiIiIiISMgpaIiIiIiISMgpaIiIiIiISMgpaIiIiIiISMgpaIiIiIiISMgN6D4aIiLiv8dWlh71udctzA9hJSIiIoenHg0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQk5BQ0REREREQm5CL8LEBE5Et3djsrGNqqb2imsaKSpvZPmtk46uhxJsZGkxgZIjo0kPioCM/O7XBERkVFLQUNEhryubsc7xdW8sKGcFzbuo6Kh7bDnRIQZqXEBJqTHMWVMAhMz4glEqBNXRERksChoiMiQtba0hqfW7OGlTfs40NhOVEQYZ0/J5NSCNNLjo1hbWkNcIIK4qAgiwozalg5qm9qpaW6nprmDyoY21pbWsHJXNRFhxvj0OE4Yk8DM3CS/X5qIiMiIp6AhIkNORX0rP35+C0+/W0ZsIJyzT8zkohnZnDUlg7ioD/5s1TZ3fOi8rMhwshKjP7Sts6ub4qpmtu9vYNv+Bp7fUM4LG8pZXVzNtQvyOWtKBhHh6ukQEREJNQUNERkyOrq6eXh5MT9/ZQftnd189ewCvnL2JGIDR/+nKiI8jILMeAoy47loZjZVjW2sKalh/d46/vnIasYkRnHVvDyuXZBPTnJMCF+NiIjI6KagISJDwsqiKv7zmY1s39/IWVMy+P4l05mQHhfy66TFR3He9CweuGker26t4PFVpfxqaSG/fm0nn5qdwxfPnMSUrISQX1dERGS0UdAQEV8553jorWJ+9PfN5CbHsPjGuXxi2pjjvmJUZHgY50/P4vzpWeyubuaht3bxxKrd/GXtXs45MZMvnjmRBRNStXKVp6W9i9qWdupbOqhr6aSupYOG1g5S4gJMyognNzmG8DC9VyIi8gEFDRHxTVe34wfPbuLht0u4YHoW91w9m5hA+KDXkZcay/cvmc7Xz5nMoytK+P3yYq5evII5+cncflYB507NHJWBo6G1g39s3Mdv3yxiV2UTrsc+A2KjImhq6+Rl9hMVEcaE9DgmZcQzPSeR5NiAX2WLiMgQoaAhIr5o6+ziyXd2s3VfA184YwLfuXAqYT5/I54SF+Dr507mC2dM5E9rdnP/60Xc+shqTsxK4PazC7hoZvaI/9a+q9vx2rYK/rpuLy9v3k9bZzepcQHOmpJJdlI0iTGRJMUE71MSHmY0tnVSVNnIzsomdlY2snVfAy9v2c/FM7OZOy7F75cjIiI+UtAQkUFX39LBI28XU17Xyh2XzeDGReP8LulDYgLhfPaU8Vy7IJ9n3y3jvtcK+drj6/jZy9v58scm0dndTUTYyFqpqr2zm6fX7eXXr+9k14EmUmIjuXp+HpfNyWVLWX2/PTrxURHMGpvMrLHJABxoaOOv7+7lL+v2snVfAxfMyCItPmowX4qIiAwRChoiMqiqGtv47Zu7aGnv4rOnjBtyIaOnyPAwPjN3LJfPyeXFTfv41dJCvvnn9STFRHLG5HTmjUsd9jcBbO3oYsnqYO/N3toWZuQmct/1J/OJaWOI9Jb93VreMOD20hOiuOX0CbxVeICXNu/n/J+/wU+umMk5J445Xi9BRESGKAUNERk09a0d/G55MR1d3dx25sRhs5xsWJhx4cxsLpiRxevbK/nBs5t4bn05S7dWcHpBOgsnphEdOfhzS45FU1snf1xZwgNv7KKyoY2541L40eUzOOuEjGOejxJmxhmTMyjIjOflzfv5/O9Xc/Np4/nexdNG5VwXEZHRSkFDRAZFS3sXv3+rmMbWTm45fcKwCRk9mRlnTcnktjNb2XWgide3V/Di5v28vqOSufkpLJqYNuSHCdW1dPDw8mIeemsXtc0dnF6Qzi+vmcOiiaFfYSs7KYanbz+Nu17Yyu/eKiY2EM7/Of/EkF5DRESGLgUNETnuOrq6eXRFMZUNbXz21HHkpcb6XdIxm5Aex4T0CeytbeGNHZW8XVTFWzurOGFMPIsmpHFCVgJhQ+jb+wONbTz05i4eebuExrZOPj41k9vPLmBO/vGdsB0dGc73L5lGW2cX9y7dSW5yLNctzD+u1xQRkaFBQUNEjquubscTq0opqWrm6vl5TM4cWTfDy02O4Zr5+dTP7OCd4mpW7armkRUlpMRGMic/hVm5SWQmRvtSm3OOogNNrNpVzfef3Uhnt+OimdncflYB03ISB60OM+OOS2ewr66V7z69gaykKM3ZEBEZBRQ0ZMR7bGXpUZ+rb16PjXOOv67by5Z9DXzqpJz3VyYaiRKjIzn3xDGcdUImm8vrWVlUxdKtFby6tYKsxGhmjU0atNff1NbJ2tIa3imu5kBjO9GRYdywaBw3LBrHpIz4Qamht4jwMH513clcs3gFt/9xHU9+cdGI/n0QEREFDRE5jv65tYK1pTWce2Imiyam+V3OoAgPM2bmJjEzN4n6lg42ltWxfk8dL23ez0ub9/PMu3tZODGVhRPSWDghNSS9HV3djj01zeyoaGTH/gb21LTggPzUWK6Ym8nM3CRuOnX8MV/nWMVFRfDg5+bx6fuW8/nfv8Nfv3LaiBhGJyIifVPQEJHjYnNZHa9ureDk/GTOOTHT73J8kRgTyamT0jl1Ujo1ze1s3FtHc3sXf127lz+sCPa0TUiPY9bYJPJTY8lLjSUvJZb8tFgyE6IIM8MAs+Dwo87ubmqbO6huaqe6qZ2apnYqG9vYdaCJts5uDBibEsPZJ2YyIyeJrCR/hmwdSmZCNL+/eQGf+fVybvrdKp796unER+mfIhGRkUh/3UUk5PbXt7JkzR7GpsRw6excLWkKpMQGOGNyBtctzKezq9sbXlXNyl1VrCmp4W/vldHtjqzNiDAjJS7ArLFJFGQmUJART0xg6C+zW5AZz/03zuXaB1Zw1wtb+NFlM/0uSUREjgMFDREJqZb2Lv6wooTI8DCuXzju/Zu+yQciwsPev5v2F86cCARX5iqvbaW0upndNc1UNrThHDic91/YVFZHckyA1LjgT0J0xJBa2aovh5ojddqkdP6wopSoiPA+545ojpSIyPCmoCEiIdPtHE+uLqWmuZ1bT59IUkyk3yUNG5HhYeSnBYdN9edYFjYYij4+dQxbyuv5y9o9fP3cyURFDP3eGBERGTh91SgiIfPK5v1s39/IJSflMD49zu9yZIgLRITx6ZPHUtvcwYub9vtdjoiIhJiChoiExMa9dby2vZL541NYMD7V73JkmJiQHseiSWmsKKqi6ECj3+WIiEgIaeiUyHFytMNchuO49MqGNp5au4e8lBgumZWjyd9yRM6flsW2fQ38Ze1evn7OZAIR+g5MRGQk0F9zETkmze2d/HFlCRFhxrUL8onQ5G85QsEhVLlUN7Xz8uZ9fpcjIiIhok8EInLUnHN8968bqWxo4+r5eSTHBvwuSYapienxLJqYyvKdVZRWN/tdjoiIhICChogctcdWlfKXdXs5Z2omkzMT/C5Hhrnzp2cRHx3B39eX0e2O8KYiIiIy5AxojoaZXQD8AggHfuucu6vX/ijgEWAuUAVc7Zwr9vZ9B7gF6AK+7px78VBtmtkE4AkgFVgL3OicazezM4GfA7OAa5xzT/W4fhewwXta6pz71BG+DyIjwrEsf3qkc0PW76nlv57dzMdOyODsKaPzzt8SWlER4Zw/LYun1u5h/Z5aYJzfJYmIyDE4bI+GmYUD9wIXAtOAa81sWq/DbgFqnHMFwD3A3d6504BrgOnABcB9ZhZ+mDbvBu5xzk0Gary2AUqBzwGP9VFmi3NutvejkCFynNU2t/PlP6wlPT7Az6+ePeRvGifDx+z8ZHKTY3hx035a2rv8LkdERI7BQIZOLQAKnXNFzrl2gr0Nl/Y65lLgYe/xU8C5Flx25lLgCedcm3NuF1Dotddnm94553ht4LV5GYBzrtg5tx7oPsrXKiIh0N3t+Lcn36WioZX7bphLSpzmZUjohJlx0cxs6lo6WLysyO9yRETkGAxk6FQusLvH8z3Awv6Occ51mlkdkOZtX9Hr3FzvcV9tpgG1zrnOPo4/lGgzWw10Anc5554ewDkichR+/sp2lm6r5I7LZjA7L9nvcmQEmpAex4ycRH61dAeBiLCjusP8cFwmWkRkpBlI0OhrTETvWXr9HdPf9r56Ug51/OHkO+fKzGwi8KqZbXDO7fxQgWa3AbcB5OfrHyCRo/Hipn388tVCrpw7lhtC9EHuWOaVyMh1wYxstuxr4KVN+7hyXp7f5YiIyFEYyNCpPUDPv/JjgbL+jjGzCCAJqD7Euf1tPwAke230d62PcM6Vef8tAl4D5vRxzGLn3Dzn3LyMjIzDNSkivRRWNPLvS95j1tgk7rhshm7KJ8dValyA0yals253LXtqtNytiMhwNJCg8Q4w2cwmmFmA4OTuZ3sd8yxwk/f4CuBV55zztl9jZlHealKTgVX9temds9RrA6/NZw5VnJmleKteYWbpwGnA5gG8LhEZoIbWDm57dDVREWH85oa5REeG+12SjAJnTckgLiqCv68vx2m5WxGRYeewQcObL/FV4EVgC7DEObfJzH5oZgdXeHoQSDOzQuAbwLe9czcBSwh+8P8HcLtzrqu/Nr22vgV8w2srzWsbM5tvZnuAK4H7zezg8VOB1Wb2HsGQcpdzTkFDJES6ux3fWPIeJVXN3Hv9yeQkx/hdkowS0ZHhnDd1DCXVzWzYW+d3OSIicoQGdB8N59zzwPO9tn2vx+NWggGgr3PvBO4cSJve9iKCq1L13v4OwaFUvbcvB2Ye9kWIyFH51dJCXt68n+9dPI1FE9P8LkdGmbnjU3i7qIqXNu9nWk4iEWG6z6yIyHChv9gi0q9n3t3Lz17ezuVzcrn5tPF+lyOjUJgZ50/PorqpnVW7qv0uR0REjoCChoj0aXnhAf7jT++xcEIq//3pmZr8Lb45YUw8E9PjeHVrBa0duomfiMhwoaAhIh+xuayeLz66hgnpcSz+7DxN/hZfmRkXzMiiub2LN3Yc8LscEREZoAHN0RAJhWO5X4JuvjV49tQ087nfrSIuKoLf37zgqG6WJhJqY1NimZmbxJuFlSycmEpitH4vRUSGOvVoiMj7apvb+dzv3qGlo4vff36+VpiSIeW8aWPo7oZXt1T4XYqIiAyAgoaIANDc3skXHllNaVUzi2+cx4lZiX6XJPIhafFRLJiQyuqSaiob2vwuR0REDkNBQ0Roae/ixgdXsaakhp9dfRKnTNIytjI0nX1iJpHhYby4aZ/fpYiIyGEoaIiMcg2tHTzwRhEb9tRx73Unc/GsHL9LEulXfFQEZ0zOYHN5PSVVTX6XIyIih6CgITKK1TS1s3hZEdVN7Tz4uXlcODPb75JEDuv0gnQSoiL4x8Z9OOf8LkdERPqhoCEySu2vb+X+ZTtpbu/i86dP4IzJGX6XJDIggYgwzpmaSUl1M1vKG/wuR0RE+qGgITIK7axs5IE3inAOvnDGRPJTY/0uSeSIzBuXSnp8FC9u2kdXt3o1RESGIgUNkVGks7ubFzaW89Cbu4gNRHDbmRPJSor2uyyRIxYeZpw/fQyVjW2sLanxuxwREemDbtgnMkpU1LeyZPVuyupaWTA+lYtmZhOI0HcNMnxNy04kPzWWV7bu56S8ZP0+i4gMMQoaIr10dHVT19IBLjjE6KBwM8YkRhMTCPexuiPnnGPlrmpe2FhOZHgYNywcx7Qc3SNDhj8z48IZWdy/rIi3dh7g7CmZfpckIiI9KGjIqNbS3kVxVRP76lvZVxf8OdDYxsER3z97ZftHzkmNC5CTHE12Ugy5yTFMHhPPtOxETsxKHFIhxDnH9v2NLN1WQWl1M5Mz4/nM3LEkRkf6XZpIyIxLi2NqdiLLtlcyf3wq8VH6Z01EZKjQX2QZlfbXt/L2zirW7a6hoysYK1LjAmQlRjNzbBKpcQHCzDit4IMb13V2OfbVt1JW20JZbQulVc0sLzxAU3sXAGEGE9LjmJaTxIycRCoa2shNjiE6cnDDR7dzbC2vZ+m2SvbWtpAUE8nls3OZOz6FMLNBrUVkMJw/bQy/+Gc9r22r0H1gRESGEAUNGTW6nWPH/gbe2llFYUUjEWHG7Lxk5uSnkJMUTVQfgeDS2bmHbNM5x56aFjaX17O5rJ7N5fWsLanhb++VAWBAenwUuSkxjE2JYWxyDNnJMUSGh34seVltC6t2VfN20QH217eRGhfg03NymZ2fTESYxq7LyJWZGM288SmsLKrm1EnppMYF/C5JRERQ0JBRoqqxjcdXlVJW10pidASfmDYmJMMszIy81FjyUmM5f3rWh673v68WsqemmT01LeysaOTd3bVAsOdjTGI0uckx5CTHkBYfIC0uiqSYSMLDBt7j0NrRxTvF1by+rZJlOyrZvj84n/XtwKIAACAASURBVCQjIYor545l1tjkI2pvtHlsZanfJUgInXviGN7dXctLm/dxzfx8v8sREREUNGQU2FJez5/W7MYwrpw7lpljk477N/xp8VGcMCaBE8YkAMGej/rWTvZ6wWNvbQubyupZ3WNZzjCDlNgAL2wsJy4QQUwgnOjIcGIiw4mMMGqbOtjfEJxHUtHQRnVTOwCB8DAWTkzlyrl5NLZ1kpkQhWmIlIwyiTGRnF6QwdJtFSyc0OR3OSIigoKGjGBd3Y57Xt7OoytKyEmO5roF43wbUmFmJMVEkhSTxLScJOCD8FHV1EZ1YztVTcGf+tZO9te30tLRRUt7N60dXbR1dpEcG5xDMjYlhrnjUhiTGM2M3EQWTUwjNhD8v7K+pZfR7GMnZLCuNDh08VsXTCHiOAxRFBGRgVPQkBGpuqmdf3liHW/sOMDccSl86qSc4zIv4lh8ED4imZj+wfbrFmrYh8jRCESEcdHMbB5bVcqjK0q4+bQJfpckIjKqKWjIiLO/vpXP/Ho5FfVt3PXpmXS7w58jIiPD9JxEJmfG87OXtnPxrBwyEqL8LklEZNRS0JARpbWji9seWU11UztPfnERc/JTNJxIhhT9Ph5fZsYls3L436U7uOuFrfzPVSf5XZKIyKg1tMaSiBwD5xzf+vN63ttTxz1Xz2ZOforfJYmID9ITorj1jIn8ee0e1pRU+12OiMiopaAhI8avX9/JM++W8R/nnfChpWZFZPT52jkFZCdF859Pb6JL4ydFRHyhoCEjwsub9/PTF7dxyUk53H52gd/liIjPYgMRfPeT09hcXs8fV5b4XY6IyKikoCHD3tZ99fzrE+uYmZvET6+YpXtIiAgAF83M4rSCNH764jb217f6XY6IyKijoCHDWm1zO7c+vJq4qAgW3ziP6Mhwv0sSkSHCzLjj0hl0dHXzrT+vxzkNoRIRGUwKGjKs/c9L2ymrbeE3N84lKyna73JEZIiZmBHPdy6cymvbKnl81W6/yxERGVUUNGTY2lwWHHt946JxnKwVpkSkHzcuGsfpBen86O+bKalq8rscEZFRQ0FDhiXnHD94dhNJMZH82ydO8LscERnCwsKMn1wxi/Aw49+XvKdVqEREBomChgxLf1tfzqriav7P+SeSHBvwuxwRGeJykmP44aXTWV1SwwNvFPldjojIqKCgIcNOc3snP/77FmbkJnL1/Dy/yxGRYeKy2blcOCOLn720nS3l9X6XIyIy4iloyLBz79JC9tW38oNLphMepqVsRWRgzIw7L59JYkwk//bku7R1dvldkojIiBbhdwEiR6KkqokHlu3i8jm5zBuf6nc5IsPGYytL/S5hSEiNC3D3Z2Zyy8Or+c+nN3L3Zz56751jea+uW5h/1Of6dV0RkeNFPRoyrNzx3BYiw41vX3ii36WIyDB17tQxfP2cApas3sODb+7yuxwRkRFLQUOGjTd2VPLKlv187dzJjEnUPTNE5Oj968dP4MIZWfz4+S0s3VrhdzkiIiOSgoYMG796tZCcpGhuPm2836WIyDAXFmb8z1UnMTU7ka89vo7t+xv8LklEZMTRHA0ZFn7yj62s3FXNRTOz+fOavX6XIyIjQGwggt/eNI9P/eotbn14NU/ffhqpcVouW0QkVNSjIcPCGzsOEB0ZxvxxugO4iIROdlIMi2+cy776Vr78hzW0d3b7XZKIyIihoCFDXnVTOxv31rFgfCpRkeF+lyMiI8yc/BR+esUsVu6q5l+eWEdnt8KGiEgoaOiUDHlv7TyAGZwyKd3vUkRkhLp0di4HGtu547nNFFU2cd3CfCLD9V2ciMix0F9RGdKa2ztZU1zDSWOTSYqJ9LscERnBbjl9Aj++fCbb9zfw8PJi3dBPROQYqUdDhrRVu6pp7+rmjMkZvlxfNzkTGV2uW5jPutIa/rx2D797q5ibThlPTEBDNkVEjoZ6NGTI6uzqZvnOKiZnxpOVpPtmiMjgmJOfwjXz89lb08Jv3yyisa3T75JERIYlBQ0Zst7dXUtjW6dvvRkiMnrNyE3ihkXjqGxo476lhZRUNfldkojIsKOgIUNSt3O8UXiA7KRoJmXE+V2OiIxCU7ISuO3MiYSFGQ+8UcSrWyvods7vskREho0BzdEwswuAXwDhwG+dc3f12h8FPALMBaqAq51zxd6+7wC3AF3A151zLx6qTTObADwBpAJrgRudc+1mdibwc2AWcI1z7qke178J+K739EfOuYeP8H2QIWb7/gYqG9q4at5YzMzvckRklBqbEstXzy7g6Xf38sqW/eysbOSqeXlanCIEjmUO3HUL80NYiYgcL4ft0TCzcOBe4EJgGnCtmU3rddgtQI1zrgC4B7jbO3cacA0wHbgAuM/Mwg/T5t3APc65yUCN1zZAKfA54LFe9aUC3wcWAguA75uZ7uo2zL29s4rE6Ahm5ib7XYqIjHLRkeFcPS+Pz5w8lr01LfzynzvYsLcOp94NEZFDGsjQqQVAoXOuyDnXTrC34dJex1wKHOxFeAo414JfQ18KPOGca3PO7QIKvfb6bNM75xyvDbw2LwNwzhU759YDve+kdD7wsnOu2jlXA7xMMNTIMFXb3E5hRSNzx6UQHqbeDBHxn5kxd1wKt59dQEpsJI+vKmXxsiLN3RAROYSBBI1cYHeP53u8bX0e45zrBOqAtEOc29/2NKDWa6O/ax1NfZjZbWa22sxWV1ZWHqZJ8dO63bU44OR8dUyJyNCSkRDFl88q4NLZOVQ3tXP/siIefbuY/fWtfpcmIjLkDGSORl9fKffuL+7vmP629xVwDnX8oQzoHOfcYmAxwLx589TfPUQ551hbUsOE9DjS4qP8LkdE5CPCw4yFE9KYk5fC8p0HeH17Jb/85w5m5yUzLSeRk8YmaW6ZiAgDCxp7gLwez8cCZf0cs8fMIoAkoPow5/a1/QCQbGYRXq9GX9fqq76zerX12mHOkSGquKqZqqZ2zp6S6XcpIiKHFIgI46wpmSwYn8rSbRWsKq7msnvf4sSsBK6en8flc3JJjg34XaaIiG8GMnTqHWCymU0wswDByd3P9jrmWeAm7/EVwKsuOEvuWeAaM4vyVpOaDKzqr03vnKVeG3htPnOY+l4EzjOzFG8S+HneNhmG1pbUEIgIY0Zukt+liIgMSGxUBJ+clcN3LpzKnZfPIBARxn/9bTMLfvxPvvb4Op5bX0ZdS4ffZYqIDLrD9mg45zrN7KsEP7yHAw855zaZ2Q+B1c65Z4EHgUfNrJBgT8Y13rmbzGwJsBnoBG53znUB9NWmd8lvAU+Y2Y+AdV7bmNl84K9ACnCJmf2Xc266c67azO4gGF4Afuicqz7G90V80NbZxYa9dcwcm0QgQrd4EZHhJToynOsW5nP9wnFsKqtjyTu7eea9Mv72XhnhYca8cSmcfWIm55yYyeTM+CEzvErLzIrI8TKg+2g4554Hnu+17Xs9HrcCV/Zz7p3AnQNp09teRHBVqt7b3yE4LKqvazwEPHTIFyFD3sa99bR3dTNXk8BFZJibnpPEf12axH9ePI13d9fy6tYKlm6r5K4XtnLXC1tJjQtwcn4yJ49LYW5+CrPGailvERl5BhQ0RAbDmpIa0uICjEuL9bsUEZGQiAgPY974VOaNT+WbF5xIeV0Ly7ZXsrq4hjWlNbyypSJ4XJgxJjGa/LRYxqXGkp8aq/kdIjLsKWjIkFDV2EZxVRPnTRszZIYT+OVYhjGIyNCWnRTD1fPzuXp+cMhRdVM760prWFtaw/Mb9rG6uJq3d1YBkBgdwbi0OMalxTIuLY6sxGjdW0hEhhUFDRkS1pTWYMAcDZsSkVEkNS7AuVPHcO7UMeQmx9LV7dhX10ppdRMl1c2UVjWzYW8dEFzlKj81Nhg8UuPIS40hKiLc51cgItI/BQ3xXbdzrCutZfKYeJJiIv0uZ9RST4qI/8LDjNyUGHJTYjhlUnBbbXM7JdXNlFQ1UVLVzKtbKoI3pLJgD8m4tFiSYyNZNDGN1DgNtxKRoUNBQ3y3s6KRupYOLpyR5XcpIiJDTnJsgOTYACd5E8ZbO7oo9YJHcVUz7xRXs3xnFWYwLTuR0wrSOXVSGgsmpBIb0D/zIuIf/QUS360prSEmMpyp2Yl+lyIiMuRFR4ZzwpgEThiTAEBXt2N6biLLCw/wVmEVv3+rmMXLiogMN+bkp3DapHROK0jjpLxkIsO1dLiIDB4FDfFVW2cXW8rrOTk/Rf8AiogchfAw4+T8FE7OT+Gr50ympb2L1SXVvFVYxVuFB/j5P7dzzysQFwhn4cQ0Tp2UxmkF6UwZk0CYJpeLyHGkoCG+2lreQEeX0xryIiIhEhMI54zJGZwxOQMIzvFYUVT1fvB4dWtwSd20uACnFqQTEWZMyojX/A4RCTkFDfHV+r113hKOuneGiMjxkBwb4IIZ2VwwIxuA8rqW90PHW4UHqGhoA4IrYE3KiGNSRjwTM+KJj9JHBBE5NvorIr5pae9i+/4GFk1IJWyU3ztDREJLq6j1LzsphivmjuWKuWNxzvGLV3aws7KRwsomNuyt453iGu+4aCZlxDMpI47x6XFaSldEjpiChvhmS3k9Xd2OmRo2JSLiCzMjMzGazMRoTpmUTle3o6y2xQsejawoquLNwgOEGeSlxnrBI5681BgiwjSvTkQOTUFDfLN+by3JsZHkpcT4XYqIiBCcWJ6XGkteaixnTcmko6ubkqpmdlY2srOykaVbK3h1awWB8DDGp8fS1NbJqQVpTM1K1MRyEfkIBQ3xRXNbJ4UVjZxekI5p2JSIyJAUGR5GQWY8BZnxQHDI664Dwd6OnRVN3Pn8FiA4v+OUSWnvL6Wbnxqrv+0ioqAh/thUVk+3Q8OmRESGkZhAONNykpiWkwTAOSdmsnznAd4sPMDywir+vr4cgNzkGM6dmslFM7OZPz6VcPV2iIxKChrii/V7a0mLC5CTFO13KSIicpSykqL59Mlj+fTJwYnlRQeaWF54gGU7DrBk9W4eebuE9PgA50/P4pMzs1kwIZUI3TNJZNRQ0JBB19DaQVFlE2dNyVDXuojICGFm708Wv/GU8TS1dfLatkqe31DOX9bu5Y8rS8lKjOa6hflcsyDP73JFZBAoaMig21RWj0PDpkRERrK4qAg+OSubT87KpqW9i6XbKnh8VSk/e3k7//vqDqZmJ3LKRM3nEBnJFDRk0K3fU0tmQhRZiRo2JSIyGsQEwrloZjYXzcymqLKRR1eU8PiqUtbvqSM3OYbzpo9hcmaC32WKSIgpaMigqmvpoKSqmXOmZvpdioiI+GBiRjzfv2Q641LjWLe7hmXbK/ndW8UUZMZz/vQscpO15LnISKGgIYNq4946HDArV8OmRERGs0BEGAsnpDE3P4WVu6pZuq2Ce5cWctLYJD4xLYvUuIDfJYrIMVLQkEG1fk8t2UnRZCRE+V2KiIgMARHhYZxWkM7ccSks217JWzsPsLGsnk9MHcPpk9MJ0/wNkWFLQUMGTW1zO7trWjh/2hi/SxERGVEeW1nqdwnHLDoynPOmZ7FwYhp/e6+Mf2zax5Z99Vxx8ljS4vXllMhwpMWsZdBsLKsHYEZuks+ViIjIUJUUE8n1C/O5cu5Y9te38r+vFrJyVxXOOb9LE5EjpB4NGTQb99aRnRStb6ZEZMQaCT0LQ4GZMSc/hQnpcfxl3V6eebeMzWX1XDF3LAnRkX6XJyIDpB4NGRTldS2UVjerN0NERAYsOTbAzaeO51Mn5VBc1cR9r+2krLbF77JEZIAUNGRQ/GPjPgBm5ihoiIjIwJkZiyamcduZkwC4f9lOnt9Q7nNVIjIQChoyKF7YsI+sxGjStdqUiIgchdzkGL5y1iSyk2L4yh/Xcs/L2+nu1rwNkaFMQUOOu4r6Vt4pqWZ6bqLfpYiIyDCWEB3JradP4Iq5Y/nFP3dw+2NraW7v9LssEemHgoYcd//YtA/nYIaGTYmIyDGKCA/jp1fM4rufnMqLm/Zx3QMrqW1u97ssEemDgoYcd89vKKcgM54xidF+lyIiIiOAmXHrGRP59Q1z2VxWzzWLV1DR0Op3WSLSi4KGHFeVDW2s2lXNRTOz/S5FRERGmPOnZ/G7m+dTWt3Mlb95m93VzX6XJCI9KGjIcfXipn10O7hoZpbfpYiIyAh0WkE6f7h1ITVN7Vz5m7cprGjwuyQR8ShoyHH1wsZyJqbHMWVMgt+liIjICHVyfgpPfvEUOrsdV92/gg176vwuSURQ0JDjqKqxjRVF1Vw4Mwsz87scEREZwaZmJ/LUl04hJjKc63+rsCEyFET4XYCMXC9t3k9Xt+PCGZqfISIyEj22stTvEj5kfHocT35xEdcsXsH1v13BH29dxMyxWvFQxC8KGnLcPL+hnHFpsUzP0f0zREQkdA4XcK6dn88DbxZx1f1v8/nTJ5CbHPP+vusW5h/v8kTEo6FTclzUNrezfGcVF87I1rApEREZVClxAb5w+kSiIsN46M1d7K1t8bskkVFJQUOOixc37aOr22m1KRER8YXChoj/FDTkuHhufTn5qbHMzNXYWBER8cf7YSMiGDbKFDZEBpWChoRcVWMby3dWcfEsDZsSERF/pcQFuPWMYNh48M1dbCrTalQig0VBQ0LuhY3BYVMXz8rxuxQRERFSvbARiAjj+t+uVNgQGSQKGhJyf3uvjIkZcUzN1k36RERkaEiNC/CFMyYSGxnO9b9dyeayer9LEhnxFDQkpPbXt7KquJqLZ+Vo2JSIiAwpqXEBHr9t0fs39VPYEDm+FDQkpJ7fUI5zcMks3aRPRESGnnFpcTxx2yKivbCxca+GUYkcLwoaElLPrS/nxKwEJo/RsCkRERmaDoaN2EAE1y5ewTvF1X6XJDIiKWhIyJTVtrCmpIaL1ZshIiJD3Li0OP70pVPISIjixgdXsmx7pd8liYw4AwoaZnaBmW0zs0Iz+3Yf+6PM7Elv/0ozG99j33e87dvM7PzDtWlmE7w2dnhtBg51DTMbb2YtZvau9/Obo30z5Nj8fX05gFabEhGRYSEnOYYlXzqFCenx3Prwav6xcZ/fJYmMKIcNGmYWDtwLXAhMA641s2m9DrsFqHHOFQD3AHd7504DrgGmAxcA95lZ+GHavBu4xzk3Gajx2u73Gp6dzrnZ3s+XjugdkJB5bn0ZM3ITGZ8e53cpIiIiA5IeH8UTX1jEjNxEbn9sLX9Zu8fvkkRGjIH0aCwACp1zRc65duAJ4NJex1wKPOw9fgo414JLDl0KPOGca3PO7QIKvfb6bNM75xyvDbw2LzvMNWQIKK1q5r09derNEBGRYScpNpJHb1nIoompfGPJe9y7tBDnnN9liQx7AwkaucDuHs/3eNv6PMY51wnUAWmHOLe/7WlArddG72v1dw2ACWa2zsxeN7MzBvCaJMSe21AGwCdnan6GiIgMP3FRETx403w+dVIOP31xG//25Lu0dnT5XZbIsBYxgGP66jXoHfP7O6a/7X0FnEMdf6hrlAP5zrkqM5sLPG1m051zH1oc28xuA24DyM/P76MpORbPvVfO7Lxk8lJj/S5FRETkqERHhvOLa2YzJSuBn764jeKqZhbfOJfMxGi/SxMZlgYSNPYAeT2ejwXK+jlmj5lFAElA9WHO7Wv7ASDZzCK8Xouex/d5DRfs22wDcM6tMbOdwAnA6p4FOucWA4sB5s2bp/7QECqqbGRzeT3f/eRUv0sRERE5pMdWlh72mJTYANcvzOdPq/fwiXuWccOiceQmx3DdQn1RKXIkBjJ06h1gsrcaVIDg5O5nex3zLHCT9/gK4FUvADwLXOOtGDUBmAys6q9N75ylXht4bT5zqGuYWYY3uRwzm+hdo2jgb4Ecq2feLcMMPqllbUVEZISYnpPEFz82EQMWL9vJmpJqzdsQOUKHDRpez8JXgReBLcAS59wmM/uhmX3KO+xBIM3MCoFvAN/2zt0ELAE2A/8AbnfOdfXXptfWt4BveG2leW33ew3gTGC9mb1HcJL4l5xzuvPOIOnudvx57R5Om5ROdlKM3+WIiIiETHZSDF85u4CxKbH8ee1evvDIGioaWv0uS2TYsNGYzufNm+dWr159+APlsJbvPMB1D6zkF9fM5tLZvdcI+LCBdFeLiIgMNd3OsXxnFa9s2U9cIJw7L5/JRVr8RAQAM1vjnJvX1z7dGVyOyVNr9pAQFcF507L8LkVEROS4CDPj9IJ0nv/66eSlxvKVP67lX55YR21zu9+liQxpChpy1BrbOnlhwz4uPimbmEC43+WIiIgcVwWZCfz5y6fyjU+cwN/Xl3PmT5Zy32uFNLd3Hv5kkVFIQUOO2vPry2np6OKKuXmHP1hERGQEiAwP4+vnTua5r5/O/PGp/OQf2/jYT1/jkbeLae/s9rs8kSFlIMvbivTpqTV7mJgex8n5yX6XIiIiMqhOzErkwc/NZ3VxNT95cRvfe2YTi5cV8dWzC7jkpBzioj78EetY5ikey7K6fl1XBNSjIUep+EATq4qr+czcsZj1dS9FERGRkW/e+FSevG0RD39+AcmxkXz7LxuYf+crfPOp91hdrCVxZXRTj4Yclb+s3UOYwadPPvRKUyIiIiOdmfGxEzI4c3I6a0pqWLJ6N39fX86S1cGe/8vn5NLS0UVOcgxh+nJORhEFDTliwXtn7OX0yRm6d4aIiIjHzJg3PpV541P5/iXTeX5DOX9as4f/eXk7ANGRYUxMj2dSRhwTM+LJSIhS8JARTUFDjtjbRVXsrW3hWxee6HcpIiIiQ1JcVARXzsvjynl5VDS08v9e3M7OykZ2VjayubwegIgwIzMxiqzEaMYkRpOZEE1qXICkmEgCERrdLsOfgoYcsafW7CEhOoLzpo3xuxQREZEhLzMhmtl5yczOS8Y5R3VTO8VVTeyra2V/Qxs7KhpZW1r7oXNiA+Ekx0SSFBPJxrI60uMCpMYFSIuPIi0+QFpc8L8psQHCw9QrIkOTgoYckfrWDl7YWM6nTx5LdKTunSEiInIkzMwLC1Ef2t7c1sn+hjZqm9upa+mgtrmDupYOapo7eHHjPmqa2+nuY165GaTEeiEkLkB6fJQXSILPt5Q3kOgFloToCA3VkkGloCFH5Pn15bR2dHPl3LF+lyIiIjJixEZFMCEqAoj7yL7rFubT1e2obW6nqqmdqsZ2qpraqG5q50BjO9VNbcFtje1s3VdPVVM7tc0dH2knzCAhOhg6DgaT93tH4gLERuljoYSWfqNkwJxzPPJ2CZMz45mdp3tniIiIDJbwsB49IQMYudzR1U1NczuPvl1CXUvHBz/NHdS2dFB8oIn3dtfSs5MkLhDOmKRosr05IzNyEzlhTIJGMMhRU9CQAXu7qIrN5fXc9emZuneGiIjIEBYZHkZmQjTZSTH9rhDZ0dVNdVO71zPSRkVDG/vqWllVXE1Hl+Mv6/YSEWZMy0lkTl4yJ49LYU5eCnmpMYf9HKAbBQooaMgReOjNXaTFBbhsju6dISIiMtxFhocxxuu96Knbm7A+OTOe9XvrWFdaw5/W7OHht0sAyEiI4vSCdE4vSOeMyelk9jr/WCmkjBwKGjIgRZWNvLKlgn85d7K6UEVEREawMDPS46O4cGY2F87MBqCzq5vt+xtZt7uGlUXVLNteyV/X7QVgypgEzpiczoUzs5iTl0KYVsESj4KGDMhDb+0iEB7GDYvG+V2KiIjIqHIs3/CHSkR4GNNyEpmWk8j1C8fR3e3YXF7Pm4UHeGNHJY+8XcL/396dR1dZnXsc/z6ZAxkIQyAkYUxAEIQAxdki9TrXCJZCq1Vvba1Drfd2rdtq23VXe+tdtbe02moLtdrJiohIlWKtIIJIRQLIHCCEMYEwQwYwAZJ9/zgbeoRESDg5B05+n7Xedd6z3/3udx82OzlP9vvu/fzCLWSlJ3Hz4CziY2PIzTjzLVYS3RRoyBkdPHyU6cvKub2gO11SE898goiIiES1mBhjUHY6g7LTeeCzfamqPcbcdbt5c1UFLy7axtH6BjokxzOsZwaf6dWR9OT4SFdZIkCBhpzRlKLt1B5r4L6r+kS6KiIiIhFzPowsnK/SkuIZU5DDmIIcqmqP8eO/FbOi7BDvrt/D/A17GJCVxmV9OtGnc3uNcrQhCjTkU9Udr+ePH2zl6vzO9O+WGunqiIiIyHkuLSmegh4ZFPTIYH9NHUVbD7B060HW7qyic0rgQfJhPTsQFxMT6apKK1OgIZ9q1soK9lbXMXHckEhXRURERMIoFCM4nVISuWlQFtcN6MrqHZUs2rSf11fsYP6GPXy2fxeG98xQwBHFFGhIk5xzPL9wC/mZKVyT3znS1REREZELVHxsDMN6ZFCQ24GNe2qYu243b6zYyXsb9irgiGIKNKRJizbtZ11FFT+9Qwv0iYiIyLkzM/p1TSU/M+UTAcf7G/dx86AsBmSl6jtHFFGgIU367YLNdGqfQOFQLdAnIiIioXNqwPH31RX8ZfE28jNTuOWSLDJTQ7sIoESGxqikUQs37uO9kr18/Zo+WqBPREREWsWJgOOR0fncMjiLsoNH+NXcjfx9dQW1x+ojXT05RxrRkNPUNzieeLOYnIxk7r2iV6SrIyIiIlEuNsa4Mq8zQ3I7MKd4F/8s3cfKskMUDu3OwO7pka6etJBGNOQ0ry4tY/2uah6/aYBGM0RERCRsUhLjGFOQw4Oj+pKSFMdfFm/n5aLt1NQdj3TVpAUUaMgn1NQdZ+LsEob3zODmwd0iXR0RERFpg3Iy2vHQqDyuG9CV4p1VPP1OCavKD+Gci3TVpBkUaMgnTJ6/iX01dfzglgGa9UFEREQiJjbGGH1RJt8cnUfH9glMXVLGS4s1unEhUaAhJ+049DG/e38zhUO7U9AjI9LVEREREaFrWhLfuKYvN17cjZLd1fzynRLWVVRFulpyFhRoyEk/+8d6AL5z40URromIiIjIv8TGGNf068LD1+aRlhzPix9uY8ZH5dRpw6gMigAADzxJREFUZqrzmgINAWBF2SFeX7GTr13dm+wOyZGujoiIiMhpuqYl8eCovny2XxeWbTvIM/NK2bb/cKSrJU1QoCHUNzh+PKuYzikJPDgqL9LVEREREWlSXEwMN1zcja9f3QfnHM8t2Mzstbs43tAQ6arJKRRoCM++W8qybQd57KYBpCRqaRURERE5//Xq3J5vjc5neM8M5pfsZfL8TZTsro50tSSIAo02buHGfTw9t4SxBdncMSw70tUREREROWuJ8bGMHZbDXZf2pPLjY9z6zEJeWLiFhgZNg3s+0J+v27DdVbU8OnU5eV1SeGLMIE1nKyIiIhekgd3TyO2Yz5KtB/jxrGLmrtvNxHFD6K7nTiNKIxpt1PH6Bh6ZspwjR+uZdNcw2iUo5hQREZELV2pSPL+7ewRPjh3MirJD3PD0At5YsUOL/EWQAo02auLsEoq2HuAnYweTl5ka6eqIiIiInDMzY8LIHrz16NX065rKo1NX8MjLyzl05Gikq9YmKdBog+au283k9zbx5Ut7cHuBnssQERGR6NKzU3umfeNy/uuG/vxjzS5ueHoB72/cG+lqtTkKNNqYdRVVfHvaSgZmpfHftw6MdHVEREREWkVsjPHwtXm8/vCVpCbF85UXivjeX1drdCOMFGi0IUVbDvDF3y6iXUIsk+4aRlJ8bKSrJCIiItKqBmWnM+uRq/jaVb2ZWrSd0T9/j2lLyjQzVRgo0Ggj5hTv5isvLCYzNZHpD15Bz07tI10lERERkbBIio/lB7cOZNYjV9Onc3u+89oqxk76gDU7KiNdtaimQKMNmLakjG+8uJSLstJ49YEryNZUbyIiItIGDeyexqsPXM7Pxw2h/OARPv/sQh6fsZqKyo8jXbWopDlNo5hzjt8u2MyTb63n6vzOTL5rOO218reIiIi0YWbGHcNzuG5gV56aU8JfPtzGa8vKmTAyl4dG5dEtPSnSVYwa+tYZpcoOHOF/ZhUzp3g3nx/SnZ+PG0JCnAawRERERADSk+P54W0Xc99VvfnN/FKmLN7O1KIyvjQylwcVcISEAo0oU3usnknzNzHpvU3ExRiP3XQR91/dh5gYrfotIiIicqrcju34ydhLeGhUHr+eV8pLi7czpWg71w/sxoSRuVzZt7O+R7WQAo0o4Zzj7bW7eeLNYsoPfsxtQ7rzvZsHKBoXEREROQu5Hdvx5B2BgOOPH2xlxvJy3lxdQW7HZMaPyGXciFy6pul7VXNYW1yWfcSIEW7p0qWRrkZI7KmqZebKncz4aAfFFVX075rKjwov5rI+nSJdtdNMWbw90lUQERGRKPblS3uErKzaY/W8vXYXU4vKWLR5PzEGBT0yuLZ/F669KJOBWWmYaaTDzJY550Y0duysRjTM7Ebgl0As8Lxz7slTjicCfwaGA/uB8c65rf7Y48B9QD3wLefc259Wppn1BqYCHYGPgK8454625BrRqrr2GO+s282Mj3bwz9J9NDi4JCed/x0ziPEjcomL1bMYIiIiIuciKT6WwqHZFA7NZsu+w/x1+Q7mrd/DxNklTJxdQmZqItf2z2RErwwG56ST1yVF38FOccYRDTOLBUqAfwPKgSXAl5xzxUF5HgIucc49YGYTgDHOufFmNhB4GRgJdAfeAfr50xot08ymATOcc1PNbDKw0jk3qbnXcM7VN/WZLqQRjdpj9RRXVLGq7BCrdlSyqrySTXtrcA6yOyQzpiCb2wuyyctMiXRVz0gjGiIiItKaQjmi0ZQ91bW8t2Ev8zfsZcHGvVTXHgcgKT6GAVlpDM5OJz8zhZyO7cjNaEdORnJUL5J8riMaI4FS59xmX9hUoBAoDspTCPzQ708HnrXAWFIhMNU5VwdsMbNSXx6NlWlm64DRwJd9nj/5cie14BqLzuKzRUTlkWNs3lfD4bp6auqOUV17nMN1x6muPc7u6loqDtVSUVnLrqpaDhw+evK8LqmJDMlJ57Yh3bm8byeG98jQw0kiIiIiYZSZmsQ4/8xGfYNjy74aVu+oZM2OKlbvqOS1ZeUcPlp/yjmJZKUnkdE+gY7tEgKv7RNIS46nXXws7RJiSU6IJTk+8BobY8SY+dfAlLyxFkiLieHksaT4WNKT4yP0L3FmZxNoZANlQe/LgUubyuOcO25mlUAnn/7hKedm+/3GyuwEHHLOHW8kf0uucV5aWLqPh6d81OixjHbxdEtPJis9iaE9OpCVlkR+11SG5KbTLS1J9wKKiIiInCdiY4y8zFTyMlMZUxBIa2hw7K2po+zAEcoOHqHswMeUHzxCRWUt+2uOUrqnhoOHj54WjLTE2GHZ/OKLQ8+5nNZyNoFGY99sT73fqqk8TaU3dgPbp+VvyTU+WUGz+4H7/dsaM9vQyHnh1hnYF5ywLUIVkVZxWvtK1FDbRje1b3RT+0axO9tY+z4FPDU+0rWgZ1MHzibQKAdyg97nADubyFNuZnFAOnDgDOc2lr4P6GBmcX5UIzh/S65xknPuOeC5s/i8YWNmS5u6p00ufGrf6KW2jW5q3+im9o1uat/zy9k8Gr8EyDez3maWAEwAZp6SZyZwj9//AvCuCzxlPhOYYGaJfjapfKCoqTL9OfN8Gfgy32jhNUREREREJELOOKLhn4f4JvA2galof++cW2tm/wMsdc7NBF4AXvQPYh8gEDjg800j8OD4ceDhE7NBNVamv+R3galm9gSw3JdNS64hIiIiIiKR0SYX7DtfmNn9/pYuiUJq3+ilto1uat/opvaNbmrf84sCDRERERERCTktXygiIiIiIiGnQOMcmVmumc0zs3VmttbMHvXpHc1sjplt9K8ZPt3M7FdmVmpmq8xsWFBZ9/j8G83snqD04Wa22p/zK9NiGmFlZrFmttzMZvn3vc1ssW+nV/yEBvgJCV7x7bTYzHoFlfG4T99gZjcEpd/o00rN7LFwf7a2zsw6mNl0M1vv+/Dl6rvRw8z+0/9cXmNmL5tZkvrvhcvMfm9me8xsTVBaq/fXpq4hodVE+/7M/3xeZWZ/NbMOQcea1S9b0vclBJxz2s5hA7KAYX4/FSgBBgL/Bzzm0x8Dfur3bwbeIrD+x2XAYp/eEdjsXzP8foY/VgRc7s95C7gp0p+7LW3At4EpwCz/fhowwe9PBh70+w8Bk/3+BOAVvz8QWAkkAr2BTQQmQYj1+32ABJ9nYKQ/b1vagD8BX/P7CUAH9d3o2Ags3LoFSPbvpwH3qv9euBtwDTAMWBOU1ur9talraAtL+14PxPn9nwa1b7P7ZXP7vrbQbBrROEfOuQrn3Ed+vxpYR+AXXCGBLzH419v9fiHwZxfwIYF1Q7KAG4A5zrkDzrmDwBzgRn8szTm3yAV6wZ+DypJWZmY5wC3A8/69AaOB6T7LqW17os2nA5/z+QuBqc65OufcFqAUGOm3UufcZufcUWCqzythYGZpBH6xvQDgnDvqnDuE+m40iQOSLbD2UjugAvXfC5ZzbgGBWSeDhaO/NnUNCaHG2tc5N9sF1lUD+JDAWmnQzH7Zwt/dEgIKNELID7cVAIuBrs65CggEI0Cmz5YNlAWdVu7TPi29vJF0CY+nge8ADf59J+BQ0A++4PY42Yb+eKXP39w2l/DoA+wF/mCBW+OeN7P2qO9GBefcDmAisJ1AgFEJLEP9N9qEo782dQ0Jr68SGGmC5rdvS353Swgo0AgRM0sBXgP+wzlX9WlZG0lzLUiXVmZmtwJ7nHPLgpMbyerOcExte36KIzBMP8k5VwAcJnBbRFPUvhcQfx99IYHbKroD7YGbGsmq/hud1J5RxMy+T2CttJdOJDWSraXtq7ZvRQo0QsDM4gkEGS8552b45N1+KBb/usenlwO5QafnADvPkJ7TSLq0viuB28xsK4Hh19EERjg6+Fsx4JPtcbIN/fF0AsPAzW1zCY9yoNw5t9i/n04g8FDfjQ7XAVucc3udc8eAGcAVqP9Gm3D016auIWHgH9i/FbjT39YGzW/ffTS/70sIKNA4R/4+vheAdc65XwQdmgmcmM3iHuCNoPS7/YwYlwGVfij2beB6M8vwf4m7HnjbH6s2s8v8te4OKktakXPucedcjnOuF4EHxN51zt0JzAO+4LOd2rYn2vwLPr/z6RP8zBa9gXwCDx0uAfL9TBgJ/hozw/DRBHDO7QLKzKy/T/ocUIz6brTYDlxmZu38v/+J9lX/jS7h6K9NXUNamZndCHwXuM05dyToULP6pe/Lze37EgqRfBI9GjbgKgJDbKuAFX67mcD9fXOBjf61o89vwK8JzIqwGhgRVNZXCTzQVAr8e1D6CGCNP+dZ/EKL2sLazqP416xTfQj8QCsFXgUSfXqSf1/qj/cJOv/7vv02EDTzkP+/UuKPfT/Sn7OtbcBQYKnvv68TmIVGfTdKNuBHwHrfBi8SmKFG/fcC3YCXCTxvc4zAX6HvC0d/beoa2sLSvqUEnp848f1qclD+ZvXLlvR9bee+aWVwEREREREJOd06JSIiIiIiIadAQ0REREREQk6BhoiIiIiIhJwCDRERERERCTkFGiIiIiIiEnIKNEREJOTM7INI10FERCJL09uKiIiIiEjIaURDRERCzsxq/OsoM5tvZtPNbL2ZveRXXsbMPmNmH5jZSjMrMrNUM0sysz+Y2WozW25m1/q895rZ62b2NzPbYmbfNLNv+zwfmllHn6+vmf3DzJaZ2ftmdlHk/hVERNq2uEhXQEREol4BcDGwE/gncKWZFQGvAOOdc0vMLA34GHgUwDk32AcJs82sny9nkC8ricAqvt91zhWY2VPA3cDTwHPAA865jWZ2KfAbYHS4PqiIiPyLAg0REWltRc65cgAzWwH0AiqBCufcEgDnXJU/fhXwjE9bb2bbgBOBxjznXDVQbWaVwN98+mrgEjNLAa4AXvWDJgCJrfzZRESkCQo0RESktdUF7dcT+N1jQGMPCVojaY2V0xD0vsGXGQMccs4NbXlVRUQkVPSMhoiIRMJ6oLuZfQbAP58RBywA7vRp/YAewIazKdCPimwxs3H+fDOzIa1ReREROTMFGiIiEnbOuaPAeOAZM1sJzCHw7MVvgFgzW03gGY57nXN1TZd0mjuB+3yZa4HC0NZcRETOlqa3FRERERGRkNOIhoiIiIiIhJwCDRERERERCTkFGiIiIiIiEnIKNEREREREJOQUaIiIiIiISMgp0BARERERkZBToCEiIiIiIiGnQENERERERELu/wGOo1r0aaYitQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 936x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (13, 5))\n",
    "sns.distplot(profile['income'].dropna());\n",
    "plt.title('Income Histogram - Without NaNs');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     14825.000000\n",
       "mean      65404.991568\n",
       "std       21598.299410\n",
       "min       30000.000000\n",
       "25%       49000.000000\n",
       "50%       64000.000000\n",
       "75%       80000.000000\n",
       "max      120000.000000\n",
       "Name: income, dtype: float64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "profile['income'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By filling the `NaN`s with some value, we'll change the distribution's shape. Since both mean and median are fairly close to each other, it doesn't matter much which one we'll use.\n",
    "I chose the mean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "processed_profile['income'].fillna(profile['income'].mean(), inplace = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxMAAAFNCAYAAAB7QTL+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeZxcZZ3v8c+vqnpJOvtK9oQsQMJOCyggm0pwg5lBjSiDXh1EQe8dZxyXO+M4iNfLjIoziuu4oA4GLs5IXBFkGZW1CYgkIQsJWchO9qWX6n7uH3USOk130ql0qCyf9+tVrzr1nOf8znOqO+n61tkipYQkSZIk7a9cpQcgSZIk6fBkmJAkSZJUFsOEJEmSpLIYJiRJkiSVxTAhSZIkqSyGCUmSJEllMUxI0iEuIrZFxLGVHsehLiLOiYiF2ft1eaXH015EPBAR7++hWj36+xARn4qIf++pepKOLoYJSYe1iHg+Il5X6XGUKyJSREzq0PaZiPjRrtcppT4ppcX7qHNBRKw4WOM8VETEhIhoi4ivdTL7BuCr2fv10578AF8pnW1D+9+HiPh+RNy4H/Ve9nuSUvo/KaXD+n2SVDmGCUnSPkVEvtJjyPwlsBGYERE1HeaNA+b01IoOoW2WpEOWYULSESMi3hMRv4+IL0TExohYEhGXtps/KCK+FxErs/k/bTfvryJiUURsiIhZETGy3bwUER/KDqHZGhGfjYiJEfFwRGyJiDsiorpd/zdHxFMRsSkiHoqIkw9wu3bvvYiIN0bE3GwcL0TE30ZEHfArYGR2CMy2iBgZETUR8eVse1dm0zXt6v5dRKzK5r2/w3q+HxFfj4hfRsR24MKIeFNEPJlt8/KI+Ey7WuOz5d+bzdsYEddGxKsi4unsvfjqgbwPmb8E/h5oAd7Sbv3PAccCP8u2//PAecBXs9dfzfodHxH3ZD/n+RHx9nY1XrbNnfws3hMRi7P3f0lEvCtr32NvUrv3o9Bu8YkR8VhEbI6IuyJiUNa3NiJ+FBEvZu/T4xExPCI+18U2pIiYFBHXAO8C/i6b/7P28zts1417+T3pOPa3RsScbCwPRMQJ7eY9n/3OPZ1tx+0RUbufP0NJR5KUkg8fPnwctg/geeB12fR7KH3I/CsgD3wQWAlENv8XwO3AQKAKOD9rvwhYD5wO1ABfAf673ToSMAvoB0wDmoDfUvrw2h+YC1yd9T0dWAuclY3h6myMNV2MPwGTOrR9BvhRZ32AVcB52fRA4PRs+gJgRYc6NwCPAMOAocBDwGezedOB1dn29AZ+2GE93wc2A+dQ+uKpNlvHSdnrk4E1wOVZ//HZ8t/I+r4BaAR+mq1/VPa+nH8AP+vzsvd+YPYzmtXV70L2+gHg/e1e1wHLgfcChexntR6Y1tU2d6hfB2wBjstej2i3bMef2a73o9BuLC8AJ2Z1frKrP/AB4GfZzyEPnAH062wbOvl9+D5w495+p9r36eL3ZPfYgSnAduD1lP6N/B2wCKhu9x4/BowEBgHzgGsr/f+ADx8+Kvdwz4SkI83SlNK3U0qtwK2UPvANj4gRwKWUPvhsTCm1pJQezJZ5F/DdlNLslFIT8Eng1RExvl3dm1JKW1JKc4BngN+klBanlDZT+rb3tKzfXwHfTCk9mlJqTSndSukD8Nl7GfPs7FvgTRGxCfjEXvq2AFMjol+2HbP30vddwA0ppbUppXXAPwFXZfPeDnwvpTQnpbQjm9fRXSmlP6SU2lJKjSmlB1JKf8pePw38GDi/wzKfzfr+htKH0h9n638B+B0vvU/luBr4VUppI3AbcGlEDNuP5d8MPJ9S+l5KqZi9dz8BrmjXZ49t7qRGG3BiRPRKKa3Kfh+664cppWdSStuBfwDeHqVDqVqAwZQCQGtK6YmU0pb9qNuT3gH8IqV0T0qpBfgC0At4Tbs+/5ZSWplS2kApBJ1agXFKOkQYJiQdaVbvmsg+JAP0AcYAG7IPoh2NBJa2W24b8CKlb9N3WdNuemcnr/tk0+OAv+kQDsZk6+jK6SmlAbsewP/dS9+/AN4ILI2IByPi1Xvpu8d2ZdMj281b3m5e++lO2yLirIi4PyLWRcRm4FpgSIdluvs+7aHdYTfbImJsJ/N7AW8D/gMgpfQwsAy4srN6XRgHnNXhZ/Mu4Jh2fTp7H8jWuZ3Sh+1rgVUR8YuIOH4/1t++9lJK3/wPobRX6G5gZnbI2T9HRNV+1O1JHf8ttFEad/t/C6vbTe+gi5+ppKODYULS0WI5MCgiBnQybyWlD5oAZMeWD6Z0WEo56/lc+3CQUuqdUvpxWaPuIKX0eErpMkqHDv0UuGPXrE6677FdwNisDUqHS41uN29MZ6vr8Po2Sod7jUkp9ad0SFPs1wZ0IZWuULTrsayTLn9G6TCzr0XE6ohYTekD7l/urWyH18uBBzv8bPqklD64l2U6jvPulNLrKe3xehb4djZrO6XDlHY5puOy7Pkej6W0R2J9tpfsn1JKUyntAXhzu+3a63i6mL9jL2PZV72O/xYiG3c5/xYkHQUME5KOCimlVZQOR/paRAyMiKqIeG02+zbgvRFxapROUP4/wKMppefLWNW3gWuzb/EjIuqidOJy3wPdhoiojoh3RUT/7BCULUBrNnsNMDgi+rdb5MfA30fE0IgYAnwa2HWi7R2UtvmEiOidzduXvpT27jRGxJns316BA3U18F1K52ycmj3OAU6NiJO6WGYNpfNadvk5MCUirsp+/lVROkH8hC6W30N2UvRbs7DZBGzjpff/KeC1ETE2+xl8spMS746Iqdn7fQNwZ0qpNSIujIiTskOetlAKGe1/rnu7p0Rn858CroyIfERMZ89D0Tr7PWnvDuBNEXFxtnfkb7JtfWgvY5B0FDNMSDqaXEXpg9qzlE4G/l8AKaXfUjqG/SeUvrGfCMwoZwUppQZK5018ldIlTBdROjG8p1wFPB8RWygdbvPubL3PUgoPi7NDeEYCNwINwNPAn4DZWRsppV8B/wbcn43x4ax+017W/SHghojYSil83LGXvj0mIkYBFwNfTimtbvd4Avg1paDRmX8FrojSlaX+LaW0ldKJ4TMofQO/GriJ0kn33ZGj9OF6JbCB0of0DwGklO6hdHL/08ATlIJLRz+kdDL0akonqX8kaz8GuJNSkJgHPMhLoW+Pbeik5nconUOzKV66Otn/pHSlq12Hce2+alkXvye0mz+f0u/UVyidnP4W4C0ppeZ9vDeSjlK7rnAiSTqKZd/OP0PpqlPFSo9HknR4cM+EJB2lIuLPskOnBlL6hv5nBglJ0v4wTEjS0esDwDrgOUrH6H9w790lSdqThzlJkiRJKot7JiRJkiSVxTAhSZIkqSyFSg/gYBoyZEgaP358pYchSZIkHbaeeOKJ9SmloZ3NO6LDxPjx42loaKj0MCRJkqTDVkQs7WqehzlJkiRJKothQpIkSVJZDBOSJEmSymKYkCRJklQWw4QkSZKkshgmJEmSJJXFMCFJkiSpLIYJSZIkSWUxTEiSJEkqi2FCkiRJUlm6FSYiYnpEzI+IRRHxiU7m10TE7dn8RyNifLt5n8za50fEJfuqGRETshoLs5rVWfvYiLg/Ip6MiKcj4o0HsuGSJEmSDkxhXx0iIg/cArweWAE8HhGzUkpz23V7H7AxpTQpImYANwHviIipwAxgGjASuDcipmTLdFXzJuDmlNLMiPhGVvvrwN8Dd6SUvp7V/SUw/gC3X5Iq6rZHl+1X/yvPGnuQRiJJ0v7rzp6JM4FFKaXFKaVmYCZwWYc+lwG3ZtN3AhdHRGTtM1NKTSmlJcCirF6nNbNlLspqkNW8PJtOQL9suj+wcv82VZIkSVJP6k6YGAUsb/d6RdbWaZ+UUhHYDAzey7JdtQ8GNmU1Oq7rM8C7I2IFpb0SH+7G2CVJkiQdJN0JE9FJW+pmn55qB3gn8P2U0mjgjcAPI+Jl44+IayKiISIa1q1b10k5SZIkST2hO2FiBTCm3evRvPwQo919IqJA6TCkDXtZtqv29cCArEbHdb0PuAMgpfQwUAsM6TjYlNK3Ukr1KaX6oUOHdmPzJEmSJJWjO2HicWBydpWlakonVM/q0GcWcHU2fQVwX0opZe0zsqs9TQAmA491VTNb5v6sBlnNu7LpZcDFABFxAqUw4a4HSZIkqUL2eTWnlFIxIq4H7gbywHdTSnMi4gagIaU0C/gOpcOOFlHaIzEjW3ZORNwBzAWKwHUppVaAzmpmq/w4MDMibgSezGoD/A3w7Yj4a0qHPr0nCx+SJEmSKiCO5M/j9fX1qaGhodLDkKQueWlYSdKhLiKeSCnVdzbPO2BLkiRJKothQpIkSVJZDBOSJEmSymKYkCRJklQWw4QkSZKkshgmJEmSJJXFMCFJkiSpLIYJSZIkSWUxTEiSJEkqi2FCkiRJUlkME5IkSZLKYpiQJEmSVBbDhCRJkqSyGCYkSZIklcUwIUmSJKkshglJkiRJZTFMSJIkSSqLYUKSJElSWQwTkiRJkspimJAkSZJUFsOEJEmSpLIYJiRJkiSVxTAhSZIkqSzdChMRMT0i5kfEooj4RCfzayLi9mz+oxExvt28T2bt8yPikn3VjIgJWY2FWc3qrP3miHgqeyyIiE0HsuGSJEmSDsw+w0RE5IFbgEuBqcA7I2Jqh27vAzamlCYBNwM3ZctOBWYA04DpwNciIr+PmjcBN6eUJgMbs9qklP46pXRqSulU4CvAf5a/2ZIkSZIOVHf2TJwJLEopLU4pNQMzgcs69LkMuDWbvhO4OCIia5+ZUmpKKS0BFmX1Oq2ZLXNRVoOs5uWdjOmdwI+7u5GSJEmSel53wsQoYHm71yuytk77pJSKwGZg8F6W7ap9MLApq9HpuiJiHDABuK8bY5ckSZJ0kHQnTEQnbambfXqqvb0ZwJ0ppdZO+hIR10REQ0Q0rFu3rrMukiRJknpAd8LECmBMu9ejgZVd9YmIAtAf2LCXZbtqXw8MyGp0ta4Z7OUQp5TSt1JK9Sml+qFDh+5z4yRJkiSVpzth4nFgcnaVpWpKH+ZndegzC7g6m74CuC+llLL2GdnVniYAk4HHuqqZLXN/VoOs5l27VhIRxwEDgYf3f1MlSZIk9aTCvjqklIoRcT1wN5AHvptSmhMRNwANKaVZwHeAH0bEIkp7JGZky86JiDuAuUARuG7X4Umd1cxW+XFgZkTcCDyZ1d7lnZRO6O546JMkSZKkV1gcyZ/L6+vrU0NDQ6WHIUlduu3RZfvV/8qzxh6kkUiS1LmIeCKlVN/ZPO+ALUmSJKkshglJkiRJZTFMSJIkSSqLYUKSJElSWQwTkiRJkspimJAkSZJUFsOEJEmSpLIYJiRJkiSVxTAhSZIkqSyGCUmSJEllMUxIkiRJKothQpIkSVJZDBOSJEmSymKYkCRJklQWw4QkSZKkshgmJEmSJJXFMCFJkiSpLIYJSZIkSWUxTEiSJEkqi2FCkiRJUlkME5IkSZLKYpiQJEmSVBbDhCRJkqSydCtMRMT0iJgfEYsi4hOdzK+JiNuz+Y9GxPh28z6Ztc+PiEv2VTMiJmQ1FmY1q9vNe3tEzI2IORFxW7kbLUmSJOnA7TNMREQeuAW4FJgKvDMipnbo9j5gY0ppEnAzcFO27FRgBjANmA58LSLy+6h5E3BzSmkysDGrTURMBj4JnJNSmgb8r7K3WpIkSdIB686eiTOBRSmlxSmlZmAmcFmHPpcBt2bTdwIXR0Rk7TNTSk0ppSXAoqxepzWzZS7KapDVvDyb/ivglpTSRoCU0tr931xJkiRJPaU7YWIUsLzd6xVZW6d9UkpFYDMweC/LdtU+GNiU1ei4rinAlIj4Q0Q8EhHTuzF2SZIkSQdJoRt9opO21M0+XbV3FmL21h9KY50MXACMBn4XESemlDbtMZCIa4BrAMaOHdtJOUmSJEk9oTt7JlYAY9q9Hg2s7KpPRBSA/sCGvSzbVft6YEBWo+O6VgB3pZRaskOm5lMKF3tIKX0rpVSfUqofOnRoNzZPkiRJUjm6EyYeByZnV1mqpnRC9awOfWYBV2fTVwD3pZRS1j4ju9rTBEof/h/rqma2zP1ZDbKad2XTPwUuBIiIIZQOe1q8vxssSZIkqWfs8zCnlFIxIq4H7gbywHdTSnMi4gagIaU0C/gO8MOIWERpj8SMbNk5EXEHMBcoAtellFoBOquZrfLjwMyIuBF4MqtN1vcNETEXaAU+llJ68cDfAkmSJEnliNLOgCNTfX19amhoqPQwJKlLtz26bL/6X3mW54JJkl5ZEfFESqm+s3neAVuSJElSWQwTkiRJkspimJAkSZJUFsOEJEmSpLIYJiRJkiSVxTAhSZIkqSyGCUmSJEllMUxIkiRJKothQpIkSVJZDBOSJEmSymKYkCRJklQWw4QkSZKkshgmJEmSJJXFMCFJkiSpLIYJSZIkSWUxTEiSJEkqi2FCkiRJUlkME5IkSZLKYpiQJEmSVBbDhCRJkqSyGCYkSZIklcUwIUmSJKkshglJkiRJZelWmIiI6RExPyIWRcQnOplfExG3Z/MfjYjx7eZ9MmufHxGX7KtmREzIaizMalZn7e+JiHUR8VT2eP+BbLgkSZKkA7PPMBEReeAW4FJgKvDOiJjaodv7gI0ppUnAzcBN2bJTgRnANGA68LWIyO+j5k3AzSmlycDGrPYut6eUTs0e/17WFkuSJEnqEd3ZM3EmsCiltDil1AzMBC7r0Ocy4NZs+k7g4oiIrH1mSqkppbQEWJTV67RmtsxFWQ2ympeXv3mSJEmSDpbuhIlRwPJ2r1dkbZ32SSkVgc3A4L0s21X7YGBTVqOzdf1FRDwdEXdGxJhujF2SJEnSQdKdMBGdtKVu9umpdoCfAeNTSicD9/LSnpA9BxJxTUQ0RETDunXrOusiSZIkqQd0J0ysANrvBRgNrOyqT0QUgP7Ahr0s21X7emBAVmOPdaWUXkwpNWXt3wbO6GywKaVvpZTqU0r1Q4cO7cbmSZIkSSpHd8LE48Dk7CpL1ZROqJ7Voc8s4Ops+grgvpRSytpnZFd7mgBMBh7rqma2zP1ZDbKadwFExIh263srMG//NlWSJElSTyrsq0NKqRgR1wN3A3nguymlORFxA9CQUpoFfAf4YUQsorRHYka27JyIuAOYCxSB61JKrQCd1cxW+XFgZkTcCDyZ1Qb4SES8NauzAXjPAW+9JEmSpLJFaWfAkam+vj41NDRUehiS1KXbHl22X/2vPGvsQRqJJEmdi4gnUkr1nc3zDtiSJEmSymKYkCRJklQWw4QkSZKkshgmJEmSJJXFMCFJkiSpLIYJSZIkSWUxTEiSJEkqi2FCkiRJUlkME5IkSZLKYpiQJEmSVBbDhCRJkqSyGCYkSZIklcUwIUmSJKkshglJkiRJZTFMSJIkSSqLYUKSJElSWQwTkiRJkspimJAkSZJUFsOEJEmSpLIYJiRJkiSVxTAhSZIkqSyGCUmSJEllMUxIkiRJKku3wkRETI+I+RGxKCI+0cn8moi4PZv/aESMbzfvk1n7/Ii4ZF81I2JCVmNhVrO6w7quiIgUEfXlbLAkSZKknrHPMBEReeAW4FJgKvDOiJjaodv7gI0ppUnAzcBN2bJTgRnANGA68LWIyO+j5k3AzSmlycDGrPausfQFPgI8Wt7mSpIkSeop3dkzcSawKKW0OKXUDMwELuvQ5zLg1mz6TuDiiIisfWZKqSmltARYlNXrtGa2zEVZDbKal7dbz2eBfwYa93M7JUmSJPWw7oSJUcDydq9XZG2d9kkpFYHNwOC9LNtV+2BgU1Zjj3VFxGnAmJTSz7sxZkmSJEkHWXfCRHTSlrrZp0faIyJH6fCpv9nLOEsDibgmIhoiomHdunX76i5JkiSpTN0JEyuAMe1ejwZWdtUnIgpAf2DDXpbtqn09MCCr0b69L3Ai8EBEPA+cDczq7CTslNK3Ukr1KaX6oUOHdmPzJEmSJJWjO2HicWBydpWlakonVM/q0GcWcHU2fQVwX0opZe0zsqs9TQAmA491VTNb5v6sBlnNu1JKm1NKQ1JK41NK44FHgLemlBrK3G5JkiRJB6iwrw4ppWJEXA/cDeSB76aU5kTEDUBDSmkW8B3ghxGxiNIeiRnZsnMi4g5gLlAErksptQJ0VjNb5ceBmRFxI/BkVluSJEnSISZKOwOOTPX19amhwZ0Xkg5dtz26bL/6X3nW2IM0EkmSOhcRT6SUOr3Hm3fAliRJklQWw4QkSZKkshgmJEmSJJXFMCFJkiSpLIYJSZIkSWUxTEiSJEkqi2FCkiRJUlkME5IkSZLKYpiQJEmSVBbDhCRJkqSyGCYkSZIklcUwIUmSJKkshglJkiRJZTFMSJIkSSqLYUKSJElSWQwTkiRJkspimJAkSZJUFsOEJEmSpLIYJiRJkiSVxTAhSYeQFRt38OzqLRTb2io9FEmS9qlQ6QFIkmDTjmZ+PWc1T6/YDEDv6jwnjx7A6WMHMGpALyKiwiOUJOnlDBOSVEHNxTYeXLCO3y1cB8CFxw1jzKBePLlsEw3Pb+CRxS8ytG8NZx87mLMnDKrwaCVJ2pNhQpIq5NfPrOJL98xnS2ORk0f3Z/q0YxjQuxqA44/px87mVp55YTMNSzfwsz+upLUt8a6zx1V41JIkvaRb50xExPSImB8RiyLiE53Mr4mI27P5j0bE+HbzPpm1z4+IS/ZVMyImZDUWZjWrs/ZrI+JPEfFURPw+IqYeyIZLUiU1PL+B6257kj61BT7w2mOZ8aqxu4PELr2q87xqwiA+cP5Epo3sx6/+tIr756+t0IglSXq5fYaJiMgDtwCXAlOBd3byQf59wMaU0iTgZuCmbNmpwAxgGjAd+FpE5PdR8ybg5pTSZGBjVhvgtpTSSSmlU4F/Br5U5jZLUkVt3tnC/5z5FCMH1PL+c49l3OC6vfbPRfC2M8Ywon8tH77tSRas2foKjVSSpL3rzp6JM4FFKaXFKaVmYCZwWYc+lwG3ZtN3AhdH6WzBy4CZKaWmlNISYFFWr9Oa2TIXZTXIal4OkFLa0m59dUDav02VpMpLKfGp//wTa7Y08m8zTqO2Kt+t5aoLOd599jhqq/K879bH2bC9+SCPVJKkfetOmBgFLG/3ekXW1mmflFIR2AwM3suyXbUPBjZlNV62roi4LiKeo7Rn4iPdGLskHVJmPr6cX/xpFX/zhuM4bezA/Vp2QO9qvv2XZ7BmSxPX/ugJmotePlaSVFndCROdXY+w416Brvr0VHtpIqVbUkoTgY8Df9/pYCOuiYiGiGhYt25dZ10kqSIWrtnKP/1sDudOGsIHXntsWTVOGzuQf7niZB5bsoF/+OkzpOROWklS5XQnTKwAxrR7PRpY2VWfiCgA/YENe1m2q/b1wICsRlfrgtJhUZd3NtiU0rdSSvUppfqhQ4fuc+Mk6ZXQ2NLKh3/8JHXVBb709lPI5cq/b8Rlp47iugsncnvDcn71zOoeHKUkSfunO2HicWBydpWlakonVM/q0GcWcHU2fQVwXyp9XTYLmJFd7WkCMBl4rKua2TL3ZzXIat4FEBGT263vTcDC/dtUSaqcz/9yHs+u3soX3nYKw/rVHnC9v37dFI4/pi+f+8U8Gltae2CEkiTtv32Giez8heuBu4F5wB0ppTkRcUNEvDXr9h1gcEQsAj4KfCJbdg5wBzAX+DVwXUqptauaWa2PAx/Nag3OagNcHxFzIuKpbB27woskHdJmL9vIrQ8v5X+cM4ELjx/WIzUL+Rz/+JZpvLBpJ998cHGP1JQkaX/FkXy8bX19fWpoaKj0MCQdxVJKvOObj7B4/XYe/NgF1NXsea/Q2x5dtl/1rjxr7B6vP/QfT3Dfs2v57d9cwKgBvQ54vJIkdRQRT6SU6jub162b1kmSynPfs2t57PkN/M/XTX5ZkOgJn3rjCaRUOoxKkqRXmmFCkg6S1rbETb9+lglD6pjxqjH7XqAMowf25trzJ/Lzp1fx6OIXD8o6JEnqimFCkg6SnzyxggVrtvGxS46jKn/w/ru99vyJjOxfy2d+NpfWtiP30FVJ0qHHMCFJB0FjSytfumcBp4wZwKUnHnNQ19WrOs+n3nQC81Zt4ceP7d85GJIkHQjDhCQdBN/7w/Os3tLIJy89nojy7ynRXW86aQRnTRjEF38zn807Wg76+iRJAsOEJPW4TTua+doDi7jo+GGcfezgV2SdEcE/vmUam3e28JX7vA2PJOmVYZiQpB52y/2L2NZU5OPTj39F1zt1ZD/+4vTR/ODhpSzfsOMVXbck6ehkmJCkHvTCpp3c+tBS/uL00Rx3TN9XfP0ffcMUIuBL9yx4xdctSTr6GCYkqQf9672lD/Efff2Uiqx/RP9evPecCfz0qReYs3JzRcYgSTp6GCYkqYcsXreNn8x+gXedPZaRFbwb9QcvmEj/XlX83189W7ExSJKODoYJSeohX753IdX5HB+6YFJFx9G/VxXXXziJ3y1cz+8Xrq/oWCRJRzbDhCT1gGdXb+FnT6/kveeMZ2jfmkoPh6tePY5RA3rx+V/No80b2UmSDhLDhCT1gC/+ZgF9agp84LUTKz0UAGoKef72kinMWVkKOZIkHQyGCUk6QE8t38Q9c9dwzXnH0r93VaWHs9tlp4xi6oh+/Mvd82kqtlZ6OJKkI5BhQpIO0Bd/M59BddW899wJlR7KHnK54BOXHs+KjTv54cNLKz0cSdIRyDAhSQfgkcUv8ruF6/ng+RPpU1Oo9HBe5rVThnL+lKF8+d6FrN3SWOnhSJKOMIYJSSpTSokv3D2f4f1quOrV4yo9nC7901un0dzaxud+Oa/SQ5EkHWEME5JUpgfmr6Nh6Uauv2gytVX5Sg+nS+OH1HHt+RO566mVPLTIS8VKknqOYUKSytBcbOOzP5/L+MG9eUf9mEoPZ58+dMFExg7qzT/c9QzNxbZKD0eSdIQwTEhSGb77hyUsXr+df3zrNKoLh/5/pbVVef7prdN4bt12/v33iys9HEnSEeLQ/wsoSYeY1Zsb+cpvF/K6E4Zz4XHDKj2cbrvw+GFcMm04X/ntIlZs3FHp4UiSjgCGCUnaT5//1Txa2hKffvPUSg9lv336LdMAuOFncys8EknSkcAwIUn74Ru+48wAACAASURBVNHFL3LXUyu59vyJjB3cu9LD2W+jBvTiIxdP5jdz13Dfs2sqPRxJ0mGuW2EiIqZHxPyIWBQRn+hkfk1E3J7NfzQixreb98msfX5EXLKvmhExIauxMKtZnbV/NCLmRsTTEfHbiDh0r8Mo6YhUbG3jH2fNYdSAXnzw/ImVHk7Z3nfuBCYN68On/vMZ1m9rqvRwJEmHsX2GiYjIA7cAlwJTgXdGRMd9++8DNqaUJgE3Azdly04FZgDTgOnA1yIiv4+aNwE3p5QmAxuz2gBPAvUppZOBO4F/Lm+TJak8P3pkKc+u3so/vHkqvaoP3UvB7kt1Ice/zjiVjTua+fBtT1Js9epOkqTydGfPxJnAopTS4pRSMzATuKxDn8uAW7PpO4GLIyKy9pkppaaU0hJgUVav05rZMhdlNchqXg6QUro/pbTrjMFHgNH7v7mSVJ7125r44j0LOG/yEC6ZNrzSwzlg00b253N/dhIPL36RL/xmQaWHI0k6THUnTIwClrd7vSJr67RPSqkIbAYG72XZrtoHA5uyGl2tC0p7K37VjbFL0gFLKfGZWXNobGnlM2+dRul7j8PfFWeM5l1njeUbDz7Hr59ZXenhSJIOQ90JE5391Uzd7NNT7S+tKOLdQD3wL530JSKuiYiGiGhYt25dZ10kab989w/P8/OnV/G/XjeFiUP7VHo4PerTb5nKKWMG8Lf/748sXret0sORJB1muhMmVgDtb+86GljZVZ+IKAD9gQ17Wbar9vXAgKzGy9YVEa8D/jfw1pRSp2cNppS+lVKqTynVDx06tBubJ0lde/i5F/k/v5zHJdOG86ELDt+TrrtSU8jztXedTlU+uPZHT7CjubjvhSRJynQnTDwOTM6uslRN6YTqWR36zAKuzqavAO5LKaWsfUZ2tacJwGTgsa5qZsvcn9Ugq3kXQEScBnyTUpBYW97mSlL3rdy0k+tvm834wb35wttOOWIOb+po1IBefOWdp7No7TY+dufTtLZ13PksSVLn9hkmsvMXrgfuBuYBd6SU5kTEDRHx1qzbd4DBEbEI+CjwiWzZOcAdwFzg18B1KaXWrmpmtT4OfDSrNTirDaXDmvoA/y8inoqIjoFGknpMY0srH/zREzQV2/jmVfX0ra2q9JAOqnMnD+Hj04/nF0+v4sM/nk1TsbXSQ5IkHQYK++4CKaVfAr/s0PbpdtONwNu6WPZzwOe6UzNrX0zpak8d21/XnbFKUk/4zKw5/HHFZr7x7jOYNOzIOk+iKx84fyL5XHDjL+axeefjfPOqevrUdOvPhCTpKOVfCUnq4EePLGXm48u57sKJTD/xmEoP5xX1/vOOZWDvav7uJ09z5bcf4fvvPZNBddWd9r3t0WX7VfvKs8b2xBAlSYcQw4QkZdraEv9230K+fO9Czp8ylI++/rhKD+kV1T4cXHnmWH782DLecPN/8z/OGc+A3p0HCknS0a07J2BL0hFve1ORD/3HbL5870L+/PRRfPOqM8jnjswTrrvjhBH9eO85E9jW1MI3HnyO57xsrCSpE4YJSUe95Rt28Bdff4jfzF3N37/pBL74tlOorcpXelgVN2FIHX913rFU5XN85/dL+OlTL9DU4onZkqSXeJiTpIPuUD62/qFF67nuttm0tiW+/94zee0U70/T3oj+vfjwRZO5d94a/rBoPQtWb+Xy00YxZXjfSg9NknQIMExIOuq0tSUeXLiO7/5+Cb9buJ5Jw/rw7b+sZ8KQukoP7ZBUXcjxxpNGcOLIfvxk9gt8/6HnOWPcQKZPO4Y6r/YkSUc1/wpIOuTsz56M/dmLsaO5yH/OfoHv/WEJz63bzrC+NXzskuO4+jXjvQRqN4wdXMf1F03ivmfX8ruF63jmhc2cN3kI50waQk3Bw8Ik6WjkX09Jh7WugkexrY11W5tYtbmR1dljxaYdNLa0cdKo/nz5HafyxpNGUF3w1LH9UZXPccm0Yzh1zADumbuGe+et5eHFG7jouKG8asIgCjnfT0k6mhgmJL0i5zRsayqydmsjL25rZltTkW1NRbZnzzuaWim2tdHalmhtSxSz5wjIRZDPRbvnPdt2XXCppTXR0tqWPRJNxVbaUmleIRcM71fLiSP783fTj+dV4wcScfReqaknDO9Xy7vPHseyDTu4e85qfvb0Kn6/aD2vO2E4p4wZQM73V5KOCoYJST0qpcSyDTt4dPEG/rhiEwvXbuOZFzazo3nPqwDVFHL0qSlQV1NgYO8qqgo58llI2PVIlM5vaEuJ1jay59LrUju0plJi6FOboyofVOdzFPI5elXlGN6vlmP61TK4T83uy7yeOWHQK/2WHNHGDurN+8+dwMK12/jNnNX8vydW8LuF63nDtOEcN7yvoU2SjnCGCUkHbPmGHfxu4XoeXfIijy7ewOotjQD0qy0wZXhfpo3sx7C+tQzrW8OQPjX0qS1Qla/M4TCH8pWlDlcRwZThfZk0rA/PvLCZe+au4QcPL2Xc4N5Mn3YM4wZ7YrskHakME5L2W0qJhWu38etnVvPrZ1Yzd9UWAIb2reGsCYM469jBnD1hEJOG9SEi9vsD/KHE8NF9uQhOHj2AaSP707B0A/fNW8s3/3sxxx/TlzdMPabSw5MkHQSGCUndtn5rE7OXb+Tff7eYxeu3A3DGuIH87zeewIXHD2Pi0DoPaxH5XHDWhMGcNmYgDz+3ngcXruMr9y1kxaYd/PXrpjBmUO9KD1GS1EMiZccbH4nq6+tTQ0NDpYchHfL29u17U0srf3phM08s3cjSDTsI4DWTBjP9xBG8YepwhverPaD6OvLtaC7y3wvW8eiSDbSlxLvOGsf1F01iSJ+aSg9NktQNEfFESqm+s3numZD0Mm0psWT9dmYv3cgzKzfT0poY0qeGS6Ydw2ljBnDtBRMrPUQdRnpXF5h+4gj++YpT+NffLuSHjyzlzidWcN2Fk3jvOeOprfIeFZJ0uDJMSNpt4/ZmZi/byOxlG9m4o4WaQo5TxwzgjLEDGTOot4cw6YAc07+Wz//5Sbz/vAl8/pfPctOvn+W2x5byqUtPYPqJx/j7JUmHIcOEdJTb1lTcHSAWryudB3Hs0Dped8Jwpo3s703d1OMmDu3Dv19dz+8XruezP5/LB/9jNmdOGMSn3zyVE0f1r/TwJEn7wTChg8or4Ryaiq1t/H7Rev7ryRf4zZw17GxpZWDvKi4+fhinjx3IwLrqSg9RR4FzJw/hFx85l9sblvPF3yzgLV/9PW87YzR/+4bjGNaNc3EkSZVnmJCOEs3FNh5Z/CL3zF3Dr55ZzfptTfTvVcWfnT6KvjUFxnoYkyqgkM/xrrPG8ZZTRvLV+xbxvT8s4RdPr+JDF07ifedO8HwKSTrEGSZ0SDmYV/05Gvd6bN7RwgML1nLP3DU8OH8dW5uK9KrKc/6UoVx+2iguPH4oNYW8V1tSxfWrreJTbzyBK88cy+d/NY9/uXs+tz26jE++8XjedNIIg64kHaIMEzosFVvb2N7cyo7mIq1tqfRIiba20pWIIko30MpFkM8FuYAv37OA6kKOmkKe6kKOfK5nP5wcCmFlzZZGHluygcef38BjSzYwf81WUoIhfap540kjeP3U4Zw7eYjf9uqQNX5IHd+8qp6HnlvPZ38+j+tve5Jvj17M315yHOdOGmKokKRDjGFCh6ztTUVWbW5k9eadrNrcyIvbm9neVGRbU5GmYtsB18/ngup8jppCjup2j5pCnl5VeXpV5ait3jWdParz1LabrspX5uTknc2tLFq7jQVrtrJg7VYWrtnG/NVbeWHTTgB6V+c5Y9xA3njSCM6ZNJhTxwzs8fAkHUyvmTiEn3/4XH4yewX/eu9CrvrOY5x97CA+dsnxnDFuYKWHJ0nKGCZ0yGhsaeXZ1VuZu3IzyzbsYEtjcfe8frUFhvSpYdTAXtTVFOhbU6CupkCvqjyFXJDLlfZA5KM0nVJpb0VbKu2paG1LtLS20Vxsoyl7NBfbaG5t3d3WnD02NDXR2NLGzuZWmlv3HloKudgdLu58Yjn9elXRP3v0q31puld1nppCjpqqfCnAVOW4d+4aUoIEkBIJaG1Lu8fXVCyNrbGllS2NRbbsbGFLYwtbdhbZ2dK6ewz5XDC0Tw3D+tVwyuj+jB9Sx0dfP4VChYKO1FPyueDt9WO47NSRzHxsOV+5bxF/8fWHuPj4YXzk4smcMmZApYcoSUe9boWJiJgO/CuQB/49pfR/O8yvAX4AnAG8CLwjpfR8Nu+TwPuAVuAjKaW791YzIiYAM4FBwGzgqpRSc0S8FvgycDIwI6V05wFstw4R25qKzFu1hTkrN/Pc2u20pkTf2gITh/ZhRP9aRvTvxTH9a+lTU5nc29qW2NnSSmNLKzubW1+abmmlMXtderTRu7rAhu3NLFm/nc07W9iys4W2HrjBfAB9agv0q61iUF0N4wfX0b9XFUOyADG4ruZlex0MEjqS1BTyXP2a8bytfjTff+h5vvHAc1x2yx84c8IgrjnvWC46fhg597xJUkXs8xNaROSBW4DXAyuAxyNiVkppbrtu7wM2ppQmRcQM4CbgHRExFZgBTANGAvdGxJRsma5q3gTcnFKaGRHfyGp/HVgGvAf42wPdaFXehu3NPLhgHbOXbaS1LTGwdxWvmTiYaSP7MXpQb3KHyHHR+VzQp6ZQVphJqbSXYWdLKy3FNoptiWJr9pyd5xEBQWTPkMvF7sOuagr53dOHyvshVVLv6gIfumASV509jtsfX853f7+E9/+ggWOH1vH+c4/lz08f5flAkvQK684npDOBRSmlxQARMRO4DGgfJi4DPpNN3wl8NUpnyV0GzEwpNQFLImJRVo/OakbEPOAi4Mqsz61Z3a+329Nx4AfLq2LWbmnkwQXr+OOKTUQEZ4wbyJnjBzGif+0Rd2JlROkQqEp8uPHqTDqS9a2t4v3nHcvVrxnPL/+0im//bjGf+q8/8flfzePNJ4/kijNGcfrYgUfc/ymSdCjqTpgYBSxv93oFcFZXfVJKxYjYDAzO2h/psOyobLqzmoOBTSmlYif9dRjbuL2ZXz2zijkrt1DIB6+ZOIRzJw2hX6+qSg9N0mGqKp/jslNH8dZTRvLI4g3c0bCc/3pyBT9+bBkThtTx56eN4vLTRjFmUO9KD1WSjljdCROdfbXT8Ujwrvp01d7ZAd17699tEXENcA3A2LGVv1Tn0a4tJR5+rnSjNAIuOG4or5k4hLoKnQMh6cgTEbx64mCWrN/OyaP688zKzcxetokv3rOAL96zgGF9azjumL4cN7wv4wbXkc/FIXEpZ0k6EnTnE90KYEy716OBlV30WRERBaA/sGEfy3bWvh4YEBGFbO9EZ+vaq5TSt4BvAdTX1/fA6a9qb38On1mzpZH/nL2C5Rt3MmV4Hy4/dRQDelcfxNFJOpKUc7heTVWeM8YN4oxxg9i4vZk5q7Ywf/UWHlr0Ir9buJ7aqhwTh/ZhR3OR08YO5MRR/agpeJ6FJJWrO2HicWBydpWlFyidUH1lhz6zgKuBh4ErgPtSSikiZgG3RcSXKJ2APRl4jNIeiJfVzJa5P6sxM6t51wFuo15hrW2JBxas5YFn11FTlePt9aM5ZfQAj1+W9IoaWFfNuZNKh1Q2tbSyaF3pfiyL1m3jxl/MA0oXWRjZv5Yxg3pzTL9ahvatYVjfWnpV7xkw3JMhSZ3bZ5jIzoG4Hrib0mVcv5tSmhMRNwANKaVZwHeAH2YnWG+gFA7I+t1B6WTtInBdSqkVoLOa2So/DsyMiBuBJ7PaRMSrgP8CBgJviYh/SilN65F3QT1my84WbntsGcs27ODk0f1588kjK3ZZV0napaYqz7SR/Zk2sj8AWxpbWL5hB8s37GDZhh08/vwGWlpf2pndt6bA0H41DOpdTf/eVRTywagBvRjRv5Zh/Wqpq87v9QuS/d2rYliRdLiKlI7cI4Hq6+tTQ0NDpYdxRNnbH8jn12/nx48to7HYyp+fXtobIUm77O8H5lfyqmRtKbFpRwtrtzSydmtT9mhk044WtjUVX9a/Op9jYF0VA3tXM6iumn61VfSuydO7Ok/v6gLPrd1GdSFHVT5HTbvn6kKOQj5HVS6oyuco5EvPVfncQbtLvUFF0oGKiCdSSvWdzfMrYx2wlBIPL36RX/5pFQN7V/PecydwTL/aSg9LkrotF8GgulIwOH7EnvOKrW1ccNwwXti0k1Wbd7J+WxMbtrewcXszG3Y0s3F7M8+t28aO7EaW25uKNBX3/yrmuYBC7qWAUVPIZZeXLt13Ztd0r6o8NVV5anfP3xViSkHmQEOJe1Uk7Q/DhA5Ic7GNnz71Ak8t38QJx/TlbfVjvGmUpCNKIZ9j7ODejB3c/UvM/uiRpbQU22hubaO5w3OxNdHS2kbL7ufSdHHXdHaDy6ZiG40trWxvauXFbc00Fttoamml2Lb3IwpqCrndwaJ3dZ5Hl7zIwN7VDOhdtcfz7um66n0etiVJXTFMqGxbG1v4wcNLWblpJ687YTgXHDfUOzVLOiLt77f1uQhqsj0IPa3Y2kZjFjQaW0p7Q3Y073oUS3tI2k0/uWwTG3c0s7Xx5Ydr7VKVDwb0rmZg7yqai4le1fndh2XV5HNUV5Weawr5UtuueYU8z63bRp+aUnCpqy6Q28eeEfd8SEcWw4TKsnZLI7c+/Dzbmoq8++xxnDCiX6WHJOkQ553Ze0Yhn6NPPtfti1vs+jBebG1j084WNu1oZuOO0mFam3a0sDF7XWpvZv7qbWza0UxTsbR3pLnYusfJ6R3d8sCiPV73qspTV1OgrqYULupqSq8H9q5mcF01KzbupK6mQJ+aPP16lfaSuEdbOnwZJrTflqzfzg8feZ58LsdfnXcsowd6d1lJOtQV8jmG9KlhSJ+avfbrLPS1pURzFi6aiq27p/d8bu3QVnr94rZmVm5qZEdzkW1NxU6DSa+q/O4T2gfXVTO8Xy3D+9UyrO/exyqp8gwT2i9/XL6JO2evYFDvaq5+zXgG1XkTOkk6lPXEHqFcxO6TvaHqgGo1FUvngWxrbGFzY5GN25uzvSPNrN3SxLOrt9KanReSC/jBI0s5YUQ/Th0zgDPGDWTqiH5UF3IHvE2SeoZhQt2SUuIbDy7m9obljB9cx7vPHkvvan99JEn7p6aQp6aQ7/LLqNa2xPptTazZ0siaLY0U8jlmL93Iz/64Mls+x8mj+3P6uIG8+tjBnDlh0B5/j/YnPHk+hnTg/DSofSq2tvHpWXO47dFlnDy6P1ecPppC3m+FJEk9L5+L3Yc5wUsf+NdsaWT20o08sXQjs5dt5Hu/f55vPriYqnxw+tiBnDtpCOdMHkJrWzpo9+yQ9HKGCe3V9qYi1982m/vnr+Pa8ycyemAvr9gkSXrFDe9Xy6UnjeDSk0o3AtnZ3ErD0g38ftF6/rBoPV+6dwFfvGcBvaryTB7eh+OG92Xy8L7dPlFdUnn8F6Yurd3ayP/4/uPMXbmFGy8/kXefPc6rsUiSDgm9qvOcN3ko500eCsCG7c38YdF6vveH51mwZitPr9hMAKMH9mLKMX05bnhfRg7wCzGppxkm1KmFa7bynu89zobtzXz7L+u5+IThlR6SJEldGlRXzVtOGcnWxiJtKbFqUyPz12xh/uqt3DdvLb+dt5a6mgLHDe/DlOF9mTysb6WHLB0RDBN6mQfmr+XDP36SmkKe2z9wNiePHlDpIUmS1G25CEYN7MWogb246PjhbG8qsnDtVuav3sq8VVuZvWwTAcz64wucM2kI50wawhnjBnq/C6kMhgntllLim/+9mJt+/SzHH9OPb111BmMGeQ8JSdLhra6mwKljBnLqmIG0pcTyDTtYuHYbW3a28K3/XszXHniOmkKOM8YN5IxxAzlt7ABOHTPQy59L3WCYEFA6ke3jP3maWX9cyZtOHsG/XHGyl36VJB1xchGMG1zHuMF1XHnWWLY1FXlsyYv8YdGLPLL4Rb72wHO773MxfnBvThs7kOOO6cuU4X2YPKwvowb0IufVoqTd/LQoXti0k2t+0MDcVVv42CXH8aELJhKeoCZJOgr0qSlw0fHDuej40rmBO5qL/GnFZp5cvonZSzfy0HPr+a8nX9jdv3d1nsnD+jB2cB0jB9QyakAvRvbvxcgBvRjat4YBvauo8vLpOooYJo5y985dw8d/8jTNxTa+c3X97v9MJUk6GvWuLnDWsYM569jBu9s272hhwdqtLFizlYVrtrFw7VaeXrGJu59ppLm17WU1+tYUGFBXxaDe1fStraK2Kle6WV9VjtqqPDWFPZ9rCzmqC3kK+SAfQSEf5CIo5IJcrtSW3zUvayvkgnvnrSUXpb0tuQgiIJ8953IvtZXmwzteNYZCLkdNIefeFfUYw8RRaktjCzf8bC53PrGCE0b04yvvPI1Jw/pUeliSJO3hULgkef/eVbxq/CAWrtnGlOF9mTK8dCWotpTY3lRk884WNu5oYXtTke3NRXY0t7KzuZXtTUU2bG+m2JYotiZa2tpKz61tFNvS7sOpXik3/mLe7uldQaZXVZ5e1aVg06s6T22h9Hrt1iaqckFVIUd1PkchH1Tnc1QXSsGotmpXECpNX3nWWPrWVlFdcK/M0cYwcRR6aNF6Pnbn06zavJPrL5zERy6e7D9+SdJR50CDSi6CvrVV9K2tYvTA/V++LZVCRrGtjZbWREqJtlRqb2tLtAFtbYm0q639/JS1t+3Z1pbI6rRv31Un0ZoohZnWNpqzYFN6JLY1FtnQ2kyxNdHcrr2ltY2WYht7iz5fvGcBAIVclEJGVZ666jy9awrUVeepqynQp6ZA39oC/WqruOrV4xjWr4aaglfQOtwZJo4i25qKfOHu+Xz/oec5dkgdP/ngazhtbBn/+0mSpAOWi6C6EFRz6H+hVwoiieZiG40tbTS2tNJYbKVp13RLK43FXdNt/7+9uw2Oq7rvOP796dmSbEuyzYORwTaPxTQ8mGdoJyUtOGkadyakeGACJGSYPmSaNtMpMLxJO30R2kzzDJSBpCGlMcSkiZNpcEgDE9rU2DyDwcTChqDKYBvZki3rWf++uEdibbSWtFhae/f3mbmz95577rl7ffas/N9z7zn0Dg6zv3+I3T0DtHdmPTa5HTH3PLEVyJ5Baa6vobmhhpb6GuY1ZOvzG2uYO6t67BnOay86sRiXbZPgYKIM9A0O82/r3+DOx1+js2eAGy9dzC0rzmBWjX8NMDMzs4lJokqiqqaC+gJGzI0IegeH6e4bYm/vIN19g3T1DtHdO8ie3gG27+nllY5uhuPdiKOmqoIFjbUcM7uWrt5BTju2kbNOmMsxs2s9UMwRxMFECRsaHuHhZ9r56s+30NHVx+WnzOdvrjqdcxZ5EjozMzObOZKor6mivqaK4+bUjZtnJILu3kHe6Rlg175+duztZ2d3P6/t3Mcdj2wey9dYW8XCprqxUbQWNs2iub76gADDPRkzx8FECeruG+Q/X9jOPU9sZevOHs5e1MSXPnE2l54yv9hvzczMzGxcFRJN9TU01ddw8oIDB4XpGxzmra4+Orp66djTR8eeXtp27By7daquuiInuKjjgsXNLF3QSKVHrZp2DiZKxPBI8N9tu3j46XbWbXqL/qERzjhuNv/yyeVceeax7g40MzOzo1ZddSWL5zeweH7DWNrg8Ahvd/eNBRcdXb2s3/oOQyPBQ0+1U1ddwRnHzWHZwjksWziXZQvncPpxs6mr9m3eh9OkgglJK4CvApXAvRHxxYP21wL3A8uBd4BrIuL1tO824CZgGPjLiFh3qDIlLQFWAy3AM8AnI2LgUOcoVzu6+9jweicbtnWybtNbvN3dz9xZ1fzJ+Yu4enkrH2id6yDCzMzMSlJ1ZQWtzfW0NtePpQ2PBDv39nPSvHo2dXSzqaOLtc938EAauauyQpy8oOGA4GLpgkaOn1PnuTcKNGEwIakS+CbwB0A7sFHS2oh4OSfbTcDuiDhF0irgDuAaSWcCq4BlwELg55JOS8fkK/MO4MsRsVrS3ansu/Kd4/3+AxwNRkaCt/f28fqu/Wzb1cNzb+5mw7ZOXn9nPwCzqiu57JR5fOGPWrnit47xMGtmZmZWliorxHFz6/j48lY+vjxLiwjad/eyqaMrBRjd75nZvK66gsXzGjh5QSMnzavnhOZZB9w2NbuuukhXdOSbTM/EhUBbRGwFkLQaWAnkBhMrgS+k9TXAN5T9JL4SWB0R/cA2SW2pPMYrU9IrwBXAtSnPd1K5d+U7R0TM7IwvUxCRjdM8MJSWnPX+g7b7BofZ0zvI7p4BOvcPsKdnkHd6+nmzs5c3OnvoG3x3hs2m+mrOP6mFay86kQuXzGPZwjlUVx75w8qZmZmZzTRJLGqpZ1FLPSvOOn4sfde+fra8vY+tu/axdWcP23b1sKmji0c2vfWeCQUba6toaaihpSEbvnZ0vbG2amwOjYbaKhpqszk2qivfnexvdL26Klsf3a6oAJHNUq70PrNXjqo7SyYTTJwAvJmz3Q5clC9PRAxJ6gLmpfT1Bx17Qlofr8x5wJ6IGBonf75z7JrENRTFp/51I4+/unPKx9VUVtDcUE1zfQ2LWur5nVPnZ/cJzmvIouWmWe6KMzMzM3sf5jfWMr+xlktOnndA+vBIsGNvzrMYe3rZ3tXH7v0DdPYMsL2rj5e3d9PZM0D/0Eie0g+P0UDjgc9c/J73eaSYTDAx3v9aD+4NyJcnX/p4P6MfKv9k3weSbgZuTpv7JL06znGHw3yO4EDGDjvXd/lwXZcX13f5cF2XketKrL4v/eLEeabZSfl2TCaYaAcW5Wy3Ah158rRLqgLmAp0THDte+i6gSVJV6p3IzZ/vHAeIiHuAeyZxXe+LpKci4vzpPo8dGVzf5cN1XV5c3+XDdV1eXN8zZzI32m8ETpW0RFIN2QPVaw/Ksxa4Ia1fDfwiPcuwFlglqTaN0nQqsCFfmemYx1IZa3LQWwAAB4BJREFUpDJ/NME5zMzMzMysCCbsmUjPJ3wWWEc2jOu3ImKTpL8HnoqItcB9wHfTA9adZMEBKd9DZA9rDwF/ERHDAOOVmU55C7Ba0j8Az6ayyXcOMzMzMzMrDvnH/cJIujndUmVlwPVdPlzX5cX1XT5c1+XF9T1zHEyYmZmZmVlBPDmBmZmZmZkVpKyDCUmLJD0m6RVJmyR9LqW3SHpU0pb02pzSJelrktokvSDpvJyybkj5t0i6ISd9uaQX0zFf09E0C0kJklQp6VlJP0nbSyQ9mertwTQgAGnQgAdTvT0paXFOGbel9FclXZWTviKltUm6daavzQ4kqUnSGkmbUxu/xG27dEn66/Q9/pKk70mqc/suDZK+JWmHpJdy0qa9Lec7h02vPPX9T+m7/AVJ/yGpKWfflNpsId8LNoGIKNsFOB44L63PBn4NnAn8I3BrSr8VuCOtfwT4KdmcFxcDT6b0FmBrem1O681p3wbgknTMT4EPF/u6y3kBPg/8O/CTtP0QsCqt3w38WVr/c+DutL4KeDCtnwk8D9QCS4DXyAYRqEzrS4GalOfMYl9vOS/Ad4DPpPUaoMltuzQXsklNtwGz0vZDwI1u36WxAL8LnAe8lJM27W053zm8FKW+rwSq0vodOfU95TY71e8FLxMvZd0zERHbI+KZtL4XeIXsj9JKsv+IkF7/OK2vBO6PzHqyOTGOB64CHo2IzojYDTwKrEj75kTE/0b26bw/pyybYZJagT8E7k3bAq4A1qQsB9f16GdgDfChlH8lsDoi+iNiG9AGXJiWtojYGhEDwOqU14pA0hyyP0j3AUTEQETswW27lFUBs5TNQ1QPbMftuyRExC9577xSM9GW853DptF49R0RP4ts/jGA9WTzkMEU22yBf/dtAmUdTORK3VnnAk8Cx0bEdsgCDuCYlO0E4M2cw9pT2qHS28dJt+L4CvC3wEjangfsyfmCyq2fsTpN+7tS/ql+Bqw4lgI7gW8ru63tXkkNuG2XpIj4P+BLwG/Igogu4GncvkvZTLTlfOew4vo0WQ8STL2+C/m7bxNwMAFIagQeBv4qIroPlXWctCgg3WaYpI8COyLi6dzkcbLGBPtc10eHKrJu8rsi4lygh+w2hXxc30exdC/7SrLbHBYCDcCHx8nq9l36XLclTNLtZPOWPTCaNE62Quvbn4UClX0wIamaLJB4ICJ+kJLfTl2fpNcdKb0dWJRzeCvQMUF66zjpNvMuAz4m6XWy7s4ryHoqmtJtEXBg/YzVado/l6zbdaqfASuOdqA9Ip5M22vIggu37dL0+8C2iNgZEYPAD4BLcfsuZTPRlvOdw4ogPTT/UeC6dEsaTL2+dzH17wWbQFkHE+leuPuAVyLin3N2rQVGR3q4AfhRTvr1abSIi4Gu1PW5DrhSUnP6hexKYF3at1fSxelc1+eUZTMoIm6LiNaIWEz2YNUvIuI64DHg6pTt4Loe/QxcnfJHSl+VRn1YApxK9vDeRuDUNEpETTrH2hm4NBtHRLwFvCnp9JT0IeBl3LZL1W+AiyXVp/oYrW+379I1E2053zlshklaAdwCfCwi9ufsmlKbTe18qt8LNpFiPv1d7AW4nKwL6wXgubR8hOweuf8CtqTXlpRfwDfJRgh4ETg/p6xPkz340wZ8Kif9fOCldMw3SBMFeilqvX+Qd0dzWkr2xdMGfB+oTel1abst7V+ac/ztqT5fJWcEn/TZ+XXad3uxr7PcF+Ac4KnUvn9INoKL23aJLsDfAZtTnXyXbHQXt+8SWIDvkT0LM0j26/FNM9GW853DS1Hqu43seYbR/6vdnZN/Sm22kO8FL4dePAO2mZmZmZkVpKxvczIzMzMzs8I5mDAzMzMzs4I4mDAzMzMzs4I4mDAzMzMzs4I4mDAzMzMzs4I4mDAzs4JJ+lWx34OZmRWPh4Y1MzMzM7OCuGfCzMwKJmlfev2gpMclrZG0WdIDaUZhJF0g6VeSnpe0QdJsSXWSvi3pRUnPSvq9lPdGST+U9GNJ2yR9VtLnU571klpSvpMlPSLpaUlPSDqjeP8KZmblq6rYb8DMzErGucAyoAP4H+AySRuAB4FrImKjpDlAL/A5gIj47RQI/EzSaamcs1JZdWSz0d4SEedK+jJwPfAV4B7gTyNii6SLgDuBK2bqQs3MLONgwszMDpcNEdEOIOk5YDHQBWyPiI0AEdGd9l8OfD2lbZb0BjAaTDwWEXuBvZK6gB+n9BeBD0hqBC4Fvp86PwBqp/nazMxsHA4mzMzscOnPWR8m+xsjYLyH8zRO2njljORsj6QyK4A9EXFO4W/VzMwOBz8zYWZm02kzsFDSBQDpeYkq4JfAdSntNOBE4NXJFJh6N7ZJ+kQ6XpLOno43b2Zmh+ZgwszMpk1EDADXAF+X9DzwKNmzEHcClZJeJHum4saI6M9f0ntcB9yUytwErDy879zMzCbDQ8OamZmZmVlB3DNhZmZmZmYFcTBhZmZmZmYFcTBhZmZmZmYFcTBhZmZmZmYFcTBhZmZmZmYFcTBhZmZmZmYFcTBhZmZmZmYFcTBhZmZmZmYF+X8oyHFzZSAXrQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 936x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (13, 5))\n",
    "sns.distplot(processed_profile['income']);\n",
    "plt.title('Income Histogram - After substitution');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see it changed a bit the distribution of the data, but we were already expecting that."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age                 0\n",
       "became_member_on    0\n",
       "gender              0\n",
       "id                  0\n",
       "income              0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "processed_profile.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "No empty cells, looks like this dataset is good enough for now.\n",
    "\n",
    "Another alteration we can work on is to transform the `became_member_on` column into a `datetime` object column, which allows us to work with time windows. This means that we can get the information for how long a customer has been a member of the program. Since we don't really know when those campaigns were build, we don't have a specifict point in time to measure a customer's \"age\" as a customer. On the other hand, we can simply use the year the user signed up for the membership."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convert_to_date(date_element):\n",
    "    return datetime.strptime(str(date_element), '%Y%m%d')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "became_member_on_date = processed_profile['became_member_on'].apply(convert_to_date)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0   2017-02-12\n",
       "1   2017-07-15\n",
       "2   2018-07-12\n",
       "3   2017-05-09\n",
       "4   2017-08-04\n",
       "Name: became_member_on, dtype: datetime64[ns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "became_member_on_date.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "became_member_on_year = became_member_on_date.apply(lambda date_point: date_point.year)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "processed_profile['became_member_on_year'] = became_member_on_year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "processed_profile = processed_profile.drop(['became_member_on'], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>gender</th>\n",
       "      <th>id</th>\n",
       "      <th>income</th>\n",
       "      <th>became_member_on_year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>118</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>68be06ca386d4c31939f3a4f0e3dd783</td>\n",
       "      <td>65404.991568</td>\n",
       "      <td>2017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>55</td>\n",
       "      <td>F</td>\n",
       "      <td>0610b486422d4921ae7d2bf64640c50b</td>\n",
       "      <td>112000.000000</td>\n",
       "      <td>2017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>118</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>38fe809add3b4fcf9315a9694bb96ff5</td>\n",
       "      <td>65404.991568</td>\n",
       "      <td>2018</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>75</td>\n",
       "      <td>F</td>\n",
       "      <td>78afa995795e4d85b5d9ceeca43f5fef</td>\n",
       "      <td>100000.000000</td>\n",
       "      <td>2017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>118</td>\n",
       "      <td>Unknown</td>\n",
       "      <td>a03223e636434f42ac4c3df47e8bac43</td>\n",
       "      <td>65404.991568</td>\n",
       "      <td>2017</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age   gender                                id         income  \\\n",
       "0  118  Unknown  68be06ca386d4c31939f3a4f0e3dd783   65404.991568   \n",
       "1   55        F  0610b486422d4921ae7d2bf64640c50b  112000.000000   \n",
       "2  118  Unknown  38fe809add3b4fcf9315a9694bb96ff5   65404.991568   \n",
       "3   75        F  78afa995795e4d85b5d9ceeca43f5fef  100000.000000   \n",
       "4  118  Unknown  a03223e636434f42ac4c3df47e8bac43   65404.991568   \n",
       "\n",
       "   became_member_on_year  \n",
       "0                   2017  \n",
       "1                   2017  \n",
       "2                   2018  \n",
       "3                   2017  \n",
       "4                   2017  "
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "processed_profile.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### `transcript` dataset preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(306534, 4)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "event     object\n",
       "person    object\n",
       "time       int64\n",
       "value     object\n",
       "dtype: object"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "event     0\n",
       "person    0\n",
       "time      0\n",
       "value     0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript.isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event</th>\n",
       "      <th>person</th>\n",
       "      <th>time</th>\n",
       "      <th>value</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>76581</th>\n",
       "      <td>offer viewed</td>\n",
       "      <td>ffdefcac307f4ca99ac1ebd51470f106</td>\n",
       "      <td>186</td>\n",
       "      <td>{'offer id': '4d5c57ea9a6940dd891ad53e9dbe8da0'}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30709</th>\n",
       "      <td>transaction</td>\n",
       "      <td>885d8e7de35e4b7abb7c6f494c82c271</td>\n",
       "      <td>48</td>\n",
       "      <td>{'amount': 7.03}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>87734</th>\n",
       "      <td>transaction</td>\n",
       "      <td>25a6a2df56ba4a4a9ad898dbf083c6b5</td>\n",
       "      <td>222</td>\n",
       "      <td>{'amount': 13.44}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>106692</th>\n",
       "      <td>offer viewed</td>\n",
       "      <td>8d20cb72c43c43699ff1074fd1e7d468</td>\n",
       "      <td>306</td>\n",
       "      <td>{'offer id': '9b98b8c7a33c4b65b9aebfe6a799e6d9'}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19710</th>\n",
       "      <td>offer viewed</td>\n",
       "      <td>95dc4202ae22461d865b13da40b3d557</td>\n",
       "      <td>12</td>\n",
       "      <td>{'offer id': 'fafdcd668e3743c1bb461111dcafc2a4'}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103990</th>\n",
       "      <td>transaction</td>\n",
       "      <td>c1dc09fa7f5940a38c2f393f89cb4a50</td>\n",
       "      <td>294</td>\n",
       "      <td>{'amount': 1.1400000000000001}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>36890</th>\n",
       "      <td>transaction</td>\n",
       "      <td>b3883064435140ce8feac3c1da259948</td>\n",
       "      <td>72</td>\n",
       "      <td>{'amount': 4.93}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>180137</th>\n",
       "      <td>transaction</td>\n",
       "      <td>eae57eab7df940d898b5b32d968ecdc7</td>\n",
       "      <td>438</td>\n",
       "      <td>{'amount': 7.78}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>185351</th>\n",
       "      <td>transaction</td>\n",
       "      <td>3977011340cf47d08a23c4b3d6f6aef6</td>\n",
       "      <td>450</td>\n",
       "      <td>{'amount': 0.66}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>197364</th>\n",
       "      <td>transaction</td>\n",
       "      <td>c0bbc13872474c63a83e8b503bb88f72</td>\n",
       "      <td>486</td>\n",
       "      <td>{'amount': 35.26}</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               event                            person  time  \\\n",
       "76581   offer viewed  ffdefcac307f4ca99ac1ebd51470f106   186   \n",
       "30709    transaction  885d8e7de35e4b7abb7c6f494c82c271    48   \n",
       "87734    transaction  25a6a2df56ba4a4a9ad898dbf083c6b5   222   \n",
       "106692  offer viewed  8d20cb72c43c43699ff1074fd1e7d468   306   \n",
       "19710   offer viewed  95dc4202ae22461d865b13da40b3d557    12   \n",
       "103990   transaction  c1dc09fa7f5940a38c2f393f89cb4a50   294   \n",
       "36890    transaction  b3883064435140ce8feac3c1da259948    72   \n",
       "180137   transaction  eae57eab7df940d898b5b32d968ecdc7   438   \n",
       "185351   transaction  3977011340cf47d08a23c4b3d6f6aef6   450   \n",
       "197364   transaction  c0bbc13872474c63a83e8b503bb88f72   486   \n",
       "\n",
       "                                                   value  \n",
       "76581   {'offer id': '4d5c57ea9a6940dd891ad53e9dbe8da0'}  \n",
       "30709                                   {'amount': 7.03}  \n",
       "87734                                  {'amount': 13.44}  \n",
       "106692  {'offer id': '9b98b8c7a33c4b65b9aebfe6a799e6d9'}  \n",
       "19710   {'offer id': 'fafdcd668e3743c1bb461111dcafc2a4'}  \n",
       "103990                    {'amount': 1.1400000000000001}  \n",
       "36890                                   {'amount': 4.93}  \n",
       "180137                                  {'amount': 7.78}  \n",
       "185351                                  {'amount': 0.66}  \n",
       "197364                                 {'amount': 35.26}  "
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript.sample(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This dataset has no empty values. The `value` column has a dictionary that has different keys and values. It will be easier to have this column expanded into multiple columns to work with the events."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>offer id</th>\n",
       "      <th>amount</th>\n",
       "      <th>offer_id</th>\n",
       "      <th>reward</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9b98b8c7a33c4b65b9aebfe6a799e6d9</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0b1e1539f2cc45b7b9fa7c272da2e1d7</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2906b810c7d4411798c6938adc9daaa5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           offer id  amount offer_id  reward\n",
       "0  9b98b8c7a33c4b65b9aebfe6a799e6d9     NaN      NaN     NaN\n",
       "1  0b1e1539f2cc45b7b9fa7c272da2e1d7     NaN      NaN     NaN\n",
       "2  2906b810c7d4411798c6938adc9daaa5     NaN      NaN     NaN\n",
       "3  fafdcd668e3743c1bb461111dcafc2a4     NaN      NaN     NaN\n",
       "4  4d5c57ea9a6940dd891ad53e9dbe8da0     NaN      NaN     NaN"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "value_expanded = transcript['value'].apply(pd.Series)\n",
    "value_expanded.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have this matrix, we can include it to the rest of the transcript data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "transcript_expanded = pd.concat([transcript, value_expanded], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event</th>\n",
       "      <th>person</th>\n",
       "      <th>time</th>\n",
       "      <th>value</th>\n",
       "      <th>offer id</th>\n",
       "      <th>amount</th>\n",
       "      <th>offer_id</th>\n",
       "      <th>reward</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>97082</th>\n",
       "      <td>transaction</td>\n",
       "      <td>98ba686b0d9943279f4ef1ee9480eaff</td>\n",
       "      <td>258</td>\n",
       "      <td>{'amount': 6.82}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.82</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89483</th>\n",
       "      <td>transaction</td>\n",
       "      <td>35faf6c136ac4eff95cb298191d4dcbe</td>\n",
       "      <td>228</td>\n",
       "      <td>{'amount': 5.42}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.42</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>263829</th>\n",
       "      <td>offer viewed</td>\n",
       "      <td>eb9e4fd1814b4fba9cdff0eb00f35278</td>\n",
       "      <td>582</td>\n",
       "      <td>{'offer id': 'f19421c1d4aa40978ebb69ca19b0e20d'}</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>58191</th>\n",
       "      <td>offer received</td>\n",
       "      <td>c77f073996314357b57c8765e85d0a01</td>\n",
       "      <td>168</td>\n",
       "      <td>{'offer id': '2298d6c36e964ae4a3e7e9706d1fb8c2'}</td>\n",
       "      <td>2298d6c36e964ae4a3e7e9706d1fb8c2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2724</th>\n",
       "      <td>offer received</td>\n",
       "      <td>cd0cb9d1599c47a0bb69c49d19c2117f</td>\n",
       "      <td>0</td>\n",
       "      <td>{'offer id': '4d5c57ea9a6940dd891ad53e9dbe8da0'}</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>176516</th>\n",
       "      <td>offer completed</td>\n",
       "      <td>dcadc299e3914526acf3eac4815b7930</td>\n",
       "      <td>426</td>\n",
       "      <td>{'offer_id': '2298d6c36e964ae4a3e7e9706d1fb8c2...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2298d6c36e964ae4a3e7e9706d1fb8c2</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>228763</th>\n",
       "      <td>transaction</td>\n",
       "      <td>777a1d8f93bb4884829eead23fd6fb53</td>\n",
       "      <td>528</td>\n",
       "      <td>{'amount': 13.97}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>13.97</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>201361</th>\n",
       "      <td>transaction</td>\n",
       "      <td>83b26151b9c9487d897a67be32d2eba4</td>\n",
       "      <td>498</td>\n",
       "      <td>{'amount': 11.76}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>11.76</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>169536</th>\n",
       "      <td>transaction</td>\n",
       "      <td>6c0f89d1092545bbb522281d8337dd2b</td>\n",
       "      <td>414</td>\n",
       "      <td>{'amount': 12.12}</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12.12</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13088</th>\n",
       "      <td>offer completed</td>\n",
       "      <td>1ba58dab3cae4787ad26d8d77ab7380f</td>\n",
       "      <td>0</td>\n",
       "      <td>{'offer_id': 'f19421c1d4aa40978ebb69ca19b0e20d...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  event                            person  time  \\\n",
       "97082       transaction  98ba686b0d9943279f4ef1ee9480eaff   258   \n",
       "89483       transaction  35faf6c136ac4eff95cb298191d4dcbe   228   \n",
       "263829     offer viewed  eb9e4fd1814b4fba9cdff0eb00f35278   582   \n",
       "58191    offer received  c77f073996314357b57c8765e85d0a01   168   \n",
       "2724     offer received  cd0cb9d1599c47a0bb69c49d19c2117f     0   \n",
       "176516  offer completed  dcadc299e3914526acf3eac4815b7930   426   \n",
       "228763      transaction  777a1d8f93bb4884829eead23fd6fb53   528   \n",
       "201361      transaction  83b26151b9c9487d897a67be32d2eba4   498   \n",
       "169536      transaction  6c0f89d1092545bbb522281d8337dd2b   414   \n",
       "13088   offer completed  1ba58dab3cae4787ad26d8d77ab7380f     0   \n",
       "\n",
       "                                                    value  \\\n",
       "97082                                    {'amount': 6.82}   \n",
       "89483                                    {'amount': 5.42}   \n",
       "263829   {'offer id': 'f19421c1d4aa40978ebb69ca19b0e20d'}   \n",
       "58191    {'offer id': '2298d6c36e964ae4a3e7e9706d1fb8c2'}   \n",
       "2724     {'offer id': '4d5c57ea9a6940dd891ad53e9dbe8da0'}   \n",
       "176516  {'offer_id': '2298d6c36e964ae4a3e7e9706d1fb8c2...   \n",
       "228763                                  {'amount': 13.97}   \n",
       "201361                                  {'amount': 11.76}   \n",
       "169536                                  {'amount': 12.12}   \n",
       "13088   {'offer_id': 'f19421c1d4aa40978ebb69ca19b0e20d...   \n",
       "\n",
       "                                offer id  amount  \\\n",
       "97082                                NaN    6.82   \n",
       "89483                                NaN    5.42   \n",
       "263829  f19421c1d4aa40978ebb69ca19b0e20d     NaN   \n",
       "58191   2298d6c36e964ae4a3e7e9706d1fb8c2     NaN   \n",
       "2724    4d5c57ea9a6940dd891ad53e9dbe8da0     NaN   \n",
       "176516                               NaN     NaN   \n",
       "228763                               NaN   13.97   \n",
       "201361                               NaN   11.76   \n",
       "169536                               NaN   12.12   \n",
       "13088                                NaN     NaN   \n",
       "\n",
       "                                offer_id  reward  \n",
       "97082                                NaN     NaN  \n",
       "89483                                NaN     NaN  \n",
       "263829                               NaN     NaN  \n",
       "58191                                NaN     NaN  \n",
       "2724                                 NaN     NaN  \n",
       "176516  2298d6c36e964ae4a3e7e9706d1fb8c2     3.0  \n",
       "228763                               NaN     NaN  \n",
       "201361                               NaN     NaN  \n",
       "169536                               NaN     NaN  \n",
       "13088   f19421c1d4aa40978ebb69ca19b0e20d     5.0  "
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript_expanded.sample(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can manipulate this dataset in order to clean it. We will deal with the duplicate `offer id`/`offer_id` column and remove unecessary columns.\n",
    "\n",
    "The `offer id`/`offer_id` duplicates seem to relate to the same information, but for different events, which means that each line can never have two values for `offer id`/`offer_id` (only one per line). To treat that, we can create a single column thar receives the presented `offer id`/`offer_id` value when one exists."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "transcript_expanded['offer_id_redux'] = np.where(transcript_expanded['offer id'].isnull() & transcript_expanded['offer_id'].notnull(),\n",
    "                                                 transcript_expanded['offer_id'],\n",
    "                                                 transcript_expanded['offer id'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can drop unnecessary columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "transcript_expanded = transcript_expanded.drop(['value', \n",
    "                                                'offer id', \n",
    "                                                'offer_id'], \n",
    "                                               axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event</th>\n",
       "      <th>person</th>\n",
       "      <th>time</th>\n",
       "      <th>amount</th>\n",
       "      <th>reward</th>\n",
       "      <th>offer_id_redux</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>offer received</td>\n",
       "      <td>78afa995795e4d85b5d9ceeca43f5fef</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>9b98b8c7a33c4b65b9aebfe6a799e6d9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>offer received</td>\n",
       "      <td>a03223e636434f42ac4c3df47e8bac43</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0b1e1539f2cc45b7b9fa7c272da2e1d7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>offer received</td>\n",
       "      <td>e2127556f4f64592b11af22de27a7932</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2906b810c7d4411798c6938adc9daaa5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>offer received</td>\n",
       "      <td>8ec6ce2a7e7949b1bf142def7d0e0586</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>offer received</td>\n",
       "      <td>68617ca6246f4fbc85e91a2a49552598</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            event                            person  time  amount  reward  \\\n",
       "0  offer received  78afa995795e4d85b5d9ceeca43f5fef     0     NaN     NaN   \n",
       "1  offer received  a03223e636434f42ac4c3df47e8bac43     0     NaN     NaN   \n",
       "2  offer received  e2127556f4f64592b11af22de27a7932     0     NaN     NaN   \n",
       "3  offer received  8ec6ce2a7e7949b1bf142def7d0e0586     0     NaN     NaN   \n",
       "4  offer received  68617ca6246f4fbc85e91a2a49552598     0     NaN     NaN   \n",
       "\n",
       "                     offer_id_redux  \n",
       "0  9b98b8c7a33c4b65b9aebfe6a799e6d9  \n",
       "1  0b1e1539f2cc45b7b9fa7c272da2e1d7  \n",
       "2  2906b810c7d4411798c6938adc9daaa5  \n",
       "3  fafdcd668e3743c1bb461111dcafc2a4  \n",
       "4  4d5c57ea9a6940dd891ad53e9dbe8da0  "
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript_expanded.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Exploratory Data Analysis\n",
    "Starting with the `profile` dataset, we can check a bit of the age and income of the customers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count    17000.000000\n",
       "mean        62.531412\n",
       "std         26.738580\n",
       "min         18.000000\n",
       "25%         45.000000\n",
       "50%         58.000000\n",
       "75%         73.000000\n",
       "max        118.000000\n",
       "Name: age, dtype: float64"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "processed_profile['age'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAucAAAFNCAYAAACjaMHdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAS3klEQVR4nO3de5Bkd1nH4e/LLoQNCCEkRtkEFmrBcJFbRQxeo6B44WaBgAZIIRZqmU20sFQEFZTCQlHBRaxCEBAhiNxBQCIQLlpBEsAQro5yCQshgZAQ2AAGfv7RZ6BZd5NssjP9zvTzVE3tdJ8z0+9uTs585syve2qMEQAAYPGut+gBAACAGXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgH4GpV1ROr6h8WPQfAZifOAQ6Rqjqrqr5QVYet8WN8paq+VFWXVdXbq+p71+rxro2q+nhV3XvRcwBsROIc4BCoqh1JfjjJSHL/NX64U8cYN05y8yRnJXnhGj8eAOtEnAMcGo9McnaS5yc5ZX5DVd28ql5bVV+sqndX1ZOr6p1z24+vqjOr6pKq+khVPeSaPOAY48okL0lyh7nPdVhVPb2qPj29PX31Sn5V/U5VnV1VW6fbv1ZVH6iqG1bVjqoaVfWY6eM+U1WPPdBjV9X9p4+9dLqaf/vp/hcmuWWS105X93/7mv3zAZCIc4BD5ZFJXjS93aeqjpnb9tdJvpzkuzIL92/Ge1XdKMmZSV6c5DuT/EKSZ1XVHa/uAavqBklOzuybglWPT3JikrsmuUuSeyR5wrTtz5J8LckTquq2SZ6S5OFjjK/MffyPJbltkp9M8rv7W55SVbdLckaS30hydJLXZxbjNxhjPCLJJ5Pcb4xx4zHGn17d3wOAbxHnANdRVf1QklsleekY49wk/53kF6dtW5I8KMkfjjH2jjE+mOQFcx9+3yQfH2M8b4xx5RjjPUlenuTBV/GQf1VVlyb5UpJTkzxpbtvJSf5ojHHRGOPiadsjkmSM8Y3Mvok4LclrkvzpGOO9+3zuJ40xvjzGeH+S52X2zcK+Hprkn8cYZ44x/jfJ05JsS/IDVzEzANeAOAe47k5J8qYxxuem2y/Ot66OH51ka5IL5vaff/9WSb5/Wh5y6RTdJ2d2lf1AThtjHJHkhpnF/cuq6s7Ttlsk+cTcvp+Y7kuSjDE+nuStSXZkdkV/X/OzfdvHzvm2x5ii/4Ik269iZgCuga2LHgBgI6uqbUkekmRLVV043X1YkiOq6i5Jzk9yZZJjk3x02n7c3Ke4IMnbxhg/cbCPPUXxO6pqJbNlKOcl+XRmwf+BabdbTvetzvszSe6Z5M2ZLXP5lX0+7XFJPry/j53z6STffIWYqqrp4/asjnawfxcAZlw5B7huHpjk65k9KfOu09vtk7wjySPHGF9P8ookT6yqw6vq+MyWlqx6XZLbVdUjqur609v3rT7B8upU1T2nx16N8TMyW1N+dFUdleQPkvzDtO9RSZ6b5Jczu7J/vynW5/3+NOcdkzwqyT/u52FfmuRnq+peVXX9JI9N8tUk/z5t/2yS21yT+QH4duIc4Lo5JcnzxhifHGNcuPqW5JlJTp5eGeXUJDdNcmFmL3t4RmYxmzHG5Zld9X5YZlekL0zy1Myuvh/IM6dXQvnS9PmeMMZ4w7TtyUnOyewq+vuTvGe6L0meneTVY4zXjzE+n+TRSZ5TVTef+9xvS7KS2ZX1p40x3rTvg48xPpLk4Ul2J/lckvtl9gTQr027/Elm3yBcWlW/dTX/fgDMqTH89BFgPVXVU5N81xjjlKvdeZ1Mr9P+sSTXn16iEYAFcOUcYI1Nr2N+55q5R2ZXrF+56LkA6McTQgHW3ndktpTlFkkuSvLnSV690IkAaMmyFgAAaMKyFgAAaEKcAwBAEwe15vyoo44aO3bsWKNRAABg8zv33HM/N8Y4en/bDirOd+zYkXPOOefQTAUAAEuoqj5xoG2WtQAAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhi66IHgGWze/furKysLHqMTWXPnj1Jku3bty94EpJk586d2bVr16LHANiQxDmss5WVlbzv/A/l64cfuehRNo0tey9Lklz4Vae0Rduy95JFjwCwoflKBgvw9cOPzBXH/8yix9g0tn349Uni37SB1f8WAFw71pwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJjZUnO/evTu7d+9e9BgAwJLSIqy1rYse4GCsrKwsegQAYIlpEdbahrpyDgAAm5k4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0MTWRQ8AAADr4aSTTvrm+2edddbC5rgqrpwDAEAT4hwAgE1v/qr5/m53saGWtezZsydXXHFFTj/99EWPAtfayspKrve1segxYE1c7ytfzMrK5c7TbForKyvZtm3bosdgE7vaK+dV9ZiqOqeqzrn44ovXYyYAAFhKV3vlfIzx7CTPTpITTjhhoZf7tm/fniR5xjOescgx4Do5/fTTc+7/fHbRY8Ca+MYNb5KdtznGeZpNy0+FWGvWnAMAQBPiHACATW/fl070UooAAMBV2lCv1gIAANdW16vl81w5BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0MTWRQ9wMHbu3LnoEQCAJaZFWGsbKs537dq16BEAgCWmRVhrlrUAAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJrYuegBYRlv2XpJtH379osfYNLbs/XyS+DdtYMveS5Ics+gxADYscQ7rbOfOnYseYdPZs+fKJMn27aJw8Y5xjANcB+Ic1tmuXbsWPQIA0JQ15wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0ESNMa75zlUXJ/nE2o2zZo5K8rlFD0EbjgfmOR6Y53hglWOBeYf6eLjVGOPo/W04qDjfqKrqnDHGCYuegx4cD8xzPDDP8cAqxwLz1vN4sKwFAACaEOcAANDEssT5sxc9AK04HpjneGCe44FVjgXmrdvxsBRrzgEAYCNYlivnAADQ3qaK86o6rqreWlUfqqoPVNXp0/1HVtWZVfVf0583W/SsrJ+q2lJV762q1023b11V75qOh3+sqhssekbWR1UdUVUvq6oPT+eJezo/LK+q+s3pa8X5VXVGVd3Q+WF5VNXfVdVFVXX+3H37PR/UzF9V1UpVnVdVd1/c5KyFAxwPfzZ9vTivql5ZVUfMbXvcdDx8pKrucyhn2VRxnuTKJI8dY9w+yYlJfr2q7pDkd5O8eYxx2yRvnm6zPE5P8qG5209N8pfT8fCFJI9eyFQswjOSvHGMcXySu2R2XDg/LKGq2p7ktCQnjDHulGRLkofF+WGZPD/JT+1z34HOBz+d5LbT22OS/M06zcj6eX7+//FwZpI7jTHunOSjSR6XJFNbPizJHaePeVZVbTlUg2yqOB9jfGaM8Z7p/csz+8K7PckDkrxg2u0FSR64mAlZb1V1bJKfTfKc6XYl+fEkL5t2cTwsiaq6SZIfSfLcJBljfG2McWmcH5bZ1iTbqmprksOTfCbOD0tjjPH2JJfsc/eBzgcPSPL3Y+bsJEdU1Xevz6Ssh/0dD2OMN40xrpxunp3k2On9ByR5yRjjq2OMjyVZSXKPQzXLporzeVW1I8ndkrwryTFjjM8ks4BP8p2Lm4x19vQkv53kG9Ptmye5dO5/tk9l9g0cm99tklyc5HnTMqfnVNWN4vywlMYYe5I8LcknM4vyy5KcG+eHZXeg88H2JBfM7efYWD6/lOQN0/trejxsyjivqhsneXmS3xhjfHHR87AYVXXfJBeNMc6dv3s/u3rJouWwNcndk/zNGONuSb4cS1iW1rSW+AFJbp3kFklulNnShX05P5D42rHUqurxmS2dftHqXfvZ7ZAdD5suzqvq+pmF+YvGGK+Y7v7s6o+fpj8vWtR8rKsfTHL/qvp4kpdk9uPqp2f248it0z7HJvn0YsZjnX0qyafGGO+abr8ss1h3flhO907ysTHGxWOM/03yiiQ/EOeHZXeg88Gnkhw3t59jY0lU1SlJ7pvk5PGt1x9f0+NhU8X5tJ74uUk+NMb4i7lNr0lyyvT+KUlevd6zsf7GGI8bYxw7xtiR2RM33jLGODnJW5M8eNrN8bAkxhgXJrmgqr5nuuteST4Y54dl9ckkJ1bV4dPXjtXjwflhuR3ofPCaJI+cXrXlxCSXrS5/YfOqqp9K8jtJ7j/G2Du36TVJHlZVh1XVrTN7ovB/HLLH3Uy/hKiqfijJO5K8P99aY/x7ma07f2mSW2Z2Qv75Mca+TwJhE6uqk5L81hjjvlV1m8yupB+Z5L1JHj7G+Ooi52N9VNVdM3ty8A2S/E+SR2V2kcL5YQlV1ZOSPDSzH1e/N8kvZ7Zu1PlhCVTVGUlOSnJUks8m+cMkr8p+zgfTN3DPzOyVOfYmedQY45xFzM3aOMDx8LgkhyX5/LTb2WOMX532f3xm69CvzGwZ9Rv2/ZzXepbNFOcAALCRbaplLQAAsJGJcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOATaYqnpVVZ1bVR+oqsdM9z26qj5aVWdV1d9W1TOn+4+uqpdX1buntx9c7PQAXBW/hAhgg6mqI6ffWrgtybuT3CfJvyW5e5LLk7wlyX+OMU6tqhcnedYY451Vdcsk/zLGuP3ChgfgKm1d9AAAHLTTqurnpvePS/KIJG8bY1ySJFX1T0luN22/d5I7zH77eJLkJlX1HWOMy9dzYACuGXEOsIFU1UmZBfc9xxh7q+qsJB9JcqCr4deb9r1ifSYE4Lqw5hxgY7lpki9MYX58khOTHJ7kR6vqZlW1NcmD5vZ/U5JTV29U1V3XdVoADoo4B9hY3phka1Wdl+SPk5ydZE+SpyR5V5J/TfLBJJdN+5+W5ISqOq+qPpjkV9d/ZACuKU8IBdgEqurGY4wvTVfOX5nk78YYr1z0XAAcHFfOATaHJ1bV+5Kcn+RjSV614HkAuBZcOQcAgCZcOQcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBP/B37v18qSRnbaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 936x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (13, 5))\n",
    "sns.boxplot(processed_profile['age']);\n",
    "plt.title('Age Boxplot');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwQAAAFNCAYAAAC+M6hvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZFElEQVR4nO3de9BtZX0f8O8vnGi9JYAcCHIRY7BK0ogMAYxpQmPCxVohk2hQGk6NCc0MjqaTJlEzHUxsrm1MYrzMmIhiKyL1Si1ejkRjzAyEgxoEjHqqBo4goChqsDaYX//Y663bw7m+73ve/Z7zfD4ze/Zez3r23r99WKx3f/fzrLWquwMAAIzpOxZdAAAAsDgCAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAdqmq3lVVmxZdBwD7hkAAsE5V1Qeq6ktV9cB9/B6/sF3b6VW1bWm5u8/u7kv34LW6qr5vX9QJwL4jEACsQ1V1XJJ/maSTPG2hxawDVbVh0TUAHKgEAoD16YIk1yR5XZJvm65TVQ+vqv9ZVV+pquuq6j9X1Yfm1j+2qjZX1d1V9YmqesZKCpkfRaiq76uqv6yqe6rqC1X1pqn9g1P3v62qr1XVz07tv1hVW6darqyqR8y97hlTffdU1Sun1116n39XVX9dVX9UVXcneXFVPbqq/qKqvji99xuq6uC51/tsVf1qVd1QVf9QVa+pqiOmKU9frar3VdUhK/m3ADgQCQQA69MFSd4w3c6sqiPm1r0iyT8k+Z7MwsL/DwxV9ZAkm5NcluTwJM9M8sqq+v5VquslSd6b5JAkRyf50yTp7h+d1j++ux/a3W+qqh9P8rtJnpHkyCR/n+Tyqc7Dkrw5yQuTPDzJJ5L88HbvdWqST0+f47eT1PR6j0jyuCTHJHnxds/56SQ/meQxSf5NkncleVGSwzL7m/e8FX5+gAOOQACwzlTVjyR5ZJIruvv6JP87ybOmdQdl9qX34u6+t7tvTjI/v/+pST7b3a/t7vu6+8NJ3pLkZ3bxli+rqi8v3ZK8cxd9/3Gq7RHd/X+6+0O76Ht+kku6+8Pd/Y3Mvvw/cZoO9ZQkN3X3W7v7viQvS/L57Z5/W3f/6fQ5vt7dW7t7c3d/o7vvSvLSJD+23XP+tLvv6O7PJfmrJNd290em939bkifsol6AIQkEAOvPpiTv7e4vTMuX5VujABuTbEhy61z/+cePTHLqdl/wz89sNGFnntfdBy/dMgsVO/Nrmf1S/zdVdVNV/fwu+j4is1GBJEl3fy3JF5McNa27dW5dJ9m23fPnP1eq6vCquryqPldVX0ny3zP75X/eHXOPv76D5Yfuol6AITlIC2AdqaoHZTbF5qCqWvrF/IFJDq6qxye5Mcl9mU3X+eS0/pi5l7g1yV9290/ui/q6+/NJfnGq9UeSvK+qPtjdW3fQ/bbMAkqm/g/JbHrQ55LcntlnWFpX88tLb7fd8u9ObT/Y3V+sqnOTvHxlnwgAIwQA68u5Sb6Z5IQkJ063x2U2/eWC7v5mkrdmdpDtg6vqsZkdb7DknUkeU1U/V1XfOd1+qKoetxrFVdXTq2rpi/uXMvuC/s1p+Y4k3zvX/bIkz66qE6dTp/5OZlN4PpvkfyX5F1V17nQGoYuy61GMJHlYkq8l+XJVHZXkV1fjMwGMTiAAWF82JXltd9/S3Z9fumX2S/j505fn5yb57szm3P+3JG9M8o0k6e6vJjkjyXmZ/UL/+SS/n9kow2r4oSTXVtXXklyZ5Pnd/Zlp3YuTXDpNVXpGd1+d5D9ldgzD7UkePdWVaTrU05P8QWbTiE5IsmXpc+zEbyY5Kck9mQWKt67SZwIYWs2mbQKwv6qq30/yPd29315NuKq+I7NjCM7v7vcvuh6AkRghANjPTNcZ+MGaOSXJczI7g85+parOrKqDp+lEL8rsYOVrFlwWwHAcVAyw/3lYZtOEHpHkziR/mOQdC61oeZ6Y2XEGD0hyc5Jzu/vriy0JYDymDAEAwMBMGQIAgIEJBAAAMLB1fQzBYYcd1scdd9yiywAAgP3a9ddf/4Xu3rijdes6EBx33HHZsmXLossAAID9WlX9/c7WmTIEAAADEwgAAGBgAgEAAAxMIAAAgIEJBAAAMDCBAAAABiYQAADAwAQCAAAYmEAAAAADEwgAAGBgAgEAAAxsw6ILAACA/cFl196yrOc969RjV7mS1WWEAAAABiYQAADAwAQCAAAYmEAAAAADEwgAAGBgAgEAAAxMIAAAgIEJBAAAMDCBAAAABiYQAADAwAQCAAAYmEAAAAADEwgAAGBgAgEAAAxMIAAAgIEJBAAAMLDdBoKqOqaq3l9VH6+qm6rq+VP7oVW1uao+Nd0fMrVXVb2sqrZW1Q1VddLca22a+n+qqjbtu48FAADsiT0ZIbgvya909+OSnJbkoqo6IckLklzd3ccnuXpaTpKzkxw/3S5M8qpkFiCSXJzk1CSnJLl4KUQAAACLsdtA0N23d/eHp8dfTfLxJEclOSfJpVO3S5OcOz0+J8nre+aaJAdX1ZFJzkyyubvv7u4vJdmc5KxV/TQAAMBe2atjCKrquCRPSHJtkiO6+/ZkFhqSHD51OyrJrXNP2za17awdAABYkD0OBFX10CRvSfLL3f2VXXXdQVvvon3797mwqrZU1Za77rprT8sDAACWYY8CQVV9Z2Zh4A3d/dap+Y5pKlCm+zun9m1Jjpl7+tFJbttF+7fp7ld398ndffLGjRv35rMAAAB7aU/OMlRJXpPk49390rlVVyZZOlPQpiTvmGu/YDrb0GlJ7pmmFL0nyRlVdch0MPEZUxsAALAgG/agz5OS/FySj1XVR6e2FyX5vSRXVNVzktyS5OnTuquSPCXJ1iT3Jnl2knT33VX1kiTXTf1+q7vvXpVPAQAALMtuA0F3fyg7nv+fJE/eQf9OctFOXuuSJJfsTYEAAMC+40rFAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGtttAUFWXVNWdVXXjXNuLq+pzVfXR6faUuXUvrKqtVfWJqjpzrv2sqW1rVb1g9T8KAACwt/ZkhOB1Sc7aQfsfdfeJ0+2qJKmqE5Kcl+T7p+e8sqoOqqqDkrwiydlJTkjyzKkvAACwQBt216G7P1hVx+3h652T5PLu/kaSz1TV1iSnTOu2dvenk6SqLp/63rzXFQMAAKtmJccQPLeqbpimFB0ytR2V5Na5Ptumtp21AwAAC7TcQPCqJI9OcmKS25P84dReO+jbu2i/n6q6sKq2VNWWu+66a5nlAQAAe2JZgaC77+jub3b3PyX5s3xrWtC2JMfMdT06yW27aN/Ra7+6u0/u7pM3bty4nPIAAIA9tKxAUFVHzi3+VJKlMxBdmeS8qnpgVT0qyfFJ/ibJdUmOr6pHVdUDMjvw+Mrllw0AAKyG3R5UXFVvTHJ6ksOqaluSi5OcXlUnZjbt57NJ/n2SdPdNVXVFZgcL35fkou7+5vQ6z03yniQHJbmku29a9U8DAADslT05y9Azd9D8ml30/+0kv72D9quSXLVX1QEAAPuUKxUDAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDABAIAABiYQAAAAAMTCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDANiy6AIC1ctm1tyzrec869dhVrgQA1g8jBAAAMLDdBoKquqSq7qyqG+faDq2qzVX1qen+kKm9quplVbW1qm6oqpPmnrNp6v+pqtq0bz4OAACwN/ZkhOB1Sc7aru0FSa7u7uOTXD0tJ8nZSY6fbhcmeVUyCxBJLk5yapJTkly8FCIAAIDF2W0g6O4PJrl7u+Zzklw6Pb40yblz7a/vmWuSHFxVRyY5M8nm7r67u7+UZHPuHzIAAIA1ttxjCI7o7tuTZLo/fGo/Ksmtc/22TW07awcAABZotQ8qrh209S7a7/8CVRdW1Zaq2nLXXXetanEAAMC3W24guGOaCpTp/s6pfVuSY+b6HZ3ktl203093v7q7T+7ukzdu3LjM8gAAgD2x3EBwZZKlMwVtSvKOufYLprMNnZbknmlK0XuSnFFVh0wHE58xtQEAAAu02wuTVdUbk5ye5LCq2pbZ2YJ+L8kVVfWcJLckefrU/aokT0myNcm9SZ6dJN19d1W9JMl1U7/f6u7tD1QGAADW2G4DQXc/cyernryDvp3kop28ziVJLtmr6gAAgH3KlYoBAGBgAgEAAAxMIAAAgIEJBAAAMDCBAAAABiYQAADAwAQCAAAYmEAAAAADEwgAAGBgAgEAAAxMIAAAgIEJBAAAMDCBAAAABiYQAADAwAQCAAAYmEAAAAADEwgAAGBgAgEAAAxMIAAAgIEJBAAAMDCBAAAABiYQAADAwAQCAAAY2IZFFwCw3l127S3Lfu6zTj12FSsBgNUnEACwIssNTMISwPpgyhAAAAxMIAAAgIGZMgTsd1Yypx8A+HZGCAAAYGBGCICF8Cs/AKwPRggAAGBgAgEAAAxMIAAAgIEJBAAAMDCBAAAABiYQAADAwJx2FGAdWu5pWZ916rGrXAkABzqBAGAfcr0FANY7gQDWIb8OAwBrxTEEAAAwMIEAAAAGJhAAAMDAHEMArIiDZg8c/lsCjEkgADiA+FIPwN4yZQgAAAYmEAAAwMAEAgAAGJhAAAAAAxMIAABgYAIBAAAMbEWBoKo+W1Ufq6qPVtWWqe3QqtpcVZ+a7g+Z2quqXlZVW6vqhqo6aTU+AAAAsHyrMULwr7r7xO4+eVp+QZKru/v4JFdPy0lydpLjp9uFSV61Cu8NAACswL64MNk5SU6fHl+a5ANJfn1qf313d5Jrqurgqjqyu2/fBzUAe8kFrQBgTCsdIegk762q66vqwqntiKUv+dP94VP7UUlunXvutqkNAABYkJWOEDypu2+rqsOTbK6qv9tF39pBW9+v0yxYXJgkxx577ArLAwAAdmVFIwTdfdt0f2eStyU5JckdVXVkkkz3d07dtyU5Zu7pRye5bQev+eruPrm7T964ceNKygMAAHZj2YGgqh5SVQ9bepzkjCQ3Jrkyyaap26Yk75geX5nkgulsQ6clucfxAwAAsFgrmTJ0RJK3VdXS61zW3e+uquuSXFFVz0lyS5KnT/2vSvKUJFuT3Jvk2St4bwAAYBUsOxB096eTPH4H7V9M8uQdtHeSi5b7fgAAwOpzpWIAABiYQAAAAAPbFxcmAxbExcUAgL1lhAAAAAYmEAAAwMBMGQJgIVYyxe1Zp7qSPcBqMUIAAAADM0IAwH5nuaMLRhYA7k8ggH3IWX8AgPXOlCEAABiYEQJWxLA9AMD+TSBgGMILAMD9mTIEAAADM0IAu+HAYGAljE4C651AsBsunAMAwIHMlCEAABiYQAAAAAMzZQiAYZjPD3B/RggAAGBgAgEAAAzMlKEDyP50RqT9qVYAgAOZEQIAABiYEQKSuPgWwK7YRwIHMoFgH/IHZN/w7woAsHpMGQIAgIEJBAAAMDCBAAAABiYQAADAwAQCAAAYmEAAAAADc9pRAFiHXNEdWCtGCAAAYGACAQAADEwgAACAgQkEAAAwMIEAAAAGJhAAAMDAnHYUAA4wyz1lqdOVwpiMEAAAwMAEAgAAGJhAAAAAA3MMAQCQZPnHHqyE4xZg8YwQAADAwIwQAAALs9ajEkYk4P6MEAAAwMAEAgAAGJhAAAAAAxMIAABgYAIBAAAMTCAAAICBCQQAADCwNb8OQVWdleRPkhyU5M+7+/fWugYAYEwrue6BaxhwoFrTEYKqOijJK5KcneSEJM+sqhPWsgYAAOBb1nqE4JQkW7v700lSVZcnOSfJzWtcBwDAXnFVZQ5Uax0Ijkpy69zytiSnrnENAADrnulNrJW1DgS1g7b+tg5VFya5cFr8WlV9Yp9XtW8cluQLiy6CdcP2wBLbAvNsD8xbte3h/NV4EVbNMv97rPb+4ZE7W7HWgWBbkmPmlo9Octt8h+5+dZJXr2VR+0JVbenukxddB+uD7YEltgXm2R6YZ3tg3lpuD2t92tHrkhxfVY+qqgckOS/JlWtcAwAAMFnTEYLuvq+qnpvkPZmddvSS7r5pLWsAAAC+Zc2vQ9DdVyW5aq3fdwH2+2lPrCrbA0tsC8yzPTDP9sC8Ndseqrt33wsAADggrfUxBAAAwDoiEKxQVR1TVe+vqo9X1U1V9fyp/dCq2lxVn5ruD1l0raydqjqoqj5SVe+clh9VVddO28ObpoPqGUBVHVxVb66qv5v2E0+0fxhXVf2H6W/FjVX1xqr6Z/YP46iqS6rqzqq6ca5th/uDmnlZVW2tqhuq6qTFVc6+sJPt4b9Mfy9uqKq3VdXBc+teOG0Pn6iqM1ezFoFg5e5L8ivd/bgkpyW5qKpOSPKCJFd39/FJrp6WGcfzk3x8bvn3k/zRtD18KclzFlIVi/AnSd7d3Y9N8vjMtgv7hwFV1VFJnpfk5O7+gcxOrnFe7B9G8rokZ23XtrP9wdlJjp9uFyZ51RrVyNp5Xe6/PWxO8gPd/YNJPpnkhUkyfbc8L8n3T895ZVUdtFqFCAQr1N23d/eHp8dfzeyP/VFJzkly6dTt0iTnLqZC1lpVHZ3kXyf582m5kvx4kjdPXWwPg6iq70ryo0lekyTd/X+7+8uxfxjZhiQPqqoNSR6c5PbYPwyjuz+Y5O7tmne2Pzgnyet75pokB1fVkWtTKWthR9tDd7+3u++bFq/J7JpdyWx7uLy7v9Hdn0myNckpq1WLQLCKquq4JE9Icm2SI7r79mQWGpIcvrjKWGN/nOTXkvzTtPzwJF+e+x98W2ahkQPf9ya5K8lrpylkf15VD4n9w5C6+3NJ/muSWzILAvckuT72D6Pb2f7gqCS3zvWzbYzn55O8a3q8T7cHgWCVVNVDk7wlyS9391cWXQ+LUVVPTXJnd18/37yDrk7vNYYNSU5K8qrufkKSf4jpQcOa5oafk+RRSR6R5CGZTQvZnv0Dib8dQ6uq38hsWvoblpp20G3VtgeBYBVU1XdmFgbe0N1vnZrvWBram+7vXFR9rKknJXlaVX02yeWZTQX448yGepeu+3F0ktsWUx5rbFuSbd197bT85swCgv3DmH4iyWe6+67u/sckb03yw7F/GN3O9gfbkhwz18+2MYiq2pTkqUnO729dH2Cfbg8CwQpN88Nfk+Tj3f3SuVVXJtk0Pd6U5B1rXRtrr7tf2N1Hd/dxmR388xfdfX6S9yf5mamb7WEQ3f35JLdW1T+fmp6c5ObYP4zqliSnVdWDp78dS9uD/cPYdrY/uDLJBdPZhk5Lcs/S1CIOXFV1VpJfT/K07r53btWVSc6rqgdW1aMyO9j8b1btfV2YbGWq6keS/FWSj+Vbc8ZflNlxBFckOTazPwJP7+7tDyTiAFZVpyf5j9391Kr63sxGDA5N8pEk/7a7v7HI+lgbVXViZgeYPyDJp5M8O7MfY+wfBlRVv5nkZzObCvCRJL+Q2Txg+4cBVNUbk5ye5LAkdyS5OMnbs4P9wRQaX57ZGWXuTfLs7t6yiLrZN3ayPbwwyQOTfHHqdk13/9LU/zcyO67gvsymqL9r+9dcdi0CAQAAjMuUIQAAGJhAAAAAAxMIAABgYAIBAAAMTCAAAICBCQQAADAwgQAAAAYmEACwS1X19qq6vqpuqqoLp7bnVNUnq+oDVfVnVfXyqX1jVb2lqq6bbk9abPUA7I4LkwGwS1V16HTl1AcluS7JmUn+OslJSb6a5C+S/G13P7eqLkvyyu7+UFUdm+Q93f24hRUPwG5tWHQBAKx7z6uqn5oeH5Pk55L8ZXffnSRV9T+SPGZa/xNJTqiqped+V1U9rLu/upYFA7DnBAIAdqqqTs/sS/4Tu/veqvpAkk8k2dmv/t8x9f362lQIwEo5hgCAXfnuJF+awsBjk5yW5MFJfqyqDqmqDUl+eq7/e5M8d2mhqk5c02oB2GsCAQC78u4kG6rqhiQvSXJNks8l+Z0k1yZ5X5Kbk9wz9X9ekpOr6oaqujnJL619yQDsDQcVA7DXquqh3f21aYTgbUku6e63LbouAPaeEQIAluPFVfXRJDcm+UySty+4HgCWyQgBAAAMzAgBAAAMTCAAAICBCQQAADAwgQAAAAYmEAAAwMAEAgAAGNj/AxJRHnjhIuZuAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 936x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (13, 5))\n",
    "sns.distplot(processed_profile['age'], kde = False);\n",
    "plt.title('Age Histogram');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This histogram and the boxplot show us a great amount of outliers near 120. That was expected, since we decided not to treat the `118` values (as discussed earlier).\n",
    "The users are mostly adults, with ages between 40 and 80 and the median in a bit lower than 60 (which is also lower than the mean, that is 62.5)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "count     17000.000000\n",
       "mean      65404.991568\n",
       "std       20169.288288\n",
       "min       30000.000000\n",
       "25%       51000.000000\n",
       "50%       65404.991568\n",
       "75%       76000.000000\n",
       "max      120000.000000\n",
       "Name: income, dtype: float64"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "processed_profile['income'].describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAucAAAFNCAYAAACjaMHdAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAU+ElEQVR4nO3df6zldX3n8debmVVAagGpREbtgNh2bZcqxVVbY+zQWK2mdhO71RAVf+26TYEtm1SNu+luNrax27XKWGutP6KNRazbWmp0K5Gytv5AgSLgCustSGVQQRGEgqjMZ/8439Ez13uZueOdOe+583gk39xzvt/v+Z7PufPNlyff8z3n1hgjAADA4h226AEAAAAz4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ7AuqqqS6rqpYseB8DBSJwDrKKqvlBVv7DoceyrqhpV9c9VdVdVfbWqzq+qoxc9rl2qaus0xs2LHgtAF+IcYGP76THGUUlOSnJMkv+62OEAcH/EOcBeqKozq+rvq+r3q+rrVXVDVT1jbvmxVfWOqrp5Wv7+uWUvq6qlqrqtqi6sqhPmlo2q+vWq+nxV3VlV/72qHlVVn6iqb1TVe6vqAXPrP6uqrqyq26vq41V1yt6Mf4zxjSQXJnnM3LZOmMZz2zS+l80t+2BV/c+5+xdU1dvnfhcfq6rtVXVHVV1bVaev8ns7rKr+c1XdWFW3VNW7quqHp8UfnX7ePp3df9LevBaAjUycA+y9JyS5LslxSX4vyduqqqZlf5rkyCQ/meShSf4gSapqW5LfTfJvkzwsyY1J3rNsu09P8jNJnpjkt5K8JckZSR6R5KeSPG/a1qlJ3p7k3yd5SJI/TnJhVT1wTwOvqmOS/EqST87NPj/JTUlOSPKcJL8zF9kvTvL8qtpWVWckeXySc5b9Lq6ffhe/neQvqurYFZ76zGn6+czO3h+V5I3TsqdMP48eYxw1xvjEnl4HwEYnzgH23o1jjD8ZY9yX5J2ZxfbxVfWwJM9I8vIxxtfHGN8eY/yf6TFnJHn7GOOKMca9SV6V5ElVtXVuu68dY3xjjPHZJNck+fAY4/oxxh1JPpTkcdN6L0vyx2OMS8cY940x3pnk3syifjVXVNXtSb6a5JGZBX2q6hFJnpzkFWOMb44xrkzy1iTPT5IxxpeTvHx6nW9I8oIxxp1z270lyeun13pBZv/T8swVnv+MJK+bXs9d0+t/ruvMAVYmzgH23pd33Rhj3D3dPCqzM9y3jTG+vsJjTsjsbPmux92V5GtJtsyt85W52/escP+o6faPJvlP0yUtt0/R/YjpOVZz6hjj6CSHJ/mjJH9XVYdPj7ltWXDfuGxcH0iyKcl1Y4y/X7bdHWOMseyxK41jt9c/3d6c5Pj7GTPAIUucA/zgvpjk2FW+CeXmzKI6SVJVD8rskpQd+/g8rxljHD03HTnGOH9PDxxjfDuzM+MnZnapzM3TmH9obrVHLhvXa5J8LsnDqup5yza5Ze6Snl2PvXmFp97t9U/rfSez/wEZK6wPcEgT5wA/oDHGlzK7/ORNVXVMVf2Lqtp1PfWfJXlRVT12ujb8d5JcOsb4wj481Z8keXlVPaFmHlRVz1wW2Cuqqk1JXpTZmfjrxxhfTPLxJL9bVYdPHyx9SZJ3T+s/ZVr/BdO0varmz6o/NMnZ02v91ST/MskHV3jq85P8ZlWdWFVHTa//gjHGd5LcmmRnZteiAxBxDrBenp/k20muzex67P+YJGOMjyT5L0n+V5IvJXlUkufuyxOMMS7L7LrzNyb5epKlzD5seX8+U1V3Teu/MMm/GWPcNi17XpKtmZ3d/sskvz3GuKiqHpzkXUl+Y4yxY7qk5W1J3jF3tvzSJI/O7Fr21yR5zhjjays8/9sz+7DsR5PckOSbSc6aXs/d02M/Nl2mc3/XzgMcEmr3SwYB4P5V1ZlJXjrGePKixwKw0ThzDgAATYhzAABowmUtAADQhDPnAADQhDgHAIAm1vTnk4877rixdevW/TQUAADY+C6//PKvjjF+ZKVla4rzrVu35rLLLlufUQEAwCGoqm5cbZnLWgAAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADSxedEDgG62b9+epaWlRQ+D+7Fjx44kyZYtWxY8koPPySefnLPOOmvRwwBgFeIclllaWsqV13wu9x157KKHwio23X1HkuTL9zqErcWmu29b9BAA2AP/ZYMV3HfksbnnJ35p0cNgFUdc+8Ek8W+0Rrt+bwD05ZpzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0MRBFefbt2/P9u3bFz0MAIBDhv46sDYvegBrsbS0tOghAAAcUvTXgXVQnTkHAICNTJwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoYvOiBwAAwMHpqU996ndvX3LJJXu9bE/Lt23blp07d2bTpk35yEc+sm7bPfvss3PVVVfl1FNPzete97rve2wHzpwDANDKzp07kyT33Xffum73qquuSpJcccUV67rd9STOAQBYs/kz1Mvv39+yPd3ftm3bbstOP/30ddnu2Wefvduyc889Nx0dVJe17NixI/fcc0/OOeecRQ+FDWxpaSmHfWssehiw7g775jeytHSnYyiwJktLSzniiCMO2PPtOmu+y3qdPd911nyXrmfP93jmvKr+XVVdVlWX3XrrrQdiTAAAcEja45nzMcZbkrwlSU477bSFnk7csmVLkuQNb3jDIofBBnfOOefk8uu/suhhwLrbefiDc/JJxzuGAmvi3bYDyzXnAAC0cdhhu+fppk2b1mW7p5xyym73Tz311HXZ7noT5wAArNnyrymcv39/y/Z0/+KLL95t2fxXKf4g2z3vvPN2W+arFAEAYC/sOnu+XmfNd9l19rzrWfPkIPu2FgAA+ljpjwDtzbI9LV9+9ny9trv87HlHzpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoYvOiB7AWJ5988qKHAABwSNFfB9ZBFednnXXWoocAAHBI0V8HlstaAACgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgCXEOAABNiHMAAGhCnAMAQBObFz0A6GjT3bfliGs/uOhhsIpNd38tSfwbrdGmu29LcvyihwHA/RDnsMzJJ5+86CGwBzt2fCdJsmWL0Fyb4+3fAM2Jc1jmrLPOWvQQAIBDlGvOAQCgCXEOAABNiHMAAGhCnAMAQBPiHAAAmhDnAADQhDgHAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKAJcQ4AAE2IcwAAaEKcAwBAE+IcAACaEOcAANCEOAcAgCbEOQAANCHOAQCgiRpj7P3KVbcmuXH/DYd1clySry56EBz07EesF/sS68W+xHrosB/96BjjR1ZasKY45+BQVZeNMU5b9Dg4uNmPWC/2JdaLfYn10H0/clkLAAA0Ic4BAKAJcb4xvWXRA2BDsB+xXuxLrBf7Euuh9X7kmnMAAGjCmXMAAGhCnDdXVZuq6h+q6gPT/ROr6tKq+nxVXVBVD5jmP3C6vzQt3zq3jVdN86+rql+cm//0ad5SVb3yQL82DpyqOrqq3ldV11bV56rqSVV1bFVdNO1LF1XVMdO6VVXnTfvFVVV16tx2Xjit//mqeuHc/J+pqqunx5xXVbWI18n+VVW/WVWfraprqur8qjrcMYm9UVVvr6pbquqauXn7/Ri02nNwcFplP/of03/brqqqv6yqo+eWrelYsy/Hs/1ijGFqPCU5N8mfJfnAdP+9SZ473X5zkv8w3f71JG+ebj83yQXT7cck+UySByY5Mck/Jtk0Tf+Y5KQkD5jWecyiX69pv+1H70zy0un2A5IcneT3krxymvfKJK+dbv9Skg8lqSRPTHLpNP/YJNdPP4+Zbh8zLftUkidNj/lQkmcs+jWb1n0f2pLkhiRHTPffm+RMxyTTXu4/T0lyapJr5ubt92PQas9hOjinVfajpyXZPN1+7dx+tOZjzVqPZ/trcua8sap6eJJnJnnrdL+SbEvyvmmVdyb5len2s6f7mZafPq3/7CTvGWPcO8a4IclSkn89TUtjjOvHGN9K8p5pXTaYqnpwZge0tyXJGONbY4zbs/s+s3xfeteY+WSSo6vqYUl+MclFY4zbxhhfT3JRkqdPyx48xvjEmB253jW3LTaWzUmOqKrNSY5M8qU4JrEXxhgfTXLbstkH4hi02nNwEFppPxpjfHiM8Z3p7ieTPHy6vaZjzT421n4hznt7fZLfSrJzuv+QJLfP7YQ3ZXY2K9PPLybJtPyOaf3vzl/2mNXms/GclOTWJO+o2SVSb62qByU5fozxpSSZfj50Wn+t+8yW6fby+WwgY4wdSX4/yT9lFuV3JLk8jknsuwNxDFrtOdiYXpzZOyfJ2vejfWms/UKcN1VVz0pyyxjj8vnZK6w69rBsrfPZeDZn9jbgH40xHpfknzN7e3c19iW+z3St7rMze3v4hCQPSvKMFVZ1TOIHZd9hzarq1Um+k+Tdu2atsNq+7kcHdB8T5339XJJfrqovZPaWy7bMzqQfPb2lnMzeurl5un1TkkckybT8hzN76+e785c9ZrX5bDw3JblpjHHpdP99mcX6V6a3gzP9vGVu/bXsMzfle28jzs9nY/mFJDeMMW4dY3w7yV8k+dk4JrHvDsQxaLXnYAOZPhz8rCRnTJc2JWvfj76atR/P9gtx3tQY41VjjIePMbZm9uGDi8cYZyT52yTPmVZ7YZK/mm5fON3PtPziaQe9MMlzp08an5jk0Zl9cObTSR49fTL5AdNzXHgAXhoH2Bjjy0m+WFU/Ps06Pcn/ze77zPJ96QXTNyY8Mckd09vBf5PkaVV1zHQW9WlJ/mZadmdVPXG6Bu8Fc9ti4/inJE+sqiOnf+dd+5FjEvvqQByDVnsONoiqenqSVyT55THG3XOL1nSsmY5Paz2e7R/789OmpnX7dPJT871vazlp2rmWkvx5kgdO8w+f7i9Ny0+ae/yrM/tk8nWZ+xaNzD4R//+mZa9e9Os07dd96LFJLktyVZL3Z/ZNBw9J8pEkn59+HjutW0n+cNovrk5y2tx2XjztY0tJXjQ3/7Qk10yPeWOmP3Bm2lhTkv+W5Nrp3/pPM/sWBMck097sO+dn9lmFb2d2FvIlB+IYtNpzmA7OaZX9aCmz68GvnKY3z62/pmPNvhzP9sfkL4QCAEATLmsBAIAmxDkAADQhzgEAoAlxDgAATYhzAABoQpwDNFFVH1/0GABYLF+lCAAATThzDtBEVd01/XxqVV1SVe+rqmur6t3TXz5MVT2+qj5eVZ+pqk9V1Q9V1eFV9Y6qurqq/qGqfn5a98yqen9V/XVV3VBVv1FV507rfLKqjp3We1RV/e+quryq/q6qfmJxvwWAQ9vmRQ8AgBU9LslPJrk5yceS/FxVfSrJBUl+bYzx6ap6cJJ7kpyTJGOMfzWF9Yer6sem7fzUtK3DM/vrdq8YYzyuqv4gsz9z/vokb0ny8jHG56vqCUnelGTbgXqhAHyPOAfo6VNjjJuSpKquTLI1yR1JvjTG+HSSjDG+MS1/cpLt07xrq+rGJLvi/G/HGHcmubOq7kjy19P8q5OcUlVHJfnZJH8+nZxPkgfu59cGwCrEOUBP987dvi+z43UlWemDQrXCvJW2s3Pu/s5pm4cluX2M8dh9HyoA68U15wAHj2uTnFBVj0+S6XrzzUk+muSMad6PJXlkkuv2ZoPT2fcbqupXp8dXVf30/hg8AHsmzgEOEmOMbyX5tSTbq+ozSS7K7FryNyXZVFVXZ3ZN+pljjHtX39L3OSPJS6ZtfjbJs9d35ADsLV+lCAAATThzDgAATYhzAABoQpwDAEAT4hwAAJoQ5wAA0IQ4BwCAJsQ5AAA0Ic4BAKCJ/w+/u+jf3kUDhwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 936x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (13, 5))\n",
    "sns.boxplot(processed_profile['income']);\n",
    "plt.title('Income Boxplot');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We already saw the histogram for this data,the boxplot shows the distribution is a bit skewed. Mean and median seem close (around $\\$65,000$) and most of the data is between $\\$60,000$ and $\\$75,000$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxEAAAFNCAYAAAB/tOLDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de7hddX3n8fdHIiqiEuSAmEBDS6qCrWhTwEu90XKzNVRBY22JyDR1Sm3t2LHaZ9ooyDw62jLqCJ20RIJjRaQiqWVqM6CoVYGAeAGkiaAQEyGYgAqKDX7nj/07shPPbYWzs3OS9+t59rPX+q7f+q3vjsdwPlmXnapCkiRJkqbqEcNuQJIkSdLMYoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkkSS1yT53ADnf2uS/9OWD07ygyR7TNPcf5vkL9vyC5Osm45523y/luSW6ZpPknYVhghJ2kklWZTk6iT3JbmrLf9hkgy7t4ejqm6vqr2r6sGJxk012FTV66rqrOnoLUklObRv7s9W1VOmY25J2pUYIiRpJ5TkjcB7gHcBTwIOAF4HPBfYc4it/YzpOqMw044tSbszQ4Qk7WSSPAE4E/jDqrqkqr5fPV+qqldX1QNt3KOSvDvJ7UnubJf1PKZte2GSdUne2M5ibEhyWt8xnphkZZLvJbkG+IVtenhqklVJNiW5Jckr+rZdkOS8JJcnuQ940Rif4ZAkVyX5fpJVwH592+a1f/Gf1dZfk+TWNva2JK9O8jTgb4Fnt0uf7hnv2K329m2O/xdJ7k7yzSSv7qt/Osl/6lv/6dmOJJ9p5S+3Y75y28ujkjytzXFPkhuTvHSbP5f3J/nn9lmuTrLVn6sk7SoMEZK083k28CjgsknGvRP4ReAI4FBgDvBXfdufBDyh1U8H3p9kdtv2fuBHwIHAa9sLgCSPBVYB/wDsD7wKODfJ4X1z/w5wNvA4YKxLjv4BuI5eeDgLWDzWB2jHei9wQlU9DngOcENV3UzvzMsX2qVP+3Q49pPacee04y5LMuklSVX1/Lb4jHbMj2zT6yOBfwL+ld6fy+uBD20z96uAtwGzgbWtT0na5RgiJGnnsx9wd1VtGS0k+Xz71+8fJnl+uy/i94E/rapNVfV94L8Di/rm+Q/gzKr6j6q6HPgB8JR2CdDLgb+qqvuq6mvAir79fhP4ZlV9oKq2VNX1wD8CJ/eNuayq/q2qflJVP+pvPsnBwK8Cf1lVD1TVZ+j98j2enwBPT/KYqtpQVTdO8ucz7rH7jB77KuCfgVeMM66Lo4G9gXdU1Y+r6krgE/SCw6iPVdU17X+7D9ELeJK0yzFESNLO57vAfqOX+wBU1XPav8Z/l97f3SPAXsB1LVzcA/xLq/90nv4gAtxP75fgEWAWcEfftm/1Lf8ccNTovG3uV9P7F/5R/ftu68nA5qq6b5z5f6qNeSW9sw4b2qVAT51g7smOzTjHfvIk+0zFk4E7quon28w9p2/9O33Lo3/ekrTLMURI0s7nC8ADwMIJxtwN/BA4vKr2aa8nVNVUfmndCGwBDuqrHdy3fAdwVd+8+7TLe/5z35iaYP4NwOx2qdJY82+lqj5ZVb9B79KqrwN/N8kxJjo24xx7fVu+j174GtUfjCazHjgoSf9/Ow8Gvt1hDknaJRgiJGknU1X30Luu/twkJyfZO8kjkhwBPLaN+Qm9X7bPSbI/QJI5SY6bwvwPAh8D3ppkrySHsfU9C58AfjHJ7yV5ZHv9arvZeSr9fwtYDbwtyZ5Jngf81lhjkxyQ5KXtl/4H6F1yNfro1zuBuUm252lUo8f+NXqXZ3201W8AXtY+96H07hXpdyfw8+PMeTW9EPKm9mfywva5LtqO/iRpRjNESNJOqKr+B/BfgDcBd9H75fZ/A38OfL4N+3N6N+9+Mcn3gP8HTPU7Df6I3qU23wEuAD7Qd+zvA8fSu79ifRvzTno3e0/V7wBHAZuApcCF44x7BPDGdpxNwAuAP2zbrgRuBL6T5O4Ox/4OsLnN+SHgdVX19bbtHODH9P48V7Tt/d4KrGiXcW11H0VV/Rh4KXACvTNB5wKn9s0tSbuNVE12VliSJEmSHuKZCEmSJEmdGCIkSZIkdWKIkCRJktSJIUKSJElSJ4YISZIkSZ3MmnzIzLPffvvVvHnzht2GJEmSNKNdd911d1fVyLb1XTJEzJs3j9WrVw+7DUmSJGlGS/KtsepeziRJkiSpE0OEJEmSpE4MEZIkSZI6MURIkiRJ6sQQIUmSJKkTQ4QkSZKkTgwRkiRJkjoxREiSJEnqxBAhSZIkqRNDhCRJkqRODBGSJEmSOpk17AYkSXo4nvu+5w67Bc1w//b6fxt2C9KM45kISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkiRJUieGCEmSJEmdGCIkSZIkdTLQEJHkT5PcmORrST6c5NFJDklydZI1ST6SZM829lFtfW3bPq9vnre0+i1Jjhtkz5IkSZImNrAQkWQO8MfAgqp6OrAHsAh4J3BOVc0HNgOnt11OBzZX1aHAOW0cSQ5r+x0OHA+cm2SPQfUtSZIkaWKDvpxpFvCYJLOAvYANwIuBS9r2FcBJbXlhW6dtPyZJWv2iqnqgqm4D1gJHDrhvSZIkSeMYWIioqm8D7wZupxce7gWuA+6pqi1t2DpgTlueA9zR9t3Sxj+xvz7GPpIkSZJ2sEFezjSb3lmEQ4AnA48FThhjaI3uMs628erbHm9JktVJVm/cuHH7mpYkSZI0qUFezvTrwG1VtbGq/gP4GPAcYJ92eRPAXGB9W14HHATQtj8B2NRfH2Ofn6qqZVW1oKoWjIyMDOLzSJIkSWKwIeJ24Ogke7V7G44BbgI+BZzcxiwGLmvLK9s6bfuVVVWtvqg9vekQYD5wzQD7liRJkjSBWZMP2T5VdXWSS4DrgS3Al4BlwD8DFyV5e6ud33Y5H/hgkrX0zkAsavPcmORiegFkC3BGVT04qL4lSZIkTWxgIQKgqpYCS7cp38oYT1eqqh8Bp4wzz9nA2dPeoCRJkqTO/MZqSZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkiRJUieGCEmSJEmdGCIkSZIkdWKIkCRJktSJIUKSJElSJ4YISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkiRJUieGCEmSJEmdDCxEJHlKkhv6Xt9L8oYk+yZZlWRNe5/dxifJe5OsTfKVJM/qm2txG78myeJB9SxJkiRpcgMLEVV1S1UdUVVHAL8C3A9cCrwZuKKq5gNXtHWAE4D57bUEOA8gyb7AUuAo4Ehg6WjwkCRJkrTj7ajLmY4BvlFV3wIWAitafQVwUlteCFxYPV8E9klyIHAcsKqqNlXVZmAVcPwO6luSJEnSNnZUiFgEfLgtH1BVGwDa+/6tPge4o2+fda02Xl2SJEnSEAw8RCTZE3gp8NHJho5Rqwnq2x5nSZLVSVZv3Lixe6OSJEmSpmRHnIk4Abi+qu5s63e2y5Ro73e1+jrgoL795gLrJ6hvpaqWVdWCqlowMjIyzR9BkiRJ0qgdESJexUOXMgGsBEafsLQYuKyvfmp7StPRwL3tcqdPAscmmd1uqD621SRJkiQNwaxBTp5kL+A3gD/oK78DuDjJ6cDtwCmtfjlwIrCW3pOcTgOoqk1JzgKubePOrKpNg+xbkiRJ0vgGGiKq6n7gidvUvkvvaU3bji3gjHHmWQ4sH0SPkiRJkrrxG6slSZIkdWKIkCRJktSJIUKSJElSJ4YISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkiRJUieGCEmSJEmdGCIkSZIkdWKIkCRJktSJIUKSJElSJ4YISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUyaxBTp5kH+DvgacDBbwWuAX4CDAP+CbwiqranCTAe4ATgfuB11TV9W2excB/a9O+vapWDLJvaVd2+5m/NOwWNMMd/FdfHXYLkqQhG/SZiPcA/1JVTwWeAdwMvBm4oqrmA1e0dYATgPnttQQ4DyDJvsBS4CjgSGBpktkD7luSJEnSOAYWIpI8Hng+cD5AVf24qu4BFgKjZxJWACe15YXAhdXzRWCfJAcCxwGrqmpTVW0GVgHHD6pvSZIkSRMb5JmInwc2Ah9I8qUkf5/kscABVbUBoL3v38bPAe7o239dq41XlyRJkjQEgwwRs4BnAedV1TOB+3jo0qWxZIxaTVDfeudkSZLVSVZv3Lhxe/qVJEmSNAWDDBHrgHVVdXVbv4ReqLizXaZEe7+rb/xBffvPBdZPUN9KVS2rqgVVtWBkZGRaP4gkSZKkhwwsRFTVd4A7kjyllY4BbgJWAotbbTFwWVteCZyanqOBe9vlTp8Ejk0yu91QfWyrSZIkSRqCgT7iFXg98KEkewK3AqfRCy4XJzkduB04pY29nN7jXdfSe8TraQBVtSnJWcC1bdyZVbVpwH1LkiRJGsdAQ0RV3QAsGGPTMWOMLeCMceZZDiyf3u4kSZIkbQ+/sVqSJElSJ4YISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkiRJUieGCEmSJEmdGCIkSZIkdWKIkCRJktSJIUKSJElSJ4YISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkiRJUicDDRFJvpnkq0luSLK61fZNsirJmvY+u9WT5L1J1ib5SpJn9c2zuI1fk2TxIHuWJEmSNLEdcSbiRVV1RFUtaOtvBq6oqvnAFW0d4ARgfnstAc6DXugAlgJHAUcCS0eDhyRJkqQdbxiXMy0EVrTlFcBJffULq+eLwD5JDgSOA1ZV1aaq2gysAo7f0U1LkiRJ6hl0iCjgX5Ncl2RJqx1QVRsA2vv+rT4HuKNv33WtNl5dkiRJ0hDMGvD8z62q9Un2B1Yl+foEYzNGrSaob71zL6QsATj44IO3p1dJkiRJUzDQMxFVtb693wVcSu+ehjvbZUq097va8HXAQX27zwXWT1Df9ljLqmpBVS0YGRmZ7o8iSZIkqRlYiEjy2CSPG10GjgW+BqwERp+wtBi4rC2vBE5tT2k6Gri3Xe70SeDYJLPbDdXHtpokSZKkIRjk5UwHAJcmGT3OP1TVvyS5Frg4yenA7cApbfzlwInAWuB+4DSAqtqU5Czg2jbuzKraNMC+JUmSJE1gYCGiqm4FnjFG/bvAMWPUCzhjnLmWA8unu0dJkiRJ3U3pcqYkV0ylJkmSJGnXN+GZiCSPBvYC9mv3I4w+KenxwJMH3JskSZKkndBklzP9AfAGeoHhOh4KEd8D3j/AviRJkiTtpCYMEVX1HuA9SV5fVe/bQT1JkiRJ2olN6cbqqnpfkucA8/r3qaoLB9SXJEmSpJ3UlEJEkg8CvwDcADzYygUYIiRJkqTdzFQf8boAOKw9hlWSJEnSbmyq31j9NeBJg2xEkiRJ0sww1TMR+wE3JbkGeGC0WFUvHUhXkiRJknZaUw0Rbx1kE5IkSZJmjqk+nemqQTciSZIkaWaY6tOZvk/vaUwAewKPBO6rqscPqjFJkiRJO6epnol4XP96kpOAIwfSkSRJkqSd2lSfzrSVqvo48OJp7kWSJEnSDDDVy5le1rf6CHrfG+F3RkiSJEm7oak+nem3+pa3AN8EFk57N5IkSZJ2elO9J+K0QTciSZIkaWaY0j0RSeYmuTTJXUnuTPKPSeYOujlJkiRJO5+p3lj9AWAl8GRgDvBPrSZJkiRpNzPVEDFSVR+oqi3tdQEwMsC+JEmSJO2kphoi7k7yu0n2aK/fBb47yMYkSZIk7ZymGiJeC7wC+A6wATgZ8GZrSZIkaTc01RBxFrC4qkaqan96oeKtU9mxnbn4UpJPtPVDklydZE2SjyTZs9Uf1dbXtu3z+uZ4S6vfkuS4Dp9PkiRJ0jSbaoj45araPLpSVZuAZ05x3z8Bbu5bfydwTlXNBzYDp7f66cDmqjoUOKeNI8lhwCLgcOB44Nwke0zx2JIkSZKm2VRDxCOSzB5dSbIvU/iOifYY2JcAf9/WA7wYuKQNWQGc1JYXtnXa9mPa+IXARVX1QFXdBqwFjpxi35IkSZKm2VS/sfqvgc8nuQQoevdHnD2F/f4n8CbgcW39icA9VbWlra+j98hY2vsdAFW1Jcm9bfwc4It9c/bvI0mSJGkHm9KZiKq6EHg5cCewEXhZVX1won2S/CZwV1Vd118ea/pJtk20T//xliRZnWT1xo0bJ2pNkiRJ0sMw1TMRVNVNwE0d5n4u8NIkJwKPBh5P78zEPklmtbMRc4H1bfw64CBgXZJZwBOATX31Uf379Pe3DFgGsGDBgp8JGZIkSZKmx1Tvieisqt5SVXOrah69G6OvrKpXA5+i94hYgMXAZW15ZVunbb+yqqrVF7WnNx0CzAeuGVTfkiRJkiY25TMR0+jPgYuSvB34EnB+q58PfDDJWnpnIBYBVNWNSS6mdxZkC3BGVT2449uWJEmSBDsoRFTVp4FPt+VbGePpSlX1I+CUcfY/m6ndyC1JkiRpwAZ2OZMkSZKkXZMhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkiRJUieGCEmSJEmdGCIkSZIkdWKIkCRJktSJIUKSJElSJ4YISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkiRJUieGCEmSJEmdGCIkSZIkdWKIkCRJktTJwEJEkkcnuSbJl5PcmORtrX5IkquTrEnykSR7tvqj2vratn1e31xvafVbkhw3qJ4lSZIkTW6QZyIeAF5cVc8AjgCOT3I08E7gnKqaD2wGTm/jTwc2V9WhwDltHEkOAxYBhwPHA+cm2WOAfUuSJEmawMBCRPX8oK0+sr0KeDFwSauvAE5qywvbOm37MUnS6hdV1QNVdRuwFjhyUH1LkiRJmthA74lIskeSG4C7gFXAN4B7qmpLG7IOmNOW5wB3ALTt9wJP7K+PsY8kSZKkHWygIaKqHqyqI4C59M4ePG2sYe0942wbr76VJEuSrE6yeuPGjdvbsiRJkqRJ7JCnM1XVPcCngaOBfZLMapvmAuvb8jrgIIC2/QnApv76GPv0H2NZVS2oqgUjIyOD+BiSJEmSGOzTmUaS7NOWHwP8OnAz8Cng5DZsMXBZW17Z1mnbr6yqavVF7elNhwDzgWsG1bckSZKkic2afMh2OxBY0Z6k9Ajg4qr6RJKbgIuSvB34EnB+G38+8MEka+mdgVgEUFU3JrkYuAnYApxRVQ8OsG9JkiRJExhYiKiqrwDPHKN+K2M8XamqfgScMs5cZwNnT3ePkiRJkrrzG6slSZIkdWKIkCRJktSJIUKSJElSJ4YISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJnRgiJEmSJHViiJAkSZLUiSFCkiRJUieGCEmSJEmdGCIkSZIkdWKIkCRJktSJIUKSJElSJ4YISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpIkSVInhghJkiRJncwadgM7s1/5rxcOuwXtAq5716nDbkGSJGlaeSZCkiRJUicDCxFJDkryqSQ3J7kxyZ+0+r5JViVZ095nt3qSvDfJ2iRfSfKsvrkWt/FrkiweVM+SJEmSJjfIMxFbgDdW1dOAo4EzkhwGvBm4oqrmA1e0dYATgPnttQQ4D3qhA1gKHAUcCSwdDR6SJEmSdryBhYiq2lBV17fl7wM3A3OAhcCKNmwFcFJbXghcWD1fBPZJciBwHLCqqjZV1WZgFXD8oPqWJEmSNLEdck9EknnAM4GrgQOqagP0ggawfxs2B7ijb7d1rTZeXZIkSdIQDDxEJNkb+EfgDVX1vYmGjlGrCerbHmdJktVJVm/cuHH7mpUkSZI0qYGGiCSPpBcgPlRVH2vlO9tlSrT3u1p9HXBQ3+5zgfUT1LdSVcuqakFVLRgZGZneDyJJkiTppwb5dKYA5wM3V9Xf9G1aCYw+YWkxcFlf/dT2lKajgXvb5U6fBI5NMrvdUH1sq0mSJEkagkF+2dxzgd8Dvprkhlb7C+AdwMVJTgduB05p2y4HTgTWAvcDpwFU1aYkZwHXtnFnVtWmAfYtSZIkaQIDCxFV9TnGvp8B4JgxxhdwxjhzLQeWT193kiRJkraX31gtSZIkqRNDhCRJkqRODBGSJEmSOjFESJIkSerEECFJkiSpE0OEJEmSpE4MEZIkSZI6MURIkiRJ6sQQIUmSJKkTQ4QkSZKkTgwRkiRJkjoxREiSJEnqxBAhSZIkqRNDhCRJkqRODBGSJEmSOjFESJIkSerEECFJkiSpE0OEJEmSpE4MEZIkSZI6MURIkiRJ6sQQIUmSJKmTgYWIJMuT3JXka321fZOsSrKmvc9u9SR5b5K1Sb6S5Fl9+yxu49ckWTyofiVJkiRNzSDPRFwAHL9N7c3AFVU1H7iirQOcAMxvryXAedALHcBS4CjgSGDpaPCQJEmSNBwDCxFV9Rlg0zblhcCKtrwCOKmvfmH1fBHYJ8mBwHHAqqraVFWbgVX8bDCRJEmStAPt6HsiDqiqDQDtff9WnwPc0TduXauNV5ckSZI0JDvLjdUZo1YT1H92gmRJktVJVm/cuHFam5MkSZL0kB0dIu5slynR3u9q9XXAQX3j5gLrJ6j/jKpaVlULqmrByMjItDcuSZIkqWdHh4iVwOgTlhYDl/XVT21PaToauLdd7vRJ4Ngks9sN1ce2miRJkqQhmTWoiZN8GHghsF+SdfSesvQO4OIkpwO3A6e04ZcDJwJrgfuB0wCqalOSs4Br27gzq2rbm7UlSZIk7UADCxFV9apxNh0zxtgCzhhnnuXA8mlsTZIkSdLDsLPcWC1JkiRphjBESJIkSerEECFJkiSpE0OEJEmSpE4MEZIkSZI6MURIkiRJ6sQQIUmSJKkTQ4QkSZKkTgwRkiRJkjoxREiSJEnqxBAhSZIkqRNDhCRJkqRODBGSJEmSOjFESJIkSerEECFJkiSpE0OEJEmSpE4MEZIkSZI6MURIkiRJ6sQQIUmSJKmTWcNuQJIkSVu76vkvGHYLmuFe8JmrBjq/ZyIkSZIkdWKIkCRJktTJjAkRSY5PckuStUnePOx+JEmSpN3VjAgRSfYA3g+cABwGvCrJYcPtSpIkSdo9zYgQARwJrK2qW6vqx8BFwMIh9yRJkiTtlmZKiJgD3NG3vq7VJEmSJO1gM+URrxmjVlsNSJYAS9rqD5LcMvCuBLAfcPewm9iZ5d2Lh92CuvFnejJLx/orWTs5f64nkD/2Z3oG8md6Mpm2n+ufG6s4U0LEOuCgvvW5wPr+AVW1DFi2I5sSJFldVQuG3Yc0XfyZ1q7In2vtavyZHr6ZcjnTtcD8JIck2RNYBKwcck+SJEnSbmlGnImoqi1J/gj4JLAHsLyqbhxyW5IkSdJuaUaECICquhy4fNh96Gd4CZl2Nf5Ma1fkz7V2Nf5MD1mqavJRkiRJktTMlHsiJEmSJO0kDBG7iSTzknxtm9pbk/zZBPu8Jsn/Gnx30vAleTDJDX2vecPuSdpeSSrJB/vWZyXZmOQTw+xLeriSzE1yWZI1Sb6R5D3toTvawQwRktTzw6o6ou/1zWE3JD0M9wFPT/KYtv4bwLeH2I/0sCUJ8DHg41U1H/hFYG/g7KE2tpsyRIgkn07yziTXJPn3JL82xpiXJPlCkv2SXJDkvUk+n+TWJCe3MUnyriRfS/LVJK9s9XOTvLQtX5pkeVs+Pcnb21mSm5P8XZIbk/xr33/4JEnb5/8CL2nLrwI+PMRepOnwYuBHVfUBgKp6EPhT4LVJ9hpqZ7shQ4RGzaqqI4E3AEv7NyT5beDNwIlVNfrtkAcCzwN+E3hHq70MOAJ4BvDrwLuSHAh8BhgNJnOAw9ry84DPtuX5wPur6nDgHuDl0/rppMk9pu9SpkuH3Yw0DS4CFiV5NPDLwNVD7kd6uA4HrusvVNX3gNuBQ4fS0W5sxjziVQ/beI/hGq1/rL1fB8zr2/4iYAFwbPs/6qiPV9VPgJuSHNBqzwM+3P5l4M4kVwG/Si8ovCHJYcBNwOwWLp4N/DHwROC2qrphnB6kHeGHVXXEsJuQpktVfaXd2/MqfES6dg1h7N9nxqtrgDwTsfv4LjB7m9q+wOiZhQfa+4NsHS5vBR5H77rDfg/0LWeb961U1bfbsY+nd1bis8ArgB9U1ffHmG/bHiRJ22cl8G68lEm7hhvp/cPmTyV5PHAQ8I2hdLQbM0TsJqrqB8CGJMcAJNmX3i/1n5tk12/Ru0zpwiSHTzL2M8Ark+yRZAR4PnBN2/YFepdKjYaIP+OhS5kkSYOxHDizqr467EakaXAFsFeSUwGS7AH8NXBBVd0/1M52Q4aI3cupwH9LcgNwJfC2qpo0uVfVLcCrgY8m+YUJhl4KfAX4cpv/TVX1nbbts/Tuu1gLXE/vLIghQpIGqKrWVdV7ht2HNB2q9w3Jvw2ckmQN8O/Aj4C/GGpjuym/sVqSJElSJ56JkCRJktSJIUKSJElSJ4YISZIkSZ0YIiRJkiR1YoiQJEmS1IkhQpK000hyQZKTh92HJGlihghJ0oyVxG+3l6Qh8C9fSdJ2SfKX9L6I8g7gbuA6el86+X5gBLgf+P2q+nqSC4DvAQuAJ9H7MspLkgR4H/Bi4DYgffP/CvA3wN5t/tdU1YYknwY+DzwXWEnvG2slSTuQIUKS1FmSBcDLgWfS+2/J9fRCxDLgdVW1JslRwLn0AgLAgcDzgKfS++X/EnrfPvsU4JeAA4CbgOVJHkkvXCysqo1JXgmcDby2zbVPVb1g4B9UkjQmQ4QkaXs8D7isqn4IkOSfgEcDzwE+2jvBAMCj+vb5eFX9BLgpyQGt9nzgw1X1IM5xWfEAAAEfSURBVLA+yZWt/hTg6cCqNtcewIa+uT4y/R9JkjRVhghJ0vbIGLVHAPdU1RHj7PPAOPvXOPPfWFXPHmeu+yZvUZI0KN5YLUnaHp8DfivJo5PsDbyE3j0QtyU5BSA9z5hkns8Ai5LskeRA4EWtfgswkuTZba5HJjl8IJ9EktSZIUKS1FlVXUvvvoYvAx8DVgP30rvR+vQkXwZuBBZOMtWlwBrgq8B5wFVt/h8DJwPvbHPdQO9SKUnSTiBVY51FliRpYkn2rqofJNmL3hmFJVV1/bD7kiQNnvdESJK217Ikh9G7oXqFAUKSdh+eiZAkSZLUifdESJIkSerEECFJkiSpE0OEJEmSpE4MEZIkSZI6MURIkiRJ6sQQIUmSJKmT/w9xaq2s9s3hkwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 936x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (13, 5))\n",
    "sns.countplot(processed_profile['gender']);\n",
    "plt.title('Gender distribution');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are more male than females in the dataset. There is a large amount of `Unknown` and a few 'others'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEWCAYAAACqitpwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAfVklEQVR4nO3de5gdVZnv8e8vNxKMGCDIJQ0GTdAhHEVpAZWDjCgGUOI4oETmEDTK0YO2zByPwgzzwFHwyHhBWiXPBBMJGSUgIxA1DDBgREcINBCRcDFtuKQhQEISICTcwnv+qNWk0uxO7+5euyud/D7P00/vvWrVqreqd+93r6raaykiMDMzy2lI1QGYmdm2x8nFzMyyc3IxM7PsnFzMzCw7JxczM8vOycXMzLJzcrHtnqSFkj6bHp8k6fqMbS+RdER6fI6kf8vY9j9K+nGu9rZFki6RdG7VcWyPnFys1yQ9JOmDVcfRCBHx04g4qqd69b5pRcSkiFjY37gkHSGpo0vb34yIz/a3bbNGcHIxawBJw6qOYWswWI/DYI17a+LkYv0i6RRJv5f0HUlrJD0o6ejS8l0k/UTSY2n51aVln5PULmm1pPmS9iotC0n/S9JSSc9K+oakt0i6RdIzkq6QNKJU/yOSFktaK+kPkt6+hZg/JOl+SU9L+iGgrvuTHkvSBZKeTHXvlnSApFOBk4CvSlon6Zep/kOSvibpbuA5ScNq9PJGSro87dOdkt7RZZ8nlJ5fIulcSa8DrgX2SttbJ2mvrqfZJB2XTsOtTaf6/qq07CFJX0n78HSKYeQW/qb/JekHqe79ko4sLX+DpFmSVkh6NMU4tMu6F0haDZxTo/1Rkuak18N9kr5a7pWlfft3SSvT66mltOyc9Le/NB3DJZKaS8vfmY7rs5IuB0Z22Xa3r5Naf79ax8fq4+RiORwCPACMBf4FmCWp8w17LrAjMAl4I3ABgKQPAP8P+ASwJ/AwMK9Lu5OBg4BDga8CMyne1PcGDgCmprbeBcwG/iewK/CvwHxJO3QNVNJY4N+Bs1K8fwHe181+HQUcDuwHjAE+CTwVETOBnwL/EhGjI+KjpXWmAscCYyLi5RptTgF+DuwC/Ay4WtLwbrYPQEQ8BxwNPJa2NzoiHuuyX/sBlwGnA7sBC4BflhMwxbGeDOwLvB04ZQubPQRYRnGMzgZ+IWmXtGwO8DIwAXgnxXH6bI113wicV6Pts4HxwJuBDwF/V9qPIcAvgT8C44AjgdMlfbi0/nEUr5UxwHzgh2ndEcDVFK+5XSiO89+W2q7nddLT38/q5ORiOTwcERdHxEaKN549gd0l7Unxpvj5iFgTES9FxG/TOicBsyPizoh4ATgTeI+k8aV2z4+IZyJiCXAPcH1ELIuIpyk+yb8z1fsc8K8RsSgiNkbEHOAFiqTU1THAvRFxZUS8BHwfeLyb/XoJeD3wNkARcV9ErOjhWLRGxPKI2NDN8jtK2/4exSfrWnH21ieBX0fEDant7wCjgPd2ie2xiFhN8QZ+4BbaexL4fvqbXU7x4eFYSbtT/E1Pj4jnIuJJig8MJ5bWfSwifhARL3dzHD4BfDO9JjqA1tKydwO7RcTXI+LFiFgGXNyl/d9HxIL0epsLdPb+DgWGl+K+Eri9tF49r5Oe/n5WJ3f7LIdX35wjYn3qtIym+PS4OiLW1FhnL+DO0nrrJD1F8Wn1oVT8RKn+hhrP90iP3wRMk/Sl0vIRaRu1tru8tN2QtLxGPSLipnTa7EfAPpKuAr4SEc/Uqp/UbKvW8oh4JZ0OqhVnb+1F0fsrt72c4nh2KifR9T1s99HYfFTbh1P9N1G8ga/Y1DllCJvvd0/HYC+6r/8mitN/a0tlQ4HflZ533Y+R6RTWXt3EXW67p9dJT7FbndxzsUZaDuwiaUyNZY9R/LMDkK4r7Ao82sftnBcRY0o/O0bEZTXqrqA4rda5XZWfdxURrRFxEMVpvf2A/9O5qLtVeoi1vO0hQBPFsYDijXLHUt09So97arfr8ezcr74cT4BxpVObAPukbSyn+LQ/tnSsd4qISb2IdQXFfncqH//lwINd/pavj4hj6oh5RTdxl9vu6XXiYeIzcXKxhkmnkK4FLpK0s6Thkg5Pi38GfFrSgemc9zeBRRHxUB82dTHweUmHqPA6ScdKen2Nur8GJkn6ePq028Lmb+KvkvTu1OZw4DngeWBjWvwExTWD3jqotO3TKd6ob03LFgOfkjRU0mTg/aX1ngB2lfSGbtq9guK01ZEp3v+d2v5DH2KE4npJS/qbnQD8FbAg/U2vB74raSdJQ1TcaPH+Lbb22ljPTK+JccAXS8tuA55JF9ZHpWNxgKR319HuLRTXglpU3EzxceDg0vLevE6sn5xcrNH+B8W1i/spzuOfDhARNwL/THFxfQXwFjY/r163iGijOJ/+Q2AN0E43F6sjYhVwAvAt4ClgIvBf3TS9E8Ub0hqK0ytPUVzLAJgF7J/uOrq6m/VruYbi+sgaimPz8XSNBODLwEeBtRTXpF5tNyLup7hgvyxtc7NTWhHxAMWF8R8Aq1I7H42IF3sRW9kiimOziuKi/PER8VRadjLF6aR7035cSXGdrV5fBzqAB4H/TOu/kPZjY4r9wLR8FfBjoLuk+qq0rx+n+NuvoTjOvygtr/t1Yv0nTxZmZmWSTgE+GxGHDdD2vgCcGBG96f3YVs49FzMbUJL2lPS+dErtrRSn8K6qOi7Ly3eLmdlAG0HxHZN9KU4BzgMuqjQiy86nxczMLDufFjMzs+x8WiwZO3ZsjB8/vuowzMwGlTvuuGNVROzWtdzJJRk/fjxtbW1Vh2FmNqhIerhWuU+LmZlZdk4uZmaWnZOLmZll5+RiZmbZObmYWb/NnTuXww8/nMsuqzUQtW2PGpZcJM1WMT3sPaWyb6uYMvVuSVeVh2KXdKaKKW8fKM86J2lyKmuXdEapfF9Ji1RMg3t554x7knZIz9vT8vGN2kczK1x88cUAzJgxo+JIbGvRyJ7LJRRTqpbdABwQEW8H/kwx+yCS9qcYEXdSWueiNNT2UIqJmo4G9gemproA5wMXRMREihFOp6fy6cCaiJhAMUPe+Y3ZPTODotdS5t6LQQO/5xIRN3ftNUTE9aWntwLHp8dTgHlputsHJbWzaR6G9jTVKZLmAVMk3Qd8APhUqjMHOAeYkdo6J5VfCfxQksLj3GxTWltbaW9v71cbHR0dADQ1NfVQs3sTJkygpaWlX3EMdp29lk4zZsxg6tSpFUVjW4sqr7l8hmIiKSimYi1PL9qRyror3xVYGxEvdynfrK20/OlU/zUknSqpTVLbypUr+71DNrhs2LCBDRs8VbpZI1TyDX1J/0QxY9xPO4tqVAtqJ7/YQv0ttfXawoiZwEyA5uZm92wGkRy9hc42Wltb+92WmW1uwJOLpGnAR4AjS6eqOth8Hu3yvOK1ylcBYyQNS72Tcv3OtjrSVLJvAFY3Yl/MzKy2AT0tluYF/xpwXESsLy2aD5yY7vTal2J61duA24GJ6c6wERQX/eenpPQbNl2zmUYxfWxnW9PS4+OBm3y9xcxsYDWs5yLpMuAIYKykDuBsirvDdgBukARwa0R8PiKWSLqCYk7ul4HT0lzaSPoicB0wFJgdEUvSJr4GzJN0LnAXxZzmpN9z000Bq+njvOxmZtZ3jbxbrNbtIrNqlHXWPw84r0b5AmBBjfJlbLqjrFz+PHBCr4I1M7Os/A19MzPLzsnFzMyyc3IxM7PsnFzMzCw7T3Nstp3LMZROV339kquH09l2uOdiZv0yfPjwLT637ZN7LmbbuRw9hcMPP/zVxzfeeGO/27PBzz0XM+u3zt7K+PHjqw3EthruuZhZv02aNAnwIKC2iXsuZmaWnZOLmZll5+RiZmbZObmYmVl2Ti5mZpadk4uZmWXn5GJmZtk5uZiZWXZOLmZmlp2Ti5mZZefkYmZm2Tm5mJlZdk4uZmaWnZOLmZll5+RiZmbZObmYmVl2Ti5mZpadk4uZmWXn5GJmZtk5uZiZWXZOLmZmlp2Ti5mZZefkYmZm2Tm5mJlZdk4uZmaWnZOLmZll17DkImm2pCcl3VMq20XSDZKWpt87p3JJapXULuluSe8qrTMt1V8qaVqp/CBJf0rrtErSlrZhZmYDp5E9l0uAyV3KzgBujIiJwI3pOcDRwMT0cyowA4pEAZwNHAIcDJxdShYzUt3O9Sb3sA0zMxsgwxrVcETcLGl8l+IpwBHp8RxgIfC1VH5pRARwq6QxkvZMdW+IiNUAkm4AJktaCOwUEbek8kuBjwHXbmEbZmYN19raSnt7e7/a6OjoAKCpqalf7UyYMIGWlpZ+tdFXDUsu3dg9IlYARMQKSW9M5eOA5aV6HalsS+UdNcq3tI3XkHQqRe+HffbZp6/7ZGaW1YYNG6oOod8GOrl0RzXKog/lvRIRM4GZAM3Nzb1e38ysqxw9hc42Wltb+91WVQb6brEn0uku0u8nU3kHsHepXhPwWA/lTTXKt7QNMzMbIAOdXOYDnXd8TQOuKZWfnO4aOxR4Op3aug44StLO6UL+UcB1admzkg5Nd4md3KWtWtswM7MB0rDTYpIuo7iwPlZSB8VdX98CrpA0HXgEOCFVXwAcA7QD64FPA0TEaknfAG5P9b7eeXEf+ALFHWmjKC7kX5vKu9uGmZkNkEbeLTa1m0VH1qgbwGndtDMbmF2jvA04oEb5U7W2YWZmA8ff0Dczs+ycXMzMLDsnFzMzy87JxczMsnNyMTOz7JxczMwsOycXMzPLzsnFzMyyc3IxM7PsnFzMzCw7JxczM8vOycXMzLJzcjEzs+ycXMzMLDsnFzMzy87JxczMsnNyMTOz7JxczMwsOycXMzPLzsnFzMyyG1Z1ALb9aW1tpb29veowWLp0KQAtLS2VxjFhwoTKYzDLzcnFBlx7ezt/vudO9hm9sdI4RrxUdNyff+j2ymJ4ZN3QyrZt1khOLlaJfUZv5KzmdVWHUblz20ZXHYJZQ/iai5mZZefkYmZm2Tm5mJlZdk4uZmaWnZOLmZll5+RiZmbZObmYmVl2Ti5mZpadk4uZmWXn5GJmZtk5uZiZWXZ1JRdJ+0m6UdI96fnbJZ3V2NDMzGywqrfncjFwJvASQETcDZzY141K+ntJSyTdI+kySSMl7StpkaSlki6XNCLV3SE9b0/Lx5faOTOVPyDpw6XyyamsXdIZfY3TzMz6pt7ksmNE3Nal7OW+bFDSOKAFaI6IA4ChFInqfOCCiJgIrAGmp1WmA2siYgJwQaqHpP3TepOAycBFkoZKGgr8CDga2B+YmuqamdkAqTe5rJL0FiAAJB0PrOjHdocBoyQNA3ZMbX0AuDItnwN8LD2ekp6Tlh8pSal8XkS8EBEPAu3AwemnPSKWRcSLwLxU18zMBki987mcBswE3ibpUeBB4O/6ssGIeFTSd4BHgA3A9cAdwNqI6OwNdQDj0uNxwPK07suSngZ2TeW3lpour7O8S/khtWKRdCpwKsA+++zTl90xM7Ma6kouEbEM+KCk1wFDIuLZvm5Q0s4UPYl9gbXAzylOYb1ms52rdLOsu/JavbGoUUZEzKRImjQ3N9esY2bbD0/Bvbn+TMFdV3KRNAY4GRgPDCvOSkFE9GWrHwQejIiVqe1fAO8FxkgalnovTcBjqX4HsDfQkU6jvQFYXSrvVF6nu3Izs261t7dz/+LF7FFxHJ2fkNcuXlxZDI/3c/16T4stoDgF9SfglX5u8xHgUEk7UpwWOxJoA34DHE9xjWQacE2qPz89vyUtvykiQtJ84GeSvgfsBUwEbqPo0UyUtC/wKMVF/0/1M2Yz207sAUyveWJk+zKr9gmfutWbXEZGxD/0a0tJRCySdCVwJ8UdZ3dRnJr6NTBP0rmpbFZaZRYwV1I7RY/lxNTOEklXAPemdk6LiI0Akr4IXEdxJ9rsiFiSI3YzM6tPvcllrqTPAb8CXugsjIjVfdloRJwNnN2leBnFnV5d6z4PnNBNO+cB59UoX0DR2zIzswrUm1xeBL4N/BObLo4H8OZGBGVmZoNbvcnlH4AJEbGqkcGYmdm2od4vUS4B1jcyEDMz23bU23PZCCyW9Bs2v+ZS7U3YZma2Vao3uVydfszMzHpU7zf056RRivdLRQ9ExEuNC8vMzAazer+hfwTF4JEPUXxJcW9J0yLi5saFZmZmg1W9p8W+CxwVEQ9AMXkYcBlwUKMCMzOzwave5DK8M7EARMSfJQ1vUExm1gtbw2CLW8tAi9C/wRYtn3qTS5ukWcDc9PwkimHyzaxi7e3t3LXkLhhTYRBpxMG7Hr2rwiAoxlm3rUK9yeULFHO6tFBcc7kZuKhRQZlZL42BV47o75iyg9+QhfV+dc8ard7kMgy4MCK+B5CmEt6hYVGZmdmgVm+avxEYVXo+CvjP/OGYmdm2oDdD7q/rfBIR69J8LGa91tHRwXPPDuXcttFVh1K5h58dyus6OqoOwyy7ensuz0l6V+cTSQdRTPRlZmb2GvX2XE4Hfi6pc7rgPYFPNiYk29Y1NTXx/MsrOKt5Xc+Vt3Hnto1mZFNT1WGYZVfv8C+3S3ob8FaKu8Xu9/AvZmbWnXp7LgDvBsandd4piYi4tCFRmZnZoFbv2GJzgbcAiymG34diJkonFzMze416ey7NwP4RET3WNDOz7V69d4vdA+zRyEDMzGzbUW/PZSxwr6Tb2HwmyuMaEpWZmQ1q9SaXcxoZhJmZbVvqvRX5t40OxMzMth1bTC6Sfh8Rh0l6luLusFcXAREROzU0OjMzG5S2mFwi4rD0+/UDE87WqbW1lWuvvbZfbaxfv56t5WY7Sey4Y9+Hhjv66KM9GZOZbZEnPzAzs+x68w397VZLS4s/qZuZ9YJ7LmZmlp2Ti5mZZefkYmZm2Tm5mJlZdk4uZmaWne8WMzNLOjo6eBaYxdbxnbQqrQDWdXT0eX33XMzMLLtKei6SxgA/Bg6gGFbmM8ADwOUUs10+BHwiItZIEnAhcAywHjglIu5M7UwDzkrNnhsRc1L5QcAlwChgAfBlz0VjZj1pampi7apVTEdVh1K5WQRjmpr6vH5VPZcLgf+IiLcB7wDuA84AboyIicCN6TnA0cDE9HMqMANA0i7A2cAhwMHA2ZJ2TuvMSHU715s8APtkZmbJgCcXSTsBhwOzACLixYhYC0wB5qRqc4CPpcdTgEujcCswRtKewIeBGyJidUSsAW4AJqdlO0XELam3cmmpLTMzGwBV9FzeDKwEfiLpLkk/lvQ6YPeIWAGQfr8x1R8HLC+t35HKtlTeUaP8NSSdKqlNUtvKlSv7v2dmZgZUc81lGPAu4EsRsUjShWw6BVZLrZOf0Yfy1xZGzARmAjQ3N/uajA1KHR0d8DQMWej7c1gLHdH3O5wsnypejR1AR0QsSs+vpEg2T6RTWqTfT5bq711avwl4rIfyphrlZmY2QAa85xIRj0taLumtEfEAcCRwb/qZBnwr/b4mrTIf+KKkeRQX75+OiBWSrgO+WbqIfxRwZkSslvSspEOBRcDJwA8GbAfNBlhTUxMrtZJXjnil6lAqN2ThEJrG9f0OJ8unqi9Rfgn4qaQRwDLg0xS9qCskTQceAU5IdRdQ3IbcTnEr8qcBUhL5BnB7qvf1iFidHn+BTbciX5t+zMxsgFSSXCJiMdBcY9GRNeoGcFo37cwGZtcob6P4Do2ZmVXAVwDNzCw7JxczM8vOycXMzLJzcjEzs+ycXMzMLDsnFzMzy86ThVklHlk3lHPbRlcawxPri89Wu+9Y3ZcPH1k3lP0q27pZ4zi52ICbMGFC1SEA8OLSpQCMHD+xshj2Y+s5HmY5ObnYgGtpaak6BGBTHK2trRVHYrbt8TUXMzPLzsnFzMyyc3IxM7PsnFzMzCw7X9A3Myt5HJhVe/LaAfNU+r1rhTE8Dozpx/pOLmZmydZyW/jKdJv8mInV3SY/hv4dDycXM7PEt8nn42suZmaWnZOLmZll5+RiZmbZObmYmVl2Ti5mZpadk4uZmWXn5GJmZtk5uZiZWXZOLmZmlp2Ti5mZZefhX8y2BWthyMIKPyuuS79HVxcCAGuBcRXHYICTi9mgtzUMtrg0DbQ4cVx1Ay0CMG7rOB7m5GI26G0Ngy1uCwMtWl6+5mJmZtk5uZiZWXZOLmZmlp2Ti5mZZefkYmZm2Tm5mJlZdpUlF0lDJd0l6Vfp+b6SFklaKulySSNS+Q7peXtaPr7Uxpmp/AFJHy6VT05l7ZLOGOh9MzPb3lXZc/kycF/p+fnABRExEVgDTE/l04E1ETEBuCDVQ9L+wInAJGAycFFKWEOBHwFHA/sDU1NdMzMbIJUkF0lNwLHAj9NzAR8ArkxV5gAfS4+npOek5Uem+lOAeRHxQkQ8CLQDB6ef9ohYFhEvAvNSXTMzGyBV9Vy+D3wVeCU93xVYGxEvp+cdbBohaBywHCAtfzrVf7W8yzrdlb+GpFMltUlqW7lyZX/3yczMkgFPLpI+AjwZEXeUi2tUjR6W9bb8tYURMyOiOSKad9ttty1EbWZmvVHF2GLvA46TdAwwEtiJoiczRtKw1DtpAh5L9TuAvYEOScOANwCrS+Wdyut0V25mZgNgwHsuEXFmRDRFxHiKC/I3RcRJwG+A41O1acA16fH89Jy0/KaIiFR+YrqbbF9gInAbcDswMd19NiJtY/4A7JqZmSVb06jIXwPmSToXuAuYlcpnAXMltVP0WE4EiIglkq4A7gVeBk6LiI0Akr4IXAcMBWZHxJIB3RMzs+1cpcklIhYCC9PjZRR3enWt8zxwQjfrnwecV6N8AbAgY6hmZtYL/oa+mZll5+RiZmbZObmYmVl2Ti5mZpadk4uZmWXn5GJmZtk5uZiZWXZOLmZmlp2Ti5mZZefkYmZm2Tm5mJlZdk4uZmaWnZOLmZll5+RiZmbZObmYmVl2KiZ1tObm5mhra6s6DKtTa2sr7e3t/Wpj6dKlAEycOLHPbUyYMIGWlpZ+xVG1reVYgo9np8F0PCXdERHNXcu3ppkozQbUqFGjqg5hm+Fjmde2cDzdc0ncczEz673uei6+5mJmZtk5uZiZWXZOLmZmlp2Ti5mZZefkYmZm2Tm5mJlZdk4uZmaWnZOLmZll5y9RJpJWAg9XHUcdxgKrqg5iG+LjmY+PZV6D5Xi+KSJ261ro5DLISGqr9W1Y6xsfz3x8LPMa7MfTp8XMzCw7JxczM8vOyWXwmVl1ANsYH898fCzzGtTH09dczMwsO/dczMwsOycXMzPLzsmlASSNl3RPl7JzJH1lC+ucIumHjY/OACRtlLS49DO+6pgGI0khaW7p+TBJKyX9qsq4BjNJTZKukbRU0l8kXShpRNVx9ZaTi22vNkTEgaWfh6oOaJB6DjhAUue8vB8CHq0wnkFNkoBfAFdHxERgP2A0cF6lgfWBk8sAk7RQ0vmSbpP0Z0n/vUadYyXdImmspEsktUr6g6Rlko5PdSTp25LukfQnSZ9M5RdJOi49vkrS7PR4uqRzU6/qPkkXS1oi6frSG4NZX1wLHJseTwUuqzCWwe4DwPMR8ROAiNgI/D3wGUk7VhpZLzm5VGNYRBwMnA6cXV4g6W+AM4BjIqJz6Ic9gcOAjwDfSmUfBw4E3gF8EPi2pD2Bm4HOhDUO2D89Pgz4XXo8EfhRREwC1gJ/m3XvBodRpVNiV1UdzCA3DzhR0kjg7cCiiuMZzCYBd5QLIuIZ4BFgQiUR9dGwqgPYRnV3f3dn+S/S7zuA8aXlfw00A0elF1SnqyPiFeBeSbunssOAy9Inmyck/RZ4N0UCOV3S/sC9wM4p6bwHaAF2BR6MiMXdxLC92BARB1YdxLYgIu5O16ymAguqjWbQE7XfP7or32q559IYTwE7dynbhU2D0L2Qfm9k8wS/DHg9xXnWshdKj9Xl92Yi4tG07ckUvZjfAZ8A1kXEszXa6xqDWV/MB76DT4n11xKKD5ivkrQTsDfwl0oi6iMnlwaIiHXACklHAkjaheLN/vc9rPowxemuSyVN6qHuzcAnJQ2VtBtwOHBbWnYLxSm3zuTyFTadEjNrhNnA1yPiT1UHMsjdCOwo6WQASUOB7wKXRMT6SiPrJSeXxjkZOEvSYuAm4P9GRI+fPCLiAeAk4OeS3rKFqlcBdwN/TO1/NSIeT8t+R3Fdpx24k6LX5ORiDRMRHRFxYdVxDHZRDJnyN8AJkpYCfwaeB/6x0sD6wMO/mJlZdu65mJlZdk4uZmaWnZOLmZll5+RiZmbZObmYmVl2Ti5mg1Aac+74quMw646Ti9l2QJJHYbAB5RecWYNJ+meKL8YupxgC6A6KL8H+CNgNWA98LiLul3QJ8AzFECB7UHw59so0FPsPKEbNfZDS8D+SDgK+RzE0+yrglIhYIWkh8AfgfRTDs3y34Ttrlji5mDWQpGaKUaffSfH/didFcpkJfD4ilko6BLiIInHAplGw30aRFK6k+Nb2W4H/BuxOMSjpbEnDKZLOlIhYmaZeOA/4TGprTES8v+E7ataFk4tZYx0GXBMRGwAk/RIYCbyXYoifzno7lNapNQr24WwaBfsxSTel8rcCBwA3pLaGAitKbV2ef5fMeubkYtZYtUavHgKs3cKQ/7VGwYbuh2JfEhHv6aat53oO0Sw/X9A3a6zfAx+VNFLSaIoZG9cDD0o6AV6dVfQdPbRzM8WEXEPT/Dx/ncofAHaT9J7U1vA6RtQ2azgnF7MGiojbKa6b/JFikrg24GmKC/zTJf2RYg6PKT00dRWwFPgTMAP4bWr/ReB44PzU1mKKU25mlfKoyGYNJml0RKxLc6DfDJwaEXdWHZdZI/mai1njzUzTTo8E5jix2PbAPRczM8vO11zMzCw7JxczM8vOycXMzLJzcjEzs+ycXMzMLLv/DwAwsHLM9pVsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.boxplot(x = 'gender',\n",
    "            y = 'income',\n",
    "            data = processed_profile);\n",
    "plt.title('Income distribution per gender');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The boxplots show there is no evidence that the income differs accross genders."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAfQElEQVR4nO3deZwdVZ338c+XhDUQEkjCkgQDYwaJKxgDmnlQiRMWlaADTnCQyIQnPr6CD24jqM8MAuKo4wDighNJIKCyiCKRFyPGyCIyLB3AsARMQCVtQtKQhU3B4O/545wmlc7trtvJrXs76e/79bqvvnXqVNXvVNWt361TdasVEZiZmfVku1YHYGZmfZ+ThZmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVkpJ4ttgKSBkkLSmFbHsq2TdKqkW1odR3+wNa5rSe2S3tHqOKrgZLEZJH1f0pwuZW+X9LSkfZqw/NslfbiB82v6Dt7oNphZtZwsNs//BY6R9PcAknYCvgt8KiJWNHJBkgY0cn62Ka/jDSQNbNFyvQ0KWrUdehQRfm3GCzgB+B0wCPh34L8L47YDPgc8BjwFXAUMLYy7FngSWAvcAhxUmPZ7wLeAnwHPA+/ostyvAC8DfwaeAy4EBgIBfARYCqwBLipMMxa4GXg6x3MFsHsedyXwV+BPeX6f7Ka97wfuB57Jy5icy9uLMQJfBC7L73cBfpCXuxa4GxhWqw25/t8BbcC6XPfQwnxvB84B7szr5SfAnjn+Z4C7gP0K9ccBvwBWA48A/9DTOgbeAywGns1t+kQ36+FU4Dbg2znOxcA7C+OHAJcCK/J8zgG2K4z/SI7nWeBB4I25/P8Bj+fyh4BjuyzzVuCivB6XAocC04FlwErgpEL9nYDzC+O+DexU0p6L8rr6QqH8kbwv/TcwOpefB1yQ3++Yt+GX8vCueXh3NmM/B4YDN+TteWde1i2Fz81FwKq83hcB47ppU6P3lW8CN5H21duAvYBv5HYt7tyGhc/DGbl8DTAb2LEw/ljgN3na24HXdZn2X4AHgJdy2eeA5TnuR+hyPGjqMa9VC94WXvnDMI90MCzufJ8Gfg2MzB/c2cAVhZ3+w8Buedw3gbYuO+ca4K257o41lns78OHCcGeyuD5/UMfkHf9defzfApOAHYARObavddlJu90JgbflnXtSjmk0cGCtadk4WczMH9SdgQHAeGDXbtowLB8ETsztOSmv16GF+r8FDgCG5g/Oo8A7c/0fAN/NdXcD/gicnMe9Oc/rwO7WMdABvC2P3wM4pJt1cSqwnnR2uT3wwbxuhuTxN5AOzrsAewMLgel53ImkA/ibAeXt0nkQ/gCwT47ng6QD015dlvmhvB6/DPyBdPDcETgmr7tdcv1vAtfl9TQYuBE4t6Q9H83z3hk4Pq/bA/P6+wLwq1x/MnBffn846QvRrwvjFm7ufk76PF2Z190bSAn3llz/3aQvEJ2JaBywdzdtavS+sgo4OLfjVtKXxA8WtsX8Lp+lRcAo0j59JxsS8FtIyfstedp/zutvh8K0C/O0OwOvzdt57zx+f+CAlh3vWn3A3ZpfpG8YzwGndylfAry9MDwaeJHCN8zCuGGkA/2gws45p2S53SWLwwplPwY+3c30xwP3FIbLksVs4D+6GddTspiRY319HW04BbijS517yN+Yc/0zCuO+Dvy0MPw+8sEI+Cfg5hpt+Hx365j07e1UYLeSdX8q6YCvQtm9pEQwknSGVvwm+aHOgwmwAJhZ5771IPDuwjIXF8YdnLf3noWydcDrSAfSPwOvKoz7X8CSHtrzeJey+cC0LvvXi7l9g/L7IaSzoTPyutuFdCZwfjfL6XE/JyXe9cCrC2VfZUOymEw66B9Kjc9RjX2rkfvKxYVxnwAe6LItnuryeTi1MHws8Gh+/13grC7LegyYWJj25MK4A0nJZRIwsJ79psqXr1lsgYhYSerWeajLqP2An0paK2kt6bQygBGSBkj6qqTHJXV26UD6MHVatpkhPVl4/wKpWwBJe0u6RtIf8zIv67K8MqNJO3VvXUY6ve9c9pd76Ivdl/QtqugPpANUp5WF93+qMbxrfv8qYGLn+s/b4B9J39w7dV3H7yN9sJ+QdIukQ3toV3vkT3Mhzn3zcncEVhaW+y3SlwroYT1K+rCk3xSmew0bb6OubX05Ip6u0f69cwzFed1AOqPsTtd18SrgW4XpnyJ1VY6KiOeB+0hnFYeTupfuJJ0hvJ30zZvN2M/3In3bLpa9sj9ExM+B7wAXk9bvdyTt1kObGrmv1DuvWu3q3Dc6l3VGl2Xtw8b7+CvTRsSjwKdIXWqrJF0pae/uGlw1J4tqtAN/HxFDCq+dIuJJ0unuMcARpFPqV+dpVJg+6FnZ+K6+Qvo2+PqIGEzqHujN8pYBf9PNuOdJ3yo7vbIzR8RLEfGFiDiIdD3ifaRvcrWWuZz0YSraj9RF0FvLgAVd1v+uEXFaoc5Gy4+IuyLiWNJB9QbSdabujKoR5/K83BeAPQrLHRwRbyjEtcl6lHQA6SD4UdLZwhDSt2h1rVuHlcBLpG6Uzhh2j4jde5im67ZYRuo6K66/nSPirjz+VuBdwOtJ3Sa3AkeTunB+lev0dj9fSUpIowtl+20UZMSFEXEI6QxqHPDJHtpUr3r2ld7q2oblhWWd3WVZu0TENYX6XffL70XERFIX1ADS9dGWcLKoxneAL0naD0DSCEnH5nG7kQ7cT7Ph1L23VpL6Y+u1G+mgvk7SaNI1ld7MbzZwqqR3StpO0ihJB+Zx9wNT8289JpAuhAMg6QhJr5O0HekC3V9IF7ZrLfMG4LWS/jHP64OkA8yNvWhnp3l5Xh+UtH1+TSjEvBFJO+e6gyPiL6SLzC/XqpvtI+m0HOdUUgL4WUQsIx04vyZpcF5Xr5Z0eJ7uEuAzkg5WMjZvj11JB4mOFI5OJZ1Z9FpEvJyXc6Gk4Xk5oyRN7sVsvgN8XtJBpICGSDq+MP5W0heORRGxnnR2MQP4bUSsznV6tZ/n9f4T4Oy8PV5H6sIjxzAhvwaS9uWX6Hkb1atX+0qdTpM0UtKewGeBq3P5LGCmpLfk7bKrpPdKGlRrJpIOyp+5HUlnMH+iMW3eLE4W1TifdJfHAknPAneQLmpBulNmeX49lMf11oXAiflU9vw66p8FTCD1a88DftRl/JdIH9K1kj7edeKIuAP436QLqutId1Z1fnv6POnAthb4V9LFw077kq6dPENq6y9IFzA3aUNEdJC6gc4gHWA+AbyncPCpW0SsA44kXSRfQeqe+3dS90x3pgF/yF0m0ykcqGq4g3TxcTXp4u8/RMSaPO4kUr/+w6QLuD8kn21FxJWks7yrSevkx6QL+ItI6/buHO9rSHfsbK5Pkbo/7iZtr5+T7oirS0T8kLQP/zCvj0Wk9dnpdlICuC0PLyJ9EbitUGdz9vOPki5IryR9Qbm0MG5ILlsL/J60ni6ot03d2cx9pcyVpH39MdKF9S/lZd1FauPFpH3jt3m53dmRdN3mqRzXUNJ1opbQxl2vZmZmm/KZhZmZlXKyMDOzUk4WZmZWysnCzMxK9b2HVTXAsGHDYsyYMa0Ow8xsq7Jw4cKnImJ4rXHbZLIYM2YMbW1trQ7DzGyrIqnrUxRe4W4oMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMrtU3+gtvM+pZvfuqnrQ6h1077z/e2OoQ+xWcWZmZWysnCzMxKOVmYmVkpJwszMyvlZGFmZqWcLMzMrJSThZmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVmpSpOFpCGSrpX0iKTFkt4qaQ9J8yUtyX+H5rqSdJGkpZIWSTqkMJ9puf4SSdOqjNnMzDZV9ZnF14GfRcRrgDcCi4EzgQURMRZYkIcBjgbG5tcM4GIASXsAZwGHAhOAszoTjJmZNUdlyULSYOBwYDZARLwUEWuBKcDcXG0ucFx+PwW4PJI7gSGS9gGOBOZHxOqIWAPMB46qKm4zM9tUlWcWBwAdwKWS7pN0iaRBwF4RsQIg/x2R648ElhWmb89l3ZVvRNIMSW2S2jo6OhrfGjOzfqzKZDEQOAS4OCIOBp5nQ5dTLapRFj2Ub1wQMSsixkfE+OHDh29OvGZm1o0qk0U70B4Rd+Xha0nJY2XuXiL/XVWoP7ow/ShgeQ/lZmbWJJUli4h4Elgm6cBcNAl4GJgHdN7RNA24Pr+fB5yc74o6DFiXu6luAiZLGpovbE/OZWZm1iRV/1vVjwHfl7QD8DhwCilBXSNpOvAEcEKueyNwDLAUeCHXJSJWSzoXuCfXOyciVlcct5mZFVSaLCLifmB8jVGTatQNYGY385kDzGlsdGZmVi//gtvMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSlSYLSb+X9ICk+yW15bI9JM2XtCT/HZrLJekiSUslLZJ0SGE+03L9JZKmVRmzmZltqhlnFu+MiDdFxPg8fCawICLGAgvyMMDRwNj8mgFcDCm5AGcBhwITgLM6E4yZmTVHK7qhpgBz8/u5wHGF8ssjuRMYImkf4EhgfkSsjog1wHzgqGYHbWbWn1WdLAL4uaSFkmbksr0iYgVA/jsil48ElhWmbc9l3ZVvRNIMSW2S2jo6OhrcDDOz/m1gxfOfGBHLJY0A5kt6pIe6qlEWPZRvXBAxC5gFMH78+E3Gm5nZ5qv0zCIilue/q4DrSNccVubuJfLfVbl6OzC6MPkoYHkP5WZm1iSVJQtJgyTt1vkemAw8CMwDOu9omgZcn9/PA07Od0UdBqzL3VQ3AZMlDc0XtifnMjMza5Iqu6H2Aq6T1LmcH0TEzyTdA1wjaTrwBHBCrn8jcAywFHgBOAUgIlZLOhe4J9c7JyJWVxi3mZl1UVmyiIjHgTfWKH8amFSjPICZ3cxrDjCn0TGamVl9/AtuMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZKScLMzMr5WRhZmalnCzMzKxUZf+D28ysvzjvpONbHUKvff571/aqvs8szMyslJOFmZmVcrIwM7NSlScLSQMk3Sfphjy8v6S7JC2RdLWkHXL5jnl4aR4/pjCPz+byRyUdWXXMZma2sWacWZwOLC4MfwW4ICLGAmuA6bl8OrAmIl4NXJDrIWkcMBV4LXAU8G1JA5oQt5mZZZUmC0mjgHcDl+RhAUcAnZfh5wLH5fdT8jB5/KRcfwpwVUS8GBG/A5YCE6qM28zMNlb1mcWFwGeAv+bhPYG1EbE+D7cDI/P7kcAygDx+Xa7/SnmNaV4haYakNkltHR0djW6HmVm/VlmykPQeYFVELCwW16gaJeN6mmZDQcSsiBgfEeOHDx/e63jNzKx7Vf4obyJwrKRjgJ2AwaQzjSGSBuazh1HA8ly/HRgNtEsaCOwOrC6UdypOY2ZmTVDZmUVEfDYiRkXEGNIF6l9GxD8BNwOdP3ecBlyf38/Lw+Txv4yIyOVT891S+wNjgburitvMzDbVisd9nAFcJemLwH3A7Fw+G7hC0lLSGcVUgIh4SNI1wMPAemBmRLzc/LDNzPqvpiSLiLgFuCW/f5wadzNFxJ+BE7qZ/jzgvOoiNDOznvgX3GZmVsrJwszMStWVLCQtqKfMzMy2TT1es5C0E7ALMEzSUDb85mEwsG/FsZmZWR9RdoH7I8DHSYlhIRuSxTPAtyqMy8zM+pAek0VEfB34uqSPRcQ3mhSTmZn1MXXdOhsR35D0NmBMcZqIuLyiuMzMrA+pK1lIugL4G+B+oPMHcQE4WZiZ9QP1/ihvPDAuP37DzMz6mXqTxYPA3sCKCmMx67duPfztrQ6h195+262tDsGaqN5kMQx4WNLdwIudhRFxbCVRmZlZn1JvsvhClUGYmVnfVu/dUD7fNDPrx+q9G+pZNvx3uh2A7YHnI2JwVYGZmVnfUe+ZxW7FYUnHUeMx42Zmtm3arKfORsRPgCMaHIuZmfVR9XZDvb8wuB3pdxf+zYWZWT9R791Q7y28Xw/8HpjS8GjMzKxPqveaxSlVB2JmZn1Xvf/8aJSk6yStkrRS0o8kjao6ODMz6xvqvcB9KTCP9H8tRgI/zWVmZtYP1JsshkfEpRGxPr8uA4ZXGJeZmfUh9SaLpySdJGlAfp0EPF1lYGZm1nfUmyz+GfgA8CTpybPHAz1e9Ja0k6S7Jf1G0kOSzs7l+0u6S9ISSVdL2iGX75iHl+bxYwrz+mwuf1TSkb1vppmZbYl6k8W5wLSIGB4RI0jJ4wsl07wIHBERbwTeBBwl6TDgK8AFETEWWANMz/WnA2si4tXABbkeksYBU4HXAkcB35Y0oM64zcysAepNFm+IiDWdAxGxGji4pwkieS4Pbp9fQfrl97W5fC5wXH4/JQ+Tx0+SpFx+VUS8GBG/A5biR42YmTVVvcliO0lDOwck7UEdv9HI1zfuB1YB84HHgLURsT5XaSfdXUX+uwwgj18H7FksrzFNcVkzJLVJauvo6KizWWZmVo96f8H9n8Adkq4lnR18ADivbKKIeBl4k6QhwHXAQbWq5b/qZlx35V2XNQuYBTB+/Hg/isTMrIHq/QX35ZLaSF1IAt4fEQ/Xu5CIWCvpFuAwYIikgfnsYRSwPFdrB0YD7ZIGArsDqwvlnYrTmJlZE9T91NmIeDgivhkR36gnUUgans8okLQz8C5gMXAz6W4qgGnA9fn9vDxMHv/LiIhcPjXfLbU/MBa4u964zcxsy9XbDbU59gHm5juXtgOuiYgbJD0MXCXpi8B9wOxcfzZwhaSlpDOKqQAR8ZCka4CHSQ8xnJm7t8zMrEkqSxYRsYgad0xFxOPUuJspIv4MnNDNvM6jjmskZmZWjc3650dmZta/OFmYmVkpJwszMyvlZGFmZqWcLMzMrJSThZmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVkpJwszMyvlZGFmZqWcLMzMrJSThZmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVkpJwszMyvlZGFmZqWcLMzMrJSThZmZlXKyMDOzUpUlC0mjJd0sabGkhySdnsv3kDRf0pL8d2gul6SLJC2VtEjSIYV5Tcv1l0iaVlXMZmZWW5VnFuuBT0XEQcBhwExJ44AzgQURMRZYkIcBjgbG5tcM4GJIyQU4CzgUmACc1ZlgzMysOSpLFhGxIiLuze+fBRYDI4EpwNxcbS5wXH4/Bbg8kjuBIZL2AY4E5kfE6ohYA8wHjqoqbjMz21RTrllIGgMcDNwF7BURKyAlFGBErjYSWFaYrD2XdVfedRkzJLVJauvo6Gh0E8zM+rXKk4WkXYEfAR+PiGd6qlqjLHoo37ggYlZEjI+I8cOHD9+8YM3MrKZKk4Wk7UmJ4vsR8eNcvDJ3L5H/rsrl7cDowuSjgOU9lJuZWZNUeTeUgNnA4og4vzBqHtB5R9M04PpC+cn5rqjDgHW5m+omYLKkofnC9uRcZmZmTTKwwnlPBD4EPCDp/lz2OeDLwDWSpgNPACfkcTcCxwBLgReAUwAiYrWkc4F7cr1zImJ1hXGbmVkXlSWLiLid2tcbACbVqB/AzG7mNQeY07jobGs08RsTWx1Cr/z6Y79udQhmDeNfcJuZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknCzMzK+VkYWZmpZwszMyslJOFmZmVcrIwM7NSThZmZlaqsmQhaY6kVZIeLJTtIWm+pCX579BcLkkXSVoqaZGkQwrTTMv1l0iaVlW8ZmbWvSrPLC4DjupSdiawICLGAgvyMMDRwNj8mgFcDCm5AGcBhwITgLM6E4yZmTVPZckiIm4DVncpngLMze/nAscVyi+P5E5giKR9gCOB+RGxOiLWAPPZNAGZmVnFmn3NYq+IWAGQ/47I5SOBZYV67bmsu/JNSJohqU1SW0dHR8MDNzPrz/rKBW7VKIseyjctjJgVEeMjYvzw4cMbGpyZWX/X7GSxMncvkf+uyuXtwOhCvVHA8h7KzcysiZqdLOYBnXc0TQOuL5SfnO+KOgxYl7upbgImSxqaL2xPzmVmZtZEA6uasaQrgXcAwyS1k+5q+jJwjaTpwBPACbn6jcAxwFLgBeAUgIhYLelc4J5c75yI6HrR3MzMKlZZsoiIE7sZNalG3QBmdjOfOcCcBoZmZma91FcucJuZWR/mZGFmZqWcLMzMrJSThZmZlarsArc13xPnvL7VIfTKfv/2QKtDMLM6+czCzMxKOVmYmVkpJwszMyvlZGFmZqWcLMzMrJSThZmZlXKyMDOzUk4WZmZWysnCzMxKOVmYmVkpJwszMyvlZGFmZqX61YME3/wvl7c6hF5b+B8ntzoEMzOfWZiZWTknCzMzK+VkYWZmpZwszMyslJOFmZmV2mqShaSjJD0qaamkM1sdj5lZf7JVJAtJA4BvAUcD44ATJY1rbVRmZv3HVpEsgAnA0oh4PCJeAq4CprQ4JjOzfkMR0eoYSkk6HjgqIk7Nwx8CDo2I0wp1ZgAz8uCBwKNNDHEY8FQTl9dsbt/WbVtu37bcNmh++14VEcNrjdhafsGtGmUbZbmImAXMak44G5PUFhHjW7HsZnD7tm7bcvu25bZB32rf1tIN1Q6MLgyPApa3KBYzs35na0kW9wBjJe0vaQdgKjCvxTGZmfUbW0U3VESsl3QacBMwAJgTEQ+1OKyilnR/NZHbt3Xbltu3LbcN+lD7tooL3GZm1lpbSzeUmZm1kJOFmZmVcrKoQdJoSTdLWizpIUmn5/I9JM2XtCT/HZrLXyPpfyS9KOnThfnsJOluSb/J8zm7VW0qalT7CvMbIOk+STc0uy21NLJ9kn4v6QFJ90tqa0V7umpw+4ZIulbSI3l+b21FmwrxNOqzd2DeZp2vZyR9vFXtKsTVyG33iTyPByVdKWmnSmP3NYtNSdoH2Cci7pW0G7AQOA74MLA6Ir6s9HyqoRFxhqQRwKtynTUR8bU8HwGDIuI5SdsDtwOnR8SdLWjWKxrVvsL8PgmMBwZHxHua2ZZaGtk+Sb8HxkdEn/nhV4PbNxf4VURconSn4S4RsbbZbSrE09B9M89zAPBH0g95/9CsttTSwGPLSNLxZFxE/EnSNcCNEXFZVbH7zKKGiFgREffm988Ci4GRpEeMzM3V5pI2IBGxKiLuAf7SZT4REc/lwe3zq+XZuVHtA5A0Cng3cEkTQq9LI9vXFzWqfZIGA4cDs3O9l1qZKHIMVWy7ScBjrU4U0PD2DQR2ljQQ2IWKf3vmZFFC0hjgYOAuYK+IWAFpowMj6ph+gKT7gVXA/Ii4q7poe29L2wdcCHwG+GtFIW6RBrQvgJ9LWqj0SJk+ZQvbdwDQAVyauxEvkTSownB7pQHbrtNU4MpGx7eltqR9EfFH4GvAE8AKYF1E/LzKeJ0seiBpV+BHwMcj4pnNmUdEvBwRbyL96nyCpNc1MsYtsaXtk/QeYFVELGx4cA3QiO0HTIyIQ0hPPJ4p6fCGBbiFGtC+gcAhwMURcTDwPNAnHv/foG1H7lo7Fvhho2JrhAZ89oaSzkb2B/YFBkk6qbFRbszJohv5GsOPgO9HxI9z8crc59jZ97iq3vnl0/tbgKMaHOpmaVD7JgLH5n79q4AjJH2vopB7pVHbLyKW57+rgOtIT0BuuQa1rx1oL5ztXktKHi3V4M/e0cC9EbGy8ZFunga1713A7yKiIyL+AvwYeFtVMYOTRU35wvRsYHFEnF8YNQ+Ylt9PA64vmc9wSUPy+51JG/iRxkfcO41qX0R8NiJGRcQY0qn+LyOi0m839Wjg9huUL0KSu2cmAw82PuLeaeD2exJYJunAXDQJeLjB4fZKo9pWcCJ9qAuqge17AjhM0i55npNI1z+qExF+dXkBf0fqq14E3J9fxwB7AguAJfnvHrn+3qRvac8Aa/P7wcAbgPvyfB4E/q3VbWtk+7rM8x3ADa1uW4O33wHAb/LrIeDzrW5bo7cf8CagLc/rJ6S7cLaVtu0CPA3s3uptVlH7ziZ9+XwQuALYscrYfeusmZmVcjeUmZmVcrIwM7NSThZmZlbKycLMzEo5WZiZWSknC7MtpOR2SUcXyj4g6WetjMuskXzrrFkD5Me4/JD0rJ8BpPvnj4qIx7ZgngMjYn2DQjTbIk4WZg0i6auk5ysNAp6NiHMlTQNmAjsAdwCnRcRfJc0iPVpjZ+DqiDgnz6Md+C/SY2EujIg+9Uwj678GtjoAs23I2cC9wEvA+Hy28T7gbRGxPieIqcAPgDMjYnV+vPTNkq6NiM5HbTwfERNb0QCz7jhZmDVIRDwv6WrguYh4UdK7gLcAbenxPewMLMvVT5Q0nfQZ3BcYx4bnMl3d3MjNyjlZmDXWX9nwvz0EzImIfy1WkDQWOB2YEBFr85N6i/8S8/mmRGrWC74byqw6vwA+IGkYgKQ9Je1Hekjhs8Az+XHUR7YwRrO6+MzCrCIR8YCks4FfSNqO9K8x/w/pKa8Pk54W+jjw69ZFaVYf3w1lZmal3A1lZmalnCzMzKyUk4WZmZVysjAzs1JOFmZmVsrJwszMSjlZmJlZqf8PR34aU3Z8H5QAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot('became_member_on_year',\n",
    "              data = processed_profile);\n",
    "plt.title('Year that customers became rewards members')\n",
    "plt.xlabel('Year');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "More customers became members in 2017, followed by 2018 then 2016. The first year the with records is 2013 and it's also the year with fewer members signing for the program."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3debxVdb3/8ddbQFFQQSVDBkFFEyJNCAdk+GlOZE43TX/XRO1GJZr9rpmW3Zyya2Wp17qWBSmloqkoDqk4izMaKoMDGskRBGRIUTHBz++P9T24OJxz1j6w99kHzvv5eOzHXvu7vuu7Pt+19t6fNe21FRGYmZk1ZqNqB2BmZi2fk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZISeLKpN0oqTJFWz/PEl/TsM9JS2T1KZMbf9W0n+l4eGSasrRbmpviKSXy9VeayXpakk/qWD7syV9sVLttyaSQtJO1Y6jIU4W9ZB0rKSnJL0naUEaPkWSqh3buoiINyKiY0SsbKxeqQksIr4VEReWI7a6H5SIeDQidilH261FuTc88hsaZk4WdUg6A7gc+AXwaWBb4FvAYGDjKoa2hnLtIaxv824pJLVtjfO2lq1i742I8CM9gC2B94B/K6i3CXAJ8AYwH/gtsGkaNxyoAc4AFgDzgJNy024NTATeAZ4GLgQm58Z/BpgELAZeBo7JjbsauBK4K8X5xXpi6w08DLyb2vk18Oc0rhcQQNv0+kTg9VT378C/A7sCy4GVwDJgaUPzTmU/qdPvHwJvA7OBf8/F9RDwH7nXJ9b2G3gkxfVemudXa9vL1d81tbEUmA4cVme5/Aa4M/XlKWDHBtZd7TIYBcxN6+eM3PiNgLOB14BFwI3AVnWm/Xpa94/U0/404Mu51+3S8tg9vd4LeDz143lgeK7uScDM1IfXgW/mxtUu37OAt4A/1ZlvY+utwWVDtmE0h+z9+CwwJJUfDPwL+Ci193wDy3M28ANgBrAE+CPQPjf+UGBq6u/jwOdy43oAtwAL07L+dSrfEXgglb0NXAt0qjPPM4EX0ntmDNlG3V9TH+8DOufqN7jMG+hPWdome7/+JI1fBtxO9vm/Ni3vZ4BeufoBfCet+7fJNlg3yo0/Ob0/lgD3ANvXmXY08Crw94p8P1ai0fX1kT4gK0hfpo3Uu4zsC38rYPP0JvjvNG54auMCsi+KEcD7tW8wYDzZF1AH4LPAm3zypdmB7IN7EtAW2CO9afql8VcD/yTby9ko/6HMxfYE8CuyhDY0vcHXSBZpXu8Au6RxXXPzOZFcAmto3qyZLFbk5j0sfdhq23+IBpJF7s2+U+71cFKySMtxFlki2hjYL/Vrl1xsi4FBqW/XAuMbWHe1y+D6tAz6k31ZfTGN/y7wJNA99eN3wPV1ph2Xpt20nva/D9yQe3048GIa7kb2BTgiLcMD0usuafyXyL4olZbf+8AedZbvz1Jc9c27ofXW4LIBjif7AmtLtoHzFul9BZxHeu808lmYTZYge5B9Hh7LvSf2INtg2hNoA4xM9TdJr58HLk3Lsj2wb5pup7RsNgG6kG1MXFZnnk+SfYl3S/N4Dvh8muYB4NxSlnkD/SlL22Tv+VlpnW5JllBfIdvQakv2Pvpjnc/Ag2k59kx1/yONOyK1tWua9kfA43WmnZSmXeO9UZbvx0p++a5vj/TBeatOWe1WwwdkX74i+xLMb53tTcrmZB/qD8glnPSG2yt9QD4CPpMb91M+SRZfBR6tM//f5d6cVwPjGom/J9kXSodc2XU0nCyWAv9W981Fw1864+opq5ss8vO+EfivNPwQa58shpB9ieW3sq4HzsvF8YfcuBHASw0so9plkF8HPwfGpOGZwP65cV3TOmubm3aHRtbBdmSJbIv0+ibg+2n4LNbcI7gHGNlAW7cCp+eWx7+oZwOhhPVW0rJJ45cAu6Xh8ygtWXyrTvuvpeErgQvr1H+ZLBHuTZakG90wS9McAfytzjzze603A1fmXp8G3LqWy7xsbZO958/Jjfsl8Nfc6y8DU+t8Bg7OvT4FuD8N/xX4em7cRmQbE9vnpt2vaFmuy8PnLFa3CNgmf8wvIvaJiE5p3EZkWzqbAc9KWippKXB3Kl/VTkSsyL1+H+iY6rQl23uo9Y/c8PbAnrXtprb/nezcSa38tHVtByyJiPcaaH+VVOerZOdj5km6U9JnGmm7aN40MO/tCqYpxXbAnIj4uE7b3XKv38oN1y7vxtRdB7Vxbg9MyC3/mWSHdrZtYNrVRMRcsq3rf5PUCTiEbGu+tu2j66zffckSEpIOkfSkpMVp3Ahgm1zzCyNieUG/6tPgspF0hqSZkv6Z5rllnXmWorFleUad/vZI43sA/6jzOamN6VOSxkt6U9I7wJ/riWl+bviDel7X9rHRZd6AcrZdalu1GluWl+fms5hsw7VbA9OWnZPF6p4APiQ7dNCQt8lWcr+I6JQeW0ZE0ZcTZFtSK8g+KLV65obnAA/n2u0U2dVL387ViUbanwd0ltShgfZXExH3RMQBZG/ul4DfF8yjsXnTwLznpuH3yJJsrXwCLDIX6CEp/37tSXYIb23VXQe1cc4BDqmzDtpHRH5eRcvhGrK91KOBJ3LTziHbEs233SEiLpa0CdlW7CXAtmkD5S6yL4RS51s0fjWShpBtHR9Ddpi0E9mhxtp5ltpeY8vyojr93Swirk/jejZwMva/07w/FxFbkC3Ltb0SscFlvpbtVbrtxpblN+vMa9OIeDxXv0nrv6mcLHIiYilwPvC/kr4iqaOkjSTtTnbYhrR1+3vgUkmfApDUTdJBJbS/kuyE3nmSNpPUl+w4bq07gJ0lfU1Su/T4gqRdS4z/H8AU4HxJG0val2xXdw2StpV0WPpy/5DsBFztJbXzge6S1ubqr9p5DyE7ufmXVD4VOCr1eyeyk8R584EdGmjzKbJk8/20TIanfo1fi/hq/VeKpR/ZOaIbUvlvgYskbQ8gqYukxjYe6nMr2fH608mOS9f6M/BlSQdJaiOpffp9SneyczGbkDYoJB0CHNjE+TZ1vW1OtvGyEGgr6cfAFnXa61UnSddntKTukrYiO69Uuyx/D3xL0p7KdJD0JUmbk13cMQ+4OJW3lzQ4F9cyYKmkbmQnnNdWY8t8XVWi7TMldZbUg+z9k39f/iC9X5G0paSj1y38pnGyqCMifg78J9mJygVkH5jfkW2B1Wbxs8hONj2ZdpPvA0r9TcCpZLueb5EdT/5jbt7vkn1BHEu2RfEWn5zQLNX/JTuhuBg4l9W/rPI2IjuhOTfVHUZ2jBSyk3jTgbckvd2Eeb9Fdsx7Ltmhl29FxEtp3KVkx9znk215X1tn2vOAa9Ju9jH5ERHxL+AwskM6bwP/C5yQa3ttPEy2Du8HLomIe1P55WQXL9wr6V2yk517NqXhiPiAbC+hN9nGQW35HLK91h+SfUHPIfsi3Cit+++QnedZQrYeJzaxT01db/eQHQt/heyQx3JWP5RRm+gXSXqukXauA+4lu4rndbIrgIiIKcA3yK7IW0K2vE9M41aSJfydyK4sqyE7LArZBtseZHs5d5Jbhk3V2DJf2zYr3PZtZFelTSXr+5g0rwlk3wXj03fONLLPQ7NROjli1ipI6kV2mXC7+o6Xl3E+PwZ2jojjKzUPs+bkH/aYlVk6HPN14GvVjsWsXHwYyqyMJH2D7HDEXyPikWrHY1YuPgxlZmaFvGdhZmaFNshzFttss0306tWr2mGYma1Xnn322bcjokt94zbIZNGrVy+mTJlS7TDMzNYrkuq94wP4MJSZmZXAycLMzAo5WZiZWaEN8pxFfT766CNqampYvnxtbtpptdq3b0/37t1p165dtUMxs2bUapJFTU0Nm2++Ob169ULr919pV01EsGjRImpqaujdu3e1wzGzZtRqDkMtX76crbfe2oliHUhi66239t6ZWSvUapIF4ERRBl6GZq1Tq0oWZma2dpwsWrgTTzyRm266qdphmFkr12pOcLcWK1asoG1br1ZrPQZfMbi4UvLYaY9VMJINW8X2LNJfDD4t6XlJ0yWdn8p7S3pK0quSbqj9C0hJm6TXs9L4Xrm2fpDKXy7l70ur5cILL+Qzn/kMBxxwAMcddxyXXHIJr732GgcffDADBgxgyJAhvPRS9uduJ554It/5znfYZ5992GGHHVbtPUQEp556Kn379uVLX/oSCxYsWNX+s88+y7BhwxgwYAAHHXQQ8+bNA2D48OH88Ic/ZNiwYVx++eXN33Ez2+BVchP0Q2C/iFgmqR0wWdJfyf6y9NKIGC/pt2R/EnNlel4SETtJOpbsLwS/mv6n+ligH7AdcJ+kndPfMrYYU6ZM4eabb+Zvf/sbK1asYI899mDAgAGMGjWK3/72t/Tp04ennnqKU045hQceeACAefPmMXnyZF566SUOO+wwvvKVrzBhwgRefvllXnzxRebPn0/fvn05+eST+eijjzjttNO47bbb6NKlCzfccAPnnHMOY8eOBWDp0qU8/PDD1VwEZrYBq1iyiOyPMpall+3SI4D9yP5fGLL/Yj6PLFkcnoYBbgJ+rezSm8OB8RHxIfB3SbOAQcATlYp9bUyePJnDDz+cTTfdFIAvf/nLLF++nMcff5yjj/7kf9U//PDDVcNHHHEEG220EX379mX+/PkAPPLIIxx33HG0adOG7bbbjv322w+Al19+mWnTpnHAAQcAsHLlSrp27bqqra9+9auYmVVKRQ9uS2pD9ufjOwG/AV4Dlub++7gG6JaGu5H+LD4iVkj6J7B1Kn8y12x+mvy8RgGjAHr27Fn2vhSp70+kPv74Yzp16sTUqVPrnWaTTTapd/r6Lk+NCPr168cTT9SfIzt06NDUkM3MSlbRq6EiYmVE7A50J9sb2LW+aum5vgv4o5HyuvO6KiIGRsTALl3qvR17Re27777cfvvtLF++nGXLlnHnnXey2Wab0bt3b/7yl7/Uxsjzzz/faDtDhw5l/PjxrFy5knnz5vHggw8CsMsuu7Bw4cJVyeKjjz5i+vTple2UmVnSLJfORsRS4CFgL6CTpNo9mu7A3DRcA/QASOO3BBbny+uZpsX4whe+wGGHHcZuu+3GUUcdxcCBA9lyyy259tprGTNmDLvtthv9+vXjtttua7SdI488kj59+tC/f3++/e1vM2zYMAA23nhjbrrpJs466yx22203dt99dx5//PHm6JqZWeX+g1tSF+CjiFgqaVPgXrKT1iOBm3MnuF+IiP+VNBroHxHfSie4j4qIYyT1A64j2zPZDrgf6NPYCe6BAwdG3T8/mjlzJrvuWt+OTfksW7aMjh078v777zN06FCuuuoq9thjj4rOsxqaY1malcqXzpaPpGcjYmB94yp5zqIrcE06b7ERcGNE3CFpBjBe0k+AvwFjUv0xwJ/SCezFZFdAERHTJd0IzABWAKNb2pVQtUaNGsWMGTNYvnw5I0eO3CAThZm1TpW8GuoF4PP1lL9OtpdQt3w5cHTd8jTuIuCicsdYbtddd121QzAzqwjf7sPMzAo5WZiZWSEnCzMzK+RkYWZmhVrt7UkHnDmurO09+4sTCuvMnj2bQw89lGnTpq0qO++88+jYsSPf+9736p3m6quvZsqUKfz6178uW6xmZk3lPQszMyvkZNFCDB8+nLPOOotBgwax88478+ijj65R584772Tvvffm7bffbvQW52eeeSaf/exn6d+/PzfccAMAp5xyChMnTgSyX4mffPLJAIwZM4Yf/ehHzJ49m1133ZVvfOMb9OvXjwMPPJAPPvigmXpvZi2dk0ULsmLFCp5++mkuu+wyzj///NXGTZgwgYsvvpi77rqLbbbZBvjkFud33HEHZ599NgC33HILU6dO5fnnn+e+++7jzDPPZN68eQwdOnRVAnrzzTeZMWMGkN0td8iQIQC8+uqrjB49munTp9OpUyduvvnm5uq6mbVwThbNqL67yebLjzrqKAAGDBjA7NmzV41/8MEH+dnPfsadd95J586dV5XXd4vzyZMnr7rF+bbbbsuwYcN45plnGDJkCI8++igzZsygb9++bLvttsybN48nnniCffbZB4DevXuz++671xuDmbVuThbNaOutt2bJkiWrlS1evHjVnkLtLcvbtGnDihUrVtXZYYcdePfdd3nllVdWm7a+W5w3dK+vbt26sWTJEu6++26GDh3KkCFDuPHGG+nYsSObb775Gu3VjcHMWjcni2bUsWNHunbtyv333w9kieLuu+9m3333bXS67bffnltuuYUTTjih8LbkQ4cO5YYbbmDlypUsXLiQRx55hEGDsrur7L333lx22WWrksUll1yy6hCUmVljWu2ls6Vc6loJ48aNY/To0ZxxxhkAnHvuuey4446F0+2yyy5ce+21HH300dx+++0N1jvyyCN54okn2G233ZDEz3/+cz796U8DMGTIEO6991522mkntt9+exYvXuxkYWYlqdgtyqupWrcoby28LK0l8S3Ky6exW5T7MJSZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr1Gp/Z/HGBf3L2l7PH79YWKdNmzb07//JfG+99VZ69epV1jjMzCqh1SaLath0002ZOnVqtcMwM2syH4YyM7NC3rNoRh988MGqu7r27t2bCRMmVDkiM7PSOFk0Ix+GMrP1lQ9DmZlZoYolC0k9JD0oaaak6ZJOT+XnSXpT0tT0GJGb5geSZkl6WdJBufKDU9ksSWdXKmYzM6tfJQ9DrQDOiIjnJG0OPCtpUhp3aURckq8sqS9wLNAP2A64T9LOafRvgAOAGuAZSRMjYsa6BFfKpa5mZpapWLKIiHnAvDT8rqSZQLdGJjkcGB8RHwJ/lzQLGJTGzYqI1wEkjU911ylZVMOyZcuqHYKZ2VpplnMWknoBnweeSkWnSnpB0lhJtX8q3Q2Yk5usJpU1VF53HqMkTZE0ZeHChWXugZlZ61bxZCGpI3Az8N2IeAe4EtgR2J1sz+OXtVXrmTwaKV+9IOKqiBgYEQO7dOlSltjNzCxT0UtnJbUjSxTXRsQtABExPzf+98Ad6WUN0CM3eXdgbhpuqNzMzJpBJa+GEjAGmBkRv8qVd81VOxKYloYnAsdK2kRSb6AP8DTwDNBHUm9JG5OdBJ9YqbjNzGxNldyzGAx8DXhRUu0v0X4IHCdpd7JDSbOBbwJExHRJN5KduF4BjI6IlQCSTgXuAdoAYyNiegXjNrMqavJNPjtvUZlAbDWVvBpqMvWfb7irkWkuAi6qp/yuxqYzM7PKarW3+xh8xeCytvfYaY8V1pHE8ccfz5/+9CcAVqxYQdeuXdlzzz254447CqY2M6se3+6jGXXo0IFp06bxwQcfADBp0iS6dWvspydmZi2Dk0UzO+SQQ7jzzjsBuP766znuuOOqHJGZWTEni2Z27LHHMn78eJYvX84LL7zAnnvuWe2QzMwKOVk0s8997nPMnj2b66+/nhEjRhRPYGbWArTaE9zVdNhhh/G9732Phx56iEWLFlU7HDOzQk4WVXDyySez5ZZb0r9/fx566KFqh2NmVqjVJotSLnWtlO7du3P66adXbf5mtn5o6iX+lfxea7XJohrqu0X58OHDGT58ePMHY2bWBD7BbWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAq12ktnHx46rKztDXvk4ZLq1dTUMHr0aGbMmMHHH3/MoYceyi9+8Qs23njjssZjZlZO3rNoRhHBUUcdxRFHHMGrr77KK6+8wrJlyzjnnHOqHZqZWaOcLJrRAw88QPv27TnppJMAaNOmDZdeeiljx47l/fffr3J0ZmYNc7JoRtOnT2fAgAGrlW2xxRb07NmTWbNmVSkqM7NiThbNKCKQ1vxb8obKzcxaCieLZtSvXz+mTJmyWtk777zDnDlz2HHHHasUlZlZMSeLZrT//vvz/vvvM27cOABWrlzJGWecwYknnshmm21W5ejMzBrWai+dLfVS13KSxIQJEzjllFO48MIL+fjjjxkxYgQ//elPmz0WM7OmaLXJolp69OjB7bffXu0wzMyaxIehzMysUMWShaQekh6UNFPSdEmnp/KtJE2S9Gp67pzKJel/JM2S9IKkPXJtjUz1X5U0slIxm5lZ/Sq5Z7ECOCMidgX2AkZL6gucDdwfEX2A+9NrgEOAPukxCrgSsuQCnAvsCQwCzq1NME0VEWvfGwO8DM1aq4oli4iYFxHPpeF3gZlAN+Bw4JpU7RrgiDR8ODAuMk8CnSR1BQ4CJkXE4ohYAkwCDm5qPO3bt2fRokX+slsHEcGiRYto3759tUMxs2bWLCe4JfUCPg88BWwbEfMgSyiSPpWqdQPm5CarSWUNldedxyiyPRJ69uy5Rgzdu3enpqaGhQsXrmNvWrf27dvTvXv3aodhZs2s4slCUkfgZuC7EfFOI79Urm9ENFK+ekHEVcBVAAMHDlxjfLt27ejdu3epYZuZWU5Fr4aS1I4sUVwbEbek4vnp8BLpeUEqrwF65CbvDsxtpNzMzJpJJa+GEjAGmBkRv8qNmgjUXtE0ErgtV35CuipqL+Cf6XDVPcCBkjqnE9sHpjIzM2smlTwMNRj4GvCipKmp7IfAxcCNkr4OvAEcncbdBYwAZgHvAycBRMRiSRcCz6R6F0TE4grGbWZmdVQsWUTEZOo/3wCwfz31AxjdQFtjgbHli87MzJrCv+A2M7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCJSULSfeXUmZmZhumRm/3Iak9sBmwTbqJX+3tO7YAtqtwbGZm1kIU3Rvqm8B3yRLDs3ySLN4BflPBuMzMrAVpNFlExOXA5ZJOi4grmikmMzNrYUq662xEXCFpH6BXfpqIGFehuMzMrAUpKVlI+hOwIzAVWJmKA3CyMDNrBUr9P4uBQN/0nxNmZtbKlPo7i2nApysZiJmZtVyl7llsA8yQ9DTwYW1hRBxWkajMzKxFKTVZnFfJIMzMrGUr9WqohysdiJmZtVylXg31LtnVTwAbA+2A9yJii0oFZmZmLUepexab519LOgIYVJGIzMysxVmru85GxK3AfmWOxczMWqhSD0MdlXu5EdnvLvybCzOzVqLUPYsv5x4HAe8Chzc2gaSxkhZImpYrO0/Sm5KmpseI3LgfSJol6WVJB+XKD05lsySd3ZTOmZlZeZR6zuKktWj7auDXrHlLkEsj4pJ8gaS+wLFAP7I73N4naec0+jfAAUAN8IykiRExYy3iMTOztVTqnx91lzQh7SnMl3SzpO6NTRMRjwCLS4zjcGB8RHwYEX8HZpGdQB8EzIqI1yPiX8B4CvZozMys/Er9Ud4fgeuAo9Pr41PZAWsxz1MlnQBMAc6IiCVAN+DJXJ2aVAYwp075nmsxT7MNyhsX9C+5bs8fv1jBSKy1KPWcRZeI+GNErEiPq4EuazG/K8nuXrs7MA/4ZSpXPXWjkfI1SBolaYqkKQsXLlyL0MzMrCGl7lm8Lel44Pr0+jhgUVNnFhHza4cl/R64I72sAXrkqnYH5qbhhsrrtn0VcBXAwIEDfaWWWTL4isEl133stMcqGImtz0rdszgZOAZ4i2yP4CtAk096S+qae3kk2d1sASYCx0raRFJvoA/wNPAM0EdSb0kbk50En9jU+ZqZ2bopdc/iQmBkOr+ApK2AS8iSSL0kXQ8MB7aRVAOcCwyXtDvZoaTZZP/xTURMl3QjMANYAYyOiJWpnVOBe4A2wNiImN7EPpqZ2ToqNVl8rjZRAETEYkmfb2yCiDiunuIxjdS/CLionvK7gLtKjNPMzCqg1MNQG0nqXPsi7VmUmmjMzGw9V+oX/i+BxyXdRHYI6Rjq2QswM7MNU6m/4B4naQrZzQMFHOVfUZuZtR4lH0pKycEJwsysFVqrW5SbmVnr4mRhZmaFnCzMzKyQk4WZmRXybyXMzJpRU+4YTOctKhdIE3nPwszMCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVqhiyULSWEkLJE3LlW0laZKkV9Nz51QuSf8jaZakFyTtkZtmZKr/qqSRlYrXzMwaVsk9i6uBg+uUnQ3cHxF9gPvTa4BDgD7pMQq4ErLkApwL7AkMAs6tTTBmZtZ8KpYsIuIRYHGd4sOBa9LwNcARufJxkXkS6CSpK3AQMCkiFkfEEmASayYgMzOrsOY+Z7FtRMwDSM+fSuXdgDm5ejWprKHyNUgaJWmKpCkLFy4se+BmZq1ZSznBrXrKopHyNQsjroqIgRExsEuXLmUNzsystWvuZDE/HV4iPS9I5TVAj1y97sDcRsrNzKwZNXeymAjUXtE0ErgtV35CuipqL+Cf6TDVPcCBkjqnE9sHpjIzM2tGbSvVsKTrgeHANpJqyK5quhi4UdLXgTeAo1P1u4ARwCzgfeAkgIhYLOlC4JlU74KIqHvS3MzMKqxiySIijmtg1P711A1gdAPtjAXGljE0MzNropZygtvMzFqwiu1ZmK0vBl8xuOS6j532WAUjMWu5vGdhZmaFvGdhZhU34MxxJdedsHkFA7G15j0LMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0JOFmZmVsjJwszMCjlZmJlZIScLMzMr5GRhZmaFnCzMzKyQk4WZmRVysjAzs0K+66xtkN64oH/plTtvUblAzDYQ3rMwM7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMysUFWShaTZkl6UNFXSlFS2laRJkl5Nz51TuST9j6RZkl6QtEc1YjYza82q+aO8/xMRb+denw3cHxEXSzo7vT4LOATokx57AlemZzOzFmHAmeNKrjth8woGUkEt6RfchwPD0/A1wENkyeJwYFxEBPCkpE6SukbEvKpE2QoNvmJwyXUfO+2xCkZiZtVSrWQRwL2SAvhdRFwFbFubACJinqRPpbrdgDm5aWtS2WrJQtIoYBRAz549Kxy+WXk1ZcsU1t+tU1t/VStZDI6IuSkhTJL0UiN1VU9ZrFGQJZyrADeIoI4AAAfCSURBVAYOHLjGeDMzW3tVOcEdEXPT8wJgAjAImC+pK0B6XpCq1wA9cpN3B+Y2X7RmZtbsyUJSB0mb1w4DBwLTgInAyFRtJHBbGp4InJCuitoL+KfPV5iZNa9qHIbaFpggqXb+10XE3ZKeAW6U9HXgDeDoVP8uYAQwC3gfOKn5QzYza92aPVlExOvAbvWULwL2r6c8gNHNEJqZmTXAv+A2M7NCThZmZlbIycLMzAo5WZiZWSEnCzMzK+RkYWZmhZwszMyskJOFmZkVcrIwM7NCThZmZlbIycLMzAo5WZiZWaGW9LeqZo1qDf9zbNZSec/CzMwKec+iDAZfMbhJ9R877bEKRWJmVhneszAzs0JOFmZmVsjJwszMCvmcRSv1xgX9S6/ceYvKBWJm6wXvWZiZWSEnCzMzK+RkYWZmhXzOwsxajYeHDiu57rBHHq5gJOsfJ4sG+ASwmdknnCzMzDYQldxzWm/OWUg6WNLLkmZJOrva8ZiZtSbrxZ6FpDbAb4ADgBrgGUkTI2JGdSNrWXxX1srb0I95b+j9s7W3XiQLYBAwKyJeB5A0HjgcaFKy8JepmdnaUURUO4ZCkr4CHBwR/5Fefw3YMyJOzdUZBYxKL3cBXm7GELcB3m7G+TU392/95v6tv5q7b9tHRJf6Rqwvexaqp2y1LBcRVwFXNU84q5M0JSIGVmPezcH9W7+5f+uvltS39eUEdw3QI/e6OzC3SrGYmbU660uyeAboI6m3pI2BY4GJVY7JzKzVWC8OQ0XECkmnAvcAbYCxETG9ymHlVeXwVzNy/9Zv7t/6q8X0bb04wW1mZtW1vhyGMjOzKnKyMDOzQk4W9ZDUQ9KDkmZKmi7p9FS+laRJkl5Nz51T+WckPSHpQ0nfy7XTXtLTkp5P7ZxfrT7llat/ufbaSPqbpDuauy/1KWf/JM2W9KKkqZKmVKM/dZW5f50k3STppdTe3tXoU52YyvX52yWtt9rHO5K+W61+pZjKue7+X2pjmqTrJbWvaOw+Z7EmSV2BrhHxnKTNgWeBI4ATgcURcbGy+1N1joizJH0K2D7VWRIRl6R2BHSIiGWS2gGTgdMj4skqdGuVcvUv195/AgOBLSLi0ObsS33K2T9Js4GBEdFifvRV5v5dAzwaEX9QdqXhZhGxtLn7lFfu92dqsw3wJtmPef/RXH2pJ45yfbd0I/s+6RsRH0i6EbgrIq6uVOzes6hHRMyLiOfS8LvATKAb2S1GrknVriFbgUTEgoh4BvioTjsREcvSy3bpUfXsXK7+AUjqDnwJ+EMzhF6ScvavJSpX/yRtAQwFxqR6/6p2okhxVGL97Q+8Vs1EAWXvW1tgU0ltgc2o8G/PnCwKSOoFfB54Ctg2IuZBttKBT5UwfRtJU4EFwKSIeKpy0TbduvYPuAz4PvBxhUJcJ2XoXwD3SnpW2S1lWpR17N8OwELgj+kw4h8kdahguE1WhvVX61jg+nLHty7WpW8R8SZwCfAGMA/4Z0TcW8l4nSwaIakjcDPw3Yh4Z23aiIiVEbE72a/OB0n6bDljXBfr2j9JhwILIuLZsgdXBuVYf8DgiNgDOAQYLWlo2QJcR2XoX1tgD+DKiPg88B7QYm7/X6b1Rzq8dhjwl3LFtq7K8NnrTLY30hvYDugg6fjyRrk6J4sGpHMMNwPXRsQtqXh+OuZYe+xxQantpd37h4CDyxzqWilT/wYDh6Xj+uOB/ST9uUIhN0m51l9EzE3PC4AJZHdArroy9a8GqMnt7d5Eljyqrsyfv0OA5yJifvkjbboy9e2LwN8jYmFEfATcAuxTqZjByaJe6cT0GGBmRPwqN2oiMDINjwRuK2ini6ROaXhTshX8Uvkjbppy9S8ifhAR3SOiF9lu/gMRUdGtm1KUcf11SCchSYdnDgSmlT/ipinj+nsLmCNpl1S0P0287X8llKt/OcfRQg5BlbFvbwB7Sdostbk/2fmPyokIP+o8gH3JjlW/AExNjxHA1sD9wKvpeatU/9NkW2nvAEvT8BbA54C/pXamAT+udt/K2b86bQ4H7qh238q8/nYAnk+P6cA51e5budcfsDswJbV1K9lVOBtS/zYDFgFbVrtfFejb+WQbn9OAPwGbVDJ2XzprZmaFfBjKzMwKOVmYmVkhJwszMyvkZGFmZoWcLMzMrJCThVkZKDNZ0iG5smMk3V3NuMzKxZfOmpVJupXLX8ju99OG7Br6gyPitXVos21ErChTiGZrzcnCrIwk/ZzsHksdgHcj4kJJI4HRwMbA48CpEfGxpKvIbq+xKXBDRFyQ2qgBfkd2a5jLIqLF3NPIWq+21Q7AbANzPvAc8C9gYNrbOBLYJyJWpARxLHAdcHZELE63mH5Q0k0RUXu7jfciYnA1OmBWHycLszKKiPck3QAsi4gPJX0R+AIwJbuFD5sCc1L14yR9nexzuB3Ql0/uzXRD80Zu1jgnC7Py+5hP/t9DwNiI+K98BUl9gNOBQRGxNN2tN/+3mO81S6RmJfLVUGaVdR9wjKRtACRtLakn2Y0K3wXeSbekPqiKMZoV8p6FWQVFxIuSzgfuk7QR2d9jfovsTq8zyO4Y+jrwWPWiNCvmq6HMzKyQD0OZmVkhJwszMyvkZGFmZoWcLMzMrJCThZmZFXKyMDOzQk4WZmZW6P8Dua32SgGL304AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(x = 'became_member_on_year',\n",
    "              hue = 'gender',\n",
    "              data = processed_profile);\n",
    "plt.xlabel('Year');\n",
    "plt.title('Gender distribution per year that became member');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In all years, there are either more male members signing up for the program (with exception of 2016, in which there were slight more female members registrations)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZcAAAEWCAYAAACqitpwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAfw0lEQVR4nO3de5wcdZ3u8c9DwiWAkIQMLCSRRMyqEVEhB+LlKEc0JCokKigckKA5xgvgehQQ1l3j4bLKsmdRVFCESGCRwOEa3UCMYIBFCAwQcgHZzAYXZokwmIAIgga+54/6jhSTnpnOTE03mTzv16tfU/WtX1X9upLup+vS1YoIzMzMqrRVsztgZmaDj8PFzMwq53AxM7PKOVzMzKxyDhczM6ucw8XMzCrncDEzs8o5XMzqIOk3kv4o6Q+lx/cGeJ0HSmrvpc3Fks4YyH6Y9cXQZnfAbDNySET8otmdMNsceM/FrB8kbSvpKUl7l2otuZeza45/WNKybPcrSfuU2v5G0omSlkt6WtIVkraTtANwA7BHaU9pjy7rng0cBZyc038q6SRJV3dp911J387hJZK+KemuXN/1kkaW2k7OPj4l6X5JBw7AZrMtgMPFrB8i4gXgGuDIUvnjwC0R8YSkfYG5wGeBXYAfAgskbdul/VRgPLAPcGxEPAtMAx6LiB3z8ViXdV8AXAb8Y04/BPgXYKqk4QCShgKfAC4tzXoM8GlgD2ADcG62HQ38K3AGMBI4EbhaUkt/tpFtmRwuZvW7Lj/Rdz4+k/Wf8Mpw+Z9ZA/gM8MOIWBoRL0bEPOAFYHKp/bkR8VhErAN+Crytrx2MiLXArcDhWZoKPBkR95SaXRoRKzPA/h74uKQhwNHAwohYGBEvRcRioBX4YF/7Y1suh4tZ/WZExPDS40dZvxkYJukASXtShMO1OW1P4CvlUALGUuw1dPptafg5YMd+9nMeRVCQfy/tMv3R0vB/AlsDo7Kvh3fp67uB3fvZH9sC+YS+WT9FxEuSrqTYe3kc+FlEPJOTHwXOjIgz+7LoPra5Djg/zwN9GDi5y/SxpeHXAn8Gnsy+XhoRn8Gsn7znYlaNn1Cc2ziKlw+JAfwI+Fzu1UjSDpI+JOk1dSzzcWAXSTv30uZ15UJEPA9clf24KyIe6TLP0ZImStoeOA24KiJepDhfc4ikgyUNyQsLDpQ0po6+mr2Cw8Wsfj/t8j2XzkNfRMRS4FmKw103lOqtFOddvgesB9qAY+tZWUT8GrgcWJOHqfao0ewiYGJOv65Unwe8hY0PiZG1iykOx20HfDHX9ygwHfhboINiT+Yk/D5hfSD/WJjZ4CPptcCvgb+KiN+X6kuAf4mIC5vVN9sy+BOJ2SAjaSvgy8D8crCYNZJP6JsNIvnly8cprgKb2uTu2BbMh8XMzKxyPixmZmaV82GxNGrUqBg3blyzu2Fmtlm55557noyIjW4R5HBJ48aNo7W1tdndMDPbrEj6z1p1HxYzM7PKOVzMzKxyDhczM6ucw8XMzCo3YOEiaa6kJyStrDHtREkhaVSOS9K5ktryF/n2LbWdKWl1PmaW6vtJWpHznCtJWR8paXG2XyxpxEA9RzMzq20g91wupsY3hCWNBT4AlO/UOg2YkI/ZwPnZdiQwBzgA2B+YUwqL87Nt53yd6zoFuCkiJgA35biZmTXQgIVLRNwKrKsx6RyK35co3xpgOnBJFO4EhkvaHTgYWBwR6yJiPbCY4idcdwd2iog7orjFwCXAjNKy5uXwvFLdzMwapKHnXCQdCvxXRNzfZdJoXvnreO1Z66neXqMOsFv+1GvnT77u2kN/ZktqldTa0dHRh2dkZma1NCxc8oeJvgZ8vdbkGrXoQ32TRMQFETEpIia1tGz0BVMzM+ujRn5Dfy9gPHB/nnsfA9wraX+KPY/yT6+OAR7L+oFd6kuyPqZGe4DHJe0eEWvz8NkTVXR+v5MuqWIxg8I9Zx/T7C6Y2atcw/ZcImJFROwaEeMiYhxFQOwbEb8FFgDH5FVjk4Gn85DWImCKpBF5In8KsCinPSNpcl4ldgxwfa5qAdB5VdnMUt3MzBpkIC9Fvhy4A3iDpHZJs3povhBYQ/ETsD8CvgAQEeuA04G783Fa1gA+D1yY8/wHL/+07LeAD0haTXFV2reqfF5mZta7ATssFhFH9jJ9XGk4gOO6aTcXmFuj3grsXaP+O+CgTeyumZlVyN/QNzOzyjlczMyscg4XMzOrnMPFzMwq53AxM7PKOVzMzKxyDhczM6ucw8XMzCrncDEzs8o5XMzMrHIOFzMzq5zDxczMKudwMTOzyjlczMyscg4XMzOrnMPFzMwq53AxM7PKOVzMzKxyDhczM6ucw8XMzCrncDEzs8oNWLhImivpCUkrS7WzJf1a0nJJ10oaXpp2qqQ2SQ9JOrhUn5q1NkmnlOrjJS2VtFrSFZK2yfq2Od6W08cN1HM0M7PaBnLP5WJgapfaYmDviNgH+HfgVABJE4EjgDfnPOdJGiJpCPB9YBowETgy2wKcBZwTEROA9cCsrM8C1kfE64Fzsp2ZmTXQgIVLRNwKrOtS+3lEbMjRO4ExOTwdmB8RL0TEw0AbsH8+2iJiTUT8CZgPTJck4H3AVTn/PGBGaVnzcvgq4KBsb2ZmDdLMcy6fBm7I4dHAo6Vp7Vnrrr4L8FQpqDrrr1hWTn86229E0mxJrZJaOzo6+v2EzMys0JRwkfQ1YANwWWepRrPoQ72nZW1cjLggIiZFxKSWlpaeO21mZnUb2ugVSpoJfBg4KCI63/TbgbGlZmOAx3K4Vv1JYLikobl3Um7fuax2SUOBnelyeM7MzAZWQ/dcJE0FvgocGhHPlSYtAI7IK73GAxOAu4C7gQl5Zdg2FCf9F2Qo/RI4LOefCVxfWtbMHD4MuLkUYmZm1gADtuci6XLgQGCUpHZgDsXVYdsCi/Mc+50R8bmIWCXpSuABisNlx0XEi7mc44FFwBBgbkSsylV8FZgv6QzgPuCirF8EXCqpjWKP5YiBeo5mZlbbgIVLRBxZo3xRjVpn+zOBM2vUFwILa9TXUFxN1rX+PHD4JnXWzMwq5W/om5lZ5RwuZmZWOYeLmZlVzuFiZmaVc7iYmVnlHC5mZlY5h4uZmVXO4WJmZpVzuJiZWeUcLmZmVjmHi5mZVc7hYmZmlXO4mJlZ5RwuZmZWOYeLmZlVzuFiZmaVc7iYmVnlHC5mZlY5h4uZmVXO4WJmZpVzuJiZWeUGLFwkzZX0hKSVpdpISYslrc6/I7IuSedKapO0XNK+pXlmZvvVkmaW6vtJWpHznCtJPa3DzMwaZyD3XC4GpnapnQLcFBETgJtyHGAaMCEfs4HzoQgKYA5wALA/MKcUFudn2875pvayDjMza5ABC5eIuBVY16U8HZiXw/OAGaX6JVG4ExguaXfgYGBxRKyLiPXAYmBqTtspIu6IiAAu6bKsWuswM7MGafQ5l90iYi1A/t0166OBR0vt2rPWU729Rr2ndWxE0mxJrZJaOzo6+vykzMzslV4tJ/RVoxZ9qG+SiLggIiZFxKSWlpZNnd3MzLrR6HB5PA9pkX+fyHo7MLbUbgzwWC/1MTXqPa3DzMwapNHhsgDovOJrJnB9qX5MXjU2GXg6D2ktAqZIGpEn8qcAi3LaM5Im51Vix3RZVq11mJlZgwwdqAVLuhw4EBglqZ3iqq9vAVdKmgU8AhyezRcCHwTagOeATwFExDpJpwN3Z7vTIqLzIoHPU1yRNgy4IR/0sA4zM2uQAQuXiDiym0kH1WgbwHHdLGcuMLdGvRXYu0b9d7XWYWZmjfNqOaFvZmaDiMPFzMwq53AxM7PKOVzMzKxyDhczM6ucw8XMzCrncDEzs8o5XMzMrHIOFzMzq5zDxczMKudwMTOzyjlczMyscg4XMzOrnMPFzMwq53AxM7PKOVzMzKxyDhczM6ucw8XMzCrncDEzs8o5XMzMrHJNCRdJ/1vSKkkrJV0uaTtJ4yUtlbRa0hWStsm22+Z4W04fV1rOqVl/SNLBpfrUrLVJOqXxz9DMbMvW8HCRNBr4IjApIvYGhgBHAGcB50TEBGA9MCtnmQWsj4jXA+dkOyRNzPneDEwFzpM0RNIQ4PvANGAicGS2NTOzBmnWYbGhwDBJQ4HtgbXA+4Crcvo8YEYOT89xcvpBkpT1+RHxQkQ8DLQB++ejLSLWRMSfgPnZ1szMGqSucJF0Uz21ekTEfwH/BDxCESpPA/cAT0XEhmzWDozO4dHAoznvhmy/S7neZZ7u6mZm1iA9hkueCxkJjJI0QtLIfIwD9ujLCiWNoNiTGJ/L2IHiEFZX0TlLN9M2tV6rL7MltUpq7ejo6K3rZmZWp6G9TP8s8CWKELiHl9+4f09xXqMv3g88HBEdAJKuAd4JDJc0NPdOxgCPZft2YCzQnofRdgbWleqdyvN0V3+FiLgAuABg0qRJNQPIzMw2XY97LhHxnYgYD5wYEa+LiPH5eGtEfK+P63wEmCxp+zx3chDwAPBL4LBsMxO4PocX5Dg5/eaIiKwfkVeTjQcmAHcBdwMT8uqzbShO+i/oY1/NzKwPettzASAivivpncC48jwRccmmrjAilkq6CrgX2ADcR7H38K/AfElnZO2inOUi4FJJbRR7LEfkclZJupIimDYAx0XEiwCSjgcWUVyJNjciVm1qP83MrO/qChdJlwJ7AcuAF7McwCaHC0BEzAHmdCmvobjSq2vb54HDu1nOmcCZNeoLgYV96ZuZmfVfXeECTAIm5uEos3575LS3NLsLrxqv/fqKZnfBrHL1fs9lJfBXA9kRMzMbPOrdcxkFPCDpLuCFzmJEHDogvTIzs81aveHyjYHshJmZDS71Xi12y0B3xMzMBo96rxZ7hpe/5b4NsDXwbETsNFAdMzOzzVe9ey6vKY9LmkGNy4bNzMygj3dFjojrKO5ibGZmtpF6D4t9tDS6FcX3XvydFzMzq6neq8UOKQ1vAH6DfyPFzMy6Ue85l08NdEfMzGzwqPfHwsZIulbSE5Iel3S1pDED3TkzM9s81XtC/8cUt63fg+JXHX+aNTMzs43UGy4tEfHjiNiQj4uBlgHsl5mZbcbqDZcnJR0taUg+jgZ+N5AdMzOzzVe94fJp4OPAb4G1FL8I6ZP8ZmZWU72XIp8OzIyI9QCSRgL/RBE6ZmZmr1Dvnss+ncECEBHrgLcPTJfMzGxzV2+4bCVpROdI7rnUu9djZmZbmHoD4v8Cv5J0FcVtXz5Ojd+uNzMzg/q/oX+JpFaKm1UK+GhEPDCgPTMzs81W3XdFjogHIuJ7EfHd/gaLpOGSrpL0a0kPSnqHpJGSFktanX9HZFtJOldSm6TlkvYtLWdmtl8taWapvp+kFTnPuZLUn/6amdmm6dMt9yvwHeDGiHgj8FbgQeAU4KaImADclOMA04AJ+ZgNnA9/Oe8zBziA4rdl5pTOC52fbTvnm9qA52RmZqnh4SJpJ+A9wEUAEfGniHiK4i7L87LZPGBGDk8HLonCncBwSbsDBwOLI2JdXsm2GJia03aKiDsiIoBLSssyM7MGaMaey+uADuDHku6TdKGkHYDdImItQP7dNduPBh4tzd+etZ7q7TXqG5E0W1KrpNaOjo7+PzMzMwOaEy5DgX2B8yPi7cCzvHwIrJZa50uiD/WNixEXRMSkiJjU0uJbpZmZVaUZ4dIOtEfE0hy/iiJsHs9DWuTfJ0rtx5bmHwM81kt9TI26mZk1SMPDJSJ+Czwq6Q1ZOgh4gOKW/p1XfM0Ers/hBcAxedXYZODpPGy2CJgiaUSeyJ8CLMppz0ianFeJHVNalpmZNUCzvmV/AnCZpG2ANRQ3wdwKuFLSLOAR4PBsuxD4INAGPJdtiYh1kk4H7s52p+VtaQA+D1wMDANuyIfZoPWu776r2V141bj9hNub3QWjSeESEcuASTUmHVSjbQDHdbOcucDcGvVWYO9+dtPMzPqoWd9zMTOzQczhYmZmlXO4mJlZ5RwuZmZWOYeLmZlVzuFiZmaVc7iYmVnlHC5mZlY5h4uZmVXO4WJmZpVzuJiZWeUcLmZmVjmHi5mZVc7hYmZmlXO4mJlZ5RwuZmZWOYeLmZlVzuFiZmaVc7iYmVnlHC5mZlY5h4uZmVWuaeEiaYik+yT9LMfHS1oqabWkKyRtk/Vtc7wtp48rLePUrD8k6eBSfWrW2iSd0ujnZma2pWvmnsvfAA+Wxs8CzomICcB6YFbWZwHrI+L1wDnZDkkTgSOANwNTgfMysIYA3wemAROBI7OtmZk1SFPCRdIY4EPAhTku4H3AVdlkHjAjh6fnODn9oGw/HZgfES9ExMNAG7B/PtoiYk1E/AmYn23NzKxBmrXn8m3gZOClHN8FeCoiNuR4OzA6h0cDjwLk9Kez/V/qXebprr4RSbMltUpq7ejo6O9zMjOz1PBwkfRh4ImIuKdcrtE0epm2qfWNixEXRMSkiJjU0tLSQ6/NzGxTDG3COt8FHCrpg8B2wE4UezLDJQ3NvZMxwGPZvh0YC7RLGgrsDKwr1TuV5+mubmZmDdDwPZeIODUixkTEOIoT8jdHxFHAL4HDstlM4PocXpDj5PSbIyKyfkReTTYemADcBdwNTMirz7bJdSxowFMzM7PUjD2X7nwVmC/pDOA+4KKsXwRcKqmNYo/lCICIWCXpSuABYANwXES8CCDpeGARMASYGxGrGvpMzMy2cE0Nl4hYAizJ4TUUV3p1bfM8cHg3858JnFmjvhBYWGFXzcxsE/gb+mZmVjmHi5mZVc7hYmZmlXO4mJlZ5V5NV4uZmTXdLe95b7O78Krx3ltv6fO83nMxM7PKOVzMzKxyDhczM6ucw8XMzCrncDEzs8o5XMzMrHIOFzMzq5zDxczMKudwMTOzyjlczMyscg4XMzOrnMPFzMwq53AxM7PKOVzMzKxyDhczM6ucw8XMzCrX8HCRNFbSLyU9KGmVpL/J+khJiyWtzr8jsi5J50pqk7Rc0r6lZc3M9qslzSzV95O0Iuc5V5Ia/TzNzLZkzdhz2QB8JSLeBEwGjpM0ETgFuCkiJgA35TjANGBCPmYD50MRRsAc4ABgf2BOZyBlm9ml+aY24HmZmVlqeLhExNqIuDeHnwEeBEYD04F52WweMCOHpwOXROFOYLik3YGDgcURsS4i1gOLgak5baeIuCMiAriktCwzM2uApp5zkTQOeDuwFNgtItZCEUDArtlsNPBoabb2rPVUb69Rr7X+2ZJaJbV2dHT09+mYmVlqWrhI2hG4GvhSRPy+p6Y1atGH+sbFiAsiYlJETGppaemty2ZmVqemhIukrSmC5bKIuCbLj+chLfLvE1lvB8aWZh8DPNZLfUyNupmZNUgzrhYTcBHwYET8c2nSAqDziq+ZwPWl+jF51dhk4Ok8bLYImCJpRJ7InwIsymnPSJqc6zqmtCwzM2uAoU1Y57uATwIrJC3L2t8C3wKulDQLeAQ4PKctBD4ItAHPAZ8CiIh1kk4H7s52p0XEuhz+PHAxMAy4IR9mZtYgDQ+XiPg3ap8XATioRvsAjutmWXOBuTXqrcDe/eimmZn1g7+hb2ZmlXO4mJlZ5RwuZmZWOYeLmZlVzuFiZmaVc7iYmVnlHC5mZlY5h4uZmVXO4WJmZpVzuJiZWeUcLmZmVjmHi5mZVc7hYmZmlXO4mJlZ5RwuZmZWOYeLmZlVzuFiZmaVc7iYmVnlHC5mZlY5h4uZmVXO4WJmZpUbtOEiaaqkhyS1STql2f0xM9uSDMpwkTQE+D4wDZgIHClpYnN7ZWa25RiU4QLsD7RFxJqI+BMwH5je5D6ZmW0xFBHN7kPlJB0GTI2I/5XjnwQOiIjju7SbDczO0TcADzW0o30zCniy2Z0YRLw9q+NtWa3NZXvuGREtXYtDm9GTBlCN2kYpGhEXABcMfHeqI6k1IiY1ux+Dhbdndbwtq7W5b8/BelisHRhbGh8DPNakvpiZbXEGa7jcDUyQNF7SNsARwIIm98nMbIsxKA+LRcQGSccDi4AhwNyIWNXkblVlszqMtxnw9qyOt2W1NuvtOShP6JuZWXMN1sNiZmbWRA4XMzOrnMOlnyQNl/SFZvejTNKxkvYojV+4udyhQNIXJT0o6TJJ20r6haRlkj7R7L6VSfqcpGMqWM44SSur6FMd62rItq1q29S5riWSKr9cd3P5f9gdSX/oZXqf3rckfUPSifW0HZQn9BtsOPAF4LxyUdKQiHixOV3iWGAlefl155dJNxNfAKZFxMOSJgNbR8Tb6p1Z0tCI2LCp0zZVRPygiuU02IBt27LNdNt01ZBt1UQ137cqFRF+9ONBcWuZPwLLKC6B/iXwE+CBnH4dcA+wCphdmu8PwJnA/cCdwG5ZP5wiGO4Hbs3aOOA24N58vLO0nJOBFdn+W8BhueyHsk/DgCXApGx/ZLZfCZzVW38GcLt9OfuwEvhS1n4A/Cn791WgDXg6n8dewH7ALbk9FwG753xLgH/IaV/psp5vUFx18/P8dxkCnJ3/VsuBz3a3LbO2F3BjrvM24I2l5Z4IvAm4q7SMccDyHO6uv/vlOu7Ivqzc3LYtxVGP3wDDS7U2YLfObdPd9st/gzUUX3YeDrwEvCfb3wa8HtgBmJv/TvcB03P6MIrX3HLgCmAp+X/71bqtctqOwI9zmcuBj9Xxejwr1/ELiltaLcntdmi2ORa4PrfvQ8Cc8vyl4ZN4+f/7/6nxvnV2d+2y/rVc/i+Ayzv/bXvdtgP5BrIlPCjeTFbm8IHAs8D40vSRpRfFSmCXHA/gkBz+R+DvcngFMDqHh+ff7YHtcngC0JrD04BfAdt3WdeS8guucxzYA3gEaKHYa70ZmNFTfwZom+2Xz3OHfNGtAt6e034DjCptz5/l8Nb5XFty/BMUl5h3Pr/zulnXN/IFOizHZ5e29bZAKzC+h215EzAhhw8Abi4tt/MNdBnwuhz+KvB3vfR3OfDeHK40XBq8bb8DfKq0bX5RY9t0t/1uBN4MfJjiDe1r+e/xcE7/B+DoztcB8O/5nL5c6ts+wAb6GC4N3lZnAd8ujY+g99fjtBy+luLD0dbAW4FlWT8WWAvswsvvL50fIv+Qf6dQfLgSxQeCnwHvofS+1Uu7zm20PbATRdDWFS4+LFa9uyLi4dL4FyV9JIfHUoTD7yg+Gf0s6/cAH8jh24GLJV0JXJO1rYHvSXob8CLw11l/P/DjiHgOICLW9dK3/wYsiYgOAEmXUfwHuq6H/gyEdwPXRsSz2Y9rgP9O8Qm1O28A9gYWS4Li0+/a0vQreph3QUT8MYenAPvk/ecAdqb4N9loW0raEXgn8P9ynVC8AXZ1JfBxij3HT+SjZn8l7UzxoeGWnPdSimCrSiO37RXA1yk+kR/RtV0v2+82iv9744FvAp+h+MR/d06fAhxaOr6/HfDanOdcgIhYLml5D8+rN43cVu+n2EYARMR6Se+h59fjjdl8BfBCRPxZ0gqKYOi0OCJ+V+r/uyk+MHWako/O57Qjxf/3R7r0r7t2r6HYRs/lOur+MrrDpXrPdg5IOpDiP9U7IuI5SUsoXiQAf478yEARGEMBIuJzkg4APgQsy0A5AXic4lPLVsDznaugxj3TelDrnmudavZngPTUj57mWRUR7+hm+rPd1LtOE3BCRCx6xcKlqWy8LbcCnorej7VfQfEGeg0QEbFa0ltq9VfS8BrrqVIjt+0dwOsltQAzgDO6TO9p+90GfI7i0/vXKQ7JHAjcWurTxyLiFTeTzTf0qrZfI7dVrddqva/Hl4AXACLiJUnl12bXZdZaxzcj4oevKErj6mz3pRrLrIuvFuu/ZyjSvZadgfUZLG8EJve2MEl7RcTSiPg6xR1Rx+Zy1kbES8AnKT4tQbGr/GlJ2+e8I3vp01LgvZJG5W/eHEnxabHRbgVmSNpe0g7ARyjebHryENAi6R0AkraW9OY+rHsR8HlJW+dy/jr7sNG2jIjfAw9LOjxrkvTWrguMiP+gCOS/5+VPrjX7GxFPAU9Lene2O6oPz6EnDdu2+eZ3LfDPwIOdn6BL03vafksp9mpeiojnKQ4tfrbU10XACco0kfT20vM7Kmt7Uxwa66tG/j/8OfCXu7JLGkE1r8cPSBopaRhFwN/eZfoiiv/XO+Z6R0valY3fI7prdyvwEUnDJL0GOKTejjlc+ilfULfn5aRnd5l8IzA0d91PpzhR3puzJa3I5d1KceL3PGCmpDspDok9m+u+keKeaa2SllGcYAa4GPhBXjo5rNTXtcCpFBcd3A/cGxHX9+Fp90tE3Jt9vIviBXZhRPR0KIIofpfnMOAsSfdTvBm9sw+rvxB4ALg3t/EPgaE9bMujgFm5zlV0/7tAVwBHUxwi662/nwK+L+kOipOqlWnCtu183t0dDqq5/SLiBeBRXn5N3EbxZrcix0+nOBy8PP+dTs/6+cCO+Zo6OZ9nnzR4W50BjJC0Muf7HxW9Hv+N4tDqMuDqiCgfEiMiOi9kuSMPqV0FvKb8viXp7B7a3Uvxb7sMuJrew/cvfPsXM7PNkKRjKU7gH99b22bwnouZmVXOey5mZlY577mYmVnlHC5mZlY5h4uZmVXO4WI2iEiaoc3kDtg2uDlczAaXGYDDxZrOV4uZNZiko4EvAttQfHlvObBnRJyc048F9ouIE2q0/UJEvKji9zq+Q3Hjxz9SfDlxL4r7wz2dj4/l3QPMGs57LmYNJOlNFDe2fFfec+tFiturf7TU7BPAFd207bxdzA7AnRHxVoo7OXwmIn5FcZeBkyLibQ4WaybfuNKssQ6iuI353XnbrGHAE8AaFT9KtZrizru3A8d10xYaexdrs03mcDFrLAHzIuLUVxSlWRS37f81xS3OI2/auFHb1Mi7WJttMh8WM2usm4DD8o6z5B1t96T47Z4ZFHfGvaKXtj3p6S7dZg3jcDFroIh4gOKXKn+ed/ZdTPEzuesp7ta8Z0Tc1VPbXlYxHzhJ0n2S9hqo52HWG18tZmZmlfOei5mZVc7hYmZmlXO4mJlZ5RwuZmZWOYeLmZlVzuFiZmaVc7iYmVnl/j+k7IMucX+RxQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.countplot(transcript_expanded['event'],\n",
    "              order = transcript_expanded['event'].value_counts().index);\n",
    "plt.title('Event type');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The event type that is most frequent is `transaction`, followed by `offer received`, `offer viewed` and `offer completed`. This makes sense, since it looks like a usual customer funnel for marketing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Merging datasets and data preprocessing\n",
    "We can get add more information to the `transcript_expanded` dataset by joining it to the `processed_portfolio` dataset, that contains information on the offers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "transcript_with_portfolio = transcript_expanded.merge(processed_portfolio,\n",
    "                                                      how = 'left',\n",
    "                                                      left_on = 'offer_id_redux',\n",
    "                                                      right_on = 'id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "# sort dataset by user and time, to make it easier for humans to see data\n",
    "transcript_with_portfolio = transcript_with_portfolio.sort_values(['person', 'time'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "event            offer_type   \n",
       "offer completed  bogo             15669\n",
       "                 discount         17910\n",
       "offer received   bogo             30499\n",
       "                 discount         30543\n",
       "                 informational    15235\n",
       "offer viewed     bogo             25449\n",
       "                 discount         21445\n",
       "                 informational    10831\n",
       "Name: offer_type, dtype: int64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transcript_with_portfolio.groupby(['event', 'offer_type'])['offer_type'].count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "BOGO and discount offers are the inly ones with an associated `offer completed` event. Also, the `transaction` event doesn't have an associated id, which means that the join won't add data to those.\n",
    "\n",
    "Basically, we can look into the BOGO and discount offers user funnels in this order:\n",
    "* offer received\n",
    "* offer viewed\n",
    "* transaction\n",
    "* offer completed\n",
    "\n",
    "For the informational offer, the funnel has this order:\n",
    "* offer received\n",
    "* offer viewed\n",
    "* transaction\n",
    "\n",
    "This means that we have to find a way to add the transaction data into the funnel. This can be done through the user id and the timestamp. If a user didn't get to the transaction part, it won't have an `offer completed` event as well. For the informational offer, the transaction data will only exist if it came after the `offer viewed` event. The timestamp will help us find the \"organic\" users: users that became members regardless of the campaigns. All those scenarios can be found by ordering data by time, offer id, person and event."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "view_to_complete = transcript_with_portfolio[['time', 'offer_id_redux', 'person', 'event']][(transcript_with_portfolio['event']=='transaction') | (transcript_with_portfolio['event'] == 'offer viewed')].groupby(['person','offer_id_redux'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>offer_id_redux</th>\n",
       "      <th>person</th>\n",
       "      <th>event</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>77705</th>\n",
       "      <td>192</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89291</th>\n",
       "      <td>228</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>transaction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>139992</th>\n",
       "      <td>372</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168412</th>\n",
       "      <td>414</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>transaction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>187554</th>\n",
       "      <td>456</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>228422</th>\n",
       "      <td>528</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>transaction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233413</th>\n",
       "      <td>540</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>237784</th>\n",
       "      <td>552</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>transaction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>258883</th>\n",
       "      <td>576</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>transaction</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85769</th>\n",
       "      <td>216</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>00116118485d4dfda04fdbaba9a87b5c</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>284472</th>\n",
       "      <td>630</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>00116118485d4dfda04fdbaba9a87b5c</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16179</th>\n",
       "      <td>6</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>0011e0d4e6b944f998e987f904e8c1e5</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75427</th>\n",
       "      <td>186</td>\n",
       "      <td>2298d6c36e964ae4a3e7e9706d1fb8c2</td>\n",
       "      <td>0011e0d4e6b944f998e987f904e8c1e5</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133370</th>\n",
       "      <td>354</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>0011e0d4e6b944f998e987f904e8c1e5</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>177937</th>\n",
       "      <td>432</td>\n",
       "      <td>0b1e1539f2cc45b7b9fa7c272da2e1d7</td>\n",
       "      <td>0011e0d4e6b944f998e987f904e8c1e5</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>222679</th>\n",
       "      <td>516</td>\n",
       "      <td>9b98b8c7a33c4b65b9aebfe6a799e6d9</td>\n",
       "      <td>0011e0d4e6b944f998e987f904e8c1e5</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18431</th>\n",
       "      <td>12</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>0020c2b971eb4e9188eac86d93036a77</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>174774</th>\n",
       "      <td>426</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>0020c2b971eb4e9188eac86d93036a77</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>293372</th>\n",
       "      <td>660</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>0020c2b971eb4e9188eac86d93036a77</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>67584</th>\n",
       "      <td>168</td>\n",
       "      <td>2298d6c36e964ae4a3e7e9706d1fb8c2</td>\n",
       "      <td>0020ccbbb6d84e358d3414a3ff76cffd</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>131476</th>\n",
       "      <td>348</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>0020ccbbb6d84e358d3414a3ff76cffd</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>165442</th>\n",
       "      <td>408</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>0020ccbbb6d84e358d3414a3ff76cffd</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>263808</th>\n",
       "      <td>582</td>\n",
       "      <td>9b98b8c7a33c4b65b9aebfe6a799e6d9</td>\n",
       "      <td>0020ccbbb6d84e358d3414a3ff76cffd</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27148</th>\n",
       "      <td>36</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>003d66b6608740288d6cc97a6903f4f0</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>105508</th>\n",
       "      <td>300</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>003d66b6608740288d6cc97a6903f4f0</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>140716</th>\n",
       "      <td>372</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>003d66b6608740288d6cc97a6903f4f0</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>173080</th>\n",
       "      <td>420</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>003d66b6608740288d6cc97a6903f4f0</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27263</th>\n",
       "      <td>36</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>00426fe3ffde4c6b9cb9ad6d077a13ea</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>76441</th>\n",
       "      <td>186</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>00426fe3ffde4c6b9cb9ad6d077a13ea</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>219332</th>\n",
       "      <td>510</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>004b041fbfe44859945daa2c7f79ee64</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26549</th>\n",
       "      <td>36</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>ffede3b700ac41d6a266fa1ba74b4f16</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>72825</th>\n",
       "      <td>180</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>ffede3b700ac41d6a266fa1ba74b4f16</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>269576</th>\n",
       "      <td>594</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>ffede3b700ac41d6a266fa1ba74b4f16</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103669</th>\n",
       "      <td>288</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>fff0f0aac6c547b9b263080f09a5586a</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>148670</th>\n",
       "      <td>396</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>fff0f0aac6c547b9b263080f09a5586a</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>287565</th>\n",
       "      <td>636</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>fff0f0aac6c547b9b263080f09a5586a</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>68851</th>\n",
       "      <td>168</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>fff29fb549084123bd046dbc5ceb4faa</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>132444</th>\n",
       "      <td>348</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>fff29fb549084123bd046dbc5ceb4faa</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>182159</th>\n",
       "      <td>438</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>fff29fb549084123bd046dbc5ceb4faa</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>221408</th>\n",
       "      <td>510</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>fff29fb549084123bd046dbc5ceb4faa</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>289457</th>\n",
       "      <td>642</td>\n",
       "      <td>ae264e3637204a6fb9bb56bc8210ddfd</td>\n",
       "      <td>fff29fb549084123bd046dbc5ceb4faa</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16325</th>\n",
       "      <td>6</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>fff3ba4757bd42088c044ca26d73817a</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>83982</th>\n",
       "      <td>210</td>\n",
       "      <td>9b98b8c7a33c4b65b9aebfe6a799e6d9</td>\n",
       "      <td>fff3ba4757bd42088c044ca26d73817a</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>233587</th>\n",
       "      <td>540</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>fff3ba4757bd42088c044ca26d73817a</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23028</th>\n",
       "      <td>24</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>fff7576017104bcc8677a8d63322b5e1</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>73167</th>\n",
       "      <td>180</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>fff7576017104bcc8677a8d63322b5e1</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168973</th>\n",
       "      <td>414</td>\n",
       "      <td>ae264e3637204a6fb9bb56bc8210ddfd</td>\n",
       "      <td>fff7576017104bcc8677a8d63322b5e1</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>226096</th>\n",
       "      <td>522</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>fff7576017104bcc8677a8d63322b5e1</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>177513</th>\n",
       "      <td>432</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>fff8957ea8b240a6b5e634b6ee8eafcf</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>293268</th>\n",
       "      <td>660</td>\n",
       "      <td>4d5c57ea9a6940dd891ad53e9dbe8da0</td>\n",
       "      <td>fff8957ea8b240a6b5e634b6ee8eafcf</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15591</th>\n",
       "      <td>6</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>fffad4f4828548d1b5583907f2e9906b</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>65899</th>\n",
       "      <td>168</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>fffad4f4828548d1b5583907f2e9906b</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>218451</th>\n",
       "      <td>510</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>fffad4f4828548d1b5583907f2e9906b</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>294735</th>\n",
       "      <td>666</td>\n",
       "      <td>9b98b8c7a33c4b65b9aebfe6a799e6d9</td>\n",
       "      <td>fffad4f4828548d1b5583907f2e9906b</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15836</th>\n",
       "      <td>6</td>\n",
       "      <td>fafdcd668e3743c1bb461111dcafc2a4</td>\n",
       "      <td>ffff82501cea40309d5fdd7edcca4a07</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>69626</th>\n",
       "      <td>174</td>\n",
       "      <td>0b1e1539f2cc45b7b9fa7c272da2e1d7</td>\n",
       "      <td>ffff82501cea40309d5fdd7edcca4a07</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>133074</th>\n",
       "      <td>354</td>\n",
       "      <td>2906b810c7d4411798c6938adc9daaa5</td>\n",
       "      <td>ffff82501cea40309d5fdd7edcca4a07</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>168022</th>\n",
       "      <td>414</td>\n",
       "      <td>2906b810c7d4411798c6938adc9daaa5</td>\n",
       "      <td>ffff82501cea40309d5fdd7edcca4a07</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>230690</th>\n",
       "      <td>534</td>\n",
       "      <td>9b98b8c7a33c4b65b9aebfe6a799e6d9</td>\n",
       "      <td>ffff82501cea40309d5fdd7edcca4a07</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>262475</th>\n",
       "      <td>582</td>\n",
       "      <td>2906b810c7d4411798c6938adc9daaa5</td>\n",
       "      <td>ffff82501cea40309d5fdd7edcca4a07</td>\n",
       "      <td>offer viewed</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>57730 rows × 4 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        time                    offer_id_redux  \\\n",
       "77705    192  5a8bc65990b245e5a138643cd4eb9837   \n",
       "89291    228                               NaN   \n",
       "139992   372  3f207df678b143eea3cee63160fa8bed   \n",
       "168412   414                               NaN   \n",
       "187554   456  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "228422   528                               NaN   \n",
       "233413   540  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "237784   552                               NaN   \n",
       "258883   576                               NaN   \n",
       "85769    216  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "284472   630  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "16179      6  3f207df678b143eea3cee63160fa8bed   \n",
       "75427    186  2298d6c36e964ae4a3e7e9706d1fb8c2   \n",
       "133370   354  5a8bc65990b245e5a138643cd4eb9837   \n",
       "177937   432  0b1e1539f2cc45b7b9fa7c272da2e1d7   \n",
       "222679   516  9b98b8c7a33c4b65b9aebfe6a799e6d9   \n",
       "18431     12  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "174774   426  4d5c57ea9a6940dd891ad53e9dbe8da0   \n",
       "293372   660  5a8bc65990b245e5a138643cd4eb9837   \n",
       "67584    168  2298d6c36e964ae4a3e7e9706d1fb8c2   \n",
       "131476   348  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "165442   408  5a8bc65990b245e5a138643cd4eb9837   \n",
       "263808   582  9b98b8c7a33c4b65b9aebfe6a799e6d9   \n",
       "27148     36  5a8bc65990b245e5a138643cd4eb9837   \n",
       "105508   300  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "140716   372  3f207df678b143eea3cee63160fa8bed   \n",
       "173080   420  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "27263     36  5a8bc65990b245e5a138643cd4eb9837   \n",
       "76441    186  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "219332   510  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "...      ...                               ...   \n",
       "26549     36  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "72825    180  4d5c57ea9a6940dd891ad53e9dbe8da0   \n",
       "269576   594  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "103669   288  3f207df678b143eea3cee63160fa8bed   \n",
       "148670   396  4d5c57ea9a6940dd891ad53e9dbe8da0   \n",
       "287565   636  3f207df678b143eea3cee63160fa8bed   \n",
       "68851    168  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "132444   348  4d5c57ea9a6940dd891ad53e9dbe8da0   \n",
       "182159   438  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "221408   510  4d5c57ea9a6940dd891ad53e9dbe8da0   \n",
       "289457   642  ae264e3637204a6fb9bb56bc8210ddfd   \n",
       "16325      6  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "83982    210  9b98b8c7a33c4b65b9aebfe6a799e6d9   \n",
       "233587   540  5a8bc65990b245e5a138643cd4eb9837   \n",
       "23028     24  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "73167    180  4d5c57ea9a6940dd891ad53e9dbe8da0   \n",
       "168973   414  ae264e3637204a6fb9bb56bc8210ddfd   \n",
       "226096   522  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "177513   432  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "293268   660  4d5c57ea9a6940dd891ad53e9dbe8da0   \n",
       "15591      6  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "65899    168  5a8bc65990b245e5a138643cd4eb9837   \n",
       "218451   510  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "294735   666  9b98b8c7a33c4b65b9aebfe6a799e6d9   \n",
       "15836      6  fafdcd668e3743c1bb461111dcafc2a4   \n",
       "69626    174  0b1e1539f2cc45b7b9fa7c272da2e1d7   \n",
       "133074   354  2906b810c7d4411798c6938adc9daaa5   \n",
       "168022   414  2906b810c7d4411798c6938adc9daaa5   \n",
       "230690   534  9b98b8c7a33c4b65b9aebfe6a799e6d9   \n",
       "262475   582  2906b810c7d4411798c6938adc9daaa5   \n",
       "\n",
       "                                  person         event  \n",
       "77705   0009655768c64bdeb2e877511632db8f  offer viewed  \n",
       "89291   0009655768c64bdeb2e877511632db8f   transaction  \n",
       "139992  0009655768c64bdeb2e877511632db8f  offer viewed  \n",
       "168412  0009655768c64bdeb2e877511632db8f   transaction  \n",
       "187554  0009655768c64bdeb2e877511632db8f  offer viewed  \n",
       "228422  0009655768c64bdeb2e877511632db8f   transaction  \n",
       "233413  0009655768c64bdeb2e877511632db8f  offer viewed  \n",
       "237784  0009655768c64bdeb2e877511632db8f   transaction  \n",
       "258883  0009655768c64bdeb2e877511632db8f   transaction  \n",
       "85769   00116118485d4dfda04fdbaba9a87b5c  offer viewed  \n",
       "284472  00116118485d4dfda04fdbaba9a87b5c  offer viewed  \n",
       "16179   0011e0d4e6b944f998e987f904e8c1e5  offer viewed  \n",
       "75427   0011e0d4e6b944f998e987f904e8c1e5  offer viewed  \n",
       "133370  0011e0d4e6b944f998e987f904e8c1e5  offer viewed  \n",
       "177937  0011e0d4e6b944f998e987f904e8c1e5  offer viewed  \n",
       "222679  0011e0d4e6b944f998e987f904e8c1e5  offer viewed  \n",
       "18431   0020c2b971eb4e9188eac86d93036a77  offer viewed  \n",
       "174774  0020c2b971eb4e9188eac86d93036a77  offer viewed  \n",
       "293372  0020c2b971eb4e9188eac86d93036a77  offer viewed  \n",
       "67584   0020ccbbb6d84e358d3414a3ff76cffd  offer viewed  \n",
       "131476  0020ccbbb6d84e358d3414a3ff76cffd  offer viewed  \n",
       "165442  0020ccbbb6d84e358d3414a3ff76cffd  offer viewed  \n",
       "263808  0020ccbbb6d84e358d3414a3ff76cffd  offer viewed  \n",
       "27148   003d66b6608740288d6cc97a6903f4f0  offer viewed  \n",
       "105508  003d66b6608740288d6cc97a6903f4f0  offer viewed  \n",
       "140716  003d66b6608740288d6cc97a6903f4f0  offer viewed  \n",
       "173080  003d66b6608740288d6cc97a6903f4f0  offer viewed  \n",
       "27263   00426fe3ffde4c6b9cb9ad6d077a13ea  offer viewed  \n",
       "76441   00426fe3ffde4c6b9cb9ad6d077a13ea  offer viewed  \n",
       "219332  004b041fbfe44859945daa2c7f79ee64  offer viewed  \n",
       "...                                  ...           ...  \n",
       "26549   ffede3b700ac41d6a266fa1ba74b4f16  offer viewed  \n",
       "72825   ffede3b700ac41d6a266fa1ba74b4f16  offer viewed  \n",
       "269576  ffede3b700ac41d6a266fa1ba74b4f16  offer viewed  \n",
       "103669  fff0f0aac6c547b9b263080f09a5586a  offer viewed  \n",
       "148670  fff0f0aac6c547b9b263080f09a5586a  offer viewed  \n",
       "287565  fff0f0aac6c547b9b263080f09a5586a  offer viewed  \n",
       "68851   fff29fb549084123bd046dbc5ceb4faa  offer viewed  \n",
       "132444  fff29fb549084123bd046dbc5ceb4faa  offer viewed  \n",
       "182159  fff29fb549084123bd046dbc5ceb4faa  offer viewed  \n",
       "221408  fff29fb549084123bd046dbc5ceb4faa  offer viewed  \n",
       "289457  fff29fb549084123bd046dbc5ceb4faa  offer viewed  \n",
       "16325   fff3ba4757bd42088c044ca26d73817a  offer viewed  \n",
       "83982   fff3ba4757bd42088c044ca26d73817a  offer viewed  \n",
       "233587  fff3ba4757bd42088c044ca26d73817a  offer viewed  \n",
       "23028   fff7576017104bcc8677a8d63322b5e1  offer viewed  \n",
       "73167   fff7576017104bcc8677a8d63322b5e1  offer viewed  \n",
       "168973  fff7576017104bcc8677a8d63322b5e1  offer viewed  \n",
       "226096  fff7576017104bcc8677a8d63322b5e1  offer viewed  \n",
       "177513  fff8957ea8b240a6b5e634b6ee8eafcf  offer viewed  \n",
       "293268  fff8957ea8b240a6b5e634b6ee8eafcf  offer viewed  \n",
       "15591   fffad4f4828548d1b5583907f2e9906b  offer viewed  \n",
       "65899   fffad4f4828548d1b5583907f2e9906b  offer viewed  \n",
       "218451  fffad4f4828548d1b5583907f2e9906b  offer viewed  \n",
       "294735  fffad4f4828548d1b5583907f2e9906b  offer viewed  \n",
       "15836   ffff82501cea40309d5fdd7edcca4a07  offer viewed  \n",
       "69626   ffff82501cea40309d5fdd7edcca4a07  offer viewed  \n",
       "133074  ffff82501cea40309d5fdd7edcca4a07  offer viewed  \n",
       "168022  ffff82501cea40309d5fdd7edcca4a07  offer viewed  \n",
       "230690  ffff82501cea40309d5fdd7edcca4a07  offer viewed  \n",
       "262475  ffff82501cea40309d5fdd7edcca4a07  offer viewed  \n",
       "\n",
       "[57730 rows x 4 columns]"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "view_to_complete.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the dataset is grouped and ordered, we can simply fill the offer id gaps with the previous value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [],
   "source": [
    "fill_offer_id = view_to_complete['offer_id_redux'].fillna(view_to_complete['offer_id_redux'].ffill()).ffill()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since this column is indexed, we can join it with the original dataset and add the missing values to either one of the redundant columns (we will actually make a new column with the values for sanity check). Then drop the unnecessary columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "with_full_offer_id = transcript_with_portfolio.join(fill_offer_id,\n",
    "                                                    rsuffix = '_right')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "with_full_offer_id['offer_id'] = np.where(with_full_offer_id['offer_id_redux'].isnull(),\n",
    "                                          with_full_offer_id['offer_id_redux_right'],\n",
    "                                          with_full_offer_id['offer_id_redux'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event</th>\n",
       "      <th>person</th>\n",
       "      <th>time</th>\n",
       "      <th>amount</th>\n",
       "      <th>reward_x</th>\n",
       "      <th>offer_id_redux</th>\n",
       "      <th>difficulty</th>\n",
       "      <th>duration</th>\n",
       "      <th>id</th>\n",
       "      <th>offer_type</th>\n",
       "      <th>reward_y</th>\n",
       "      <th>email</th>\n",
       "      <th>mobile</th>\n",
       "      <th>social</th>\n",
       "      <th>web</th>\n",
       "      <th>offer_id_redux_right</th>\n",
       "      <th>offer_id</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>55972</th>\n",
       "      <td>offer received</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>168</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>informational</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77705</th>\n",
       "      <td>offer viewed</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>192</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>informational</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>89291</th>\n",
       "      <td>transaction</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>228</td>\n",
       "      <td>22.16</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>113605</th>\n",
       "      <td>offer received</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>336</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>informational</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>139992</th>\n",
       "      <td>offer viewed</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>372</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>informational</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 event                            person  time  amount  \\\n",
       "55972   offer received  0009655768c64bdeb2e877511632db8f   168     NaN   \n",
       "77705     offer viewed  0009655768c64bdeb2e877511632db8f   192     NaN   \n",
       "89291      transaction  0009655768c64bdeb2e877511632db8f   228   22.16   \n",
       "113605  offer received  0009655768c64bdeb2e877511632db8f   336     NaN   \n",
       "139992    offer viewed  0009655768c64bdeb2e877511632db8f   372     NaN   \n",
       "\n",
       "        reward_x                    offer_id_redux  difficulty  duration  \\\n",
       "55972        NaN  5a8bc65990b245e5a138643cd4eb9837         0.0       3.0   \n",
       "77705        NaN  5a8bc65990b245e5a138643cd4eb9837         0.0       3.0   \n",
       "89291        NaN                               NaN         NaN       NaN   \n",
       "113605       NaN  3f207df678b143eea3cee63160fa8bed         0.0       4.0   \n",
       "139992       NaN  3f207df678b143eea3cee63160fa8bed         0.0       4.0   \n",
       "\n",
       "                                      id     offer_type  reward_y  email  \\\n",
       "55972   5a8bc65990b245e5a138643cd4eb9837  informational       0.0    1.0   \n",
       "77705   5a8bc65990b245e5a138643cd4eb9837  informational       0.0    1.0   \n",
       "89291                                NaN            NaN       NaN    NaN   \n",
       "113605  3f207df678b143eea3cee63160fa8bed  informational       0.0    1.0   \n",
       "139992  3f207df678b143eea3cee63160fa8bed  informational       0.0    1.0   \n",
       "\n",
       "        mobile  social  web              offer_id_redux_right  \\\n",
       "55972      1.0     1.0  0.0                               NaN   \n",
       "77705      1.0     1.0  0.0  5a8bc65990b245e5a138643cd4eb9837   \n",
       "89291      NaN     NaN  NaN  5a8bc65990b245e5a138643cd4eb9837   \n",
       "113605     1.0     0.0  1.0                               NaN   \n",
       "139992     1.0     0.0  1.0  3f207df678b143eea3cee63160fa8bed   \n",
       "\n",
       "                                offer_id  \n",
       "55972   5a8bc65990b245e5a138643cd4eb9837  \n",
       "77705   5a8bc65990b245e5a138643cd4eb9837  \n",
       "89291   5a8bc65990b245e5a138643cd4eb9837  \n",
       "113605  3f207df678b143eea3cee63160fa8bed  \n",
       "139992  3f207df678b143eea3cee63160fa8bed  "
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_full_offer_id.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "with_full_offer_id = with_full_offer_id.drop(['offer_id_redux', \n",
    "                                              'offer_id_redux_right'],\n",
    "                                             axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['event', 'person', 'time', 'amount', 'reward_x', 'difficulty',\n",
       "       'duration', 'id', 'offer_type', 'reward_y', 'email', 'mobile', 'social',\n",
       "       'web', 'offer_id'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_full_offer_id.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can merge it again to get the transaction events data. This means we'll duplicate many columns, but all we have to do is drop them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "with_transaction = with_full_offer_id.merge(processed_portfolio,\n",
    "                                            how = 'left',\n",
    "                                            left_on = 'offer_id',\n",
    "                                            right_on = 'id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['event', 'person', 'time', 'amount', 'reward_x', 'difficulty_x',\n",
       "       'duration_x', 'id_x', 'offer_type_x', 'reward_y', 'email_x', 'mobile_x',\n",
       "       'social_x', 'web_x', 'offer_id', 'difficulty_y', 'duration_y', 'id_y',\n",
       "       'offer_type_y', 'reward', 'email_y', 'mobile_y', 'social_y', 'web_y'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_transaction.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>reward</th>\n",
       "      <th>reward_x</th>\n",
       "      <th>reward_y</th>\n",
       "      <th>difficulty_x</th>\n",
       "      <th>difficulty_y</th>\n",
       "      <th>duration_x</th>\n",
       "      <th>duration_y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306504</th>\n",
       "      <td>2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306505</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306506</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306507</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306508</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306509</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306510</th>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>20</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306511</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306512</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306513</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306514</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>20</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306515</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306516</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306517</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306518</th>\n",
       "      <td>2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306519</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306520</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306521</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306522</th>\n",
       "      <td>2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306523</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306524</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306525</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306526</th>\n",
       "      <td>5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306527</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306528</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306529</th>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306530</th>\n",
       "      <td>2</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306531</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>10</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306532</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>306533</th>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>306534 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        reward  reward_x  reward_y  difficulty_x  difficulty_y  duration_x  \\\n",
       "0            0       NaN       0.0           0.0             0         3.0   \n",
       "1            0       NaN       0.0           0.0             0         3.0   \n",
       "2            0       NaN       NaN           NaN             0         NaN   \n",
       "3            0       NaN       0.0           0.0             0         4.0   \n",
       "4            0       NaN       0.0           0.0             0         4.0   \n",
       "5            5       NaN       5.0           5.0             5         5.0   \n",
       "6            0       NaN       NaN           NaN             0         NaN   \n",
       "7            5       5.0       5.0           5.0             5         5.0   \n",
       "8            5       NaN       5.0           5.0             5         5.0   \n",
       "9            2       NaN       2.0          10.0            10        10.0   \n",
       "10           5       NaN       NaN           NaN             5         NaN   \n",
       "11           2       2.0       2.0          10.0            10        10.0   \n",
       "12           2       NaN       2.0          10.0            10        10.0   \n",
       "13           2       NaN       NaN           NaN            10         NaN   \n",
       "14           2       NaN       2.0          10.0            10         7.0   \n",
       "15           2       NaN       NaN           NaN            10         NaN   \n",
       "16           2       2.0       2.0          10.0            10         7.0   \n",
       "17           2       NaN       NaN           NaN            10         NaN   \n",
       "18           2       NaN       NaN           NaN            10         NaN   \n",
       "19           2       NaN       NaN           NaN            10         NaN   \n",
       "20           5       NaN       5.0           5.0             5         5.0   \n",
       "21           5       NaN       5.0           5.0             5         5.0   \n",
       "22           5       NaN       NaN           NaN             5         NaN   \n",
       "23           5       NaN       NaN           NaN             5         NaN   \n",
       "24           5       NaN       NaN           NaN             5         NaN   \n",
       "25           5       NaN       5.0           5.0             5         5.0   \n",
       "26           5       NaN       5.0           5.0             5         5.0   \n",
       "27           0       NaN       0.0           0.0             0         4.0   \n",
       "28           0       NaN       0.0           0.0             0         4.0   \n",
       "29           0       NaN       NaN           NaN             0         NaN   \n",
       "...        ...       ...       ...           ...           ...         ...   \n",
       "306504       2       2.0       2.0          10.0            10        10.0   \n",
       "306505       2       NaN       NaN           NaN            10         NaN   \n",
       "306506       2       NaN       NaN           NaN            10         NaN   \n",
       "306507       5       NaN       5.0          20.0            20        10.0   \n",
       "306508       5       NaN       5.0          20.0            20        10.0   \n",
       "306509       5       NaN       NaN           NaN            20         NaN   \n",
       "306510       5       5.0       5.0          20.0            20        10.0   \n",
       "306511       5       NaN       NaN           NaN            20         NaN   \n",
       "306512       5       NaN       NaN           NaN            20         NaN   \n",
       "306513       5       NaN       NaN           NaN            20         NaN   \n",
       "306514       5       NaN       NaN           NaN            20         NaN   \n",
       "306515       2       NaN       2.0          10.0            10         7.0   \n",
       "306516       2       NaN       2.0          10.0            10         7.0   \n",
       "306517       2       NaN       NaN           NaN            10         NaN   \n",
       "306518       2       2.0       2.0          10.0            10         7.0   \n",
       "306519       2       NaN       2.0          10.0            10         7.0   \n",
       "306520       2       NaN       2.0          10.0            10         7.0   \n",
       "306521       2       NaN       NaN           NaN            10         NaN   \n",
       "306522       2       2.0       2.0          10.0            10         7.0   \n",
       "306523       2       NaN       NaN           NaN            10         NaN   \n",
       "306524       5       NaN       5.0           5.0             5         7.0   \n",
       "306525       2       NaN       NaN           NaN            10         NaN   \n",
       "306526       5       5.0       5.0           5.0             5         7.0   \n",
       "306527       5       NaN       5.0           5.0             5         7.0   \n",
       "306528       2       NaN       2.0          10.0            10         7.0   \n",
       "306529       5       NaN       NaN           NaN             5         NaN   \n",
       "306530       2       2.0       2.0          10.0            10         7.0   \n",
       "306531       2       NaN       2.0          10.0            10         7.0   \n",
       "306532       2       NaN       NaN           NaN            10         NaN   \n",
       "306533       2       NaN       NaN           NaN            10         NaN   \n",
       "\n",
       "        duration_y  \n",
       "0                3  \n",
       "1                3  \n",
       "2                3  \n",
       "3                4  \n",
       "4                4  \n",
       "5                5  \n",
       "6                4  \n",
       "7                5  \n",
       "8                5  \n",
       "9               10  \n",
       "10               5  \n",
       "11              10  \n",
       "12              10  \n",
       "13              10  \n",
       "14               7  \n",
       "15              10  \n",
       "16               7  \n",
       "17              10  \n",
       "18              10  \n",
       "19              10  \n",
       "20               5  \n",
       "21               5  \n",
       "22               5  \n",
       "23               5  \n",
       "24               5  \n",
       "25               5  \n",
       "26               5  \n",
       "27               4  \n",
       "28               4  \n",
       "29               4  \n",
       "...            ...  \n",
       "306504          10  \n",
       "306505          10  \n",
       "306506          10  \n",
       "306507          10  \n",
       "306508          10  \n",
       "306509          10  \n",
       "306510          10  \n",
       "306511          10  \n",
       "306512          10  \n",
       "306513          10  \n",
       "306514          10  \n",
       "306515           7  \n",
       "306516           7  \n",
       "306517           7  \n",
       "306518           7  \n",
       "306519           7  \n",
       "306520           7  \n",
       "306521           7  \n",
       "306522           7  \n",
       "306523           7  \n",
       "306524           7  \n",
       "306525           7  \n",
       "306526           7  \n",
       "306527           7  \n",
       "306528           7  \n",
       "306529           7  \n",
       "306530           7  \n",
       "306531           7  \n",
       "306532           7  \n",
       "306533           7  \n",
       "\n",
       "[306534 rows x 7 columns]"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_transaction[['reward',\n",
    "                  'reward_x', \n",
    "                  'reward_y', \n",
    "                  'difficulty_x',\n",
    "                  'difficulty_y',\n",
    "                  'duration_x',\n",
    "                  'duration_y']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "reward               0\n",
       "reward_x        272955\n",
       "reward_y        138953\n",
       "difficulty_x    138953\n",
       "difficulty_y         0\n",
       "duration_x      138953\n",
       "duration_y           0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_transaction[['reward',\n",
    "                  'reward_x', \n",
    "                  'reward_y', \n",
    "                  'difficulty_x',\n",
    "                  'difficulty_y',\n",
    "                  'duration_x',\n",
    "                  'duration_y']].isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll keep the `reward_y` column instead of the other ones because the lack of information there is related to the event type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['event', 'person', 'time', 'amount', 'reward_x', 'difficulty_x',\n",
       "       'duration_x', 'id_x', 'offer_type_x', 'reward_y', 'email_x', 'mobile_x',\n",
       "       'social_x', 'web_x', 'offer_id', 'difficulty_y', 'duration_y', 'id_y',\n",
       "       'offer_type_y', 'reward', 'email_y', 'mobile_y', 'social_y', 'web_y'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_transaction.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['event', 'person', 'time', 'amount', 'reward_y', 'offer_id',\n",
       "       'difficulty_y', 'duration_y', 'offer_type_y', 'email_y', 'mobile_y',\n",
       "       'social_y', 'web_y'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_transaction = with_transaction.drop(['reward',\n",
    "                                          'reward_x',\n",
    "                                          'difficulty_x',\n",
    "                                          'duration_x',\n",
    "                                          'id_x',\n",
    "                                          'id_y',\n",
    "                                          'offer_type_x', \n",
    "                                          'email_x',\n",
    "                                          'mobile_x',\n",
    "                                          'social_x', \n",
    "                                          'web_x'\n",
    "                                         ],\n",
    "                                         axis = 1)\n",
    "with_transaction.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "with_transaction = with_transaction.rename(columns = {'reward_y': 'reward',\n",
    "                                                      'difficulty_y': 'difficulty',\n",
    "                                                      'duration_y': 'duration',\n",
    "                                                      'offer_type_y': 'offer_type',\n",
    "                                                      'email_y': 'email',\n",
    "                                                      'mobile_y': 'mobile',\n",
    "                                                      'social_y': 'social',\n",
    "                                                      'web_y': 'web'\n",
    "                                                     })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>event</th>\n",
       "      <th>person</th>\n",
       "      <th>time</th>\n",
       "      <th>amount</th>\n",
       "      <th>reward</th>\n",
       "      <th>offer_id</th>\n",
       "      <th>difficulty</th>\n",
       "      <th>duration</th>\n",
       "      <th>offer_type</th>\n",
       "      <th>email</th>\n",
       "      <th>mobile</th>\n",
       "      <th>social</th>\n",
       "      <th>web</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>offer received</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>168</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>informational</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>offer viewed</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>192</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>informational</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>transaction</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>228</td>\n",
       "      <td>22.16</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>informational</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>offer received</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>336</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>informational</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>offer viewed</td>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>372</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>informational</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            event                            person  time  amount  reward  \\\n",
       "0  offer received  0009655768c64bdeb2e877511632db8f   168     NaN     0.0   \n",
       "1    offer viewed  0009655768c64bdeb2e877511632db8f   192     NaN     0.0   \n",
       "2     transaction  0009655768c64bdeb2e877511632db8f   228   22.16     NaN   \n",
       "3  offer received  0009655768c64bdeb2e877511632db8f   336     NaN     0.0   \n",
       "4    offer viewed  0009655768c64bdeb2e877511632db8f   372     NaN     0.0   \n",
       "\n",
       "                           offer_id  difficulty  duration     offer_type  \\\n",
       "0  5a8bc65990b245e5a138643cd4eb9837           0         3  informational   \n",
       "1  5a8bc65990b245e5a138643cd4eb9837           0         3  informational   \n",
       "2  5a8bc65990b245e5a138643cd4eb9837           0         3  informational   \n",
       "3  3f207df678b143eea3cee63160fa8bed           0         4  informational   \n",
       "4  3f207df678b143eea3cee63160fa8bed           0         4  informational   \n",
       "\n",
       "   email  mobile  social  web  \n",
       "0      1       1       1    0  \n",
       "1      1       1       1    0  \n",
       "2      1       1       1    0  \n",
       "3      1       1       0    1  \n",
       "4      1       1       0    1  "
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "with_transaction.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Finding successfull offers\n",
    "### Defining success\n",
    "To train the proposed models, we need to define success. For this problem, it makes sense that sucess is when a user has completed the funnels mentioned above successfully.\n",
    "\n",
    "Let's start by finding which users got to which steps of the funnel. For that, we'll separate the events in it's own tables adn create auxiliary columns. Then we'll join those datasets to come up with a single dataset that has the offer id, the person id, and three booleans for received, viewed and completed. We'll also include the considerations for the order of the performed events to be considered valid in this step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "def creating_event_df(original_df, event):\n",
    "    '''\n",
    "    this function takes in a dataframe that contains the 'event', \n",
    "    'person' and 'time' columns and creates a new dataframe for only \n",
    "    one selected event sorted by people and time.\n",
    "    \n",
    "    INPUT\n",
    "    - original_df (df): dataframe that contains the 'event', \n",
    "      'person' and 'time' columns\n",
    "    - event (str): the target event\n",
    "    \n",
    "    OUTPUT\n",
    "    - target_df (df): sorted dataframe with the desired filters\n",
    "    '''\n",
    "    target_df = original_df[original_df['event'] == event].copy()\n",
    "    target_df.sort_values(['person', 'time'])\n",
    "    \n",
    "    return target_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "offer_received = creating_event_df(with_transaction, 'offer received')\n",
    "\n",
    "offer_received.rename(columns = {'time': 'time_received'},\n",
    "                      inplace = True)\n",
    "\n",
    "offer_received = offer_received.drop(['event',\n",
    "                                      'amount',\n",
    "                                      'difficulty',\n",
    "                                      'offer_type',\n",
    "                                      'email',\n",
    "                                      'mobile',\n",
    "                                      'social',\n",
    "                                      'web'],\n",
    "                                     axis = 1)\n",
    "\n",
    "offer_received['received'] = np.ones(len(offer_received))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "offer_viewed = creating_event_df(with_transaction, 'offer viewed')\n",
    "\n",
    "offer_viewed.rename(columns = {'time': 'time_viewed'},\n",
    "                    inplace = True)\n",
    "\n",
    "offer_viewed = offer_viewed.drop(['event',\n",
    "                                  'amount',\n",
    "                                  'reward',\n",
    "                                  'duration',\n",
    "                                  'difficulty',\n",
    "                                  'offer_type',\n",
    "                                  'email',\n",
    "                                  'mobile',\n",
    "                                  'social',\n",
    "                                  'web'],\n",
    "                                axis = 1)\n",
    "\n",
    "offer_viewed['viewed'] = np.ones(len(offer_viewed))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "offer_transaction = creating_event_df(with_transaction, 'transaction')\n",
    "\n",
    "offer_transaction.rename(columns = {'time': 'time_transaction'},\n",
    "                         inplace = True)\n",
    "\n",
    "offer_transaction = offer_transaction.drop(['event',\n",
    "                                            'reward',\n",
    "                                            'duration',\n",
    "                                            'difficulty',\n",
    "                                            'offer_type',\n",
    "                                            'email',\n",
    "                                            'mobile',\n",
    "                                            'social',\n",
    "                                            'web'],\n",
    "                                           axis = 1)\n",
    "\n",
    "offer_transaction['has_transaction'] = np.ones(len(offer_transaction))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "offer_completed = creating_event_df(with_transaction, 'offer completed')\n",
    "\n",
    "offer_completed.rename(columns = {'time': 'time_completed'},\n",
    "                       inplace = True)\n",
    "\n",
    "offer_completed = offer_completed.drop(['event',\n",
    "                                        'amount',\n",
    "                                        'reward',\n",
    "                                        'duration',\n",
    "                                        'difficulty',\n",
    "                                        'offer_type',\n",
    "                                        'email',\n",
    "                                        'mobile',\n",
    "                                        'social',\n",
    "                                        'web'],\n",
    "                                      axis = 1)\n",
    "\n",
    "offer_completed['completed'] = np.ones(len(offer_completed))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "# merge event dataframes into one\n",
    "offers_received_viewed = offer_received.merge(offer_viewed,\n",
    "                                              how = 'left',\n",
    "                                              on = ['person', 'offer_id'])\n",
    "\n",
    "offers_received_viewed_transaction = offers_received_viewed.merge(offer_transaction,\n",
    "                                                                  how = 'left',\n",
    "                                                                  on = ['person', 'offer_id'])\n",
    "\n",
    "offers = offers_received_viewed_transaction.merge(offer_completed,\n",
    "                                                  how = 'left',\n",
    "                                                  on = ['person', 'offer_id'])\n",
    "\n",
    "# reorder columns\n",
    "offers = offers[['person',\n",
    "                 'offer_id', \n",
    "                 'received', \n",
    "                 'viewed', \n",
    "                 'has_transaction',\n",
    "                 'completed', \n",
    "                 'duration', \n",
    "                 'time_received',\n",
    "                 'time_viewed',\n",
    "                 'time_transaction',\n",
    "                 'time_completed',\n",
    "                 'amount',\n",
    "                 'reward']]\n",
    "\n",
    "# add expiration time for the offer\n",
    "offers['time_expiry'] = offers['duration'] + offers['time_received']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/julia.tessler/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:11: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  # This is added back by InteractiveShellApp.init_path()\n"
     ]
    }
   ],
   "source": [
    "# filter only pairs of events and person where the \n",
    "# funnel order was followed correctly and the offer\n",
    "# was successful\n",
    "successful_offers = offers[(offers['has_transaction'] == 1) &                          # the offer must have had a transaction\n",
    "                           (offers['time_transaction'] >= offers['time_viewed']) &     # the moment the transaction happened must be after the offer was viewed\n",
    "                           (offers['time_viewed'] >= offers['time_received']) &        # the moment the offer was viewed must be after the offer was received\n",
    "                           ((offers['time_expiry'] < offers['time_completed']) |       # the offers must have been completed - either by transaction in the case\n",
    "                           (offers['time_expiry'] < offers['time_transaction']))]      ## informational offers or by event - before the expiration time\n",
    "\n",
    "# add a column of ones signaling those are sucessful offers\n",
    "successful_offers['is_successful'] = np.ones(len(successful_offers))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(217659, 15)"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "successful_offers.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/julia.tessler/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:7: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy\n",
      "  import sys\n"
     ]
    }
   ],
   "source": [
    "# filter only pairs of events and person where the \n",
    "# funnel order was followed correctly but the offer \n",
    "# was not successful\n",
    "non_successful_offers = offers[offers['has_transaction'].isna()]      # if the offer has no transaction, it wasn't successful\n",
    "\n",
    "# add a column of ones signaling those are sucessful offers\n",
    "non_successful_offers['is_successful'] = np.zeros(len(non_successful_offers))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(27573, 15)"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "non_successful_offers.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can merge it back into one dataset and then merge this data with the rest of the profile data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "offers = successful_offers.merge(non_successful_offers,\n",
    "                                how = 'outer')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(245232, 15)"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "offers.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also need to refine user data. Let's start by joining the `profile` data with the transactions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "16993"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(offers['person'].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17000"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(processed_profile['id'].unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "17000"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(with_transaction['person'].unique())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The amount of distinct users in our `offers` data isn't the same as it is in the `profile` data, which means there is a chance we have most of our offers covered by the `profile` data.\n",
    "\n",
    "The `offers` dataset has 245,232 rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [],
   "source": [
    "selected_users_profile = offers['person'].isin(processed_profile['id'])\n",
    "selected_users_offers = offers[selected_users_profile]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [],
   "source": [
    "selected_users = with_transaction['person'].isin(selected_users_offers['person'])\n",
    "selected_transactions = with_transaction[selected_users]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [],
   "source": [
    "selected_transactions = selected_transactions.drop(['event',\n",
    "                                                    'time'], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(306508, 11)"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "selected_transactions.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have to filter this dataset only for the successful or non-successful offers we found above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [],
   "source": [
    "successful_transactions = offers.merge(selected_transactions,\n",
    "                                       how = 'inner',\n",
    "                                       right_on = ['person', 'offer_id'],\n",
    "                                       left_on = ['person', 'offer_id'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['person', 'offer_id', 'received', 'viewed', 'has_transaction',\n",
       "       'completed', 'duration_x', 'time_received', 'time_viewed',\n",
       "       'time_transaction', 'time_completed', 'amount_x', 'reward_x',\n",
       "       'time_expiry', 'is_successful', 'amount_y', 'reward_y', 'difficulty',\n",
       "       'duration_y', 'offer_type', 'email', 'mobile', 'social', 'web'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "successful_transactions.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [],
   "source": [
    "successful_transactions = successful_transactions.drop(['reward_x',\n",
    "                                                        'received', \n",
    "                                                        'viewed', \n",
    "                                                        'has_transaction',\n",
    "                                                        'completed',\n",
    "                                                        'duration_x', \n",
    "                                                        'time_received', \n",
    "                                                        'time_viewed',\n",
    "                                                        'time_transaction',\n",
    "                                                        'time_completed',\n",
    "                                                        'amount_x', \n",
    "                                                        'time_expiry',\n",
    "                                                       ],\n",
    "                                                       axis = 1)\n",
    "\n",
    "successful_transactions.rename(columns = {'reward_y': 'reward',\n",
    "                                          'duration_y': 'duration',\n",
    "                                          'amount_y': 'amount'},\n",
    "                         inplace = True)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2472021, 12)"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "successful_transactions.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have multiple lines for the same successful transaction. Let's leave only one line of informations for each person and offer pair."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>person</th>\n",
       "      <th>offer_id</th>\n",
       "      <th>is_successful</th>\n",
       "      <th>amount</th>\n",
       "      <th>reward</th>\n",
       "      <th>difficulty</th>\n",
       "      <th>duration</th>\n",
       "      <th>offer_type</th>\n",
       "      <th>email</th>\n",
       "      <th>mobile</th>\n",
       "      <th>social</th>\n",
       "      <th>web</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>informational</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>5a8bc65990b245e5a138643cd4eb9837</td>\n",
       "      <td>1.0</td>\n",
       "      <td>22.16</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>informational</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>informational</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>3f207df678b143eea3cee63160fa8bed</td>\n",
       "      <td>1.0</td>\n",
       "      <td>8.57</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>informational</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>0009655768c64bdeb2e877511632db8f</td>\n",
       "      <td>f19421c1d4aa40978ebb69ca19b0e20d</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>bogo</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             person                          offer_id  \\\n",
       "0  0009655768c64bdeb2e877511632db8f  5a8bc65990b245e5a138643cd4eb9837   \n",
       "2  0009655768c64bdeb2e877511632db8f  5a8bc65990b245e5a138643cd4eb9837   \n",
       "3  0009655768c64bdeb2e877511632db8f  3f207df678b143eea3cee63160fa8bed   \n",
       "5  0009655768c64bdeb2e877511632db8f  3f207df678b143eea3cee63160fa8bed   \n",
       "6  0009655768c64bdeb2e877511632db8f  f19421c1d4aa40978ebb69ca19b0e20d   \n",
       "\n",
       "   is_successful  amount  reward  difficulty  duration     offer_type  email  \\\n",
       "0            1.0     NaN     0.0           0         3  informational      1   \n",
       "2            1.0   22.16     NaN           0         3  informational      1   \n",
       "3            1.0     NaN     0.0           0         4  informational      1   \n",
       "5            1.0    8.57     NaN           0         4  informational      1   \n",
       "6            1.0     NaN     5.0           5         5           bogo      1   \n",
       "\n",
       "   mobile  social  web  \n",
       "0       1       1    0  \n",
       "2       1       1    0  \n",
       "3       1       0    1  \n",
       "5       1       0    1  \n",
       "6       1       1    1  "
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "successful_transactions = successful_transactions.drop_duplicates()\n",
    "successful_transactions.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We still have some duplicates due to the multiple lines related to events... Since the `amount` column only happens when a user makes a transaction, this feature is highly important to predict the success because it's part of the definition of success. Therefore, it can't be used in the model and should be dropped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [],
   "source": [
    "transactions_logs = successful_transactions.drop(['amount'], axis = 1)\n",
    "\n",
    "transactions_logs = transactions_logs.dropna().drop_duplicates()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(62398, 11)"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions_logs.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can join the user data with the transaction logs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [],
   "source": [
    "transactions_with_users = transactions_logs.merge(processed_profile,\n",
    "                                                  how = 'left',\n",
    "                                                  left_on = 'person',\n",
    "                                                  right_on = 'id')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['person', 'offer_id', 'is_successful', 'reward', 'difficulty',\n",
       "       'duration', 'offer_type', 'email', 'mobile', 'social', 'web', 'age',\n",
       "       'gender', 'id', 'income', 'became_member_on_year'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transactions_with_users.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cool! Now we're almost ready to start modelling. To model data, we need to remove the `person`/`id` and `offer_id` columns as well as encode the `offer_type` and `gender` (we'll use [Scikit-learn's `LabelEncoder()`](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html) for this). The `amount` coly"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_dataset = transactions_with_users.drop(['person',\n",
    "                                              'offer_id',\n",
    "                                              'id'],\n",
    "                                             axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "le = LabelEncoder()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "offer_types_list = list(final_dataset['offer_type'].unique())\n",
    "types_encoder = le.fit(offer_types_list)\n",
    "offer_type_encoded = le.transform(final_dataset['offer_type'])\n",
    "\n",
    "final_dataset['offer_type_encoded'] = offer_type_encoded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "gender_list = list(final_dataset['gender'].unique())\n",
    "gender_encoder = le.fit(gender_list)\n",
    "gender_encoded = le.transform(final_dataset['gender'])\n",
    "\n",
    "final_dataset['gender_encoded'] = gender_encoded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_dataset = final_dataset.drop(['offer_type',\n",
    "                                    'gender'], axis = 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['is_successful', 'reward', 'difficulty', 'duration', 'email', 'mobile',\n",
       "       'social', 'web', 'age', 'income', 'became_member_on_year',\n",
       "       'offer_type_encoded', 'gender_encoded'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_dataset.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "is_successful            0\n",
       "reward                   0\n",
       "difficulty               0\n",
       "duration                 0\n",
       "email                    0\n",
       "mobile                   0\n",
       "social                   0\n",
       "web                      0\n",
       "age                      0\n",
       "income                   0\n",
       "became_member_on_year    0\n",
       "offer_type_encoded       0\n",
       "gender_encoded           0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_dataset.isna().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(62398, 13)"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_dataset.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>is_successful</th>\n",
       "      <th>reward</th>\n",
       "      <th>difficulty</th>\n",
       "      <th>duration</th>\n",
       "      <th>email</th>\n",
       "      <th>mobile</th>\n",
       "      <th>social</th>\n",
       "      <th>web</th>\n",
       "      <th>age</th>\n",
       "      <th>income</th>\n",
       "      <th>became_member_on_year</th>\n",
       "      <th>offer_type_encoded</th>\n",
       "      <th>gender_encoded</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>33</td>\n",
       "      <td>72000.000000</td>\n",
       "      <td>2017</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>72000.000000</td>\n",
       "      <td>2017</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>72000.000000</td>\n",
       "      <td>2017</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>10</td>\n",
       "      <td>10</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>33</td>\n",
       "      <td>72000.000000</td>\n",
       "      <td>2017</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>118</td>\n",
       "      <td>65404.991568</td>\n",
       "      <td>2018</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   is_successful  reward  difficulty  duration  email  mobile  social  web  \\\n",
       "0            1.0     0.0           0         3      1       1       1    0   \n",
       "1            1.0     0.0           0         4      1       1       0    1   \n",
       "2            1.0     5.0           5         5      1       1       1    1   \n",
       "3            1.0     2.0          10        10      1       1       1    1   \n",
       "4            1.0     5.0           5         5      1       1       1    1   \n",
       "\n",
       "   age        income  became_member_on_year  offer_type_encoded  \\\n",
       "0   33  72000.000000                   2017                   2   \n",
       "1   33  72000.000000                   2017                   2   \n",
       "2   33  72000.000000                   2017                   0   \n",
       "3   33  72000.000000                   2017                   1   \n",
       "4  118  65404.991568                   2018                   0   \n",
       "\n",
       "   gender_encoded  \n",
       "0               1  \n",
       "1               1  \n",
       "2               1  \n",
       "3               1  \n",
       "4               3  "
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_dataset.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Modeling\n",
    "### Separating data into train and test\n",
    "We'll use [Scikit-learn's `train_test_split` function](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.train_test_split.html) to separate data, with 80% as train and 20% as test.\n",
    "\n",
    "Our independent variables - or features - will be:\n",
    "* reward\n",
    "* difficulty\n",
    "* duration\n",
    "* email\n",
    "* mobile\n",
    "* social\n",
    "* web\n",
    "* amount\n",
    "* age\n",
    "* income\n",
    "* became_member_on_year\n",
    "* offer_type_encoded\n",
    "* gender_encoded\n",
    "\n",
    "We'll train supervised models for this. Our dependent variable will be `is_successful`. \n",
    "\n",
    "The supervised models that will be trained are:\n",
    "* Model 1: Logistic Regression\n",
    "* Model 2: Naïve Bayes\n",
    "* Model 3: Support Vector Machines (SVM)\n",
    "* Model 4: Decision Tree\n",
    "\n",
    "We'll use Scikit-learn's framework for all models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = final_dataset[['reward', \n",
    "                          'difficulty', \n",
    "                          'duration',\n",
    "                          'email', \n",
    "                          'mobile',\n",
    "                          'social',\n",
    "                          'web', \n",
    "                          'age',\n",
    "                          'income', \n",
    "                          'became_member_on_year',\n",
    "                          'offer_type_encoded',\n",
    "                          'gender_encoded']]\n",
    "\n",
    "y = final_dataset['is_successful']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(features, \n",
    "                                                    y, \n",
    "                                                    test_size = 0.8, \n",
    "                                                    random_state = 42)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To train all models, we'll use [Scikit-learn's `cross_val_score`](https://scikit-learn.org/stable/modules/cross_validation.html) with 5 folds."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Baseline model\n",
    "As baseline model we'll use a naive one, given by [Scikit-learn's `DummyClassifier`](https://scikit-learn.org/stable/modules/generated/sklearn.dummy.DummyClassifier.html). The model shall predict by most frequent class. This is a great baseline model because it's no better than just saying that an offer will be a success just because that's the marjority's class (and, since this is an unbalanced problem, this means that predictions by the marjority will be better than flipping a fair coin to predict the class)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [],
   "source": [
    "dummy_clf = DummyClassifier(strategy = \"most_frequent\")\n",
    "\n",
    "cross_val_baseline = cross_val_score(dummy_clf, \n",
    "                                     X_train, \n",
    "                                     y_train, \n",
    "                                     cv = 5, \n",
    "                                     scoring = 'f1_micro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6441221316975467"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_val_baseline.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The baseline model has 64.41% of F1-Score. We're looking for models that can perform better than this one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 1: Logistic Regression\n",
    "The [logistic regression](https://en.wikipedia.org/wiki/Logistic_regression) is a fairly simple model that presents good results in robust datasets, such as this one. We expect it to be better than the baseline and, since this is one of the most explainable models there is, we might favor it depending on the results. The logistic regression (along with the Decision Tree) is one of the models with the easiest explanation we chose to train, which makes it a favourite to be te best model (but first let's see how the models perform).\n",
    "\n",
    "Even though the logistic regression has analytical solution, Scikit-learn's implementation uses computational methods to solve it. Since this is a small dataset, we'll use the `liblinear` solver as suggested by [the documentation](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = LogisticRegression(solver = 'liblinear', random_state = 42)\n",
    "\n",
    "cross_val_lr = cross_val_score(lr, \n",
    "                               X_train, \n",
    "                               y_train, \n",
    "                               cv = 5, \n",
    "                               scoring = 'f1_micro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6662358493158751"
      ]
     },
     "execution_count": 110,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_val_lr.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see some marginal gain in the F1-Score: this model reached 66.62%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 2: Naïve Bayes\n",
    "We'll train a Gaussian Näive Bayes model. This model usually performs well in well defined classification models. The chosen classifier, Gaussian, assumes that the likelihood of the features comes from a Gaussian distribution (see [documentation](https://scikit-learn.org/stable/modules/naive_bayes.html) for more details).\n",
    "\n",
    "Naïve Bayes classifiers use [Bayesian statistics](http://www.scholarpedia.org/article/Bayesian_statistics) to find the parameter values. An advantage of Naïve Bayes is that it only requires a small number of training data to estimate the parameters necessary for classification, even though this is a naïve model, which makes it great for this problem. Also, it can perform very well for this type of problem, even when the data breaks the Gaussian assumption."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "nb = GaussianNB()\n",
    "\n",
    "cross_val_nb = cross_val_score(nb, \n",
    "                               X_train, \n",
    "                               y_train, \n",
    "                               cv = 5, \n",
    "                               scoring = 'f1_micro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.702621055986057"
      ]
     },
     "execution_count": 112,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_val_nb.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This model performed a bit better than the Logistic Regression and baseline models when it comes to F1-Score: 70.26%."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 3: Support Vector Machines (SVM)\n",
    "We'll train support vector machines. SVMs are considered more complex than the models above. We expect it to perform better than the baseline, buti it might start to overfit the model.\n",
    "\n",
    "An SVM model builds a representation of the categories as points in space, mapped in a way that the categories are each divided by a clear gap that is as wide as possible (see more [here](https://en.wikipedia.org/wiki/Support-vector_machine). Also, SVMs can efficiently perform a non-linear classification using what is called the kernel trick, implicitly mapping their inputs into high-dimensional feature spaces.\n",
    "\n",
    "For this problem, we'll use an [SVC kernel](https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html), which is a linear one."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/julia.tessler/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:193: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "/home/julia.tessler/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:193: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "/home/julia.tessler/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:193: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "/home/julia.tessler/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:193: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n",
      "/home/julia.tessler/anaconda3/lib/python3.7/site-packages/sklearn/svm/base.py:193: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n",
      "  \"avoid this warning.\", FutureWarning)\n"
     ]
    }
   ],
   "source": [
    "sv = svm.SVC()\n",
    "\n",
    "cross_val_sv = cross_val_score(sv, \n",
    "                               X_train, \n",
    "                               y_train, \n",
    "                               cv = 5, \n",
    "                               scoring = 'f1_micro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6510127408915569"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_val_sv.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This model has 65.10% of F1-Score. It is better than the baseline model, but not better than the Naïve Bayes model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 4: Decision Tree\n",
    "We'll train a decision tree classifier. This is the most powerful of all the chosen models and is very likely to overfit.\n",
    "\n",
    "Decision trees are highly explainable because they produce a [flow-chart like structure](https://en.wikipedia.org/wiki/Decision_tree) as outcome, which means that we have a linear flow of decision making in order to classify the offers. It's a favourite to best model due to it's explainability (as mentioned above)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [],
   "source": [
    "dt = DecisionTreeClassifier(random_state = 42)\n",
    "\n",
    "cross_val_dt = cross_val_score(dt, \n",
    "                               X_train, \n",
    "                               y_train, \n",
    "                               cv = 5, \n",
    "                               scoring = 'f1_micro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6508533831240313"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cross_val_dt.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This better than the baseline model, since the F1-Score is 65.08%. But it's not the best. The Linear Regression model has better scores and it's explainable as well."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Choosing the best model\n",
    "Based on the resuls we found above, the best model is the Naïve Bayes. Let's retrain it so we can keep it's coefficients and apply it to the test features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = nb.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>col_0</th>\n",
       "      <th>0.0</th>\n",
       "      <th>1.0</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_successful</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0.0</th>\n",
       "      <td>6095</td>\n",
       "      <td>12012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.0</th>\n",
       "      <td>3097</td>\n",
       "      <td>28715</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "col_0           0.0    1.0\n",
       "is_successful             \n",
       "0.0            6095  12012\n",
       "1.0            3097  28715"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the confusion matrix\n",
    "pd.crosstab(y_test, predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6973296740719966"
      ]
     },
     "execution_count": 120,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.f1_score(y_test, predictions, average = 'micro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "              precision    recall  f1-score   support\n",
      "\n",
      "         0.0       0.66      0.34      0.45     18107\n",
      "         1.0       0.71      0.90      0.79     31812\n",
      "\n",
      "    accuracy                           0.70     49919\n",
      "   macro avg       0.68      0.62      0.62     49919\n",
      "weighted avg       0.69      0.70      0.67     49919\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(metrics.classification_report(y_test, predictions))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model performed not very well in the test dataset, reaching a 69.73% F1-Score. The decrease in the score was already expected, since this data is new to the model. But since this score is close to what we found during the train stage, this means that this model can be applied to other datapoints and still be trustworthy, as it'll produce similar results.\n",
    "\n",
    "The other usual metrics performed fairly well, with exception of the recall for non-successful offers, which was 34% - meaning that it's prediction is worse than flipping a coin to predict the success of the offer.\n",
    "\n",
    "Unfortunatelly, Naïve Bayes model do not have easy explanations for the coefficients, which means that we can't easily explain what variables are related to the success of the offer. What we can do is look into each variable and the associated probability to look into the relations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>non successful</th>\n",
       "      <th>successful</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>reward</td>\n",
       "      <td>0.204731</td>\n",
       "      <td>0.795269</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>difficulty</td>\n",
       "      <td>0.925091</td>\n",
       "      <td>0.074909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>duration</td>\n",
       "      <td>0.346706</td>\n",
       "      <td>0.653294</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>email</td>\n",
       "      <td>0.294267</td>\n",
       "      <td>0.705733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>mobile</td>\n",
       "      <td>0.408526</td>\n",
       "      <td>0.591474</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>social</td>\n",
       "      <td>0.093768</td>\n",
       "      <td>0.906232</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>web</td>\n",
       "      <td>0.924745</td>\n",
       "      <td>0.075255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>age</td>\n",
       "      <td>0.337013</td>\n",
       "      <td>0.662987</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>income</td>\n",
       "      <td>0.398634</td>\n",
       "      <td>0.601366</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>became_member_on_year</td>\n",
       "      <td>0.288445</td>\n",
       "      <td>0.711555</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>offer_type_encoded</td>\n",
       "      <td>0.118418</td>\n",
       "      <td>0.881582</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>gender_encoded</td>\n",
       "      <td>0.086060</td>\n",
       "      <td>0.913940</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                        0  non successful  successful\n",
       "0                  reward        0.204731    0.795269\n",
       "1              difficulty        0.925091    0.074909\n",
       "2                duration        0.346706    0.653294\n",
       "3                   email        0.294267    0.705733\n",
       "4                  mobile        0.408526    0.591474\n",
       "5                  social        0.093768    0.906232\n",
       "6                     web        0.924745    0.075255\n",
       "7                     age        0.337013    0.662987\n",
       "8                  income        0.398634    0.601366\n",
       "9   became_member_on_year        0.288445    0.711555\n",
       "10     offer_type_encoded        0.118418    0.881582\n",
       "11         gender_encoded        0.086060    0.913940"
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result_df = pd.DataFrame(X_train.columns)\n",
    "result_df[['non successful', 'successful']] = pd.DataFrame(model.predict_proba(X_train).tolist())\n",
    "\n",
    "result_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also we can get the features that had the most predictive contribution to each class, but still no explainability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['social', 'mobile', 'web', 'gender_encoded', 'offer_type_encoded',\n",
      "       'email', 'reward', 'duration', 'difficulty', 'age'],\n",
      "      dtype='object')\n",
      "\n",
      "Index(['email', 'web', 'mobile', 'social', 'offer_type_encoded',\n",
      "       'gender_encoded', 'became_member_on_year', 'duration', 'reward',\n",
      "       'difficulty'],\n",
      "      dtype='object')\n"
     ]
    }
   ],
   "source": [
    "# source: https://stackoverflow.com/questions/50526898/how-to-get-feature-importance-in-naive-bayes\n",
    "# prints the top 10 most predictive features for the non-successful class\n",
    "neg = model.theta_[0].argsort()\n",
    "print(np.take(X_train.columns, neg[:10]))\n",
    "\n",
    "print('')\n",
    "\n",
    "# prints the top 10 most predictive features for the successful class\n",
    "neg = model.sigma_[0].argsort()\n",
    "print(np.take(X_train.columns, neg[:10]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we consider the Logist Regression as the best explainable model, we can look into the coefficients to get the most important features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_lr = lr.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions_lr = model_lr.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.6438029607964904"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "metrics.f1_score(y_test, predictions_lr, average = 'micro')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>col_0</th>\n",
       "      <th>0.0</th>\n",
       "      <th>1.0</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>is_successful</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0.0</th>\n",
       "      <td>523</td>\n",
       "      <td>17584</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.0</th>\n",
       "      <td>197</td>\n",
       "      <td>31615</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "col_0          0.0    1.0\n",
       "is_successful            \n",
       "0.0            523  17584\n",
       "1.0            197  31615"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get the confusion matrix\n",
    "pd.crosstab(y_test, predictions_lr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    coefficient                feature\n",
      "0      1.051222                 reward\n",
      "5      1.023997                 social\n",
      "4      1.008203                 mobile\n",
      "9      1.000550  became_member_on_year\n",
      "3      1.000011                  email\n",
      "8      0.999996                 income\n",
      "11     0.999768         gender_encoded\n",
      "7      0.998785                    age\n",
      "6      0.995864                    web\n",
      "2      0.992067               duration\n",
      "10     0.987936     offer_type_encoded\n",
      "1      0.960984             difficulty\n"
     ]
    }
   ],
   "source": [
    "# Logistic Regression needs a small transformation to get the right coefficients\n",
    "transformed_coefficients = list(np.exp(model_lr.coef_))\n",
    "cdf = pd.DataFrame(list(X_train.columns), transformed_coefficients).reset_index()\n",
    "\n",
    "cdf.columns = ['coefficient', 'feature']\n",
    "print(cdf.sort_values(by = 'coefficient', ascending = False))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The supervised models trained, with respective F1-Scores, were:\n",
    "* Baseline Model: Dummy Classifier for most frequent class\n",
    "    * F1-Score: 64.41%\n",
    "* Model 1: Logistic Regression\n",
    "    * F1-Score: 66.62%\n",
    "* Model 2: Naïve Bayes\n",
    "    * F1-Score: 70.26%\n",
    "* Model 3: Support Vector Machines (SVM)\n",
    "    * 65.10%\n",
    "* Model 4: Decision Tree\n",
    "    * F1-Score: 65.08%\n",
    "\n",
    "The best model was Naïve Bayes, but it has not explainability. The second best and explainable model was the Logistic Regression.\n",
    "\n",
    "In order to have explainability, we loose predictive power: the F1-Score of the Logistic Regression model is 64.38%. But the value of the reward, followed by the offer sent by social networks. For the increase of one unit in each feature, we expect a increase of the respective coefficient in the success of the offer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion and refinement\n",
    "When we started, we wanted to make better purchasing offers to Starbucks’ customers. For\n",
    "this, we used customer’s past behaviour to find patterns and try to be more assertive. As given\n",
    "by the Udacity’s Starbucks Project Overview, the basic task was to use the data to identify which\n",
    "groups of people are most responsive to each type of offer, and how best to present each type of\n",
    "offer. In other words, this is a classification problem where the model takes user behaviour data as\n",
    "input and produces a group as output (either previously defined or not).\n",
    "\n",
    "For this project, we spent quite some time dealing with the features, manipulating tose to fit into the models. For that to happen, we found a way to define an offer success based on the user funnel performed from the transcript dataset.\n",
    "\n",
    "Once we had the dataset, we trained 4 supervised learning models and a baseline one. The baseline was a incredibly naïve model that classified the items based on the most frequent class. The model with best performance was a Naïve Bayes. This model doesn't have easy explainability, which means we fail to find understainable patterns to provide offers. But this model achieved reasonable results when applied to the test dataset, meaning it can be applied to other datapoints and still produce the same level of results.\n",
    "\n",
    "In order to get explainability, we chose the Logistic Regression model, that has lower predictive power. But with this model, we identified the top three most important features: reward, social and mobile.\n",
    "\n",
    "Next steps would include better feature engineering and selection (we just used all features we could) and other classification models. The model selection should probably account for model explainability, which failed in this case. To make better decisions here, we could dive deeper into the post-mortem analysis of those models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}