028 - How to Automate Building Your Foreclosure List with Selenium in Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to Automate Building Your Foreclosure List with Selenium in Python | Easy Tutorial\n",
    "\n",
    "## Overview\n",
    "| Detail Tag            | Information                                                                                        |\n",
    "|-----------------------|----------------------------------------------------------------------------------------------------|\n",
    "| Originally Created By | Ariel Herrera arielherrera@analyticsariel.com                                                      |\n",
    "| External References   | <a href=\"https://www.hillsborough.realforeclose.com/index.cfm\" target=\"_blank\">County Foreclosure Site</a>|\n",
    "| Input Datasets        | Login Credentials                                                                                    |\n",
    "| Output Datasets       | Table of Foreclosure Properties                                              |\n",
    "| Input Data Source     | Dataframe                                                                                                |\n",
    "| Output Data Source    | Dataframe                                                                                                   |\n",
    "\n",
    "## History\n",
    "| Date         | Developed By  | Reason                                                |\n",
    "|--------------|---------------|-------------------------------------------------------|\n",
    "| 6th Feb 2021 | Ariel Herrera | Notebook created to automate creating foreclosure list. |\n",
    "\n",
    "## Other Details\n",
    "This Notebook is a prototype."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=\"blue\">Install Packages</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: beautifulsoup4 in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (4.8.0)\n",
      "Requirement already satisfied: soupsieve>=1.2 in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from beautifulsoup4) (1.9.5)\n",
      "Requirement already satisfied: selenium in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (3.141.0)\n",
      "Requirement already satisfied: urllib3 in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from selenium) (1.25.8)\n",
      "Requirement already satisfied: webdriver-manager in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (3.3.0)\n",
      "Requirement already satisfied: requests in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from webdriver-manager) (2.22.0)\n",
      "Requirement already satisfied: crayons in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from webdriver-manager) (0.4.0)\n",
      "Requirement already satisfied: configparser in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from webdriver-manager) (5.0.1)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from requests->webdriver-manager) (2019.11.28)\n",
      "Requirement already satisfied: idna<2.9,>=2.5 in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from requests->webdriver-manager) (2.8)\n",
      "Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from requests->webdriver-manager) (3.0.4)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from requests->webdriver-manager) (1.25.8)\n",
      "Requirement already satisfied: colorama in /Users/arielherrera/opt/anaconda3/lib/python3.7/site-packages (from crayons->webdriver-manager) (0.4.3)\n"
     ]
    }
   ],
   "source": [
    "# Install a pip package in the current Jupyter kernel\n",
    "import sys\n",
    "!{sys.executable} -m pip install beautifulsoup4\n",
    "!{sys.executable} -m pip install selenium\n",
    "!{sys.executable} -m pip install webdriver-manager"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# install chromedriver\n",
    "# https://sites.google.com/a/chromium.org/chromedriver/home"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=\"blue\">Imports</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from bs4 import BeautifulSoup\n",
    "from selenium import webdriver\n",
    "from webdriver_manager.chrome import ChromeDriverManager\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=\"blue\">Local and Constant Variables</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set chrome options\n",
    "chrome_options = webdriver.ChromeOptions()\n",
    "# chrome_options.add_argument('--headless') # run in background\n",
    "chrome_options.add_argument('--no-sandbox')\n",
    "chrome_options.add_argument('--disable-dev-shm-usage')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "aherrera614\n",
      "2021-02-26\n"
     ]
    }
   ],
   "source": [
    "# set parameters\n",
    "url = \"https://www.hillsborough.realforeclose.com/index.cfm\"\n",
    "\n",
    "df_login = pd.read_csv('login_credentials.csv')\n",
    "username = df_login.loc[df_login['key'] == \"username\"]['value'].iloc[0]\n",
    "password = df_login.loc[df_login['key'] == \"password\"]['value'].iloc[0]\n",
    "\n",
    "# date\n",
    "date_str = \"2021-02-26\"\n",
    "\n",
    "print(username)\n",
    "print(date_str)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <font color=\"blue\">Navigate Website</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WDM] - ====== WebDriver manager ======\n",
      "[WDM] - Current google-chrome version is 88.0.4324\n",
      "[WDM] - Get LATEST driver version for 88.0.4324\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WDM] - Driver [/Users/arielherrera/.wdm/drivers/chromedriver/mac64/88.0.4324.96/chromedriver] found in cache\n"
     ]
    }
   ],
   "source": [
    "# set up driver\n",
    "driver = webdriver.Chrome(ChromeDriverManager().install(), options=chrome_options)\n",
    "# get url\n",
    "driver.get(url)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# enter in credentials\n",
    "# username\n",
    "login_user_name_elem = driver.find_element_by_id(\"LogName\")\n",
    "login_user_name_elem.send_keys(username)\n",
    "\n",
    "# password\n",
    "login_password_elem = driver.find_element_by_id(\"LogPass\")\n",
    "login_password_elem.send_keys(password)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# login\n",
    "login_elem = driver.find_element_by_id(\"LogButton\")\n",
    "login_elem.click()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# bypass notice\n",
    "enter_notice_elem = driver.find_element_by_xpath(\"//input[@type='button'][@value='OK']\")\n",
    "enter_notice_elem.click()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# calendar\n",
    "calendar_elem = driver.find_element_by_xpath(\"//span[contains(text(), 'Calendar')]\")\n",
    "calendar_elem.click()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-02-26\n",
      "Day: 26\n"
     ]
    }
   ],
   "source": [
    "# day input\n",
    "day_str = str(int(date_str.split(\"-\")[-1])) # get single value if single digit\n",
    "print(date_str)\n",
    "print(\"Day:\", day_str)\n",
    "\n",
    "# calendar date\n",
    "calendar_date_elem = driver.find_element_by_xpath(\"//span[@class='CALNUM'][contains(text(), '{0}')]\"\\\n",
    "                                                   .format(day_str))\n",
    "calendar_date_elem.click()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### <font color=\"green\">Get Auction Content</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# get page contents\n",
    "soup = BeautifulSoup(driver.page_source, 'html.parser')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### <font color=\"purple\">Get Auction Detail</font>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create list\n",
    "auction_details_list = []\n",
    "\n",
    "# find auctions stats\n",
    "auction_details_content = soup.findAll(\"div\", {\"class\": \"AUCTION_DETAILS\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<div class=\"AUCTION_DETAILS\">\n",
      " <table class=\"ad_tab\" tabindex=\"0\">\n",
      "  <tbody>\n",
      "   <tr>\n",
      "    <th class=\"AD_LBL\" scope=\"row\">\n",
      "     Auction Type:\n",
      "    </th>\n",
      "    <td class=\"AD_DTA\">\n",
      "     FORECLOSURE\n",
      "    </td>\n",
      "   </tr>\n",
      "   <tr>\n",
      "    <th aria-label=\"Case Number\" class=\"AD_LBL\" scope=\"row\">\n",
      "     Case #:\n",
      "    </th>\n",
      "    <td class=\"AD_DTA\">\n",
      "     <a href=\"/index.cfm?zaction=auction&amp;zmethod=details&amp;AID=1282294&amp;bypassPage=1\">\n",
      "      292019CA012595A001HC\n",
      "     </a>\n",
      "    </td>\n",
      "   </tr>\n",
      "   <tr>\n",
      "    <th class=\"AD_LBL\" scope=\"row\">\n",
      "     Final Judgment Amount:\n",
      "    </th>\n",
      "    <td class=\"AD_DTA\">\n",
      "     $197,273.19\n",
      "    </td>\n",
      "   </tr>\n",
      "   <tr>\n",
      "    <th class=\"AD_LBL\" scope=\"row\">\n",
      "     Parcel ID:\n",
      "    </th>\n",
      "    <td class=\"AD_DTA\">\n",
      "     2031065WA000015000450U\n",
      "    </td>\n",
      "   </tr>\n",
      "   <tr>\n",
      "    <th class=\"AD_LBL\" scope=\"row\">\n",
      "     Property Address:\n",
      "    </th>\n",
      "    <td class=\"AD_DTA\">\n",
      "     10313 HUNTERS HAVEN BLVD\n",
      "    </td>\n",
      "   </tr>\n",
      "   <tr>\n",
      "    <th class=\"AD_LBL\" scope=\"row\">\n",
      "    </th>\n",
      "    <td class=\"AD_DTA\">\n",
      "     RIVERVIEW, FL- 33578\n",
      "    </td>\n",
      "   </tr>\n",
      "   <tr>\n",
      "    <th class=\"AD_LBL\" scope=\"row\">\n",
      "     Assessed Value:\n",
      "    </th>\n",
      "    <td class=\"AD_DTA\">\n",
      "     $147,997.00\n",
      "    </td>\n",
      "   </tr>\n",
      "   <tr>\n",
      "    <th class=\"AD_LBL\" scope=\"row\">\n",
      "     Plaintiff Max Bid:\n",
      "    </th>\n",
      "    <td class=\"AD_DTA ASTAT_MSGPB\">\n",
      "     Hidden\n",
      "    </td>\n",
      "   </tr>\n",
      "  </tbody>\n",
      " </table>\n",
      "</div>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# view contents of first element\n",
    "print(auction_details_content[0].prettify())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<th class=\"AD_LBL\" scope=\"row\">Auction Type:</th>,\n",
       " <th aria-label=\"Case Number\" class=\"AD_LBL\" scope=\"row\">Case #:</th>,\n",
       " <th class=\"AD_LBL\" scope=\"row\">Final Judgment Amount:</th>,\n",
       " <th class=\"AD_LBL\" scope=\"row\">Parcel ID:</th>,\n",
       " <th class=\"AD_LBL\" scope=\"row\">Property Address:</th>,\n",
       " <th class=\"AD_LBL\" scope=\"row\"></th>,\n",
       " <th class=\"AD_LBL\" scope=\"row\">Assessed Value:</th>,\n",
       " <th class=\"AD_LBL\" scope=\"row\">Plaintiff Max Bid:</th>]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "auction_details_content[0].findAll(\"th\", {\"class\": \"AD_LBL\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<td class=\"AD_DTA\">FORECLOSURE</td>,\n",
       " <td class=\"AD_DTA\"><a href=\"/index.cfm?zaction=auction&amp;zmethod=details&amp;AID=1282294&amp;bypassPage=1\">292019CA012595A001HC</a></td>,\n",
       " <td class=\"AD_DTA\">$197,273.19</td>,\n",
       " <td class=\"AD_DTA\">2031065WA000015000450U</td>,\n",
       " <td class=\"AD_DTA\">10313 HUNTERS HAVEN BLVD</td>,\n",
       " <td class=\"AD_DTA\">RIVERVIEW, FL- 33578</td>,\n",
       " <td class=\"AD_DTA\">$147,997.00</td>,\n",
       " <td class=\"AD_DTA ASTAT_MSGPB\">Hidden</td>]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "auction_details_content[0].findAll(\"td\", {\"class\": \"AD_DTA\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set params\n",
    "prior_label = \"\"\n",
    "a_detail_row = 0\n",
    "d = {}\n",
    "\n",
    "# iterate through details\n",
    "for a_detail in auction_details_content:\n",
    "    d[a_detail_row] = {}\n",
    "    \n",
    "    # get all labels\n",
    "    auction_starts = soup.findAll(\"div\", {\"class\": \"ASTAT_MSGB Astat_DATA\"})\n",
    "    auction_detail_labels = a_detail.findAll(\"th\", {\"class\": \"AD_LBL\"})\n",
    "    auction_detail_values = a_detail.findAll(\"td\", {\"class\": \"AD_DTA\"})\n",
    "    \n",
    "    # iterate through labels\n",
    "    for i in range(len(auction_detail_labels)):\n",
    "        # get label\n",
    "        label = auction_detail_labels[i].text\n",
    "        \n",
    "        if label == \"Auction Type:\":\n",
    "            auction_type = auction_detail_values[i].text\n",
    "        elif label == \"Case #:\":\n",
    "            case_num = auction_detail_values[i].text\n",
    "        elif label == \"Final Judgment Amount:\":\n",
    "            final_judgement_amt = auction_detail_values[i].text\n",
    "        elif label == \"Parcel ID:\":\n",
    "            parcel_id = auction_detail_values[i].text\n",
    "        elif label == \"Property Address:\":\n",
    "            property_address_street = auction_detail_values[i].text\n",
    "        elif (label == \"\") & (prior_label == \"Property Address:\"):\n",
    "            property_address_city = auction_detail_values[i].text\n",
    "        elif label == \"Assessed Value:\":\n",
    "            assessed_val = auction_detail_values[i].text\n",
    "        elif label == \"Plaintiff Max Bid:\":\n",
    "            plaintiff_max_bid = auction_detail_values[i].text\n",
    "        \n",
    "        # set prior label before moving on to next\n",
    "        prior_label = label\n",
    "        \n",
    "    d[a_detail_row] = {\n",
    "        'AUCTION_STARTS': auction_starts[a_detail_row].text,\n",
    "        'AUCTION_TYPE': auction_type, \n",
    "        'CASE_NUMBER': case_num,\n",
    "        'FINAL_JUDGEMENT_AMOUNT': final_judgement_amt, \n",
    "        'PARCEL_ID': parcel_id,\n",
    "        'PROPERTY_STREET': property_address_street, \n",
    "        'PROPERTY_CITY': property_address_city,\n",
    "        'ASSESSED_VALUE': assessed_val, \n",
    "        'PLAINTIFF_MAX_BID': plaintiff_max_bid}\n",
    "    \n",
    "    # reset values\n",
    "    auction_type = \"\"\n",
    "    case_num = \"\"\n",
    "    final_judgement_amt = \"\"\n",
    "    parcel_id = \"\"\n",
    "    property_address_street = \"\"\n",
    "    property_address_city = \"\"\n",
    "    assessed_val = \"\"\n",
    "    plaintiff_max_bid = \"\"\n",
    "        \n",
    "    a_detail_row += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AUCTION_STARTS</th>\n",
       "      <th>AUCTION_TYPE</th>\n",
       "      <th>CASE_NUMBER</th>\n",
       "      <th>FINAL_JUDGEMENT_AMOUNT</th>\n",
       "      <th>PARCEL_ID</th>\n",
       "      <th>PROPERTY_STREET</th>\n",
       "      <th>PROPERTY_CITY</th>\n",
       "      <th>ASSESSED_VALUE</th>\n",
       "      <th>PLAINTIFF_MAX_BID</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>02/26/2021 10:00 AM ET</td>\n",
       "      <td>FORECLOSURE</td>\n",
       "      <td>292019CA012595A001HC</td>\n",
       "      <td>$197,273.19</td>\n",
       "      <td>2031065WA000015000450U</td>\n",
       "      <td>10313 HUNTERS HAVEN BLVD</td>\n",
       "      <td>RIVERVIEW, FL- 33578</td>\n",
       "      <td>$147,997.00</td>\n",
       "      <td>Hidden</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>02/26/2021 10:00 AM ET</td>\n",
       "      <td>FORECLOSURE</td>\n",
       "      <td>292019CC065643A001HC</td>\n",
       "      <td>$4,578.43</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>$4,578.43</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>02/26/2021 10:00 AM ET</td>\n",
       "      <td>FORECLOSURE</td>\n",
       "      <td>292020CA007527A001HC</td>\n",
       "      <td>$154,930.44</td>\n",
       "      <td>20302080P000010000020U</td>\n",
       "      <td>9225 STONE RIVER PL</td>\n",
       "      <td>RIVERVIEW, FL- 33578</td>\n",
       "      <td>$152,322.00</td>\n",
       "      <td>Hidden</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>02/26/2021 10:00 AM ET</td>\n",
       "      <td>FORECLOSURE</td>\n",
       "      <td>292020CC037960A001HC</td>\n",
       "      <td>$9,132.88</td>\n",
       "      <td>1932099NP000004000370U</td>\n",
       "      <td>2022 PEACEFUL PALM ST</td>\n",
       "      <td>RUSKIN, FL- 33570</td>\n",
       "      <td>$136,764.00</td>\n",
       "      <td>Hidden</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>02/26/2021 10:00 AM ET</td>\n",
       "      <td>FORECLOSURE</td>\n",
       "      <td>292020CC040346A001HC</td>\n",
       "      <td>$10,449.27</td>\n",
       "      <td>1830053X9000000000060A</td>\n",
       "      <td>4405 W FAIR OAKS AVE 6</td>\n",
       "      <td>TAMPA, FL- 33611</td>\n",
       "      <td>$71,687.00</td>\n",
       "      <td>$10,805.57</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>02/26/2021 10:00 AM ET</td>\n",
       "      <td>FORECLOSURE</td>\n",
       "      <td>292020CC043311A001HC</td>\n",
       "      <td>$5,938.50</td>\n",
       "      <td>2030189CO000060000080U</td>\n",
       "      <td>6919 TOWERING SPRUCE DR</td>\n",
       "      <td>RIVERVIEW, FL- 33578</td>\n",
       "      <td>$113,756.00</td>\n",
       "      <td>Hidden</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Canceled per County</td>\n",
       "      <td>FORECLOSURE</td>\n",
       "      <td>292019CA012595A001HC</td>\n",
       "      <td>$0.00</td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td></td>\n",
       "      <td>Hidden</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           AUCTION_STARTS AUCTION_TYPE           CASE_NUMBER  \\\n",
       "0  02/26/2021 10:00 AM ET  FORECLOSURE  292019CA012595A001HC   \n",
       "1  02/26/2021 10:00 AM ET  FORECLOSURE  292019CC065643A001HC   \n",
       "2  02/26/2021 10:00 AM ET  FORECLOSURE  292020CA007527A001HC   \n",
       "3  02/26/2021 10:00 AM ET  FORECLOSURE  292020CC037960A001HC   \n",
       "4  02/26/2021 10:00 AM ET  FORECLOSURE  292020CC040346A001HC   \n",
       "5  02/26/2021 10:00 AM ET  FORECLOSURE  292020CC043311A001HC   \n",
       "6     Canceled per County  FORECLOSURE  292019CA012595A001HC   \n",
       "\n",
       "  FINAL_JUDGEMENT_AMOUNT               PARCEL_ID           PROPERTY_STREET  \\\n",
       "0            $197,273.19  2031065WA000015000450U  10313 HUNTERS HAVEN BLVD   \n",
       "1              $4,578.43                                                     \n",
       "2            $154,930.44  20302080P000010000020U       9225 STONE RIVER PL   \n",
       "3              $9,132.88  1932099NP000004000370U     2022 PEACEFUL PALM ST   \n",
       "4             $10,449.27  1830053X9000000000060A    4405 W FAIR OAKS AVE 6   \n",
       "5              $5,938.50  2030189CO000060000080U   6919 TOWERING SPRUCE DR   \n",
       "6                  $0.00                                                     \n",
       "\n",
       "          PROPERTY_CITY ASSESSED_VALUE PLAINTIFF_MAX_BID  \n",
       "0  RIVERVIEW, FL- 33578    $147,997.00            Hidden  \n",
       "1                                              $4,578.43  \n",
       "2  RIVERVIEW, FL- 33578    $152,322.00            Hidden  \n",
       "3     RUSKIN, FL- 33570    $136,764.00            Hidden  \n",
       "4      TAMPA, FL- 33611     $71,687.00        $10,805.57  \n",
       "5  RIVERVIEW, FL- 33578    $113,756.00            Hidden  \n",
       "6                                                 Hidden  "
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame(data=d).T # convert dictionary to dataframe\n",
    "df.to_csv('{0}_auction_properties.csv'.format(date_str), index=False) # save\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# End Notebook"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}