How to do Web Scraping for Real Estate Data.ipynb

{"cells":[{"cell_type":"markdown","source":["# How to do Web Scraping for Real Estate Data\nExtracting data from the web to use in decision-making"],"metadata":{}},{"cell_type":"markdown","source":["### New Jersey Map by County"],"metadata":{}},{"cell_type":"markdown","source":["<img src=\"https://www.njfuture.org/wp-content/uploads/2012/05/new-jersey-county-map.png\" style=\"height:600px;\">"],"metadata":{}},{"cell_type":"markdown","source":["### What comes to your mind when you think of New Jersey?"],"metadata":{}},{"cell_type":"markdown","source":["md\n<img src=\"https://townsquare.media/site/394/files/2016/03/six-flags.jpg?w=980&q=75\" style=\"width:500px\">\n<img src=\"https://tvseriesfinale.com/wp-content/uploads/2020/01/jersey-shore-family-vacation-e1580134194375.jpg\" style=\"width:500px\">\n<img src=\"https://advancelocal-adapter-image-uploads.s3.amazonaws.com/image.lehighvalleylive.com/home/lvlive-media/width2048/img/breaking-news_impact/photo/garden-state-parkway-49b4ef47daea1aad.jpg\" style=\"width:500px\">"],"metadata":{}},{"cell_type":"markdown","source":["### Did you know New Jersey has incredible real estate too?"],"metadata":{}},{"cell_type":"markdown","source":["<img src=\"https://images.beachhouse.com/files/hal_62581838_0.jpg\" style=\"height:600px;\">"],"metadata":{}},{"cell_type":"markdown","source":["## Overview\nWe are <b>long distance</b> real estate investors looking to invest in the state of New Jersey.\n\n### Problem\nWe have limited resources since we do not live in New Jersey. We cannot rely on word-of-mouth or local conversation to find great deals.\n\n### Goal\nWe want to use <b>BIG DATA</b> technologies to get local town data. Data in an easy format (csv file). That way we can make better investment decisions.\n\n### Solution\nUse <b>WEB SCRAPING</b> to build a dataset of New Jersey towns. Use this data to identify trends and areas for opportunity."],"metadata":{}},{"cell_type":"markdown","source":["## What is web scraping?\nA tool to automatically collect new or updated data.\n\n<img src=\"https://drive.google.com/uc?id=1xTMvXBH8cnegL3VbBuA1pf9XcWiByxep\" style=\"width:1000px;\">\n\n### Benefits\n1. Collect Data for market research\n2. Extract Contact Info\n3. Track prices for mulitple markets"],"metadata":{}},{"cell_type":"markdown","source":["## Widgets"],"metadata":{}},{"cell_type":"code","source":["dbutils.widgets.removeAll()"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\"></div>"]}}],"execution_count":11},{"cell_type":"code","source":["# create widgets\ndbutils.widgets.text(\"websiteUrl\", \"https://www.njstatelib.org/research_library/new_jersey_resources/highlights/municipalities_by_county/\", \"01) Website URL\")\ndbutils.widgets.text(\"newsUrl\", \"https://patch.com/new-jersey/eastbrunswick/real-estate\", \"02) News URL\")"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\"></div>"]}}],"execution_count":12},{"cell_type":"code","source":["websiteUrl = dbutils.widgets.get(\"websiteUrl\")\nnewsUrl = dbutils.widgets.get(\"newsUrl\")"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\"></div>"]}}],"execution_count":13},{"cell_type":"markdown","source":["## Imports"],"metadata":{}},{"cell_type":"code","source":["from bs4 import BeautifulSoup\nimport requests\nimport pandas as pd"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\"></div>"]}}],"execution_count":15},{"cell_type":"markdown","source":["## Functions"],"metadata":{}},{"cell_type":"code","source":["def get_url_contents(url):\n  # get page contents\n  page = requests.get(url)\n  # prase the html page\n  soup = BeautifulSoup(page.text, 'html5lib')\n  print('Extracted website contents!')\n  return soup"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\"></div>"]}}],"execution_count":17},{"cell_type":"markdown","source":["## Data Exploration"],"metadata":{}},{"cell_type":"markdown","source":["### 1) City & County Data"],"metadata":{}},{"cell_type":"code","source":["# parse text for nj state lib contents\nsoup = get_url_contents(websiteUrl)"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\">Extracted website contents!\n</div>"]}}],"execution_count":20},{"cell_type":"code","source":["# find the html table\ntable = soup.find('table', attrs={\"class\": \"omsc-custom-table omsc-style-1\"})\ntable_body = table.find('tbody')\nprint(table_body.prettify())"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\">&lt;tbody&gt;\n &lt;tr&gt;\n  &lt;td&gt;\n   &lt;p&gt;\n   &lt;/p&gt;\n   &lt;h2&gt;\n    Atlantic\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Absecon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Atlantic City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Brigantine\n    &lt;/li&gt;\n    &lt;li&gt;\n     Buena\n    &lt;/li&gt;\n    &lt;li&gt;\n     Buena Vista\n    &lt;/li&gt;\n    &lt;li&gt;\n     Corbin City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Egg Harbor\n    &lt;/li&gt;\n    &lt;li&gt;\n     Egg Harbor City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Estell Manor\n    &lt;/li&gt;\n    &lt;li&gt;\n     Folsom\n    &lt;/li&gt;\n    &lt;li&gt;\n     Galloway\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hamilton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hammonton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Linwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Longport\n    &lt;/li&gt;\n    &lt;li&gt;\n     Margate City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mullica\n    &lt;/li&gt;\n    &lt;li&gt;\n     Northfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pleasantville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Port Republic\n    &lt;/li&gt;\n    &lt;li&gt;\n     Somers Point\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ventnor City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Weymouth\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Bergen\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Allendale\n    &lt;/li&gt;\n    &lt;li&gt;\n     Alpine\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bergenfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bogota\n    &lt;/li&gt;\n    &lt;li&gt;\n     Carlstadt\n    &lt;/li&gt;\n    &lt;li&gt;\n     Cliffside Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Closter\n    &lt;/li&gt;\n    &lt;li&gt;\n     Cresskill\n    &lt;/li&gt;\n    &lt;li&gt;\n     Demarest\n    &lt;/li&gt;\n    &lt;li&gt;\n     Dumont\n    &lt;/li&gt;\n    &lt;li&gt;\n     East Rutherford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Edgewater\n    &lt;/li&gt;\n    &lt;li&gt;\n     Elmwood Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Emerson\n    &lt;/li&gt;\n    &lt;li&gt;\n     Englewood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Englewood Cliffs\n    &lt;/li&gt;\n    &lt;li&gt;\n     Fair Lawn\n    &lt;/li&gt;\n    &lt;li&gt;\n     Fairview\n    &lt;/li&gt;\n    &lt;li&gt;\n     Fort Lee\n    &lt;/li&gt;\n    &lt;li&gt;\n     Franklin Lakes\n    &lt;/li&gt;\n    &lt;li&gt;\n     Garfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Glen Rock\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hackensack\n    &lt;/li&gt;\n    &lt;li&gt;\n     Harrington Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hasbrouck Heights\n    &lt;/li&gt;\n    &lt;li&gt;\n     Haworth\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hillsdale\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ho-Ho-Kus\n    &lt;/li&gt;\n    &lt;li&gt;\n     Leonia\n    &lt;/li&gt;\n    &lt;li&gt;\n     Little Ferry\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lodi\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lyndhurst\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mahwah\n    &lt;/li&gt;\n    &lt;li&gt;\n     Maywood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Midland Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Montvale\n    &lt;/li&gt;\n    &lt;li&gt;\n     Moonachie\n    &lt;/li&gt;\n    &lt;li&gt;\n     New Milford\n    &lt;/li&gt;\n    &lt;li&gt;\n     North Arlington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Northvale\n    &lt;/li&gt;\n    &lt;li&gt;\n     Norwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Oakland\n    &lt;/li&gt;\n    &lt;li&gt;\n     Old Tappan\n    &lt;/li&gt;\n    &lt;li&gt;\n     Oradell\n    &lt;/li&gt;\n    &lt;li&gt;\n     Palisades Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Paramus\n    &lt;/li&gt;\n    &lt;li&gt;\n     Park Ridge\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ramsey\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ridgefield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ridgefield Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ridgewood\n    &lt;/li&gt;\n    &lt;li&gt;\n     River Edge\n    &lt;/li&gt;\n    &lt;li&gt;\n     River Vale\n    &lt;/li&gt;\n    &lt;li&gt;\n     Rochelle Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Rockleigh\n    &lt;/li&gt;\n    &lt;li&gt;\n     Rutherford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Saddle Brook\n    &lt;/li&gt;\n    &lt;li&gt;\n     Saddle River\n    &lt;/li&gt;\n    &lt;li&gt;\n     South Hackensack\n    &lt;/li&gt;\n    &lt;li&gt;\n     Teaneck\n    &lt;/li&gt;\n    &lt;li&gt;\n     Tenafly\n    &lt;/li&gt;\n    &lt;li&gt;\n     Teterboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Upper Saddle River\n    &lt;/li&gt;\n    &lt;li&gt;\n     Waldwick\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wallington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Washington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Westwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Woodcliff Lake\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wood-Ridge\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wyckoff\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Burlington\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Bass River\n    &lt;/li&gt;\n    &lt;li&gt;\n     Beverly\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bordentown city\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bordentown township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Burlington city\n    &lt;/li&gt;\n    &lt;li&gt;\n     Burlington township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Chesterfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Cinnaminson\n    &lt;/li&gt;\n    &lt;li&gt;\n     Delanco\n    &lt;/li&gt;\n    &lt;li&gt;\n     Delran\n    &lt;/li&gt;\n    &lt;li&gt;\n     Eastampton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Edgewater Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Evesham\n    &lt;/li&gt;\n    &lt;li&gt;\n     Fieldsboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Florence\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hainesport\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lumberton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mansfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Maple Shade\n    &lt;/li&gt;\n    &lt;li&gt;\n     Medford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Medford Lakes\n    &lt;/li&gt;\n    &lt;li&gt;\n     Moorestown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mount Holly\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mount Laurel\n    &lt;/li&gt;\n    &lt;li&gt;\n     New Hanover\n    &lt;/li&gt;\n    &lt;li&gt;\n     North Hanover\n    &lt;/li&gt;\n    &lt;li&gt;\n     Palmyra\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pemberton borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pemberton township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Riverside\n    &lt;/li&gt;\n    &lt;li&gt;\n     Riverton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Shamong\n    &lt;/li&gt;\n    &lt;li&gt;\n     Southampton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Springfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Tabernacle\n    &lt;/li&gt;\n    &lt;li&gt;\n     Washington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Westampton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Willingboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Woodland\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wrightstown\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Camden\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Audubon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Audubon Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Barrington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bellmawr\n    &lt;/li&gt;\n    &lt;li&gt;\n     Berlin borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Berlin township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Brooklawn\n    &lt;/li&gt;\n    &lt;li&gt;\n     Camden\n    &lt;/li&gt;\n    &lt;li&gt;\n     Cherry Hill\n    &lt;/li&gt;\n    &lt;li&gt;\n     Chesilhurst\n    &lt;/li&gt;\n    &lt;li&gt;\n     Clementon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Collingswood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Gibbsboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Gloucester township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Gloucester City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Haddon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Haddon Heights\n    &lt;/li&gt;\n    &lt;li&gt;\n     Haddonfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hi-Nella\n    &lt;/li&gt;\n    &lt;li&gt;\n     Laurel Springs\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lawnside\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lindenwold\n    &lt;/li&gt;\n    &lt;li&gt;\n     Magnolia\n    &lt;/li&gt;\n    &lt;li&gt;\n     Merchantville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mount Ephraim\n    &lt;/li&gt;\n    &lt;li&gt;\n     Oaklyn borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pennsauken\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pine Hill\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pine Valley\n    &lt;/li&gt;\n    &lt;li&gt;\n     Runnemede\n    &lt;/li&gt;\n    &lt;li&gt;\n     Somerdale\n    &lt;/li&gt;\n    &lt;li&gt;\n     Stratford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Tavistock\n    &lt;/li&gt;\n    &lt;li&gt;\n     Voorhees\n    &lt;/li&gt;\n    &lt;li&gt;\n     Waterford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Winslow\n    &lt;/li&gt;\n    &lt;li&gt;\n     Woodlynne\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Cape May\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Avalon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Cape May\n    &lt;/li&gt;\n    &lt;li&gt;\n     Cape May Point\n    &lt;/li&gt;\n    &lt;li&gt;\n     Dennis\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lower\n    &lt;/li&gt;\n    &lt;li&gt;\n     Middle\n    &lt;/li&gt;\n    &lt;li&gt;\n     North Wildwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ocean City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Sea Isle City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Stone Harbor\n    &lt;/li&gt;\n    &lt;li&gt;\n     Upper\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Cape May\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Wildwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wildwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wildwood Crest\n    &lt;/li&gt;\n    &lt;li&gt;\n     Woodbine\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Cumberland\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     &lt;span class=&#34;Apple-style-span&#34;&gt;\n      Bridgeton\n     &lt;/span&gt;\n    &lt;/li&gt;\n    &lt;li&gt;\n     Commercial\n    &lt;/li&gt;\n    &lt;li&gt;\n     Deerfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Downe\n    &lt;/li&gt;\n    &lt;li&gt;\n     Fairfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Greenwich\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hopewell\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lawrence\n    &lt;/li&gt;\n    &lt;li&gt;\n     Maurice River\n    &lt;/li&gt;\n    &lt;li&gt;\n     Millville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Shiloh\n    &lt;/li&gt;\n    &lt;li&gt;\n     Stow Creek\n    &lt;/li&gt;\n    &lt;li&gt;\n     Upper Deerfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Vineland\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Essex\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Belleville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bloomfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Caldwell Borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Cedar Grove\n    &lt;/li&gt;\n    &lt;li&gt;\n     City of Orange\n    &lt;/li&gt;\n    &lt;li&gt;\n     East Orange\n    &lt;/li&gt;\n    &lt;li&gt;\n     Essex Fells\n    &lt;/li&gt;\n    &lt;li&gt;\n     Fairfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Glen Ridge\n    &lt;/li&gt;\n    &lt;li&gt;\n     Irvington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Livingston\n    &lt;/li&gt;\n    &lt;li&gt;\n     Maplewood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Millburn\n    &lt;/li&gt;\n    &lt;li&gt;\n     Montclair\n    &lt;/li&gt;\n    &lt;li&gt;\n     Newark\n    &lt;/li&gt;\n    &lt;li&gt;\n     North Caldwell\n    &lt;/li&gt;\n    &lt;li&gt;\n     Nutley\n    &lt;/li&gt;\n    &lt;li&gt;\n     Roseland\n    &lt;/li&gt;\n    &lt;li&gt;\n     Verona\n    &lt;/li&gt;\n    &lt;li&gt;\n     Village of South Orange\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Caldwell\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Orange\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;p&gt;\n   &lt;/p&gt;\n  &lt;/td&gt;\n  &lt;td&gt;\n   &lt;p&gt;\n   &lt;/p&gt;\n   &lt;h2&gt;\n    Gloucester\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Clayton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Deptford\n    &lt;/li&gt;\n    &lt;li&gt;\n     East Greenwich\n    &lt;/li&gt;\n    &lt;li&gt;\n     Elk\n    &lt;/li&gt;\n    &lt;li&gt;\n     Franklin\n    &lt;/li&gt;\n    &lt;li&gt;\n     Glassboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Greenwich\n    &lt;/li&gt;\n    &lt;li&gt;\n     Harrison\n    &lt;/li&gt;\n    &lt;li&gt;\n     Logan\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mantua\n    &lt;/li&gt;\n    &lt;li&gt;\n     Monroe\n    &lt;/li&gt;\n    &lt;li&gt;\n     National Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Newfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Paulsboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pitman\n    &lt;/li&gt;\n    &lt;li&gt;\n     South Harrison\n    &lt;/li&gt;\n    &lt;li&gt;\n     Swedesboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Washington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wenonah\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Deptford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Westville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Woodbury\n    &lt;/li&gt;\n    &lt;li&gt;\n     Woodbury Heights\n    &lt;/li&gt;\n    &lt;li&gt;\n     Woolwich\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Hudson\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Bayonne\n    &lt;/li&gt;\n    &lt;li&gt;\n     East Newark\n    &lt;/li&gt;\n    &lt;li&gt;\n     Guttenberg\n    &lt;/li&gt;\n    &lt;li&gt;\n     Harrison\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hoboken\n    &lt;/li&gt;\n    &lt;li&gt;\n     Jersey City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Kearny\n    &lt;/li&gt;\n    &lt;li&gt;\n     North Bergen\n    &lt;/li&gt;\n    &lt;li&gt;\n     Secaucus\n    &lt;/li&gt;\n    &lt;li&gt;\n     Union City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Weehawken\n    &lt;/li&gt;\n    &lt;li&gt;\n     West New York\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Hunterdon\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Alexandria\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bethlehem\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bloomsbury\n    &lt;/li&gt;\n    &lt;li&gt;\n     Califon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Clinton town\n    &lt;/li&gt;\n    &lt;li&gt;\n     Clinton township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Delaware\n    &lt;/li&gt;\n    &lt;li&gt;\n     East Amwell\n    &lt;/li&gt;\n    &lt;li&gt;\n     Flemington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Franklin\n    &lt;/li&gt;\n    &lt;li&gt;\n     Frenchtown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Glen Gardner\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hampton\n    &lt;/li&gt;\n    &lt;li&gt;\n     High Bridge\n    &lt;/li&gt;\n    &lt;li&gt;\n     Holland\n    &lt;/li&gt;\n    &lt;li&gt;\n     Kingwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lambertville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lebanon borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lebanon township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Milford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Raritan\n    &lt;/li&gt;\n    &lt;li&gt;\n     Readington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Stockton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Tewksbury\n    &lt;/li&gt;\n    &lt;li&gt;\n     Union\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Amwell\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Mercer\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     East Windsor\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ewing\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hamilton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hightstown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hopewell borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hopewell township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lawrence\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pennington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Princeton township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Trenton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Washington\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Windsor\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Middlesex\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Carteret\n    &lt;/li&gt;\n    &lt;li&gt;\n     Cranbury\n    &lt;/li&gt;\n    &lt;li&gt;\n     Dunellen\n    &lt;/li&gt;\n    &lt;li&gt;\n     East Brunswick\n    &lt;/li&gt;\n    &lt;li&gt;\n     Edison\n    &lt;/li&gt;\n    &lt;li&gt;\n     Helmetta\n    &lt;/li&gt;\n    &lt;li&gt;\n     Highland Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Jamesburg\n    &lt;/li&gt;\n    &lt;li&gt;\n     Metuchen\n    &lt;/li&gt;\n    &lt;li&gt;\n     Middlesex\n    &lt;/li&gt;\n    &lt;li&gt;\n     Milltown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Monroe\n    &lt;/li&gt;\n    &lt;li&gt;\n     New Brunswick\n    &lt;/li&gt;\n    &lt;li&gt;\n     North Brunswick\n    &lt;/li&gt;\n    &lt;li&gt;\n     Old Bridge\n    &lt;/li&gt;\n    &lt;li&gt;\n     Perth Amboy\n    &lt;/li&gt;\n    &lt;li&gt;\n     Piscataway\n    &lt;/li&gt;\n    &lt;li&gt;\n     Plainsboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Sayreville\n    &lt;/li&gt;\n    &lt;li&gt;\n     South Amboy\n    &lt;/li&gt;\n    &lt;li&gt;\n     South Brunswick\n    &lt;/li&gt;\n    &lt;li&gt;\n     South Plainfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     South River\n    &lt;/li&gt;\n    &lt;li&gt;\n     Spotswood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Woodbridge\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Monmouth\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Aberdeen\n    &lt;/li&gt;\n    &lt;li&gt;\n     Allenhurst\n    &lt;/li&gt;\n    &lt;li&gt;\n     Allentown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Asbury Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Atlantic Highlands\n    &lt;/li&gt;\n    &lt;li&gt;\n     Avon-by-the-Sea\n    &lt;/li&gt;\n    &lt;li&gt;\n     Belmar\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bradley Beach\n    &lt;/li&gt;\n    &lt;li&gt;\n     Brielle\n    &lt;/li&gt;\n    &lt;li&gt;\n     Colts Neck\n    &lt;/li&gt;\n    &lt;li&gt;\n     Deal\n    &lt;/li&gt;\n    &lt;li&gt;\n     Eatontown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Englishtown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Fair Haven\n    &lt;/li&gt;\n    &lt;li&gt;\n     Farmingdale\n    &lt;/li&gt;\n    &lt;li&gt;\n     Freehold borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Freehold township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hazlet\n    &lt;/li&gt;\n    &lt;li&gt;\n     Highlands\n    &lt;/li&gt;\n    &lt;li&gt;\n     Holmdel\n    &lt;/li&gt;\n    &lt;li&gt;\n     Howell\n    &lt;/li&gt;\n    &lt;li&gt;\n     Interlaken\n    &lt;/li&gt;\n    &lt;li&gt;\n     Keansburg\n    &lt;/li&gt;\n    &lt;li&gt;\n     Keyport\n    &lt;/li&gt;\n    &lt;li&gt;\n     Little Silver\n    &lt;/li&gt;\n    &lt;li&gt;\n     Loch Arbour\n    &lt;/li&gt;\n    &lt;li&gt;\n     Long Branch\n    &lt;/li&gt;\n    &lt;li&gt;\n     Manalapan\n    &lt;/li&gt;\n    &lt;li&gt;\n     Manasquan\n    &lt;/li&gt;\n    &lt;li&gt;\n     Marlboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Matawan\n    &lt;/li&gt;\n    &lt;li&gt;\n     Middletown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Millstone\n    &lt;/li&gt;\n    &lt;li&gt;\n     Monmouth Beach\n    &lt;/li&gt;\n    &lt;li&gt;\n     Neptune\n    &lt;/li&gt;\n    &lt;li&gt;\n     Neptune City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ocean\n    &lt;/li&gt;\n    &lt;li&gt;\n     Oceanport\n    &lt;/li&gt;\n    &lt;li&gt;\n     Red Bank\n    &lt;/li&gt;\n    &lt;li&gt;\n     Roosevelt\n    &lt;/li&gt;\n    &lt;li&gt;\n     Rumson\n    &lt;/li&gt;\n    &lt;li&gt;\n     Sea Bright\n    &lt;/li&gt;\n    &lt;li&gt;\n     Sea Girt\n    &lt;/li&gt;\n    &lt;li&gt;\n     Shrewsbury borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Shrewsbury township\n    &lt;/li&gt;\n    &lt;li&gt;\n     South Belmar\n    &lt;/li&gt;\n    &lt;li&gt;\n     Spring Lake Boro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Spring Lake Heights\n    &lt;/li&gt;\n    &lt;li&gt;\n     Tinton Falls\n    &lt;/li&gt;\n    &lt;li&gt;\n     Union Beach\n    &lt;/li&gt;\n    &lt;li&gt;\n     Upper Freehold\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wall\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Long Branch\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Morris\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Boonton town\n    &lt;/li&gt;\n    &lt;li&gt;\n     Boonton township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Butler\n    &lt;/li&gt;\n    &lt;li&gt;\n     Chatham borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Chatham township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Chester borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Chester township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Denville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Dover\n    &lt;/li&gt;\n    &lt;li&gt;\n     East Hanover\n    &lt;/li&gt;\n    &lt;li&gt;\n     Florham Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hanover\n    &lt;/li&gt;\n    &lt;li&gt;\n     Harding\n    &lt;/li&gt;\n    &lt;li&gt;\n     Jefferson\n    &lt;/li&gt;\n    &lt;li&gt;\n     Kinnelon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lincoln Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Long Hill\n    &lt;/li&gt;\n    &lt;li&gt;\n     Madison\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mendham borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mendham township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mine Hill\n    &lt;/li&gt;\n    &lt;li&gt;\n     Montville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Morris\n    &lt;/li&gt;\n    &lt;li&gt;\n     Morris Plains\n    &lt;/li&gt;\n    &lt;li&gt;\n     Morristown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mount Arlington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mount Olive\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mountain Lakes\n    &lt;/li&gt;\n    &lt;li&gt;\n     Netcong\n    &lt;/li&gt;\n    &lt;li&gt;\n     Parsippany-Troy Hills\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pequannock\n    &lt;/li&gt;\n    &lt;li&gt;\n     Randolph\n    &lt;/li&gt;\n    &lt;li&gt;\n     Riverdale\n    &lt;/li&gt;\n    &lt;li&gt;\n     Rockaway borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Rockaway township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Roxbury\n    &lt;/li&gt;\n    &lt;li&gt;\n     Victory Gardens\n    &lt;/li&gt;\n    &lt;li&gt;\n     Washington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wharton\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;p&gt;\n   &lt;/p&gt;\n  &lt;/td&gt;\n  &lt;td&gt;\n   &lt;p&gt;\n   &lt;/p&gt;\n   &lt;h2&gt;\n    Ocean\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Barnegat\n    &lt;/li&gt;\n    &lt;li&gt;\n     Barnegat Light\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bay Head\n    &lt;/li&gt;\n    &lt;li&gt;\n     Beach Haven\n    &lt;/li&gt;\n    &lt;li&gt;\n     Beachwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Berkeley\n    &lt;/li&gt;\n    &lt;li&gt;\n     Brick\n    &lt;/li&gt;\n    &lt;li&gt;\n     Dover\n    &lt;/li&gt;\n    &lt;li&gt;\n     Eagleswood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Harvey Cedars\n    &lt;/li&gt;\n    &lt;li&gt;\n     Island Heights\n    &lt;/li&gt;\n    &lt;li&gt;\n     Jackson\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lacey\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lakehurst\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lakewood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lavallette\n    &lt;/li&gt;\n    &lt;li&gt;\n     Little Egg Harbor\n    &lt;/li&gt;\n    &lt;li&gt;\n     Long Beach\n    &lt;/li&gt;\n    &lt;li&gt;\n     Manchester\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mantoloking\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ocean\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ocean Gate\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pine Beach\n    &lt;/li&gt;\n    &lt;li&gt;\n     Plumsted\n    &lt;/li&gt;\n    &lt;li&gt;\n     Point Pleasant\n    &lt;/li&gt;\n    &lt;li&gt;\n     Point Pleasant Beach\n    &lt;/li&gt;\n    &lt;li&gt;\n     Seaside Heights\n    &lt;/li&gt;\n    &lt;li&gt;\n     Seaside Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ship Bottom\n    &lt;/li&gt;\n    &lt;li&gt;\n     South Toms River\n    &lt;/li&gt;\n    &lt;li&gt;\n     Stafford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Surf City\n    &lt;/li&gt;\n    &lt;li&gt;\n     Tuckerton\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Passaic\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Bloomingdale\n    &lt;/li&gt;\n    &lt;li&gt;\n     Clifton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Haledon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hawthorne\n    &lt;/li&gt;\n    &lt;li&gt;\n     Little Falls\n    &lt;/li&gt;\n    &lt;li&gt;\n     North Haledon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Passaic\n    &lt;/li&gt;\n    &lt;li&gt;\n     Paterson\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pompton Lakes\n    &lt;/li&gt;\n    &lt;li&gt;\n     Prospect Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ringwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Totowa\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wanaque\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wayne\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Milford\n    &lt;/li&gt;\n    &lt;li&gt;\n     West Paterson\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Salem\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     &lt;span class=&#34;Apple-style-span&#34;&gt;\n      Alloway\n     &lt;/span&gt;\n    &lt;/li&gt;\n    &lt;li&gt;\n     Carneys Point\n    &lt;/li&gt;\n    &lt;li&gt;\n     Elmer\n    &lt;/li&gt;\n    &lt;li&gt;\n     Elsinboro\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lower Alloways Creek\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mannington\n    &lt;/li&gt;\n    &lt;li&gt;\n     Oldmans\n    &lt;/li&gt;\n    &lt;li&gt;\n     Penns Grove\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pennsville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pilesgrove\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pittsgrove\n    &lt;/li&gt;\n    &lt;li&gt;\n     Quinton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Salem\n    &lt;/li&gt;\n    &lt;li&gt;\n     Upper Pittsgrove\n    &lt;/li&gt;\n    &lt;li&gt;\n     Woodstown\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Somerset\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     &lt;span class=&#34;Apple-style-span&#34;&gt;\n      Bedminster\n     &lt;/span&gt;\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bernards\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bernardsville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bound Brook\n    &lt;/li&gt;\n    &lt;li&gt;\n     Branchburg\n    &lt;/li&gt;\n    &lt;li&gt;\n     Bridgewater\n    &lt;/li&gt;\n    &lt;li&gt;\n     Far Hills\n    &lt;/li&gt;\n    &lt;li&gt;\n     Franklin\n    &lt;/li&gt;\n    &lt;li&gt;\n     Green Brook\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hillsborough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Manville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Millstone\n    &lt;/li&gt;\n    &lt;li&gt;\n     Montgomery\n    &lt;/li&gt;\n    &lt;li&gt;\n     North Plainfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Peapack &amp;amp; Gladstone\n    &lt;/li&gt;\n    &lt;li&gt;\n     Raritan\n    &lt;/li&gt;\n    &lt;li&gt;\n     Rocky Hill\n    &lt;/li&gt;\n    &lt;li&gt;\n     Somerville\n    &lt;/li&gt;\n    &lt;li&gt;\n     South Bound Brook\n    &lt;/li&gt;\n    &lt;li&gt;\n     Warren\n    &lt;/li&gt;\n    &lt;li&gt;\n     Watchung\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Sussex\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Andover borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Andover township\n    &lt;/li&gt;\n    &lt;li&gt;\n     Branchville\n    &lt;/li&gt;\n    &lt;li&gt;\n     Byram\n    &lt;/li&gt;\n    &lt;li&gt;\n     Frankford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Franklin\n    &lt;/li&gt;\n    &lt;li&gt;\n     Fredon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Green\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hamburg\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hampton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hardyston\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hopatcong\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lafayette\n    &lt;/li&gt;\n    &lt;li&gt;\n     Montague\n    &lt;/li&gt;\n    &lt;li&gt;\n     Newton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Ogdensburg\n    &lt;/li&gt;\n    &lt;li&gt;\n     Sandyston\n    &lt;/li&gt;\n    &lt;li&gt;\n     Sparta\n    &lt;/li&gt;\n    &lt;li&gt;\n     Stanhope\n    &lt;/li&gt;\n    &lt;li&gt;\n     Stillwater\n    &lt;/li&gt;\n    &lt;li&gt;\n     Sussex\n    &lt;/li&gt;\n    &lt;li&gt;\n     Vernon\n    &lt;/li&gt;\n    &lt;li&gt;\n     Walpack\n    &lt;/li&gt;\n    &lt;li&gt;\n     Wantage\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Union\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     Berkeley Heights\n    &lt;/li&gt;\n    &lt;li&gt;\n     Clark\n    &lt;/li&gt;\n    &lt;li&gt;\n     Cranford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Elizabeth\n    &lt;/li&gt;\n    &lt;li&gt;\n     Fanwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Garwood\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hillside\n    &lt;/li&gt;\n    &lt;li&gt;\n     Kenilworth\n    &lt;/li&gt;\n    &lt;li&gt;\n     Linden\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mountainside\n    &lt;/li&gt;\n    &lt;li&gt;\n     New Providence\n    &lt;/li&gt;\n    &lt;li&gt;\n     Plainfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Rahway\n    &lt;/li&gt;\n    &lt;li&gt;\n     Roselle\n    &lt;/li&gt;\n    &lt;li&gt;\n     Roselle Park\n    &lt;/li&gt;\n    &lt;li&gt;\n     Scotch Plains\n    &lt;/li&gt;\n    &lt;li&gt;\n     Springfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Summit\n    &lt;/li&gt;\n    &lt;li&gt;\n     Union\n    &lt;/li&gt;\n    &lt;li&gt;\n     Westfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Winfield\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;h2&gt;\n    Warren\n   &lt;/h2&gt;\n   &lt;ul&gt;\n    &lt;li&gt;\n     &lt;span class=&#34;Apple-style-span&#34;&gt;\n      Allamuchy\n     &lt;/span&gt;\n    &lt;/li&gt;\n    &lt;li&gt;\n     Alpha\n    &lt;/li&gt;\n    &lt;li&gt;\n     Belvidere\n    &lt;/li&gt;\n    &lt;li&gt;\n     Blairstown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Franklin\n    &lt;/li&gt;\n    &lt;li&gt;\n     Frelinghuysen\n    &lt;/li&gt;\n    &lt;li&gt;\n     Greenwich\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hackettstown\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hardwick\n    &lt;/li&gt;\n    &lt;li&gt;\n     Harmony\n    &lt;/li&gt;\n    &lt;li&gt;\n     Hope\n    &lt;/li&gt;\n    &lt;li&gt;\n     Independence\n    &lt;/li&gt;\n    &lt;li&gt;\n     Knowlton\n    &lt;/li&gt;\n    &lt;li&gt;\n     Liberty\n    &lt;/li&gt;\n    &lt;li&gt;\n     Lopatcong\n    &lt;/li&gt;\n    &lt;li&gt;\n     Mansfield\n    &lt;/li&gt;\n    &lt;li&gt;\n     Oxford\n    &lt;/li&gt;\n    &lt;li&gt;\n     Phillipsburg\n    &lt;/li&gt;\n    &lt;li&gt;\n     Pohatcong\n    &lt;/li&gt;\n    &lt;li&gt;\n     Washington borough\n    &lt;/li&gt;\n    &lt;li&gt;\n     Washington township\n    &lt;/li&gt;\n    &lt;li&gt;\n     White\n    &lt;/li&gt;\n   &lt;/ul&gt;\n   &lt;p&gt;\n   &lt;/p&gt;\n  &lt;/td&gt;\n &lt;/tr&gt;\n&lt;/tbody&gt;\n</div>"]}}],"execution_count":21},{"cell_type":"code","source":["table_dict = {}\n# list all counties from header tags\ncounties = [county.text for county in table_body.findAll('h2')]\n\ncity_list = []\n# get a list of all cities within unordered list tag\nfor city_elem in table_body.findAll('ul'):\n    cities = [city_name.text for city_name in city_elem.findAll('li')]\n    city_list.append(cities)\n# create a dictionary that maps county -> city\nfor i in range(len(counties)):\n    table_dict[counties[i]] = city_list[i]\ntable_dict"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\">Out[8]: {&#39;Atlantic&#39;: [&#39;Absecon&#39;,\n  &#39;Atlantic City&#39;,\n  &#39;Brigantine&#39;,\n  &#39;Buena&#39;,\n  &#39;Buena Vista&#39;,\n  &#39;Corbin City&#39;,\n  &#39;Egg Harbor&#39;,\n  &#39;Egg Harbor City&#39;,\n  &#39;Estell Manor&#39;,\n  &#39;Folsom&#39;,\n  &#39;Galloway&#39;,\n  &#39;Hamilton&#39;,\n  &#39;Hammonton&#39;,\n  &#39;Linwood&#39;,\n  &#39;Longport&#39;,\n  &#39;Margate City&#39;,\n  &#39;Mullica&#39;,\n  &#39;Northfield&#39;,\n  &#39;Pleasantville&#39;,\n  &#39;Port Republic&#39;,\n  &#39;Somers Point&#39;,\n  &#39;Ventnor City&#39;,\n  &#39;Weymouth&#39;],\n &#39;Bergen&#39;: [&#39;Allendale&#39;,\n  &#39;Alpine&#39;,\n  &#39;Bergenfield&#39;,\n  &#39;Bogota&#39;,\n  &#39;Carlstadt&#39;,\n  &#39;Cliffside Park&#39;,\n  &#39;Closter&#39;,\n  &#39;Cresskill&#39;,\n  &#39;Demarest&#39;,\n  &#39;Dumont&#39;,\n  &#39;East Rutherford&#39;,\n  &#39;Edgewater&#39;,\n  &#39;Elmwood Park&#39;,\n  &#39;Emerson&#39;,\n  &#39;Englewood&#39;,\n  &#39;Englewood Cliffs&#39;,\n  &#39;Fair Lawn&#39;,\n  &#39;Fairview&#39;,\n  &#39;Fort Lee&#39;,\n  &#39;Franklin Lakes&#39;,\n  &#39;Garfield&#39;,\n  &#39;Glen Rock&#39;,\n  &#39;Hackensack&#39;,\n  &#39;Harrington Park&#39;,\n  &#39;Hasbrouck Heights&#39;,\n  &#39;Haworth&#39;,\n  &#39;Hillsdale&#39;,\n  &#39;Ho-Ho-Kus&#39;,\n  &#39;Leonia&#39;,\n  &#39;Little Ferry&#39;,\n  &#39;Lodi&#39;,\n  &#39;Lyndhurst&#39;,\n  &#39;Mahwah&#39;,\n  &#39;Maywood&#39;,\n  &#39;Midland Park&#39;,\n  &#39;Montvale&#39;,\n  &#39;Moonachie&#39;,\n  &#39;New Milford&#39;,\n  &#39;North Arlington&#39;,\n  &#39;Northvale&#39;,\n  &#39;Norwood&#39;,\n  &#39;Oakland&#39;,\n  &#39;Old Tappan&#39;,\n  &#39;Oradell&#39;,\n  &#39;Palisades Park&#39;,\n  &#39;Paramus&#39;,\n  &#39;Park Ridge&#39;,\n  &#39;Ramsey&#39;,\n  &#39;Ridgefield&#39;,\n  &#39;Ridgefield Park&#39;,\n  &#39;Ridgewood&#39;,\n  &#39;River Edge&#39;,\n  &#39;River Vale&#39;,\n  &#39;Rochelle Park&#39;,\n  &#39;Rockleigh&#39;,\n  &#39;Rutherford&#39;,\n  &#39;Saddle Brook&#39;,\n  &#39;Saddle River&#39;,\n  &#39;South Hackensack&#39;,\n  &#39;Teaneck&#39;,\n  &#39;Tenafly&#39;,\n  &#39;Teterboro&#39;,\n  &#39;Upper Saddle River&#39;,\n  &#39;Waldwick&#39;,\n  &#39;Wallington&#39;,\n  &#39;Washington&#39;,\n  &#39;Westwood&#39;,\n  &#39;Woodcliff Lake&#39;,\n  &#39;Wood-Ridge&#39;,\n  &#39;Wyckoff&#39;],\n &#39;Burlington&#39;: [&#39;Bass River&#39;,\n  &#39;Beverly&#39;,\n  &#39;Bordentown city&#39;,\n  &#39;Bordentown township&#39;,\n  &#39;Burlington city&#39;,\n  &#39;Burlington township&#39;,\n  &#39;Chesterfield&#39;,\n  &#39;Cinnaminson&#39;,\n  &#39;Delanco&#39;,\n  &#39;Delran&#39;,\n  &#39;Eastampton&#39;,\n  &#39;Edgewater Park&#39;,\n  &#39;Evesham&#39;,\n  &#39;Fieldsboro&#39;,\n  &#39;Florence&#39;,\n  &#39;Hainesport&#39;,\n  &#39;Lumberton&#39;,\n  &#39;Mansfield&#39;,\n  &#39;Maple Shade&#39;,\n  &#39;Medford&#39;,\n  &#39;Medford Lakes&#39;,\n  &#39;Moorestown&#39;,\n  &#39;Mount Holly&#39;,\n  &#39;Mount Laurel&#39;,\n  &#39;New Hanover&#39;,\n  &#39;North Hanover&#39;,\n  &#39;Palmyra&#39;,\n  &#39;Pemberton borough&#39;,\n  &#39;Pemberton township&#39;,\n  &#39;Riverside&#39;,\n  &#39;Riverton&#39;,\n  &#39;Shamong&#39;,\n  &#39;Southampton&#39;,\n  &#39;Springfield&#39;,\n  &#39;Tabernacle&#39;,\n  &#39;Washington&#39;,\n  &#39;Westampton&#39;,\n  &#39;Willingboro&#39;,\n  &#39;Woodland&#39;,\n  &#39;Wrightstown&#39;],\n &#39;Camden&#39;: [&#39;Audubon&#39;,\n  &#39;Audubon Park&#39;,\n  &#39;Barrington&#39;,\n  &#39;Bellmawr&#39;,\n  &#39;Berlin borough&#39;,\n  &#39;Berlin township&#39;,\n  &#39;Brooklawn&#39;,\n  &#39;Camden&#39;,\n  &#39;Cherry Hill&#39;,\n  &#39;Chesilhurst&#39;,\n  &#39;Clementon&#39;,\n  &#39;Collingswood&#39;,\n  &#39;Gibbsboro&#39;,\n  &#39;Gloucester township&#39;,\n  &#39;Gloucester City&#39;,\n  &#39;Haddon&#39;,\n  &#39;Haddon Heights&#39;,\n  &#39;Haddonfield&#39;,\n  &#39;Hi-Nella&#39;,\n  &#39;Laurel Springs&#39;,\n  &#39;Lawnside&#39;,\n  &#39;Lindenwold&#39;,\n  &#39;Magnolia&#39;,\n  &#39;Merchantville&#39;,\n  &#39;Mount Ephraim&#39;,\n  &#39;Oaklyn borough&#39;,\n  &#39;Pennsauken&#39;,\n  &#39;Pine Hill&#39;,\n  &#39;Pine Valley&#39;,\n  &#39;Runnemede&#39;,\n  &#39;Somerdale&#39;,\n  &#39;Stratford&#39;,\n  &#39;Tavistock&#39;,\n  &#39;Voorhees&#39;,\n  &#39;Waterford&#39;,\n  &#39;Winslow&#39;,\n  &#39;Woodlynne&#39;],\n &#39;Cape May&#39;: [&#39;Avalon&#39;,\n  &#39;Cape May&#39;,\n  &#39;Cape May Point&#39;,\n  &#39;Dennis&#39;,\n  &#39;Lower&#39;,\n  &#39;Middle&#39;,\n  &#39;North Wildwood&#39;,\n  &#39;Ocean City&#39;,\n  &#39;Sea Isle City&#39;,\n  &#39;Stone Harbor&#39;,\n  &#39;Upper&#39;,\n  &#39;West Cape May&#39;,\n  &#39;West Wildwood&#39;,\n  &#39;Wildwood&#39;,\n  &#39;Wildwood Crest&#39;,\n  &#39;Woodbine&#39;],\n &#39;Cumberland&#39;: [&#39;Bridgeton&#39;,\n  &#39;Commercial&#39;,\n  &#39;Deerfield&#39;,\n  &#39;Downe&#39;,\n  &#39;Fairfield&#39;,\n  &#39;Greenwich&#39;,\n  &#39;Hopewell&#39;,\n  &#39;Lawrence&#39;,\n  &#39;Maurice River&#39;,\n  &#39;Millville&#39;,\n  &#39;Shiloh&#39;,\n  &#39;Stow Creek&#39;,\n  &#39;Upper Deerfield&#39;,\n  &#39;Vineland&#39;],\n &#39;Essex&#39;: [&#39;Belleville&#39;,\n  &#39;Bloomfield&#39;,\n  &#39;Caldwell Borough&#39;,\n  &#39;Cedar Grove&#39;,\n  &#39;City of Orange&#39;,\n  &#39;East Orange&#39;,\n  &#39;Essex Fells&#39;,\n  &#39;Fairfield&#39;,\n  &#39;Glen Ridge&#39;,\n  &#39;Irvington&#39;,\n  &#39;Livingston&#39;,\n  &#39;Maplewood&#39;,\n  &#39;Millburn&#39;,\n  &#39;Montclair&#39;,\n  &#39;Newark&#39;,\n  &#39;North Caldwell&#39;,\n  &#39;Nutley&#39;,\n  &#39;Roseland&#39;,\n  &#39;Verona&#39;,\n  &#39;Village of South Orange&#39;,\n  &#39;West Caldwell&#39;,\n  &#39;West Orange&#39;],\n &#39;Gloucester&#39;: [&#39;Clayton&#39;,\n  &#39;Deptford&#39;,\n  &#39;East Greenwich&#39;,\n  &#39;Elk&#39;,\n  &#39;Franklin&#39;,\n  &#39;Glassboro&#39;,\n  &#39;Greenwich&#39;,\n  &#39;Harrison&#39;,\n  &#39;Logan&#39;,\n  &#39;Mantua&#39;,\n  &#39;Monroe&#39;,\n  &#39;National Park&#39;,\n  &#39;Newfield&#39;,\n  &#39;Paulsboro&#39;,\n  &#39;Pitman&#39;,\n  &#39;South Harrison&#39;,\n  &#39;Swedesboro&#39;,\n  &#39;Washington&#39;,\n  &#39;Wenonah&#39;,\n  &#39;West Deptford&#39;,\n  &#39;Westville&#39;,\n  &#39;Woodbury&#39;,\n  &#39;Woodbury Heights&#39;,\n  &#39;Woolwich&#39;],\n &#39;Hudson&#39;: [&#39;Bayonne&#39;,\n  &#39;East Newark&#39;,\n  &#39;Guttenberg&#39;,\n  &#39;Harrison&#39;,\n  &#39;Hoboken&#39;,\n  &#39;Jersey City&#39;,\n  &#39;Kearny&#39;,\n  &#39;North Bergen&#39;,\n  &#39;Secaucus&#39;,\n  &#39;Union City&#39;,\n  &#39;Weehawken&#39;,\n  &#39;West New York&#39;],\n &#39;Hunterdon&#39;: [&#39;Alexandria&#39;,\n  &#39;Bethlehem&#39;,\n  &#39;Bloomsbury&#39;,\n  &#39;Califon&#39;,\n  &#39;Clinton town&#39;,\n  &#39;Clinton township&#39;,\n  &#39;Delaware&#39;,\n  &#39;East Amwell&#39;,\n  &#39;Flemington&#39;,\n  &#39;Franklin&#39;,\n  &#39;Frenchtown&#39;,\n  &#39;Glen Gardner&#39;,\n  &#39;Hampton&#39;,\n  &#39;High Bridge&#39;,\n  &#39;Holland&#39;,\n  &#39;Kingwood&#39;,\n  &#39;Lambertville&#39;,\n  &#39;Lebanon borough&#39;,\n  &#39;Lebanon township&#39;,\n  &#39;Milford&#39;,\n  &#39;Raritan&#39;,\n  &#39;Readington&#39;,\n  &#39;Stockton&#39;,\n  &#39;Tewksbury&#39;,\n  &#39;Union&#39;,\n  &#39;West Amwell&#39;],\n &#39;Mercer&#39;: [&#39;East Windsor&#39;,\n  &#39;Ewing&#39;,\n  &#39;Hamilton&#39;,\n  &#39;Hightstown&#39;,\n  &#39;Hopewell borough&#39;,\n  &#39;Hopewell township&#39;,\n  &#39;Lawrence&#39;,\n  &#39;Pennington&#39;,\n  &#39;Princeton township&#39;,\n  &#39;Trenton&#39;,\n  &#39;Washington&#39;,\n  &#39;West Windsor&#39;],\n &#39;Middlesex&#39;: [&#39;Carteret&#39;,\n  &#39;Cranbury&#39;,\n  &#39;Dunellen&#39;,\n  &#39;East Brunswick&#39;,\n  &#39;Edison&#39;,\n  &#39;Helmetta&#39;,\n  &#39;Highland Park&#39;,\n  &#39;Jamesburg&#39;,\n  &#39;Metuchen&#39;,\n  &#39;Middlesex&#39;,\n  &#39;Milltown&#39;,\n  &#39;Monroe&#39;,\n  &#39;New Brunswick&#39;,\n  &#39;North Brunswick&#39;,\n  &#39;Old Bridge&#39;,\n  &#39;Perth Amboy&#39;,\n  &#39;Piscataway&#39;,\n  &#39;Plainsboro&#39;,\n  &#39;Sayreville&#39;,\n  &#39;South Amboy&#39;,\n  &#39;South Brunswick&#39;,\n  &#39;South Plainfield&#39;,\n  &#39;South River&#39;,\n  &#39;Spotswood&#39;,\n  &#39;Woodbridge&#39;],\n &#39;Monmouth&#39;: [&#39;Aberdeen&#39;,\n  &#39;Allenhurst&#39;,\n  &#39;Allentown&#39;,\n  &#39;Asbury Park&#39;,\n  &#39;Atlantic Highlands&#39;,\n  &#39;Avon-by-the-Sea&#39;,\n  &#39;Belmar&#39;,\n  &#39;Bradley Beach&#39;,\n  &#39;Brielle&#39;,\n  &#39;Colts Neck&#39;,\n  &#39;Deal&#39;,\n  &#39;Eatontown&#39;,\n  &#39;Englishtown&#39;,\n  &#39;Fair Haven&#39;,\n  &#39;Farmingdale&#39;,\n  &#39;Freehold borough&#39;,\n  &#39;Freehold township&#39;,\n  &#39;Hazlet&#39;,\n  &#39;Highlands&#39;,\n  &#39;Holmdel&#39;,\n  &#39;Howell&#39;,\n  &#39;Interlaken&#39;,\n  &#39;Keansburg&#39;,\n  &#39;Keyport&#39;,\n  &#39;Little Silver&#39;,\n  &#39;Loch Arbour&#39;,\n  &#39;Long Branch&#39;,\n  &#39;Manalapan&#39;,\n  &#39;Manasquan&#39;,\n  &#39;Marlboro&#39;,\n  &#39;Matawan&#39;,\n  &#39;Middletown&#39;,\n  &#39;Millstone&#39;,\n  &#39;Monmouth Beach&#39;,\n  &#39;Neptune&#39;,\n  &#39;Neptune City&#39;,\n  &#39;Ocean&#39;,\n  &#39;Oceanport&#39;,\n  &#39;Red Bank&#39;,\n  &#39;Roosevelt&#39;,\n  &#39;Rumson&#39;,\n  &#39;Sea Bright&#39;,\n  &#39;Sea Girt&#39;,\n  &#39;Shrewsbury borough&#39;,\n  &#39;Shrewsbury township&#39;,\n  &#39;South Belmar&#39;,\n  &#39;Spring Lake Boro&#39;,\n  &#39;Spring Lake Heights&#39;,\n  &#39;Tinton Falls&#39;,\n  &#39;Union Beach&#39;,\n  &#39;Upper Freehold&#39;,\n  &#39;Wall&#39;,\n  &#39;West Long Branch&#39;],\n &#39;Morris&#39;: [&#39;Boonton town&#39;,\n  &#39;Boonton township&#39;,\n  &#39;Butler&#39;,\n  &#39;Chatham borough&#39;,\n  &#39;Chatham township&#39;,\n  &#39;Chester borough&#39;,\n  &#39;Chester township&#39;,\n  &#39;Denville&#39;,\n  &#39;Dover&#39;,\n  &#39;East Hanover&#39;,\n  &#39;Florham Park&#39;,\n  &#39;Hanover&#39;,\n  &#39;Harding&#39;,\n  &#39;Jefferson&#39;,\n  &#39;Kinnelon&#39;,\n  &#39;Lincoln Park&#39;,\n  &#39;Long Hill&#39;,\n  &#39;Madison&#39;,\n  &#39;Mendham borough&#39;,\n  &#39;Mendham township&#39;,\n  &#39;Mine Hill&#39;,\n  &#39;Montville&#39;,\n  &#39;Morris&#39;,\n  &#39;Morris Plains&#39;,\n  &#39;Morristown&#39;,\n  &#39;Mount Arlington&#39;,\n  &#39;Mount Olive&#39;,\n  &#39;Mountain Lakes&#39;,\n  &#39;Netcong&#39;,\n  &#39;Parsippany-Troy Hills&#39;,\n  &#39;Pequannock&#39;,\n  &#39;Randolph&#39;,\n  &#39;Riverdale&#39;,\n  &#39;Rockaway borough&#39;,\n  &#39;Rockaway township&#39;,\n  &#39;Roxbury&#39;,\n  &#39;Victory Gardens&#39;,\n  &#39;Washington&#39;,\n  &#39;Wharton&#39;],\n &#39;Ocean&#39;: [&#39;Barnegat&#39;,\n  &#39;Barnegat Light&#39;,\n  &#39;Bay Head&#39;,\n  &#39;Beach Haven&#39;,\n  &#39;Beachwood&#39;,\n  &#39;Berkeley&#39;,\n  &#39;Brick&#39;,\n  &#39;Dover&#39;,\n  &#39;Eagleswood&#39;,\n  &#39;Harvey Cedars&#39;,\n  &#39;Island Heights&#39;,\n  &#39;Jackson&#39;,\n  &#39;Lacey&#39;,\n  &#39;Lakehurst&#39;,\n  &#39;Lakewood&#39;,\n  &#39;Lavallette&#39;,\n  &#39;Little Egg Harbor&#39;,\n  &#39;Long Beach&#39;,\n  &#39;Manchester&#39;,\n  &#39;Mantoloking&#39;,\n  &#39;Ocean&#39;,\n  &#39;Ocean Gate&#39;,\n  &#39;Pine Beach&#39;,\n  &#39;Plumsted&#39;,\n  &#39;Point Pleasant&#39;,\n  &#39;Point Pleasant Beach&#39;,\n  &#39;Seaside Heights&#39;,\n  &#39;Seaside Park&#39;,\n  &#39;Ship Bottom&#39;,\n  &#39;South Toms River&#39;,\n  &#39;Stafford&#39;,\n  &#39;Surf City&#39;,\n  &#39;Tuckerton&#39;],\n &#39;Passaic&#39;: [&#39;Bloomingdale&#39;,\n  &#39;Clifton&#39;,\n  &#39;Haledon&#39;,\n  &#39;Hawthorne&#39;,\n  &#39;Little Falls&#39;,\n  &#39;North Haledon&#39;,\n  &#39;Passaic&#39;,\n  &#39;Paterson&#39;,\n  &#39;Pompton Lakes&#39;,\n  &#39;Prospect Park&#39;,\n  &#39;Ringwood&#39;,\n  &#39;Totowa&#39;,\n  &#39;Wanaque&#39;,\n  &#39;Wayne&#39;,\n  &#39;West Milford&#39;,\n  &#39;West Paterson&#39;],\n &#39;Salem&#39;: [&#39;Alloway&#39;,\n  &#39;Carneys Point&#39;,\n  &#39;Elmer&#39;,\n  &#39;Elsinboro&#39;,\n  &#39;Lower Alloways Creek&#39;,\n  &#39;Mannington&#39;,\n  &#39;Oldmans&#39;,\n  &#39;Penns Grove&#39;,\n  &#39;Pennsville&#39;,\n  &#39;Pilesgrove&#39;,\n  &#39;Pittsgrove&#39;,\n  &#39;Quinton&#39;,\n  &#39;Salem&#39;,\n  &#39;Upper Pittsgrove&#39;,\n  &#39;Woodstown&#39;],\n &#39;Somerset&#39;: [&#39;Bedminster&#39;,\n  &#39;Bernards&#39;,\n  &#39;Bernardsville&#39;,\n  &#39;Bound Brook&#39;,\n  &#39;Branchburg&#39;,\n  &#39;Bridgewater&#39;,\n  &#39;Far Hills&#39;,\n  &#39;Franklin&#39;,\n  &#39;Green Brook&#39;,\n  &#39;Hillsborough&#39;,\n  &#39;Manville&#39;,\n  &#39;Millstone&#39;,\n  &#39;Montgomery&#39;,\n  &#39;North Plainfield&#39;,\n  &#39;Peapack &amp; Gladstone&#39;,\n  &#39;Raritan&#39;,\n  &#39;Rocky Hill&#39;,\n  &#39;Somerville&#39;,\n  &#39;South Bound Brook&#39;,\n  &#39;Warren&#39;,\n  &#39;Watchung&#39;],\n &#39;Sussex&#39;: [&#39;Andover borough&#39;,\n  &#39;Andover township&#39;,\n  &#39;Branchville&#39;,\n  &#39;Byram&#39;,\n  &#39;Frankford&#39;,\n  &#39;Franklin&#39;,\n  &#39;Fredon&#39;,\n  &#39;Green&#39;,\n  &#39;Hamburg&#39;,\n  &#39;Hampton&#39;,\n  &#39;Hardyston&#39;,\n  &#39;Hopatcong&#39;,\n  &#39;Lafayette&#39;,\n  &#39;Montague&#39;,\n  &#39;Newton&#39;,\n  &#39;Ogdensburg&#39;,\n  &#39;Sandyston&#39;,\n  &#39;Sparta&#39;,\n  &#39;Stanhope&#39;,\n  &#39;Stillwater&#39;,\n  &#39;Sussex&#39;,\n  &#39;Vernon&#39;,\n  &#39;Walpack&#39;,\n  &#39;Wantage&#39;],\n &#39;Union&#39;: [&#39;Berkeley Heights&#39;,\n  &#39;Clark&#39;,\n  &#39;Cranford&#39;,\n  &#39;Elizabeth&#39;,\n  &#39;Fanwood&#39;,\n  &#39;Garwood&#39;,\n  &#39;Hillside&#39;,\n  &#39;Kenilworth&#39;,\n  &#39;Linden&#39;,\n  &#39;Mountainside&#39;,\n  &#39;New Providence&#39;,\n  &#39;Plainfield&#39;,\n  &#39;Rahway&#39;,\n  &#39;Roselle&#39;,\n  &#39;Roselle Park&#39;,\n  &#39;Scotch Plains&#39;,\n  &#39;Springfield&#39;,\n  &#39;Summit&#39;,\n  &#39;Union&#39;,\n  &#39;Westfield&#39;,\n  &#39;Winfield&#39;],\n &#39;Warren&#39;: [&#39;Allamuchy&#39;,\n  &#39;Alpha&#39;,\n  &#39;Belvidere&#39;,\n  &#39;Blairstown&#39;,\n  &#39;Franklin&#39;,\n  &#39;Frelinghuysen&#39;,\n  &#39;Greenwich&#39;,\n  &#39;Hackettstown&#39;,\n  &#39;Hardwick&#39;,\n  &#39;Harmony&#39;,\n  &#39;Hope&#39;,\n  &#39;Independence&#39;,\n  &#39;Knowlton&#39;,\n  &#39;Liberty&#39;,\n  &#39;Lopatcong&#39;,\n  &#39;Mansfield&#39;,\n  &#39;Oxford&#39;,\n  &#39;Phillipsburg&#39;,\n  &#39;Pohatcong&#39;,\n  &#39;Washington borough&#39;,\n  &#39;Washington township&#39;,\n  &#39;White&#39;]}</div>"]}}],"execution_count":22},{"cell_type":"code","source":["full_list = []\n# create a list that maps county -> city\nfor county in list(table_dict.keys()):\n    full_list.append([[county, city] for city in table_dict[county]])\nflatten = [item for sublist in full_list for item in sublist]\nflatten[:5]"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\">Out[9]: [[&#39;Atlantic&#39;, &#39;Absecon&#39;],\n [&#39;Atlantic&#39;, &#39;Atlantic City&#39;],\n [&#39;Atlantic&#39;, &#39;Brigantine&#39;],\n [&#39;Atlantic&#39;, &#39;Buena&#39;],\n [&#39;Atlantic&#39;, &#39;Buena Vista&#39;]]</div>"]}}],"execution_count":23},{"cell_type":"code","source":["# view table\ndf = pd.DataFrame(flatten, columns=['County', 'City'])\ndisplay(df.head())"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .table-result-container {\n    max-height: 300px;\n    overflow: auto;\n  }\n  table, th, td {\n    border: 1px solid black;\n    border-collapse: collapse;\n  }\n  th, td {\n    padding: 5px;\n  }\n  th {\n    text-align: left;\n  }\n</style><div class='table-result-container'><table class='table-result'><thead style='background-color: white'><tr><th>County</th><th>City</th></tr></thead><tbody><tr><td>Atlantic</td><td>Absecon</td></tr><tr><td>Atlantic</td><td>Atlantic City</td></tr><tr><td>Atlantic</td><td>Brigantine</td></tr><tr><td>Atlantic</td><td>Buena</td></tr><tr><td>Atlantic</td><td>Buena Vista</td></tr></tbody></table></div>"]}}],"execution_count":24},{"cell_type":"code","source":["# view target area\ndisplay(df.loc[df['City'] == 'East Brunswick'])"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .table-result-container {\n    max-height: 300px;\n    overflow: auto;\n  }\n  table, th, td {\n    border: 1px solid black;\n    border-collapse: collapse;\n  }\n  th, td {\n    padding: 5px;\n  }\n  th {\n    text-align: left;\n  }\n</style><div class='table-result-container'><table class='table-result'><thead style='background-color: white'><tr><th>County</th><th>City</th></tr></thead><tbody><tr><td>Middlesex</td><td>East Brunswick</td></tr></tbody></table></div>"]}}],"execution_count":25},{"cell_type":"markdown","source":["### 2) News Data"],"metadata":{}},{"cell_type":"code","source":["# parse nj news content\nsoup = get_url_contents(newsUrl)"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\">Extracted website contents!\n</div>"]}}],"execution_count":27},{"cell_type":"code","source":["# view top 10 most recent blog titles\nblog_titles = soup.findAll('a', attrs={\"class\": \"near-black-link\"})\nblog_titles_list = [title.text for title in blog_titles]\nfor i in range(len(blog_titles_list[:10])):\n  print('Blog title {}: {}'.format(i + 1, blog_titles_list[i]))"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\">Blog title 1: 5 New Open Houses In The East Brunswick Area\nBlog title 2: First Look At Dramatic New Rt. 18 Development In East Brunswick\nBlog title 3: East Brunswick Plans Massive Development Project Along Rt. 18\nBlog title 4: Sayreville: Massive Luxury Apt. Complex &#39;Riverton&#39; May Be Coming\nBlog title 5: The Most Expensive Home In Middlesex County ....\nBlog title 6: The Cheapest Home In Middlesex County Is ...\nBlog title 7: 2018 Best Cities For Middle-Class Buyers — And The Worst\nBlog title 8: Top Home In Highly-Rated East Brunswick School District\nBlog title 9: Most Expensive Home On The Market In East Brunswick\nBlog title 10: Explore This Lakefront Home In East Brunswick\n</div>"]}}],"execution_count":28},{"cell_type":"code","source":["# view top 10 most recent blog descriptions\nblog_descs = soup.findAll('p', attrs={\"class\": \"d-none d-sm-block lineheight-1-5 m-0 py-2 text-secondary text-serif text-xs\"})\nblog_descs_list = [desc.text.replace(\"\\n  \", \"\") for desc in blog_descs]\nfor i in range(len(blog_descs_list[:10])):\n  print('Blog descriptions {}: {}'.format(i + 1, blog_descs_list[i]))"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .ansiout {\n    display: block;\n    unicode-bidi: embed;\n    white-space: pre-wrap;\n    word-wrap: break-word;\n    word-break: break-all;\n    font-family: \"Source Code Pro\", \"Menlo\", monospace;;\n    font-size: 13px;\n    color: #555;\n    margin-left: 4px;\n    line-height: 19px;\n  }\n</style>\n<div class=\"ansiout\">Blog descriptions 1: Here are the most recent properties to hit the local open-house circuit.\nBlog descriptions 2: On Tuesday, East Brunswick released never-before-seen renderings of the dramatic —​ and somewhat controversial —​ Rt. 18 redevelopment plan.\nBlog descriptions 3: A new bus stop, public parking, a hotel, residential and commercial units, a &#34;Tech Center&#34; and more are planned for this stretch of Rt. 18.\nBlog descriptions 4: If built, this project promises to dramatically re-shape Sayreville and the Raritan riverfront in this part of Central New Jersey.\nBlog descriptions 5:  ... is this one, located in ....\nBlog descriptions 6: But there are some very good reasons it has a $4,900 price tag.\nBlog descriptions 7: Sky-high prices got you down? Here are the top middle-class housing meccas.\nBlog descriptions 8: WOW House: Enjoy access to East Brunswick&#39;s blue-ribbon schools in this stunning villa with a pool and tennis courts near a peach orchard.\nBlog descriptions 9: WOW House: Explore this five-bedroom with a pool on 3.5 acres.\nBlog descriptions 10: WOW House: Asking $2.1 million in East Brunswick Twp., this property overlooks Farrington Lake.\n</div>"]}}],"execution_count":29},{"cell_type":"code","source":["# readable format\ndf_news = pd.DataFrame([[*range(1, 11)], blog_titles_list[:10], blog_descs_list[:10]]).T.rename(\n  columns={0: 'Blog Num', 1: 'Blog Title', 2: 'Blog Desc'})\ndf_news['City'] = 'East Brunswick'\ndisplay(df_news.head())"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .table-result-container {\n    max-height: 300px;\n    overflow: auto;\n  }\n  table, th, td {\n    border: 1px solid black;\n    border-collapse: collapse;\n  }\n  th, td {\n    padding: 5px;\n  }\n  th {\n    text-align: left;\n  }\n</style><div class='table-result-container'><table class='table-result'><thead style='background-color: white'><tr><th>Blog Num</th><th>Blog Title</th><th>Blog Desc</th><th>City</th></tr></thead><tbody><tr><td>1</td><td>5 New Open Houses In The East Brunswick Area</td><td>Here are the most recent properties to hit the local open-house circuit.</td><td>East Brunswick</td></tr><tr><td>2</td><td>First Look At Dramatic New Rt. 18 Development In East Brunswick</td><td>On Tuesday, East Brunswick released never-before-seen renderings of the dramatic —​ and somewhat controversial —​ Rt. 18 redevelopment plan.</td><td>East Brunswick</td></tr><tr><td>3</td><td>East Brunswick Plans Massive Development Project Along Rt. 18</td><td>A new bus stop, public parking, a hotel, residential and commercial units, a \"Tech Center\" and more are planned for this stretch of Rt. 18.</td><td>East Brunswick</td></tr><tr><td>4</td><td>Sayreville: Massive Luxury Apt. Complex 'Riverton' May Be Coming</td><td>If built, this project promises to dramatically re-shape Sayreville and the Raritan riverfront in this part of Central New Jersey.</td><td>East Brunswick</td></tr><tr><td>5</td><td>The Most Expensive Home In Middlesex County ....</td><td> ... is this one, located in ....</td><td>East Brunswick</td></tr></tbody></table></div>"]}}],"execution_count":30},{"cell_type":"code","source":["# join both datasets\ndf_join = df.merge(df_news, how='left', on=['City'])\ndisplay(df_join.loc[df_join['City'] == 'East Brunswick'])"],"metadata":{},"outputs":[{"metadata":{},"output_type":"display_data","data":{"text/html":["<style scoped>\n  .table-result-container {\n    max-height: 300px;\n    overflow: auto;\n  }\n  table, th, td {\n    border: 1px solid black;\n    border-collapse: collapse;\n  }\n  th, td {\n    padding: 5px;\n  }\n  th {\n    text-align: left;\n  }\n</style><div class='table-result-container'><table class='table-result'><thead style='background-color: white'><tr><th>County</th><th>City</th><th>Blog Num</th><th>Blog Title</th><th>Blog Desc</th></tr></thead><tbody><tr><td>Middlesex</td><td>East Brunswick</td><td>1</td><td>5 New Open Houses In The East Brunswick Area</td><td>Here are the most recent properties to hit the local open-house circuit.</td></tr><tr><td>Middlesex</td><td>East Brunswick</td><td>2</td><td>First Look At Dramatic New Rt. 18 Development In East Brunswick</td><td>On Tuesday, East Brunswick released never-before-seen renderings of the dramatic —​ and somewhat controversial —​ Rt. 18 redevelopment plan.</td></tr><tr><td>Middlesex</td><td>East Brunswick</td><td>3</td><td>East Brunswick Plans Massive Development Project Along Rt. 18</td><td>A new bus stop, public parking, a hotel, residential and commercial units, a \"Tech Center\" and more are planned for this stretch of Rt. 18.</td></tr><tr><td>Middlesex</td><td>East Brunswick</td><td>4</td><td>Sayreville: Massive Luxury Apt. Complex 'Riverton' May Be Coming</td><td>If built, this project promises to dramatically re-shape Sayreville and the Raritan riverfront in this part of Central New Jersey.</td></tr><tr><td>Middlesex</td><td>East Brunswick</td><td>5</td><td>The Most Expensive Home In Middlesex County ....</td><td> ... is this one, located in ....</td></tr><tr><td>Middlesex</td><td>East Brunswick</td><td>6</td><td>The Cheapest Home In Middlesex County Is ...</td><td>But there are some very good reasons it has a $4,900 price tag.</td></tr><tr><td>Middlesex</td><td>East Brunswick</td><td>7</td><td>2018 Best Cities For Middle-Class Buyers — And The Worst</td><td>Sky-high prices got you down? Here are the top middle-class housing meccas.</td></tr><tr><td>Middlesex</td><td>East Brunswick</td><td>8</td><td>Top Home In Highly-Rated East Brunswick School District</td><td>WOW House: Enjoy access to East Brunswick's blue-ribbon schools in this stunning villa with a pool and tennis courts near a peach orchard.</td></tr><tr><td>Middlesex</td><td>East Brunswick</td><td>9</td><td>Most Expensive Home On The Market In East Brunswick</td><td>WOW House: Explore this five-bedroom with a pool on 3.5 acres.</td></tr><tr><td>Middlesex</td><td>East Brunswick</td><td>10</td><td>Explore This Lakefront Home In East Brunswick</td><td>WOW House: Asking $2.1 million in East Brunswick Twp., this property overlooks Farrington Lake.</td></tr></tbody></table></div>"]}}],"execution_count":31},{"cell_type":"markdown","source":["### What do we have so far?\nTwo datasets merged into one file. This can be used to find trends per city.\n\nData set <b>features</b> (columns): County, City, Blog Num, Blog Title, Blog Desc\n\n### How do we make this a process?\nLabel each row as <i>important</i> (1) or <i>not important</i> (0). \n\n### How do we automate this process?\n1. Hire someone to manually label each new blog entry\n  - <b>Problem</b>: not scalable, cost of training someone, error of mislabeling \n  - \"Staying busy and being productive are NOT the same thing\"\n2. Develop a machine learning model that auto tags the important & non important label per blog entry. Set email alerts only when data labeled \"important\" is processed.\n  - <b>Benefit</b>: scalable, low-cost, low error (if trained with correct data - labels)\n  \n<img src=\"https://avo-translations.co.uk/wp-content/uploads/2019/05/1.png\" style=\"height:300px;\">"],"metadata":{}},{"cell_type":"markdown","source":["## How do we train a model to hand label what's important and what's not?\n\n1. NLP to extract text\n2. Clustering to group similar titles\n3. Label cluster\n4. Train model"],"metadata":{}},{"cell_type":"markdown","source":["### NLP example\n<img src=\"https://www.researchgate.net/profile/Pyry_Kettunen/publication/277248905/figure/fig4/AS:614337920106508@1523480982617/A-schematic-example-of-the-natural-language-processing-analysis-of-the-think-aloud.png\" style=\"height:600px;\">"],"metadata":{}},{"cell_type":"markdown","source":["### Train Model\n\n<img src=\"https://drive.google.com/uc?id=1cVv3brTMrvrsoPYaulogpRDaP5uthRUo\" style=\"height:600px;\">"],"metadata":{}},{"cell_type":"markdown","source":["## Is there an easier way to get data?\n### APIs(Application Programming Interface)\n<img src=\"https://img.deusm.com/darkreading/MarilynCohodas/TraditionalVModern.jpg\" style=\"height:400px;\">\n\n### Web tools with UI (User Interface)\n\n<img src=\"https://www.import.io/wp-content/uploads/2017/04/import-io-selezionare-attributo-3-1.png\" style=\"height:400px;\">\n<img src=\"https://www.import.io/wp-content/uploads/2017/04/import-io-pagina-tabellata-automaticamente-2-e1493125570920-1.png\" style=\"height:400px;\">\n\n<a href=\"https://www.import.io/post/importio-extract-pricing-data-web-page/\" target=\"_blank\">Import IO Pricing Data Example</a>"],"metadata":{}},{"cell_type":"markdown","source":["## Future\n- Other areas to extract data:\n  - municipality websites (i.e. permits)\n  - local forums (i.e. Facebook local community)\n  \n  <img src=\"https://drive.google.com/uc?id=1pGdcp0zeNxpBgqdT9ToxdEWqrtq5Vd99\" style=\"height:600px;\">"],"metadata":{}},{"cell_type":"markdown","source":["# End Notebook"],"metadata":{}}],"metadata":{"name":"How to do Web Scraping for Real Estate Data","notebookId":4206987172902979},"nbformat":4,"nbformat_minor":0}