README.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# How to use pandas_cub\n",
    "\n",
    "The README.ipynb notebook will serve as the documentation and usage guide to pandas_cub.\n",
    "\n",
    "## Installation\n",
    "\n",
    "`pip install pandas-cub`\n",
    "\n",
    "## What is pandas_cub?\n",
    "pandas_cub is a simple data analysis library that emulates the functionality of the pandas library. The library is not meant for serious work. It was built as an assignment for one of Ted Petrou's Python classes. If you would like to complete the assignment on your own, visit [this repository][1]. There are about 40 steps and 100 tests that you must pass in order to rebuild the library. It is a good challenge and teaches you the fundamentals of how to build your own data analysis library.\n",
    "\n",
    "## pandas_cub functionality\n",
    "\n",
    "pandas_cub has limited functionality but is still capable of a wide variety of data analysis tasks.\n",
    "\n",
    "* Subset selection with the brackets\n",
    "* Arithmetic and comparison operators (+, -, <, !=, etc...)\n",
    "* Aggregation of columns with most of the common functions (min, max, mean, median, etc...)\n",
    "* Grouping via pivot tables\n",
    "* String-only methods for columns containing strings\n",
    "* Reading in simple comma-separated value files\n",
    "* Several other methods\n",
    "\n",
    "\n",
    "## pandas_cub DataFrame\n",
    "\n",
    "pandas_cub has a single main object, the DataFrame, to hold all of the data. The DataFrame is capable of holding 4 data types - booleans, integers, floats, and strings. All data is stored in NumPy arrays. panda_cub DataFrames have no index (as in pandas). The columns must be strings.\n",
    "\n",
    "### Missing value representation\n",
    "Boolean and integer columns will have no missing value representation. The NumPy NaN is used for float columns and the Python None is used for string columns.\n",
    "\n",
    "## Code Examples\n",
    "\n",
    "pandas_cub syntax is very similar to pandas, but implements much fewer methods. The below examples will cover just about all of the API.\n",
    "\n",
    "[1]: https://github.com/tdpetrou/pandas_cub"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reading data with `read_csv`\n",
    "\n",
    "pandas_cub consists of a single function, `read_csv`, that has a single parameter, the location of the file you would like to read in as a DataFrame. This function can only handle simple CSV's and the delimiter must be a comma. A sample employee dataset is provided in the data directory. Notice that the visual output of the DataFrame is nearly identical to that of a pandas DataFrame. The `head` method returns the first 5 rows by default."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas_cub as pdc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>dept      </th><th>race      </th><th>gender    </th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63166</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     66614</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     71680</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42390</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115ad1d0>"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pdc.read_csv('data/employee.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### DataFrame properties"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `shape` property returns a tuple of the number of rows and columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1535, 4)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `len` function returns just the number of rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1535"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `dtypes` property returns a DataFrame of the column names and their respective data type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>Column Name</th><th>Data Type </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>dept      </td><td>string    </td></tr><tr><td><strong>1</strong></td><td>race      </td><td>string    </td></tr><tr><td><strong>2</strong></td><td>gender    </td><td>string    </td></tr><tr><td><strong>3</strong></td><td>salary    </td><td>int       </td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115adfd0>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `columns` property returns a list of the columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['dept', 'race', 'gender', 'salary']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set new columns by assigning the `columns` property to a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63166</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     66614</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     71680</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42390</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b0160>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns = ['department', 'race', 'gender', 'salary']\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `values` property returns a single numpy array of all the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([['Houston Police Department-HPD', 'White', 'Male', 45279],\n",
       "       ['Houston Fire Department (HFD)', 'White', 'Male', 63166],\n",
       "       ['Houston Police Department-HPD', 'Black', 'Male', 66614],\n",
       "       ...,\n",
       "       ['Houston Police Department-HPD', 'White', 'Male', 43443],\n",
       "       ['Houston Police Department-HPD', 'Asian', 'Male', 55461],\n",
       "       ['Houston Fire Department (HFD)', 'Hispanic', 'Male', 51194]],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Subset selection\n",
    "\n",
    "Subset selection is handled with the brackets. To select a single column, place that column name in the brackets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>White     </td></tr><tr><td><strong>1</strong></td><td>White     </td></tr><tr><td><strong>2</strong></td><td>Black     </td></tr><tr><td><strong>3</strong></td><td>Asian     </td></tr><tr><td><strong>4</strong></td><td>White     </td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b0f60>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['race'].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Select multiple columns with a list of strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>White     </td><td>     45279</td></tr><tr><td><strong>1</strong></td><td>White     </td><td>     63166</td></tr><tr><td><strong>2</strong></td><td>Black     </td><td>     66614</td></tr><tr><td><strong>3</strong></td><td>Asian     </td><td>     71680</td></tr><tr><td><strong>4</strong></td><td>White     </td><td>     42390</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b29e8>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['race', 'salary']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simultaneously select rows and columns by passing the brackets the row selection followed by the column selection separated by a comma. Here we use integers for rows and strings for columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th><th>race      </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>     77076</td><td>Black     </td></tr><tr><td><strong>1</strong></td><td>     81239</td><td>White     </td></tr><tr><td><strong>2</strong></td><td>     81239</td><td>White     </td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b3b38>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rows = [10, 50, 100]\n",
    "cols = ['salary', 'race']\n",
    "df[rows, cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use integers for the columns as well."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>gender    </th><th>department</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Male      </td><td>Houston Police Department-HPD</td></tr><tr><td><strong>1</strong></td><td>Male      </td><td>Houston Police Department-HPD</td></tr><tr><td><strong>2</strong></td><td>Male      </td><td>Houston Police Department-HPD</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b5208>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rows = [10, 50, 100]\n",
    "cols = [2, 0]\n",
    "df[rows, cols]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use a single integer and not just a list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>     66614</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b3940>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[99, 3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or a single string for the columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>     66614</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b5c50>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[99, 'salary']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use a slice for the rows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th><th>gender    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>White     </td><td>Male      </td></tr><tr><td><strong>1</strong></td><td>White     </td><td>Male      </td></tr><tr><td><strong>2</strong></td><td>Hispanic  </td><td>Male      </td></tr><tr><td><strong>3</strong></td><td>White     </td><td>Male      </td></tr><tr><td><strong>4</strong></td><td>White     </td><td>Male      </td></tr><tr><td><strong>5</strong></td><td>Hispanic  </td><td>Male      </td></tr><tr><td><strong>6</strong></td><td>Hispanic  </td><td>Male      </td></tr><tr><td><strong>7</strong></td><td>Black     </td><td>Female    </td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b7c88>"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[20:100:10, ['race', 'gender']]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also slice the columns with either integers or strings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Hispanic  </td></tr><tr><td><strong>3</strong></td><td>Houston Police Department-HPD</td><td>White     </td></tr><tr><td><strong>4</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td></tr><tr><td><strong>5</strong></td><td>Houston Police Department-HPD</td><td>Hispanic  </td></tr><tr><td><strong>6</strong></td><td>Houston Fire Department (HFD)</td><td>Hispanic  </td></tr><tr><td><strong>7</strong></td><td>Houston Police Department-HPD</td><td>Black     </td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b3588>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[20:100:10, :2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Hispanic  </td><td>Male      </td></tr><tr><td><strong>3</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td></tr><tr><td><strong>4</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td></tr><tr><td><strong>5</strong></td><td>Houston Police Department-HPD</td><td>Hispanic  </td><td>Male      </td></tr><tr><td><strong>6</strong></td><td>Houston Fire Department (HFD)</td><td>Hispanic  </td><td>Male      </td></tr><tr><td><strong>7</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Female    </td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b9c18>"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[20:100:10, 'department':'gender']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can do boolean selection if you pass the brackets a one-column boolean DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>False</td></tr><tr><td><strong>1</strong></td><td>False</td></tr><tr><td><strong>2</strong></td><td>False</td></tr><tr><td><strong>3</strong></td><td>False</td></tr><tr><td><strong>4</strong></td><td>False</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b70f0>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filt = df['salary'] > 100000\n",
    "filt.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Public Works & Engineering-PWE</td><td>White     </td><td>Male      </td><td>    107962</td></tr><tr><td><strong>1</strong></td><td>Health & Human Services</td><td>Black     </td><td>Male      </td><td>    180416</td></tr><tr><td><strong>2</strong></td><td>Houston Fire Department (HFD)</td><td>Hispanic  </td><td>Male      </td><td>    165216</td></tr><tr><td><strong>3</strong></td><td>Health & Human Services</td><td>White     </td><td>Female    </td><td>    100791</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>    120916</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b74a8>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[filt].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>White     </td><td>    107962</td></tr><tr><td><strong>1</strong></td><td>Black     </td><td>    180416</td></tr><tr><td><strong>2</strong></td><td>Hispanic  </td><td>    165216</td></tr><tr><td><strong>3</strong></td><td>White     </td><td>    100791</td></tr><tr><td><strong>4</strong></td><td>White     </td><td>    120916</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b98d0>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[filt, ['race', 'salary']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Assigning Columns\n",
    "You can only assign an entire new column or overwrite an old one. You cannot assign a subset of the data. You can assign a new column with a single value like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td><td>      1000</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63166</td><td>      1000</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     66614</td><td>      1000</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     71680</td><td>      1000</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42390</td><td>      1000</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115b9710>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['bonus'] = 1000\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can assign with a numpy array the same length as a column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td><td>      3536</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63166</td><td>      1296</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     66614</td><td>       511</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     71680</td><td>      4267</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42390</td><td>      3766</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bb160>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "df['bonus'] = np.random.randint(100, 5000, len(df))\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can assign a new column with a one column DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>     48815</td></tr><tr><td><strong>1</strong></td><td>     64462</td></tr><tr><td><strong>2</strong></td><td>     67125</td></tr><tr><td><strong>3</strong></td><td>     75947</td></tr><tr><td><strong>4</strong></td><td>     46156</td></tr><tr><td><strong>5</strong></td><td>    110001</td></tr><tr><td><strong>6</strong></td><td>     53738</td></tr><tr><td><strong>7</strong></td><td>    185348</td></tr><tr><td><strong>8</strong></td><td>     32575</td></tr><tr><td><strong>9</strong></td><td>     57918</td></tr><tr><strong><td>...</td></strong><td>...</td></tr><tr><td><strong>1525</strong></td><td>     32936</td></tr><tr><td><strong>1526</strong></td><td>     49294</td></tr><tr><td><strong>1527</strong></td><td>     34218</td></tr><tr><td><strong>1528</strong></td><td>     82795</td></tr><tr><td><strong>1529</strong></td><td>    104900</td></tr><tr><td><strong>1530</strong></td><td>     46408</td></tr><tr><td><strong>1531</strong></td><td>     67050</td></tr><tr><td><strong>1532</strong></td><td>     47368</td></tr><tr><td><strong>1533</strong></td><td>     60013</td></tr><tr><td><strong>1534</strong></td><td>     52624</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bb588>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['salary'] + df['bonus']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td><td>      3536</td><td>     48815</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63166</td><td>      1296</td><td>     64462</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     66614</td><td>       511</td><td>     67125</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     71680</td><td>      4267</td><td>     75947</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42390</td><td>      3766</td><td>     46156</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bbb70>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['total salary'] = df['salary'] + df['bonus']\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Arithmetic and comparison operators"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th><th>bonus     </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>    226395</td><td>     17680</td></tr><tr><td><strong>1</strong></td><td>    315830</td><td>      6480</td></tr><tr><td><strong>2</strong></td><td>    333070</td><td>      2555</td></tr><tr><td><strong>3</strong></td><td>    358400</td><td>     21335</td></tr><tr><td><strong>4</strong></td><td>    211950</td><td>     18830</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bbb00>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df[['salary', 'bonus']] * 5\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th><th>bonus     </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>False</td><td>False</td></tr><tr><td><strong>1</strong></td><td>False</td><td>False</td></tr><tr><td><strong>2</strong></td><td>False</td><td>False</td></tr><tr><td><strong>3</strong></td><td>False</td><td>False</td></tr><tr><td><strong>4</strong></td><td>False</td><td>False</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bd8d0>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df[['salary', 'bonus']] > 100000\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>True</td></tr><tr><td><strong>1</strong></td><td>True</td></tr><tr><td><strong>2</strong></td><td>False</td></tr><tr><td><strong>3</strong></td><td>False</td></tr><tr><td><strong>4</strong></td><td>True</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bdeb8>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df['race'] == 'White'\n",
    "df1.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Aggregation\n",
    "\n",
    "Most of the common aggregation methods are available. They only work down the columns and not across the rows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Health & Human Services</td><td>Asian     </td><td>Female    </td><td>     24960</td><td>       101</td><td>     25913</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bba58>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.min()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Columns that the aggregation does not work are dropped."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td> 56278.746</td><td>  2594.283</td><td> 58873.029</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115be3c8>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>         3</td><td>         0</td><td>         0</td><td>       145</td><td>      1516</td><td>       145</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115be5c0>"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.argmax()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>       347</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115c0160>"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['salary'].argmin()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check if all salaries are greater than 20000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>salary    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>True</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115c0710>"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = df['salary'] > 20000\n",
    "df1.all()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Count the number of non-missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>      1535</td><td>      1535</td><td>      1535</td><td>      1535</td><td>      1535</td><td>      1535</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115c0e10>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.count()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get number of unique values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>         6</td><td>         5</td><td>         2</td><td>       548</td><td>      1318</td><td>      1524</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bd6a0>"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.nunique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Non-Aggregating Methods\n",
    "These are methods that do not return a single value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get the unique values of each column. The `unique` method returns a list of DataFrames containing the unique values for each column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfs = df.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Health & Human Services</td></tr><tr><td><strong>1</strong></td><td>Houston Airport System (HAS)</td></tr><tr><td><strong>2</strong></td><td>Houston Fire Department (HFD)</td></tr><tr><td><strong>3</strong></td><td>Houston Police Department-HPD</td></tr><tr><td><strong>4</strong></td><td>Parks & Recreation</td></tr><tr><td><strong>5</strong></td><td>Public Works & Engineering-PWE</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115beac8>"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfs[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Asian     </td></tr><tr><td><strong>1</strong></td><td>Black     </td></tr><tr><td><strong>2</strong></td><td>Hispanic  </td></tr><tr><td><strong>3</strong></td><td>Native American</td></tr><tr><td><strong>4</strong></td><td>White     </td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115beeb8>"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfs[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>gender    </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Female    </td></tr><tr><td><strong>1</strong></td><td>Male      </td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bbcc0>"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfs[2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rename columns with a dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>dept      </th><th>race      </th><th>gender    </th><th>salary    </th><th>BONUS     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td><td>      3536</td><td>     48815</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63166</td><td>      1296</td><td>     64462</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     66614</td><td>       511</td><td>     67125</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     71680</td><td>      4267</td><td>     75947</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42390</td><td>      3766</td><td>     46156</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115c6550>"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.rename({'department':'dept', 'bonus':'BONUS'}).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Drop columns with a string or list of strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>Male      </td><td>     45279</td><td>      3536</td><td>     48815</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>Male      </td><td>     63166</td><td>      1296</td><td>     64462</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Male      </td><td>     66614</td><td>       511</td><td>     67125</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Male      </td><td>     71680</td><td>      4267</td><td>     75947</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>Male      </td><td>     42390</td><td>      3766</td><td>     46156</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115c6c88>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop('race').head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>     45279</td><td>      3536</td><td>     48815</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>     63166</td><td>      1296</td><td>     64462</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>     66614</td><td>       511</td><td>     67125</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>     71680</td><td>      4267</td><td>     75947</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>     42390</td><td>      3766</td><td>     46156</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115c8710>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.drop(['race', 'gender']).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Non-aggregating methods that keep all columns\n",
    "The next several methods are non-aggregating methods that return a DataFrame with the same exact shape as the original. They only work on boolean, integer and float columns and ignore string columns."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Absolute value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td><td>      3536</td><td>     48815</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63166</td><td>      1296</td><td>     64462</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     66614</td><td>       511</td><td>     67125</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     71680</td><td>      4267</td><td>     75947</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42390</td><td>      3766</td><td>     46156</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115c9320>"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.abs().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cumulative min, max, and sum"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td><td>      3536</td><td>     48815</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63166</td><td>      3536</td><td>     64462</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     66614</td><td>      3536</td><td>     67125</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     71680</td><td>      4267</td><td>     75947</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     71680</td><td>      4267</td><td>     75947</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115c9c18>"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.cummax().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clip values to be within a range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td><td>     40000</td><td>     48815</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     60000</td><td>     40000</td><td>     60000</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     60000</td><td>     40000</td><td>     60000</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     60000</td><td>     40000</td><td>     60000</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42390</td><td>     40000</td><td>     46156</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115cb908>"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.clip(40000, 60000).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Round numeric columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45000</td><td>      4000</td><td>     49000</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63000</td><td>      1000</td><td>     64000</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     67000</td><td>      1000</td><td>     67000</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     72000</td><td>      4000</td><td>     76000</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42000</td><td>      4000</td><td>     46000</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115cc0f0>"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.round(-3).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copy the DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     45279</td><td>      3536</td><td>     48815</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     63166</td><td>      1296</td><td>     64462</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     66614</td><td>       511</td><td>     67125</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     71680</td><td>      4267</td><td>     75947</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>     42390</td><td>      3766</td><td>     46156</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115ccba8>"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.copy().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Take the nth difference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>       nan</td><td>       nan</td><td>       nan</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>       nan</td><td>       nan</td><td>       nan</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td> 21335.000</td><td> -3025.000</td><td> 18310.000</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>  8514.000</td><td>  2971.000</td><td> 11485.000</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>-24224.000</td><td>  3255.000</td><td>-20969.000</td></tr><tr><td><strong>5</strong></td><td>Public Works & Engineering-PWE</td><td>White     </td><td>Male      </td><td> 36282.000</td><td> -2228.000</td><td> 34054.000</td></tr><tr><td><strong>6</strong></td><td>Houston Fire Department (HFD)</td><td>Hispanic  </td><td>Male      </td><td> 10254.000</td><td> -2672.000</td><td>  7582.000</td></tr><tr><td><strong>7</strong></td><td>Health & Human Services</td><td>Black     </td><td>Male      </td><td> 72454.000</td><td>  2893.000</td><td> 75347.000</td></tr><tr><td><strong>8</strong></td><td>Public Works & Engineering-PWE</td><td>Black     </td><td>Male      </td><td>-22297.000</td><td>  1134.000</td><td>-21163.000</td></tr><tr><td><strong>9</strong></td><td>Health & Human Services</td><td>Black     </td><td>Male      </td><td>-125147.000</td><td> -2283.000</td><td>-127430.000</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115ce828>"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.diff(2).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Find the nth percentage change."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>       nan</td><td>       nan</td><td>       nan</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>       nan</td><td>       nan</td><td>       nan</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     0.471</td><td>    -0.855</td><td>     0.375</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     0.135</td><td>     2.292</td><td>     0.178</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>White     </td><td>Male      </td><td>    -0.364</td><td>     6.370</td><td>    -0.312</td></tr><tr><td><strong>5</strong></td><td>Public Works & Engineering-PWE</td><td>White     </td><td>Male      </td><td>     0.506</td><td>    -0.522</td><td>     0.448</td></tr><tr><td><strong>6</strong></td><td>Houston Fire Department (HFD)</td><td>Hispanic  </td><td>Male      </td><td>     0.242</td><td>    -0.710</td><td>     0.164</td></tr><tr><td><strong>7</strong></td><td>Health & Human Services</td><td>Black     </td><td>Male      </td><td>     0.671</td><td>     1.419</td><td>     0.685</td></tr><tr><td><strong>8</strong></td><td>Public Works & Engineering-PWE</td><td>Black     </td><td>Male      </td><td>    -0.424</td><td>     1.037</td><td>    -0.394</td></tr><tr><td><strong>9</strong></td><td>Health & Human Services</td><td>Black     </td><td>Male      </td><td>    -0.694</td><td>    -0.463</td><td>    -0.688</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115cf400>"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pct_change(2).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sort the DataFrame by one or more columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Female    </td><td>     24960</td><td>       953</td><td>     25913</td></tr><tr><td><strong>1</strong></td><td>Public Works & Engineering-PWE</td><td>Hispanic  </td><td>Male      </td><td>     26104</td><td>      4258</td><td>     30362</td></tr><tr><td><strong>2</strong></td><td>Public Works & Engineering-PWE</td><td>Black     </td><td>Female    </td><td>     26125</td><td>      3247</td><td>     29372</td></tr><tr><td><strong>3</strong></td><td>Houston Airport System (HAS)</td><td>Hispanic  </td><td>Female    </td><td>     26125</td><td>       832</td><td>     26957</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>Black     </td><td>Female    </td><td>     26125</td><td>      2461</td><td>     28586</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115bd940>"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values('salary').head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sort descending"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>    210588</td><td>      3724</td><td>    214312</td></tr><tr><td><strong>1</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>    199596</td><td>       848</td><td>    200444</td></tr><tr><td><strong>2</strong></td><td>Houston Airport System (HAS)</td><td>Black     </td><td>Male      </td><td>    186192</td><td>      1778</td><td>    187970</td></tr><tr><td><strong>3</strong></td><td>Health & Human Services</td><td>Black     </td><td>Male      </td><td>    180416</td><td>      4932</td><td>    185348</td></tr><tr><td><strong>4</strong></td><td>Public Works & Engineering-PWE</td><td>White     </td><td>Female    </td><td>    178331</td><td>      2124</td><td>    180455</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115ce358>"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values('salary', asc=False).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sort by multiple columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Airport System (HAS)</td><td>Asian     </td><td>Female    </td><td>     26125</td><td>      4446</td><td>     30571</td></tr><tr><td><strong>1</strong></td><td>Houston Police Department-HPD</td><td>Asian     </td><td>Male      </td><td>     27914</td><td>      2855</td><td>     30769</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Asian     </td><td>Male      </td><td>     28169</td><td>      2572</td><td>     30741</td></tr><tr><td><strong>3</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     28995</td><td>      2874</td><td>     31869</td></tr><tr><td><strong>4</strong></td><td>Public Works & Engineering-PWE</td><td>Asian     </td><td>Male      </td><td>     30347</td><td>      4938</td><td>     35285</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d1278>"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values(['race', 'salary']).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Randomly sample the DataFrame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     62540</td><td>      2995</td><td>     65535</td></tr><tr><td><strong>1</strong></td><td>Public Works & Engineering-PWE</td><td>White     </td><td>Male      </td><td>     63336</td><td>      1547</td><td>     64883</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     52514</td><td>      1150</td><td>     53664</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115cfe80>"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sample(n=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Randomly sample a fraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>Hispanic  </td><td>Female    </td><td>     60347</td><td>      1200</td><td>     61547</td></tr><tr><td><strong>1</strong></td><td>Public Works & Engineering-PWE</td><td>Black     </td><td>Male      </td><td>     49109</td><td>      3598</td><td>     52707</td></tr><tr><td><strong>2</strong></td><td>Health & Human Services</td><td>Black     </td><td>Female    </td><td>     48984</td><td>      4602</td><td>     53586</td></tr><tr><td><strong>3</strong></td><td>Houston Police Department-HPD</td><td>White     </td><td>Male      </td><td>     55461</td><td>      2813</td><td>     58274</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>Black     </td><td>Female    </td><td>     29286</td><td>      1877</td><td>     31163</td></tr><tr><td><strong>5</strong></td><td>Houston Police Department-HPD</td><td>Asian     </td><td>Male      </td><td>     66614</td><td>      4480</td><td>     71094</td></tr><tr><td><strong>6</strong></td><td>Houston Fire Department (HFD)</td><td>White     </td><td>Male      </td><td>     28024</td><td>      4475</td><td>     32499</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d2080>"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sample(frac=.005)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sample with replacement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>race      </th><th>gender    </th><th>salary    </th><th>bonus     </th><th>total salary</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Parks & Recreation</td><td>Black     </td><td>Female    </td><td>     31075</td><td>      1665</td><td>     32740</td></tr><tr><td><strong>1</strong></td><td>Public Works & Engineering-PWE</td><td>Hispanic  </td><td>Male      </td><td>     67038</td><td>       644</td><td>     67682</td></tr><tr><td><strong>2</strong></td><td>Houston Police Department-HPD</td><td>Black     </td><td>Male      </td><td>     37024</td><td>      1532</td><td>     38556</td></tr><tr><td><strong>3</strong></td><td>Health & Human Services</td><td>Black     </td><td>Female    </td><td>     57433</td><td>      3106</td><td>     60539</td></tr><tr><td><strong>4</strong></td><td>Public Works & Engineering-PWE</td><td>Black     </td><td>Male      </td><td>     53373</td><td>       924</td><td>     54297</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d2c18>"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sample(n=10000, replace=True).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### String-only methods\n",
    "\n",
    "Use the `str` accessor to call methods available just to string columns. Pass the name of the string column as the first parameter for all these methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>         2</td></tr><tr><td><strong>1</strong></td><td>         0</td></tr><tr><td><strong>2</strong></td><td>         2</td></tr><tr><td><strong>3</strong></td><td>         2</td></tr><tr><td><strong>4</strong></td><td>         0</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d5240>"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.str.count('department', 'P').head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>houston police department-hpd</td></tr><tr><td><strong>1</strong></td><td>houston fire department (hfd)</td></tr><tr><td><strong>2</strong></td><td>houston police department-hpd</td></tr><tr><td><strong>3</strong></td><td>public works & engineering-pwe</td></tr><tr><td><strong>4</strong></td><td>houston airport system (has)</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d6080>"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.str.lower('department').head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th></tr></thead><tbody><tr><td><strong>0</strong></td><td>         0</td></tr><tr><td><strong>1</strong></td><td>         0</td></tr><tr><td><strong>2</strong></td><td>         0</td></tr><tr><td><strong>3</strong></td><td>        -1</td></tr><tr><td><strong>4</strong></td><td>         0</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d6940>"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.str.find('department', 'Houston').head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Grouping\n",
    "\n",
    "pandas_cub provides the `value_counts` method for simple frequency counting of unique values and `pivot_table` for grouping and aggregating.\n",
    "\n",
    "The `value_counts` method returns a list of DataFrames, one for each column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfs = df[['department', 'race', 'gender']].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>count     </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Houston Police Department-HPD</td><td>       570</td></tr><tr><td><strong>1</strong></td><td>Houston Fire Department (HFD)</td><td>       365</td></tr><tr><td><strong>2</strong></td><td>Public Works & Engineering-PWE</td><td>       341</td></tr><tr><td><strong>3</strong></td><td>Health & Human Services</td><td>       103</td></tr><tr><td><strong>4</strong></td><td>Houston Airport System (HAS)</td><td>       103</td></tr><tr><td><strong>5</strong></td><td>Parks & Recreation</td><td>        53</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d8470>"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfs[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th><th>count     </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>White     </td><td>       542</td></tr><tr><td><strong>1</strong></td><td>Black     </td><td>       518</td></tr><tr><td><strong>2</strong></td><td>Hispanic  </td><td>       381</td></tr><tr><td><strong>3</strong></td><td>Asian     </td><td>        87</td></tr><tr><td><strong>4</strong></td><td>Native American</td><td>         7</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d85c0>"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfs[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>gender    </th><th>count     </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Male      </td><td>      1135</td></tr><tr><td><strong>1</strong></td><td>Female    </td><td>       400</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d86d8>"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dfs[2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If your DataFrame has one column, a DataFrame and not a list is returned. You can also return the relative frequency by setting the `normalize` parameter to `True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th><th>count     </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>White     </td><td>     0.353</td></tr><tr><td><strong>1</strong></td><td>Black     </td><td>     0.337</td></tr><tr><td><strong>2</strong></td><td>Hispanic  </td><td>     0.248</td></tr><tr><td><strong>3</strong></td><td>Asian     </td><td>     0.057</td></tr><tr><td><strong>4</strong></td><td>Native American</td><td>     0.005</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115dd0f0>"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['race'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `pivot_table` method allows to group by one or two columns and aggregate values from another column. Let's find the average salary for each race and gender. All parameters must be strings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th><th>Female    </th><th>Male      </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Asian     </td><td> 58304.222</td><td> 60622.957</td></tr><tr><td><strong>1</strong></td><td>Black     </td><td> 48133.382</td><td> 51853.000</td></tr><tr><td><strong>2</strong></td><td>Hispanic  </td><td> 44216.960</td><td> 55493.064</td></tr><tr><td><strong>3</strong></td><td>Native American</td><td> 58844.333</td><td> 68850.500</td></tr><tr><td><strong>4</strong></td><td>White     </td><td> 66415.528</td><td> 63439.196</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115dd9e8>"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_table(rows='race', columns='gender', values='salary', aggfunc='mean')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you don't provide `values` or `aggfunc` then by default it will return frequency (a contingency table)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>race      </th><th>Female    </th><th>Male      </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Asian     </td><td>        18</td><td>        69</td></tr><tr><td><strong>1</strong></td><td>Black     </td><td>       207</td><td>       311</td></tr><tr><td><strong>2</strong></td><td>Hispanic  </td><td>       100</td><td>       281</td></tr><tr><td><strong>3</strong></td><td>Native American</td><td>         3</td><td>         4</td></tr><tr><td><strong>4</strong></td><td>White     </td><td>        72</td><td>       470</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115d5f98>"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_table(rows='race', columns='gender')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can group by just a single column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>department</th><th>mean      </th></tr></thead><tbody><tr><td><strong>0</strong></td><td>Health & Human Services</td><td> 51324.981</td></tr><tr><td><strong>1</strong></td><td>Houston Airport System (HAS)</td><td> 53990.369</td></tr><tr><td><strong>2</strong></td><td>Houston Fire Department (HFD)</td><td> 59960.441</td></tr><tr><td><strong>3</strong></td><td>Houston Police Department-HPD</td><td> 60428.746</td></tr><tr><td><strong>4</strong></td><td>Parks & Recreation</td><td> 39426.151</td></tr><tr><td><strong>5</strong></td><td>Public Works & Engineering-PWE</td><td> 50207.806</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115c1940>"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_table(rows='department', values='salary', aggfunc='mean')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table><thead><tr><th></th><th>Health & Human Services</th><th>Houston Airport System (HAS)</th><th>Houston Fire Department (HFD)</th><th>Houston Police Department-HPD</th><th>Parks & Recreation</th><th>Public Works & Engineering-PWE</th></tr></thead><tbody><tr><td><strong>0</strong></td><td> 51324.981</td><td> 53990.369</td><td> 59960.441</td><td> 60428.746</td><td> 39426.151</td><td> 50207.806</td></tr></tbody></table>"
      ],
      "text/plain": [
       "<pandas_cub.DataFrame at 0x1115de748>"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.pivot_table(columns='department', values='salary', aggfunc='mean')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}