Candy Template.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project: cleanning - [candy data]\n",
    "### this notebook is for educational purpose . in this notebook we will do some cleanning\n",
    "\n",
    "## Table of Contents\n",
    "<ul>\n",
    "<li><a href=\"#intro\">Introduction</a></li>\n",
    "<li><a href=\"#wrangling\">wrangling</a></li>\n",
    "  \n",
    "</ul>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='intro'></a>\n",
    "## Introduction\n",
    "\n",
    "### Dataset Description \n",
    "\n",
    "> i will go through candy data 2017 , this data has :\n",
    "- Internal_ID\n",
    "- Q1-Going Out?: Are you actually going trick or treating yourself? \"has values `Yes or No`\"\n",
    "- Q2-Gender: It has four different options, `Femal ,Male ,Other ,I'd rather not say`\n",
    "- Q3-Age: Numerical field\n",
    "- Q4-Country: Text Field, but users have written their own version of the names. Example, for America, there are entries such as USA, us, US, America so we should consider that while cleanning\n",
    "- Q5-State/Province: Text Field, but users have written their own version of the names. Same as the country data.\n",
    "- Q6-Joy Or Despair: All kinds of chocolate bars are the questions with three distinct options to choose from (Joy, Meh, Despair).\n",
    "- Q7-Joy Other: Text Field to write items not included above that give you JOY. Lots of missing values.\n",
    "- Q8-Despair Other: Text Field ti write items not included above that give you DESPAIR. Lots of missing values.\n",
    "- Q9-Other Comments: Text Field. Lots of missing values.\n",
    "- Q10-Dress: Binary field. Missing values present\n",
    "- Unnamed: 113\n",
    "- Q11-Day: Binary Answer Field. Missing values present\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1- import packages and load data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id='wrangling'></a>\n",
    "## Data Wrangling\n",
    "\n",
    "\n",
    "\n",
    "### General Properties"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2- show some general properties for the data "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3- write summary about what you get form the above functions\n",
    "> #### tips : this summary can be about : if there is any missing values , if there a wrong data type , if there a wrong range of values , ...."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4- drop all the useless columns and  rows with too much NaN , \n",
    "> tip : useless columns can be like `Internal ID`and can be also columns with too much NaN \n",
    "- rename columns for easy access"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5 - clean `going out` column \n",
    "> tip : you should choose which best value to put in place of these NaNs ( mode, unknown,other ....etc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6 - clean ` gender` column \n",
    "> tip : you should choose which best value to put in place of these NaNs ( mode, unknown,other ....etc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 7 - clean ` age` column \n",
    "> tip : note the type of age column , and NaN values\n",
    " - try to use `to_numeric` \n",
    " - tip : you should choose which best value to put in place of these NaNs ( mean, median, ....etc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 8 - clean ` country` column\n",
    "> tip : note that country column has country names with different writing ways \n",
    "  - use fuzzywuzzy to deal with this problem\n",
    "  - you should choose which best value to put in place of these NaNs ( mode, unknown,other ....etc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 9 - clean ` area` column\n",
    "> tip : area column has the same problem as country columns , but looking at all it's unique values may give us another way to deal with it rather than try to fix it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 10 - clean ` Q6` columns\n",
    ">- tip : Q6 has all kinds of chocolate bars and questions with three distinct options to choose from (Joy, Meh, Despair).\n",
    "- so all the column that has Q6 is a candy name , so you should go through all these column and you should choose which best value to put in place of these NaNs ( mode, unknown,other ....etc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 11 - clean ` dress` column\n",
    ">tip : you should choose which best value to put in place of these NaNs ( mode, unknown,other ....etc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 12 - clean ` day` column\n",
    ">tip : you should choose which best value to put in place of these NaNs ( mode, unknown,other ....etc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 13 - check\n",
    "> after finish cleaning you should make sure your data is clean and that :\n",
    "- there is no wrong data type\n",
    "- there is no NaN values \n",
    "- all the column are cleaing and ready for the next step `EDA`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 14 - write a summary about all the cleaning steps "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}