Solution_Philipp_Munkes.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "2e99d708",
   "metadata": {},
   "source": [
    "*Data Challenge for MODE 2022 Workshop on Differentiable Programming, Crete\n",
    "Copyright (C) 2022 Giles C Strong*\n",
    "\n",
    "*This program is free software: you can redistribute it and/or modify\n",
    "it under the terms of the GNU Affero General Public License as published\n",
    "by the Free Software Foundation, version 3 only of the License.*\n",
    "\n",
    "*This program is distributed in the hope that it will be useful,\n",
    "but WITHOUT ANY WARRANTY; without even the implied warranty of\n",
    "MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\n",
    "GNU Affero General Public License for more details.*\n",
    "\n",
    "*You should have received a copy of the GNU Affero General Public License\n",
    "along with this program.  If not, see <https://www.gnu.org/licenses/>.*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06bb7f65",
   "metadata": {},
   "source": [
    "# A Voting model to reconstruct walls from muon scattering data\n",
    "\n",
    "This is the reference code to reproduce my solution for the Data Challenge for MODE 2022 Workshop on Differentiable Programming."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3fba77db",
   "metadata": {},
   "source": [
    "This is a list of all packages used in this notebook. Since not all of them are going to be installed in your environment, I recommend running this setup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "75104ffc",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!pip install h5py matplotlib numpy seaborn tqdm torch torchvision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "405caca5",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "%reload_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "d0d16b8e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import h5py\n",
    "from pathlib import Path\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import numpy as np\n",
    "from tqdm import tqdm\n",
    "import math\n",
    "from matplotlib.colors import LogNorm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8107755",
   "metadata": {},
   "source": [
    "## Data loading\n",
    "\n",
    "*This section is taken from the reference notebook.*\n",
    "\n",
    "If you haven't already downloaded the dataset, do so using, e.g. the uncommented command below. You only need to run this once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cfcbe72c",
   "metadata": {},
   "outputs": [],
   "source": [
    "#!mkdir data\n",
    "#!wget -O data/train.h5 https://zenodo.org/record/6866891/files/train.h5\n",
    "#!wget -O data/test.h5 https://zenodo.org/record/6866891/files/test.h5    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "987dab53",
   "metadata": {},
   "source": [
    "Point this to the directory where you have the data stored"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ec1e9e9d",
   "metadata": {},
   "outputs": [],
   "source": [
    "DATA = Path('data')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8015e84e",
   "metadata": {},
   "source": [
    "Data files are in HDf5 format. The training file contains two datasets:\n",
    "- `'x0'`: a set of voxelwise X0 predictions (float32)\n",
    "- `'targs'`: a set of voxelwise classes (int):\n",
    "    - 0 = soil\n",
    "    - 1 = wall\n",
    "    \n",
    "The format of the datasets is a rank-4 array, with dimensions corresponding to (samples, z position, x position, y position).\n",
    "All passive volumes are of the same size: 10x10x10 m, with cubic voxels of size 1x1x1 m, i.e. every passive volume contains 1000 voxels.\n",
    "The arrays are ordered such that zeroth z layer is the bottom layer of the passive volume, and the ninth layer is the top layer.\n",
    "\n",
    "Our job is to convert the X0 values into the target class values, for every voxel.\n",
    "\n",
    "Since we want to evaluate the performance of our methods, we split off a portion of the training data to be used for validating performance, see e.g. https://www.fast.ai/2017/11/13/validation-sets/."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "54f5ed7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "h5 = h5py.File(DATA/'train.h5', 'r')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d40390aa",
   "metadata": {},
   "source": [
    "Let's check the number of samples available:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "df0c03aa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(106129, 10, 10, 10)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "h5['targs'].shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "129034c6",
   "metadata": {},
   "source": [
    "So in total we have over 100,000 samples. We will use most of these for training and set a small set of 1024 samples aside for a test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "06e4af7e",
   "metadata": {},
   "outputs": [],
   "source": [
    "n = 106129\n",
    "n_train = n-1024"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e3631c00",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_x = h5['x0'][:n_train]\n",
    "train_y = h5['targs'][:n_train]\n",
    "valid_x = h5['x0'][n_train:n]\n",
    "valid_y = h5['targs'][n_train:n]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "345a8f73",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((105105, 10, 10, 10), (1024, 10, 10, 10))"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_x.shape, valid_y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "670c09d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "h5.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e6fe46d",
   "metadata": {},
   "source": [
    "## Data viewing\n",
    "\n",
    "Looking at raw 3D arrays of data is a bit uninformative, so in this section we'll plot it out.\n",
    "\n",
    "Fist, we'll look at the targets from a top-down perspective, and plot them in x & y.\n",
    "\n",
    "Below, we plot out 5 examples and flatten them along the z axis (`.any(0)`). The black sections are soil (class=0) and the white sections are the walls (class=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "49a4f81b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASsAAAD8CAYAAAAv6IKXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAU30lEQVR4nO3de5BkZX3G8e+zN10ugiVIYBcCKF6IF8RlISqIIrqLllumTAUwgpS4UgEvyR+RxKpQlJVUqHjDEt2sgEqpEAWMK264aIKSKLCoXHa56LIgOywIRAS5lDDTT/44Z00zNdPdw5yenrfn+VintrvPOb9+HXZ++77v+Z3zyjYREbPdvEE3ICKiF0lWEVGEJKuIKEKSVUQUIckqIoqQZBURRUiyiohGSTpf0gOSNk6yX5I+J2mzpJslHdxL3CSriGjaV4AVHfavBA6ot9XAF3sJmmQVEY2y/SPgNx0OWQVc4Mq1wK6S9uwWd0FTDZz0CxYtSYl8APDktmsaj7l4r8Mbjwn9aWs/Ldxtf03n/Kcf2tLz7+mi3V/0Qaoe0XZrba+dwtctAba2vR+pP7uv00l9T1YRUYDWWM+H1olpKslpvIkSa9dkmWQVEeDWTH7bCLB32/ulwLZuJ2XOKiKg1ep9m751wAn1VcHDgEdsdxwCQnpWEQG4wZ6VpAuBI4HdJI0AZwALq+/xGmA9cAywGXgCOKmXuElWEQFjo42Fsn1cl/0GTp1q3CSriJjSBPugJFlFxExPsD8rSVYR0dTEeV8lWUVEoxPs/dI1WUl6GVV5/BKqwq1twDrbt/W5bRExUwroWXWss5L0MeAiqorT64EN9esLJZ3e4bzVkm6QdEOr9XiT7Y2Ifhh7uvdtQLr1rN4P/IntZ7RQ0qeBTcA/T3RSezl+7g2MKMAQDANbwF7Ar8Z9vme9LyKGQQHDwG7J6qPADyT9kv+/S3of4MXAaX1sV0TMpNJ7VrYvl/QSYDnVBLuobkLcYHv2V5FFRG+GoGeFq2ua185AWyJiQNwa3MR5r1JnFRHD0bOKiDmg9DmriJgjciNzRBQhPauIKELmrCLKNOdWzWnw4Xv9kmQVEelZRUQZSqjxTrKKiPSsIqIQuRoYEUVIzyoiipCrgRFRhAwDI6IIGQZGRBGSrCKiCAUMAzuubtOJpJM67MvqNhElGRvtfRuQZ52sgDMn22F7re1ltpfNm7fjNL4iImZEq9X7NiAdh4GSbp5sF7BH882JiIEoYBjYbc5qD+BtwMPjPhfw4760KCJm3hBMsF8G7GT7xvE7JF3djwZFxACUnqxsv7/DvuObb05EDIRn/8LpKV2ICBjN7TYRUYICJtinU7oQEcOi4dIFSSsk3SFps6TTJ9i/i6TvSrpJ0qZOdZvbJVlFRDVn1evWhaT5wDnASuBA4DhJB4477FTgVtuvBo4EPiVpUae4GQZGRNNXA5cDm21vAZB0EbAKuLXtGAM7SxKwE/AboOPEWXpWETGlYWD77XT1tnpctCXA1rb3I/Vn7T4PvBzYBtwCfMTuPHGWnlVE4LHeF4ywvRZY2+EQTXTauPdvA24E3gy8CLhK0jW2H50saHpWEdH0BPsIsHfb+6VUPah2JwGXurIZuAt4WaegSVYRUZUu9Lp1twE4QNJ+9aT5scC6ccfcAxwFIGkP4KXAlk5BMwyMCGg1V8Fue1TSacAVwHzgfNubJJ1S718DfAL4iqRbqIaNH7P9UKe4SVYR0fi9gbbXA+vHfbam7fU24K1TiZlkFREwhQn2QUmyiojyn7oQEXNEg3NW/ZJkFRHDcSOzpJdJOkrSTuM+X9G/ZkXEjGq5921AOiYrSR8GvgN8CNgoaVXb7n/qcF5Wt4koiFutnrdB6TYM/ADwWtuPSdoXuFjSvrbPZuKSeuCZ5fgLFi2Z/YPhiLluCK4Gzrf9GIDtuyUdSZWw/pgOySoiClPABHu3Oav7JR20/U2duN4B7Aa8so/tioiZVPq6gcAJjHvGjO1R4ARJ/9q3VkXEzCqgZ9VtdZuRDvv+p/nmRMRAFFC6kDqriCi/ZxURc4NHy78aGBFzQXpWEVGEzFlFRBHSs4p+enLbNYNuQgwJJ1lFRBEywR4RRUjPKiKKkGQVESWwk6wiogTpWUVEEZKsIqIEHk1RaESUYPbnqu7JStJywLY3SDoQWAHcXq+4GhFDoPiiUElnACuBBZKuAg4FrgZOl/Qa2/84yXmrgdUAmr8L8+bt2GijI6JhpScr4N3AQcBzgPuBpbYflfQvwHXAhMkqC0ZEFGYIhoGjtseAJyTdaftRANtPSirg/15E9KL4YSDwlKQdbD8BvHb7h5J2oYhcHBG98Gj5yeoI278HsJ/xwJuFwIl9a1VEzKwCuh7dFoz4/SSfPwQ81JcWRcSMK+DZe6mzigjK71lFxNxQQs+q24rMETEHeLT3rReSVki6Q9JmSadPcsyRkm6UtEnSD7vFTM8qIhrtWUmaD5wDHA2MABskrbN9a9sxuwJfAFbYvkfSC7vFTc8qInCr960Hy4HNtrfYfgq4CFg17pjjgUtt3wNg+4FuQdOzKtjivQ4fdBNiivr132z0qXunF8Dq+dD22+lqa+u7VrZbAmxtez9Cdateu5cACyVdDewMnG37gk7fm2QVEVMaBrbfTjeJiTLf+KrTBVSF5kcBi4GfSLrW9i8mC5pkFRG41XvPqgcjwN5t75cC2yY45iHbjwOPS/oR8Gpg0mSVOauIoDWmnrcebAAOkLSfpEXAscC6ccd8Bzhc0gJJO1ANE2/rFDQ9q4ho9Gqg7VFJpwFXAPOB821vknRKvX+N7dskXQ7cTFWSeq7tjZ3iqt+rWuQRMRH9N/rUvdMax2095Kief0/33vCDRseMvUrPKiIoYCWuJKuIaHyCvS+SrCKi14nzgUqyiogielZTLl2Q1LHKNCLKY6vnbVC6rW4zvjZCwJvqmxCx/c5JzsvqNhEFKeERMd2GgUuBW4FzqcrlBSwDPtXppKxuE1GW1gB7TL3qNgxcBvwU+DjwiO2rgSdt/9B21+fPREQZih8G1otEfEbSt+o/f93tnIgoz9BcDbQ9Avy5pLcDj/a3SREx00q4GjilXpLt7wHf61NbImJASpizypAuIgY6F9WrJKuIyL2BEVGGDAMjogitYZtgj4jhlJ5VRBQhE+wRUYT0rCKiCAVcDEyyiggYa83+ha6SrCKCAp4Qk2QVEeAJF1GeXZKsIoJWAZNWSVYRQSs9q4goQYaBEVGEsWFLVpLeACwHNtq+sj9NioiZVsLVwI7FFZKub3v9AeDzwM7AGZJO73Deakk3SLqh1Xq8scZGRH+0prANSrdKsIVtr1cDR9s+E3gr8J7JTrK91vYy28uyDFfE7GfU8zYo3YaB8yQ9nyqpyfaDALYflzTa99ZFxIwo4AkxXZPVLlRLcQmwpD+yfb+knerPImIIFF+6YHvfSXa1gHc13pqIGIixQTegB8+qdMH2E8BdDbclIgakpcJ7VhExNxRwt02SVUQMQZ1VRMwNLfW+9ULSCkl3SNrcpSbzEEljkt7dLWZ6VhHR6O02kuYD5wBHAyPABknrbN86wXFnAVf0Ejc9q4houme1HNhse4vtp4CLgFUTHPch4BLggV6CpmdVsCe3XTPoJkzJ4r0OH3QTYhJTmbOStJrqjpbt1tpe2/Z+CbC17f0IcOi4GEuoyp/eDBzSy/cmWUXElK4G1olpbYdDJup/jf+KzwIfsz2mHssmkqwiounbbUaAvdveLwW2jTtmGXBRnah2A46RNGr73ycLmmQVEU2XLmwADpC0H3AvcCxwfPsBtvfb/lrSV4DLOiUqSLKKCGCswZ6V7VFJp1Fd5ZsPnG97k6RT6v1rnk3cJKuIaLwo1PZ6YP24zyZMUrbf10vMJKuIKKKCPckqInJvYESUYRgevhcRc0AJw8BuC0YcKul59evFks6U9F1JZ0naZWaaGBH9NjaFbVC63Rt4PvBE/fpsqsccn1V/9uXJTsrqNhFlafqpC/3QdcEI29sXhlhm++D69X9LunGyk9rL8RcsWlLC3F3EnFb8MBDYKOmk+vVNkpYBSHoJ8HRfWxYRM8ZT2AalW7I6GXijpDuBA4GfSNoCfKneFxFDoIV73gal2+o2jwDvk7QzsH99/IjtX89E4yJiZgzN6ja2fwfc1Oe2RMSAlDBnlTqriEhRaESUYZBzUb1KsoqIAlJVklVEkDmriCjEWAF9qySrgvVrtZjSVs2J6UvPKiKKkAn2iCjC7E9VSVYRQYaBEVGITLBHRBEyZxURRZj9qSrJKiJIzyoiClHCBHu3BSM+LGnvmWpMRAyGp/C/Qen2pNBPANdJukbSX0navZegWTAioixjuOdtULolqy3AUqqk9VrgVkmXSzqxfnrohGyvtb3M9rJ583ZssLkR0Q+tKWyD0m3OyrZbwJXAlZIWAiuB44BPAj31tCJidmu5/An2Zzw/0PbTwDpgnaTFfWtVRMyo2Z+quierv5hsh+0nG25LRAxI8aULtn8xUw2JiMEZ5FW+XqXOKiIYTbKKiBKU0LPqVroQEXNA06ULklZIukPSZkmnT7D/PZJurrcfS3p1t5jpWUUEbrB0QdJ84BzgaGAE2CBpne1b2w67C3ij7YclrQTWAod2iptkFRFNXw1cDmy2vQVA0kXAKuAPycr2j9uOv5aq+LyjDAMjYkq327TfTldvq8eFWwJsbXs/Un82mfcD/9GtjelZRcSUela211IN2yYz0WL0E36BpDdRJas3dPveJKuIaHTOiqon1f60lqXAtvEHSXoVcC6w0vb/dguaYWBENH01cANwgKT9JC0CjqW6Te8PJO0DXAq8t9fi8/SsIqLROivbo5JOA64A5gPn294k6ZR6/xrgH4AXAF+QBDBqe1mnuElWEdH4vYG21wPrx322pu31ycDJU4mZZBURjHn2P9g4ySoiirjdJskqIobi4XsRMQfM/lTVJVm1XXbcZvv7ko4HXgfcBqytnxwaEYUr/uF7wJfrY3aQdCKwE1VtxFFU9/+cONFJdfn9agDN34UsGhExuw1Dsnql7VdJWgDcC+xle0zS14CbJjupvRx/waIls/+nEDHHDcPVwHn1UHBHYAdgF+A3wHOAhX1uW0TMkGG4GngecDtVFerHgW9J2gIcBlzU57ZFxAxp+N7Avui2YMRnJP1b/XqbpAuAtwBfsn39TDQwIvpvGOassL2t7fVvgYv72aCImHnF96wiYm4YG+jC8L1JsoqIVLBHRBmG4WpgRMwB6VlFRBHSs4qIIqRnFRFFGIbbbSJiDsgwMCKK4PSsIqIEQ3G7TUQMv9xuExFFSM8qIoow1sqcVUQUIFcDI6IIQzFnJelFwLuAvYFR4JfAhbYf6XPbImKGlDBnNa/TTkkfBtYAzwUOARZTJa2fSDqyw3mrJd0g6YZW6/HmWhsRfWG7521Q1OnLJd0CHFSvaLMDsN72kZL2Ab5j+zXdviCr25TnyW3X9CXu4r0O70vcgNGn7tV0zn/+Ti/u+ff04cc2T+u7nq1e5qwWAGNUK9rsDGD7HklZ3SZiSJQwDOyWrM4FNki6FjgCOAtA0u5US3JFxBAofoLd9tmSvg+8HPi07dvrzx+kSl4RMQSG4hExtjcBm2agLRExIKmziogiDEXPKiKGX6uAR8R0rLOKiLmh6TorSSsk3SFps6TTJ9gvSZ+r998s6eBuMZOsIqLRZCVpPnAOsBI4EDhO0oHjDlsJHFBvq4EvdoubZBUReApbD5YDm21vsf0UcBGwatwxq4ALXLkW2FXSnp0bOYWM2u8NWD3X45bU1tLiltTWfsZtol3ADW3b6nH73w2c2/b+vcDnxx1zGfCGtvc/AJZ1+t7Z1rNanbhFtbW0uCW1tZ9xp8X2WtvL2ra14w6Z6Hac8Z2yXo55htmWrCKifCNUDzzYbimw7Vkc8wxJVhHRtA3AAZL2k7QIOBZYN+6YdcAJ9VXBw4BHbN/XKehsq7Ma352ci3FLamtpcUtqaz/j9pXtUUmnAVcA84HzbW+SdEq9fw2wHjgG2Aw8AZzULW7HR8RERMwWGQZGRBGSrCKiCLMmWXUrz3+WMc+X9ICkjU3Eq2PuLem/JN0maZOkjzQU97mSrpd0Ux33zCbi1rHnS/q5pMsajHm3pFsk3Sjphgbj7irpYkm31z/jP20g5kvrdm7fHpX00Qbi/nX932qjpAslPXe6Meu4H6ljbmqinUNj0AVm9ZzZfOBOYH9gEXATcGADcY8ADgY2NtjWPYGD69c7A79oqK0CdqpfLwSuAw5rqM1/A3wDuKzBn8PdwG59+LvwVeDk+vUiYNc+/F27H/jjacZZAtwFLK7ffxN4XwPtewWwEdiB6gLY94EDmv45l7jNlp5VL+X5U2b7RzT8RFPb99n+Wf36d8BtVH9xpxvXth+r3y6st2lf/ZC0FHg71VNfZzVJz6P6B+Y8ANtP2f5tw19zFHCn7V81EGsBsFjSAqrk0rFOqEcvB661/YTtUeCHVKtLzXmzJVktAba2vR+hgQTQb5L2BV5D1QtqIt58STcCDwBX2W4i7meBvwWafgaIgSsl/VRSU5XW+wMPAl+uh63nStqxodjbHQtcON0gtu8FPgncA9xHVSd05XTjUvWqjpD0gnqRlmN4ZvHknDVbktWUS+8HTdJOwCXAR20/2kRM22O2D6Kq5l0u6RXTiSfpHcADtn/aRPvGeb3tg6nunj9VUhOPuV5ANWz/oquVkx4HGpm/BKgLFN8JfKuBWM+n6v3vB+wF7CjpL6cb1/ZtVGsdXAVcTjUlMjrduMNgtiSrKZfeD1K9ss8lwNdtX9p0/HroczWwYpqhXg+8U9LdVEPrN0v62jRjAmB7W/3nA8C3qYby0zUCjLT1KC+mSl5NWQn8zPavG4j1FuAu2w/afhq4FHhdA3GxfZ7tg20fQTWN8csm4pZutiSrXsrzZwVJoppTuc32pxuMu7ukXevXi6l+GW6fTkzbf2d7qe19qX6m/2l72v/6S9pR0s7bXwNvpRq+TIvt+4Gtkl5af3QUcOt047Y5jgaGgLV7gMMk7VD/nTiKav5y2iS9sP5zH+DPaK7NRZsVt9t4kvL86caVdCFwJLCbpBHgDNvnTTPs66keeXFLPb8E8Pe2108z7p7AV+sHl80Dvmm7sVKDhu0BfLv6HWUB8A3blzcU+0PA1+t/tLbQw20Yvajnf44GPthEPNvXSboY+BnVMO3nNHd7zCWSXgA8DZxq++GG4hYtt9tERBFmyzAwIqKjJKuIKEKSVUQUIckqIoqQZBURRUiyiogiJFlFRBH+DwzPvTXh8Xr0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASsAAAD8CAYAAAAv6IKXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAU+UlEQVR4nO3de7BdZX3G8e+ThGggFBxRCwkIarzgDTEGqoIoook6MnbsCFhRRo1MxUv7R6V1pgzjtDOMWi8jmh4RL6NCFbFGTAG1orSKBpVLQkRjQDgERCqCXMZwzn76x1qxm+05e++Ts/bZ+z3n+TBrsvdea/32S5Lzy/u+67fWK9tERIy6RcNuQEREP5KsIqIISVYRUYQkq4goQpJVRBQhySoiipBkFRGNknS+pDslbZlmvyR9VNJ2SddJOrKfuElWEdG0zwBru+xfB6yqt/XAJ/oJmmQVEY2y/T3gt10OORH4nCtXAftLOrBX3CVNNXDaL1i6IiXyEQM2ses2zeb8h+7a0ffP6dLHPPFtVD2i3cZsj83g61YAt7a9H68/u73bSQNPVhFRgNZk34fWiWkmyanTVIm1Z7JMsooIcGsuv20cOLjt/UpgZ6+TMmcVEdBq9b/N3kbg1Pqq4NHAPba7DgEhPauIANxgz0rSBcBxwAGSxoGzgL2q7/EGYBPwCmA78ABwWj9xk6wiAiYnGgtl++Qe+w28faZxk6wiYkYT7MOSZBURcz3BvkeSrCKiqYnzgUqyiohGJ9gHpWeykvRUqvL4FVSFWzuBjba3DbhtETFXCuhZda2zkvQe4EKqitMfAZvr1xdIOrPLeeslXS3p6lbr/ibbGxGDMPlQ/9uQqNvqNpJ+Djzd9kMdny8Fttpe1esLcm9gxODN9t7AP2z7Tt8/p4942otn9V17qtcwsAUcBPyq4/MD630RMR8UMAzslazeDXxb0i/4/7ukDwGeBJwxwHZFxFwqfYLd9qWSngysoZpgF9VNiJttj34VWUT0Zx70rHB1TfOqOWhLRAyJW8ObOO9X6qwiYn70rCJiASh9zioiFojcyBwRRUjPKiKKkDmriMF6cOeVA4m77KBjBhJ3ZDX48L1BSbKKiPSsIqIMJdR4J1lFRHpWEVGIXA2MiCKkZxURRcjVwIgoQoaBEVGEDAMjoghJVhFRhAKGgV1Xt+lG0mld9mV1m4iSTE70vw3JHicr4Ozpdtges73a9upFi/aZxVdExJxotfrfhqTrMFDSddPtAh7XfHMiYigKGAb2mrN6HPBy4O6OzwV8fyAtioi5Nw8m2C8Bltu+pnOHpCsG0aCIGILSk5XtN3fZd0rzzYmIoeiyMvuoSOlCRMBEbreJiBIUMME+m9KFiJgvGi5dkLRW0o2Stks6c4r9+0n6uqRrJW3tVre5W5JVRFRzVv1uPUhaDJwLrAMOB06WdHjHYW8HbrD9bOA44IOSlnaLm2FgRDR9NXANsN32DgBJFwInAje0HWNgX0kClgO/BbpOnCVZdRjUaikRI20GyUrSemB920djtsfa3q8Abm17Pw4c1RHmY8BGYCewL/A6u/vEWZJVRODJ/heMqBPTWJdDNNVpHe9fDlwDvAR4IvBNSVfavne6oJmzioimJ9jHgYPb3q+k6kG1Ow242JXtwE3AU7sFTbKKiKp0od+tt83AKkmH1ZPmJ1EN+drdAhwPIOlxwFOAHd2CZhgYEdBqroLd9oSkM4DLgMXA+ba3Sjq93r8BeB/wGUnXUw0b32P7rm5xk6wiovF7A21vAjZ1fLah7fVO4GUziZlkFREwgwn2YUmyiojyn7oQEQtEg3NWg5JkFRHz40ZmSU+VdLyk5R2frx1csyJiTrXc/zYkXZOVpHcCXwPeAWyRdGLb7n/pcl5Wt4koiFutvrdh6TUMfCvwXNv3SToUuEjSobY/wtQl9cDDy/GXLF0x+oPhiIVuHlwNXGz7PgDbN0s6jiphPZ4uySoiClPABHuvOas7JB2x+02duF4FHAA8c4Dtioi5VPq6gcCpdDxjxvYEcKqkfxtYqyJibhXQs+q1us14l33/03xzImIoCihdSJ1VRJTfs4qIhcET5V8NjIiFID2riChC5qwiogjpWQXAsoOOGXYTIrpyklVEFCET7BFRhPSsIqIISVYRUQI7ySoiSpCeVUQUIckqIkrgiRSFRkQJRj9X9U5WktYAtr1Z0uHAWuBn9YqrETEPFF8UKuksYB2wRNI3gaOAK4AzJT3H9j9Pc956YD2AFu/HokX7NNroiGhY6ckKeC1wBPAI4A5gpe17Jb0f+CEwZbLKghERhZkHw8AJ25PAA5J+afteANsPSirgfy8i+lH8MBDYJWlv2w8Az939oaT9KCIXR0Q/PFF+sjrW9h8A7Ic98GYv4I0Da1VEzK0Cuh69Foz4wzSf3wXcNZAWRcScK+DZe6mzigjK71lFxMJQQs+q14rMEbEAeKL/rR+S1kq6UdJ2SWdOc8xxkq6RtFXSd3vFTM8qIhrtWUlaDJwLnACMA5slbbR9Q9sx+wMfB9bavkXSY3vFTc8qInCr/60Pa4DttnfY3gVcCJzYccwpwMW2bwGwfWevoOlZFezBnVcOJG4WuBicQf2ZzZrV96Htt9PVxuq7VnZbAdza9n6c6la9dk8G9pJ0BbAv8BHbn+v2vUlWETGjYWD77XTTmCrzdVadLqEqND8eWAb8QNJVtn8+XdAkq4jArf57Vn0YBw5ue78S2DnFMXfZvh+4X9L3gGcD0yarzFlFBK1J9b31YTOwStJhkpYCJwEbO475GnCMpCWS9qYaJm7rFjQ9q4ho9Gqg7QlJZwCXAYuB821vlXR6vX+D7W2SLgWuoypJPc/2lm5xk6wioulhIPXDOTd1fLah4/37gff3GzPJKiIoYCWuJKuIaL5nNQhJVhHR78T5UCVZRUQRPasZly5I6lplGhHlsdX3Niy9VrfprI0Q8OL6JkRsv3qa87K6TURBSnhETK9h4ErgBuA8qnJ5AauBD3Y7KavbRJSlNcQeU796DQNXAz8G3gvcY/sK4EHb37Xd8/kzEVGG4oeB9SIRH5L05frXX/c6JyLKM2+uBtoeB/5K0iuBewfbpIiYayVcDZxRL8n2N4BvDKgtETEkJcxZZUgXEUOdi+pXklVE5N7AiChDhoERUYTWfJtgj4j5KT2rAmVll1iIMsEeEUVIzyoiilDAxcAkq4iAydboL3SVZBURFPCEmCSriABPuYjyaEmyighaBUxaJVlFBK30rCKiBBkGRkQRJudbspL0QmANsMX25YNpUkTMtRKuBnYtrpD0o7bXbwU+BuwLnCXpzC7nrZd0taSrW637G2tsRAxGawbbsPSqBNur7fV64ATbZwMvA14/3Um2x2yvtr06y3BFjD6jvrdh6TUMXCTpUVRJTbZ/A2D7fkkTA29dRMyJAp4Q0zNZ7Ue1FJcAS/pz23dIWl5/FhHzQPGlC7YPnWZXC3hN462JiKGYHHYD+rBHpQu2HwBuargtETEkLRXes4qIhaGAu22SrCJiHtRZRcTC0FL/Wz8krZV0o6TtPWoynydpUtJre8VMzyoiGr3dRtJi4FzgBGAc2Cxpo+0bpjjuHOCyfuKmZxURTfes1gDbbe+wvQu4EDhxiuPeAXwFuLOfoOlZxZ94cOeVw25CzLGZzFlJWk91R8tuY7bH2t6vAG5tez8OHNURYwVV+dNLgOf1871JVhExo6uBdWIa63LIVP2vzq/4MPAe25Pqs2wiySoimr7dZhw4uO39SmBnxzGrgQvrRHUA8ApJE7b/Y7qgSVYR0XTpwmZglaTDgNuAk4BT2g+wfdju15I+A1zSLVFBklVEAJMN9qxsT0g6g+oq32LgfNtbJZ1e79+wJ3GTrCKi8aJQ25uATR2fTZmkbL+pn5hJVhFRRAV7klVE5N7AiCjDfHj4XkQsACUMA3stGHGUpD+rXy+TdLakr0s6R9J+c9PEiBi0yRlsw9Lr3sDzgQfq1x+heszxOfVnn57upKxuE1GWpp+6MAg9F4ywvXthiNW2j6xf/7eka6Y7qb0cf8nSFSXM3UUsaMUPA4Etkk6rX18raTWApCcDDw20ZRExZzyDbVh6Jau3AC+S9EvgcOAHknYAn6z3RcQ80MJ9b8PSa3Wbe4A3SdoXeEJ9/LjtX89F4yJibsyb1W1s/x64dsBtiYghKWHOKnVWEZGi0IgowzDnovqVZBURBaSqJKuIIHNWEVGIyQL6VklWBVt20DHDbkKMiIldt83q/PSsIqIImWCPiCKMfqpKsooIMgyMiEJkgj0iipA5q4gowuinqiSriCA9q4goRAkT7L0WjHinpIPnqjERMRyewX/D0utJoe8DfijpSkl/I+kx/QTNghERZZnEfW/D0itZ7QBWUiWt5wI3SLpU0hvrp4dOyfaY7dW2Vy9atE+DzY2IQWjNYBuWXnNWtt0CLgcul7QXsA44GfgA0FdPKyJGW8vlT7A/7PmBth8CNgIbJS0bWKsiYk6NfqrqnaxeN90O2w823JaIGJLiSxds/3yuGhIRwzPMq3z9Sp1VRDCRZBURJSihZ9WrdCEiFoCmSxckrZV0o6Ttks6cYv/rJV1Xb9+X9OxeMdOzigjcYOmCpMXAucAJwDiwWdJG2ze0HXYT8CLbd0taB4wBR3WLm2QVEU1fDVwDbLe9A0DShcCJwB+Tle3vtx1/FVXxeVcZBkbEjG63ab+drt7Wd4RbAdza9n68/mw6bwb+s1cb07OKiBn1rGyPUQ3bpjPVYvRTfoGkF1Mlqxf2+t4kq4hodM6KqifV/rSWlcDOzoMkPQs4D1hn+397Bc0wMCKavhq4GVgl6TBJS4GTqG7T+yNJhwAXA2/ot/g8PauIaLTOyvaEpDOAy4DFwPm2t0o6vd6/Afgn4NHAxyUBTNhe3S2uGu7+/YklS1eMfrVZROEmdt021TxR31568Mv7/jn91q2Xzeq79lR6VhHBpEf/wcZJVhFRxO02SVYRMS8evhcRC8Dop6oeyartsuNO29+SdArwfGAbMFY/OTQiClf8w/eAT9fH7C3pjcByqtqI46nu/3njVCfV5ffrAbR4P7JoRMRomw/J6pm2nyVpCXAbcJDtSUmfB66d7qT2cvyULkSMvvlwNXBRPRTcB9gb2A/4LfAIYK8Bty0i5sh8uBr4KeBnVFWo7wW+LGkHcDRw4YDbFhFzZNDF4U3otWDEhyT9e/16p6TPAS8FPmn7R3PRwIgYvPkwZ4XtnW2vfwdcNMgGRcTcK75nFRELw+RQF4bvT5JVRKSCPSLKMB+uBkbEApCeVUQUIT2riChCelYRUYT5cLtNRCwAGQZGRBGcnlVElGBe3G4TEfNfbreJiCKkZxURRZhsZc4qIgqQq4ERUYR5MWcl6YnAa4CDgQngF8AFtu8ZcNsiYo6UMGe1qNtOSe8ENgCPBJ4HLKNKWj+QdFyX89ZLulrS1a3W/c21NiIGwnbf27Co25dLuh44ol7RZm9gk+3jJB0CfM32c3p9QVa3iRi8iV23aTbnP2r5k/r+Ob37vu2z+q491c+c1RJgkmpFm30BbN8iKavbRMwTJQwDeyWr84DNkq4CjgXOAZD0GKoluSJiHihhgr3rMBBA0tOBpwFbbP9spl+QYWDE4M12GLh878P6/jm974GbRnMYaHsrsHUO2hIRQ5I6q4goQh6+FxFFaBXwiJiudVYRsTA0XWclaa2kGyVtl3TmFPsl6aP1/uskHdkrZpJVRDSarCQtBs4F1gGHAydLOrzjsHXAqnpbD3yiV9wkq4jAM9j6sAbYbnuH7V3AhcCJHcecCHzOlauA/SUd2L2RM8iog96A9Qs9bkltLS1uSW0dZNwm2gVc3bat79j/WuC8tvdvAD7WccwlwAvb3n8bWN3te0etZ7U+cYtqa2lxS2rrIOPOiu0x26vbtrGOQ6aqw+rslPVzzMOMWrKKiPKNUz3wYLeVwM49OOZhkqwiommbgVWSDpO0FDgJ2NhxzEbg1Pqq4NHAPbZv7xZ01OqsOruTCzFuSW0tLW5JbR1k3IGyPSHpDOAyYDFwvu2tkk6v928ANgGvALYDDwCn9Yrb897AiIhRkGFgRBQhySoiijAyyapXef4exjxf0p2StjQRr455sKTvSNomaaukdzUU95GSfiTp2jru2U3ErWMvlvRTSZc0GPNmSddLukbS1Q3G3V/SRZJ+Vv8e/0UDMZ9St3P3dq+kdzcQ92/rP6stki6Q9MjZxqzjvquOubWJds4bwy4wq+fMFgO/BJ4ALAWuBQ5vIO6xwJFUz+Jqqq0HAkfWr/cFft5QWwUsr1/vBfwQOLqhNv8d8EXgkgZ/H24GDhjA34XPAm+pXy8F9h/A37U7gMfPMs4K4CZgWf3+S8CbGmjfM4AtwN5UF8C+Baxq+ve5xG1Uelb9lOfPmO3v0fATTW3fbvsn9evfA9uo/uLONq5t31e/3aveZn31Q9JK4JVUT30daZL+jOofmE8B2N5l+3cNf83xwC9t/6qBWEuAZZKWUCWXrnVCfXoacJXtB2xPAN+lWl1qwRuVZLUCuLXt/TgNJIBBk3Qo8ByqXlAT8RZLuga4E/im7Sbifhj4e6DpZ4AYuFzSjyU1VWn9BOA3wKfrYet5kvZpKPZuJwEXzDaI7duADwC3ALdT1QldPtu4VL2qYyU9ul6k5RU8vHhywRqVZDXj0vthk7Qc+Arwbtv3NhHT9qTtI6iqeddIesZs4kl6FXCn7R830b4OL7B9JNXd82+XdGwDMZdQDds/4WrlpPuBRuYvAeoCxVcDX24g1qOoev+HAQcB+0j669nGtb2Naq2DbwKXUk2JTMw27nwwKslqxqX3w1Sv7PMV4Au2L246fj30uQJYO8tQLwBeLelmqqH1SyR9fpYxAbC9s/71TuCrVEP52RoHxtt6lBdRJa+mrAN+YvvXDcR6KXCT7d/Yfgi4GHh+A3Gx/SnbR9o+lmoa4xdNxC3dqCSrfsrzR4IkUc2pbLP9rw3GfYyk/evXy6h+GGa8QEc72/9ge6XtQ6l+T//L9qz/9Ze0j6R9d78GXkY1fJkV23cAt0p6Sv3R8cANs43b5mQaGALWbgGOlrR3/XfieKr5y1mT9Nj610OAv6S5NhdtJG638TTl+bONK+kC4DjgAEnjwFm2PzXLsC+geuTF9fX8EsA/2t40y7gHAp+tH1y2CPiS7cZKDRr2OOCr1c8oS4Av2r60odjvAL5Q/6O1gz5uw+hHPf9zAvC2JuLZ/qGki4CfUA3Tfkpzt8d8RdKjgYeAt9u+u6G4RcvtNhFRhFEZBkZEdJVkFRFFSLKKiCIkWUVEEZKsIqIISVYRUYQkq4gowv8BERGuCX5B0RwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASsAAAD8CAYAAAAv6IKXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAU+ElEQVR4nO3de5BcZZ3G8e+TmwbCgiXKQgISNV7wjjGyXhBENEHLlFtuCbqClBqpFS+7f6zsWrWWZe1WUep6KdHsiHgplaiAa8QsoK4ouwokKpeEiMagZIiIrApyKcNMP/vHOXGb2ZnuHub0dL8zz4c6le4+p3/nrTDzy/u+53fOK9tERAy7BYNuQEREL5KsIqIISVYRUYQkq4goQpJVRBQhySoiipBkFRGNknSBpDskbZ9ivyR9VNIuSTdIOraXuElWEdG0zwBrO+xfB6yqtw3AJ3oJmmQVEY2y/T3gtx0OWQ98zpWrgUMkHd4t7qKmGjjlCZYsT4l8RJ+N7btNM/n+A3fu7vn3dMmjHvcWqh7RfiO2R6ZxuuXAnrb3o/Vnv+r0pb4nq4goQGu850PrxDSd5DTRZIm1a7JMsooIcGs2zzYKHNn2fgWwt9uXMmcVEdBq9b7N3Gbg9Pqq4HHAXbY7DgEhPauIANxgz0rShcAJwKGSRoH3AIur83gjsAU4BdgF3Aec2UvcJKuIgPGxxkLZPq3LfgNvnW7cJKuImNYE+6AkWUXEbE+wPyRJVhHR1MR5XyVZRUSjE+z90jVZSXoSVXn8cqrCrb3AZts7+9y2iJgtBfSsOtZZSXoXsImq4vRaYGv9+kJJ53T43gZJ2yRta7XubbK9EdEP4w/0vg2IOq1uI+mnwFNsPzDh8yXADturup0g9wZG9N9M7w38487v9Px7+rAnnzijcz1U3YaBLeAI4JcTPj+83hcRc0EBw8BuyeqdwLcl/Yz/u0v6KODxwNl9bFdEzKbSJ9htXybpCcAaqgl2Ud2EuNX28FeRRURv5kDPClfXNK+ehbZExIC4NbiJ816lzioi5kbPKiLmgdLnrCJinsiNzBFRhPSsIqIImbOKiCI0+PC9fkmyioj0rCKiDCXUeCdZRUR6VhFRiFwNjIgipGcVEUXI1cCIKEKGgRFRhAwDI6IISVYRUYQChoEdV7fpRNKZHfZldZuIkoyP9b4NyENOVsB7p9phe8T2aturFyw4cAaniIhZ0Wr1vg1Ix2GgpBum2gUc1nxzImIgChgGdpuzOgx4GfC7CZ8L+H5fWhQRs28OTLBfCiyzfd3EHZKu7EeDImIASk9Wtt/YYd9rm29ORAxEh5XZh0VKFyICxnK7TUSUoIAJ9pmULkTEXNFw6YKktZJulrRL0jmT7D9Y0tclXS9pR6e6zf2SrCKimrPqdetC0kLgPGAdcAxwmqRjJhz2VuAm288ATgA+KGlJp7gZBkZE01cD1wC7bO8GkLQJWA/c1HaMgYMkCVgG/BboOHGWnlVETGsY2H47Xb1tmBBtObCn7f1o/Vm7jwFPBvYCNwLvsDtPnKVnFRF4vPcFI2yPACMdDtFkX5vw/mXAdcCLgccB35R0le27pwqanlVEND3BPgoc2fZ+BVUPqt2ZwCWu7AJuAZ7UKWiSVURUpQu9bt1tBVZJWllPmp8KbJ5wzK3ASQCSDgOeCOzuFDTDwIiAVnMV7LbHJJ0NXA4sBC6wvUPSWfX+jcD7gM9IupFq2Pgu23d2iptkFRGN3xtoewuwZcJnG9te7wVeOp2YSVYRAdOYYB+UJKuIKP+pCxExTzQ4Z9UvSVYRMTduZJb0JEknSVo24fO1/WtWRMyqlnvfBqRjspL0duBrwNuA7ZLWt+3+lw7fy+o2EQVxq9XzNijdhoFvBp5t+x5JRwMXSTra9keYvKQeeHA5/qIly4d/MBwx382Bq4ELbd8DYPsXkk6gSliPoUOyiojCFDDB3m3O6nZJz9z/pk5crwAOBZ7Wx3ZFxGwqfd1A4HQmPGPG9hhwuqR/61urImJ2FdCz6ra6zWiHff/dfHMiYiAKKF1InVVElN+zioj5wWPlXw2MiPkgPauIKELmrCKiCOlZRUQJnGQVEUXIBHtEFCE9q4goQpJVRJTATrKKiBKkZxURRUiyiogSeCxFoRFRguHPVd2TlaQ1gG1vlXQMsBb4Sb3iakTMAcUXhUp6D7AOWCTpm8BzgSuBcyQ9y/Y/T/G9DcAGAC08mAULDmy00RHRsAKSlTpdspR0I/BM4GHA7cAK23dLWgpcY/vp3U6QBSMi+m9s320zWhPh9685seff00O+9J2BrL/QbRg4ZnscuE/Sz23fDWD7fkkFjHIjohfFDwOBfZIOsH0f8Oz9H0o6mCKm5CKiFx4rP1kdb/uPAPaDHnizGDijb62KiNlVQNej24IRf5zi8zuBO/vSooiYdQU8ey91VhFB+T2riJgfSuhZdVuROSLmAY/1vvVC0lpJN0vaJemcKY45QdJ1knZI+m63mOlZRUSjPStJC4HzgJOBUWCrpM22b2o75hDg48Ba27dKenS3uOlZRQRu9b71YA2wy/Zu2/uATcD6Cce8FrjE9q0Atu/oFjQ9q1lw/96rBt2EmKalR7xw0E2YXe69KL39drraiO2RtvfLgT1t70epbtVr9wRgsaQrgYOAj9j+XKfzJllFxLSGgXViGulwyGSZb2LV6SKqQvOTgKXADyRdbfunUwVNsooI3Gr0dr9R4Mi29yuAvZMcc6fte4F7JX0PeAYwZbLKnFVE0BpXz1sPtgKrJK2UtAQ4Fdg84ZivAS+UtEjSAVTDxJ2dgqZnFRGNXg20PSbpbOByYCFwge0dks6q92+0vVPSZcANVCWp59ve3iluklVEND0MpH4455YJn22c8P79wPt7jZlkFREUsBJXklVENN+z6ockq4jodeJ8oJKsIqKIntW0SxckdawyjYjy2Op5G5Ruq9tMrI0QcGJ9EyK2XznF97K6TURBSnhETLdh4ArgJuB8qnJ5AauBD3b6Uns5fla3iRh+rQH2mHrVbRi4Gvgh8G7gLttXAvfb/q7trs+fiYgyFD8MrBeJ+JCkr9R//rrbdyKiPHPmaqDtUeCvJL0cuLu/TYqI2VbC1cBp9ZJsfwP4Rp/aEhEDUsKcVYZ0ETHQuaheJVlFRO4NjIgyZBgYEUVozbUJ9oiYm9Kzir6adyuwTCIrBzUjE+wRUYT0rCKiCAVcDEyyiggYbw3/QldJVhFBAU+ISbKKCPCkiygPlySriKBVwKRVklVE0ErPKiJKkGFgRBRhfK4lK0kvANYA221f0Z8mRcRsK+FqYMfiCknXtr1+M/Ax4CDgPZLO6fC9DZK2SdrWat3bWGMjoj9a09gGpVsl2OK21xuAk22/F3gp8LqpvmR7xPZq26uzDFfE8DPqeRuUbsPABZIeQZXUZPs3ALbvlTTW99ZFxKwo4AkxXZPVwVRLcQmwpD+3fbukZfVnETEHFF+6YPvoKXa1gFc13pqIGIjxQTegBw+pdMH2fcAtDbclIgakpcJ7VhExPxRwt02SVUTMgTqriJgfWup964WktZJulrSrS03mcySNS3p1t5jpWUVEo7fbSFoInAecDIwCWyVttn3TJMedC1zeS9z0rCKi6Z7VGmCX7d229wGbgPWTHPc24GLgjl6CJllFxLRut2m/na7eNkwItxzY0/Z+tP7sTyQtpyp/2thrGzMMjIhpXQ20PQKMdDhksv7XxFN8GHiX7XH1WDaRZBURTd9uMwoc2fZ+BbB3wjGrgU11ojoUOEXSmO1/nypoklVENF26sBVYJWklcBtwKvDa9gNsr9z/WtJngEs7JSpIsooIYLzBnpXtMUlnU13lWwhcYHuHpLPq/T3PU7VLsoqIxotCbW8Btkz4bNIkZfsNvcRMsoqIIirYk6wiIvcGRkQZ5sLD9yJiHihhGNhtwYjnSvqz+vVSSe+V9HVJ50o6eHaaGBH9Nj6NbVC63W5zAXBf/fojVI85Prf+7NNTfSmr20SUpemnLvRD1wUjbO9fGGK17WPr1/8l6bqpvtRejr9oyfIS5u4i5rXih4HAdkln1q+vl7QaQNITgAf62rKImDWexjYo3ZLVm4AXSfo5cAzwA0m7gU/W+yJiDmjhnrdB6ba6zV3AGyQdBDy2Pn7U9q9no3ERMTvmzOo2tv8AXN/ntkTEgJQwZ5U6q4hIUWhElGGQc1G9SrKKiAJSVZJVRJA5q4goxHgBfaskq4Ldv/eqQTch5oj0rCKiCJlgj4giDH+qSrKKCDIMjIhCZII9IoqQOauIKMLwp6okq4ggPauIKEQJE+zdFox4u6QjZ6sxETEYnsZ/g9LtSaHvA66RdJWkv5H0qF6CZsGIiLKM4563QemWrHYDK6iS1rOBmyRdJumM+umhk7I9Ynu17dULFhzYYHMjoh9a09gGpduclW23gCuAKyQtBtYBpwEfAHrqaUXEcGu5/An2Bz0/0PYDwGZgs6SlfWtVRMyq4U9V3ZPVa6baYfv+htsSEQNSfOmC7Z/OVkMiYnAGeZWvV6mzigjGkqwiogQl9Ky6lS5ExDzQdOmCpLWSbpa0S9I5k+x/naQb6u37kp7RLWZ6VhGBGyxdkLQQOA84GRgFtkrabPumtsNuAV5k+3eS1gEjwHM7xU2yioimrwauAXbZ3g0gaROwHvhTsrL9/bbjr6YqPu8oyWoWLD3ihYNuQkRH07mNRtIGYEPbRyO2R9reLwf2tL0fpXOv6Y3Af3Q7b5JVREyrZ1UnppEOh0y2GP2kJ5B0IlWyekG38yZZRUSjc1ZUPan2p7WsAPZOPEjS04HzgXW2/6db0FwNjIimrwZuBVZJWilpCXAq1W16fyLpKOAS4PW9Fp+nZxURjdZZ2R6TdDZwObAQuMD2Dkln1fs3Av8EPBL4uCSAMdurO8VVw92//2fRkuXDX20WUbixfbdNNk/Us5cc+bKef0+/tefyGZ3roUrPKiIY9/A/2DjJKiKKuN0mySoi5sTD9yJiHhj+VNUlWbVddtxr+1uSXgs8D9hJVbX6wCy0MSL6rPiH7wGfro85QNIZwDKq2oiTqO7/OWOyL7WX42vhwWTRiIjhNheS1dNsP13SIuA24Ajb45I+D1w/1Zfay/FTuhAx/ObC1cAF9VDwQOAA4GDgt8DDgMV9bltEzJK5cDXwU8BPqKpQ3w18RdJu4DhgU5/bFhGzpN/F4U3otmDEhyR9qX69V9LngJcAn7R97Ww0MCL6by7MWWF7b9vr3wMX9bNBETH7iu9ZRcT8MD7QheF7k2QVEalgj4gyzIWrgRExD6RnFRFFSM8qIoqQnlVEFGEu3G4TEfNAhoERUQSnZxURJZgTt9tExNyX220iogjpWUVEEcZbmbOKiALkamBEFGFOzFlJehzwKuBIYAz4GXCh7bv63LaImCUlzFkt6LRT0tuBjcDDgecAS6mS1g8kndDhexskbZO0rdW6t7nWRkRf2O55GxR1OrmkG4Fn1ivaHABssX2CpKOAr9l+VrcTZHWbiP4b23ebZvL9Ryx7fM+/p7+7Z9eMzvVQ9TJntQgYp1rR5iAA27dKyuo2EXNECcPAbsnqfGCrpKuB44FzASQ9impJroiYA0qYYO84DASQ9BTgycB22z+Z7gkyDIzov5kOA5cdsLLn39N77rtlOIeBtncAO2ahLRExIKmziogi5OF7EVGEVgGPiOlYZxUR80PTdVaS1kq6WdIuSedMsl+SPlrvv0HSsd1iJllFRKPJStJC4DxgHXAMcJqkYyYctg5YVW8bgE90i5tkFRF4GlsP1gC7bO+2vQ/YBKyfcMx64HOuXA0cIunwzo2cRkbt9wZsmO9xS2praXFLams/4zbRLmBb27Zhwv5XA+e3vX898LEJx1wKvKDt/beB1Z3OO2w9qw2JW1RbS4tbUlv7GXdGbI/YXt22jUw4ZLI6rImdsl6OeZBhS1YRUb5Rqgce7LcC2PsQjnmQJKuIaNpWYJWklZKWAKcCmyccsxk4vb4qeBxwl+1fdQo6bHVWE7uT8zFuSW0tLW5Jbe1n3L6yPSbpbOByYCFwge0dks6q928EtgCnALuA+4Azu8Xtem9gRMQwyDAwIoqQZBURRRiaZNWtPP8hxrxA0h2StjcRr455pKTvSNopaYekdzQU9+GSrpV0fR33vU3ErWMvlPRjSZc2GPMXkm6UdJ2kbQ3GPUTSRZJ+Uv8d/0UDMZ9Yt3P/drekdzYQ92/r/1fbJV0o6eEzjVnHfUcdc0cT7ZwzBl1gVs+ZLQR+DjwWWAJcDxzTQNzjgWOpnsXVVFsPB46tXx8E/LShtgpYVr9eDFwDHNdQm/8O+CJwaYN/D78ADu3Dz8JngTfVr5cAh/ThZ+124DEzjLMcuAVYWr//MvCGBtr3VGA7cADVBbBvAaua/nsucRuWnlUv5fnTZvt7NPxEU9u/sv2j+vUfgJ1UP7gzjWvb99RvF9fbjK9+SFoBvJzqqa9DTdKfUf0D8ykA2/ts/77h05wE/Nz2LxuItQhYKmkRVXLpWCfUoycDV9u+z/YY8F2q1aXmvWFJVsuBPW3vR2kgAfSbpKOBZ1H1gpqIt1DSdcAdwDdtNxH3w8DfA00/A8TAFZJ+KKmpSuvHAr8BPl0PW8+XdGBDsfc7FbhwpkFs3wZ8ALgV+BVVndAVM41L1as6XtIj60VaTuHBxZPz1rAkq2mX3g+apGXAxcA7bd/dREzb47afSVXNu0bSU2cST9IrgDts/7CJ9k3wfNvHUt09/1ZJxzcQcxHVsP0TrlZOuhdoZP4SoC5QfCXwlQZiPYKq978SOAI4UNJfzzSu7Z1Uax18E7iMakpkbKZx54JhSVbTLr0fpHpln4uBL9i+pOn49dDnSmDtDEM9H3ilpF9QDa1fLOnzM4wJgO299Z93AF+lGsrP1Cgw2tajvIgqeTVlHfAj279uINZLgFts/8b2A8AlwPMaiIvtT9k+1vbxVNMYP2sibumGJVn1Up4/FCSJak5lp+1/bTDuoyQdUr9eSvXLMO0FOtrZ/gfbK2wfTfV3+p+2Z/yvv6QDJR20/zXwUqrhy4zYvh3YI+mJ9UcnATfNNG6b02hgCFi7FThO0gH1z8RJVPOXMybp0fWfRwF/SXNtLtpQ3G7jKcrzZxpX0oXACcChkkaB99j+1AzDPp/qkRc31vNLAP9oe8sM4x4OfLZ+cNkC4Mu2Gys1aNhhwFer31EWAV+0fVlDsd8GfKH+R2s3PdyG0Yt6/udk4C1NxLN9jaSLgB9RDdN+THO3x1ws6ZHAA8Bbbf+uobhFy+02EVGEYRkGRkR0lGQVEUVIsoqIIiRZRUQRkqwioghJVhFRhCSriCjC/wJ1pLT+8sNqkAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASsAAAD8CAYAAAAv6IKXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAUyUlEQVR4nO3de7BdZX3G8e+ThGggCI4ohQQKarxQrxgj9YIoogk6MnbsFLSKjBiZipf2j0rrTBnGaTuMVxzRNGJURiVVwBoxBdR6oVUkUbkkBDQGhENAoCIoMMI5++kfa8Vu9pyz9z6ctc8+7z7Px1mTvfda67dfDzm//N53vWu9sk1ExFy3YNgNiIjoR5JVRBQhySoiipBkFRFFSLKKiCIkWUVEEZKsIqJRkjZIulPStin2S9InJO2UdK2kI/uJm2QVEU37PLC6y/41wIp6Wwt8up+gSVYR0SjbPwB+0+WQE4DzXbkS2F/SQb3iLmqqgVN5+O5dRU2RX3Lwy4bdhIhpG3/oNs3k/On8ni5+4lPeSVUR7bHe9vppfN0y4Na292P1Z7d3O2ngySoiCtCa6PvQOjFNJzl1miyx9kyWSVYRAW7N5reNAYe0vV8O7O51UsasIgJarf63mdsEvLW+KngUcK/trl1ASGUVEYAbrKwkXQAcAxwgaQw4E9ir+h6vAzYDxwM7gQeAU/qJm2QVETAx3lgo2yf12G/gXdONm2QVEdMaYB+WJKuImO0B9kclySoimho4H6gkq4hodIB9UHomK0nPoJoev4xq4tZuYJPtHQNuW0TMlgIqq67zrCS9H9hINeP0KmBL/foCSWd0OW+tpK2Stp53/gVNtjciBmHi4f63IVG31W0k/Rz4M9sPd3y+GNhue0WvL8i9gRGDN9N7A/+w47t9/54+5pmvmNF3PVq9uoEt4GDgVx2fH1Tvi4hRUEA3sFeyeh/wHUm/4P/vkj4UeCpw+gDbFRGzqfQBdtuXSnoasIpqgF1UNyFusT33Z5FFRH9GoLLC1TXNK2ehLRExJG4Nb+C8X5lnFRGjUVlFxDxQ+phVRMwTuZE5IoqQyioiipAxq4goQoMP3xuUJKuISGUVEWUoYY53klVEpLKKiELkamBEFCGVVUQUIVcDI6II6QZGRBHSDYyIIiRZRUQRCugGdl3dphtJp3TZl9VtIkoyMd7/NiQzqazOAj432Q7b64H1UN7qNhHzUundQEnXTrULOLD55kTEUBTQDexVWR0IvAa4p+NzAT8cSIsiYvaVXlkBlwBLbV/duUPS9wbRoIgYgtKTle23d9n3puabExFD0WVl9rkiUxciAsZzu01ElKCAAfZHPc8qIkZIq9X/1gdJqyXdKGmnpDMm2b+fpG9IukbS9m7zNvdIsoqIasyq360HSQuBc4E1wBHASZKO6DjsXcD1tp8LHAN8RNLibnHTDYyIpq8GrgJ22t4FIGkjcAJwfdsxBvaVJGAp8Bug68BZKquImFY3sP12unpb2xFtGXBr2/ux+rN2nwSeCewGrgPea3cfOEtlFRF4ov8FI9pvp5uCJjut4/1rgKuBVwJPAb4l6Qrb900VNJVVRDQ9wD4GHNL2fjlVBdXuFOBiV3YCNwHP6BY0ySoiqqkL/W69bQFWSDq8HjQ/EdjUccwtwLEAkg4Eng7s6hY03cCIgFZzM9htj0s6HbgMWAhssL1d0mn1/nXAB4HPS7qOqtv4ftt3d4ubZBURjd8baHszsLnjs3Vtr3cDr55OzCSriIBpDLAPS5JVRJT/1IWImCcaHLMalCSriBiNG5klPUPSsZKWdny+enDNiohZ1XL/25B0TVaS3gN8HXg3sE3SCW27/6XLeVndJqIgbrX63oalVzfwHcALbP9e0mHAhZIOs30Ok0+pB7K6TURxRuBq4ELbvwewfbOkY6gS1p/SJVlFRGEKGGDvNWZ1h6Tn7XlTJ67XAQcAzx5guyJiNjX88L1B6FVZvZWOZ8zYHgfeKunfBtaqiJhdBVRWvVa3Geuy73+ab05EDEUBUxcyzyoiyq+sImJ+8Hj5VwMjYj5IZRURRciYVUQUIZVVRJTASVYRUYQMsEdEEVJZRUQRkqwiogR2klVElCCVVUQUIckqIkrg8UwKjYgSzP1c1TtZSVoF2PYWSUcAq4Eb6hVXI2IElDAptNeCEWcCnwA+LelfgU8CS4EzJH2gy3lZMCKiJAWsbtOrsnoj8DzgMcAdwHLb90n6EPBj4J8nOykLRkQUZgS6geO2J4AHJP3S9n0Ath+UVMD/vYjoRwndwF7J6iFJe9t+AHjBng8l7UcRuTgi+uHx8pPV0bb/AGA/4oE3ewEnD6xVETG7Cig9ei0Y8YcpPr8buHsgLYqIWVfAs/cyzyoiKL+yioj5oYTKqteKzBExD3i8/60fklZLulHSTklnTHHMMZKulrRd0vd7xUxlFRGNVlaSFgLnAscBY8AWSZtsX992zP7Ap4DVtm+R9KRecVNZRQRu9b/1YRWw0/Yu2w8BG4ETOo55E3Cx7VsAbN/ZK+jAK6slB79sIHEf3H3FQOJGzEtW34dKWgusbftofX3Xyh7LgFvb3o8BL+oI8zRgL0nfA/YFzrF9frfvTTcwIqbVDWy/nW4Kk2W+zlmni6gmmh8LLAF+JOlK2z+fKmiSVUTgVv+VVR/GgEPa3i8Hdk9yzN227wful/QD4LnAlMkqY1YRQWtCfW992AKskHS4pMXAicCmjmO+DrxM0iJJe1N1E3d0C5rKKiIavRpoe1zS6cBlwEJgg+3tkk6r96+zvUPSpcC1VFNSz7O9rVvcJKuIaLobSP1wzs0dn63reP8h4EP9xkyyiggKWIkrySoimq+sBiHJKiL6HTgfqiSriCiispr21AVJXWeZRkR5bPW9DUvXykpS59wIAa+ob0LE9uunOO+P0/G1cD8WLNhn5i2NiIEp4RExvbqBy4HrgfOopssLWAl8pNtJ7dPxFy1eVsB1hoj5rTXEiqlfvbqBK4GfAB8A7rX9PeBB29+33fP5MxFRhuK7gfUiER+T9NX6z1/3OiciyjMyVwNtjwF/Kem1wH2DbVJEzLYSrgZOq0qy/U3gmwNqS0QMSQljVunSRcRQx6L6lWQVEbk3MCLKkG5gRBShNWoD7BExmlJZDdCgVs2JmI8ywB4RRUhlFRFFKOBiYJJVRMBEa+4vdJVkFREU8ISYJKuIAE+6iPLckmQVEbQKGLRKsooIWqmsIqIE6QZGRBEmRi1ZSXopsArYZvvywTQpImZbCVcDu06ukHRV2+t3AJ8E9gXOlHRGl/PWStoqaWurdX9jjY2IwWhNYxuWXjPB9mp7vRY4zvZZwKuBN091ku31tlfaXplluCLmPqO+t2Hp1Q1cIOnxVElNtu8CsH2/pPGBty4iZkUBT4jpmaz2o1qKS4Al/YntOyQtrT+LiBFQ/NQF24dNsasFvKHx1kTEUEwMuwF9eFRTF2w/ANzUcFsiYkhaKryyioj5oYC7bZKsImIE5llFxPzQUv9bPyStlnSjpJ095mS+UNKEpDf2ipnKKiIavd1G0kLgXOA4YAzYImmT7esnOe5s4LJ+4qayioimK6tVwE7bu2w/BGwETpjkuHcDFwF39hM0lVWHB3df0XjMrMQTc910xqwkraW6o2WP9bbXt71fBtza9n4MeFFHjGVU059eCbywn+9NsoqIaV0NrBPT+i6HTFZ/dX7Fx4H3255Qn9MmkqwiounbbcaAQ9reLwd2dxyzEthYJ6oDgOMljdv+j6mCJllFRNNTF7YAKyQdDtwGnAi8qf0A24fveS3p88Al3RIVJFlFBDDRYGVle1zS6VRX+RYCG2xvl3RavX/do4mbZBURjU8Ktb0Z2Nzx2aRJyvbb+omZZBURRcxgT7KKiNwbGBFlGIWH70XEPFBCN7DXghEvkvS4+vUSSWdJ+oaksyXtNztNjIhBm5jGNiy97g3cADxQvz6H6jHHZ9effW6qk7K6TURZmn7qwiD0XDDC9p6FIVbaPrJ+/d+Srp7qpPbp+IsWLyth7C5iXiu+Gwhsk3RK/foaSSsBJD0NeHigLYuIWeNpbMPSK1mdCrxc0i+BI4AfSdoFfKbeFxEjoIX73oal1+o29wJvk7Qv8OT6+DHbv56NxkXE7BiZ1W1s/w64ZsBtiYghKWHMKvOsIiKTQiOiDMMci+pXklVEFJCqkqwigoxZRUQhJgqorZKsIiKVVUSUIQPsEVGEuZ+qkqwignQDI6IQGWCPiCJkzCoiijD3U1WSVUSQyioiClHCAHuvBSPeI+mQ2WpMRAyHp/G/Yen1pNAPAj+WdIWkv5H0xH6CZsGIiLJM4L63YemVrHYBy6mS1guA6yVdKunk+umhk7K93vZK2ysXLNinweZGxCC0prENS68xK9tuAZcDl0vaC1gDnAR8GOir0oqIua3l8gfYH/H8QNsPA5uATZKWDKxVETGr5n6q6p2s/mqqHbYfbLgtETEkxU9dsP3z2WpIRAzPMK/y9SvzrCKC8SSriChBCZVVr6kLETEPND11QdJqSTdK2inpjEn2v1nStfX2Q0nP7RUzlVVE4AanLkhaCJwLHAeMAVskbbJ9fdthNwEvt32PpDXAeuBF3eImWUVE01cDVwE7be8CkLQROAH4Y7Ky/cO246+kmnzeVbqBETGt223ab6ert7Ud4ZYBt7a9H6s/m8rbgf/s1cZUVhExrcrK9nqqbttUJluMftIvkPQKqmT10l7fm2QVEY2OWVFVUu1Pa1kO7O48SNJzgPOANbb/t1fQdAMjoumrgVuAFZIOl7QYOJHqNr0/knQocDHwln4nn6eyiohG51nZHpd0OnAZsBDYYHu7pNPq/euAfwKeAHxKEsC47ZXd4iZZRUTj9wba3gxs7vhsXdvrU4FTpxMzySoimPDcf7BxklVEFHG7TZJVRIzEw/ciYh6Y+6mqR7Jqu+y42/a3Jb0JeDGwA1hfPzk0IgpX/MP3gM/Vx+wt6WRgKdXciGOp7v85ebKT6un3awG0cD+yaETE3DYKyerZtp8jaRFwG3Cw7QlJXwSumeqk9un4ixYvm/s/hYh5bhSuBi6ou4L7AHsD+wG/AR4D7DXgtkXELBmFq4GfBW6gmoX6AeCrknYBRwEbB9y2iJglDd8bOBC9Foz4mKR/r1/vlnQ+8CrgM7avmo0GRsTgjcKYFbZ3t73+LXDhIBsUEbOv+MoqIuaHiaEuDN+fJKuIyAz2iCjDKFwNjIh5IJVVRBQhlVVEFCGVVUQUYRRut4mIeSDdwIgoglNZRUQJRuJ2m4gYfbndJiKKkMoqIoow0cqYVUQUIFcDI6IIIzFmJekpwBuAQ4Bx4BfABbbvHXDbImKWlDBmtaDbTknvAdYBjwVeCCyhSlo/knRMl/PWStoqaWurdX9zrY2IgbDd9zYs6vblkq4DnlevaLM3sNn2MZIOBb5u+/m9vqC01W0e3H1F4zGXHPyyxmNGtBt/6DbN5PzHL31q37+n9/x+54y+69HqZ8xqETBBtaLNvgC2b5GU1W0iRkQJ3cBeyeo8YIukK4GjgbMBJD2RakmuiBgBxQ+w2z5H0reBZwIftX1D/fldVMkrIkbASDwixvZ2YPsstCUihiTzrCKiCCNRWUXE6GsV8IiYrvOsImJ+aHqelaTVkm6UtFPSGZPsl6RP1PuvlXRkr5hJVhHRaLKStBA4F1gDHAGcJOmIjsPWACvqbS3w6V5xk6wiAk9j68MqYKftXbYfAjYCJ3QccwJwvitXAvtLOqh7I6eRUQe9AWvne9yS2lpa3JLaOsi4TbQL2Nq2re3Y/0bgvLb3bwE+2XHMJcBL295/B1jZ7XvnWmW1NnGLamtpcUtq6yDjzojt9bZXtm3rOw6Z7HaczqKsn2MeYa4lq4go3xjVAw/2WA7sfhTHPEKSVUQ0bQuwQtLhkhYDJwKbOo7ZBLy1vip4FHCv7du7BZ1r86w6y8n5GLektpYWt6S2DjLuQNkel3Q6cBmwENhge7uk0+r964DNwPHATuAB4JRecbs+IiYiYq5INzAiipBkFRFFmDPJqtf0/EcZc4OkOyVtayJeHfMQSd+VtEPSdknvbSjuYyVdJemaOu5ZTcStYy+U9DNJlzQY82ZJ10m6WtLWBuPuL+lCSTfUP+M/byDm0+t27tnuk/S+BuL+bf3fapukCyQ9dqYx67jvrWNub6KdI2PYE8zqMbOFwC+BJwOLgWuAIxqIezRwJLCtwbYeBBxZv94X+HlDbRWwtH69F/Bj4KiG2vx3wJeBSxr8OdwMHDCAvwtfAE6tXy8G9h/A37U7gD+dYZxlwE3Akvr9V4C3NdC+ZwHbgL2pLoB9G1jR9M+5xG2uVFb9TM+fNts/oOEnmtq+3fZP69e/A3ZQ/cWdaVzb/n39dq96m/HVD0nLgddSPfV1TpP0OKp/YD4LYPsh279t+GuOBX5p+1cNxFoELJG0iCq5dJ0n1KdnAlfafsD2OPB9qtWl5r25kqyWAbe2vR+jgQQwaJIOA55PVQU1EW+hpKuBO4Fv2W4i7seBvweafgaIgcsl/URSUzOtnwzcBXyu7raeJ2mfhmLvcSJwwUyD2L4N+DBwC3A71Tyhy2cal6qqOlrSE+pFWo7nkZMn5625kqymPfV+2CQtBS4C3mf7viZi2p6w/Tyq2byrJD1rJvEkvQ640/ZPmmhfh5fYPpLq7vl3SWriMdeLqLrtn3a1ctL9QCPjlwD1BMXXA19tINbjqar/w4GDgX0k/fVM49reQbXWwbeAS6mGRMZnGncUzJVkNe2p98NUr+xzEfAl2xc3Hb/u+nwPWD3DUC8BXi/pZqqu9SslfXGGMQGwvbv+807ga1Rd+ZkaA8baKsoLqZJXU9YAP7X96wZivQq4yfZdth8GLgZe3EBcbH/W9pG2j6YaxvhFE3FLN1eSVT/T8+cESaIaU9lh+6MNxn2ipP3r10uofhlumElM2/9ge7ntw6h+pv9le8b/+kvaR9K+e14Dr6bqvsyI7TuAWyU9vf7oWOD6mcZtcxINdAFrtwBHSdq7/jtxLNX45YxJelL956HAX9Bcm4s2J2638RTT82caV9IFwDHAAZLGgDNtf3aGYV9C9ciL6+rxJYB/tL15hnEPAr5QP7hsAfAV241NNWjYgcDXqt9RFgFftn1pQ7HfDXyp/kdrF33chtGPevznOOCdTcSz/WNJFwI/peqm/Yzmbo+5SNITgIeBd9m+p6G4RcvtNhFRhLnSDYyI6CrJKiKKkGQVEUVIsoqIIiRZRUQRkqwioghJVhFRhP8D8CzBae5QeUIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASsAAAD8CAYAAAAv6IKXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAU50lEQVR4nO3de5BcZZ3G8e+ThGi4CJQgCwlIVLzgDTFG1guiiCZoSbmlJeCKUmKkFhR3/1jZtWopytqtpcp7iWZHQGVVUAHXiFkuuqLsKhBQLgkXDQFhCIisCHIpYaaf/eOcsM3UTHcPc3q635nnQ51Kd59zfv1WmPnlfd/zO+eVbSIiht2CQTcgIqIXSVYRUYQkq4goQpJVRBQhySoiipBkFRFFSLKKiEZJOkvSvZI2TrFfkr4gabOk6yUd2EvcJKuIaNrXgFUd9q8G9qu3NcCXewmaZBURjbL9M+APHQ45AjjblSuAXSTt2S3uoqYaOOUXLF6aEvk+eXTr5X2Ju2Sv1/clbvTP2GN3aSbnP37flp5/Txfv/twPU/WIthmxPTKNr1sK3Nn2frT+7O5OJ/U9WUVEAVrjPR9aJ6bpJKeJJkusXZNlklVEgFuz+W2jwN5t75cBW7udlDmriIBWq/dt5tYBx9RXBQ8CHrDdcQgI6VlFBOAGe1aSzgEOAXaTNAqcAmxXfY/XAuuBw4HNwCPAsb3ETbKKCBgfayyU7aO67DdwwnTjJllFxLQm2AclySoiZnuC/SlJsoqIpibO+yrJKiIanWDvl67JStILqcrjl1IVbm0F1tm+qc9ti4jZUkDPqmOdlaSPA+dSVZxeBWyoX58j6eQO562RdLWkq1uth5tsb0T0w/jjvW8D0q1n9UHgxbaf1EJJnwE2Af862Unt5fi5NzCiAHNgGNgC9gJ+O+HzPet9ETEXFDAM7JasPgb8WNJv+P+7pPcBngec2Md2RcRsKr1nZfsiSc8HVlJNsIvqJsQNtoe/iiwiejMHela4uqZ5xSy0JSIGxK3BTZz3KnVWETE3elYRMQ+UPmcVEfNEbmSOiCKkZxURRcicVUQUocGH7/VLklVEpGcVEWUoocY7ySoi0rOKiELkamBEFCE9q4goQq4GRkQRMgyMiCJkGBgRRUiyiogiFDAM7Li6TSeSju2wL6vbRJRkfKz3bUCecrICTp1qh+0R2ytsr1iwYIcZfEVEzIpWq/dtQDoOAyVdP9UuYI/mmxMRA1HAMLDbnNUewFuB+yd8LuDnfWlRRMy+OTDBfiGwo+1rJ+6QdFk/GhQRA1B6srL9wQ77jm6+ORExEB7+hdNTuhARMJbbbSKiBAVMsM+kdCEi5oqGSxckrZJ0i6TNkk6eZP/Okn4g6TpJmzrVbW6TZBUR1ZxVr1sXkhYCpwOrgf2BoyTtP+GwE4Abbb8cOAT4tKTFneJmGBgRTV8NXAlstr0FQNK5wBHAjW3HGNhJkoAdgT8AHSfO0rOKiGkNA9tvp6u3NROiLQXubHs/Wn/W7ovAi4CtwA3ASXbnibP0rCICj/e+YITtEWCkwyGa7LQJ798KXAu8CXgucKmky20/OFXQ9KwioukJ9lFg77b3y6h6UO2OBS5wZTNwG/DCTkGTrCKiKl3odetuA7CfpOX1pPmRwLoJx9wBHAogaQ/gBcCWTkEzDIwIaDVXwW57TNKJwMXAQuAs25skHV/vXwt8EviapBuoho0ft31fp7hJVhHR+L2BttcD6yd8trbt9VbgLdOJmWQVETCNCfZBSbKKiPKfuhAR80SDc1b9kmQVEXPjRmZJL5R0qKQdJ3y+qn/NiohZ1XLv24B0TFaSPgp8H/gIsFHSEW27/6XDeVndJqIgbrV63gal2zDwQ8ArbT8kaV/gPEn72v48k5fUA08ux1+0eOnwD4Yj5rs5cDVwoe2HAGzfLukQqoT1bDokq4goTAET7N3mrO6RdMC2N3XiejuwG/DSPrYrImZT6esGAscw4RkztseAYyT9W99aFRGzq4CeVbfVbUY77Puf5psTEQNRQOlC6qwiovyeVUTMDx4r/2pgRMwH6VlFRBEyZxURRUjPKvppyV6vH3QTYo5wklVEFCET7BFRhPSsIqIISVYRUQI7ySoiSpCeVUQUIckqIkrgsRSFRkQJhj9XdU9WklYCtr1B0v7AKuDmesXViJgDii8KlXQKsBpYJOlS4NXAZcDJkl5h+5+nOG8NsAZAC3dmwYIdGm10RDSsgGSlTpcsJd0AHAA8DbgHWGb7QUlLgCttv6zbF2TBiIj+G3vsrhmtifDH97yx59/TXb79k4Gsv9BtGDhmexx4RNKtth8EsP2opAJGuRHRi+KHgcBjkra3/Qjwym0fStqZIqbkIqIXHis/WR1s+88A9pMeeLMd8P6+tSoiZlcBXY9uC0b8eYrP7wPu60uLImLWFfDsvdRZRQTl96wiYn4ooWfVbUXmiJgHPNb71gtJqyTdImmzpJOnOOYQSddK2iTpp91ipmcVEY32rCQtBE4HDgNGgQ2S1tm+se2YXYAvAats3yHpWd3ipmcVEbjV+9aDlcBm21tsPwacCxwx4ZijgQts3wFg+95uQZOsIgKsnjdJayRd3batmRBtKXBn2/vR+rN2zwd2lXSZpGskHdOtiRkGRsS0hoG2R4CRDodMdjvOxKrTRVSF5ocCS4BfSLrC9q+nCppkFRG41ejtfqPA3m3vlwFbJznmPtsPAw9L+hnwcmDKZJVhYETQGlfPWw82APtJWi5pMXAksG7CMd8HXi9pkaTtqZ7oclOnoOlZRUSjVwNtj0k6EbgYWAicZXuTpOPr/Wtt3yTpIuB6qpLUM2xv7BS34yNimpBHxET030wfEXPnqw7t+fd07w0/HspHxETEPFDASlxJVhHR+AR7XyRZRUSvE+cDlWQVEUX0rKZduiDp7H40JCIGx1bP26B0W91mYm2EgDfWNyFi+x1TnJfVbSIKUsIjYroNA5cBNwJnUJXLC1gBfLrTSe3l+CldiBh+rQH2mHrVbRi4ArgG+ATwgO3LgEdt/9R21+fPREQZih8G1otEfFbSd+s/f9ftnIgoz5y5Gmh7FHi3pLcBD/a3SREx20q4GjitXpLtHwI/7FNbImJASpizypAuIgY6F9WrJKuIyL2BEVGGDAMjogituTbBHhFzU3pWEVGETLBHRBHSs4qIIhRwMTDJKiJgvDX8C10lWUUEBTwhJskqIsCTLqI8XJKsIoJWAZNWSVYRQSs9q4goQYaBEVGE8bmWrCS9DlgJbLR9SX+aFBGzrYSrgR2LKyRd1fb6Q8AXgZ2AUySd3OG8NZKulnR1q/VwY42NiP5oTWMblG6VYNu1vV4DHGb7VOAtwHunOsn2iO0VtldkGa6I4WfU8zYo3YaBCyTtSpXUZPv3ALYfljTW99ZFxKwo4AkxXZPVzlRLcQmwpL+wfY+kHevPImIOKL50wfa+U+xqAe9svDURMRDjg25AD55S6YLtR4DbGm5LRAxIS4X3rCJifijgbpskq4iYA3VWETE/tNT71gtJqyTdImlzl5rMV0kal/SubjHTs4qIRm+3kbQQOB04DBgFNkhaZ/vGSY47Dbi4l7jpWUVE0z2rlcBm21tsPwacCxwxyXEfAc4H7u0laJJVREzrdpv22+nqbc2EcEuBO9vej9afPUHSUqryp7W9tjHDwIiY1tVA2yPASIdDJut/TfyKzwEftz2uHssmkqwiounbbUaBvdveLwO2TjhmBXBunah2Aw6XNGb7P6YKmmQVEU2XLmwA9pO0HLgLOBI4uv0A28u3vZb0NeDCTokKkqwiAhhvsGdle0zSiVRX+RYCZ9neJOn4en/P81TtkqwiovGiUNvrgfUTPps0Sdn+QC8xk6wioogK9iSriMi9gRFRhrnw8L2ImAdKGAZ2WzDi1ZKeUb9eIulUST+QdJqknWeniRHRb+PT2Aal2+02ZwGP1K8/T/WY49Pqz7461UlZ3SaiLE0/daEfui4YYXvbwhArbB9Yv/5vSddOdVJ7Of6ixUtLmLuLmNeKHwYCGyUdW7++TtIKAEnPBx7va8siYtZ4GtugdEtWxwFvkHQrsD/wC0lbgK/U+yJiDmjhnrdB6ba6zQPAByTtBDynPn7U9u9mo3ERMTvmzOo2tv8EXNfntkTEgJQwZ5U6q4hIUWhElGGQc1G9SrKKiAJSVZJVRJA5q4goxHgBfaskq4hIzyoiypAJ9ogowvCnqiSriCDDwIgoRCbYI6IImbOKiCIMf6pKsooI0rOKiEKUMMHebcGIj0rae7YaExGD4Wn8NyjdnhT6SeBKSZdL+htJu/cSNAtGRJRlHPe8DUq3ZLUFWEaVtF4J3CjpIknvr58eOinbI7ZX2F6xYMEODTY3IvqhNY1tULrNWdl2C7gEuETSdsBq4CjgU0BPPa2IGG4tlz/B/qTnB9p+HFgHrJO0pG+tiohZNfypqnuyes9UO2w/2nBbImJAii9dsP3r2WpIRAzOIK/y9Sp1VhHBWJJVRJSghJ5Vt9KFiJgHmi5dkLRK0i2SNks6eZL975V0fb39XNLLu8VMzyoicIOlC5IWAqcDhwGjwAZJ62zf2HbYbcAbbN8vaTUwAry6U9wkq4ho+mrgSmCz7S0Aks4FjgCeSFa2f952/BVUxecd9T1ZPbr18n5/xdBbstfrB92EiI6mcxuNpDXAmraPRmyPtL1fCtzZ9n6Uzr2mDwL/2e1707OKiGn1rOrENNLhkMkWo5/0CyS9kSpZva7b9yZZRUSjc1ZUPan2p7UsA7ZOPEjSy4AzgNW2/7db0FwNjIimrwZuAPaTtFzSYuBIqtv0niBpH+AC4H29Fp+nZxURjdZZ2R6TdCJwMbAQOMv2JknH1/vXAv8EPBP4kiSAMdsrOsVNsoqIxu8NtL0eWD/hs7Vtr48DjptOzCSriGDcw/9g4ySriCjidpskq4iYEw/fi4h5YPhTVZdk1XbZcavtH0k6GngNcBNV1erjs9DGiOizEh6+163O6qvA24CTJP078G7gSuBVVMVck2pf3eaMs89prLER0R8t3PM2KN2GgS+1/TJJi4C7gL1sj0v6BnDdVCe1l+M/ft+W4U/ZEfPcXLgauKAeCu4AbA/sDPwBeBqwXZ/bFhGzZC5cDTwTuJmqCvUTwHclbQEOAs7tc9siYpY0fG9gX3RbMOKzkr5dv94q6WzgzcBXbF81Gw2MiP4rYYK9a+mC7a1tr/8InNfPBkXE7Cu+ZxUR88P4QBeG702SVUSkgj0iyjAXrgZGxDyQnlVEFCE9K7KyS0QJ0rOKiCLMhdttImIeyDAwIorg9KwiogRz4nabiJj7crtNRBQhPauIKMJ4K3NWEVGAXA2MiCLMiTkrSc8F3gnsDYwBvwHOsf1An9sWEbOkhDmrjqvbSPoosBZ4OtWKNkuoktYvJB3S4bwnVrdptR5urrUR0Re2e94GRZ2+XNINwAH1ijbbA+ttHyJpH+D7tl/R7QsWLV46/Ck7onBjj92lmZy/647P6/n39P6HNs/ou56qXuasFgHjVCva7ARg+w5JWd0mYo4oYRjYLVmdAWyQdAVwMHAagKTdqZbkiog5oIQJ9o7DQABJLwZeBGy0ffN0vyDDwIj+m+kwcMftl/f8e/rQI7cN5zDQ9iZg0yy0JSIGJHVWEVGEPHwvIorQKuARMR3rrCJifmi6zkrSKkm3SNos6eRJ9kvSF+r910s6sFvMJKuIaDRZSVoInA6sBvYHjpK0/4TDVgP71dsa4Mvd4iZZRQSextaDlcBm21tsPwacCxwx4ZgjgLNduQLYRdKenRs5jYza7w1YM9/jltTW0uKW1NZ+xm2iXcDVbduaCfvfBZzR9v59wBcnHHMh8Lq29z8GVnT63mHrWa1J3KLaWlrcktraz7gzYnvE9oq2bWTCIZPVYU3slPVyzJMMW7KKiPKNUj3wYJtlwNancMyTJFlFRNM2APtJWi5pMXAksG7CMeuAY+qrggcBD9i+u1PQYauzmtidnI9xS2praXFLams/4/aV7TFJJwIXAwuBs2xvknR8vX8tsB44HNgMPAIc2y1u13sDIyKGQYaBEVGEJKuIKMLQJKtu5flPMeZZku6VtLGJeHXMvSX9RNJNkjZJOqmhuE+XdJWk6+q4pzYRt469UNKvJF3YYMzbJd0g6VpJVzcYdxdJ50m6uf47/ssGYr6gbue27UFJH2sg7t/W/682SjpH0tNnGrOOe1Idc1MT7ZwzBl1gVs+ZLQRuBZ4DLAauA/ZvIO7BwIFUz+Jqqq17AgfWr3cCft1QWwXsWL/eDrgSOKihNv8d8C3gwgb/Hm4HduvDz8LXgePq14uBXfrws3YP8OwZxlkK3AYsqd9/B/hAA+17CbAR2J7qAtiPgP2a/nsucRuWnlUv5fnTZvtnNPxEU9t32/5l/fpPwE1UP7gzjWvbD9Vvt6u3GV/9kLQMeBvVU1+HmqRnUP0DcyaA7cds/7HhrzkUuNX2bxuItQhYImkRVXLpWCfUoxcBV9h+xPYY8FOq1aXmvWFJVkuBO9vej9JAAug3SfsCr6DqBTURb6Gka4F7gUttNxH3c8DfA00/A8TAJZKukdRUpfVzgN8DX62HrWdI2qGh2NscCZwz0yC27wI+BdwB3E1VJ3TJTONS9aoOlvTMepGWw3ly8eS8NSzJatql94MmaUfgfOBjth9sIqbtcdsHUFXzrpT0kpnEk/R24F7b1zTRvglea/tAqrvnT5B0cAMxF1EN27/sauWkh4FG5i8B6gLFdwDfbSDWrlS9/+XAXsAOkv56pnFt30S11sGlwEVUUyJjM407FwxLspp26f0g1Sv7nA980/YFTcevhz6XAatmGOq1wDsk3U41tH6TpG/MMCYAtrfWf94LfI9qKD9To8BoW4/yPKrk1ZTVwC9t/66BWG8GbrP9e9uPAxcAr2kgLrbPtH2g7YOppjF+00Tc0g1LsuqlPH8oSBLVnMpNtj/TYNzdJe1Sv15C9csw7QU62tn+B9vLbO9L9Xf6X7Zn/K+/pB0k7bTtNfAWquHLjNi+B7hT0gvqjw4Fbpxp3DZH0cAQsHYHcJCk7eufiUOp5i9nTNKz6j/3Af6K5tpctKG43cZTlOfPNK6kc4BDgN0kjQKn2D5zhmFfS/XIixvq+SWAf7S9foZx9wS+Xj+4bAHwHduNlRo0bA/ge9XvKIuAb9m+qKHYHwG+Wf+jtYUebsPoRT3/cxjw4Sbi2b5S0nnAL6mGab+iudtjzpf0TOBx4ATb9zcUt2i53SYiijAsw8CIiI6SrCKiCElWEVGEJKuIKEKSVUQUIckqIoqQZBURRfg/OqvMJD+N4KEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "for targ in train_y[:5]:\n",
    "    sns.heatmap(targ.any(0), square=True)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "8f1c5796",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_x0_targ(x0:np.ndarray, targ:np.ndarray) -> None:\n",
    "    n_layers = x0.shape[0]\n",
    "    fig, axs = plt.subplots(n_layers,2, figsize=(8, 8*16/9))\n",
    "    pred_cbar_ax = fig.add_axes([0.45, 0.25, 0.03, 0.5])\n",
    "    true_cbar_ax = fig.add_axes([0.90, 0.25, 0.03, 0.5])\n",
    "\n",
    "    for layer in range(n_layers-1,-1,-1):\n",
    "        sns.heatmap(x0[layer], ax=axs[n_layers-layer-1,0], vmin=np.nanmin(x0), vmax=np.nanmax(x0), cbar=(layer==0), cbar_ax=pred_cbar_ax if layer == 0 else None, square=True, cmap='viridis')\n",
    "        sns.heatmap(targ[layer], ax=axs[n_layers-layer-1,1], vmin=np.nanmin(targ), vmax=np.nanmax(targ), cbar=(layer==0), cbar_ax=true_cbar_ax if layer == 0 else None, square=True, cmap='viridis')\n",
    "        axs[n_layers-layer-1][0].set_ylabel(f\"Layer {layer}\")\n",
    "    axs[-1][0].set_xlabel(\"X0\")\n",
    "    axs[-1][1].set_xlabel(\"Target\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37db1c83",
   "metadata": {},
   "source": [
    "Due to the large range of X0s the colour scale gets compressed. Instead we can plot on a log-scale to better see differences in the low X0 region"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "b559366f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_x0_targ_log(x0:np.ndarray, targ:np.ndarray) -> None:\n",
    "    #targ=targ.astype(\"int\")\n",
    "    n_layers = x0.shape[0]\n",
    "    fig, axs = plt.subplots(n_layers,2, figsize=(8, 8*16/9))\n",
    "    pred_cbar_ax = fig.add_axes([0.45, 0.25, 0.03, 0.5])\n",
    "    true_cbar_ax = fig.add_axes([0.90, 0.25, 0.03, 0.5])\n",
    "    cbar_ticks = [math.pow(10, i) for i in range(math.floor(math.log10(np.nanmin(x0))), 1+math.ceil(math.log10(np.nanmax(x0))))]\n",
    "    log_norm = LogNorm(vmin=np.nanmin(x0), vmax=np.nanmax(x0))\n",
    "    \n",
    "    for layer in range(n_layers-1,-1,-1):\n",
    "        sns.heatmap(x0[layer], ax=axs[n_layers-layer-1,0], vmin=np.nanmin(x0), vmax=np.nanmax(x0), cbar=(layer==0), cbar_ax=pred_cbar_ax if layer == 0 else None, square=True, cmap='viridis', norm=log_norm, cbar_kws={\"ticks\": cbar_ticks})\n",
    "        sns.heatmap(targ[layer], ax=axs[n_layers-layer-1,1], vmin=np.nanmin(targ), vmax=np.nanmax(targ), cbar=(layer==0), cbar_ax=true_cbar_ax if layer == 0 else None, square=True, cmap='viridis')\n",
    "        axs[n_layers-layer-1][0].set_ylabel(f\"Layer {layer}\")\n",
    "    axs[-1][0].set_xlabel(\"X0\")\n",
    "    axs[-1][1].set_xlabel(\"Target\")\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45f77934",
   "metadata": {},
   "source": [
    "## The Intersection over Unity\n",
    "\n",
    "The intersection is the area of overlap between predicted walls and true wall, and the union is the summed area of predicted walls and true walls. By taking the ratio, we quantify the precision of our predictions: if we predicted a large area for one walls, then we would identify all the voxels with walls, however if there are only a few voxels that actually are walls, then the large union would reduce the IOU. This is computed separately for each sample, and then we compute the mean IOU across the samples.\n",
    "\n",
    "The IOU is a float between 0 and 1, with 1 being a perfect prediction, in which only every voxel is correctly identified as either soil or wall."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "18ac06e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "def binary_iou(preds:np.ndarray, targs:np.ndarray, as_mean:bool=True, smooth:float=1e-17) -> float:\n",
    "    r'''\n",
    "    Assumes that preds and targs have a batch dimension, and no class dimension, i.e. (batch, z, x, y)\n",
    "    If the batch dimesion is missing, e.g. your are computing the IOU for a single sample,\n",
    "    add it with e.g. preds[None], targs[None]\n",
    "    '''\n",
    "    \n",
    "    preds = preds.reshape(len(preds), 1, -1).astype(bool)\n",
    "    targs = targs.reshape(len(targs), 1, -1).astype(bool)\n",
    "    \n",
    "    inter = (preds*targs).sum(-1)\n",
    "    union = (preds+targs).sum(-1)\n",
    "    iou = ((inter+smooth)/(union+smooth))  # Small epsilon in case of zero union\n",
    "\n",
    "    if as_mean: iou = iou.mean() \n",
    "    return iou"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ebe7276",
   "metadata": {},
   "source": [
    "## Setting up the models for training\n",
    "\n",
    "The underlying idea is to take a bunch of relatively weak models and have them vote on individual voxels. A voxel is classified as a wall if a majority of these models agree that it is likely a wall."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "45de6909",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch.utils.data import Dataset\n",
    "from torchvision import datasets\n",
    "from torchvision.transforms import ToTensor\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c814fd3",
   "metadata": {},
   "source": [
    "Training can be sped up a lot by utilizing a GPU. Nvidia GPUs require the use of the CUDA version, while AMD GPUs require the ROCm based version. Depending on the specific AMD GPU, additional configurations may be necessary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "26fdff8e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using cpu device\n"
     ]
    }
   ],
   "source": [
    "device = \"cuda\" if torch.cuda.is_available() else \"cpu\"\n",
    "print(f\"Using {device} device\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d2ac9c5",
   "metadata": {},
   "source": [
    "The data set will be enhanced by applying a set of rotations and mirror operations in order to gain additional samples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "aa8f75f8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([105105, 10, 10, 10])\n",
      "torch.Size([840840, 10, 10, 10])\n",
      "torch.Size([1024, 10, 10, 10])\n",
      "torch.Size([1024, 10, 10, 10])\n"
     ]
    }
   ],
   "source": [
    "#import os\n",
    "\n",
    "import torchvision.transforms.functional as tft\n",
    "\n",
    "class CustomImageDataset(Dataset,):\n",
    "    def __init__(self, maps, labels, transform=None, target_transform=None, do_permutations=False):\n",
    "        tempimg_labels = np.asarray(labels,dtype='float32')\n",
    "        self.img_labels = torch.from_numpy(tempimg_labels)\n",
    "        tempmaps = np.asarray(maps,dtype='float32')\n",
    "        self.maps = torch.from_numpy(tempmaps)\n",
    "        self.transform = transform\n",
    "        self.target_transform = target_transform\n",
    "        print(self.maps.shape)\n",
    "        if do_permutations==True:\n",
    "            self.__add_permutations__()\n",
    "        print(self.maps.shape)\n",
    "\n",
    "        \n",
    "    def __add_permutations__(self):\n",
    "        tmpim = torch.zeros((self.maps.shape[0]*4*2,10,10,10))\n",
    "        tmplbl = torch.zeros((self.maps.shape[0]*4*2,10,10,10))\n",
    "        for idx in range(self.maps.shape[0]):\n",
    "            transform = range (0,8)\n",
    "            for x in transform:\n",
    "                image=self.maps[idx]\n",
    "                mask=self.img_labels[idx]\n",
    "                if x > 0 and x < 4:\n",
    "                    image=tft.rotate(image,x*90.)\n",
    "                    mask=tft.rotate(mask,x*90.)\n",
    "                elif x < 6:\n",
    "                    image = tft.hflip(image)\n",
    "                    mask = tft.hflip(mask)\n",
    "                else :\n",
    "                    if x == 5 :\n",
    "                        image=tft.rotate(self.maps[idx],90.)\n",
    "                        mask=tft.rotate(self.img_labels[idx],90.)\n",
    "                    image = tft.vflip(image)\n",
    "                    mask = tft.vflip(mask)\n",
    "                tmpim[idx*4*2+x] = image\n",
    "                tmplbl[idx*4*2+x]=mask\n",
    "        self.maps=tmpim\n",
    "        self.img_labels=tmplbl\n",
    "\n",
    "\n",
    "    def __len__(self):\n",
    "        return len(self.img_labels)\n",
    "\n",
    "    def __getitem__(self, idx):\n",
    "        image = self.maps[idx]\n",
    "        label = self.img_labels[idx]\n",
    "        return image, label\n",
    "\n",
    "training_data=CustomImageDataset(train_x,train_y,do_permutations=True)\n",
    "test_data=CustomImageDataset(valid_x,valid_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "1a1ae848",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch.utils.data import DataLoader\n",
    "from torch.utils.data.dataloader import default_collate\n",
    "\n",
    "learning_rate = 1e-3\n",
    "set_batch_size = 512\n",
    "#The data needs to be uploaded to the device.\n",
    "\n",
    "train_dataloader = DataLoader(training_data, batch_size=set_batch_size, shuffle=True, collate_fn=lambda x: tuple(x_.to(device) for x_ in default_collate(x)))\n",
    "test_dataloader = DataLoader(test_data, batch_size=1024, shuffle=True, collate_fn=lambda x: tuple(x_.to(device) for x_ in default_collate(x)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f10363a",
   "metadata": {},
   "source": [
    "Define the 5 models in use. These were selected in experiments and were the best performing models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d34291ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch import nn\n",
    "import random\n",
    "class NeuralNetwork1(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(NeuralNetwork1, self).__init__()\n",
    "        self.flatten = nn.Flatten()\n",
    "        self.linear_relu_stack = nn.Sequential(\n",
    "            nn.Flatten(),\n",
    "            nn.Linear(1000,1280),\n",
    "            nn.Tanh(),\n",
    "            nn.Linear(1280,1000),\n",
    "            nn.ReLU(),\n",
    "            nn.Unflatten(1,(10,10,10))\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        logits = self.linear_relu_stack(x)\n",
    "        \n",
    "        return logits\n",
    "\n",
    "class NeuralNetwork2(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(NeuralNetwork2, self).__init__()\n",
    "        self.flatten = nn.Flatten()\n",
    "        self.linear_relu_stack = nn.Sequential(\n",
    "            nn.Flatten(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.Tanh(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.ReLU(),\n",
    "            nn.Unflatten(1,(10,10,10))\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        logits = self.linear_relu_stack(x)\n",
    "        \n",
    "        return logits\n",
    "\n",
    "class NeuralNetwork3(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(NeuralNetwork3, self).__init__()\n",
    "        self.flatten = nn.Flatten()\n",
    "        self.linear_relu_stack = nn.Sequential(\n",
    "            nn.Flatten(),\n",
    "            nn.Tanh(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.ReLU(),\n",
    "            nn.Unflatten(1,(10,10,10))\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        logits = self.linear_relu_stack(x)\n",
    "        \n",
    "        return logits\n",
    "    \n",
    "class NeuralNetwork4(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(NeuralNetwork4, self).__init__()\n",
    "        self.flatten = nn.Flatten()\n",
    "        self.linear_relu_stack = nn.Sequential(\n",
    "            nn.Flatten(),\n",
    "            nn.Linear(1000,1100),\n",
    "            nn.Tanh(),\n",
    "            nn.Linear(1100,1000),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.ReLU(),\n",
    "            nn.Unflatten(1,(10,10,10))\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        logits = self.linear_relu_stack(x)\n",
    "        \n",
    "        return logits\n",
    "\n",
    "class NeuralNetwork5(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(NeuralNetwork5, self).__init__()\n",
    "        self.flatten = nn.Flatten()\n",
    "        self.linear_relu_stack = nn.Sequential(\n",
    "            nn.Flatten(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.Tanh(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.Tanh(),\n",
    "            nn.Linear(1000,1000),\n",
    "            nn.ReLU(),\n",
    "            nn.Unflatten(1,(10,10,10))\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        logits = self.linear_relu_stack(x)\n",
    "        \n",
    "        return logits\n",
    "\n",
    "models=[]\n",
    "models.append(NeuralNetwork1().to(device))\n",
    "models.append(NeuralNetwork2().to(device))\n",
    "models.append(NeuralNetwork3().to(device))\n",
    "models.append(NeuralNetwork4().to(device))\n",
    "models.append(NeuralNetwork5().to(device))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "e88f1f73",
   "metadata": {},
   "outputs": [],
   "source": [
    "loss_fn=nn.MSELoss().to(device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "fd3434c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "#from torch.autograd import Variable\n",
    "def train_loop(dataloader, model, loss_fn, optimizer):\n",
    "    size = len(dataloader.dataset)\n",
    "    for batch, (X, y) in enumerate(dataloader):\n",
    "        #Compute prediction and loss\n",
    "        pred = model(X)\n",
    "        loss = loss_fn(pred, y)\n",
    "\n",
    "        # Backpropagation\n",
    "        optimizer.zero_grad()\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "\n",
    "        if batch % (128*4) == 0:\n",
    "            loss, current = loss.item(), batch * len(X)\n",
    "            print(f\"loss: {loss:>7f}  [{current:>5d}/{size:>5d}]\")\n",
    "\n",
    "\n",
    "def test_loop(dataloader, model, loss_fn):\n",
    "    size = len(dataloader.dataset)\n",
    "    num_batches = len(dataloader)\n",
    "    test_loss, correct = 0, 0\n",
    "\n",
    "    with torch.no_grad():\n",
    "        for X, y in dataloader:\n",
    "            pred = model(X)\n",
    "            test_loss += loss_fn(pred>.35, y)#.item()#binary_iou(pred>.35, y).item()\n",
    "\n",
    "    test_loss /= num_batches\n",
    "    print(f\"Test Error: \\n Accuracy: {(100*0):>0.1f}%, Avg loss: {test_loss:>8f} \\n\")\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "305aee88",
   "metadata": {},
   "outputs": [],
   "source": [
    "optimizers = []\n",
    "for model in models:\n",
    "    optimizers.append(torch.optim.Adam(model.parameters(), lr=learning_rate))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "1ba52de4",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.065447  [    0/840840]\n",
      "loss: 0.041031  [262144/840840]\n",
      "loss: 0.034690  [524288/840840]\n",
      "loss: 0.031855  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.045247 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.070994  [    0/840840]\n",
      "loss: 0.042435  [262144/840840]\n",
      "loss: 0.033987  [524288/840840]\n",
      "loss: 0.032516  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.050390 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.067456  [    0/840840]\n",
      "loss: 0.043472  [262144/840840]\n",
      "loss: 0.038825  [524288/840840]\n",
      "loss: 0.035774  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.048554 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.063750  [    0/840840]\n",
      "loss: 0.048516  [262144/840840]\n",
      "loss: 0.041858  [524288/840840]\n",
      "loss: 0.038223  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.051160 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.064755  [    0/840840]\n",
      "loss: 0.048735  [262144/840840]\n",
      "loss: 0.042452  [524288/840840]\n",
      "loss: 0.038862  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.055558 \n",
      "\n",
      "Epoch 2\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.033015  [    0/840840]\n",
      "loss: 0.030006  [262144/840840]\n",
      "loss: 0.030000  [524288/840840]\n",
      "loss: 0.030799  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.045216 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.032020  [    0/840840]\n",
      "loss: 0.031621  [262144/840840]\n",
      "loss: 0.030569  [524288/840840]\n",
      "loss: 0.030149  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.042265 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.036678  [    0/840840]\n",
      "loss: 0.034248  [262144/840840]\n",
      "loss: 0.032696  [524288/840840]\n",
      "loss: 0.032165  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.043764 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.036647  [    0/840840]\n",
      "loss: 0.034408  [262144/840840]\n",
      "loss: 0.031989  [524288/840840]\n",
      "loss: 0.030876  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.043168 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.036947  [    0/840840]\n",
      "loss: 0.036201  [262144/840840]\n",
      "loss: 0.034849  [524288/840840]\n",
      "loss: 0.033581  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.045671 \n",
      "\n",
      "Epoch 3\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.030353  [    0/840840]\n",
      "loss: 0.029339  [262144/840840]\n",
      "loss: 0.030047  [524288/840840]\n",
      "loss: 0.028289  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040994 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.029785  [    0/840840]\n",
      "loss: 0.028629  [262144/840840]\n",
      "loss: 0.029390  [524288/840840]\n",
      "loss: 0.029273  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.043447 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.033601  [    0/840840]\n",
      "loss: 0.032416  [262144/840840]\n",
      "loss: 0.032649  [524288/840840]\n",
      "loss: 0.031578  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.044273 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.029339  [    0/840840]\n",
      "loss: 0.029929  [262144/840840]\n",
      "loss: 0.028660  [524288/840840]\n",
      "loss: 0.027480  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039161 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.033237  [    0/840840]\n",
      "loss: 0.034079  [262144/840840]\n",
      "loss: 0.032188  [524288/840840]\n",
      "loss: 0.030588  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.042825 \n",
      "\n",
      "Epoch 4\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.027566  [    0/840840]\n",
      "loss: 0.028565  [262144/840840]\n",
      "loss: 0.029016  [524288/840840]\n",
      "loss: 0.028492  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.041121 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.029132  [    0/840840]\n",
      "loss: 0.029439  [262144/840840]\n",
      "loss: 0.028803  [524288/840840]\n",
      "loss: 0.028227  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.041490 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.031786  [    0/840840]\n",
      "loss: 0.032059  [262144/840840]\n",
      "loss: 0.031232  [524288/840840]\n",
      "loss: 0.029800  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.041436 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.028035  [    0/840840]\n",
      "loss: 0.027123  [262144/840840]\n",
      "loss: 0.025843  [524288/840840]\n",
      "loss: 0.026070  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037595 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.031952  [    0/840840]\n",
      "loss: 0.030454  [262144/840840]\n",
      "loss: 0.028900  [524288/840840]\n",
      "loss: 0.030169  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.043563 \n",
      "\n",
      "Epoch 5\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.028765  [    0/840840]\n",
      "loss: 0.028704  [262144/840840]\n",
      "loss: 0.028196  [524288/840840]\n",
      "loss: 0.027607  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.041205 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.028415  [    0/840840]\n",
      "loss: 0.028578  [262144/840840]\n",
      "loss: 0.028296  [524288/840840]\n",
      "loss: 0.028180  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040479 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.030316  [    0/840840]\n",
      "loss: 0.030413  [262144/840840]\n",
      "loss: 0.030794  [524288/840840]\n",
      "loss: 0.029840  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.041268 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.024359  [    0/840840]\n",
      "loss: 0.024352  [262144/840840]\n",
      "loss: 0.024847  [524288/840840]\n",
      "loss: 0.025477  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037053 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.031447  [    0/840840]\n",
      "loss: 0.029203  [262144/840840]\n",
      "loss: 0.030674  [524288/840840]\n",
      "loss: 0.030879  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040163 \n",
      "\n",
      "Epoch 6\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.027944  [    0/840840]\n",
      "loss: 0.028252  [262144/840840]\n",
      "loss: 0.028414  [524288/840840]\n",
      "loss: 0.028271  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039516 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.027130  [    0/840840]\n",
      "loss: 0.027914  [262144/840840]\n",
      "loss: 0.028121  [524288/840840]\n",
      "loss: 0.028255  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039493 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.028782  [    0/840840]\n",
      "loss: 0.029509  [262144/840840]\n",
      "loss: 0.029437  [524288/840840]\n",
      "loss: 0.029764  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.041393 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.023803  [    0/840840]\n",
      "loss: 0.023180  [262144/840840]\n",
      "loss: 0.024041  [524288/840840]\n",
      "loss: 0.023742  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.035924 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.029134  [    0/840840]\n",
      "loss: 0.029274  [262144/840840]\n",
      "loss: 0.029243  [524288/840840]\n",
      "loss: 0.028552  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040183 \n",
      "\n",
      "Epoch 7\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.026815  [    0/840840]\n",
      "loss: 0.026494  [262144/840840]\n",
      "loss: 0.029761  [524288/840840]\n",
      "loss: 0.027307  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040123 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.027728  [    0/840840]\n",
      "loss: 0.027437  [262144/840840]\n",
      "loss: 0.027756  [524288/840840]\n",
      "loss: 0.028383  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.043342 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.029170  [    0/840840]\n",
      "loss: 0.029404  [262144/840840]\n",
      "loss: 0.029589  [524288/840840]\n",
      "loss: 0.028968  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039522 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.023341  [    0/840840]\n",
      "loss: 0.022759  [262144/840840]\n",
      "loss: 0.022306  [524288/840840]\n",
      "loss: 0.022724  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.033709 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.028610  [    0/840840]\n",
      "loss: 0.029649  [262144/840840]\n",
      "loss: 0.028344  [524288/840840]\n",
      "loss: 0.027711  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.041190 \n",
      "\n",
      "Epoch 8\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.028005  [    0/840840]\n",
      "loss: 0.028197  [262144/840840]\n",
      "loss: 0.027540  [524288/840840]\n",
      "loss: 0.027357  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.041521 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.025915  [    0/840840]\n",
      "loss: 0.028003  [262144/840840]\n",
      "loss: 0.027696  [524288/840840]\n",
      "loss: 0.026411  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040084 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.029672  [    0/840840]\n",
      "loss: 0.028919  [262144/840840]\n",
      "loss: 0.028978  [524288/840840]\n",
      "loss: 0.028131  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040799 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.022640  [    0/840840]\n",
      "loss: 0.021877  [262144/840840]\n",
      "loss: 0.022800  [524288/840840]\n",
      "loss: 0.022014  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.033461 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.028721  [    0/840840]\n",
      "loss: 0.028158  [262144/840840]\n",
      "loss: 0.027704  [524288/840840]\n",
      "loss: 0.027892  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039694 \n",
      "\n",
      "Epoch 9\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.027482  [    0/840840]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "loss: 0.027989  [262144/840840]\n",
      "loss: 0.028289  [524288/840840]\n",
      "loss: 0.027633  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039522 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.027518  [    0/840840]\n",
      "loss: 0.028254  [262144/840840]\n",
      "loss: 0.027282  [524288/840840]\n",
      "loss: 0.026620  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040362 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.029808  [    0/840840]\n",
      "loss: 0.028768  [262144/840840]\n",
      "loss: 0.028125  [524288/840840]\n",
      "loss: 0.029233  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039412 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.022641  [    0/840840]\n",
      "loss: 0.022730  [262144/840840]\n",
      "loss: 0.021941  [524288/840840]\n",
      "loss: 0.022699  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.033898 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.028492  [    0/840840]\n",
      "loss: 0.028009  [262144/840840]\n",
      "loss: 0.026480  [524288/840840]\n",
      "loss: 0.027272  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039296 \n",
      "\n",
      "Epoch 10\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.026748  [    0/840840]\n",
      "loss: 0.027515  [262144/840840]\n",
      "loss: 0.026980  [524288/840840]\n",
      "loss: 0.027414  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038545 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.027632  [    0/840840]\n",
      "loss: 0.026029  [262144/840840]\n",
      "loss: 0.026869  [524288/840840]\n",
      "loss: 0.028582  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040681 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.026453  [    0/840840]\n",
      "loss: 0.028440  [262144/840840]\n",
      "loss: 0.028093  [524288/840840]\n",
      "loss: 0.027882  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037997 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.021824  [    0/840840]\n",
      "loss: 0.021716  [262144/840840]\n",
      "loss: 0.022132  [524288/840840]\n",
      "loss: 0.022585  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.033515 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.027159  [    0/840840]\n",
      "loss: 0.028613  [262144/840840]\n",
      "loss: 0.027981  [524288/840840]\n",
      "loss: 0.025258  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.040613 \n",
      "\n",
      "Epoch 11\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.027088  [    0/840840]\n",
      "loss: 0.027609  [262144/840840]\n",
      "loss: 0.026395  [524288/840840]\n",
      "loss: 0.026805  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039762 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.027793  [    0/840840]\n",
      "loss: 0.027809  [262144/840840]\n",
      "loss: 0.026980  [524288/840840]\n",
      "loss: 0.028552  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039646 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.028293  [    0/840840]\n",
      "loss: 0.029031  [262144/840840]\n",
      "loss: 0.028442  [524288/840840]\n",
      "loss: 0.028825  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038758 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.021578  [    0/840840]\n",
      "loss: 0.021466  [262144/840840]\n",
      "loss: 0.021043  [524288/840840]\n",
      "loss: 0.021330  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.032184 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.026059  [    0/840840]\n",
      "loss: 0.027229  [262144/840840]\n",
      "loss: 0.026630  [524288/840840]\n",
      "loss: 0.026030  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038734 \n",
      "\n",
      "Epoch 12\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.027262  [    0/840840]\n",
      "loss: 0.027775  [262144/840840]\n",
      "loss: 0.027398  [524288/840840]\n",
      "loss: 0.027414  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.039627 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.026582  [    0/840840]\n",
      "loss: 0.026938  [262144/840840]\n",
      "loss: 0.027628  [524288/840840]\n",
      "loss: 0.026993  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038830 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.028221  [    0/840840]\n",
      "loss: 0.028371  [262144/840840]\n",
      "loss: 0.028329  [524288/840840]\n",
      "loss: 0.027516  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038735 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.020261  [    0/840840]\n",
      "loss: 0.020860  [262144/840840]\n",
      "loss: 0.021189  [524288/840840]\n",
      "loss: 0.021208  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.032603 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.025816  [    0/840840]\n",
      "loss: 0.026666  [262144/840840]\n",
      "loss: 0.026825  [524288/840840]\n",
      "loss: 0.025248  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038001 \n",
      "\n",
      "Epoch 13\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.026587  [    0/840840]\n",
      "loss: 0.026010  [262144/840840]\n",
      "loss: 0.026900  [524288/840840]\n",
      "loss: 0.026539  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038562 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.027291  [    0/840840]\n",
      "loss: 0.027146  [262144/840840]\n",
      "loss: 0.026945  [524288/840840]\n",
      "loss: 0.026567  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038376 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.027988  [    0/840840]\n",
      "loss: 0.028422  [262144/840840]\n",
      "loss: 0.027832  [524288/840840]\n",
      "loss: 0.027265  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038796 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.020103  [    0/840840]\n",
      "loss: 0.021143  [262144/840840]\n",
      "loss: 0.021560  [524288/840840]\n",
      "loss: 0.019806  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.032110 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.025575  [    0/840840]\n",
      "loss: 0.025369  [262144/840840]\n",
      "loss: 0.025340  [524288/840840]\n",
      "loss: 0.025573  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037529 \n",
      "\n",
      "Epoch 14\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.027230  [    0/840840]\n",
      "loss: 0.026527  [262144/840840]\n",
      "loss: 0.027235  [524288/840840]\n",
      "loss: 0.026795  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038517 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.026858  [    0/840840]\n",
      "loss: 0.026921  [262144/840840]\n",
      "loss: 0.026469  [524288/840840]\n",
      "loss: 0.028097  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038295 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.027442  [    0/840840]\n",
      "loss: 0.027935  [262144/840840]\n",
      "loss: 0.028199  [524288/840840]\n",
      "loss: 0.027765  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038360 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.020099  [    0/840840]\n",
      "loss: 0.021131  [262144/840840]\n",
      "loss: 0.021134  [524288/840840]\n",
      "loss: 0.021816  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.030619 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.024459  [    0/840840]\n",
      "loss: 0.024760  [262144/840840]\n",
      "loss: 0.024803  [524288/840840]\n",
      "loss: 0.024691  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037526 \n",
      "\n",
      "Epoch 15\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.026354  [    0/840840]\n",
      "loss: 0.026556  [262144/840840]\n",
      "loss: 0.027062  [524288/840840]\n",
      "loss: 0.025766  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038984 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.027087  [    0/840840]\n",
      "loss: 0.026403  [262144/840840]\n",
      "loss: 0.026745  [524288/840840]\n",
      "loss: 0.025124  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038727 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.027536  [    0/840840]\n",
      "loss: 0.028073  [262144/840840]\n",
      "loss: 0.028044  [524288/840840]\n",
      "loss: 0.028723  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037662 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.020301  [    0/840840]\n",
      "loss: 0.020490  [262144/840840]\n",
      "loss: 0.019243  [524288/840840]\n",
      "loss: 0.018572  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.031897 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.024249  [    0/840840]\n",
      "loss: 0.024264  [262144/840840]\n",
      "loss: 0.025000  [524288/840840]\n",
      "loss: 0.024129  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.035832 \n",
      "\n",
      "Epoch 16\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.026864  [    0/840840]\n",
      "loss: 0.026991  [262144/840840]\n",
      "loss: 0.026651  [524288/840840]\n",
      "loss: 0.026504  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038726 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.026083  [    0/840840]\n",
      "loss: 0.026404  [262144/840840]\n",
      "loss: 0.026456  [524288/840840]\n",
      "loss: 0.026025  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038597 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.027508  [    0/840840]\n",
      "loss: 0.028056  [262144/840840]\n",
      "loss: 0.027198  [524288/840840]\n",
      "loss: 0.027296  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037243 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.020447  [    0/840840]\n",
      "loss: 0.020507  [262144/840840]\n",
      "loss: 0.020697  [524288/840840]\n",
      "loss: 0.020447  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.031647 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.024808  [    0/840840]\n",
      "loss: 0.024590  [262144/840840]\n",
      "loss: 0.024926  [524288/840840]\n",
      "loss: 0.024881  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.036227 \n",
      "\n",
      "Epoch 17\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.026639  [    0/840840]\n",
      "loss: 0.026110  [262144/840840]\n",
      "loss: 0.025605  [524288/840840]\n",
      "loss: 0.026497  [786432/840840]\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038358 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.026385  [    0/840840]\n",
      "loss: 0.026603  [262144/840840]\n",
      "loss: 0.026460  [524288/840840]\n",
      "loss: 0.025380  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038395 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.027722  [    0/840840]\n",
      "loss: 0.027579  [262144/840840]\n",
      "loss: 0.029380  [524288/840840]\n",
      "loss: 0.028414  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037619 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.019771  [    0/840840]\n",
      "loss: 0.020993  [262144/840840]\n",
      "loss: 0.020927  [524288/840840]\n",
      "loss: 0.020050  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.031561 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.024307  [    0/840840]\n",
      "loss: 0.023732  [262144/840840]\n",
      "loss: 0.023628  [524288/840840]\n",
      "loss: 0.024026  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.035356 \n",
      "\n",
      "Epoch 18\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.025938  [    0/840840]\n",
      "loss: 0.025926  [262144/840840]\n",
      "loss: 0.027518  [524288/840840]\n",
      "loss: 0.026775  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038791 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.025799  [    0/840840]\n",
      "loss: 0.026179  [262144/840840]\n",
      "loss: 0.025800  [524288/840840]\n",
      "loss: 0.026103  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037475 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.026717  [    0/840840]\n",
      "loss: 0.027628  [262144/840840]\n",
      "loss: 0.026336  [524288/840840]\n",
      "loss: 0.027088  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.036938 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.019672  [    0/840840]\n",
      "loss: 0.018834  [262144/840840]\n",
      "loss: 0.019302  [524288/840840]\n",
      "loss: 0.020783  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.031471 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.023558  [    0/840840]\n",
      "loss: 0.024777  [262144/840840]\n",
      "loss: 0.024074  [524288/840840]\n",
      "loss: 0.024346  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.034308 \n",
      "\n",
      "Epoch 19\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.025619  [    0/840840]\n",
      "loss: 0.025344  [262144/840840]\n",
      "loss: 0.026214  [524288/840840]\n",
      "loss: 0.027491  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.036802 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.026297  [    0/840840]\n",
      "loss: 0.027631  [262144/840840]\n",
      "loss: 0.027107  [524288/840840]\n",
      "loss: 0.026468  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037714 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.026154  [    0/840840]\n",
      "loss: 0.028833  [262144/840840]\n",
      "loss: 0.027350  [524288/840840]\n",
      "loss: 0.027873  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037217 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.019414  [    0/840840]\n",
      "loss: 0.020194  [262144/840840]\n",
      "loss: 0.019619  [524288/840840]\n",
      "loss: 0.020346  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.031703 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.023720  [    0/840840]\n",
      "loss: 0.024421  [262144/840840]\n",
      "loss: 0.022996  [524288/840840]\n",
      "loss: 0.023302  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.034332 \n",
      "\n",
      "Epoch 20\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.025957  [    0/840840]\n",
      "loss: 0.025918  [262144/840840]\n",
      "loss: 0.025403  [524288/840840]\n",
      "loss: 0.026041  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037204 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.025159  [    0/840840]\n",
      "loss: 0.024979  [262144/840840]\n",
      "loss: 0.026370  [524288/840840]\n",
      "loss: 0.025599  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037680 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.027972  [    0/840840]\n",
      "loss: 0.028090  [262144/840840]\n",
      "loss: 0.027081  [524288/840840]\n",
      "loss: 0.027054  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037288 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.019247  [    0/840840]\n",
      "loss: 0.019897  [262144/840840]\n",
      "loss: 0.019710  [524288/840840]\n",
      "loss: 0.019617  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.031121 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.022781  [    0/840840]\n",
      "loss: 0.021973  [262144/840840]\n",
      "loss: 0.022978  [524288/840840]\n",
      "loss: 0.023151  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.034063 \n",
      "\n",
      "Epoch 21\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.024912  [    0/840840]\n",
      "loss: 0.024381  [262144/840840]\n",
      "loss: 0.025522  [524288/840840]\n",
      "loss: 0.025270  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.036929 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.025929  [    0/840840]\n",
      "loss: 0.026597  [262144/840840]\n",
      "loss: 0.025793  [524288/840840]\n",
      "loss: 0.026019  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037480 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.027643  [    0/840840]\n",
      "loss: 0.028213  [262144/840840]\n",
      "loss: 0.026893  [524288/840840]\n",
      "loss: 0.027223  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.036095 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.018578  [    0/840840]\n",
      "loss: 0.019458  [262144/840840]\n",
      "loss: 0.019775  [524288/840840]\n",
      "loss: 0.019940  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.030530 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.023718  [    0/840840]\n",
      "loss: 0.022408  [262144/840840]\n",
      "loss: 0.022946  [524288/840840]\n",
      "loss: 0.022358  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.032952 \n",
      "\n",
      "Epoch 22\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.025313  [    0/840840]\n",
      "loss: 0.026415  [262144/840840]\n",
      "loss: 0.027037  [524288/840840]\n",
      "loss: 0.025590  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.038101 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.025333  [    0/840840]\n",
      "loss: 0.024852  [262144/840840]\n",
      "loss: 0.026026  [524288/840840]\n",
      "loss: 0.026391  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.036162 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.027017  [    0/840840]\n",
      "loss: 0.027737  [262144/840840]\n",
      "loss: 0.028167  [524288/840840]\n",
      "loss: 0.027524  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.036295 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.019171  [    0/840840]\n",
      "loss: 0.019256  [262144/840840]\n",
      "loss: 0.019362  [524288/840840]\n",
      "loss: 0.019506  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.030919 \n",
      "\n",
      "Training model 5\n",
      "loss: 0.022700  [    0/840840]\n",
      "loss: 0.022240  [262144/840840]\n",
      "loss: 0.022449  [524288/840840]\n",
      "loss: 0.022693  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.035329 \n",
      "\n",
      "Epoch 23\n",
      "-------------------------------\n",
      "Training model 1\n",
      "loss: 0.024897  [    0/840840]\n",
      "loss: 0.025191  [262144/840840]\n",
      "loss: 0.024918  [524288/840840]\n",
      "loss: 0.025275  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037000 \n",
      "\n",
      "Training model 2\n",
      "loss: 0.024765  [    0/840840]\n",
      "loss: 0.026263  [262144/840840]\n",
      "loss: 0.025689  [524288/840840]\n",
      "loss: 0.025406  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037382 \n",
      "\n",
      "Training model 3\n",
      "loss: 0.028284  [    0/840840]\n",
      "loss: 0.027951  [262144/840840]\n",
      "loss: 0.027702  [524288/840840]\n",
      "loss: 0.026972  [786432/840840]\n",
      "Test Error: \n",
      " Accuracy: 0.0%, Avg loss: 0.037061 \n",
      "\n",
      "Training model 4\n",
      "loss: 0.018704  [    0/840840]\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "Input \u001b[0;32mIn [24]\u001b[0m, in \u001b[0;36m<cell line: 3>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[38;5;28;01mfor\u001b[39;00m i,model \u001b[38;5;129;01min\u001b[39;00m \u001b[38;5;28menumerate\u001b[39m(models):\n\u001b[1;32m      6\u001b[0m         \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mTraining model \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mi\u001b[38;5;241m+\u001b[39m\u001b[38;5;241m1\u001b[39m\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m----> 7\u001b[0m         \u001b[43mtrain_loop\u001b[49m\u001b[43m(\u001b[49m\u001b[43mtrain_dataloader\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mmodel\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mloss_fn\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43moptimizers\u001b[49m\u001b[43m[\u001b[49m\u001b[43mi\u001b[49m\u001b[43m]\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m      8\u001b[0m         test_loop(test_dataloader, model, loss_fn)\n\u001b[1;32m      9\u001b[0m \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mDone!\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n",
      "Input \u001b[0;32mIn [22]\u001b[0m, in \u001b[0;36mtrain_loop\u001b[0;34m(dataloader, model, loss_fn, optimizer)\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[38;5;66;03m# Backpropagation\u001b[39;00m\n\u001b[1;32m     10\u001b[0m optimizer\u001b[38;5;241m.\u001b[39mzero_grad()\n\u001b[0;32m---> 11\u001b[0m \u001b[43mloss\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mbackward\u001b[49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\n\u001b[1;32m     12\u001b[0m optimizer\u001b[38;5;241m.\u001b[39mstep()\n\u001b[1;32m     14\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m batch \u001b[38;5;241m%\u001b[39m (\u001b[38;5;241m128\u001b[39m\u001b[38;5;241m*\u001b[39m\u001b[38;5;241m4\u001b[39m) \u001b[38;5;241m==\u001b[39m \u001b[38;5;241m0\u001b[39m:\n",
      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/torch/_tensor.py:396\u001b[0m, in \u001b[0;36mTensor.backward\u001b[0;34m(self, gradient, retain_graph, create_graph, inputs)\u001b[0m\n\u001b[1;32m    387\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m has_torch_function_unary(\u001b[38;5;28mself\u001b[39m):\n\u001b[1;32m    388\u001b[0m     \u001b[38;5;28;01mreturn\u001b[39;00m handle_torch_function(\n\u001b[1;32m    389\u001b[0m         Tensor\u001b[38;5;241m.\u001b[39mbackward,\n\u001b[1;32m    390\u001b[0m         (\u001b[38;5;28mself\u001b[39m,),\n\u001b[0;32m   (...)\u001b[0m\n\u001b[1;32m    394\u001b[0m         create_graph\u001b[38;5;241m=\u001b[39mcreate_graph,\n\u001b[1;32m    395\u001b[0m         inputs\u001b[38;5;241m=\u001b[39minputs)\n\u001b[0;32m--> 396\u001b[0m \u001b[43mtorch\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mautograd\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mbackward\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgradient\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mretain_graph\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreate_graph\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43minputs\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[43minputs\u001b[49m\u001b[43m)\u001b[49m\n",
      "File \u001b[0;32m~/.local/lib/python3.10/site-packages/torch/autograd/__init__.py:173\u001b[0m, in \u001b[0;36mbackward\u001b[0;34m(tensors, grad_tensors, retain_graph, create_graph, grad_variables, inputs)\u001b[0m\n\u001b[1;32m    168\u001b[0m     retain_graph \u001b[38;5;241m=\u001b[39m create_graph\n\u001b[1;32m    170\u001b[0m \u001b[38;5;66;03m# The reason we repeat same the comment below is that\u001b[39;00m\n\u001b[1;32m    171\u001b[0m \u001b[38;5;66;03m# some Python versions print out the first line of a multi-line function\u001b[39;00m\n\u001b[1;32m    172\u001b[0m \u001b[38;5;66;03m# calls in the traceback and some print out the last line\u001b[39;00m\n\u001b[0;32m--> 173\u001b[0m \u001b[43mVariable\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_execution_engine\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mrun_backward\u001b[49m\u001b[43m(\u001b[49m\u001b[43m  \u001b[49m\u001b[38;5;66;43;03m# Calls into the C++ engine to run the backward pass\u001b[39;49;00m\n\u001b[1;32m    174\u001b[0m \u001b[43m    \u001b[49m\u001b[43mtensors\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mgrad_tensors_\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mretain_graph\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mcreate_graph\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43minputs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[1;32m    175\u001b[0m \u001b[43m    \u001b[49m\u001b[43mallow_unreachable\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43maccumulate_grad\u001b[49m\u001b[38;5;241;43m=\u001b[39;49m\u001b[38;5;28;43;01mTrue\u001b[39;49;00m\u001b[43m)\u001b[49m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "epochs = 2000\n",
    "\n",
    "for t in range(epochs):\n",
    "    print(f\"Epoch {t+1}\\n-------------------------------\")#\n",
    "    for i,model in enumerate(models): \n",
    "        print(f\"Training model {i+1}\")\n",
    "        train_loop(train_dataloader, model, loss_fn, optimizers[i])\n",
    "        test_loop(test_dataloader, model, loss_fn)\n",
    "print(\"Done!\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "bac683e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "with torch.no_grad():\n",
    "    pred=[]\n",
    "    for model in models:\n",
    "        dataiter = iter(test_dataloader)\n",
    "        X, y = dataiter.next()\n",
    "        #m=nn.Unflatten()\n",
    "        pred.append(model(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4336415f",
   "metadata": {},
   "source": [
    "We can now visualize the result for a randomly chosen sample from the test dataset. This shows the output of the vote on the left and the ground truth on the right."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "cc101415",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([1024, 10, 10, 10])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAc8AAAMyCAYAAAAL60rFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAB0OUlEQVR4nO39fdxcZZ3n+36+SUjzFMABZ48kIGQEAqKEkI6Ac9ICOgbhgPYeZqMiaDdGRwSinlH20H32YdrhNLtVoB1HjTz4gI12kN7mgINEW+PI5ikg0WBQEdHchCfbMeHBV0Ny/84f67pD5c6qqlV116q7rqrv+/WqV6rWWtdaVyX5Xletp2spIjAzM7PqZkx3BczMzHLjztPMzKxD7jzNzMw65M7TzMysQ+48zczMOuTO08zMrEOzprsCzbxpxlm+hwZYM75K010Hs6lynp3lYeM9TzMzsw7VtucpaQFwJjAXCGAzsDoiNta1TTPrPWfZbFe17HlK+hjwNUDAPcC96f2Nki5pUW65pHWS1o3FI3VUzcw60G2WU1nn2YaW6hieT9LPgVdHxIuTps8GHoyIw9qtw+dICj5PYtOpF1kG5xmc5WFT1znPceDAkumvSPPMLA/OslmJus55rgC+K+kXwKY07WDgVcAHa9qmmfXeCpxls13U0nlGxG2SDgeWUFxkIGAMuDcittexTTPrPWfZrFxtV9tGxDhwV13rN7P+cJbNduX7PM3MzDrkztPMzKxD7jzNzMw65M7TzMysQ+48zczMOuTO08zMrEO1dZ6SFkg6RdLek6Yvq2ubZtZ7zrLZrlp2npL2lvTvJH1I0oWSlklq2+FKugj4JnAhsEHSmQ2zL29RzgNJmw2QbrOcyjrPNrSadoSS/j3wPWAZxTBcS4B3AQ9Iek2b9b4XOC4i3gq8AfhLSRdPrLpZoYhYGRGLI2LxPM2v/CXMrDZdZRmcZxturUYY+gvg+Ih4XtIBwFcj4s2SXgt8HjixRdmZEfEsQEQ8KukNwE2SXkmbwJnZQHGWzUq0OgQr4A/p/XPAvwSIiB8D+7RZ7xOSFk58SOE7HTgAaLfXamaDw1k2K9Fqz/NbwG2S1gKnAqsAJP0L2v/iPBfY1jghIrYB50r6fPfVNbM+c5bNSjTtPCPiY5LeAhwF/OeIWJNm/R5Y1GqlETHWYt4dXdTTzKaBs2xWruVTVSLiWxR7oI3TxoF/rrNSZmZmg8yDJJiZmXXInaeZmVmH2g2SMFPSDf2qjJmZWQ5adp4RsR14uaTZfaqPmZnZwGt5wVDyKHCHpNUU93sCEBGfqqtSZmZmg6zKOc/NwC1p2TkNr45I+nKnZcxs8DjLZhX2PCPiMgBJe0XEc+2WT8uunjwJOEnSfmmdZzQptxxYDrCARXg8TLPp1W2WU1nn2YZW285T0gnAtcDewMGSjgHeFxEfaFFsHvBT4BogKAK3GPhkq21FxEpgJcCbZpwVVb6AmdWqqyyD82zDrcph26uANwP/BBAR64GlbcosBu4DLgW2RMT3gT9ExNqIWNt1bc2s35xlsxJVLhgiIjZJOw1nu73N8uPAlZJWpT+frLotMxsczrJZuSoh2CTpRCDSLSsXARurrDyNi3mWpNOArd1X08ymk7NstrMqnef7gauBucAYcDtwQScbiYhbgVs7rp2ZDRRn2axQpfMcj4h31l4TMzOzTFS5YOhuSasknapJJz7NzMxGUZXO83CKy83PBR6WdLmkw+utlpmZ2eBq23lGYU1EvB04HzgPuEfS2nQPqJmZ2UipMkjC/sA5wLuAJ4ELgdXAQmAVcGiN9TMzMxs4VS4YuhP4CvDWdLn6hHWSPldPtczMzAZXlc7ziIgoHVorIq4omy7pdcDGiNgqaQ/gEmARxTBfl0fElm4rbGb94yyblatywdABkv5G0rck/ePEq02Z64Dn0/urgX2BK9K065sVkrRc0jpJ68bikSr1N7N6dZVlcJ5tuFXZ8/wq8HXgdIoBE84Dnm5TZkZEbEvvF0fEovT+h5IeaFbIA0mbDZyusgzOsw23Knue+0fEtcCLaTDoPwOOb1Nmg6T3pPfrJS0GSLe4vNh9dc2sz5xlsxJVOs+JgDwu6TRJx1I8pqiV84E/kfRL4CjgTkmPAF9I88wsD86yWYkqh20/Lmlf4CPAp4F9gBWtCqSLCN4taQ4wP21nLCKenFp1zayfnGWzcm07z4i4Jb3dApwEIGlFlZVHxDPA+m4rZ2aDwVk221mVw7ZlPtzTWpiZmWWk287TA8SbmdnI6rbz9GXnZmY2spqe85T0DOWdpIA9aquRmZnZgGvaeUbEnH5WxMzMLBfdHrY1MzMbWVXu8+yYpNnA2cDmiPiOpHcAJwIbgZUR4ZFJzDLgLJuVq6XzpBgwehawp6TzgL2Bm4FTgCUU4+PuQtJyYDnAAhYxT/Nrqp6ZVdRVlsF5tuGmJk8bm9pKpR9HxGslzQIeAw6MiO2SBKyPiNe2W4cHki6sGV/l24Js2vQiy+A8g7M8bOo65zkjHe6ZA+xJ8RgjgD8Cdqtpm2bWe86yWYm6DtteCzwEzAQuBValwaSPB75W0zbNrPecZbMStRy2BZB0IEBEbJa0H/BG4DcRcU+V8j7MU/ChHptuU80yOM/gLA+buvY8iYjNDe9/D9xU17bMrD7OstmufJ+nmZlZh9x5mpmZdcidp5mZWYfceZqZmXUqIgb+BSzv5bxcy/jl1zC8BiFL013Gr/xfuex5Lu/xvFzLmA2DQcjSdJexzOXSeZqZmQ0Md55mZmYdyqXzXNnjebmWMRsGg5Cl6S5jmatteD4zM7Nhlcuep5mZ2cBw52lmZtahge88JS2T9DNJD0u6pGH6dZKekrRh0vIHSfqepI2SHpR0ccO83SXdI2l9mnfZpLIzJf1I0i2Tpj8q6SeSHpC0rmH6fpJukvRQ2t4JafoRadmJ11ZJK9K8D6Vtb5B0o6TdG9Z3cZr+4MTyZsOi0yyneaV57nWW07xd8twqy6lMaZ6d5REw3TeatnpRPEPwl8B8YDawHjgqzVsKLAI2TCrzCmBRej8H+HlDGQF7p/e7AXcDxzeU/TDwd8Atk9b5KHBASf2+BJyf3s8G9mvyHZ4AXgnMBX4F7JHm/T3w7vT+aGADxQOHZwHfAQ6b7n8Dv/zqxaubLKd5pXnudZbTvJZ5bsxy+lyaZ2d5NF6Dvue5BHg4Ih6JiBcoHr57JkBE/AD43eQCEfF4RNyf3j8DbKT4T04Unk2L7pZeRa8qzQNOA66pUjFJ+1CE/tq07heieFzTZKcAv4yIX6fPs4A9JM2iCNfE456OBO6KiOcjYhuwFnhblbqYZaDjLKd5pXnuZZZTmSp5npxlKM+zszwCBr3znAtsavg8lqZVIukQ4FiKX6UT02ZKegB4ClgTERPzrgI+CoyXrCqA2yXdJ2lixJD5wNPA9enw0DWS9iopezZwI0BEPAZ8AvgN8DiwJSJuT8ttAJZK2l/SnsBbgIOqflezATelLMOuee5hlqFanndkGVrm2VkeAYPeeZY9eb3SvTWS9ga+AayIiK07Ckdsj4iFwDxgiaSjJZ0OPBUR9zVZ3esjYhFwKnCBpKUUvzgXAZ+NiGOB54BLGgtJmg2cAaxKn19G8Wv7UOBAYC9J56R6bQSuANYAt1Ec1tpW5buaZaDrLEN5nnuYZWiT58lZTtNK8+wsj4aBvc/zTTPOGsyK9dma8VVljY5ZVpxnZ3nYDPqep5mZ2cCZVdeKJS2gOKQxl+LwzGZgdTqkYWaZcJbNdlXLnqekj1FcTSfgHuDe9P7Gxvu7zGywOctm5Wo55ynp58CrI+LFSdNnAw9GxGFNyi0nPf9uAYuOm6f5Pa9bbnyexKZTt1lOyzjPDZzl4VLXOc9xiqvPJnsF5ZePAxARKyNicUQsHvWgmQ2IrrIMzrMNt7rOea4AvivpF7x0b9fBwKuAD9a0TTPrvRU4y2a7qKXzjIjbJB1OMarIXIpzJGPAvRGxvY5tmlnvOctm5Wq72jYixoG76lq/mfWHs2y2K9/naWZm1iF3nmZmZh1y52lmZtYhd55mZmYdcudpZmbWIXeeZmZmHaqt85S0QNIp6Tl8jdOX1bVNM+s9Z9lsVy07T0lLJR2R3v8bSf8vSae1W6mki4BvAhcCGySd2TD78qlU2Mz6x1k2K9d0kARJV1GMKjJL0reBU4D/DnxI0hsi4j+2WO97geMi4llJhwA3STokIq6m/InyE9tsHEgaj4dpNu26yjI4zzbcmj5VRdKDwNHAHsBjwNyIeF7SbsCPIuLopiuVfhoRRzV83hu4CfgpcHJELGxXMT95vuAnMdh06kWWwXkGZ3nYtDpsG1H0rBNPTpj4zz/ephzAE5IWNqzoWeB04ADgNd1V1cymgbNsVqLV2La3SvofwO7ANcDfS7oL+BPgB23Wey6wrXFCRGwDzpX0+SnU18z6y1k2K9G084yIj0k6oXgbd0n618DbKDrSm1qtNCLGWsy7o9vKmll/Octm5Vo+VSUi7mx4/0vgE7XXyMzMbMB5kAQzM7MOufM0MzPrULtBEmZKuqFflTEzM8tBy84zIrYDL5c0u0/1MTMzG3gtLxhKHgXukLQaeG5iYkR8qq5KmZmZDbIq5zw3A7ekZec0vDoi6cudljGzweMsm1XY84yIywAk7RURz7VbPi27evIk4CRJ+6V1ntFhPc1sGjjLZuXadp5poIRrgb2BgyUdA7wvIj7Qotg8irEvr6EY1k/AYuCTbbblgaTNBktXWQbn2YZblcO2VwFvBv4JICLWA0vblFkM3AdcCmyJiO8Df4iItRGxtlmhiFgZEYsjYrGDZjYQusoyOM823KpcMEREbJJ2eiDA9jbLjwNXSlqV/nyy6rbMbHA4y2blqoRgk6QTgUi3rFwEbKyy8jQu5lnpAdpbu6+mmU0nZ9lsZ1U6z/cDVwNzgTHgduCCTjYSEbcCt3ZcOzMbKM6yWaFK5zkeEe+svSZmZmaZqHLB0N2SVkk6VZNOfJqZmY2iKp3n4cBKiofiPizpckmH11stMzOzwdW284zCmoh4O3A+cB5wj6S16R5QMzOzkVJlkIT9gXOAdwFPAhcCq4GFwCrg0BrrZ2ZmNnCqXDB0J/AV4K3pcvUJ6yR9rp5qmZmZDa4qnecRERFlMyLiih7Xx8zMbOBV6TwPkPRR4NXA7hMTI+LkZgUkvQ7YGBFbJe0BXAIsohgj8/KI2DK1aptZPzjLZuWqXG37VeAhinObl1E83/PeNmWuA55P768G9gWuSNOub1ZI0nJJ6yStG4tHKlTNzGrWVZbBebbhpiZHZF9aQLovIo6T9OOIeG2atjYi/qRFmY0RcWR6f39ELGqY90BELGxXsTfNOKt1xUbEmvFVvrfWpk0vsgzOMzjLw6bKnueL6c/HJZ0m6ViKxxS1skHSe9L79ZIWA6T7Q19sXszMBoyzbFaiyjnPj0vaF/gI8GlgH2BFmzLnA1dL+gvgt8CdkjYBm9I8M8uDs2xWom3nGRG3pLdbgJMAJK1oU2YL8G5Jc4D5aTtjEfHklGprZn3lLJuV6/a5fB+meEh2SxHxDLC+y22Y2YBwls12VuWcZxmf+DYzs5HVbec58lfOmZnZ6GraeUp6RtLWktczwIF9rKNZxyRdJ+kpSRuazJekv5X0sKQfS1pUtpyZ5a2utqBp5xkRcyJin5LXnIjo9lypWb98EVjWYv6pwGHptRz4bB/qZGb990VqaAu6PWxrNtAi4gfA71oscibw5fTIvbuA/SS9oj+1M7N+qastcOdpo2ouxb2KE8bSNDMbLV21BbUcfpU0Gzgb2BwR35H0DuBEYCOwMiI8MskIG3/i8ClfcDbzFb94H8UhlgkrI2JlB6sou2LcF8JN4ixbnXJuC+o6d3l9Wveeks4D9gZuBk4BlgDnlRWStJz0l7CARczT/JqqZ7lL4egkIJONAQc1fJ4HbJ5SpYZTV1kG59n6Y7ragro6z9dExGslzQIeAw6MiO2SbqDFjdaNfwkeSHp4jTM+5XX04HzDauCDkr4GvA7YEhGPT321Q6erLIPzbO3l3BbU1XnOSId79gL2pHiM0e+APwJ2q2mbZjtIuhF4A8XzaMeA/4P0fy8iPgd8C3gL8DDF47XeU76mkecsW9bqagvq6jyvpXgG6EzgUmCVpEeA44Gv1bRNy8T2mPqvzXb/cSPi7W3mB3DBlCsy/Jxlq03ObUEtnWdEXCnp6+n9ZklfBt4IfCEi7qljm2bWe86yWbnaBjuIiM0N738P3FTXtiwv476oNSvOstUl57bA93mamZl1yMPsWd/14go7M8tfzm2B9zzNzMw65D1P67vtke95DjPrnZzbAkUGlZe0vNlwS93My7XMsNi6+eAp/6fb58Df+IHsmRqELE13GSvk3Bbkcth2eY/n5VpmKIwTU35Z1gYhS9Ndxsi7Lcil8zQzMxsYPudpfbfde45mRt5tQS57nq3OGXQzL9cyZsNgELI03WUsc1lcMGTD5enNc6f8n+7lBz7mC4bMMpdzW5DLnqeZmdnAGPjOU9IyST+T9LCkSxqmXyfpKUkbJi1/kKTvSdoo6UFJFzfM213SPZLWp3mXTSo7U9KPJN0yafqjkn4i6QFJ6xqm7yfpJkkPpe2dkKYfkZadeG2VtCLN+1Da9gZJN0ravWF9F6fpD04sP4y2R0z5ZfnpNMtpXmmee53lNG+XPLfKcipTmudRyfJU5dwWDPRhW0kzgZ8Db6J42ve9wNsj4qeSlgLPAl+OiKMbyrwCeEVE3C9pDnAf8NZURsBeEfGspN2AHwIXR8RdqeyHgcXAPhFxesM6HwUWR8RvJ9XvS8D/iIhrVDzzcM80cPbk7/AYxUNWt6VtHhURf5D098C3IuKLko6meMTTEuAF4DbgP0TEL6b8FzlgnnjswCn/p/tXczf7sG1GuslyKleaZ2AjPcxymtcyz41ZjohfS5pLSZ6BdYxIlqcq57Zg0Pc8lwAPR8QjEfECxX/IMwEi4gcUD+XdSUQ8HhH3p/fPUIRsbvocEfFsWnS39AoASfOA04BrqlRM0j7AUornHRIRL0zuOJNTgF9GxK/T51nAHpJmUTxceOKJFUcCd0XE8xGxDVgLvK1KXXIz3oOXZafjLKd5pXnuZZZTmSp5npxlKM/zyGR5qnJuCwa985wLbGr4PJamVSLpEOBY4O6GaTMlPQA8BayJiIl5VwEfpfzfI4DbJd0naeKm5/nA08D16fDQNZL2Kil7NnAjQEQ8BnwC+A3wOLAlIm5Py20AlkraX9KeFE82P6jqdzUbcFPKMuya5x5mGarleUeWoWWeneURMOidZ9nueKXdfEl7A98AVkTE1h2FI7ZHxEJgHrBE0tGSTgeeioj7mqzu9RGxCDgVuCAdZpoFLAI+GxHHAs8BlzQWSod+zgBWpc8vo/i1fShwILCXpHNSvTYCVwBrKA7zrKc4zDt0thNTfll2us4ylOe5h1mGNnmenOU0rTTPo5Tlqcq5LRj0znOMnX+xzeOlw5xNpXMg3wC+GhE3ly2TDsl8H1gGvB44I50P+RpwsqQbGpbdnP58CvgHikNQY8BYw6/dmyjC1+hU4P6IeDJ9fiPwq4h4OiJeBG4GTmzYzrURsSgillIcxvI5EhsWXWUZ2ue5B1meqF+rPE/OMrTIs7M8/Ab2gqE3zThrMCvWZ2vGVw3dhTGPjr1iyv+2h8x7fOj+XoaZ8zycWZ6qnNuCQd/zNDMzGzi1jW0raQHF+YC5FOc2NgOr0/kAG2G+WjYvzrLVJee2oJY9T0kfozjfIOAeinu6BNzYeHN0SbnlktZJWjcWj9RRNTPrQLdZTmWdZxtatZzzlPRz4NXpJHrj9NnAgxFxWLt1+BxJYRjPk/xibOo3Rh82z4Mk9EMvsgzOMwxnlqcq57agrnOe4xSXbk/2CvLeUzcbNc6yWYm6znmuAL4r6Re8dGP0wcCrgA/WtE3LxPjI74NkZQXOstUk57agls4zIm6TdDjFPVRzKc6RjAH3RsT2OrZpZr3nLJuVq+1q24gYB+6qa/2Wr+2lg83YoHKWrS45twW+z9PMzKxDte15mjWT869NM+udnNsC73mamZl1yHue1nfjke+vTTPrnZzbAu95mpmZdch7ntZ3OZ/nMLPeybktqG3PU9ICSaekh9g2Tl9W1zbNrPecZbNdNe08JZ0hafduVirpIuCbwIXABklnNsy+vEU5DyQ9ArYzY8ov649us5zKOs/WUs5tQastfx0Yk/QVSW+RNLOD9b4XOC4i3gq8AfhLSReneU330yNiZUQsjojF8zS/g82ZWU26yjI4zzbcWp3zfAg4Gfh3wEeA6yX9A3BjRKxts96ZEfEsQEQ8KukNwE2SXkmbwNnwy/kKuxHkLFttcm4LWu15RkT8z4j4QkScAhwD/BT4a0mbWpQDeELSwoYVPQucDhwAvGaKdTaz/nGWzUq02vPc6SdBRDwB/C3wt+lXZyvnAtsmld8GnCvp891U1IZHzlfYjSBn2WqTc1vQqvP8ULMZEfHrViuNiLEW8+6oUC8zGwDOslm5pp1nRHy/j/WwEbI9fLWsmeXdFuRbczMzs2niEYas78b9m83MyLstaFlzSTMl3dCvypiZmeWg5Z5nRGyX9HJJsyPihX5VyoZbzlfY2fT79ub1Tee9+cBj+lgTm6qc24Iqh20fBe6QtBp4bmJiRHyqrkqZmZkNsiqd5+b0mgHM6XZDkr4cEed2W96GR85X2JmzbL2Tc1vQtvOMiMsAJO0VEc+1Wz4tu3ryJOAkSfuldZ7RpNxyYDnAAhbh8TDNple3WU5lnWcbWm07T0knANcCewMHSzoGeF9EfKBFsXkUQ/ldAwRF4BYDn2y1rYhYCawEeNOMs6LKF7D8jGd8nmMEdZVlcJ6tvZzbgir7zFcBbwb+CSAi1gNL25RZDNwHXApsSQMu/CEi1lYYVN7MBoezbFai0n2eEbFJ2ukXwvY2y48DV0palf58suq2bPj5eZz5cJatTjm3BVVCsEnSiUBImg1cBGyssvI0LuZZkk4DtnZfzWp8CbtZPaaa5VbZNMtRlc7z/cDVwFxgDLgduKCTjUTErcCtHdfOhlLOV9iNOmfZeinntqBK5zkeEe+svSZmZmaZqNJ53i3pAeA64LaI8FVzNiU5j2dpZr2Tc1tQpeaHU1xufi7wsKTLJR1eb7XMzMwGV5VBEgJYA6yRdBJwA/ABSeuBSyLizprraENme+R7b5eZ9U7ObUGVQRL2B84B3gU8CVwIrAYWAquAQ2usX0d8Ra3ZYHI2bdhUOed5J/AV4K3pcvUJ6yR9rp5q2TDL+d4uM+udnNuCKp3nEc0uEoqIK8qmS3odsDEitkraA7gEWEQxzNflEbGl2wqbWf84y2blqnT7B0j6G0nfkvSPE682Za4Dnk/vrwb2Ba5I067vvro2DMZjxpRf1jfOstUm57agyp7nV4GvA6dTDJhwHvB0mzIzImJber84Ihal9z9Mt72U8lMYzAZOV1kG59mGW5Vue/+IuBZ4MQ0G/WfA8W3KbJD0nvR+vaTFAOkWlxebFYqIlRGxOCIWO2jDazszpvyyvukqy+A8W3s5twVVtjwRkMclnSbpWIrHFLVyPvAnkn4JHAXcKekR4AtpnpnlwVk2K1HlsO3HJe0LfAT4NLAPsKJVgXQRwbslzQHmp+2MRcSTU6uuDYOc7+0aNc6y1SnntqDKIAm3pLdbgJMAJK2osvKIeAbw4xTMMucsm+2s2+fyfZjiIdlmHct5PEsz652c24Jua57vvraZmdkUdbvn6SerWNdyfoafmfVOzm1B085T0jOUd5IC9qitRmZmZgOuaecZEXP6WREbHeM+6m9m5N0W5LvPbGZmNk26PefZkqTZwNnA5oj4jqR3ACcCG4GVEdFyZBIbbjmf5xg1zrLVKee2oJbOk2LA6FnAnpLOA/YGbgZOAZZQjI9rZoPPWTYrUVfn+ZqIeK2kWcBjwIERsV3SDbS40doDSY8Gj02bla6yDM6ztZdzW1BXzWekwz1zgD0pHmME8EfAbs0KeSBps4HTVZbBebbhVtee57XAQ8BM4FJgVRpM+njgazVt0zIxnvF4liPIWbba5NwW1NJ5RsSVkr6e3m+W9GXgjcAXIuKeOrZpZr3nLJuVq2vPk4jY3PD+98BNdW3L8pLzeY5R5CxbXXJuC/KtuZmZ2TSpbc/TrJnxjO/tMrPeybktyLfmZmZm08R7ntZ32zMez9LMeifntmBgO88146t2/K1KWh4RK8uW62ZermXMcjXVPA9bGctfLodtl/d4Xq5lhsJ4zJjyy7I2CFma7jJG3m2BWyEzM7MODexhWxteOZ/nMLPeybktyGXPs9U5g27m5VrGbBgMQpamu4xlThEx3XWwEfNfHjx9yv/pLn31Lfn+ZDUzIO+2IJc9TzMzs4Hhc57Wdzk/Pd7MeifntmDgay5pmaSfSXpY0iUN06+T9JSkDZOWP0jS9yRtlPSgpIsb5u0u6R5J69O8yyaVnSnpR5JumTT9UUk/kfSApHUN0/eTdJOkh9L2TkjTj0jLTry2SlqR5n0obXuDpBsl7d6wvovT9AcnljcbFp1mOc0rzXOvs5zm7ZLnVllOZUrz7CwPv4HuPCXNBD4DnAocBbxd0lFp9heBZSXFtgEfiYgjKZ45eEFDmX8GTo6IY4CFwDJJxzeUvRjY2KQ6J0XEwohY3DDtauC2iFgAHDNRNiJ+lpZdCBwHPA/8g6S5wEXA4og4muIZiWen73o08F5gSVrX6ZIOa/NXlKVxNOWX5aXLLEPzPPc6y1CS52ZZTt+pNM+jlOWpyrktGOjOk+I/38MR8UhEvEDx8N0zASLiB8DvJheIiMcj4v70/hmKAM1NnyMink2L7pZeASBpHnAacE2ViknaB1hK8bBgIuKF9LimyU4BfhkRv06fZwF7SJoF7AlMPO7pSOCuiHg+IrYBa4G3VamLlWu2p9Mwf19J/7+GvZf3TEc9R0THWU7zSvPcyyynMlXyPDnLUJ5nZ3nA1NEWDHrnORfY1PB5LE2rRNIhwLHA3Q3TZkp6AHgKWBMRE/OuAj4KjJesKoDbJd0naWLEkPnA08D16fDQNZL2Kil7NnAjQEQ8BnwC+A3wOLAlIm5Py20AlkraX9KewFuAg6p+15xsjxlTfrXTZk9nwgXAT9PeyxuAT0qa3dtva8mUsgy75rmHWYZqed6RZWiZ55HJ8lTl3BYMeudZtk9e6dJmSXsD3wBWRMTWHYUjtqdDMPOAJZKOlnQ68FRE3Ndkda+PiEUUf/kXSFpK8YtzEfDZiDgWeA7Y6RdN+ss/A1iVPr+M4tf2ocCBwF6Szkn12ghcAawBbgPWUxyysu403dNpEMAcSQL2ptj78d95PbrOMpTnuYdZhjZ5npzlNK00z87ywKmlLRj0znOMnX+xzeOlw5xNSdqNImhfjYiby5ZJh2S+T3Gu5fXAGZIepfiLPVnSDQ3Lbk5/PkVxvmNJqttYw6/dmyjC1+hU4P6IeDJ9fiPwq4h4OiJeBG4GTmzYzrURsSgillL84/2i3XfN0Xhoyi9JyyWta3hNHkO0yp7Of6U4xLYZ+AlwcUSU7a3Y1HWVZWif5x5keaJ+rfI8OcvQIs+jkuWpyrktGNhBEt4046zBrFifNT6NYlhc+uM/nfK/7X957c0t/14knQW8OSLOT5/fBSyJiAsblvl3FI3th4F/TbGncEzjkQrrDed5OLM8VTm3BYO+52lDaDszpvyqoMqeznuAm9PFJw8DvwIW9ORLmllbObcFtQ2SIGkBxXHluRTHkzcDq9P5ALO63QscJulQ4DGKiz3eMWmZ31BcQfk/JP0vwBHAI32tZQacZctcLW1BLXuekj5Gcb5BwD0UlRdwY9llwg3ldhy7Hgu3YcOqF+c52km3CHwQ+DbF7Q1/HxEPSnq/pPenxf4KOFHST4DvAh+LiN/W9LWz1G2WU1nn2VrKuS2o5ZynpJ8Dr04n0RunzwYejIi2Nwz7HElhGM+TfHT91P9t/89jhu/vZRD1IsvgPMNwZnmqcm4L6jpsO05x6favJ01/BeX3XtkIGfep9pw4y1abnNuCujrPFcB3Jf2Cly4RPhh4FcXus5nlYQXOstkuauk8I+I2SYdT3EM1l+IcyRhwb0Rsr2Oblo/tFc5T2GBwlq1OObcFtV1tm24wvauu9ZtZfzjLZrvy8zyt76pcIWdmwy/ntiDfs7VmZmbTxHue1nfjGT893sx6J+e2IN+am5mZTRPveVrfbZ/Gp7+b2eDIuS3wnqeZmVmHmnaekg6WtHt6L0nvkfRpSf9BUts9VkkLJJ2SHmLbOH3Z1KttOevHeJbWO86y1SXntqDVnue3Gub/NXAacDfwx8DKViuVdBHwTeBCYIOkxqd2X96inAeSNhsg3WY5lXWebWi12oOcERHPp/dvBP443Sx9g6T1bdb7XuC4iHhW0iHATZIOiYiroflB7ohYSeqYPZD08Mr5CrsR1FWWwXm29nJuC1p1npsknRwR/wg8SvEw0V9L2r/CemdGxLMAEfGopDdQhO6VtAmcmQ0UZ9msRKtu/3zgLyX9AJgNPCDpH4HvAB9us94nJC2c+JDCdzpwAPCaKdXYsjeOpvyyvnGWrTY5twVN9zwjYhNwkqQjgcOBL/LSgNDtHkV0LrBt0vq2AedK+vyUamxm/eQsm5Voe9VsRGykePp2ZREx1mLeHZ2sy4ZPzk9SGDXOstUp57Yg37O1ZmZm08QjDFnf5XyFnZn1Ts5tQcuaS5op6YZ+VcbMzCwHLfc8I2K7pJdLmh0RL/SrUjbcPEKQmUHebUGVw7aPAndIWg08NzExIj5VV6XMzJr59ubmY7S8+cBj+lgTG2VVOs/N6TUDmFNvdWwU+D5NM4O824Iqt6pcBiBpr4h4rt3yzUj6ckSc2215MxsMzrJZhc5T0gnAtcDewMGSjgHeFxEfaFFm9eRJFAMu7AcQEWc0KbccWA6wgEXM0/wq38Eyk/N5jlHTbZZTWefZWsq5Lahy2PYq4M3AaoCIWC9paZsy84CfAtcAQRG4xcAnWxXyQNJmA6erLIPzbMOt0k02aai+RtvbFFkM3AdcCmyJiO8Df4iItRGxtuNa2lAZjxlTflnfOMtWm5zbgip7npsknQiEpNnARbQZri+NfXulpFXpzycrbmtKfBWeWW/1KsutsmmWoyoheD9wNTCXYmD424ELqqw8jYt5lqTTgK3dVtKGS87nOUaVs2x1yLktqNJ5jkfEO6eykYi4Fbh1Kusws+nnLJsVqnSed0t6ALgOuC0ifOLfpiTne7vMrHdybguqnG09nOKKuXOBhyVdLunweqtlZmY2uKoMkhDAGmCNpJOAG4APSFoPXBIRd9ZcRxsyOZ/nMLPeybktqDJIwv7AOcC7gCeBCynu+VwIrAIOrbF+ZmZmA6fKOc87ga8Ab530VPl1kj5XT7W649tR8pDzr03rjrNpZXJuC6p0nkc0u0goIq7ocX3MzMwGXpXO8wBJHwVeDew+MTEiTm5WQNLrgI0RsVXSHsAlwCKKYb4uj4gtU6u25SznX5ujxlm2OuXcFlS52varwEMU5zYvo3i+571tylwHPJ/eXw3sC1yRpl3frJCk5ZLWSVo3Fo9UqJqZ1ayrLIPzbMOtyp7n/hFxraSL01iWayW1G9NyRkRsS+8XR8Si9P6H6Z7RUh5IejTk/GtzBHWVZXCerb2c24Iqe54vpj8fl3SapGMpnrTQygZJ70nv10taDJDuD32xeTEzGzDOslmJKnueH5e0L/AR4NPAPsCKNmXOB66W9BfAb4E7JW0CNqV5NsJyHlVkBDnLVpuc24IqgyTckt5uAU4CkLSiTZktwLslzQHmp+2MRcSTU6qtmfWVs2xWrtvHhH2Y4iHZLUXEM4CfRWQ7yfk8x6hylq0OObcF3T5JNN9vbGZmNkXd7nn6yjnrWs6/Ns2sd3JuC5p2npKeobyTFLBHbTUyMzMbcE07z4iY08+K2OjI+demmfVOzm1Bt+c8zczMRla35zzNupbzr00z652c24JaOk9Js4Gzgc0R8R1J7wBOBDYCKyPCI5OYZcBZNitX157n9Wnde0o6D9gbuBk4BVgCnFdWSNJyYDnAAhYxT/Nrqp5Np8j41+YI6irL4Dxbezm3BXV1nq+JiNdKmgU8BhwYEdsl3UCLG609kLTZwOkqy+A823Crq/OckQ737AXsSfEYo98BfwTsVtM2LRM5j2c5gpxlq03ObUFdnee1FM8AnQlcCqyS9AhwPPC1mrZpZr3nLJuVqKXzjIgrJX09vd8s6cvAG4EvRMQ9dWzT8pHzFXajxlm2OuXcFtR2q0pEbG54/3vgprq2ZWb1cZbNduX7PK3vcr7Czsx6J+e2wCMMmZmZdch7ntZ3OZ/nMLPeybkt8J6nmZlZhwZ2z3PN+KodP0kkLU83XO+im3m5lhkWOZ/nsO5MNc/DVsYKObcFuex5Lu/xvFzLmA2DQcjSdJexzA3snqcNr5zPc5hZ7+TcFuSy52lmZjYwctnzbHXOoJt5uZYZCuEhwkfdIGRpussYebcFipxrb1n64//+n6b8n+7eUy/P93iPmQF5twW57HnaEMn5SQpm1js5twU+52lmZtahge88JS2T9DNJD0u6pGH6dZKekrRh0vIHSfqepI2SHpR0ccO83SXdI2l9mnfZpLIzJf1I0i2Tpj8q6SeSHpC0rmH6fpJukvRQ2t4JafoRadmJ11ZJK9K8D6Vtb5B0o6TdG9Z3cZr+4MTywyhCU35ZfjrNcppXmudeZznN2yXPrbKcypTmeVSyPFU5twUD3XlKmgl8BjgVOAp4u6Sj0uwvAstKim0DPhIRR1I8c/CChjL/DJwcEccAC4Flko5vKHsxsLFJdU6KiIURsbhh2tXAbRGxADhmomxE/CwtuxA4Dnge+AdJc4GLgMURcTTFMxLPTt/1aOC9wJK0rtMlHdbmr8gsC11mGZrnuddZhpI8N8ty+k6leXaWR8NAd54U//kejohHIuIFiofvngkQET+geKL9TiLi8Yi4P71/hiJAc9PniIhn06K7pVcASJoHnAZcU6VikvYBllI8LJiIeCE9rmmyU4BfRsSv0+dZwB6SZgF7AhOPezoSuCsino+IbcBa4G1V6pKb8dCUX5adjrOc5pXmuZdZTmWq5HlylqE8zyOT5anKuS0Y9M5zLrCp4fNYmlaJpEOAY4G7G6bNlPQA8BSwJiIm5l0FfBQYL1lVALdLuk/SxIgh84GngevT4aFrJO1VUvZs4EaAiHgM+ATwG+BxYEtE3J6W2wAslbS/pD2BtwAHVf2uZgNuSlmGXfPcwyxDtTzvyDK0zLOzPAIGvfMs+1lR6dJmSXsD3wBWRMTWHYUjtqdDMPOAJZKOlnQ68FRE3Ndkda+PiEUUh5wukLSU4hfnIuCzEXEs8BxwSWMhSbOBM4BV6fPLKH5tHwocCOwl6ZxUr43AFcAa4DZgPcUhq6ETMfWXZafrLEN5nnuYZWiT58lZTtNK8zxKWZ6qnNuCgb3P800zzhrMivVZ44Daw+KYW/5yyv+260//q6H7exlmzvNwZnmqcm4LfJ+n9Z2vljUzyLstqK3zlLSA4pDGXIrDM5uB1emQhpllwlk221Ut5zwlfYziajoB9wD3pvc3Nt7fVVJuuaR1ktaNxSN1VM0GQM73do2abrOcyjrP1lLObUFde55/Drw6Il5snCjpU8CDwF+XFUoPjV0JPkdiNiC6yjI4zzbc6uo8xymuPvv1pOmvoPzycRshvk8zK86y1SbntqCuznMF8F1Jv+Cle7sOBl4FfLCmbZpZ763AWTbbRS2dZ0TcJulwilFF5lKcIxkD7o2I7XVs0/IxoHdHWQln2eqUc1tQ29W2ETEO3FXX+s2sP5xls135Pk/rO18ta2aQd1sw6MPzmZmZDRzveVrf5fxr08x6J+e2wHueZmZmHfKep/VdxhfYmVkP5dwWeM/TzMysQ7V1npIWSDolPYevcfqyurZpech5PMtR5CxbXXJuCyp3npIOlfSn6QkL7Za9CPgmcCGwQdKZDbMv77yaZjYdnGWzck07T0n/V8P7M4F/BP6fwDclvbvNet8LHBcRbwXeAPylpIsnVtdim34KwyiIHrysX7rKMjjPVkHGbUGrC4Ze2fD+Y8DJEfErSQcA3wW+2KLszIh4FiAiHpX0BuAmSa+kReD8FAazgdNVltPyzrMNrVaHbRv/s8+KiF8BRMRvaf80hSckLdyxoiJ8pwMHAK/prqo2LHI+zzGCnGWrTc5tQavO8xhJWyU9AyyU9K8AJM0GZrZZ77nAE40TImJbRJwLLJ1Khc2qkrRM0s8kPdzswc2S3iDpAUkPSlrb7zpmwFm27NXRFjQ9bBsRzTrIPYH3tVppRIy1mHdHu0rZcOvHkxQkzQQ+A7yJ9BQQSasj4qcNy+wH/DdgWUT8RtK/rL9meXGWrU45twUd36oSEb+PiDs7LWfWZ0uAhyPikYh4AfgacOakZd4B3BwRvwGIiKf6XEczq18tbYEHSbC+69N5jrm89PBmKH5xzp20zOHAyyR9X9J9ks7t0Vc0swpybgs8PJ9lSdJyYHnDpJXp6s4di5QUm3yQaBZwHHAKsAdwp6S7IuLnPa2smdVmutqClp1nOlb8pYg4p9VyZh3pwRVyjbdBNDEGHNTweR6wuWSZ30bEc8Bzkn4AHAO48zTrh4zbgpadZ0Rsl/RySbPTsWKzXNwLHCbpUOAx4GyK8xqNvgn8V0mzgNnA64Ar+1pL69i3N69vOu/NBx7Tx5pYJmppC6octn0UuEPSauC5iYkR8anKVTdr0I8r7CJim6QPAt+muLXquoh4UNL70/zPRcRGSbcBP6a4d/maiNhQf+3MDPJuC6p0npvTawYwp9svIOnL6d4ws76IiG8B35o07XOTPv8N8Df9rFfunGXLTR1tQdvOMyIuA5C0Vzoe3FbaS91pEnBSupeGiDijagVtCHmgtmw4y1arjNuCtp2npBOAa4G9gYMlHQO8LyI+0KLYPOCnwDUUfz0CFgOfbLOtHVdNLWAR8zS/yncws/p0lWVwnm24VbnP8yrgzcA/AUTEetoPy7UYuA+4FNgSEd8H/hARayOi6bBHEbEyIhZHxGIHbXjlPJ7lCOoqy+A8W3s5twWV7vOMiE3STpXc3mb5ceBKSavSn09W3ZaZDQ5n2axclRBsknQiEGlQ+IuAjVVWnsbFPEvSacDW7qtZjS9hz0TG5zlG1VSz3CqbNsIybguqdJ7vB66mGM5oDLgduKCTjUTErcCtHdfOzAaKs2xWqNJ5jkfEO2uviY0Mn7M0M8i7LahywdDdklZJOlWTTnyamZmNoiqd5+EU4waeCzws6XJJh9dbLRtq0YOXmeUv47agbecZhTUR8XbgfOA84B5Ja9M9oGZmZiOlyiAJ+wPnAO8CngQuBFYDC4FVwKE11q8jvqI2Fz76P2qcTSuXb1tQ5YKhO4GvAG9Nl6tPWCfpc03KmJmZDa0qnecREeVj30fEFWXTJb0O2BgRWyXtAVwCLKIY5uvyiNjSbYVtCPicZTacZatVxm1BlQuGDpD0N5K+JekfJ15tylwHPJ/eXw3sC1yRpl3ffXXNrM+cZbMSVfY8vwp8HTidYsCE84Cn25SZERHb0vvFEbEovf+hpAeaFfJA0iMi41+bI6irLIPzbBVk3BZU2fPcPyKuBV5Mg0H/GXB8mzIbJL0nvV8vaTFAusXlxWaFPJC02cDpKsvgPNtwq7LnORGQx9O4lpspHlPUyvnA1ZL+AvgtcKekTcCmNM9GWcajiowgZ9nqk3FbUKXz/LikfYGPAJ8G9gFWtCqQLiJ4t6Q5wPy0nbGIeHJq1TWzfnKWzcq17Twj4pb0dgtwEoCkFVVWHhHPAH6cgu2k/NptG2TOstUh57agyjnPMh/uaS3MzMwy0u1DbfM9UG3TL+Nfm2bWQxm3Bd3ueWb8lc3MzKam6Z6npGco7yQF7FFbjWz4ZXyFnZn1UMZtQdPOMyLm9LMiZmZmuej2nKdZ1+SD/mZG3m1Bt+c8zczMRlYte56SZgNnA5sj4juS3gGcCGwEVkZEy2G9bMhl/Gtz1DjLVquM24K6Dtten9a9p6TzgL2Bm4FTgCUUg8vvwgNJmw2crrIMzrMNt7o6z9dExGslzQIeAw6MiO2SbqDFKCURsRJYCfCmGWdl/JvEWsr4CrsR1FWWwXm2CjJuC+o65zkjHe6ZA+xJ8QxAgD8Cdqtpm2bWe86yWYm69jyvBR4CZgKXAqskPULxKLOv1bRNy4X3QXLiLFt9Mm4Lauk8I+JKSV9P7zdL+jLwRuALEXFPHds0s95zls3K1XafZ0Rsbnj/e+CmurZlmcn41+YocpatNhm3Bb7P08zMrEMeYcj6L+Nfm2bWQxm3Bd7zNDMz65D3PK3/Mr63y8x6KOO2YGA7zzXjq3b8rUpanm643kU383ItY5arqeZ52MpY/nI5bLu8x/NyLTMUFFN/WdYGIUvTXcbIuy3IpfM0MzMbGAN72NaGmPcczQyybgty2fNsdc6gm3m5ljEbBoOQpekuY5lTRMZdv2Xp0E9/csr/6X514UfyvUzPzIC824Jc9jzNzMwGxsB3npKWSfqZpIclXdIw/TpJT0naMGn5gyR9T9JGSQ9Kurhh3u6S7pG0Ps27bFLZmZJ+JOmWSdMflfQTSQ9IWtcwfT9JN0l6KG3vhDT9iLTsxGurpBVp3ofStjdIulHS7g3ruzhNf3Bi+WGU8xV21r1Os5zmlea511lO83bJc6sspzKleR6VLE9Vzm3BQHeekmYCnwFOBY4C3i7pqDT7i8CykmLbgI9ExJEUj026oKHMPwMnR8QxwEJgmaTjG8peDGxsUp2TImJhRCxumHY1cFtELACOmSgbET9Lyy4EjgOeB/5B0lzgImBxRBxN8Zins9N3PRp4L7Akret0SYe1+Ssyy0KXWYbmee51lqEkz82ynL5TaZ6d5dEw0J0nxX++hyPikYh4geL5gWcCRMQPgN9NLhARj0fE/en9MxQBmps+R0Q8mxbdLb0CQNI84DTgmioVk7QPsJTieYdExAvpiROTnQL8MiJ+nT7PAvaQNIvi4cITT6w4ErgrIp6PiG3AWuBtVeqSndDUX5abjrOc5pXmuZdZTmWq5HlylqE8z6OT5anKuC0Y9M5zLrCp4fNYmlaJpEOAY4G7G6bNlPQA8BSwJiIm5l0FfBQYL1lVALdLuk/SxE3P84GngevT4aFrJO1VUvZs4EaAiHgM+ATwG+BxYEtE3J6W2wAslbS/pD2BtwAHVf2uZgNuSlmGXfPcwyxDtTzvyDK0zLOzPAIGvfMs+1lR6Si3pL2BbwArImLrjsIR29MhmHnAEklHSzodeCoi7muyutdHxCKKQ04XSFpK8YtzEfDZiDgWeA64pLGQpNnAGcCq9PllFL+2DwUOBPaSdE6q10bgCmANcBuwnuKQ1fCJHrwsN11nGcrz3MMsQ5s8T85ymlaa55HK8lRl3BYMeuc5xs6/2Obx0mHOpiTtRhG0r0bEzWXLpEMy36c41/J64AxJj1IcTjpZ0g0Ny25Ofz5Fcb5jSarbWMOv3ZsowtfoVOD+iHgyfX4j8KuIeDoiXgRuBk5s2M61EbEoIpZSHMb6RbvvapaJrrIM7fPcgyxP1K9VnidnGVrk2VkefgN7n+ebZpw1mBXrs8YBtYfF/Cs/NeV/20c+9OGh+3sZZs7zcGZ5qnJuCwZ9z9PMzGzg1Da2raQFFOcD5lIcmd4MrE7nA2yE+T7NvDjLVpec24Ja9jwlfYzifIOAe4B70/sbG2+OLim3XNI6SevG4pE6qmZmHeg2y6ms82xDq649zz8HXp1Oou8g6VPAg8BflxVKD41dCT5HMtT8L5uTrrIMzrNVkPH/irrOeY5TXLo92Ssov/fKzAaTs2xWoq49zxXAdyX9gpdujD4YeBXwwZq2abnI+NfmCFqBs2x1ybgtqKXzjIjbJB1OcQ/VXIpzJGPAvRGxvY5tmlnvOctm5Wq72jYixoG76lq/5SvnK+xGkbNsdcm5LfB9nmZmZh2qbc/TrCk/FcXMIOu2wHueZmZmHfKep/Vfxuc5zKyHMm4LvOdpZmbWIe95Wt/lfIWdmfVOzm1BbXuekhZIOiU9xLZx+rK6tmlmvecsm+2qo85T0uUVl7sI+CZwIbBB0pkNs5uuwwNJj4iMnx4/arrNcirrPFtrGbcFTQ/bSvrbyZOAd038+oyIi1qs973AcRHxrKRDgJskHRIRV6f1lPJA0mYDp6ssg/Nsw63VOc8/Bb4P3M5LITkbuK/CemdGxLMAEfGopDdQhO6VtAmcDb+cz3OMIGfZapNzW9DqsO2RwG+BZcB3IuJLwDMR8aX0vpUnJC2c+JDCdzpwAPCaqVXZzPrIWTYr0XTPMyKeAVZIOg64QdKtVD9Hei6wbdL6tgHnSvp8t5W1IZHxr80R5CxbfTJuC9p2hhFxH3Ay8Afgh1VWGhFjEfFEk3l3dFRDM5s2zrJZuUr3eUZEAJ9JL7OpyfjXppn1UMZtgUcYMjMz65BHGLK+y/kKOzPrnZzbgpZ7npJmSrqhX5UxMzPLQcs9z4jYLunlkmZHxAv9qpSZWTPf3ry+6bw3H3hMH2tio6zKYdtHgTskrQaem5gYEZ+qq1JmZmaDrMoFQ5uBW9KycxpeHZH05U7L2JDKeDxLc5athzJuC9rueUbEZQCS9oqI59otn5ZdPXkScJKk/dI6z2hSbjmwHGABi5in+VU2Z2Y16TbLqazzbEOrbecp6QTgWmBv4GBJxwDvi4gPtCg2D/gpcA3FbwMBi4FPttqWB5IeDTlfYTeCusoyOM/WXs5tQZXDtlcBbwb+CSAi1gNL25RZTDGA/KXAloj4PvCHiFgbEWu7rq2Z9ZuzbFai6ghDm6SdHqCwvc3y48CVklalP5+sui0bARn/2hw1zrLVKuO2oEoINkk6EQhJs4GLgI1VVh4RY8BZkk4DtnZfzWp8CbtZPaaa5VbZNMtRlc7z/cDVwFxgjOL5nhd0spGIuBW4tePa2XDK+NfmqHOWracybguqdJ7jEfHO2mtiZmaWiSqd592SHgCuA25LT1gx61rOV9iZWe/k3BZUudr2cIrLzc8FHpZ0uaTD662WmZnZ4KryMOyIiDUR8XbgfOA84B5Ja9M9oGadyXhUETProYzbgiqDJOwPnAO8C3gSuBBYDSwEVgGH1li/jviKWrPB5GzasKlyzvNO4CvAW9Pl6hPWSfpcPdWyYZbzeQ4z652c24IqnecRzS4SiogryqZLeh2wMSK2StoDuARYRDHM1+URsaXbCptZ/zjLZuWqXDB0gKS/kfQtSf848WpT5jrg+fT+amBf4Io07fpmhSQtl7RO0rqxeKRK/S1HGZ/nGEFdZRmcZ6sg47agyp7nV4GvA6dTDJhwHvB0mzIzImJber84Ihal9z9Mt72U8kDSZgOnqyyD82zDrcqe5/4RcS3wYhoM+s+A49uU2SDpPen9ekmLAdItLi92X10bCn36tSlpmaSfSXpY0iUtlvtjSdsl/btuv9IQc5atPhm3BVU6z4mAPC7pNEnHUjymqJXzgT+R9EvgKOBOSY8AX0jzzGolaSbwGeBUiv+Db5d0VJPlrgC+3d8aZsNZtqzV1RZUOWz7cUn7Ah8BPg3sA6xoVSBdRPBuSXOA+Wk7YxHxZJVK2XDr0xV2S4CHI4qTbZK+BpxJcaFLowuBbwB/3JdaZcZZtjrl3Ba07Twj4pb0dgtwUtr4iiorj4hnAD9OwXpO0nJgecOklekc24S5wKaGz2PA6yatYy7wNuBk3Hm25CzboJqutqDb5/J9mOIh2Wad68GvzcaLUZpQybTJW74K+FhEbJ/0vFoz64eM24JuO0+3NDboxoCDGj7PAzZPWmYx8LUUlgOAt0jaFhH/V19qaGb9UEtb0G3n6cvOrXv9+d9zL3CYpEOBx4CzgXfsVI2IHUNLSvoicIs7TrM+yrgtaNp5SnqG8q8mYI+qtTabDhGxTdIHKa6cmwlcFxEPSnp/mu+hJc1GQF1tQdPOMyLmdFVTszb6NZ5lRHwL+NakaaVBiYh396NOZvaSnNuCKvd5mpmZWYNuz3m2JGk2xXHlzRHxHUnvAE4ENlJcRuyRSUaZz5hnw1m2WmXcFtTSeVIMGD0L2FPSecDewM3AKRQ3rJ5XVqjxfp0FLGKe5tdUPTOrqKssg/Nsw62uzvM1EfFaSbMorm46MN0/cwMtbrT2QNKjIedn+I2grrIMzrO1l3NbUNc5zxnpcM8cYE+KxxgB/BGwW03bNLPec5bNStS153kt8BDFZcGXAqvSYNLHA1+raZuWi4x/bY4gZ9nqk3FbUEvnGRFXSvp6er9Z0peBNwJfiIh76timmfWes2xWrq49TyJic8P73wM31bUty0zGvzZHkbNstcm4LfB9nmZmZh2qbc/TrBk/VcDMIO+2wHueZmZmHfKep/Vfxuc5zKyHMm4LBrbzXDO+ascevaTlk54MzlTm5VrGLFdTzfOwlbH85XLYdnmP5+VaZigopv6yrA1Clqa7jJF3W5BL52lmZjYwBvawrQ0x7zmaGWTdFuSy59nqnEE383ItYzYMBiFL013GMqeIjLt+y9IxF1055f906//2QznfImZm5N0W5LLnaWZmNjB8ztP6zlfLmhnk3RYM/J6npGWSfibpYUmXNEy/TtJTkjZMWv4gSd+TtFHSg5Iubpi3u6R7JK1P8y6bVHampB9JumXS9Ecl/UTSA5LWNUzfT9JNkh5K2zshTT8iLTvx2ippRZr3obTtDZJulLR7w/ouTtMfnFjebFh0muU0rzTPvc5ymrdLnltlOZUpzbOzPPwGuvOUNBP4DHAqcBTwdklHpdlfBJaVFNsGfCQijqR45uAFDWX+GTg5Io4BFgLLJB3fUPZiYGOT6pwUEQsjYnHDtKuB2yJiAXDMRNmI+FladiFwHPA88A+S5gIXAYsj4miKZySenb7r0cB7gSVpXadLOqzNX1Geogcvy0qXWYbmee51lqEkz82ynL5TaZ5HKstTlXFbMNCdJ8V/vocj4pGIeIHi4btnAkTED4DfTS4QEY9HxP3p/TMUAZqbPkdEPJsW3S29AkDSPOA04JoqFZO0D7CU4mHBRMQL6XFNk50C/DIifp0+zwL2kDQL2BOYeNzTkcBdEfF8RGwD1gJvq1IXswx0nOU0rzTPvcxyKlMlz5OzDOV5dpZHwKB3nnOBTQ2fx9K0SiQdAhwL3N0wbaakB4CngDURMTHvKuCjwHjJqgK4XdJ9kiZGDJkPPA1cnw4PXSNpr5KyZwM3AkTEY8AngN8AjwNbIuL2tNwGYKmk/SXtCbwFOKjqd81JzqOKWNemlGXYNc89zDJUy/OOLEPLPI9Mlqcq57Zg0DvPskuQK/11Sdob+AawIiK27igcsT0dgpkHLJF0tKTTgaci4r4mq3t9RCyiOOR0gaSlFL84FwGfjYhjgeeASxoLSZoNnAGsSp9fRvFr+1DgQGAvSeekem0ErgDWALcB6ykOWZkNg66zDOV57mGWoU2eJ2c5TSvNs7M8Ggb2Ps83zThrMCvWZ40Dag+LYz8w9Xu7fvTffJ9nTpzn4czyVOXcFgz6nqeZmdnAqe0+T0kLKA5pzKU4PLMZWJ0OadgI8znLvDjLVpec24Ja9jwlfYziajoB9wD3pvc3Nt7fZWaDzVk2K1fXnuefA6+OiBcbJ0r6FPAg8NdlhdLVb8sBFrCIeZpfU/VsWmX8a3MEdZXltIzzbK1l3BbUdc5znOLqs8leQfnl4wBExMqIWBwRix00s4HQVZbBebbhVtee5wrgu5J+wUv3dh0MvAr4YE3btFxk/GtzBK3AWba6ZNwW1NJ5RsRtkg6nGFVkLsU5kjHg3ojYXsc2zaz3nGWzcrVdbRsR48Bdda3f8pXzFXajyFm2uuTcFvg+TzMzsw75eZ7Wfxn/2jSzHsq4LfCep5mZWYe852l9pwEdT9nM+ivntsB7nmZmZh3ynqf1X74/Ns2slzJuC7znaWZm1qGWnaekfyXpX6X3L5f0p5JeXWXFkhZIOiU9xLZx+rLuq2vDIOenx48iZ9nqknNb0LTzlPQ+4E7gLkn/AbgFOB24WdKft1qppIuAbwIXAhskndkw+/IW5ZZLWidp3Vg80sHXMLM6dJvlVNZ5tqHV6pznB4FXA3sAvwZeFRFPSHoZ8D3g2hZl3wscFxHPSjoEuEnSIRFxNcXwXqUiYiWwEvzk+aHmf9mcdJVlcJ6tgoz/V7TqPF+MiOeB5yX9MiKeAIiI/ym13VmeGRHPpuUflfQGitC9kjaBM7OB4iyblWh1znNc0m7p/WkTEyXt3qYcwBOSFk58SOE7HTgAeE13VbVhkfN5jhHkLFttcm4LWnWCf0raqY6IsYbp+wMfabPec4EnGidExLaIOBdY2kU9zWx6OMtmJZoeto2I3zSZ/hjwWKuVTupsJ8+7o3LtbDh5zzEbzrLVKuO2wPd5mpmZdcgjDFnf+ZylmUHebUG7QRJmSrqhX5UxMzPLQcs9z4jYnkYWmh0RL/SrUjbkMv61aWY9lHFbUOWw7aPAHZJWA89NTIyIT9VVKTOzZr69eX3TeW8+8Jg+1sRGWZXOc3N6zQDm1FsdGwU5n+cws97JuS1o23lGxGUAkvaKiOfaLd+MpC+ne8PMLGPOslmFzlPSCRTj2O4NHCzpGOB9EfGBFmVWT54EnCRpP4CIOKNJueXAcoAFLGKe5lf5DpabjJ8eP2q6zXIq6zxbaxm3BVUO214FvBlYDRAR6yW1G1lkHvBT4BqKU8ICFgOfbFXIA0mbDZyusgzOsw23SoMkRMSmSZO2tymyGLgPuBTYEhHfB/4QEWsjYm3HtbShkvN4liPIWbba5NwWVNnz3CTpRCAkzQYuAja2KhAR48CVklalP5+suC0zGyC9ynKrK2TNclQlBO8HrgbmAmPA7cAFVVaexsU8S9JpwNZuK2lDxnuO2XGWrRYZtwVVOs/xiHjnVDYSEbcCt05lHWY2/Zxls0KVzvNuSQ8A1wG3RWR8eZQNBI1Pdw3MbBDk3BZUuWDocIor5s4FHpZ0uaTD662WmZnZ4GrbeUZhTUS8HTgfOA+4R9LadA+oWWeiBy8zy1/GbUGVQRL2B84B3gU8CVxIcc/nQmAVcGiN9TMzMxs4Vc553gl8BXjrpKfKr5P0uXqqZcPM92mOHg/YbmVybguqdJ5HNLtIKCKu6HF9zMzMBl6VzvMASR8FXg3sPjExIk5uVkDS64CNEbFV0h7AJcAiimG+Lo+ILVOrtmXNF2xnw1m2WmXcFlS52varwEMU5zYvo3i+571tylwHPJ/eXw3sC1yRpl3frJCk5ZLWSVo3Fo9UqJqZ1ayrLIPzbMOtyp7n/hFxraSL01iWayW1G9NyRkRsS+8XR8Si9P6H6Z7RUh5IejTkfJ5jBHWVZXCerb2c24Iqe54vpj8fl3SapGMpnrTQygZJ70nv10taDJDuD32xeTEzGzDOslmJKnueH5e0L/AR4NPAPsCKNmXOB66W9BfAb4E7JW0CNqV5Nsoy/rU5gpxlq0/GbUHbzjMibklvtwAnAUha0abMFuDdkuYA89N2xiLiySnV1sz6ylk2K9ftY8I+TPGQ7JYi4hnAzyKyneR8nmNUOctWh5zbgkoPwy6hntbCzMwsI93ueWb8e8GmXcb3dplZD2XcFjTtPCU9Q3knKWCP2mpkZmY24Jp2nhExp58VsdGR83kOM+udnNuCbs95mpmZjaxuz3madS/jX5tm1kMZtwW1dJ6SZgNnA5sj4juS3gGcCGwEVkaERyYxy4CzbFaurj3P69O695R0HrA3cDNwCrAEOK+skKTlwHKABSxinubXVD2bTjmf5xhBXWUZnGdrL+e2oK7O8zUR8VpJs4DHgAMjYrukG2hxo7UHkjYbOF1lGZxnG251dZ4z0uGevYA9KR5j9Dvgj4Ddatqm5WLc7WhGnGWrT8ZtQV2d57UUzwCdCVwKrJL0CHA88LWatmlmvecsm5WopfOMiCslfT293yzpy8AbgS9ExD11bNMyku+PzZHjLFutMm4LartVJSI2N7z/PXBTXdsys/o4y2a78n2e1nc5X2FnZr2Tc1vgEYbMzMw65D1P67+Mn6RgZj2UcVvgPU8zM7MODeye55rxVTseuC1pebrhehfdzMu1zLDI+TyHdWeqeR62MlbIuS3IZc9zeY/n5VrGbBgMQpamu4xlLpfO04ZJ9OBVgaRlkn4m6WFJl5TMf6ekH6fX/y3pmKl+NTPrQMZtgTtPG0qSZgKfAU4FjgLeLumoSYv9CviTiHgt8FekcVjNbHjU1RYM7DnPSVp9kW7m5VpmKKg/V9gtAR6OiEcAJH0NOBP46cQCEfF/Nyx/FzCvHxWzgcjSdJcx8m4LstjzbHXCvZt5uZaxl0haLmldw2vyuaW5wKaGz2NpWjN/Dvz3XtfTdjUIWZruMtY709UW5LLnacNkfOqraHzcVRMqmVb6M1fSSRSB+TdTr5mZVZZxWzDwe57NTvRKuk7SU5I2TFr+IEnfk7RR0oOSLm6Yt7ukeyStT/Mum1R2pqQfSbpl0vRHJf1E0gOS1jVM30/STZIeSts7IU0/Ii078doqaUWa96G07Q2SbpS0e8P6Lk7TH5xY3ro2BhzU8HkesHnyQpJeC1wDnBkR/9Snuo2kTrOc5pXmuddZTvN2yXOrLKcypXl2lgdKLW3BQHeebU70fhFYVlJsG/CRiDiS4rFJFzSU+Wfg5Ig4BlgILJN0fEPZi4GNTapzUkQsjIjFDdOuBm6LiAXAMRNlI+JnadmFwHHA88A/SJoLXAQsjoijKR7zdHb6rkcD76U4Pn8McLqkw9r8FWVJEVN+VXAvcJikQ1U8j/JsYPVO9ZAOBm4G3hURP+/5F7UduswyNM9zr7MMJXluluX0nUrzPEpZnqqc24KB7jxpONEbES9QPD/wTICI+AHFQ3l3EhGPR8T96f0zFAGamz5HRDybFt0tvQJA0jzgNIpfHm1J2gdYSvG8QyLihfTEiclOAX4ZEb9On2cBe0iaRfFw4YlfQEcCd0XE8xGxDVgLvK1KXWxX6e/wg8C3Kf4P/H1EPCjp/ZLenxb7fwP7A/+tbE/EeqrjLKd5pXnuZZZTmSp5npxlKM+zszxA6moLBv2cZ9mJ3tdVLSzpEOBY4O6GaTOB+4BXAZ+JiIl5VwEfBeaUrCqA2yUF8Pl0jH0+8DRwvYp7gu4DLo6I5yaVPRu4ESAiHpP0CeA3wB+A2yPi9rTcBuC/SNo/zXsLMJyNeZ9GFYmIbwHfmjTtcw3vzwfO709tRt6Usgy75rmHWYZqed6RZWieZ0lHMipZnqqM24JB3/OsfKJ3l4LS3sA3gBURsXVH4Yjt6RDMPGCJpKMlnQ48FRH3NVnd6yNiEcUhpwskLaX44bEI+GxEHAs8B+x08206RHAGsCp9fhnFr+1DgQOBvSSdk+q1EbgCWAPcBqynOGRlNgy6zjKU57mHWYY2eZ6c5TStNM/O8mgY9M6z0oneySTtRhG0r0bEzWXLpEMy36c41/J64AxJj1IcTjpZ0g0Ny25Ofz5Fcb5jSarbWMOv3ZsowtfoVOD+iHgyfX4j8KuIeDoiXqQ4xn5iw3aujYhFEbGU4jDWL9p91yxFTP1luekqy9A+zz3I8kT9WuV5cpahRZ5HJstTlXFboBjQhuhNM84azIr1WeOA2sPilDdcPuV/2+9+/z8N3d/LMHOehzPLU5VzWzDo5zxtCOX8JAUz652c24LaOk9JCyjOB8ylOLexGVidzgeYWSacZbNd1XLOU9LHKM43CLiH4j4bATeqZET7hnI7hlkaK4YhtGGU8XmOUdNtllNZ59lay7gtqGvP88+BV6eT6DtI+hTwIPDXZYUah1nyORKzgdBVlsF5tuFWV+c5TnHp9q8nTX8FPRnN0HIm/w/IibNstcm5Lair81wBfFfSL3jpxuiDKW5m/mBN2zSz3luBs2y2i1o6z4i4TdLhFPdQzaU4RzIG3BsR2+vYpmXE5yyz4SxbrTJuC2q72jYixikeKmpmGXOWzXbl+zyt//L9sWlmvZRxWzDow/OZmZkNHO95Wt9VfAafmQ25nNsC73mamZl1yHue1n8Z/9o0sx7KuC3wnqeZmVmHaus8JS2QdEp6iG3j9GV1bdMyMd6Dl/WNs2y1ybgtaNl5StpH0r8umf7aNuUuAr4JXAhskHRmw+zLW5TzQNJmA6TbLKeyzrMNrabnPCX9e+Aq4Kn0JPd3R8S9afYX2fkp65O9FzguIp6VdAhwk6RDIuJqihFKSnkg6dGQ8xV2I6irLIPzbO3l3Ba0umDoP1GE5nFJS4CvSPpPEXEzbUIDzIyIZwEi4lFJb6AI3SsrlDWzweEsm5Voddh2ZkQ8DhAR9wAnAZemwzjtfi48IWnhxIcUvtOBA4DXTKnGlr+Mn+E3gpxlq0/GbUGrzvOZxvOdqSN9A8UT5V/dZr3nAk80ToiIbRFxLrC0u6qa2TRwls1KtDps+x+YdFgmIp5JV9j9+1YrjYixFvPu6KiGNny855gNZ9lqlXFb0LTzjIj1Taa/CHy1thqZmZkNOI8wZP3n+zTNDLJuCzzCkJmZWYda7nlKmgl8KSLO6VN9bATkfG+XmfVOzm1By84zIrZLermk2RHxQr8qZWbWzLc3l16OAcCbDzymjzWxUVblnOejwB2SVgPPTUyMiE/VVSkbchn/2jSzHsq4LahyznMzcEtadk7DqyOSvtxpGTMbPM6yWYU9z4i4DEDSXhHxXLvl07KrJ08CTpK0X1rnGR3W04ZJxr82R42zbLXKuC1o23lKOgG4FtgbOFjSMcD7IuIDLYrNA34KXEMxlJ+AxcAn22xrObAcYAGLmKf5Vb6DmdWnqyyD82zDrcph26uANwP/BDsGT2g3LNdi4D7gUmBLRHwf+ENErI2Itc0KRcTKiFgcEYsdtCGW8XiWI6irLIPzbBVk3BZUGiQhIjZJO43Ut73N8uPAlZJWpT+frLotMxsczrJZuSoh2CTpRCAkzQYuAjZWWXkaF/MsSacBW7uvpg2VjEcVGVVTzXKr20tshGXcFlTpPN8PXA3MBcaA24ELOtlIRNwK3Npx7cxsoDjLZoUqned4RLyz9prYyMh5VBEz652c24IqFwzdLWmVpFM16cSnmZnZKKrSeR4OrKR4KO7Dki6XdHi91bKhlvEVdmbWQxm3BW07zyisiYi3A+cD5wH3SFqb7gE1MzMbKVUGSdgfOAd4F/AkcCGwGlgIrAIOrbF+NozGvec4ajxgu5XKuC2ocsHQncBXgLemy9UnrJP0uXqqZWZmNriqdJ5HRJQfWI6IK8qmS3odsDEitkraA7gEWEQxzNflEbGl2wrbEPA5y2w4y1arjNuCKhcMHSDpbyR9S9I/TrzalLkOeD69vxrYF7giTbu+++qaWZ85y2Ylqux5fhX4OnA6xYAJ5wFPtykzIyK2pfeLI2JRev9DSQ80K+SBpEdExr82R1BXWQbn2SrIuC2osue5f0RcC7yYBoP+M+D4NmU2SHpPer9e0mKAdIvLi80KeSBps4HTVZbBebbhVmXPcyIgj6dxLTdTPKaolfOBqyX9BfBb4E5Jm4BNaZ6Nsox/bY4gZ9nqk3FbUKXz/LikfYGPAJ8G9gFWtCqQLiJ4t6Q5wPy0nbGIeHJq1TWzfnKWzcq17Twj4pb0dgtwEoCkFVVWHhHPAH6cgu0s43u7RpWzbLXIuC2ocs6zzId7WgszM7OMdPtQWw8Qb92LjB/iZ2a9k3Fb0O2eZ7772mZmZlPUdM9T0jOUd5IC9qitRjb8Mr7Czsx6KOO2oGnnGRFz+lkRMzOzXHR7ztOsexlfYWdmPZRxW1BL5ylpNnA2sDkiviPpHcCJwEZgZUS0HJnEzAaDs2xWrq49z+vTuveUdB6wN3AzcAqwhGJ8XBtVGZ/nGEHOstUn47agrs7zNRHxWkmzgMeAAyNiu6QbaHGjtQeSNhs4XWUZnGcbbt3eqtJ2velwzxxgT4rHGAH8EbBbs0IeSHpEREz9Zf3SVZbBebYKMm4L6trzvBZ4CJgJXAqskvQIxdNYvlbTNs2s95xlsxK1dJ4RcaWkr6f3myV9GXgj8IWIuKeObVpGvOeYDWfZapVxW1DbrSoRsbnh/e+Bm+ralpnVx1k225Xv87T+G893PEsz66GM24K6LhgyMzMbWt7ztP7L+DyHmfVQxm2B9zzNzMw65D1P67+Mf22aWQ9l3BYMbOe5ZnzVjgduS1oeESvLlutmXq5lzHI11TwPWxnLXy6HbZf3eF6uZYbDeEz9ZTkbhCxNdxmDrNuCXDpPMzOzgTGwh21teEXke2+XmfVOzm1BLnuerc4ZdDMv1zJmw2AQsjTdZSxzioyvdrI8LfsX753yf7rbfvcFtV/KzAZZzm1BLnueZmZmA2PgO09JyyT9TNLDki5pmH6dpKckbZi0/EGSvidpo6QHJV3cMG93SfdIWp/mXTap7ExJP5J0y6Tpj0r6iaQHJK1rmL6fpJskPZS2d0KafkRaduK1VdKKNO9DadsbJN0oafeG9V2cpj84sfxQyvgZfta9TrOc5pXmuddZTvN2yXOrLKcypXkemSxPVcZtwUB3npJmAp8BTgWOAt4u6ag0+4vAspJi24CPRMSRFM8cvKChzD8DJ0fEMcBCYJmk4xvKXgxsbFKdkyJiYUQsbph2NXBbRCwAjpkoGxE/S8suBI4Dngf+QdJc4CJgcUQcTfGMxLPTdz0aeC+wJK3rdEmHtfkrMstCl1mG5nnudZahJM/Nspy+U2meneXRMNCdJ8V/vocj4pGIeIHi4btnAkTED4DfTS4QEY9HxP3p/TMUAZqbPkdEPJsW3S29AkDSPOA04JoqFZO0D7CU4mHBRMQL6XFNk50C/DIifp0+zwL2kDQL2BOYeNzTkcBdEfF8RGwD1gJvq1KX7IyPT/1luek4y2leaZ57meVUpkqeJ2cZyvM8OlmeqozbgkHvPOcCmxo+j6VplUg6BDgWuLth2kxJDwBPAWsiYmLeVcBHgbJ/jQBul3SfpImbnucDTwPXp8ND10jaq6Ts2cCNABHxGPAJ4DfA48CWiLg9LbcBWCppf0l7Am8BDqr6Xc0G3JSyDLvmuYdZhmp53pFlaJlnZ3kEDHrnWXYVVaWD3JL2Br4BrIiIrTsKR2xPh2DmAUskHS3pdOCpiLivyepeHxGLKA45XSBpKcUvzkXAZyPiWOA54JLGQpJmA2cAq9Lnl1H82j4UOBDYS9I5qV4bgSuANcBtwHqKQ1bDJ+PzHNa1rrMM5XnuYZahTZ4nZzlNK83zSGV5qjJuCwa98xxj519s83jpMGdTknajCNpXI+LmsmXSIZnvU5xreT1whqRHKQ4nnSzphoZlN6c/n6I437Ek1W2s4dfuTRTha3QqcH9EPJk+vxH4VUQ8HREvAjcDJzZs59qIWBQRSykOY/2i3Xc1y0RXWYb2ee5Blifq1yrPk7MMLfLsLA+/gb3P800zzhrMivVZ44Daw+LNe5835X/bbz/7paH7exlmzvNwZnmqcm4LBn3P08zMbODUNratpAUU5wPmUpzb2AysTucDbJQN6NEOK+csW20ybgtq2fOU9DGK8w0C7gHuTe9vbLw5uqTccknrJK0bi0fqqJqZdaDbLKeyzrMNrbr2PP8ceHU6ib6DpE8BDwJ/XVYoPTR2JfgcyVDz8zhz0lWWwXm2CjJuC+o65zlOcen2ZK+g/N4rMxtMzrJZibr2PFcA35X0C166Mfpg4FXAB2vapuUi42f4jaAVOMtWl4zbglo6z4i4TdLhFPdQzaU4RzIG3BsR2+vYppn1nrNsVq62q22jeET4XXWt3/IVGZ/nGEXOstUl57bA93mamZl1qLY9T7OmMj7PYWY9lHFb4D1PMzOzDrnztL6L8ZjyqwpJyyT9TNLDZTf0q/C3af6PJU0e2N/MapRzW+DO04aSpJnAZyiehnEU8HZJR01a7FTgsPRaDny2r5U0s9rV1Ra487T+i/Gpv9pbAjwcEY9ExAsUQ8ydOWmZM4EvR+EuYD9Jr+jtlzWzpjJuC2rrPCUtkHRKeoht4/RldW3TrMFcXrqpH4p7E+d2sczIc5Ytc7W0BV1dbSvpTRGxpsX8i4ALgI3AtZIujohvptmXUzxdvazccopdZhawiHma3031bMD14rmGjf9XkpVpLNUdi5QUm3yCpMoyI63bLKeyzrO1lHNb0O2tKtdSDNHVzHuB4yLiWUmHADdJOiQirm5SScADSVt1jf9XmhgDDmr4PI/iUVqdLjPqusoyOM/WH9PVFjTtPCWtbjYL2L/VSoGZEfEsQEQ8KukNFKF7JW0CZ9Yj9wKHSToUeAw4G3jHpGVWAx+U9DXgdcCWiHi8v9UceM6y5a6WtqDVnuf/AzgHeHbSdFGcgG3lCUkLI+IBgPSr9XTgOuA1bcqaTVlEbJP0QeDbwEzguoh4UNL70/zPAd8C3gI8DDwPvGe66jvAnGXLWl1tgaLJk7wl/Xfg/4yI75XM+0FELG26UmkesC0iniiZ9/qIuKNdxXyYp9CLcwJm3epFlsF5Bmd52DTd84yIU1vMa9pxpvljLeZVCpuZTT9n2ayc7/M0MzPrkDtPMzOzDrnzNDMz61DLzlPSTEk39KsyZmZmOWjZeUbEduDlkmb3qT5mZmYDr8oIQ48Cd6RBE56bmBgRn6qrUmZmZoOsyjnPzcAtadk5Da+OSPpyp2XMbPA4y2YV9jwj4jIASXtFxHPtlk/LTh7aT8BJkvZL6zyjSTkPJG02QLrNcirrPNvQatt5SjqBYiD4vYGDJR0DvC8iPtCi2Dzgp8A1FCPTC1gMfLLVtjyQtNnA6SrL4DzbcKty2PYq4M3APwFExHqg5QhDFOG6D7iUYoDd7wN/iIi1EbG269qaWb85y2YlKj2SLCI2STsNy7i9zfLjwJWSVqU/n6y6LTMbHM6yWbkqIdgk6UQg0i0rF1E8GLetNC7mWZJOA7Z2X00zm07OstnOqnSe7weuBuZSPDD0doony1cWEbcCt3ZcOzMbKM6yWaFK5zkeEe+svSZmZmaZqHLB0N2SVkk6VZNOfJqZmY2iKp3n4RSXm58LPCzpckmH11stMzOzwdW284zCmoh4O3A+cB5wj6S16R5QMzOzkVJlkIT9gXOAdwFPAhcCq4GFwCrg0BrrZ2ZmNnCqXDB0J/AV4K3pcvUJ6yR9rp5qmZmZDa4qnecREVE6tFZEXFE2XdLrgI0RsVXSHsAlwCKKYb4uj4gt3VbYzPrHWTYrV+WCoQMk/Y2kb0n6x4lXmzLXAc+n91cD+wJXpGnXNyskabmkdZLWjcUjVepvZvXqKsvgPNtwq7Ln+VXg68DpFAMmnAc83abMjIjYlt4vjohF6f0PJT3QrJAHkjYbOF1lGZxnG25V9jz3j4hrgRfTYNB/BhzfpswGSe9J79dLWgyQbnF5sfvqmlmfOctmJap0nhMBeVzSaZKOpXhMUSvnA38i6ZfAUcCdkh4BvpDmmVkenGWzElUO235c0r7AR4BPA/sAK1oVSBcRvFvSHGB+2s5YRDw5teqaWT85y2bl2naeEXFLersFOAlA0ooqK4+IZ4D13VbOzAaDs2y2syqHbct8uKe1MDMzy0i3nacHiDczs5HVbefpy87NzGxkNT3nKekZyjtJAXvUViMzM7MB17TzjIg5/ayImZlZLro9bGtmZjayqtzn2TFJs4Gzgc0R8R1J7wBOBDYCKyPCI5OYZcBZNitXS+dJMWD0LGBPSecBewM3A6cASyjGx92FpOXAcoAFLGKe5tdUPTOrqKssg/Nsw01NnjY2tZVKP46I10qaBTwGHBgR2yUJWB8Rr223Dg8kXVgzvsq3Bdm06UWWwXkGZ3nY1HXOc0Y63DMH2JPiMUYAfwTsVtM2zaz3nGWzEnUdtr0WeAiYCVwKrEqDSR8PfK2mbZpZ7znLZiVqOWwLIOlAgIjYLGk/4I3AbyLinirlfZin4EM9Nt2mmmVwnsFZHjZ17XkSEZsb3v8euKmubZlZfZxls135Pk8zM7MOufM0MzPrkDtPMzOzDrnzNDMz61REDPwLWN7LebmW8cuvYXgNQpamu4xf+b9y2fNc3uN5uZYxGwaDkKXpLmOZy6XzNDMzGxjuPM3MzDqUS+e5ssfzci1jNgwGIUvTXcYyV9vwfGZmZsMqlz1PMzOzgeHO08zMrEMD33lKWibpZ5IelnRJw/TrJD0lacOk5Q+S9D1JGyU9KOnihnm7S7pH0vo077JJZWdK+pGkWyZNf1TSTyQ9IGldw/T9JN0k6aG0vRPS9CPSshOvrZJWpHkfStveIOlGSbs3rO/iNP3BieXNhkWnWU7zSvPc6yynebvkuVWWU5nSPDvLI2C6bzRt9aJ4huAvgfnAbGA9cFSatxRYBGyYVOYVwKL0fg7w84YyAvZO73cD7gaObyj7YeDvgFsmrfNR4ICS+n0JOD+9nw3s1+Q7PAG8EpgL/ArYI837e+Dd6f3RwAaKBw7PAr4DHDbd/wZ++dWLVzdZTvNK89zrLKd5LfPcmOX0uTTPzvJovAZ9z3MJ8HBEPBIRL1A8fPdMgIj4AfC7yQUi4vGIuD+9fwbYSPGfnCg8mxbdLb2KXlWaB5wGXFOlYpL2oQj9tWndL0TxuKbJTgF+GRG/Tp9nAXtImkURronHPR0J3BURz0fENmAt8LYqdTHLQMdZTvNK89zLLKcyVfI8OctQnmdneQQMeuc5F9jU8HksTatE0iHAsRS/SiemzZT0APAUsCYiJuZdBXwUGC9ZVQC3S7pP0sSIIfOBp4Hr0+GhayTtVVL2bOBGgIh4DPgE8BvgcWBLRNyeltsALJW0v6Q9gbcAB1X9rmYDbkpZhl3z3MMsQ7U878gytMyzszwCBr3zLHvyeqV7ayTtDXwDWBERW3cUjtgeEQuBecASSUdLOh14KiLua7K610fEIuBU4AJJSyl+cS4CPhsRxwLPAZc0FpI0GzgDWJU+v4zi1/ahwIHAXpLOSfXaCFwBrAFuozista3KdzXLQNdZhvI89zDL0CbPk7OcppXm2VkeDQN7n+ebZpw1mBXrszXjq8oaHbOsOM/O8rAZ9D1PMzOzgTOrrhVLWkBxSGMuxeGZzcDqdEjDzDLhLJvtqpY9T0kfo7iaTsA9wL3p/Y2N93eZ2WBzls3K1XLOU9LPgVdHxIuTps8GHoyIw5qUW056/t0CFh03T/N7Xrfc+DyJTadus5yWcZ4bOMvDpa5znuMUV59N9grKLx8HICJWRsTiiFg86kEzGxBdZRmcZxtudZ3zXAF8V9IveOneroOBVwEfrGmbZtZ7K3CWzXZRS+cZEbdJOpxiVJG5FOdIxoB7I2J7Hds0s95zls3K1Xa1bUSMA3fVtX4z6w9n2WxXvs/TzMysQ+48zczMOuTO08zMrEPuPM3MzDrkztPMzKxD7jzNzMw6VFvnKWmBpFPSc/gapy+ra5tm1nvOstmuWnaekvaV9L9J+rCkD6X3+7VbqaSLgG8CFwIbJJ3ZMPvyKdXYzPrGWTYr17TzlHQucD/wBmBPYC/gJOC+NK+V9wLHRcRbU/m/lHTxxKpbbHO5pHWS1o3FI1W/g5nVp6ssg/Nsw63pU1Uk/Qx4XUT8ftL0lwF3R8ThTVcq/TQijmr4vDdwE/BT4OSIWNiuYn7yfMFPYrDp1Issg/MMzvKwaXXYVhQPvp1snDa/OIEnJC2c+BARzwKnAwcAr+mwjmY2fZxlsxKtxrb9L8D9km5n56cpvAn4qzbrPRfY1jghIrYB50r6fJd1NbP+c5bNSjTtPCPiS5JWA2/mpacpfB/43yPif7ZaaUSMtZh3R3dVNbN+c5bNyrV8qkrqJL/Wp7qYmZllwYMkmJmZdcidp5mZWYfaDZIwU9IN/aqMmZlZDlp2nhGxHXi5pNl9qo+ZmdnAa3nBUPIocEe68va5iYkR8am6KmVmZjbIqpzz3Azckpad0/DqiKQvd1rGzAaPs2xWYc8zIi4DkLRXRDzXbvm07OrJk4CTJgaVj4gzOqynmU0DZ9msXNvOU9IJwLXA3sDBko4B3hcRH2hRbB7F2JfXUAzxJ2Ax8Mk221oOLAdYwCLmaX6V72Bm9ekqy+A823Crctj2KopRhv4JICLWA0vblFkM3AdcCmyJiO8Df4iItRGxtlmhiFgZEYsjYrGDZjYQusoyOM823KpcMEREbJJ2Ggt+e5vlx4ErJa1Kfz5ZdVtmNjicZbNyVUKwSdKJQKRbVi4CNlZZeRoX8yxJpwFbu6+mmU0nZ9lsZ1U6z/cDV1MMDj8G3A5c0MlGIuJW4NaOa2dmA8VZNitU6TzHI+KdtdfEzMwsE1UuGLpb0ipJp2rSiU8zM7NRVKXzPBxYSfFQ3IclXS7p8HqrZWZmNrjadp5RWBMRbwfOB84D7pG0Nt0DamZmNlKqDJKwP3AO8C7gSeBCYDWwEFgFHFpj/czMzAZOlQuG7gS+Arw1Xa4+YZ2kz9VTLTMzs8FVpfM8IiKibEZEXNHj+piZmQ28Kp3nAZI+Crwa2H1iYkSc3KyApNcBGyNiq6Q9gEuARRRjZF4eEVumVm0z6wdn2axclattvwo8RHFu8zKK53ve26bMdcDz6f3VwL7AFWna9c0KSVouaZ2kdWPxSIWqmVnNusoyOM823NTkiOxLC0j3RcRxkn4cEa9N09ZGxJ+0KLMxIo5M7++PiEUN8x6IiIXtKvamGWe1rtiIWDO+yvfW2rTpRZbBeQZnedhU2fN8Mf35uKTTJB1L8ZiiVjZIek96v17SYoB0f+iLzYuZ2YBxls1KVDnn+XFJ+wIfAT4N7AOsaFPmfOBqSX8B/Ba4U9ImYFOaZ2Z5cJbNSrTtPCPilvR2C3ASgKQVbcpsAd4taQ4wP21nLCKenFJtzayvnGWzct0+l+/DFA/JbikingHWd7kNMxsQzrLZzqqc8yzjE99mZjayuu08R/7KOTMzG11ND9tKeobyTlLAHrXVyMzMbMA17TwjYk4/K2JmZpaLbg/bmpmZjSx3nmZmZh3q9laVliTNBs4GNkfEdyS9AzgR2AisjAiPTGKWAWfZrFwtnSfFgNGzgD0lnQfsDdwMnAIsAc4rKyRpObAcYAGLmKf5NVXPzCrqKsvgPNtwazswfFcrTYPIS5oFPAYcGBHbJQlYPzHAfCseSLrgwaRtOvUiy+A8g7M8bOo65zkjHe6ZA+xJ8RgjgD8Cdqtpm2bWe86yWYm6DtteS/EM0JnApcAqSY8AxwNfq2mbZtZ7zrJZiVoO2wJIOhAgIjZL2g94I/CbiLinSnkf5in4UI9Nt6lmGZxncJaHTV17nkTE5ob3vwduqmtbZlYfZ9lsV77P08zMrEPuPM3MzDrkztPMzKxD7jzNzMw6FRED/wKW93JermX88msYXoOQpeku41f+r1z2PJf3eF6uZcyGwSBkabrLWOZy6TzNzMwGhjtPMzOzDuXSea7s8bxcy5gNg0HI0nSXsczVNjyfmZnZsMplz9PMzGxgDHznKWmZpJ9JeljSJQ3Tr5P0lKQNk5Y/SNL3JG2U9KCkixvm7S7pHknr07zLJpWdKelHkm6ZNP1RST+R9ICkdQ3T95N0k6SH0vZOSNOPSMtOvLZKWpHmfShte4OkGyXt3rC+i9P0ByeWNxsWnWY5zSvNc6+znObtkudWWU5lSvPsLI+A6b5XptWL4jFIvwTmA7OB9cBRad5SYBGwYVKZVwCL0vs5wM8bygjYO73fDbgbOL6h7IeBvwNumbTOR4EDSur3JeD89H42sF+T7/AE8EpgLvArYI807++Bd6f3RwMbKJ6ZOAv4DnDYdP8b+OVXL17dZDnNK81zr7Oc5rXMc2OW0+fSPDvLo/Ea9D3PJcDDEfFIRLxA8fzAMwEi4gfA7yYXiIjHI+L+9P4ZYCPFf3Ki8GxadLf0KnpVaR5wGnBNlYpJ2oci9Nemdb8QxRMnJjsF+GVE/Dp9ngXsIWkWRbgmnlhxJHBXRDwfEduAtcDbqtTFLAMdZznNK81zL7OcylTJ8+QsQ3meneURMOid51xgU8PnsTStEkmHAMdS/CqdmDZT0gPAU8CaiJiYdxXwUWC8ZFUB3C7pPkkTNz3PB54Grk+Hh66RtFdJ2bOBGwEi4jHgE8BvgMeBLRFxe1puA7BU0v6S9gTeAhxU9buaDbgpZRl2zXMPswzV8rwjy9Ayz87yCBj0zrPs4bGVLg+WtDfwDWBFRGzdUThie0QsBOYBSyQdLel04KmIuK/J6l4fEYuAU4ELJC2l+MW5CPhsRBwLPAdc0lhI0mzgDGBV+vwyil/bhwIHAntJOifVayNwBbAGuI3isNa2Kt/VLANdZxnK89zDLEObPE/OcppWmmdneTQMeuc5xs6/2Obx0mHOpiTtRhG0r0bEzWXLpEMy3weWAa8HzpD0KMXhpJMl3dCw7Ob051PAP1AcghoDxhp+7d5EEb5GpwL3R8ST6fMbgV9FxNMR8SJwM3Biw3aujYhFEbGU4jDWL9p9V7NMdJVlaJ/nHmR5on6t8jw5y9Aiz87y8Bv0zvNe4DBJh6ZffmcDq1sVkCSK8xYbI+JTk+a9XNJ+6f0eFP/5H4qI/z0i5kXEIWkb/xgR56Tl9pI0Z+I98G8pLmx4Atgk6Yi0+lOAn06qzttpOMxDcXjneEl7pnqeQnEOZ6J+/zL9eTDwp5PKmuWs4yxD8zz3MssAFfI8OcvQIs/O8vCbNd0VaCUitkn6IPBtiivdrouIBwEk3Qi8AThA0hjwf0TEtRS/PN8F/CSdDwH4TxHxLYor974kaSbFD4e/j4idLmUv8b8A/1Bkg1nA30XEbWnehcBXU2PwCPCeiULpXMebgPc1fJ+7Jd0E3E9xGOdH7DwCyTck7Q+8CFwQEf+z2t+U2WDrMsvQJM8Ue4q9zDI0yXNZltN3apVnZ3nIeYQhMzOzDg36YVszM7OB487TzMysQ+48zczMOuTO08zMrEPuPM3MzDrkzpMdT274laR/kT6/LH1+paTzJP0ivc6b7rqajao03N3E002ekPRYw+fZPd7WfpI+0Mt12nDxrSqJpI8Cr4qI5ZI+T/H0hc8D64DFFEOJ3Qcc53u2zKaXpP8P8GxEfKLCsrPSAO2drP8QiieyHN1dDW3Yec/zJVdSjBayAvg3wCeBN1MMOP271GGuoRgCzMwGgKT3SrpXxXM9v5EGNEDSFyV9StL3gCsk/WtJd6Vl/7OkZxvW8R/T9B/rpeeC/jXwr9Ne7d9Mw1ezATfQIwz1U0S8KOk/Ugzk/G8j4gVJU34ShJnV6uaI+AKApI8Dfw58Os07HHhjRGxX8VDsqyPiRknvnygs6d8Ch1GMcStgdRos/hLg6DTwvNkuvOe5s1MpHi00cahmSk+CMLPaHS3pf0j6CfBO4NUN81ZFxPb0/gReeiLK3zUs82/T60cUw+wtoOhMzVrynmciaSHF+JXHAz+U9DWKPc03NCw2j+LpDWY2GL4IvDUi1kt6Nzvn9bkK5QX8fyPi8ztNLM55mjXlPU92PLnhsxTPCvwN8DcUD7n9NvBv09W3L6P4hfrt6aupmU0yB3g8PbbsnS2Wuwv4X9P7sxumfxv4MxXPC0XS3PRElGfSus1KufMsvBf4TUSsSZ//G8Xhm9cAf0XxOKV7gf8cEb+bniqaWYm/BO6muJjvoRbLrQA+LOkeiqcrbQGIiNspDuPemQ793gTMiYh/Au6QtMEXDFkZ36piZkMvXYX7h4gISWcDb4+IM6e7XpYvn/M0s1FwHPBf0yma3wN/Nr3Vsdx5z9PMzKxDPudpZmbWIXeeZmZmHXLnaWZm1iF3nmZmZh1y52lmZtYhd55mZmYd+v8DdNyV2qrUCYUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x1024 with 22 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "idx=74\n",
    "print((pred[0].shape))\n",
    "thr=0.35\n",
    "vote=(pred[0]>thr).long()\n",
    "for i in range(1,5):\n",
    "    vote+=(pred[i]>thr).long()\n",
    "plot_x0_targ((vote>2).detach().cpu().numpy()[idx], y.detach().cpu().numpy()[idx])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c09f2dbb",
   "metadata": {},
   "source": [
    "With this we can finally calculate the IOU for the whole model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "fa96b9ea",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.04370796152042775"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "binary_iou((vote>2).long().detach().cpu().numpy(), y.detach().cpu().numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12dcadae",
   "metadata": {},
   "source": [
    "## Submission prep\n",
    "\n",
    "The final step is to load the test dataset and perform the inference step to calculate the prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "60566983",
   "metadata": {},
   "outputs": [],
   "source": [
    "with h5py.File(DATA/'test.h5', 'r') as h5:\n",
    "    test2_x = h5['x0'][()]\n",
    "h5.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "e369c968",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "30036"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(test2_x)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28b52549",
   "metadata": {},
   "source": [
    "We compute our predictions just like we did on the validation data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "551e15fe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([30036, 10, 10, 10])\n",
      "torch.Size([30036, 10, 10, 10])\n"
     ]
    }
   ],
   "source": [
    "test_data2=CustomImageDataset(test2_x,test2_x)\n",
    "\n",
    "test_dataloader2 = DataLoader(test_data2, batch_size=30036, shuffle=False, collate_fn=lambda x: tuple(x_.to(device) for x_ in default_collate(x)))\n",
    "\n",
    "with torch.no_grad():\n",
    "    pred=[]\n",
    "    for model in models:\n",
    "        dataiter = iter(test_dataloader2)\n",
    "        X, y = dataiter.next()\n",
    "        #m=nn.Unflatten()\n",
    "        pred.append(model(X))\n",
    "    \n",
    "thr=0.35\n",
    "vote=(pred[0]>thr).long()\n",
    "for i in range(1,5):\n",
    "    vote+=(pred[i]>thr).long()\n",
    "\n",
    "vote= vote>2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f688fd9",
   "metadata": {},
   "source": [
    "To submit predictions, please save them to a HDF5 file using the function below. The file name should use the format `[your_full_name]_test_preds.h5`. Ypu can also include an option ID number in case you submit multiple times. Your final score, and score for the Friday announcements, will be computed on your submission with the highest ID number."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "68ad2036",
   "metadata": {},
   "outputs": [],
   "source": [
    "FULL_NAME_AND_ID = 'PHILIPP_MUNKES_ID0X'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "6408cf91",
   "metadata": {},
   "outputs": [],
   "source": [
    "with h5py.File(f'{FULL_NAME_AND_ID}_test_preds.h5', 'w') as h5:\n",
    "    h5.create_dataset(\"preds\", data=vote.cpu().detach().numpy().astype(\"int\"), dtype=\"int\", compression='lzf')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8563c1e7",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "vscode": {
   "interpreter": {
    "hash": "3a3c528b3647304ed48111236c271ef4a9b919270978bc3379f9e9bd1a237755"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}