{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classification of Pet's Faces\n",
    "\n",
    "Lab Assignment from [AI for Beginners Curriculum](https://github.com/microsoft/ai-for-beginners).\n",
    "\n",
    "### Getting the Data\n",
    "\n",
    "In this assignment, we will focus on relatively simple classification task - classification of pet's faces. This dataset consists of cut-out faces from [Oxford-IIIT Dataset](https://www.robots.ox.ac.uk/~vgg/data/pets/). Let's start by loading and visualizing the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2022-02-17 12:32:43--  https://mslearntensorflowlp.blob.core.windows.net/data/petfaces.tar.gz\n",
      "Resolving mslearntensorflowlp.blob.core.windows.net... 20.150.90.68\n",
      "Connecting to mslearntensorflowlp.blob.core.windows.net|20.150.90.68|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 24483412 (23M) [application/x-gzip]\n",
      "Saving to: ‘petfaces.tar.gz’\n",
      "\n",
      "petfaces.tar.gz     100%[===================>]  23.35M  12.5MB/s    in 1.9s    \n",
      "\n",
      "2022-02-17 12:32:45 (12.5 MB/s) - ‘petfaces.tar.gz’ saved [24483412/24483412]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!wget https://mslearntensorflowlp.blob.core.windows.net/data/petfaces.tar.gz\n",
    "!tar xfz petfaces.tar.gz\n",
    "!rm petfaces.tar.gz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will define generic function to display a series of images from a list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "from PIL import Image\n",
    "import numpy as np\n",
    "\n",
    "def display_images(l,titles=None,fontsize=12):\n",
    "    n=len(l)\n",
    "    fig,ax = plt.subplots(1,n)\n",
    "    for i,im in enumerate(l):\n",
    "        ax[i].imshow(im)\n",
    "        ax[i].axis('off')\n",
    "        if titles is not None:\n",
    "            ax[i].set_title(titles[i],fontsize=fontsize)\n",
    "    fig.set_size_inches(fig.get_size_inches()*n)\n",
    "    plt.tight_layout()\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's traverse all class subdirectories and plot first few images of each class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for cls in os.listdir('petfaces'):\n",
    "    print(cls)\n",
    "    display_images([Image.open(os.path.join('petfaces',cls,x)) \n",
    "                    for x in os.listdir(os.path.join('petfaces',cls))[:10]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also define the number of classes in our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "35"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "num_classes = len(os.listdir('petfaces'))\n",
    "num_classes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing dataset for Deep Learning\n",
    "\n",
    "To start training our neural network, we need to convert all images to tensors, and also create tensors corresponding to labels (class numbers). Most neural network frameworks contain simple tools for dealing with images:\n",
    "* In Tensorflow, use `tf.keras.preprocessing.image_dataset_from_directory`\n",
    "* In PyTorch, use `torchvision.datasets.ImageFolder`\n",
    "\n",
    "As you have seen from the pictures above, all of them are close to square image ratio, so we need to resize all images to square size. Also, we can organize images in minibatches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CODE TO LOAD DATASET"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we need to separate dataset into train and test portions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CODE TO DO TRAIN/TEST SPLIT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's print the size of tensors in our dataset. If you have done everything correctly, the size of training elements should be\n",
    " * `(batch_size,image_size,image_size,3)` for Tensorflow, `batch_size,3,image_size,image_size` for PyTorch\n",
    " * `batch_size` for Labels\n",
    " \n",
    " Labels should contain numbers of classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print tensor sizes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 448592 characters]",
      "text/plain": [
       "<Figure size 3024x2016 with 7 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 999712 characters]",
      "text/plain": [
       "<Figure size 3024x2016 with 7 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 1226520 characters]",
      "text/plain": [
       "<Figure size 3024x2016 with 7 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Display the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define a neural network\n",
    "\n",
    "For image classification, you should probably define a convolutional neural network with several layers. What to keep an eye for:\n",
    "* Keep in mind the pyramid architecture, i.e. number of filters should increase as you go deeper\n",
    "* Do not forget activation functions between layers (ReLU) and Max Pooling\n",
    "* Final classifier can be with or without hidden layers, but the number of output neurons should be equal to number of classes.\n",
    "\n",
    "An important thing is to get the activation function on the last layer + loss function right:\n",
    "* In Tensorflow, you can use `softmax` as the activation, and `sparse_categorical_crossentropy` as loss. The difference between sparse categorical cross-entropy and non-sparse one is that the former expects output as the number of class, and not as one-hot vector.\n",
    "* In PyTorch, you can have the final layer without activation function, and use `CrossEntropyLoss` loss function. This function applies softmax automatically. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CODE TO DEFINE NEURAL NETWORK"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the Neural Network\n",
    "\n",
    "Now we are ready to train the neural network. During training, please collect accuracy on train and test data on each epoch, and then plot the accuracy to see if there is overfitting.\n",
    "\n",
    "> To speed up training, you need to use GPU if available. While TensorFlow/Keras will automatically use GPU, in PyTorch you need to move both the model and data to GPU during training using `.to()` method in order to take advantage of GPU acceleration. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TRAIN THE NETWORK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 17140 characters]",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# PLOT THE ACCURACY on train and validation dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What can you say about overfitting? What can be done to improve the accuracy of the model\n",
    "\n",
    "## Optional: Calculate Top3 Accuracy\n",
    "\n",
    "In this exercise, we were dealing with classification with quite high number of classes (35), so our result - around 50% validation accuracy - is pretty good. Standard ImageNet dataset has even more - 1000 classes.\n",
    "\n",
    "In such cases it is difficult to ensure that model **always** correctly predicts the class. There are cases when two breeds are very similar to each other, and the model returns very similar probablities (eg., 0.45 and 0.43). If we measure standard accuracy, it will be considered a wrong case, even though the model did very small mistake. This, we often measure another metric - an accuracy within top 3 most probable predictions of the model.\n",
    "\n",
    "We consider the case accurate if target label is contained within top 3 model predictions. \n",
    "\n",
    "To compute top-3 accuracy on the test dataset, you need to manually go over the dataset, apply the neural network to get the prediction, and then do the calculations. Some hints:\n",
    "\n",
    "* In Tensorflow, use `tf.nn.in_top_k` function to see if the `predictions` (output of the model) are in top-k (pass `k=3` as parameter), with respect to `targets`. This function returns a tensor of boolean values, which can be converted to `int` using `tf.cast`, and then accumulated using `tf.reduce_sum`.\n",
    "* In PyTorch, you can use `torch.topk` function to get indices of classes with highers probabilities, and then see if the correct class belongs to them. See [this](https://gist.github.com/weiaicunzai/2a5ae6eac6712c70bde0630f3e76b77b) for more hints.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CALCULATE TOP-3 ACCURACY"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optional: Build Cats vs. Dogs classification\n",
    "\n",
    "We also want to see how accurate our binary cats vs. dogs classification would be on the same dateset. To do it, we need to adjust labels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define dataset that contains only two labels: 0 = cat, 1 = dog\n",
    "# Hint: use class name prefix to figure out which one is which"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define neural network architecture and train"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "86193a1ab0ba47eac1c69c1756090baa3b420b3eea7d4aafab8b85f8b312f0c5"
  },
  "kernelspec": {
   "display_name": "py37_pytorch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
