{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jUWeYIfl6h8C"
      },
      "source": [
        "\n",
        "# Genereative Adversarial Networks\n",
        "\n",
        "The main goal of **Generative Adversarial Network** (GAN) is to generate images that are similar (but not identical) to training dataset.\n",
        "\n",
        "GAN consists of two neural networks that are trained against each other:\n",
        "\n",
        " * **Generator** takes a random vector, and should generate an image from it\n",
        " * **Discriminator** is a networks that should distinguish between original image (from training dataset), and the one generated by the generator.\n",
        "\n",
        "<img src=\"./images/gan_architecture.png\" width=\"70%\"/>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "id": "vh9JiNcw80sd",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "import tensorflow as tf\n",
        "import tensorflow.keras as keras\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import *\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MlNHFbgyJ0t-"
      },
      "source": [
        "## Generator\n",
        "\n",
        "The role of a generator is to take a random vector of some size (it is similar to latent vector in autoencoders) and generate the target image. It is very similar to the generative side of autoencoder.\n",
        "\n",
        "In our example, we will use dense neural networks and MNIST dataset."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "iOWm55Bd5DNl",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "generator = Sequential()\n",
        "generator.add(Dense(256, input_shape=(100,)))\n",
        "generator.add(LeakyReLU(alpha=0.2))\n",
        "generator.add(BatchNormalization(momentum=0.8))\n",
        "generator.add(Dense(512))\n",
        "generator.add(LeakyReLU(alpha=0.2))\n",
        "generator.add(BatchNormalization(momentum=0.8))\n",
        "generator.add(Dense(1024))\n",
        "generator.add(LeakyReLU(alpha=0.2))\n",
        "generator.add(BatchNormalization(momentum=0.8))\n",
        "generator.add(Dense(784, activation='tanh'))\n",
        "generator.add(Reshape((28,28)))\n",
        "\n",
        "optimizer = keras.optimizers.Adam(lr=0.0002, decay=8e-9)\n",
        "\n",
        "generator.compile(loss='binary_crossentropy',optimizer=optimizer,metrics=['accuracy'])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tKbYa6ABKDlt"
      },
      "source": [
        "A few tricks used in generator:\n",
        "* Instead of ReLU, we use **Leaky ReLU**, i.e. a ReLU which is not exactly 0 for negative $x$, but rather another linear function with very small slope. This is important, because it helps gradient descent to propagate values even if we are on the negative side of ReLU (where values are 0)\n",
        "* We use Batch Normalization in order to stabilize training\n",
        "* The activation function on last layer is `tanh`, so the output is in the range [-1,1]\n",
        "\n",
        "## Discriminator\n",
        "\n",
        "Discriminator is a classical image classification network. In our first example, we will also use dense classifier."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "krxyG2oh87pU",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "discriminator = Sequential()\n",
        "discriminator.add(Flatten(input_shape=(28,28)))\n",
        "discriminator.add(Dense(784))\n",
        "\n",
        "discriminator.add(LeakyReLU(alpha=0.2))\n",
        "discriminator.add(Dense(784//2))\n",
        "discriminator.add(LeakyReLU(alpha=0.2))\n",
        "\n",
        "discriminator.add(Dense(1, activation='sigmoid'))\n",
        "\n",
        "discriminator.compile(loss='binary_crossentropy',optimizer=optimizer,metrics=['accuracy'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kH2ox9EIKO8F"
      },
      "source": [
        "We will also define an adversarial network, which is generator followed by discriminator. This network starts with a noise vector, and returns a binary result."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "TsJH1VWQ9yeB",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "discriminator.trainable = False\n",
        "adversarial = Sequential()\n",
        "adversarial.add(generator)\n",
        "adversarial.add(discriminator)\n",
        "adversarial.compile(loss='binary_crossentropy', optimizer=optimizer)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Loading dataset\n",
        "\n",
        "We will use MNIST dataset, built into Keras:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "id": "ZzX0kMrW-dQA",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "(X_train, _), (_, _) = keras.datasets.mnist.load_data()\n",
        "X_train = (X_train.astype(np.float32) - 127.5) / 127.5"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EhyAVbdMKWaq"
      },
      "source": [
        "## Network training\n",
        "\n",
        "On each step of the training, we have two phases:\n",
        "\n",
        "* Training discriminator:\n",
        "   - We generate some random vectors `noise`. Training happens in minibatches, so we use `batch//2` vectors to produce `batch//2` generated images\n",
        "   - Sample `batch//2` random images from the dataset\n",
        "   - Train discriminator on 50% real and 50% generated images, providing corresponding labels (0 or 1)\n",
        " * Train the generator by using combined adversarial model, passing random vectors as input, and expecting 1's as output (which corresponds to real images)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "id": "aISx_wo4W1qJ",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "def plotn(n):\n",
        "  noise = np.random.normal(0, 1, (n,100))\n",
        "  imgs = generator.predict(noise)\n",
        "  fig,ax = plt.subplots(1,n)\n",
        "  for i,im in enumerate(imgs):\n",
        "    ax[i].imshow(im.reshape(28,28))\n",
        "  plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 673
        },
        "id": "mNdcpEa3_Pqs",
        "outputId": "75ebec09-e608-4747-8298-0cb2281d95af",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 0, [Discriminator :: d_loss: 0.601463], [ Generator :: loss: 0.640677]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 55440 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 500, [Discriminator :: d_loss: 0.192001], [ Generator :: loss: 12.124918]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 56512 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 1000, [Discriminator :: d_loss: 0.141956], [ Generator :: loss: 1.900380]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 37860 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 1500, [Discriminator :: d_loss: 0.293635], [ Generator :: loss: 2.443017]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 34096 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 2000, [Discriminator :: d_loss: 0.547135], [ Generator :: loss: 2.680543]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 30628 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 2500, [Discriminator :: d_loss: 0.491767], [ Generator :: loss: 2.633016]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 29280 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "batch=32\n",
        "for cnt in range(3000):\n",
        "## train discriminator\n",
        "  random_index =  np.random.randint(0, len(X_train) - batch//2)\n",
        "  legit_images = X_train[random_index : random_index + batch//2].reshape(batch//2, 28, 28)\n",
        "  gen_noise = np.random.normal(0, 1, (batch//2,100))\n",
        "  syntetic_images = generator.predict(gen_noise)\n",
        "  x_combined_batch = np.concatenate((legit_images, syntetic_images))\n",
        "  y_combined_batch = np.concatenate((np.ones((batch//2, 1)), np.zeros((batch//2, 1))))\n",
        "  d_loss = discriminator.train_on_batch(x_combined_batch, y_combined_batch)\n",
        "# train generator\n",
        "  noise = np.random.normal(0, 1, (batch,100))\n",
        "  y_mislabled = np.ones((batch, 1))\n",
        "  g_loss = adversarial.train_on_batch(noise, y_mislabled)\n",
        "  if cnt%500==0:\n",
        "    print ('epoch: %d, [Discriminator :: d_loss: %f], [ Generator :: loss: %f]' % (cnt, d_loss[0], g_loss))\n",
        "    plotn(5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fJH1n4wRkqPy"
      },
      "source": [
        "> **Task**: You can train this GAN on the whole MNIST dataset and see how good can it get"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L-wH0b7pT0CI"
      },
      "source": [
        "## DCGAN\n",
        "\n",
        "In the previous example, we have used dense networks for both generator and discriminator, but we know that CNNs provide better performance when dealing with images. **Deep Convolutional GAN** is similar to the architecture above, but it uses convolutional layers for generator and discriminator. \n",
        "\n",
        "The main difficulty here is to build an architecture for denerator, because it has to do an inverse task compared to traditional CNN - it has to generate image from feature vector. In a way, this is similar to decoder part of autoencoders.That's why we will be using `Conv2DTranspose` layers in the generator."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fkD2WufskBdm",
        "outputId": "b4a22c66-a493-4abd-dc79-55c9fd7be137",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "-1.0 1.0\n"
          ]
        }
      ],
      "source": [
        "(X_train, _), (_, _) = keras.datasets.mnist.load_data()\n",
        "X_train = (X_train.astype(np.float32)-127.5) / 127.5\n",
        "print(X_train.min(),X_train.max())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tAk-hsF0TzEv",
        "outputId": "68bba515-be2d-44f5-f079-b461456ed082",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Model: \"sequential_3\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_7 (Dense)              (None, 6272)              633472    \n",
            "_________________________________________________________________\n",
            "reshape_1 (Reshape)          (None, 7, 7, 128)         0         \n",
            "_________________________________________________________________\n",
            "up_sampling2d (UpSampling2D) (None, 14, 14, 128)       0         \n",
            "_________________________________________________________________\n",
            "conv2d_transpose (Conv2DTran (None, 14, 14, 128)       147584    \n",
            "_________________________________________________________________\n",
            "batch_normalization_3 (Batch (None, 14, 14, 128)       512       \n",
            "_________________________________________________________________\n",
            "activation (Activation)      (None, 14, 14, 128)       0         \n",
            "_________________________________________________________________\n",
            "up_sampling2d_1 (UpSampling2 (None, 28, 28, 128)       0         \n",
            "_________________________________________________________________\n",
            "conv2d_transpose_1 (Conv2DTr (None, 28, 28, 64)        73792     \n",
            "_________________________________________________________________\n",
            "batch_normalization_4 (Batch (None, 28, 28, 64)        256       \n",
            "_________________________________________________________________\n",
            "activation_1 (Activation)    (None, 28, 28, 64)        0         \n",
            "_________________________________________________________________\n",
            "conv2d_transpose_2 (Conv2DTr (None, 28, 28, 1)         577       \n",
            "_________________________________________________________________\n",
            "activation_2 (Activation)    (None, 28, 28, 1)         0         \n",
            "=================================================================\n",
            "Total params: 856,193\n",
            "Trainable params: 855,809\n",
            "Non-trainable params: 384\n",
            "_________________________________________________________________\n"
          ]
        }
      ],
      "source": [
        "generator = Sequential()\n",
        "generator.add(Dense(128 * 7 * 7, activation=\"relu\", input_dim=100))\n",
        "generator.add(Reshape((7, 7, 128)))\n",
        "generator.add(UpSampling2D())\n",
        "generator.add(Conv2DTranspose(128, kernel_size=3, padding=\"same\"))\n",
        "generator.add(BatchNormalization(momentum=0.8))\n",
        "generator.add(Activation(\"relu\"))\n",
        "generator.add(UpSampling2D())\n",
        "generator.add(Conv2DTranspose(64, kernel_size=3, padding=\"same\"))\n",
        "generator.add(BatchNormalization(momentum=0.8))\n",
        "generator.add(Activation(\"relu\"))\n",
        "generator.add(Conv2DTranspose(1, kernel_size=3, padding=\"same\"))\n",
        "generator.add(Activation(\"tanh\"))\n",
        "\n",
        "optimizer = keras.optimizers.Adam(0.0001) #, 0.5)\n",
        "\n",
        "generator.compile(loss='binary_crossentropy',optimizer=optimizer,metrics=['accuracy'])\n",
        "generator.summary()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "t9FZHScTUsCd",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "discriminator = Sequential()\n",
        "\n",
        "discriminator.add(Conv2D(32, kernel_size=3, strides=2, input_shape=(28,28,1), padding=\"same\"))\n",
        "discriminator.add(LeakyReLU(alpha=0.2))\n",
        "discriminator.add(Dropout(0.25))\n",
        "discriminator.add(Conv2D(64, kernel_size=3, strides=2, padding=\"same\"))\n",
        "discriminator.add(ZeroPadding2D(padding=((0,1),(0,1))))\n",
        "discriminator.add(BatchNormalization(momentum=0.8))\n",
        "discriminator.add(LeakyReLU(alpha=0.2))\n",
        "discriminator.add(Dropout(0.25))\n",
        "discriminator.add(Conv2D(128, kernel_size=3, strides=2, padding=\"same\"))\n",
        "discriminator.add(BatchNormalization(momentum=0.8))\n",
        "discriminator.add(LeakyReLU(alpha=0.2))\n",
        "discriminator.add(Dropout(0.25))\n",
        "discriminator.add(Conv2D(256, kernel_size=3, strides=1, padding=\"same\"))\n",
        "discriminator.add(BatchNormalization(momentum=0.8))\n",
        "discriminator.add(LeakyReLU(alpha=0.2))\n",
        "discriminator.add(Dropout(0.25))\n",
        "discriminator.add(Flatten())\n",
        "discriminator.add(Dense(1, activation='sigmoid'))\n",
        "\n",
        "discriminator.compile(loss='binary_crossentropy',optimizer=optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "2Jz166pQWRba",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [],
      "source": [
        "discriminator.trainable = False\n",
        "adversarial = Sequential()\n",
        "adversarial.add(generator)\n",
        "adversarial.add(discriminator)\n",
        "adversarial.compile(loss='binary_crossentropy', optimizer=optimizer)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "kohaucz-WTue",
        "outputId": "edf64874-5ec9-4c10-cb36-95e2bb5111cf",
        "vscode": {
          "languageId": "python"
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 0, [Discriminator :: d_loss: 0.957905], [ Generator :: loss: 0.695994]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 44480 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 100, [Discriminator :: d_loss: 0.826593], [ Generator :: loss: 1.488088]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 47732 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 200, [Discriminator :: d_loss: 0.602254], [ Generator :: loss: 1.362499]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 41312 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 300, [Discriminator :: d_loss: 0.711605], [ Generator :: loss: 1.224355]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 42252 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 400, [Discriminator :: d_loss: 0.650690], [ Generator :: loss: 0.899742]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 41184 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 500, [Discriminator :: d_loss: 0.413256], [ Generator :: loss: 1.106550]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 40876 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 600, [Discriminator :: d_loss: 0.574668], [ Generator :: loss: 1.045400]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 35324 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 700, [Discriminator :: d_loss: 0.522089], [ Generator :: loss: 1.068755]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 37196 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 800, [Discriminator :: d_loss: 0.300957], [ Generator :: loss: 1.292961]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 31180 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "epoch: 900, [Discriminator :: d_loss: 0.389556], [ Generator :: loss: 0.942707]\n"
          ]
        },
        {
          "data": {
            "image/png": "[encoded data, 34428 characters]",
            "text/plain": [
              "<Figure size 432x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "needs_background": "light",
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "batch=32\n",
        "y_labeled = np.ones((batch, 1))\n",
        "y_mislabeled = np.zeros((batch, 1))\n",
        "for cnt in range(1000):\n",
        "## train discriminator\n",
        "  random_index =  np.random.randint(0, len(X_train) - batch)\n",
        "  legit_images = X_train[random_index : random_index + batch].reshape(batch, 28, 28, 1)\n",
        "  gen_noise = np.random.normal(0, 1, (batch,100))\n",
        "  syntetic_images = generator.predict(gen_noise)\n",
        "  d_loss_1 = discriminator.train_on_batch(legit_images, y_labeled)\n",
        "  d_loss_2 = discriminator.train_on_batch(syntetic_images, y_mislabeled)\n",
        "  d_loss = 0.5*np.add(d_loss_1,d_loss_2)\n",
        "# train generator\n",
        "  g_loss = adversarial.train_on_batch(gen_noise, y_labeled)\n",
        "  if cnt%100==0:\n",
        "    print ('epoch: %d, [Discriminator :: d_loss: %f], [ Generator :: loss: %f]' % (cnt, d_loss, g_loss))\n",
        "    plotn(5)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bqkGxSk8mXFr"
      },
      "source": [
        "> **Task**: Try generating more complex color images with DCGAN - for example, take one class from [CIFAR-10](https://keras.io/api/datasets/cifar10/) dataset."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Wc8rDu7uk98o"
      },
      "source": [
        "## Training on Paintings\n",
        "\n",
        "One of the good candidates for GAN training are paintings created by human artists. Below is a sample image produced by DCGAN trained on a dataset from [WikiArt](http://wikiart.org). [KeraGAN](http://github.com/shwars/keragan) library was used to produce this image [using Azure Machine Learning](https://soshnikov.com/scienceart/creating-generative-art-using-gan-on-azureml/)\n",
        "\n",
        "![](https://soshnikov.com/images/artartificial/Flo1.jpg)\n",
        "\n",
        "(Photo from [Art of Artificial](https://soshnikov.com/museum/art-artificial/) collection)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q_A3nR10Tkih"
      },
      "source": [
        "## References\n",
        "\n",
        "* [Keras implementation of different toy GAN architectures](https://github.com/eriklindernoren/Keras-GAN)\n",
        "* [KeraGAN Library](http://github.com/shwars/Keragan)\n",
        "* [Blog post about creating GANs on Azure ML](https://soshnikov.com/scienceart/creating-generative-art-using-gan-on-azureml-ru/)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [],
      "name": "GANs.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
