Even outside the computer science industry, people are familiar with many well-known neural network architectures, such as CNNs, which are very powerful in image processing, and RNNs, which can model sequential data. However, neural network architectures like CNNs and RNNs cannot be used to perform tasks such as content and style separation of images, generating realistic images, classifying images with limited label information, or performing data compression. This is because these tasks require specialized network structures and training algorithms.
Is there a network architecture that can handle all of the above tasks? Obviously, there is: the Adversarial Autoencoder (AAE). In this paper, we will build an AAE to compress data, separate the content and style of images, classify images with a small number of samples, and then generate them.
Implementing a self-encoder in PyTorch
First, let's review what an autoencoder is, and then implement it simply using PyTorch.
1. Self-encoder
As shown in the figure, the input and output of an autoencoder are the same, meaning it does not require supervisory information (labels). It mainly consists of two parts:
• Encoder : Input data (can be text, image, video, or audio), output latent code. For example, in the image above, the input data is an image, and the output is the hidden layer value h, or latent code. The value of h can be set arbitrarily. In this configuration, the encoder compresses the image, transforming it from a single image to a more complex one, just like compressing an image using compression software (such as WinRAR). If we denote the encoder as a function q, then the encoder does the following:
• Decoder: The input data is the output data h from the previous step. It attempts to reconstruct h into x. In the example above, the Decoder needs to reconstruct h back into x, making it as similar as possible to the original x, just like compressing a compressed file. If we denote the Decoder as a function p, then the Decoder is doing the following:
This model seems like a natural dimensionality reduction model. But besides dimensionality reduction, what else can an Autoencoder do?
Image denoising involves taking a noisy image as input and using an autoencoder to generate a clear, noise-free image. When data is input into an autoencoder, we can force the hidden layer of the autoencoder to learn more robust features instead of simply recognizing them. Such an autoencoder, trained on the left image below, can reconstruct the noisy data in the middle into the image on the right.
Semantic hashing can reduce the dimensionality of data and accelerate information retrieval; currently, many people are researching this area.
Generative models, such as the AdversarialAutoencoder (AAE), which will be introduced in this series of articles.
Other numerous applications
2. Implementation in PyTorch
We'll start our first part with a simple, fully connected network.
This encoder contains an input layer, two hidden layers (each with 1000 nodes), and an output layer with 2^36 nodes.
Therefore, the entire model is:
After the model is implemented, we need to prepare the data:
We choose the MSE loss function to measure the similarity between the reconstructed image and the original image x.
The training steps can then be implemented:
Let's take a look at the reconstructed image:
We can observe that some strange parts of the input image of the number 3 have been removed (the top left corner of the number 3).
Next, let's look at latentcode, which is only 2-dimensional. We can fill in any value and let the decoder generate an image, for example, let's consider a value like 'let', and then input it into the decoder:
This looks like an image of a 6, but it could also be a 0; in any case, it's not a clear image of a number. This is because the Encoder's output doesn't cover the entire 2D space (its output distribution has a lot of gaps). Therefore, if we input some values that the Decoder hasn't seen before, we'll see some strange output images. This can be achieved by restricting the Encoder's output to a random distribution (e.g., a normal distribution with a mean of 0.0 and a standard deviation of 2.0) when generating the latent code. AdversarialAutoencoder does just that, and we'll look at its implementation in Part 2.
