Unveiling Denoising Autoencoders

Introduction

Denoising Autoencoders are neural network models that remove noise from corrupted or noisy data by learning to reconstruct the initial data from its noisy counterpart. We train the model to minimize the disparity between the original and reconstructed data. We can stack these autoencoders together to form deep networks, increasing their performance.

Additionally, tailor this architecture to handle a variety of data formats, including images, audio, and text. Additionally, customise the noise, such as including salt-and-pepper or Gaussian noise. As the DAE reconstructs the image, it effectively learns the input features, leading to enhanced extraction of latent representations. It is important to highlight that the Denoising Autoencoder reduces the likelihood of learning the identity function compared to a regular autoencoder.

Learning Objectives

• An overview of denoising automatic encoders (DAEs) and their use in obtaining a low-dimensional representation by reconstructing the original data from noisy types.
• We will also cover aspects of DAE architecture, including encoder and decoder components.
• Examining their performance can provide insight into their role in reconstructing the original data from their noisy counterparts.
• Furthermore, we consider various applications of DAE such as denoising, compression, feature extraction, and representation learning. As an illustrative example, we focus on the DAE implementation for image denoising using the Keras dataset.

This article was published as a part of the Data Science Blogathon.

What is Denoising Autoencoders?

Denoising autoencoders are a specific type of neural network that enables unsupervised learning of data representations or encodings. Their primary objective is to reconstruct the original version of the input signal corrupted by noise. This capability proves valuable in problems such as image recognition or fraud detection, where the goal is to recover the original signal from its noisy form.

An autoencoder consists of two main components:

• Encoder: This component maps the input data into a low-dimensional representation or encoding.
• Decoder: This component returns the encoding to the original data space.

During the training phase, present the autoencoder with a set of clean input examples along with their corresponding noisy versions. The objective is to learn a task using an encoder-decoder architecture that efficiently transforms noisy input into clean output.

Architecture of DAE

The denoising autoencoder (DAE) architecture is similar to a standard autoencoder. It consists of two main components:

Encoder

• The encoder creates a neural network equipped with one or more hidden layers.
• Its purpose is to receive noisy input data and generate an encoding, which represents a low-dimensional representation of the data.
• Understand an encoder as a compression function because the encoding has fewer parameters than the input data.

Decoder

• Decoder acts as an expansion function, which is responsible for reconstructing the original data from the compressed encoding.
• It takes as input the encoding generated by the encoder and reconstructs the original data.
• Like encoders, decoders are implemented as neural networks featuring one or more hidden layers.

During the training phase, present the denoising autoencoder (DAE) with a collection of clean input examples along with their respective noisy counterparts. The objective is to acquire a function that maps a noisy input to a relatively clean output using an encoder-decoder architecture. To achieve this, a reconstruction loss function is typically employed to evaluate the disparity between the clean input and the reconstructed output. A DAE is trained by minimizing this loss through the use of backpropagation, which involves updating the weights of both encoder and decoder components.

Applications of Denoising Autoencoders (DAEs) span a variety of domains, including computer vision, speech processing, and natural language processing.

Examples

• Image Denoising: DAEs are effective in removing noise from images, such as Gaussian noise or salt-and-pepper noise.
• Fraud Detection: DAEs can contribute to identifying fraudulent transactions by learning to reconstruct common transactions from their noisy counterparts.
• Data Imputation: To reconstruct missing values ​​from available data by learning, DAEs can facilitate data imputation in datasets with incomplete information.
• Data Compression: DAEs can compress data by obtaining a concise representation of the data in the encoding space.
• Anomaly Detection: Using DAEs, anomalies in a dataset can be detected by training a model to reconstruct normal data and then flag challenging inputs as potentially abnormal.

Image Denoising with Denoising Autoencoder

Denoising autoencoders (DAEs) provide a powerful solution for reconstructing the original data from its noisy version. In particular, when it comes to image denoising, DAE can be extremely effective. By providing a noisy image as input, DAE creates a reconstructed version of the original image. The training process involves minimizing the discrepancy between the original and reconstructed images. Once the DAE has completed its training, it can be employed to denoise new images by removing unwanted noise and reconstructing the original image.

To illustrate this, let’s consider an example using the Keras digit MNIST dataset. This dataset consists of 60,000 28×28 grayscale images (0-9) of handwritten digits for training and an additional 10,000 images for testing. We can use a denoising autoencoder to denoise these images.

Importing Libraries and Dataset

To begin, make sure you have the required libraries installed and load the dataset.

# Installing libraries
!pip install tensorflow numpy matplotlib

# Loading  dataset
from tensorflow.keras.datasets import mnist
(x_train, _), (x_test, _) = mnist.load_data()

Preprocess Data

# Add noise to the images
noise_factor = 0.5
x_train_noisy = x_train + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=x_train.shape)
x_test_noisy = x_test + noise_factor * np.random.normal(loc=0.0, scale=1.0, size=x_test.shape)

# Clip the images to the valid pixel range
x_train_noisy = np.clip(x_train_noisy, 0., 1.)
x_test_noisy = np.clip(x_test_noisy, 0., 1.)

# Normalize the pixel values
x_train_noisy = x_train_noisy / 255.
x_test_noisy = x_test_noisy / 255.

Define Model

Afterwards, you can proceed to define the denoising autoencoder model utilizing the Keras functional API.

from tensorflow.keras.layers import Input, Conv2D, MaxPooling2D, UpSampling2D
from tensorflow.keras.models import Model

# Define the input layer
input_layer = Input(shape=(28, 28, 1))

# Encoder
x = Conv2D(32, (3, 3), activation='relu', padding='same')(input_layer)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)

# Decoder
x = Conv2D(32, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)

# Define the model
autoencoder = Model(input_layer, decoded)
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')

Train Model

Lastly, you can train the model and utilize it to denoise the images.

# Train the model
autoencoder.fit(x_train_noisy, x_train, epochs=10, batch_size=128,
validation_data=(x_test_noisy, x_test))

Denoise Images

# Using model to denoise the images
denoised_images = autoencoder.predict(x_test_noisy)

To visualize the original, noisy, and denoised images, you can utilize the Matplotlib library.

import matplotlib.pyplot as plt

# Display the first image in the test set
plt.imshow(x_test[0], cmap='gray')
plt.show()

# Display the first image in the noisy test set
plt.imshow(x_test_noisy[0], cmap='gray')
plt.show()

# Display the first image in the denoised test set
plt.imshow(denoised_images[4].squeeze(), cmap='gray')
plt.show()

Output:

In the provided output, the images are organized as follows:

• The first row represents the original test images.
• The second row displays the corresponding noisy images.
• The third row showcases the cleaned (reconstructed) images.

Notice how the reconstructed images closely resemble the original ones.

However, upon closer inspection, you may notice that the updated image looks a bit blurry. There are several possible reasons for this confusion in the decoder network output. One of the reasons for this is the use of less time during the study period. Therefore, your task is to increase the number of periods and re-evaluate the resulting image. Compare the increase in the number of results obtained with the previous period.

Conclusion

Denoising autoencoders (DAEs) offer several advantages over traditional noise reduction techniques. They effectively avoid the problem of creating oversimplified images, and they compute quickly. Unlike traditional filtering methods, DAEs use an improved autoencoder approach that involves inserting noise into the input and reconstructing the output from the corrupted image.

This modification to the standard autoencoder approach prevents the DAE from copying input to output. Instead, DAEs need to remove noise from the input before extracting meaningful information.

In our specific DAE approach, we have used CNN due to its effectiveness in deducing and preserving spatial relationships within an image. Additionally, employing CNNs helps reduce dimensions and computational complexity, making it possible to use arbitrary-sized images as input.

Key Takeaways

• Denoising autoencoders are a type of neural network designed to eliminate noise from images.
• They comprise two main components: an encoder and a decoder, working together to reconstruct a clean image from a noisy input.
• The training process involves adding noise to the images and minimizing a reconstruction loss function, aiming to minimize the discrepancy between the original and reconstructed images.
• Denoising autoencoders prove valuable for image preprocessing tasks, effectively removing noise. However, it’s important to note that they may require further enhancements to handle more complex noise patterns or different types of data.

Frequently Asked Questions

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

ANURAG SINGH CHOUDHARY

Passionate Machine learning professional and data-driven analyst with the ability to apply ML techniques and various algorithms to solve real-world business problems. I have always been fascinated by Mathematics and Numbers. Over the past few months, I have dedicated a considerable amount of time and effort to Machine Learning Studies.