Introduction
Can you guess what’s common among all the faces in this image?
None of these people are real! These faces were generated by a computer vision technique called GANs, or Generative Adversarial Networks. Full marks to you if you guessed it correctly!
The term ‘GAN’ was introduced by the Ian Goodfellow in 2014 but the concept has been around since as far back as 1990 (pioneered by Jürgen Schmidhuber). But it was only after Goodfellow’s paper on the subject that they gained popularity in the community. And since then, there’s been no looking back for GANs!
In fact, GANs are now ubiquitous. Data scientists and deep learning researchers use this technique to generate photorealistic images, change facial expressions, create computer game scenes, visualize designs, and more recently, even generate awe-inspiring artwork! That’s right – the recent news about AI generated art? It was GANs at work:
In this article, we will look at five intriguing applications of GANs that are prevalent in the industry. You might even have come across a few of them without realizing how they worked. I have also provided links to research papers for each GAN application which I encourage you to check out.
To learn more about generative models and how GANs works from scratch, feel free to check out this detailed article:
What are Generative Models and GANs? The Magic of Computer Vision
Strap in – this is going to be fun!
Table of Contents
- Introductory Explanation of GANs
- Applications of GANs
- GANs for Image Editing
- Using GANs for Security
- Generating Data using GANs
- GANs for Attention Prediction
- GANs for 3D Object Generation
Introductory Explanation of GANs
Right, we have a sense of what GANs can do. But how do they work? What goes on underneath all the wonderful applications this powerful algorithm produces? Let’s understand this using a popular example.
There’s a forger (who creates fake artistry) and an investigator tasked with detecting these fake artworks.
The task of this forger is to create fraudulent imitations of original paintings by famous artists (like Leonardo Da Vinci). If he/she can pass off this work as the original art piece, the forger can potentially net a lot of money.
On the other side of this situation, the art investigator’s task is to catch these forgers. How does he/she do it? The investigator knows what are the properties which set the original artist apart and what kind of painting he/she would have created. The investigator leverages this knowledge against the piece at hand to check if it is real or not.
This contest of forger vs investigator goes on, which ultimately makes world-class investigators (and unfortunately world-class forgers); a battle between good and evil.
Now, consider both forger and investigator as robots. When you train the forger to be a painter and the investigator to tell a fake painting from the real one – you now have an algorithmic painter at hand! That’s essentially how GANs work on the inside. Awesome, aren’t they?
I haven’t got into the intricate details of GANs here. This is just the tip of the iceberg. If you are interested in learning more about GANs, you should go through this article:
Applications of GANs
Now that we have an intuition of how GANs work, let’s put on our exploration hats! It’s time to dive into the interesting applications of GANs that are commonly used in the industry right now.
GANs for Image Editing
Most image editing software these days don’t give us much flexibility to make creative changes in pictures. For example, let’s say you want to change the appearance of a 90-year-old person by changing his/her hairstyle. This can’t be done by the current image editing tools out there. But guess what? Using GANs, we can reconstruct images and attempt to change the appearance drastically.
This amazing paper demonstrates this very cutting edge application.
Another similar application is image de-raining (or literally removing rainy texture from images). Want an example? Check out the below image taken from this paper:
Using GANs for Security
The rise of artificial intelligence has been wonderful for most industries. But there’s a real concern that has shadowed the entire AI revolution – cyber threats. Even deep neural networks are susceptible to being hacked.
A constant concern of industrial applications is that they should be robust to cyber attacks. There’s a lot of confidential information on the line! GANs are proving to be of immense help here, directly addressing the concern of “adversarial attacks”.
These adversarial attacks use a variety of techniques to fool deep learning architectures. GANs are used to make existing deep learning models more robust to these techniques. How? By creating more such fake examples and training the model to identify them. Pretty clever stuff.
A technique called SSGAN is used to do steganalysis of images and detect harmful encodings which shouldn’t have been there.
Generating Data with GANs
Who among us wouldn’t love to collect more data for building our deep learning model? The availability of data in certain domains is a necessity, especially in domains where training data is needed to model supervideepeeop learning algorithms. The healthcare industry comes to mind here.
GANs shine again as they can be used to generate synthetic data for supervision. That’s right! You know where to go next time you need more data. 🙂
For instance, this paper explores the creation of synthetic data with the help of GANs for training deep learning algorithms by creating realistic eye images.
GANs for Attention Prediction
When we see an image, we tend to focus on a particular part (rather than the entire image as a whole). This is called attention and is an important human trait. Knowing where a person would look beforehand would certainly be a useful feature for businesses, as they can optimize and position their products better.
For example, game designers can focus on a particular portion of the game to enhance the features and make it more engrossing.
This enthralling idea is explored in this paper, where the authors try to identify the most appealing parts of given images using GANs.
GANs for 3D Object Generation
It won’t surprise you to know GANs are quite popular in the gaming industry.
Game designers work countless hours recreating 3D avatars and backgrounds to give them a realistic feel. And let me assure you, it certainly takes a lot of effort to create 3D models by imagination. Does this seem unrealistic? Then I suggest you watch this video. You might believe the incredible power of GANs, wherein they can be used to automate the entire process!
Here is an open source implementation of the same. Go ahead and try it out if you find the idea interesting!
End Notes
There are a plethora of applications of GANs regularly published in research. I’m sure many more will follow soon. I hope the above applications encouraged you to think of more such ideas – perhaps you could come up with your own GAN! If you have any ideas or suggestions, do comment below.
Finally, let me share with you a few additional resources in case you want to try things on your own, or just understand how GANs work in general. You can start out with our beginner’s guide to grasp the basics or refer to the below resources:
- List of Papers published on GANs
- A Brief Chapter on Deep Generative Modelling
- Workshop on Generative Adversarial Network by Ian Goodfellow
- NIPS 2016 Workshop on Adversarial Training
Faizan is a Data Science enthusiast and a Deep learning rookie. A recent Comp. Sc. undergrad, he aims to utilize his skills to push the boundaries of AI research.
