Introduction to Artificial Neural Network : Simplified

Here is yet another algorithm used by the industry to scare ignorant freshers. The tag line for this algorithm is “It works in a way, similar to human brain.” The first time I started reading about the algorithm, I realized that even though brain functionality might be very complex, artificial neural network algorithm is not as complex as its impression in the industry.

Have you ever wondered, if human brain were as simple as a CPU of a desktop, what would have changed? Probably, every human being would have ended up being identical. For each situation, every one of us would have taken the exact same decisions. What could be the differentiating factor responsible for creating such diversity among human race? Answer to this question is simply beyond the scope of discussion here.

Here is what I think differentiates human brain from a simple machine. A simple machine is a set of algorithm, which converts input(s) to output(s). In this scenario, the same input will always lead to the same output. Human brain, on the other hand, has a unique characteristic of creating transient states through neurons in between the sensory organs and the brain (decision taking unit). Hence, the probabilistic interim state brings out a factor of randomness, which brings out what we call “Creativity”. In ANN (Artificial neural network) or rather all machine learning algorithm, we build some kind of transient states, which allows the machine to learn in a more sophisticated manner. The objective of this article is to bring out the framework of ANN algorithm in parallel to the functionality of human brain.

How does a simple predictive algorithm work?

A simple predictive algorithm tries to mimic the relationship between the input and the output variables. The function derived in such routines is a direct linear or non-linear function between input and output variables. For instance, if we try to predict the total work experience of a person using his age, following is the kind of relationship we will observe:

Such relationships can easily be predicted using simple regression algorithms. But it becomes difficult to make predictions in case of complex non-linear relationships and significant covariate terms. In such cases, we need more sophisticated machine learning tools. To make such predictions, we have two options – either predict a complex non linear function or break this problem into multiple steps and solve for each step. The later can be achieved easily using an artificial neural network (ANN).

How does ANN work?

It is truly said that the working of ANN takes its roots from the neural network residing in human brain. ANN operates on something referred to as Hidden State. These hidden states are similar to neurons. Each of these hidden state is a transient form which has a probabilistic behavior. A grid of such hidden state act as a bridge between the input and the output.

Let’s try to understand what the above diagram actually means. We have a vector of three inputs and we intend to find the probability that the output event will fall into class 1 or class 2. For this prediction we need to predict a series of hidden classes in between (the bridge). The vector of the three inputs in some combination predicts the probability of activation of hidden nodes from 1 – 4. The probabilistic combination of hidden state 1-4 are then used to predict the activation rate of hidden nodes 5-8. These hidden nodes 5-8 in turn are used to predict hidden nodes 9-12, which finally predicts the outcome. The intermediate latent states allows the algorithm to learn from every prediction.

Interesting? Let’s think at a different scale

Now that you understand how neural networks work, you might be questioning its application and usage. Neural networks are one of the hottest machine learning and researching area today. All data giants (Google, Facebook, Baidu etc. have invested heavily in mastering and researching these algorithms. For example, imagine solving Voice recoginition problem – all the past dialogues spoken by you become input for a neural network and the words from a dictionary become the output. By training on huge number of inputs from several people, can we create a neural net which is able to predict the right words as output? The answer is the subject matter of some of the best research labs across the globe. This subject is now popularly being called as deep learning. You can read about applications and some recent development in deep learning here.

End Notes

This article gave you a sneak peak into the scary black box known as Artificial neural network. The name itself scares people away. But after reading this article, I hope I might have ignited some level of interest in your mind to know more about this algorithm. The content of this article is just a tip of the iceberg. In next few articles on ANN, we will get into more details of this algorithm. In these articles, we will read about how the algorithm learns about the relationship between the linkages (between input & Hidden state, among Hidden state, Hidden state & output). We will also learn about the backward learning (or feedback) mechanism which helps us to optimize the linkage grid of the hidden states. Even though the last two lines might look scary, but  the actual content will be as simple as this article. And to gain expertise in working in neural network try out our deep learning practice problem – Identify the Digits.

Did you find the article useful? Have you used any other machine learning tool recently? Do you plan to use ANN in any of your business problems? If yes, share with us how you plan to go about it.

If you like what you just read & want to continue your analytics learning, subscribe to our emails, follow us on twitter or like our facebook page.

Tavish Srivastava

Tavish Srivastava, co-founder and Chief Strategy Officer of Analytics Vidhya, is an IIT Madras graduate and a passionate data-science professional with 8+ years of diverse experience in markets including the US, India and Singapore, domains including Digital Acquisitions, Customer Servicing and Customer Management, and industry including Retail Banking, Credit Cards and Insurance. He is fascinated by the idea of artificial intelligence inspired by human intelligence and enjoys every discussion, theory or even movie related to this idea.