What are Neural Networks and How Do They Work? A Deep Learning Tutorial

Introduction

In recent years, there has been a tremendous increase in the evolution of technology, and nowadays, deep learning is widely used in many domains. This has achieved great success in many fields, like computer vision tasks and natural language processing. Though we have traditional machine learning algorithms, deep learning plays an important role in many tasks better than machine learning techniques.

Machine learning, which is a subset of Artificial intelligence, includes deep learning. Deep learning operates based on artificial neural networks, processing data through multiple layers and extracting high-level features.In machine learning, this feature extraction happens manually, but in deep learning, feature extraction happens automatically because of the neural networks.

Learning Objectives

1. To understand the importance of deep learning and how it is different from machine learning
2. Understanding neural networks and their architecture
3. How to build our own neural networks
4. How the artificial neural networks work and the importance of activation function in it.

This article was published as a part of the Data Science Blogathon.

Why do We Need Deep Learning?

Before the evolution of deep learning, we had traditional machine learning algorithms which performed quite very well. But still, there is something that machine learning cannot perform well. Machine learning cannot process large unstructured data. It runs well with a large amount of structured data. But it cannot perform complex algorithms, and for the same problem statement, deep learning achieves better performance than traditional machine learning algorithms.

Machine learning algorithms cannot perform complex operations, but deep learning algorithms can handle complex algorithms. Finally, feature extraction takes place automatically with deep learning. Traditional machine learning techniques require domain expertise to identify the applicable features in order to simplify the data and make patterns more apparent to the learning algorithms. The main benefit of deep learning algorithms is that they attempt to learn high-level features from data incrementally. This does away with the necessity for hard-core feature extraction and domain expertise.

What is a Neural Network?

The human brain mainly inspires a neural network. It contains artificial neurons similar to biological neurons in the human brain. The human brain mainly inspires it. It teaches computers to process data. In this layered formation, interconnected nodes, called neurons, transmit data. The ANN primarily consists of three layers. These include the input layer, hidden layers in between, and the output layer.

Types of neural networks.

Shallow neural networks

Shallow neural networks have only one hidden layer positioned between the input and output layers, while deep neural networks contain at least two hidden layers situated between the input and output layers.

Source: DiSHA

The simple three-layer multi-layer perception architectures will make up the shallow neural network (MLP). The shallow architecture consists of an input layer, one hidden layer, and an output layer. The specially designed input layer accepts network packet data, which demonstrates numerical information reflecting network activity. The output layer will divide into corresponding classes.

Deep Neural networks.

DNN , which are more complicated and have more layers, support a significantly greater level of variety. Deep neural networks’ multi-layer structure enables more creative topologies, such as connecting layers from supervised and unsupervised learning methods into one network and varying the number of hidden layers.

How Does an Artificial Neural Network Work?

In an Artificial Neural Network, the inputs initially pass through the input layer, with random weights assigned to each input. The weights are multiplied by their respective input values, and the products are summed to generate a weighted sum. Following the addition of a bias to this sum, the result undergoes an activation function. Subsequently, all activated neurons progress to the next layer, iterating this process until reaching the output layer, where the final output is predicted.

What is an Activation Function?

In artificial neural networks, the activation function determines how the weighted sum of the input layer transitions from the input layer to the output layer through the hidden layers in between. When the input provided is sufficiently large, the corresponding neuron becomes activated and is subsequently passed to the next layer of the network. The goal of this activation function is to introduce non-linearity in the network.

Types of Activation Function

1. Linear Activation Function
2. Binary step function
3. Non-linear Activation Function

Some examples of activation functions are Sigmoid, ReLU(Rectified Linear Unit), ELU(Exponential Linear Units), Softmax, and Tanh.

Create Your Own Neural Network

Now let’s see how to build a simple neural network. First, import the necessary libraries, and to begin with, we have to initialize the bias, learning rate, and weights.

Then define the perceptron function defining how to update weights if an error occurs. Finally, if you test the network the output will be 0 or 1, determining whether to fire the neuron or not.

Python Code:

import numpy
import random
import os
#initialize learning rate, bias, and weights
learning_rate = 1 
bias = 1
# Random weights
weights = [random.random(),random.random(),random.random()]

def Perceptron(input1, input2, output) :
   out = input1*weights[0]+input2*weights[1]+bias*weights[2]
   if out > 0: #activation function (here Heaviside)
      out = 1
   else :
      out = 0
   error = output-out
   #update weights
   weights[0] += error * input1 * learning_rate
   weights[1] += error * input2 * learning_rate
   weights[2] += error * bias * learning_rate
for i in range(50) :
   Perceptron(1,1,1) # For True or true
   Perceptron(1,0,1) #For True or false
   Perceptron(0,1,1) #For False or true
   Perceptron(0,0,0) #For False or false
   
x = int(input("enter a value for x: "))
y = int(input("enter a value for y: "))
output_perceptron = x*weights[0] + y*weights[1] + bias*weights[2]
if output_perceptron > 0 :
   output_perceptron = 1
else :
   output_perceptron = 0
print( "Output is : ", output_perceptron)

What are Deep Learning Algorithms

Deep learning algorithms process data by passing it through all the layers of the network, transferring a simplified representation of the data from the last layer to the next layer.As it goes through NN layers, deep learning algorithms learn gradually more about the data. Feature extraction and feature aggregation take place here. Low-level feature detection learning takes place at the early layers of the network.

Deep learning algorithms process data across many NN “layers,” each of which sends a condensed version of the data to the following layer. Most machine learning methods perform effectively on datasets with up to some hundred features or columns. Deep learning algorithms are critical for identifying features and can handle a large number of operations for both structured and unstructured data.

Convolutional Neural Networks(CNN): 

A convolutional neural network (CNN or ConvNet) is a DNN architecture that learns directly from data. CNNs excel at detecting patterns in images, enabling the identification of objects and categories with high precision.

Recurrent Neural Networks(RNN):

A recurrent neural network are a type of Artificial Neural Network (ANN) specifically designed to handle time series data or data containing sequences. While feedforward neural networks are suitable for processing independent data points, specialized networks such as RNNs are employed for dependent data.

Long Short-Term Memory Networks(LSTMs):

It is a class of recurrent neural networks (RNNs) that can learn long-term dependencies mainly in sequence prediction problems. In traditional RNNs, the repeating module will have a simple structure.

Multilayer Perceptron(MLPs):

In simple words, MLP is a fully connected layer. An MLP is a common type of feedforward ANN. It is made up of three layers: the input layer, the output layer, and the hidden layer.

Applications of Neural Network

Health Care

Deep learning plays a significant role in the analysis of biomedical data, garnering attention from organizations of diverse sizes and specialties. These organizations are keen to harness artificial intelligence to elevate patient care, cut costs, and enhance operational efficiencies. Moreover, it finds application in disease recognition through pattern analysis.

Convolutional neural networks

CNNs, a subset of deep learning, excel at analyzing image data such as MRI results or x-rays, while NLP tools are utilized for tasks such as dictating documentation and transcribing speech to text. Meanwhile, deep learning developers are specifically concentrating on precision medicine and drug discovery.

Autonomous Vehicles

Deep learning algorithms are integral in autonomous vehicles for making critical decisions regarding navigation, obstacle detection, and other driving tasks. Also used in computer vision tasks.

In autonomous vehicles, for detection and classification, DL is used. This primarily includes using camera-based systems to detect and classify objects. The data collected by the vehicle sensors is collected and interpreted by the system. Also, this monitors the driver. Because neural networks can identify patterns, they can be used to monitor the driver in vehicles. Vehicle powertrains generate a time series of data points. This data can be used to improve motor control and battery management using machine learning.

Computer Vision

Computer vision tasks include object detection, image classification, face recognition, image captioning, and so on. Deep learning algorithms play an important role in these tasks.

Image classification with localization entails assigning a class label to an image and using a bounding box to show the location of an object in the image. Style transfer of images is an application of this. Image colorization, image reconstruction, image super-resolution, and image synthesis, like changing a horse to zebra, are all applications of computer vision that use DL.

Source: Tensorflow

Virtual Assistants

Virtual assistants like Alexa, Siri, Google Assistant, and Microsoft Cortana are developed using deep learning algorithms.

AI assistants greatly simplify daily tasks such as adding events to the calendar, setting reminders, or tracking expenses. Chatbots, AI avatars, voice assistants,  and domain-specific virtual assistants are all types of AI virtual assistants. Voice assistants, like well-known Siri and Google Assistant products, use automatic speech recognition and natural language processing to provide vocal responses to queries. AI avatars are 3D models resembling humans, utilized for entertainment purposes or to infuse a human touch into virtual customer care interactions.

Conclusion

Deep Learning is a new set of tools, algorithms, and incredibly novel approaches to problem-solving. It is about machines becoming conscious and taking over the duties assigned only to humans due to their superior intellectual abilities than machines. Their structure is most fundamentally similar to that of the human brain.

– With feature learning, deep learning algorithms explore and evaluate the unknown structure in the input distribution and find meaningful representations.

– Deep Learning offers a wide range of uses, which has increased its appeal and led to its use in a number of different industries. Many organizations from many fields or industries use it.

Connect with me on LinkedIn.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Amrutha

This is Amrutha, I am pursuing B.Tech in the Computer science Department. I am interested in developing ML Models with python and Data Analysis. And also I have an interest in Web Development. I hope my articles in Analytics Vidhya help you to learn better. Thank you!!