This article was published as a part of the Data Science Blogathon.
Introduction
We often see that when people are giving the definition of Machine learning or deep learning that they use the words like “the process that mimics the human brain”. Now, what does that mean when they are saying “mimic the human brain”? If the system has enough computing power and enough data for processing then it can solve the most challenging problems.
For example anomalies in stock fluctuations and signs of cancer in mammograms, computers with well-trained programs can recognize these much better than humans. Now we have to understand that what is the complex process lies behind these algorithms and how these are mimicking human brains and giving better results.
What is Pattern Recognition?
Pattern recognition is a process of finding regularities and similarities in data using machine learning data. Now, these similarities can be found based on statistical analysis, historical data, or the already gained knowledge by the machine itself.
A pattern is a regularity in the world or in abstract notions. If we discuss sports, a description of a type would be a pattern. If a person keeps watching videos related to cricket, YouTube wouldn’t recommend them chess tutorials videos.
Examples: Speech recognition, speaker identification, multimedia document recognition (MDR), automatic medical diagnosis.
Before searching for a pattern there are some certain steps and the first one is to collect the data from the real world. The collected data needs to be filtered and pre-processed so that its system can extract the features from the data. Then based on the type of the data system will choose the appropriate algorithm among Classification, Regression, and Regression to recognize the pattern.
Classification. In classification, the algorithm assigns labels to data based on the predefined features. This is an example of supervised learning.
Clustering. An algorithm splits data into a number of clusters based on the similarity of features. This is an example of unsupervised learning.
Regression. Regression algorithms try to find a relationship between variables and predict unknown dependent variables based on known data. It is based on supervised learning. [2]
Features can be represented as continuous, discrete, or discrete binary variables. A feature is basically a function of one or more measurements, computed to quantify the significant characteristics of the object. The feature is one of the most important components in the Pattern Recognition system.
Example: consider a football, shape, size and color, etc. are features of the football.
A feature vector is a set of features that are taken together.
Example: In the above example of football, if all the features (shape, size, colour etc.) taken together then the sequence is feature vector ([shape, size, colour]). The feature vector is the sequence of features represented as an n-dimensional column vector. In the case of speech, MFCC (Mel-frequency Cepstral Coefficient) is the spectral features of the speech. The sequence of the first 13 features forms a feature vector. [1]
Features of Pattern Recognition
1. Speed and accuracy for the familiar is high
2. It can recognize an unfamiliar object
3. It has the ability to recognize different shapes and object from all angles.
4. It identifies the patterns and objects when partly hidden.
5. During analysis quickly catch the patterns with automaticity.
Training a Pattern Recognition system
Fig (1): Dataset
Training Set: The training set plays an important part to train the model. Program process this dataset by using training rules. To get the better result one need to collect quite a large dataset because the program will always give better results with a handful of training data. But it may not give the same results in the case of the test dataset. If someone is building a masked face recognizer then he/she will need a lot of images of people wearing a mask. From that dataset, the necessary information will be gathered by the program. Generally, 80% of the total dataset is used as the training dataset.
Validation Set: Fine-tuning helps to train the model. If for the training dataset the accuracy is increasing then a certain portion of data from the training dataset which is unknown to the model is selected to check that for that dataset also the accuracy is increasing. If accuracy is not increasing for the validation set then the program is overfitting the model. In that case, the developer needs to check the value of the parameters or he/she may have to reconsider the model.
Test Set: The test set is used to take the output from the model. After the training, it is used to check how accurate the model is. The rest of the 20% of the dataset is used as a test set.
How does it work?
Fig (2): Step by step process
A pattern recognition system will perceive some input from the real world with sensors. Such a system can work with any type of data: images, texts, videos, or numbers.
After receiving some information as the input, the algorithm starts to pre-process the data. That is segmenting something interesting from the background. For example, when you are given a photo of a park and a familiar face or any object that attracts the user’s attention, this is pre-processing.
While the data is in the pre-processing phase it is important to filter the noise from the main dataset. Depending on the working function of the application, the filter algorithm will change. For example, consider a face recognition system where the system is collecting the images for training purposes.
In order to process the data, it will first convert the images from RGB to greyscale. Also, the system doesn’t need other areas than the face. So to filter out unwanted portions of the images and replace them with white or black background some filter mechanisms are required. Once those filter mechanisms are used on the data it will be easier for the system to extract features from the filtered images.
Feature extraction is a process of uncovering some characteristic traits that are similar to more than one data sample. The derived information may be general features, which are evaluated to ease further processing. For example, in image recognition, the extracted features will contain information about grey shade, texture, shape, or context of the image. This is the main information used in image processing. The methods of feature extraction and the extracted features are application dependent.
After extracting the features from the processed data the result of a pattern recognition system will be either a class assignment (labeled dataset), or cluster assignment (dataset without labels), or predicted values (where regression is applied).
Neural Approach
There are quite a few approaches for pattern recognition like Statistical, Syntactical, and Neural. The statistical approach is nothing but to collect historical data and based on the observations and analyses from those data new patterns are recognized. The syntactical approach is also known as the structural approach as it mainly relies upon sub-patterns called primitives like words.
The pattern recognition approaches discussed so far are based on direct computation through machines. Direct computations are based on math and stats related techniques. Other than those techniques another one is the neural approach, neural networks related topics are discussed here to recognize the patterns. As it is known to all neuron is the basic unit of brain cells and together these neurons create networks to control the specific tasks. This neural network is implemented in systems. The outcome of this effort is the invention of artificial neural networks. [3]
An artificial neural network is a computing system that tries to stimulate the working function of a biological neural network of human brains. In this network, all the neurons are well connected and that helps to achieve massive parallel distributing. The input units receive various forms and structures of information based on an internal weighting system and the neural network attempts to learn about the information presented to produce one output report [4].
The advantages of neural networks are their adaptive-learning, self-organization, and fault-tolerance capabilities. For these outstanding capabilities, neural networks are used for pattern recognition applications. An ANN initially goes through a training phase where it learns to recognize patterns in data, whether visually, aurally, or textually [4]. Some of the best neural models are back-propagation, high-order nets, time-delay neural networks, and recurrent nets.
Fig (3): Basic structure of a feed-forward neural network
Normally, only feed-forward networks are used for pattern recognition. Feed-forward means that there is no feedback to the input. Similar to the way that human beings learn from mistakes, neural networks also could learn from their mistakes by giving feedback to the input patterns. This kind of feedback would be used to reconstruct the input patterns and make them free from error; thus increasing the performance of the neural networks. Of course, it is very complex to construct such types of neural networks. These kinds of networks are called auto-associative neural networks [3]. This complexity of constructing the network can be avoided by using back-propagation algorithms. During this supervised phase, the network compares its actual output produced with what it was meant to produce—the desired output.
The difference between both outcomes is adjusted using backpropagation. This means that the network works backward, going from the output unit to the input units to adjust the weight of its connections between the units until the difference between the actual and desired outcome produces the lowest possible error [4]. Local minima is one of the main problems associated with back-propagation algorithms. In addition, neural networks have issues associated with hyper-parameters like learning rate, architecture selection, feature representation, modularity, and scaling. Though there are problems and obstacles, the application of neural networks has spread everywhere.
Real-life example
The pattern is the most basic thing for anyone to learn anything. When a baby starts learning he/she tries to search for patterns to identify different objects. Many people use face recognition in photos when posting to social media. Pattern recognition is used to build this face recognition system similar to fingerprint identification.
Basically, a machine tries to capture features of the object and store those features into a vector. These elements in vectors are the attributes of the pattern. Example: While representing different types of balls, (circumference, weight, shape, and class) will be Vector and each feature is an element. If the first ball belongs to class 1, the vector would be (22.4cm, 163gm, round, 1), where the value of the last element represents the “cricket ball”. Each type of ball falls under a separate class and is denoted by a number.
Applications
-
Image processing, segmentation, and analysis
Pattern Recognition is efficient enough to give machines human recognition intelligence. This is used for image processing, segmentation, and analysis. For example, computers can detect different types of insects better than humans.
-
Computer Vision
Using a pattern recognition system one can extract important features from the images and videos. This is helpful in computer vision which is applied in different fields’, especially biomedical imaging.
-
Seismic Analysis
Decision-theoretic and syntactic pattern recognition techniques are employed to detect the physical anomalies (bright spots) and to recognize the structural seismic patterns in two-dimensional seismograms. Here, decision-theoretic methods include Bayes classification, linear and quadratic classifications, tree classification, partitioning-method, and tree classification, and sequential classification [5].
-
Radar Signal Classification
Pattern recognition and signal processing methods are used in a large dataset to find similar characteristics like amplitude, frequencies, type of modulation, scanning type, pulse repetition intervals, etc. Basically, it helps to classify the radio signals, and based upon their class the conversion to digital form is accomplished.
-
Speech Recognition
All of us have heard the names Siri, Alexa, and Cortona. These are all the applications of speech recognition. Pattern recognition plays a huge part in this technique.
-
Fingerprint Identification
Many recognition approaches are there to perform Fingerprint Identification. But pattern recognition system is the most used approach.
-
Medical Diagnosis
Algorithms of pattern recognition deal with real data. It has been found that pattern recognition has a huge role in today’s medical diagnosis. From breast cancer detection to covid-19 checking algorithms are giving results with more than 90% accuracy.
-
Stock Market Analysis
Patterns are everywhere and nobody can ignore that. Though the stock market is hard to predict still some AI-based applications are there which are doing using a pattern recognition approach to predict the market. Example: Blumberg, Tinkoff, SofiWealth, and Kosho.
References
1. https://www.geeksforgeeks.org/pattern-recognition-introduction/
2. https://serokell.io/blog/pattern-recognition
3. https://ubiquity.acm.org/article.cfm?id=985625
5. Pattern Recognition to Seismic Exploration by Kou-Yuan Huang (https://link.springer.com/chapter/10.1007/978-1-4612-4388-5_7#:~:text=Decision%2Dtheoretic%20and%20syntactic%20pattern,patterns%20in%20two%2Ddimensional%20seismograms.)
