Optical Character Recognition(OCR): Tesseract, OpenCV, and Python

Introduction

We, humans, read text almost every minute of our life. Wouldn’t it be great if our machines or systems could also read the text just like the way we do? But the bigger question is “How do we make our machines read”? This is where Optical Character Recognition (OCR) comes into the picture. OCR engines leverage algorithms for text localization, segmentation, and sometimes deep learning to accurately interpret text.

Learning Objectives

• In this tutorial you will learn about OCR
• You will learn about the applications of OCR
• You will understand and build OCR applications even if you are a beginner
• Build a word cloud of words from a corpus
• Create a text to audio application

Optical Character Recognition (OCR)

Optical Character Recognition (OCR) is a technique to extract text from printed or scanned photos, handwritten text images and convert them into a digital format that can be editable and searchable.

Applications of OCR

OCR has plenty of applications in today’s business. A few of them are listed below:

• Passport recognition in Airports
• Automation of Data Entry
• Number plates recognition
• Text detection from business cards
• Converting handwritten documents into electronic images
• Creating Searchable PDFs
• Create audible files (text to audio)
• Real-time image recognition

Some of the Open Source OCR tools are Tesseract, OCRopus.

In this article, we will focus on Tesseract OCR. And to read the images we need OpenCV.

Installation of Tesseract OCR

Download the latest installer for windows 10 from this website. Execute the .exe file once it is downloaded.

Note: Don’t forget to copy the file software installation path. We will require it later as we need to add the path of the tesseract executable in the code if the directory of installation is different from the default.

The typical installation path in Windows systems is C:Program Files.

So, in my case, it is “C: Program FilesTesseract-OCRtesseract.exe“.

Next, to install the Python wrapper for Tesseract, open the command prompt and execute the command

pip install pytesseract

For linux, run the following command in command line:

sudo apt-get install tesseract-ocr

OpenCV(Open Source Computer Vision) is an open-source library for computer vision, machine learning, and image processing applications.

OpenCV-Python is the Python API for OpenCV.

To install it, open the command prompt and execute the command in the command line:

pip install opencv-python

Build sample OCR Script

Reading a sample Image

import cv2

Read the input image using cv2.imread() method and store it in a variable “img”.

img = cv2.imread("image.jpg")

If needed, resize the image using cv2.resize() method

img = cv2.resize(img, (400, 400))

Display the image using cv2.imshow() method

cv2.imshow("Image", img)

Display the window infinitely (to prevent the kernel from crashing)

cv2.waitKey(0)

Close all open windows.

cv2.destroyAllWindows()

Converting Image to String

import pytesseract

Set the tesseract path in the code

pytesseract.pytesseract.tesseract_cmd=r'C:Program FilesTesseract-OCRtesseract.exe'

The below error occurs if we do not set the path.

To convert an image to string use pytesseract.image_to_string(img) and store it in a variable “text”.

text = pytesseract.image_to_string(img)

Print the result.

print(text)

Complete code: 

import cv2
import pytesseract
pytesseract.pytesseract.tesseract_cmd=r'C:Program FilesTesseract-OCRtesseract.exe'
img = cv2.imread("image.jpg")
img = cv2.resize(img, (400, 450))
cv2.imshow("Image", img)
text = pytesseract.image_to_string(img)
print(text)
cv2.waitKey(0)
cv2.destroyAllWindows()

The output for the above code:

If we observe the output, the main quote is extracted perfectly, but the philosopher’s name and the text at the very bottom of the image are not obtained.

To extract the text accurately and to avoid accuracy drop, we need to do some preprocessing of the image. I found this article quite helpful. Refer to it for a better understanding of preprocessing techniques.

Perfect! Now that we have got the basics required, Let us see some simple applications of OCR.

Applications of OCR

Building word clouds on Review images

Word cloud is a visual representation of word frequency. The bigger the word appears in a word cloud, the more commonly the word is used in the text.

For this, I took some snapshots of reviews from Amazon for the product Apple iPad 8th Generation and created a dataset. The goal is to recognize text from it and create a wordcloud.

Sample image

Steps: 

1. Create a list of all the available review images
2. If needed view the images using cv2.imshow() method
3. Read text from images using pytesseract
4. Create a data frame
5. Preprocess the text – remove special characters, stop words
6. Build positive, negative word clouds

Step 1: Create a list of all the available review images

import os
folderPath = "Reviews"
myRevList = os.listdir(folderPath)

Step 2: If needed view the img. using cv2.imshow() method

for image in  myRevList:
    img = cv2.imread(f'{folderPath}/{image}')
    cv2.imshow("Image", img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

Step 3: Read text from images using pytesseract

import cv2
import pytesseract
pytesseract.pytesseract.tesseract_cmd=r'C:Program FilesTesseract-OCRtesseract.exe'
corpus = []
for images in myRevList:
    img = cv2.imread(f'{folderPath}/{images}')
    if img is None:
        corpus.append("Could not read the image.")
    else:
        rev = pytesseract.image_to_string(img)
        corpus.append(rev)
list(corpus)
corpus

The output of the above code

Step 4: Create a data frame

Provide the corpus of words and column name as parameters.

import pandas as pd
data = pd.DataFrame(list(corpus), columns=['Review'])
data

Step 5: Preprocess the text – remove special characters, stopwords

#removing special characters
import re
def clean(text):
    return re.sub('[^A-Za-z0-9" "]+', ' ', text)
data['Cleaned Review'] = data['Review'].apply(clean)
data

Removing stopwords from the ‘Cleaned Review’ and appending all the remaining words to a list variable “final_list”. Here is the python code for that:

# removing stopwords 
import nltk 
from nltk.corpus 
import stopwords 
nltk.download("punkt") 
from nltk import word_tokenize 

stop_words = stopwords.words('english') 
final_list = [] 

for column in data[['Cleaned Review']]: 
    columnSeriesObj = data[column] 
    all_rev = columnSeriesObj.values 

    for i in range(len(all_rev)): 
        tokens = word_tokenize(all_rev[i]) 
        for word in tokens: 
              if word.lower() not in stop_words: 
                   final_list.append(word)

Step 6: Build positive, negative word clouds

Install word cloud library using the command “pip install wordcloud“.

In the English language, we have a predefined set of positive, negative words called Opinion Lexicons. These files can be downloaded from the link or directly from my GitHub repo.

Once the files are downloaded, read those files in the code and create a list of positive, negative words.

with open(r"opinion-lexicon-Englishpositive-words.txt","r") as pos:
  poswords = pos.read().split("n")
with open(r"opinion-lexicon-Englishnegative-words.txt","r") as neg:
  negwords = neg.read().split("n")

Importing libraries to generate and show word clouds.

import matplotlib.pyplot as plt
from wordcloud import WordCloud

Positive Word Cloud

# Choosing the only words which are present in poswords
pos_in_pos = " ".join([w for w in final_list if w in poswords])
wordcloud_pos = WordCloud(
                      background_color='black',
                      width=1800,
                      height=1400
                     ).generate(pos_in_pos)
plt.imshow(wordcloud_pos)

The word “good” being the most used word catches our attention. If we look back at the reviews, people have written reviews saying the iPad has a good display, good sound, good software, and hardware.

Negative Word Cloud

# Choosing the only words which are present in negwords
neg_in_neg = " ".join([w for w in final_list if w in negwords])
wordcloud_neg = WordCloud(
                      background_color='black',
                      width=1800,
                      height=1400
                     ).generate(neg_in_neg)
plt.imshow(wordcloud_neg) 

The words expensive, stuck, struck, disappoint stood out in the negative word cloud. If we look at the context of the word stuck, it says “Though it has just 3 GB RAM, it never gets stuck” which is a positive thing about the device.

So, it’s good to build bigram/trigram word clouds to not miss out on the context by using text recognition.

Create audible files (Text to Audio)

gTTS is a Python Library with Google Translate’s text-to-speech API.

To install, execute the command “pip install gtts” in the command prompt.

Import necessary libraries

import cv2
import pytesseract
from gtts import gTTS
import os

Set the tesseract path

pytesseract.pytesseract.tesseract_cmd=r'C:Program FilesTesseract-OCRtesseract.exe'

Read the image using cv2.imread() and grab the text from the image using pytesseract and store it in a variable.

rev = cv2.imread("Reviews\15.PNG")
# display the image using cv2.imshow() method
# cv2.imshow("Image", rev)
# cv2.waitKey(0)
# cv2.destroyAllWindows()
# grab the text from image using pytesseract
txt = pytesseract.image_to_string(rev)
print(txt)

Set language and create a convert the text to audio using gTTS bypassing the text, language

language = 'en'

outObj = gTTS(text=txt, lang=language, slow=False)

Save the audio file as “rev.mp3”

outObj.save("rev.mp3")

play the audio file

os.system('rev.mp3')

Complete Code

import cv2
import pytesseract
from gtts import gTTS
import os
rev = cv2.imread("Reviews\15.PNG")

# cv2.imshow("Image", rev)
# cv2.waitKey(0)
# cv2.destroyAllWindows()

txt = pytesseract.image_to_string(rev)
print(txt)
language = 'en'
outObj = gTTS(text=txt, lang=language, slow=False)
outObj.save("rev.mp3")
print('playing the audio file')
os.system('rev.mp3')

Conclusion

By the end of this article, we have gained a solid understanding of Optical Character Recognition (OCR) and learned how to use libraries like OpenCV and Tesseract to extract text from images. We explored practical applications like building word clouds and creating spoken audio from text. For more advanced tasks like object detection and accurate character recognition in diverse image scenarios, powerful techniques like neural networks, bounding boxes, and object detection templates can be employed, regardless of the specific framework used (e.g., TensorFlow, PyTorch).

The media shown in this article are not owned by Analytics Vidhya and are used at the Author’s discretion.

Frequently Asked Questions

Harika

Hi, my name is Harika. I am a Data Engineer and I thrive on creating innovative solutions and improving user experiences. My passion lies in leveraging data to drive innovation and create meaningful impact.