Computer Vision to Detect License Number Plate

This article was published as a part of the Data Science Blogathon

Introduction

In this fastest era of technology, it is very difficult to stop every vehicle on the road and check its number plate in the search for one criminal car. With the increase in road fraud, the cops are also becoming smarter. They are using deep learning and computer vision to detect the number plate in the car and extract the license number from that. Today, we’re going to build one such project to detect number plates using computer vision which helps in e-challan, and security monitoring.

In this blog, we will learn how to detect a number plate of a car and extract its values using computer vision. We are going to use the OpenCV library of Computer vision to detect the number plate of cars and the pytesseract library of deep learning to read image types and fetch characters and digits from the number plates. Finally, we build a graphical user interface using Tkinter to display our project’s work.

Prerequisites for Computer Vision

In order to understand this blog, you should be familiar with Computer Vision techniques and the OpenCV library. If you want to learn computer vision deeply click here.

Read our blog here on Computer Vision. 

To begin with, Install the libraries:

pip3 install OpenCV-python
pip3 install pytesseract

What is OpenCV?

OpenCV is a huge open-source cross-platform library that enables computer vision to perform real-world applications like autonomous driving, image annotation, drone-based crop monitoring, etc. It majorly focuses on capturing images and video to analyze the important features like object detection, face detection, emotion detection, etc. It also plays a significant role in Image processing-based AI applications. Refer to the below guide to deep dive into OpenCV.

OpenCV:https://www.analyticsvidhya.com/blog/2019/03/opencv-functions-computer-vision-python/

Here, we are just using a few basic features/functions of openCV to identify our number plate in the inputted image of a car.

• Contours: Contours are generally treated as boundary pixels as they are just simple curves that combine all the continuous points in the boundary with the same intensity and color. The use of contours is more clear in shape analysis, object detection and recognition, motion detection, and, background/foreground image segmentation. To reduce the task of contour detection, OpenCV provides in-built cv2.findContours() functions for this.

cv2.findContours(morph_img_threshold,mode=cv2.RETR_EXTERNAL,method=cv2.CHAIN_APPROX_NONE)

Our cv.find contours() function takes three parameters including input image, the mode of contour retrieval, and last is the contour approximation method. The function results to a modified image, the hierarchy, and contours in the form of a Python list.

Official link:- https://docs.opencv.org/3.4/d4/d73/tutorial_py_contours_begin.html

• Morphological Transformation: It refers to some simple operations that are performed only on binary images and depend on the shape of the image. Some common morphological operations are Opening, Closing, Erosion, Dilation. Every function takes two parameters including the input image and structuring element or kernel to decide the nature of the operation. OpenCV provides some in-built functions to perform these operations:

  • cv2.erode()

  • cv2.dilate()

  • cv2.morphologyEx()

Official link:- https://docs.opencv.org/3.4/d9/d61/tutorial_py_morphological_ops.html

• Gaussian Blur: Gaussian function is used to blur and smoothen the input image and output the Gaussian blur images. It is widely used in the reduction of image noise effects. OpenCV provides an in-built function cv2.GaussianBlur() for this.

Official link:- https://docs.opencv.org/4.x/d4/d13/tutorial_py_filtering.html

• Sobel: This function is used for the calculation of image derivatives which in turn help in the gradients. OpenCV provides an in-built function cv2.Sobel() for this.

Official link:- https://docs.opencv.org/3.4/d2/d2c/tutorial_sobel_derivatives.html

Steps to Build a Number Plate using Computer Vision

Step 1. Import the necessary libraries

import numpy as np
import cv2
from PIL import Image
import pytesseract as pytess

Step 2. Identify unnecessary contours

Now we will focus on identifying some unnecessary contours present in the picture that can be misidentified by OpenCV as there is very little chance of it being a license plate. We will define three different functions to find these contours.

1. Firstly, we create a function named “ratioCheck” to identify the range of area and ratio between width and height.

def ratioCheck(Ar, breatth, height):
   ratio = float(breatth) / float(height)
if ratio < 1:
       ratio = 1 / ratio
if (Ar  73862.5) or (ratio  6):
return False
return True

2. Secondly, we create a function named”isMaxWhite” to identify average of image matrix:

def isMaxWhite(plate):
   avg = np.mean(plate)
if(avg>=115):
return True
else:
return False

3. Lastly, we create a function named “ratio_and_rotation” to find the rotation of contours:

def ratio_and_rotation(rect):
   (x, y), (breatth, height), rect_angle = rect
if(breatth>height):
       angle = -rect_angle
else:
       angle = 90 + rect_angle
if angle>15:
return False
if height == 0 or breatth == 0:
return False
   Ar = height*breatth
if not ratioCheck(Ar, breatth,height):
return False
else:
return True

Step 3 Clean the identified number plate

Now our task is to create a function to prepare a number plate for preprocessing by removing all the unnecessary elements and make the image ready to feed to pytesseract:

def clean2_plate(plate):
   gray_img = cv2.cvtColor(plate, cv2.COLOR_BGR2GRAY)
   _, thresh_val = cv2.threshold(gray_img, 110, 255, cv2.THRESH_BINARY)
if cv2.waitKey(0) & 0xff == ord('q'):
       pass
   num_contours,hierarchy = cv2.findContours(thresh_val.copy(),cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
if num_contours:
       conto_ar = [cv2.contourArea(c) for c in num_contours]
       max_cntr_index = np.argmax(conto_ar)
       max_cnt = num_contours[max_cntr_index]
       max_cntArea = conto_ar[max_cntr_index]
       x,y,w,h = cv2.boundingRect(max_cnt)
if not ratioCheck(max_cntArea,w,h):
return plate,None
       final_img = thresh_val[y:y+h, x:x+w]
return final_img,[x,y,w,h]
else:
return plate, None

Step 4 Recognize the number and characters

Now our task is to take user input in the form of an image. Then, we’ll perform the three discussed cv2 functions Gaussian Blur, Sobel, and morphological operations and identify the image contours and from each contour, we’ll find the loop to recognize the number plate. Finally, we’ll use the pytesseract library and feed it with images to extract the number and characters.

img = cv2.imread("testData/img1.jpg")
print("Number  input image...",)
cv2.imshow("input",img)
if cv2.waitKey(0) & 0xff == ord('q'):
   pass
img2 = cv2.GaussianBlur(img, (3,3), 0)
img2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
img2 = cv2.Sobel(img2,cv2.CV_8U,1,0,ksize=3)   
_,img2 = cv2.threshold(img2,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
element = cv2.getStructuringElement(shape=cv2.MORPH_RECT, ksize=(17, 3))
morph_img_threshold = img2.copy()
cv2.morphologyEx(src=img2, op=cv2.MORPH_CLOSE, kernel=element, dst=morph_img_threshold)
num_contours, hierarchy= cv2.findContours(morph_img_threshold,mode=cv2.RETR_EXTERNAL,method=cv2.CHAIN_APPROX_NONE)
cv2.drawContours(img2, num_contours, -1, (0,255,0), 1)
for i,cnt in enumerate(num_contours):
   min_rect = cv2.minAreaRect(cnt)
if ratio_and_rotation(min_rect):
       x,y,w,h = cv2.boundingRect(cnt)
       plate_img = img[y:y+h,x:x+w]
       print("Number  identified number plate...")
       cv2.imshow("num plate image",plate_img)
if cv2.waitKey(0) & 0xff == ord('q'):
           pass
if(isMaxWhite(plate_img)):
           clean_plate, rect = clean2_plate(plate_img)
if rect:
               fg=0
               x1,y1,w1,h1 = rect
               x,y,w,h = x+x1,y+y1,w1,h1
              # cv2.imwrite("clena.png",clean_plate)
               plate_im = Image.fromarray(clean_plate)
               text = tess.image_to_string(plate_im, lang='eng')
               print("Number  Detected Plate Text : ",text)

Code for Project GUI

Now we’ll create a python file for graphical user interface named “gui.py” to create a webform that accepts the image as input and outputs the license number on the screen.

import tkinter as tk #python library for GUI
from tkinter import filedialog
from tkinter import *
from PIL import ImageTk, Image
from tkinter import PhotoImage
import numpy as np
import cv2
import pytesseract as tess
def clean2_plate(plate):#to clean the identified number plate using above discussed openCV methods
   gray_img = cv2.cvtColor(plate, cv2.COLOR_BGR2GRAY)
  _, thresh_val = cv2.threshold(gray_img, 110, 255, cv2.THRESH_BINARY)
   num_contours,hierarchy = cv2.findContours(thresh_val.copy(),cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
if num_contours:
       conto_ar = [cv2.contourArea(c) for c in num_contours]
       max_cntr_index = np.argmax(conto_ar)
       max_cnt = num_contours[max_cntr_index]
       max_cntArea = conto_ar[max_cntr_index]
       x,y,w,h = cv2.boundingRect(max_cnt)
if not ratioCheck(max_cntArea,w,h):
return plate,None
       final_img = thresh_val[y:y+h, x:x+w]
return final_img,[x,y,w,h]
else:
return plate,None
#method to identify the range of area and ratio between width and height
def ratioCheck(Ar, breatth, height):
   ratio = float(breatth) / float(height)
if ratio < 1:
       ratio = 1 / ratio
if (Ar  73862.5) or (ratio  6):
return False
return True
#method to identify average of image matrix:
def isMaxWhite(plate):
   avg = np.mean(plate)
if(avg>=115):
return True
else:
return False
# to find the rotation of contours:
def ratio_and_rotation(rect):
   (x, y), (breatth, height), rect_angle = rect
if(breatth>height):
       angle = -rect_angle
else:
       angle = 90 + rect_angle
if angle>15:
return False
if height == 0 or breatth == 0:
return False
   Ar = height*breatth#area calculation
if not ratioCheck(Ar,breatth,height):
return False
else:
return True
top=tk.Tk()
top.geometry('900x700')#window size
top.title('Number Plate Recognition')#title of GUI
top.iconphoto(True, PhotoImage(file="/home/shikha/GUI/logo.png"))#give the path of folder where your test image is available
img = ImageTk.PhotoImage(Image.open("logo.png"))#to open your image
top.configure(background='#CDCDCD')#background color
label=Label(top,background='#CDCDCD', font=('arial',35,'bold'))#to set background,font,and size of the label
sign_image = Label(top,bd=10)
plate_image=Label(top,bd=10)
def classify(file_path):
   res_text=[0]
   res_img=[0]
   img = cv2.imread(file_path)
   img2 = cv2.GaussianBlur(img, (3,3), 0)
   img2 = cv2.cvtColor(img2, cv2.COLOR_BGR2GRAY)
   img2 = cv2.Sobel(img2,cv2.CV_8U,1,0,ksize=3)   
   _,img2 = cv2.threshold(img2,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
   element = cv2.getStructuringElement(shape=cv2.MORPH_RECT, ksize=(17, 3))
   morph_img_threshold = img2.copy()
   cv2.morphologyEx(src=img2, op=cv2.MORPH_CLOSE, kernel=element, dst=morph_img_threshold)
   num_contours, hierarchy= cv2.findContours(morph_img_threshold,mode=cv2.RETR_EXTERNAL,method=cv2.CHAIN_APPROX_NONE)
   cv2.drawContours(img2, num_contours, -1, (0,255,0), 1)
for i,cnt in enumerate(num_contours):
       min_rect = cv2.minAreaRect(cnt)
if ratio_and_rotation(min_rect):
           x,y,w,h = cv2.boundingRect(cnt)
           plate_img = img[y:y+h,x:x+w]
           print("Number  identified number plate...")
           res_img[0]=plate_img
           cv2.imwrite("result.png",plate_img)
#method to identify average of image matrix:
if(isMaxWhite(plate_img)):
               clean_plate, rect = clean2_plate(plate_img)
if rect:
                   fg=0
                   x1,y1,w1,h1 = rect
                   x,y,w,h = x+x1,y+y1,w1,h1
                   plate_im = Image.fromarray(clean_plate)
                   text = tess.image_to_string(plate_im, lang='eng')
                   res_text[0]=text
if text:
                       break
   label.configure(foreground='#011638', text=res_text[0])
   uploaded=Image.open("result.png")
   im=ImageTk.PhotoImage(uploaded)
   plate_image.configure(image=im)
   plate_image.image=im
   plate_image.pack()
   plate_image.place(x=560,y=320)
def show_classify_button(file_path):
   classify_b=Button(top,text="Classify Image",command=lambda: classify(file_path),padx=10,pady=5)
   classify_b.configure(background='#364156', foreground='white',font=('arial',15,'bold'))
   classify_b.place(x=490,y=550)
def upload_image():
try:
       file_path=filedialog.askopenfilename()
       uploaded=Image.open(file_path)
       uploaded.thumbnail(((top.winfo_width()/2.25),(top.winfo_height()/2.25)))
       im=ImageTk.PhotoImage(uploaded)
       sign_image.configure(image=im)
       sign_image.image=im
       label.configure(text='')
       show_classify_button(file_path)
except:
       pass
upload=Button(top,text="Upload an image",command=upload_image,padx=10,pady=5)
upload.configure(background='#364156', foreground='white',font=('arial',15,'bold'))
upload.pack()
upload.place(x=210,y=550)
sign_image.pack()
sign_image.place(x=70,y=200)
label.pack()
label.place(x=500,y=220)
heading = Label(top,image=img)
heading.configure(background='#CDCDCD',foreground='#364156')
heading.pack()
top.mainloop()

Computer Vision Output

Source:- https://medium.com/programming-fever/license-plate-recognition-using-opencv-python-7611f85cdd6c

Conclusion

In this blog, we use computer vision and deep learning to create a number plate recognition and license number extraction system. Here we create a GUI to upload the vehicle’s image and identify the number. We majorly focused on two libraries OpenCV to clean the number plate and pytesseract to identify the number plate digits and characters. We also learned some special features of OpenCV namely Morphological transformations, Gaussian blur, and Sobel operators.