Food Classification Using Transfer Learning And TensorFlow

This article was published as a part of the Data Science Blogathon

Abstract

In today’s report, we will analyze food items to predict whether they food or not. We apply state of the art Transfer Leanirng approach and Tensorflow framework to build a machine learning model for food classification.

Introduction

Image classification is a job where a machine will predict a picture belongs to which category. Before deep learning begins booming, tasks like image classification cannot attain human-level achievement.

It’s because the machine learning model cannot determine the neighbour knowledge of an image. The model only receives the pixel-level command.

Thanks to the potential of deep learning, image classification tasks can transfer a human-level performance utilizing a model described as Convolutional Neural Network (CNN).

CNN is a sort of deep learning model that studies representation from an image. This model can determine from flat to high-level features without individual engagement.

The model receives not only data on a pixel level. The model also gets the neighbour data from an image by a mechanism called convolution.

Convolution will aggregate neighbourhood data by multiplying the compilation of pixels in a range and sum them into a value. ML model will accept those features to classify the picture into a group.

Although deep learning can accomplish human-level production, it needs a considerable volume of data.

What if we don’t have them? We can apply a theory called transfer learning.

Transfer learning is a technique to train a model on massive-scale data for our query. Accordingly, we only prepare them by fine-tuning the model. The advantage that we will notice is the model will learn in a bit of time.

The article will advance you to practice transfer learning for Food image classification using TensorFlow (Python). Because the preprocessing step is the fundamental process, I will also explain how to serve the data to our deep learning model.

Without further delay, let’s get ignited!

Implementation

Step1: Import Libraries

The initial step that we demand to do is to import libraries. We want TensorFlow, NumPy, os, and pandas. If you haven’t fixed the package still, you can apply the pip command to install the required libraries.

Please remark that the TensorFlow that I command use is version 2.4.1, so please install that version.

Also, if you desire to adopt GPU for training the deep learning model, please install CUDA with version 11.0 because that version holds TensorFlow with version 2.4.1.

Download the CUDA software.

Below is the code to install and load the required libraries.

# ! pip install tensorflow==2.4.1
# ! pip install pandas
# ! pip install numpy
import os
import numpy as np
import pandas as pd
import tensorflow as tf

Step2: Prepare The Data

After you arrange the libraries, the following step is to fix our dataset. In this example, we will apply a dataset named Food-5K.

This dataset consists of 5000 pictures with two categories, i.e. food and non-food. FOOD-5K is partitioned into training, validation, and a test collection of data.

The formation of the dataset folder goes like this:

As you can observe above, every folder consists of pictures. Each picture filename includes the category and the identifier of it broken by an underscore.

We need to produce the dataframe with columns as the image filename and the label with that folder arrangement.

The code for fixing the dataset looks like this,

def dframe(dtype):
    X = []
    y = []
    path = 'Food-5K/' + dtype + '/'
    for i in os.listdir(path):
        # Image
        X.append(i)
        # Label
        y.append(i.split('_')[0])
    X = np.array(X)
    y = np.array(y)
    df = pd.DataFrame()
    df['filename'] = X
    df['label'] = y
    return df
df_train = dframe('training')
df_val = dframe('validation')
df_test = dframe('evaluation')

df_train.head()

And here is how the dataframe views like,

The subsequent step is to make an object to put the pictures into the model. We will practice the ImageDataGenerator object of tf.keras.preprocessing.image library.

With this object, we will produce image batches. Also, we can augment our picture to largen the product of the dataset. Because we also extend those pictures, we further set parameters for the image augmentations technique.

Also, because we apply a dataframe as the knowledge about the dataset, we will exercise the flow_from_dataframe method to produce batches and augment the pictures.

Code for above looks like,

from tensorflow.keras.preprocessing.image import ImageDataGenerator

# Create the ImageDataGenerator object
train_datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True,
)
val_datagen = ImageDataGenerator(
    featurewise_center=True,
    featurewise_std_normalization=True,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    horizontal_flip=True,
)
# Generate batches and augment the images
train_generator = train_datagen.flow_from_dataframe(
    df_train,
    directory='Food-5K/training/',
    x_col='filename',
    y_col='label',
    class_mode='binary',
    target_size=(224, 224),
)
val_generator = train_datagen.flow_from_dataframe(
    df_val,
    directory='Food-5K/validation/',
    x_col='filename',
    y_col='label',
    class_mode='binary',
    target_size=(224, 224),
)

Step3: Train the model

After we produce the batches, now we can train the model by the transfer learning technique. Because we apply that method, we don’t demand to perform CNN architecture from scratch. Instead, we will use the present and previously pretrained architecture.

We will apply ResNet-50 as the spine for our new model. We will generate the input and adjust the last linear layer of ResNet-50 with the modern one based on the number of categories.

Code to construct the model is below,

from tensorflow.keras.applications import ResNet50

# Initialize the Pretrained Model
feature_extractor = ResNet50(weights='imagenet', 
                             input_shape=(224, 224, 3),
                             include_top=False)

# Set this parameter to make sure it's not being trained
feature_extractor.trainable = False

# Set the input layer
input_ = tf.keras.Input(shape=(224, 224, 3))

# Set the feature extractor layer
x = feature_extractor(input_, training=False)

# Set the pooling layer
x = tf.keras.layers.GlobalAveragePooling2D()(x)

# Set the final layer with sigmoid activation function
output_ = tf.keras.layers.Dense(1, activation='sigmoid')(x)

# Create the new model object
model = tf.keras.Model(input_, output_)

# Compile it
model.compile(optimizer='adam',
             loss='binary_crossentropy',
             metrics=['accuracy'])

# Print The Summary of The Model
model.summary()

To train the model, we apply the fit method for preparing it. Here is the code,

model.fit(train_generator, epochs=20, validation_data=val_generator)

Step4: Test the model

Following training the model, now let’s examine the model on the test dataset. In extension, we need to combine a pillow library to load and resize the picture and scikit-learn for determining the model performance.

We will practice classification_report from the scikit-learn library to produce a report regarding model execution. Also, we will fancy the confusion matrix from it.

Here is the code to predict the experiment data and its decision,

from PIL import Image
from sklearn.metrics import classification_report, confusion_matrix

y_true = []
y_pred = []

for i in os.listdir('Food-5K/evaluation'):
    img = Image.open('Food-5K/evaluation/' + i)
    img = img.resize((224, 224))
    img = np.array(img)
    img = np.expand_dims(img, 0)
    
    y_true.append(int(i.split('_')[0]))
    y_pred.append(1 if model.predict(img) > 0.5 else 0)
    
print(classification_report(y_true, y_pred))
print()
print(confusion_matrix(y_true, y_pred))

As you can observe from the preceding, the model has already touched beyond 95 per cent on performance. Accordingly, we can accept this model in the event of establishing an image classifier API.

Step5: Save the model

If you desire to use the model for the subsequent use or deployment, you can save the model utilizing the save method like this,

model.save('./resnet50_food_model')

If you require to load the model, you can practice the load_model method like this,

model = tf.keras.models.load_model('./resnet50_food_model')

What’s Next

Well Done! Now you acknowledge how to perform transfer learning using TensorFlow. I hope this study encourages you, especially those who aspire to train a deep learning model with inadequate data.

Thank you for browsing my article!

If you are affected by this report, you can follow me on Medium. Also, if you desire to discuss with me, you can connect with me on LinkedIn.

The media shown in this article on Food Classification using Transfer learning are not owned by Analytics Vidhya and is used at the Author’s discretion.

Mrinal Singh

Data Scientist and a Technical Writer! I will give you the best of Open-Source and AI.

Talks about #chatgpt, #opensource, #contentcreation, #communitybuilding, and #artificialintelligence

Technical Writer | Data Science, ML, AI, Open-Source | Do More with Data - Litmus