Agriculture is the lifeblood of our civilization, providing sustenance and nourishment to billions around the globe. However, this vital industry faces a relentless adversary: plant diseases. These microscopic threats can wreak havoc on crops, leading to significant economic losses and food shortages. The key to safeguarding our agricultural heritage lies in early detection and timely intervention, where cutting-edge technology steps in. This comprehensive guide will embark on a journey into plant disease classification using Monk, a powerful machine-learning library. By the end of this article, you’ll be equipped with the knowledge to harness the potential of artificial intelligence in identifying and combating plant diseases effectively.
So, fasten your seatbelts as we explore how Monk empowers us to create, train, and optimize deep learning models for plant disease classification. But before we dive into the technical aspects, let’s set up the stage by understanding the significance of this endeavor and why Monk plays a pivotal role.
This article was published as a part of the Data Science Blogathon.
Monk is a versatile software library and deep learning tool focusing on computer vision. It serves as a low-code solution for various AI applications, particularly in the field of computer vision. Monk’s capabilities encompass object detection, plant disease classification, and more. It provides a unified wrapper for computer vision tasks, making it easier to perform tasks like object detection efficiently and without extensive coding. This library has found applications in areas like protecting crops from diseases, detecting marine creatures, and simplifying computer vision tasks.
Monk is an AI and computer vision tool designed to evaluate items based on their visual aspects, with an initial focus on mobility applications. It streamlines data modeling, manipulation, and validation while supporting various databases. Monk is a valuable tool for developers and researchers working on computer vision and AI projects.
This section will walk you through the step-by-step process of building your Monk model for plant disease classification. Whether you’re new to machine learning or a seasoned data scientist, follow these instructions to get started on your plant disease classification journey.
In this first step, we will gather the necessary dataset for our plant disease classification project. Follow these steps to collect the data:
The fantastic team at Plant Village gathered the dataset
1. Upload Kaggle API Token:
from google.colab import files
files.upload()
2. Install the Kaggle Python Package:
!pip install kaggle
3. Set Kaggle Configuration Directory:
import os
os.environ['KAGGLE_CONFIG_DIR'] = '/content'
4. Set Appropriate Permissions:
!chmod 600 /content/kaggle.json
5. Download the Dataset:
!kaggle datasets download -d "prashantmalge/plant-disease"
6. Unzip the Dataset:
!unzip plant-disease.zip
This will make the dataset available in your Colab environment for further processing and training. Adjust the dataset name and paths as needed for your specific project.
First, you must set up Monk on your local machine or cloud environment. Follow these steps to get Monk up and running:
1. Clone the Monk repository:
# Clone the Monk repository
!git clone https://github.com/Tessellate-Imaging/monk_v1.git
# Add the Monk repository to the Python path
import sys
sys.path.append("./monk_v1/monk")
# Install any required packages or dependencies if needed
# Uncomment and add your installation command here
# !pip install -r /kaggle/working/monk_v1/installation/requirements_kaggle.txt
2. Install Monk using pip:
!pip install -U monk-colab
Now, let’s create a Monk experiment. An experiment is a structured environment where you define the parameters and configurations for training your disease classifier. Here’s a code snippet to get you started:
from pytorch_prototype import prototype
# Create an experiment
ptf = prototype(verbose=1)
ptf.Prototype("plant_disease", "exp1")
In this code, we’ve named our experiment “plant_disease” with the tag “exp1.” You can adjust the names according to your project.
Load your dataset and apply data transforms for preprocessing. Monk provides convenient functions for loading data and using transforms. Here’s an example:
# Load the dataset and define transforms
ptf.Default(dataset_path=["./dataset/train", "./dataset/val"],
model_name="resnet18", freeze_base_network=True, num_epochs=5)
In this snippet, we specify the dataset path and the model architecture (ResNet-18) and freeze the base network’s weights.
Choose a model architecture that suits your needs and initiate the training process. Monk supports various pre-trained models, and you can fine-tune them as required. Here’s how you can do it:
# Analysis - 1
# Analysis Project Name
analysis_name = "Model_Finder";
# Models to analyse
# First element in the list- Model Name
# Second element in the list - Boolean value to freeze base network or not
# Third element in the list - Boolean value to use pretrained model as the starting point or not
models = [["resnet34", True, True], ["resnet50", False, True],
["densenet121", False, True], ["densenet169", True, True], ["densenet201", True, True]];
# Num epochs for each experiment to run
epochs=5;
# Percentage of original dataset to take in for experimentation
percent_data=10;
# "keep_all" - Keep all the sub experiments created
# "keep_non" - Delete all sub experiments created
ptf.Analyse_Models(analysis_name, models, percent_data, num_epochs=epochs, state="keep_none");
This code is helpful for quickly testing multiple models with different configurations to find the best-performing model for your task. Selecting an appropriate model architecture is a standard step in machine learning and deep learning projects.
Output :
## Update Model Architecture
ptf.update_model_name("densenet121");
ptf.update_freeze_base_network(True);
ptf.update_use_pretrained(True);
ptf.Reload();
In summary, this code switches the model architecture to DenseNet-121, freezes the base network, and uses pre-trained weights. These changes are then applied to the model by reloading it.
Output :
#Analysis - 2
# Analysis Project Name
analysis_name = "Batch_Size_Finder";
# Batch sizes to explore
batch_sizes = [4, 8, 16, 32];
# Num epochs for each experiment to run
epochs = 10;
# Percentage of original dataset to take in for experimentation
percent_data = 10;
# "keep_all" - Keep all the sub experiments created
# "keep_non" - Delete all sub experiments created
ptf.Analyse_Batch_Sizes(analysis_name, batch_sizes, percent_data,
num_epochs=epochs, state="keep_none");
This analysis aims to determine the batch size that results in the best training and validation performance for your deep learning model. Different batch sizes can affect training speed and convergence; this analysis helps you find an optimal value.
Output :
## Update Batch Size
ptf.update_batch_size(8);
ptf.Reload();
Setting the batch size to 8 means that your model should process data in 8 samples per batch during training. This value was determined from your batch size analysis (as shown in the previous code snippet) to find an optimal batch size for your specific deep-learning task.
Output :
#Analysis - 3
# Analysis Project Name
analysis_name = "Input_Size_Finder";
# Input sizes to explore
input_sizes = [224, 256, 512];
# Num epochs for each experiment to run
epochs=5;
# Percentage of original dataset to take in for experimentation
percent_data=10;
# "keep_all" - Keep all the sub experiments created
# "keep_non" - Delete all sub experiments created
ptf.Analyse_Input_Sizes(analysis_name, input_sizes, percent_data,
num_epochs=epochs, state="keep_none");
This analysis lets you determine which input size works best for your specific deep-learning task. Different input sizes can impact model performance and training speed, so finding the right balance for your project is essential.
Output :
## Update Input Size
ptf.update_input_size(224);
ptf.Reload();
By setting the input size to 224×224 pixels, you have effectively prepared your model to accept images of this size during training and inference. The choice of input size should align with your dataset and task requirements, and it’s often a critical hyperparameter to tune for optimal results.
Output :
#Analysis - 4
# Analysis Project Name
analysis_name = "Learning_Rate_Finder"
# Learning rates to explore
lrs = [0.01, 0.005, 0.001, 0.0001];
# Num epochs for each experiment to run
epochs=5
# Percentage of original dataset to take in for experimentation
percent_data=10
# "keep_all" - Keep all the sub experiments created
# "keep_non" - Delete all sub experiments created
ptf.Analyse_Learning_Rates(analysis_name, lrs, percent_data,
num_epochs=epochs, state="keep_none");
After running this code, the analysis will explore the specified learning rates, train the model for a few epochs with each learning rate, and collect performance metrics. This information will help you choose your model and dataset’s most suitable learning rate.
Output :
## Update Learning Rate
ptf.update_learning_rate(0.01);
ptf.Reload();
Setting the learning rate to 0.01 allows you to specify a new learning rate for your model. Adjusting the learning rate is common in fine-tuning deep learning models to improve training stability and convergence.
Output :
# Analysis - 5
# Analysis Project Name
analysis_name = "Optimiser_Finder";
# Optimizers to explore
optimizers = ["sgd", "adam", "adamax", "rmsprop"]; #Model name, learning rate
# Num epochs for each experiment to run
epochs = 5;
# Percentage of original dataset to take in for experimentation
percent_data = 10;
# "keep_all" - Keep all the sub experiments created
# "keep_non" - Delete all sub experiments created
ptf.Analyse_Optimizers(analysis_name, optimizers, percent_data,
num_epochs=epochs, state="keep_none");
By analyzing different optimizers, you can identify which works best for your dataset and deep learning model. The choice of optimizer can significantly impact the training and performance of your neural network.
Output :
## Update Optimiser
ptf.optimizer_adamax(0.001);
ptf.Reload();
You are changing the optimization algorithm used during training by updating the optimizer to “Adamax” with a specific learning rate. Different optimizers may converge at different rates, leading to variations in the training process and final model performance. It’s common to experiment with other optimizers and learning rates to find the best combination for your specific deep-learning task.
ptf.update_save_intermediate_models(False);
Intermediate models are snapshots of your model’s parameters saved at specific intervals during training. They can help resume training from a particular checkpoint or analyze the model’s performance at various stages of training.
Setting this option to False means that your code will not save intermediate models during the training process, which can be beneficial if you want to conserve disk space or do not need to keep track of intermediate checkpoints.
Next, we create a new experiment using ‘copy_experiment’ and resume training to achieve further improvement.
ptf = prototype(verbose=1);
ptf.Prototype("plant_disease", "exp2", copy_from=["plant_disease", "exp1"]);
By creating a new experiment with the same settings as an existing one, you can quickly iterate on your experiments while maintaining consistency in configurations and tracking progress.
Summary of the experiment configuration:
ptf.Summary()
Output :
Monk provides a streamlined “Compare” function that gathers statistics, visualizes results, and helps users decide which model architectures and hyperparameters are most effective. This step aids in the iterative process of fine-tuning models and progressively improving their performance, ultimately guiding the selection of the best-performing model configurations for deployment.
from compare_prototype import compare
ctf = compare(verbose=1);
ctf.Comparison("plant_disease");
ctf.Add_Experiment("plant_disease", "exp1");
ctf.Add_Experiment("plant_disease", "exp2");
These code snippets demonstrate the use of Monk’s “compare_prototype” module. First, it imports the necessary “compare” function. Then, it initializes a comparison object “ctf” with verbosity set to 1 for reporting. The comparison is named “plant_disease” using ctf.Comparison.
Following this, two experiments, “exp1” and “exp2,” conducted under the project “plant_disease,” are added to the comparison using ctf.Add_Experiment. This allows users to analyze and compare these two experiments’ results, metrics, and performance to make informed decisions about model selection and fine-tuning.
Inference in Monk uses a trained model to predict new, unseen data. It allows you to utilize your trained model for real-world applications, such as classifying images, recognizing objects, or making decisions based on the model’s output. Inference typically involves loading a trained model, providing input data, and obtaining predictions or classifications from the model. Monk offers tools and functions to streamline the inference process and simplify deploying machine-learning models for practical use.
import numpy as np
import PIL.Image as Image
import requests
test_url = "https://gardenerspath.com/wp-content/uploads/2019/08/
Frogeye-spots-Botryosphaeria-obtusa-on-apple-leaf-FB.jpg"
# URL of the image to be downloaded is defined as image_url
r = requests.get(test_url) # create HTTP response object
with open('test.jpg','wb') as f:
f.write(r.content)
test = Image.open('./test.jpg')
test
Output :
ptf.Prototype("plant_disease", "exp2", eval_infer=True);
Output :
img_name = "test.jpg";
predictions = ptf.Infer(img_name=img_name, return_raw=True);
print(predictions)
This code allows you to analyze the specified image using the trained model and obtain predictions for various classes and their confidence scores. The printed output will provide insights into the model’s prediction for the given image.
import csv
def read_labels():
mydict = {}
with open('./dataset/labels.csv', mode='r') as infile:
reader = csv.reader(infile)
with open('labels_new.csv', mode='w') as outfile:
writer = csv.writer(outfile)
mydict = {rows[0]:rows[1] for rows in reader}
return mydict
def predict_label(predictions):
pred_class = predictions['predicted_class']
label_dict = read_labels()
out_label = label_dict[pred_class]
return out_label
print("Predicted class for test image is : {}".format(predict_label(predictions)))
In summary, this code helps you translate the numeric class ID predicted by the model into a human-readable label, making it easier to understand the model’s output.
Output: Predicted class for test image is: Apple___Black_rot
Let’s explore advanced techniques and strategies to take your plant disease classification project to the next level.
While using pre-trained models is an excellent starting point, you can further improve your classifier’s accuracy by fine-tuning it with custom data. Collect more images specific to your target plant species and diseases. You can adapt a pre-trained model on your dataset by fine-tuning it to recognize unique patterns and symptoms.
# Fine-tuning a pre-trained model with custom data
ptf.update_model_name("resnet50")
ptf.update_freeze_base_network(False) # Unfreeze base network for fine-tuning
ptf.update_use_pretrained(True)
ptf.Reload()
ptf.Train()
Data augmentation is a powerful technique to increase the diversity of your training dataset artificially. By applying random transformations such as rotation, cropping, and brightness adjustments to your images, you can enhance your model’s ability to generalize. Monk provides convenient functions for data augmentation:
# Apply data augmentation transforms
ptf.apply_random_rotation(train=True, val=True)
ptf.apply_random_crop(scale=[0.8, 1.0], ratio=[0.8, 1.2], train=True)
ptf.apply_random_brightness(train=True)
In real-world scenarios, you may encounter class imbalance, where some plant diseases are more prevalent than others. To address this, consider techniques like oversampling minority classes or applying class weights during training. Monk offers class-weighted loss functions to handle imbalanced datasets.
# Use a class-weighted loss function
ptf.loss_weighted_softmax_crossentropy(class_weights)
Ensemble learning involves combining predictions from multiple models to improve accuracy and robustness. You can create an ensemble of different architectures or variations of the same model to achieve better results. Monk supports ensemble learning, allowing you to experiment with different combinations of models.
# Create an ensemble of models
ensemble = ptf.create_ensemble(models)
ensemble.Train()
Fine-tuning hyperparameters is crucial for optimizing your model’s performance. Monk provides tools for hyperparameter tuning, allowing you to explore various learning rates, batch sizes, and optimization algorithms.
# Hyperparameter tuning - Learning rate, batch size, optimizer
ptf.Analyse_Learning_Rates("LR_Finder", lrs, percent_data=10, num_epochs=5)
ptf.Analyse_Batch_Sizes("Batch_Size_Finder", batch_sizes, percent_data=10, num_epochs=5)
ptf.Analyse_Optimizers("Optimizer_Finder", optimizers, percent_data=10, num_epochs=5)
Understanding why a model makes specific predictions is essential, especially in critical applications like disease classification. Monk provides interpretability tools like Grad-CAM (Gradient-weighted Class Activation Mapping) to visualize which regions of an image are most influential for a prediction.
# Visualize model predictions with Grad-CAM
heatmap = ptf.Visualize_With_GradCAM(image_path, target_layer)
In this comprehensive guide, we’ve explored the fascinating world of plant disease classification using Monk. We’ve covered everything from setting up your experiments to fine-tuning models and understanding real-world applications. As we conclude, let’s recap the key takeaways and discuss the exciting future directions of this technology.
Key Takeaways:
To dive deeper into plant disease classification with Monk, here are some valuable resources:
Monk remains at the forefront for plant disease classification solutions. Its user-friendly interface and powerful capabilities empower individuals and organizations to contribute to the well-being of our planet’s ecosystems and the global food supply.
A1: The Monk framework is a powerful deep-learning tool designed to streamline the process of building and training machine-learning models. It offers a user-friendly interface, pre-built models, and various optimization tools. Using Monk for plant disease classification simplifies model development, saving time and effort while achieving accurate results.
A2: Setting up Monk is easy! Install it, configure your project directory, and prepare your dataset. Monk provides step-by-step instructions to help you get started quickly, making the setup process accessible even for beginners.
A3: When choosing a model architecture, consider factors like model complexity, available computational resources, and the size of your dataset. Monk’s “Quick Model Finder” feature helps you analyze various pre-trained models, making it easier to select the one that strikes the right balance between accuracy and efficiency for your specific project.
A4: Monk provides tools to fine-tune your model’s hyperparameters, such as batch size, learning rate, and optimizer. You can use its “Batch Size Optimization,” “Learning Rate Exploration,” and “Optimizing the Optimizer” features to experiment with different settings and discover the best configuration for your model.
The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.