Counting leaves from plant images using convolution neural network
Plants become the essential source of food, fuel, etc. for human beings. So, the researchers and agriculture-related industries with great efforts are involving in research, to continue agriculture for a prolonged period without any breach. Plant phenotyping addresses the rural requirements without any constraint. One of the important rural requirements is increasing the crop yield, which requires an exceptional amount of research. Plant phenotyping based on image analysis is used to analyze the plant traits for predicting the crop yield. Plant image analysis deals with plant measurements like growth, anatomy, shape, surface, etc. by analyzing the images of different plant organs like leaves, roots, etc. Counting leaves one of the important phenotypes that gives a clear idea of the plant’s health and its current development stage. In this blog, we talk about counting leaves that is one of the important phenotypes.
Our Case Study Overview
1. Problem Description
2. Data Preprocessing
3. Segmentation Model
4. Regression Models based on Segmentation model
5. Results Analysis
6. Future Work/Improvements
7. Conclusion
In this case study, we investigate the problem of counting leaves from an RGB image, an important task in plant phenotyping. It is a regression problem. We propose a data-driven approach for this task generalized over different plant species and imaging setups. To accomplish this task, we use state-of-the-art deep learning architectures: a deconvolutional network for initial segmentation and a convolutional network for leaf counting
Business strategy
Counting leaves is an important task of every day in the nursery because it gives an update on plant growth. Our model helps to solve this problem in the nursery to count leaves every day fast. It saves a lot of time and cost to do this thing automatically instead of manually. It helps to count leaves fast and saves labor costs.
Performance Metric
As we know this is a regression problem because our prediction will be real numbers.To solve this problem we use mean squared error to calculate our loss.
Dataset overview
For solving this problem we use the CVPP2017 challenge dataset (https://www.plant-phenotyping.org/CVPPP2017-challenge). The dataset contains 4 directories namely A1, A2, A3, A4 in which each directory contains Arabidopsis and Tobacco plant images or a combination of both tobacco and Arabidopsis with their leaves counts in a CSV file. Plant images are encoded as PNG files and their size may vary. In total, there are 27 Tobacco images in A3, and 783 Arabidopsis images in the rest of the directories.
Data preprocessing
- Reading image path
- Load image
- Resizing image
- Rescaling image
- Converting image data type
- Doing the same steps for mask images
Training and test dataset
As the dataset is small, so we split our dataset in an 80–20 ratio.Here we use an 80 percent dataset for training and 10 percent dataset for testing. I created a separate train And test CSV file for training. In the CSV file, we have two-column.In one column image address of the RGB image file and in another column the image address of segmented(mask) images.here I attach below my images of the CSV file.
Data Pipeline
In the data pipeline method, define the steps of loading of image, then resizing, changing shapes of the image, etc.
Segnet architecture
In segnet architecture, there are two-layer, encoder, and decoder layers. In the encoder layer, there are13 convolution layer, similarly their are13 convolution in the decoding layer. This architecture takes the RGB image and returns the segmented(mask) image.
Loss function
In this case study, For calculating the loss of our model, I used dice loss.
Training and results
After training segnet architecture on training data for 150 epochs and adam as an optimizer, a dice loss of 0.08 and 0.09 on train data and test data. We used image shape (512 * 512) for training our model. Also got a 94.05 accuracy score on train data and 90.02 accuracies on test data.
Here I attached below some predicted output by the Segnet model.
Unet Model
We also tried Unet architecture for getting mask image. After training Unet architecture on datasets for 100 epochs, we get a 93% iou score on the test dataset and dice loss 0.06. We used image shape(512 * 512) for training our Unet model.
Counting network (Regression Model based on segmentation images)
Model architecture
Counting architecture used for estimating the number of leaves from SRGB (Segmentation + RGB) channels. Each of the convolution blocks is a combination of convolution, batch normalization, and rectified linear unit (ReLU). All the pooling operations are 2 _ 2 max-pooling with the stride of 2.This network will take the SRGB image as input and return the number of leaves in the image as output. Here below I show the approach of our model
Data pipeline for creating SRGB images
As pipeline process steps did in previous steps for predicting segment image, similarly we did here pipeline for creating SRGB images. The only difference is that we only concatenating RGB and mask images so that will get SRGB images.
Here below I attached an image of the SRGB image. This image will get by concatenating RGB and grayscale images.
Training & results
After training our counting network architecture on our dataset for 30 epochs we got a total 1.73 mean squared error(MSE). We used SRGB images for training our model which have an image shape (512 * 512).
Our model shows sometimes accurate prediction on the test dataset images.
Pipelining Of Our Model Prediction
Here below I shows some prediction of our counting network model.
Result analysis on different directory of images
As our image folder contains 4 different directories of images.Since each directory contains different types of images. To evaluate the performance of our model, we also perform an analysis of these different directories. We analyze that our model did not perform well on the A2 directory. We get maximum MSE on the A2 directory.
But when we use Unet and counting network model for our prediction, we observe that our model did not perform well on the A4 directory. We get a maximum mean squared error on the A4 directory.
Model Quantization
We also quantize our model. Quantization works by reducing the precision of the numbers used to represent a model’s parameters, which by default are 32-bit floating-point numbers. This results in a smaller model size and faster computation. We analyze that our quantization model’s result is the same as the non-quantization model.
Deployment
I used streamlit app for the deployment of my model. Here below I am showing the video of deployment of my video. In this video, you can see I am drag and drop my images and my model is predicting the number of output in my images.
Future work
- We observe that the dataset is so small if we increase the dataset size our model prediction will improve.
- We can also do data augmentation that will help the model to improve results.
Conclusion
The most important takeaway from this case study is that how to do work with a small dataset, segmented images. We also observe that in our case there is no overfitting with our model. we get the best iou score (0.91) and dice loss (0.06) using our segnet model. By training the counting network model on SRGB images we get 1.73 MSE. During prediction time Segnet model works better than the Unet model.
I hope you enjoyed reading this blog as I enjoyed during solving this problem.
If you want to see my code, you can click on this Github link.
Linkedin: https://www.linkedin.com/in/ashish-saha-07212a131/
Github: https://github.com/sahaashish95
References
Dataset: CVPP2017 challenge dataset (https://www.plant-phenotyping.org/CVPPP2017-challenge).
Reference research paper: https://openaccess.thecvf.com/content_ICCV_2017_workshops/papers/w29/Aich_Leaf_Counting_With_ICCV_2017_paper.pdf
