An Intelligent Approach for Food Standards Prediction Using Machine Learning

Abstract

Nowadays, standard intake of healthy food is necessary for keeping a balanced diet to avoid health issues in the human body. This project proposes a food recognition system that uses a convolution neural network as a base model for image prediction and then returns nutrition facts such as calories in the given single food image. Knowing the nutrition content of the food that we are consuming helps in maintaining balanced diet. We have aimed with a variety of food categories, each containing thousands of images, and through machine learning training to achieve higher classification accuracy. Firstly, we have planned to train and optimize a CNN, state-of-art model using Tensorflow, we are using CNN as the convolution layers are tweak able and easy to implement. Second, we adapt our model with GUI features as well as nutrition analysis.

Introduction

I. INTRODUCTION

People are very keen on measuring weight, healthy diets, and staying away from obesity, there is an increasing demand for food calorie measurement.

Adult obesity is increasing at an alarming rate. The main source of obesity is the difference between dietary intake and the energy people get from the diet. High-calorie intake may be injurious and lead to various diseases. Hypertension, heart attack, diabetes, obesity, hypertension, high cholesterol Breast, colon and prostate cancers are caused by high calorie intake. High calorie intake is the second leading cause of cancer. Dieticians have determined that the standard intake of a number of calories is required to keep the right balance of calories in the human body.

As reported by the world health organization, more than 110th of the adult population in the world is obese. Obesity is a medical condition in which excess body fat has accumulated to the extent that it may have a negative effect on health. If the amount of food a person takes daily is higher than the amount of Energy utilized then we can say that the respective person is becoming obese. Obesity and being overweight are interconnected to many dangerous and chronic diseases. In 2013, The American Medical Association officially declared obesity as the disease that has serious consequences on patients health and therefore requires medical treatment. Therefore, daily intake measurements are important for losing weight and maintaining healthy diet and weight for normal people. Only a timely measurement of daily food consumption can make obese people lose weight in a healthier way, and can also make healthy people better healthy. Abstract Nowadays, standard intake of healthy food is necessary for keeping a balanced diet to avoid obesity in the human body. In this paper, we present a novel system based on machine learning that automatically performs accurate classification of food images and estimates food attributes. This paper proposes a deep learning model consisting of a convolutional neural network that classifies food into specific categories in the training part of the prototype system. The main purpose of the proposed method is to improve the accuracy of the pre-training model.

A. Problem Statement

In this growing digital world it is very important to keep the track of calorie intake in the form of food As and how the world is growing digitally problems like obesity , weight gains etc are also equally growing it becomes more inevitable. The system uses image processing and computational intelligence for food item recognition, convolutional neural network has been applied in food classification and resulted in a high accuracy.

B. Objectives

The Objectives of our project are :

1. To detect the image of the Food using input images.
2. To employ an algorithm which can accurately predict the food item from the given food image.

C. Proposed System

The main aim of the project is to build a new approach of image classification system to positively identify different standards in food. In this project, we first perform different pre- processing techniques on food categories to enhance clarity of the images, then we perform training of the model, testing of the model and classification of food based on standards.

II. LITERATURE SURVEY

1. Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z.(2016).Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2818-2826) They have provided several design principles to scale up convolutional networks and studied them in the context of the Inception architecture. This guidance can lead to high performance vision networks that have a relatively modest computation cost compared to simpler, more monolithic architectures. The combination of lower parameter count and additional regularization with batch- normalized auxiliary classifiers and label-smoothing allows for training high quality networks on relatively modest sized training sets.
2. J. D. A. Berg and L. Fei-Fei, “Large scale visual recognition challenge 2010,” [Online accessed 29-Jan- 2018]. Our final contribution is a hashing scheme for bilinear similarity on probability distributions that is shown to provide efficient (sublinear) retrieval in our setting, and may be useful in a wide range of applications. Using our technique, one query against a database of 600K images takes only 3 milliseconds with an accuracy close to 90% of linear scan and the computational cost 0.001 times that of linear scan. This completes an end-to-end system for very large scale hierarchical retrieval that has inherent parallelization, linear time training, sublinear time retrieval, and better accuracy and scalability than the state of the art.
3. Mohammed A. Subhi and Sawal Md. Ali. “A Deep Convolutional Neural Network for Food Detection and Recognition” (2019) In this paper, we have implemented a basic CNN approach to detect and recognize food items We have presented a new dataset for local Malaysian food which contains food categories with (5800) images. Two datasets were used for performance evaluation of our proposed approach, Food-101 were utilized for food/non-food classification and our proposed. Local Malaysian food dataset, which was used for the categorization of food items. Additionally, we have implemented very deep convolutional networks (24 weight layers) for food image classification. Large kernel size at the beginning of the layers ensures that shape. An Intelligent Approach For Food Standards Prediction Using Machine Learning features are maintained in the learning process. It was shown that it is beneficial for classification accuracy to have this depth. The results confirm the significance of network depth in training visual representations.
4. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh,S. Ma, Z.Huang, A. Karpathy, A. Khosla, M. Bernstein “ImageNet Large Scale Visual Recognition Challenge” (2019) In this paper we described the large-scale data collection process of ILSVRC, provided a summary of the most successful algorithms on this data, and analyzed the success and failure modes of these algorithms..
5. Lukas Bossard and Matthieu Guillaumin and Luc Van Gool “Food Calorie Measurement Using Deep Learning Neural Network” (2014). Diabetes and cardiovascular diseases. In 2014, about 13% of the world's adult population (11% of men and 15% of women) were obese. Accurate estimation of dietary intake is important for assessing the effectiveness of weight loss interventions. In order to reduce bias and improve the accuracy of self-report, we proposed new algorithms to analyze the food image captured by mobile devices (e.g., smartphone). The key technique innovation in this paper is the deep learning-based food image recognition algorithms. Our proposed algorithms are based on Convolutional Neural Network (CNN). Our experimental results on two challenging data sets using our proposed approach exceed the results from all existing approaches. In the future, we learn to improve performance of the algorithms and integrate our system into a real world mobile devices and cloud computing-based system to enhance the accuracy of current measurements of dietary caloric intake.
6. Kaiming He, XiangyuZhang, Shaoqing Ren, Jian Sun “Deep Residual Learning for Image Recognition” (2015). Above method has good generalization performance on other recognition tasks. Table 7 and 8 show the object detection baseline results on PASCAL VOC 2007 and 2012 [5] and COCO [26]. We adopt Faster R-CNN [32] as the detection method. Here we are interestedin the improvements of replacing VGG-16 [40] with ResNet-101. An Intelligent Approach For Food Standards Prediction Using Machine Learning
7. Gözde ÖZSERT Y???T and Buse MelisÖZYILDIRIM “Comparison of Convolutional Neural Network Models for Food Image Classification” (2018). In this study, the development of a pre-trained structure for food recognition was aimed. To achieve this aim, three different models were implemented, and their performances were compared with that of popular pre-trained models Alex-Net and Caffe-Net. The transfer learning technique was utilized to be able to use these pre-trained models for our problem. Three similar structures were proposed for analyzing the effectiveness of layers separately. While the first structure includes local response normalization layers as in Alex-Net and Caffe-Net, the third structure is the same as the first structure except local response normalization.
8. Simon Mezgec, Barbara Korousic Seljak “Nutri Net: A Deep Learning Food and Drink Image-Recognition System for Dietary Assessment” (2017). In this paper, they have introduced an intelligent cloud broker for EHSH, supporting different food categories of Single Food Object, Multiple Food Object and Mixed Food Object. We have further introduced a ingredient testing approach for the mixed food category. Also ,introduced two decision algorithms (Decision Algo. 1 and Decision Algo. that enabled the cloud broker to make decisions of whether or not to offload to other cloud instances and if so to which cloud instance in particular. We discussed the various components of the intelligent cloud broker, including the food recognition components (deep learning feature extraction and deep learning classification model), calorie measurement, indexing the food images, scheduling and queue management. Finally, scalability, consistency, and the performance of the system in terms of response time was studied and presented. The proposed model helped us achieve an improved performance and efficient utilization of the cloud resources on one hand and also to address the aspect of the system, wherein the system.
9. Z. Ning, F. Xia, X. Kong and Z. Chen “Social Oriented Resource Management in Based Mobile Networks” (2016). In this paper, we have studied the social-oriented adaptive transmission in opportunistic Internet of smartphones. Our method has two steps. First, the social attributes of smart phones are modeled by their friendships, and the social oriented double auction based relay selection method has been presented by considering node trust to stimulate the cooperation among different smartphones. After that, a spec trum spatial reuse-aware relay. An Intelligent Approach For Food Standards Prediction Using Machine Learning method selection scheme is designed to study the interplay between NC and spatial reuse by concurrently activating links in an optimal way. Due to the formulated problem is NP- complete, a firefly algorithm based heuristic scheme is presented to approach the optimal network performance with low computational complexity.
10. M. Artuso and H. Christiansen “Optimizing TCP for Cloud Based MobileNetworks” (2016). In this paper, the proposed method has achieved a high accuracy in classifying the images. We have implemented the convolutional neural network using Tensor Flow. It has been shown that classifier performs well in terms of classification accuracy. However, we use a CPU based system, so experiment has taken extra training time, if we would have used a GPU based system, the training time would have been less. This model can be applied for solving complex image classification problem related to medical imaging and other fields.
11. Vaibhavee Gamit1, Mr. Swarndeep Saket, “Food Recognition And Nutrients Identification For Making Healthy Food Choices”(2017). In the proposed system, a system for food recognition and identification of different nutrients contained in the recognized food object is carried out. The system helps people by helping them make healthy food choices and maintaining a healthy life. The system focuses on feature extraction and classification for accurate food recognition. The accuracy of the proposed system is 96.25%. The system works well for single foods, in future the accuracy of mixed foods can be improved significantly.
12. Keiji Yanai, “Image Recognition of 85 Food Categories by Feature Fusion ”(2016). In this paper, we extended the food recognition system proposed in terms of the number of categories and employed image features. By integrating seventeen kinds of image features with Multiple Kernel Learning, we obtained the 62.52% classification rate for 85-food- category classification evaluated by five-fold cross-validation. If we allow the system to return three candidate categories, the classification rate exceeded 80%. In addition, we implemented a prototype system to recognize food images taken by cellular-phone cameras, and we obtained 45.3% as the classification rate for 785 food images which were actually uploaded by the trial users.

III. SYSTEM DESIGN

A. System Architecture

The figure 1 depicts the architectural diagram of Food Standard Prediction system. The framework can be comprehensively sorted into following significant stages:

1. Acquisition of Image: Images are obtained either by lens or by secretly deleting them from the contraction. Whatever the source may be, it is very important that the image of the data is transparent and cautious. An incredible picture is needed for this.
2. Pre-Processing of Image: In this process, the photo is standardized by clearing the commotion, as it may confuse the evaluation. Similarly, the image given as the information may not be of standard size as required by the figure, so it is vital that the image size needed is obtained.
3. Data Storage Aspect To Preserve Information Images For Testing And Training: If controlled learning will occur, as is the case here, it is important to prepare data sets. The sample database is the images collected during the photo procurement process
4. Classifier to classify the FOOD: The classifier used here is the last layer of the system which gives the true probability of each experience. The project involves two major parts Image preparation unit and grouping unit. The object processing system enhances the image by removing the clatter and noisy bits. The Food image will then be isolated into different segments to isolate the image from running the mill after the image features are evacuated to check whether or not the Food is contaminated.

• Noise reduction unit: removes from the photo the unwanted colors .
• Image enhancement unit and segmentation: carries the affected part to the middle by improving the area and dividing the area into different segments in order to isolate it from the normal Scanned Image.
• Feature Extraction Component: One of the notable developments in any gathering- centered issues is highlighting extraction. Looks are the cornerstone for both purposes of planning and screening. This feature contains noteworthy image information that will be used to identify the Food.
• Identification unit for Food: See if the Food is considerate or hazardous.
• Input Attributes: For example, all noteworthy attributes, asymmetry, edge, concealment, distance, progression, etc. that have been expelled from the image are now provided as a dedication to Part II, which is the classifier part.
• Classifier engine: characterizes the images by grouping the calculation into one of the predefined Food.

B. Use case Diagram

A Use Case Diagram is a lot of situations that reflect a client-frame relationship. A use case chart shows the entertainer-to-use relationship. Usage cases and on-screen characters are the two main elements of an usage case diagram. An on-screen character refers to an user or other person connected with the demonstrated process. A use case chart in figure is an out- of - the-box perspective that speaks to some activity each module will perform to complete an errand.

C. Flowchart

1. Food Image Dataset – It contains different classes of food and each class has sample images. The dataset inherently comes with a lot of noise since there are images in which there is more than one food item. The image samples also contain a lot of color and few of them are wrongly labeled too. The figure below shows the sample food images from the Indian Food dataset.
2. Image pre-processing- The dataset contains different classes of food images. Each class of image is divided into training and testing images wherein images from each class are considered as training samples and the remaining samples as test samples. Overall, there are training samples and test samples. The training set images are fed to the CNN model and validation is made using the test dataset
3. Training the CNN classifier using pretrained models– The model will be using the technique of Transfer Learning for training the model - “Feature Extraction from pre-trained model and training a classifier using extracted features”.
4. Validation and Testing - Once the model is trained using the train dataset (the sample of data used to fit the model) then validated using validation dataset (The sample of data used to provide an unbiased evaluation of a model fit on the training dataset while tuning model hyper parameters.) and finally tested using the test dataset.
5. Calories Extraction of the Classified Image - Our classifier can be used to estimate the calorific content of the classified food from the internet. Suitable python or any scripts can be used to perform web scraping to fetch the nutrition facts for the classified image from the web and provide it to the user.

IV. IMPLEMENTATION

A. Methodology Of Proposed System

The proposed food detection and recognition model is based on the implementation of the concepts of image processing and computer vision. These concepts are bundled together to get the desired result, the implementation.

The first Convolutional 2D layer consists of 32 kernels of 3x3. Takes an input of size 100x100x3 where 100x100 is the rescaled size of images from Food101 dataset. RGB, the color aspect of the image is denoted by 3. The second layer with a pool size of 2x2 is the max- pooling layer. For better feature extraction, these layers are repeated once again. Then, to get more filtered images for the fully connected layers, the kernel’s size is increased from 32 to 64. Two fully connected layers are used next with 128 and 90 neurons respectively. To prevent overfitting, dropouts have been added in between the dense layers. All the convolutional 2D layers and the fully connected layers have an activation function of Rectified Linear Unit (ReLu). The last final layers consist of 101 neurons that are equal to the number of categories in our Food101 Dataset. The model predicts the category to be the one with the highest probability Transfer learning is the reuse of a pre-trained model for a new problem, it is very popular nowadays in deep learning because it can train deep neural networks with relatively little data, and it is very useful in data science because of most real problems. , you don't have millions of data points marked to train these complex models

B. Algorithm

Convolutional Neural Network (CNN)

Convolutional neural framework is one of the principal categories for the photos affirmation and pictures portrayals. Articles disclosures, affirmation faces, etc., are a bitof the regions where CNNs are commonly utilized. The 6 shows the Neural Network with various convolutional layers. 1n certainty, the possibility of significant learning CNN models t can be used for train and attempted, every data picture will be adhered to the course of action of convolution layers with procedures (Kernals), Pooling, totally related layers (FC) by applying Soft max work can arrange an article with probabilistic characteristics runs some place in the scope of 0 and 1. The underneath figure is a complete stream of CNN to process an information picture and requests the articles subject to values.

We must have a basic idea on how the human brain recognizes an object in spite of its varying attributes from one another. Our brain has a complex layer of neurons,each layer holds some information about the object and all the features of the object are extracted by the neurons and stored in our memory, next time when we see the same object the brain matches the stored features to recognize the object, but one can easily mistake it as a simple “IF- THEN” function, yes it is to some extent but it has an extra feature that gives it an edge over other algorithms that is Self-Learning, although it cannot match a human brain but still it can give it a tough competition . Image is processed using the Basic CNN to detect the Calories in Food.

VI. FUTURE SCOPE

As far as the future enhancement is concerned, the task of classification can be improved by removing noise from the dataset. The same research can be carried out on larger dataset with more number of classes and more number of images in each class, as larger dataset improves the accuracy by learning more features and reduces the loss rate. The weights of the model can be saved and used to design a web app or mobile app for image classification and further calories extraction of the classified food

Conclusion

In this work, we trained our model with different test sets and got a mean average accuracy of about 85%. A lot of data augmentation and segmentation that has to be performed to clean pixel values in other classifiers which is not mandatory in CNN. Once our trained model has produced the most probable output, we call the Nutritionix and return the one serving food item nutrition facts on the user’s screen. The whole process takes at most 5 seconds. The model proposed in this paper is doing well with the given data set both in terms of speed and accuracy but state of-art models such YOLO can also be integrated with it to get better results and accuracy. To achieve fast multiple object detection with boundary boxes in real time in a single image can also be achieved by using pre-trained models. Also, an effective and reliable system can be developed for real time food recognition and calorie estimation system. In this research study, the Convolutional Neural Network, a Deep learning technique is used to classify the food images in to their respective classes. The dataset considered is the Indian food dataset. The Flowchart shows the flow of operation done to detect the particular livestock and count them accordingly that is shown in result. Here first the image is captured by using a camera and which is then converted to a grey scale image to make it feasible for comparison with the existing data set values.

References

[1] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, and Wojna Z. “Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition” (2021) [2] J.D.A Berg and L Fei-Fei, “Large scale visual recognition challenge” (Jan 2018) [3] Mohammed A. Subhi and Sawal Md. Ali. “A Deep Convolutional Neural Network for Food Detection and Recognition” (2019) [4] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh,S. Ma, Z.Huang, A. Karpathy, A. Khosla, M. Bernstein “ImageNet Large Scale Visual Recognition Challenge” (2019) [5] Lukas Bossard and Matthieu Guillaumin and Luc Van Gool “Food Calorie Measurement Using Deep Learning Neural Network” (2014) [6] Kaiming He, XiangyuZhang, Shaoqing Ren, Jian Sun “Deep Residual Learning for Image Recognition” (2015) [7] Gözde ÖZSERT Y???T and Buse Melis ÖZYILDIRIM “Comparison of Convolutional Neural Network Models for Food Image Classification” (2018) [8] Simon Mezgec, Barbara Korousic Seljak “Nutri Net: A Deep Learning Food and Drink Image-Recognition System for Dietary Assessment” (2017) [9] Z. Ning, F. Xia, X. Kong and Z. Chen “Social Oriented Resource Management in Based Mobile Networks” (2016). [10] M. Artuso and H. Christiansen “Optimizing TCP for Cloud Based Mobile Networks” (2016) [11] Vaibhavee Gamit1, Mr. Swarndeep Saket, “Food Recognition And Nutrients Identification For Making Healthy Food Choices” (2017) [12] Keiji Yanai, “Image Recognition of 85 Food Categories by Feature Fusion ”(2016).

Copyright

Copyright © 2022 Dr. Ashoka K, Keerthana S, Sumangala K M, Laxmi Angadi, T Kavya. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.