A Study on Autism Detection from Facial Images

Abstract

Autism is a serious type of neurodevelopmental condition that disrupts cognitive functioning, language, and social behavior. Individuals with autism spectrum disorders have a broad range of intellectual functioning, from significant disability to outstanding abilities. The severity and long-term repercussions of ASD can be avoided with an early diagnosis. Medical professionals currently use a number of methods to predict autism, including brain scan analysis, autism diagnostic interviews, autism diagnostic observations, and physical facial trait analysis. These traditional ways of diagnosing autism are quite time-consuming, expensive, and complicated. People with autism have a distinctive set of facial traits which distinguish them from normal ones. One of the most interesting areas of autism study is the application of facial traits as a physical indicator for autism diagnosis. The use of deep learning and machine learning has become increasingly widespread in recent years, particularly in the field of image classification. These algorithms are capable of identifying hidden autism patterns from vast amounts of facial data, making them useful as autism predictor. Hence, this study reviews the various autism prediction methods based on facial features using deep learning and machine learning techniques.

Introduction

I. INTRODUCTION

An individual with autism spectrum disorder (ASD)may experience challenges with cognition and difficulties interacting with others. Those with ASD might be from any ethnicity, race, or socioeconomic status. Symptoms can vary greatly in range and severity. This disease can cause compulsive behaviors, obsessive interests, and communication issues in the sufferer. Even though there is no known cure for autism, many kids can benefit greatly from early treatment. For autistic patients to acquire the mental abilities necessary for social and interpersonal communication, they require specialized treatment [1].

The conventional autism detection methods involve physical identification by doctors and conducting   Autism Diagnostic Interview-Revised (ADI-R) and Autism Diagnostic Observation Schedule Revised (ADOS-R) tests. However, these are tedious processes requiring a lot of time and patience [2]. Consequently, the longer it takes the patient to get the right diagnosis and medication, the less likely their symptoms will be minimized. In addition, the expenses linked with receiving a diagnosis can significantly strain families. Compared to children without autism, autistic children exhibit a few distinct facial traits. Based on a study conducted by scientists at the University of Missouri [3], autistic children have broad upper faces, wide eyes, short noses, narrow cheeks, etc. The use of machine learning models and deep learning can help reduce the need for time-consuming and expensive assessments to some extent by automatically extracting autism-related unique face features and classifying them as autism or not. Therefore the main objective of the study is to find out the different deep learning and machine learning techniques of autism detection by extracting autism related facial features.

II. REVIEW ON EXISTING APPROACHES

Autism spectrum disorder (ASD) cannot be diagnosed with a blood test or other medical test, making diagnosis challenging [4].  Technological advancements are enabling researchers to identify autism in new ways. Autism detection can be performed by analyzing structural brain images, face images, and behavioral patterns. Brain image-based detection methods can provide a more precise diagnosis of ASD, however, image acquisition is costly and requires a lot of time[5]. Behavioural-based approaches also require a great deal of time and preparation. Face image-based approaches are simple, economical, and require less time, so no discomfort is caused to patients. This study focuses on various face-based autism detection methods that have been recently introduced by researchers.

According to the findings in [6], facial morphology is a useful biomarker for distinguishing different categories of autism spectrum disorders (ASD). A machine learning-based approach for autism detection from thermal face images has been proposed

in [7]. Here, the autism-related facial features are extracted using GLCM (Gray-Level Co-occurrence Matrix).GLCM is a statistical technique for textural feature extraction in image processing that exhibits the correlation between two nearby pixels with respect to their grey level, distance, entropy, homogeneity, and orientation. After feature extraction, the classification stage uses a support vector Machine Classifier. Thermal images have been used to determine the face's temperature at various emotional states. The fuzzy C-means technique has been used for thermal image segmentation.

In [8], an autism classification model that combines transfer learning procedures with deep learning has been proposed. The convolution neural network classifier uses facial traits in images to detect the early signs of autism in children. A CNN is a deep learning approach that can recognize different objects and attributes in an input image and distinguish between them by assigning weights and biases that can be learned. Using a model that was developed and trained for a specific task as a base for another is known as transfer learning. It involves the use of a pre-trained model. VGG 19(Visual Geometry Group) has been used to extract and classify facial features for autism prediction automatically.

Five classification algorithms (K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Gaussian Naïve Bayes (NB), Neural Networks (NN), and Random Forests (RF)) have been used for autism detection in [9]. Here, two-dimensional digital facial photos have been used to train each prediction model. This paper also introduces an image acquisition methodology. Multiple frontal images have been captured for each individual, and the best image based on some criteria has been selected for feature extraction. A bilateral filter has been used for noise removal from images and a Canny detector for edge detection. Sixty-eight facial landmarks have been obtained using a Convolutional Neural Network (CNN), Histogram of Oriented Gradients (HOG, Deep Neural Network (DNN), and Haar cascade algorithm. The extracted features are fed to different classifiers for autism prediction and performance comparison.

By implementing the VGG16-based transfer learning approach on a special ASD dataset of children with clinical diagnoses, a facial image-based ASD screening solution has been presented in [10]. The dataset consists of facial images of children in an age group of 2-12 years. Unlike other methods, an additional dataset comprising facial images of white children and children of various colors has been used to emphasize racial characteristics' influence on the development of facial-image-based prediction systems. The automatic facial feature extraction and classification have been done by VGG16 in which 70% of base layers are frozen.

Xception, VGG19, and NASNETMobile were the three pre-trained CNN models used for ASD prediction in [8]. With these CNN models, important facial features are retrieved and classified into autistic or non-autistic groups from face images. The ImageNet dataset has been applied to train the Xception model. A 19-layer variation of the VGG model is called VGG19. NASNetMobile is the latest pre-trained model proposed by the Google brain team. Every model underwent training using an openly accessible dataset over the Internet.

A hybrid method for detecting ASD has been presented in [11], which combines XGBoost and RF algorithms with three different CNN models. Overfitting is a common problem with deep learning models, particularly when there is a shortage of training data. By combining XGBoost and RF algorithms, this problem is lessened. CNNs are made up of several layers, each of which has a distinct function related to feature extraction and categorization. At first, the Gaussian filter is used to eliminate noise and distortion from the facial images. Then, feature map extraction, face trait analysis, and eye monitoring were done using VGG16, ResNet101, and MobileNet. The feature vectors that were obtained have dimensions of 2940 × 670, 2940 × 715, and 2940 × 610. In order to categorize and differentiate between people with ASD and TD, these feature vectors were then fed into the XGBoost and RF algorithms.

A different method for identifying autism based on distinctive features of the face is given in [12]. ASD has been determined by measuring the distances between important facial points (eyes, lips, nose, mouth, etc.). This paper also introduces a way to acquire object shape characteristics from binary images. Then Naive Bayes Classifier was used for the classification task and its performance was compared with SVM and KNN classifier. Naïve Classifier is a basic machine-learning technique that relies on the Bayes Theorem. Images of children with autism were collected from Google Images.

III. RESULTS

Table 1. Comparison of various autism detection methods

Most of the autism prediction methods reviewed were based on convolutional neural networks. The accuracy of detection in the GLCM and SVM-based approach [7] was 89.22%. 40 thermal photographs of autistic and normal subjects are used to train and test GLCM and SVM. The approach proved more efficient and effective than the K means classification. The K means classification approach obtained a maximum accuracy of 73%. A training accuracy of 96% and a testing accuracy of 84% has been obtained in the VGG-based model for autism prediction [8]. CNN uses the pre-trained VGG19 version of ImageNet, the ReLU actuation function, the Adam Optimizer, and a 33-epoch categorical cross-entropy loss function.

In [9], five classifiers ( (K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Gaussian Naïve Bayes (NB), Neural Networks (NN), and Random Forests (RF) ) were used for autism prediction, where SVM classifier gives highest accuracy of 86.2%  and highest precision of 90.8%.The least accurate was the Random Forest classifier. 882 training images and 232 validation images, equally divided between ASD and non-ASD categories, have been used in the VGG 16-based transfer learning strategy for autism identification. 95% classification accuracy and an F1 score of 0.95 was achieved by a deep learning model for the East Asian dataset. Here, the inaccurate classification arises from the fact that the normal face anthropometric metrics of one race may coincide with the aberrant dimensions of another race. The Xception model has the highest classification accuracy, while NASNETMobile achieves the lowest, among the three pre-trained deep learning algorithms used for ASD detection (Xception, VGG19, and NASNETMobile) in [10]. For the purpose of detecting ASD, XGBoost and RF algorithms were combined with features obtained from CNN models in [11]. In particular, an RF that made use of the VGG16-MobileNet attributes showed a remarkable 99.25% AUC, 98.8% accuracy, and 98.9% precision. Here, the strategy of combining the features of two CNN models was able to negotiate the restrictions of feature extraction using a single CNN model. On comparing the results of the Naive Bayes Classifier with SVM and KNN, the Naive Bayes classifier was found to be the best model for autism detection in [12].

Conclusion

A study on different methods to detect autism from face images has been presented in this paper. ASD is a neurodevelopmental impairment because of variations in a child\'s brain. Individuals with ASD may exhibit distinct behaviors, interactions, and learning styles than other people. Compared to other autism detection methods (behavioral pattern analysis method, neuroimaging method), face-based approaches are simple, economical, and take only less amount poof time, Most of the recently developed facial image-based approaches are based on pre-trained CNN models. This is because CNN can perform automatic feature extraction and classification on large amounts of data with high accuracy. Out of the approaches reviewed, RF combined with the VGG16-Mobilenet prediction model gives the highest accuracy in ASD detection. Nevertheless, almost all of the techniques are limited to the identification of autism in youngsters. Most toddlers with autism receive a diagnosis, but adults may not receive one. Adult-specific diagnostic techniques do not yet exist. Observations and discussions with autistic adults by medical professionals are the main sources of diagnostic information for adults. Therefore prediction models that can be used for autism detection of individuals of all ages should be developed. Additionally, in order to overcome the disparities in face anthropometrics between races, race-specific prediction models based on 2D facial photos should be developed in order to increase their accuracy and reliability A. Conflict of Interest The authors have no conflicts of interest to declare.

References

[1] https://www.cdc.gov/ncbddd/autism/signs.html [2] https://munewsarchives.missouri.edu/news-releases/2015/0114-advanced-3-d-facial-imaging-may-aid-in-early-detection-of-autism [3] https://www.cbsnews.com/pictures/is-it-autism-facial-features-that-show-disorder [4] https://www.cdc.gov/ncbddd/autism/screening.html [5] Alam MS, Rashid MM, Faizabadi AR, Mohd Zaki HF, Alam TE, Ali MS, Gupta KD, Ahsan MM. Efficient Deep Learning-Based Data-Centric Approach for Autism Spectrum Disorder Diagnosis from Facial Images Using Explainable AI. Technologies. 2023; 11(5):115. https://doi.org/10.3390/technologies11050115 [6] Tamilarasi, F. C., & Shanmugarn, J. (2020, August). Evaluation of autism classification using machine learning techniques. In 2020 Third International Conference on Smart Systems and Inventive Technology (ICSSIT) (pp. 757-761). IEEE. [7] Jahanara, S., & Padmanabhan, S. (2021). Detecting autism from facial image. International Journal of Advance Research, Ideas and Innovations in Technology, 7(2), 219-225. [8] Michelassi, G. C., Bortoletti, H. S., Pinheiro, T. D., Nobayashi, T., Barros, F. R., Testa, R. L., ... & Machado-Lima, A. (2021). Classification of Facial Images to Assist in the Diagnosis of Autism Spectrum Disorder. [9] Lu, A., & Perkowski, M. (2021). Deep learning approach for screening autism spectrum disorder in children with facial images and analysis of ethnoracial factors in model development and application. Brain Sciences, 11(11), 1446. [10] Alsaade, F. W., & Alzahrani, M. S. (2022). Classification and detection of autism spectrum disorder based on deep learning algorithms. Computational Intelligence and Neuroscience, 2022. [11] Awaji, B., Senan, E. M., Olayah, F., Alshari, E. A., Alsulami, M., Abosaq, H. A., ... & Janrao, P. (2023). Hybrid Techniques of Facial Feature Image Analysis for Early Detection of Autism Spectrum Disorder Based on Combined CNN Features. Diagnostics, 13(18), 2948. [12] F. Catherine Tamilarasi, F. Catherine Tamilarasi, Dr. J. Shanmugam. (2020). Child autism detection based on facial feature classification, International Journal of Advanced Research in Engineering and Technology (IJARET) Volume 11, Issue 10, October 2020, pp. 468-475 [13] Al Banna, M.H.; Ghosh, T.; Taher, K.A.; Kaiser, M.S.; Mahmud, M. A monitoring system for patients of autism spectrum disorder using artificial intelligence. In Proceedings of the Brain Informatics: 13th International Conference, BI 2020, Padua, Italy, 19 September 2020; Proceedings 13. Springer: Cham, Switzerland, 2020; pp. 251–262. [14] Perinelli, M.G.; Cloherty, M. Identification of autism in cognitively able adults with epilepsy: A narrative review and discussion of available screening and diagnostic tools. Seizure 2023, 104, 6–11. [15] A. Abraham, M. P. Milham, A. Di Martino, R. C. Craddock, D. Samaras, B. Thirio et al., \"Deriving reproducible biomarkers from multi-site resting-state data: an autism-based example\", NeuroImage, vol. 147, pp. 736-745, 2017 [16] Patrick Dwyer, Buyun Xu, James W. Tanaka Patrick Dwyer, Buyun Xu, James W. Tanaka Patrick Dwyer, Buyun Xu, James W. Tanaka,” Investigating the perception of face identity in adults on the autism spectrum using behavioural and electrophysiological measures”, https://doi.org/10.1016/j.visres.2018.02.013. [17] Yongqiang Li, Shangfei Wang, Member, IEEE, Yongping Zhao, and Qiang Ji, Senior Member, IEEE,” Simultaneous Facial Feature Tracking and Facial Expression Recognition”, IEEE Transactions on Image Processing, VOL. 22, NO. 7, JULY 2013, Digital Object Identifier 10.1109/TIP.2013.2253477. [18] Uddin, M.J.; Ahamad, M.M.; Sarker, P.K.; Aktar, S.; Alotaibi, N.; Alyami, S.A.; Kabir, M.A.; Moni, M.A. An Integrated Statistical and Clinically Applicable Machine Learning Framework for the Detection of Autism Spectrum Disorder. Computers 2023, 12, 92. [19] Alalayah, K.M.; Senan, E.M.; Atlam, H.F.; Ahmed, I.A.; Shatnawi, H.S.A. Automatic and Early Detection of Parkinson’s Disease by Analyzing Acoustic Signals Using Classification Algorithms Based on Recursive Feature Elimination Method. Diagnostics 2023, 13, 1924. [20] Deng, L.; Rattadilok, P. A Sensor and Machine Learning-Based Sensory Management Recommendation System for Children with Autism Spectrum Disorders. Sensors 2022, 22, 5803.

Copyright

Copyright © 2023 Asha Sukumaran, Munavar Jasim K. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.