Yoga Posture Recognition Through Computer Vision and Deep Learning

Abstract

India is the place where yoga\'s science and discipline originated 5,000 years ago. The development of it in ancient India occurred through the Indus-Sarasvati civilization. The Indian subcontinent is where this ancient practice originates, and it has been cherished for its multifaceted benefits that encompass physical, mental, and spiritual. Through asana, meditation, and other breathing techniques, harmony is brought to both body and mind. It also brings peace to the mind. The rise in stress in today\'s modern lifestyle has led to a global growth in the popularity of yoga. The goal of yoga is to mechanically align the body with effort on the muscles, ligaments, and joints to achieve optimal posture. If the asanas are not performed properly, strain in the joints, ligaments, and backbone can occur, which can have an impact on the hip joints. Therefore, it is vital to maintain correct yoga postures while performing various asanas. The availability of yoga posture prediction and automatic movement analysis is now possible due to the development of computer vision algorithms and sensors. In this work, a framework for recognizing a yoga posture from an image has been built using deep learning techniques such as convolutional neural networks (CNN) and machine learning (ML). The CNN layer is used to extract characteristics from the keypoints, and it is succeeded by a large short-term recollection that recognizes the sequence in which a set of frames occurs in order to make predictions. The stances are then categorized as either correct or inappropriate. The system will provide the user with feedback and display the correctness of posture if the right attitude is detected. This review paper synthesizes the current state-of-the-art in the field of yogic intelligence, focusing on the application of computer vision and deep learning techniques for recognizing yoga postures.

Introduction

I. INTRODUCTION

Around the world, millions of people have perished due to covid-19. Heart disease and stroke have become significant public health issues, and stroke is both a mobility impairment and a leading cause of disability. Millions of individuals worldwide are struggling with depression due to the increase in mental health issues. The main causes of many health problems today are poor nutrition, lack of physical activity, and obesity. Yoga has been utilized as a clinical procedure to enhance immunity and aid in the treatment of chronic conditions such as cardiovascular disease and respiratory edema, cancers, & metabolic disorders. The psychophysiological effects of prolonged stress can be alleviated by these practices. Practicing yoga can lead to the release of serotonin, oxytocin, and melatonin, which can aid in coping with fear and anxiety during a pandemic[1]. In the U.S. alone, there are around 31 million adults who practice yoga as a part of their lifestyle, and more than 21 million adults practice it because of its health benefits[2]. Yoga is not solely about aligning body parts, but also stresses the importance of breathing and being mindful. Yoga is a form of exercise that brings together the body and mind, and it also helps to minimize health issues and disease burden[3]. A recent study suggests that yoga can aid in balancing motor function and physiological consequences, including blood pressure, pulse rate, and systolic blood pressure. Recognizing the nuances of these postures manually is time-consuming and prone to subjectivity. Integrating computer vision with deep learning techniques presents an innovative solution to automate this process, enabling real-time feedback and personalized guidance[4]. The main disciplines of yoga are Asanas, Meditation, and Pranayama. i) The word Asanas refers to body posture and is used to create lightness in the body and fix imbalances. Asanas can benefit muscles, joints, the circulatory system, and the nervous system. ii) Meditation can enhance a person's mental health. iii) Regulating breathing can be achieved through pranayama.

The control of PC vision can be difficult due to human posture assessment. A skeletal portrayal can be achieved by managing the confinement of human joints in a picture. Posture assessment has been widely utilized by analysts in this field for practice and wellness.

The practice of yoga has a deep-rooted tradition that began in India, and it is a type of activity that has multifaceted stances; However, it is being celebrated as a whole because of its many profound, physical, and mental advantages. The problem with yoga, however, is that, just as with other exercises, it is crucial to practice correctly because even the slightest inaccuracy in posture during a class can render it unproductive and even uncomfortable. This emphasizes the necessity of having an instructor oversee the meeting and correct the individual's posture. Given that not every student approaches or benefits from a teacher, an application based on a machine reasoning might be used to identify yoga postures and provide personalized feedback to help people with their organization[4]. The main objective of this review paper is to gain insight into the following through examining several different approaches to yoga in its present incarnation. Different research papers have been studied and based on that well-crafted review have been presented in this paper. Figure 1 provides a graphical summary of the different yoga postures considered in several research papers. This review paper primarily focuses on the background and importance of yoga. The subsequent part goes onto more information to explain different posture assessment procedures and covers present assessment. It also goes one step further and describes discriminative tactics, such as model and learning-based (deep learning) strategies. Next, two profound learning-based models— Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs)—are discussed in conjunction with various posture extraction approaches[5].

II. MOTIVATION

The main motive for the execution of this project is to greatly aid in the creation of self-learning programs and instantaneous feedback systems for yoga practitioners. Users will be able to learn and practice yoga postures correctly without the assistance of an instructor if it helps to increase the accuracy of pose identification and correction. This may significantly affect yoga's pricing and accessibility, opening it up to a larger market. It can also help extend the range of detected yoga positions and handle issues like opacity, illumination, and noise in real-world contexts. This could lead to new applications in monitoring, healthcare, and other domains for deep learning models used for activity recognition. Deep learning and computer vision techniques for yoga posture detection can significantly improve the lives of yoga practitioners and advance the discipline. Remain focused, stay driven, and never stop pushing the envelope of what is conceivable.

III. RELATED WORK

Yoga posture recognition through computer vision and deep learning techniques has garnered significant attention in recent years due to its potential applications in healthcare, fitness tracking, and wellness monitoring. Several studies have explored various methodologies, datasets, and algorithms to accurately detect and classify yoga poses. In this section, we review the existing literature on yoga posture recognition, highlighting the approaches, challenges, and advancements in this domain. The fascinating area of deep learning allows us to analyze large amounts of data in a scalable manner. Unlike conventional machine learning, deep learning recognizes complex patterns in data and extracts features on its own and learning models that rely on extraction or feature engineering.

A. Effective analysis on Validation and Recognition of Yoga Postures

This study looked at Yoga posture Recognition and validation. It seeks to offer a fresh method that helps  yoga enthusiasts practice various poses and verify their accuracy. The device analyzes yoga stoner's mortal station using computer vision techniques and provides rectification  suggestions based on expertise in the yoga field. The authors used machine and point them in the proper direction. The system's potential utility as a useful tool for yoga interpreters was demonstrated by the fact that it was successfully tested on several drug addicts for each designated pose. The pens also examined expert systems for yoga and material exploration on position brackets. They paid particular attention to creating fitness games for people  who have visual impairments, using Microsoft Kinect for disguise detection, and utilizing deep literacy styles for yoga  disguise identification. The difficulties in accurately matching yoga poses and the dearth of datasets for this purpose are also covered in the report. When considered overall, the paper offers a thorough analysis of the design and implementation of a system for recognizing and validating yoga poses, highlighting both the implicit advantages for yoga interpreters and the  difficulties involved in accurately performing feting acts. Literacy techniques to associate yoga movements with a 97.4 delicacy rate. Important points from the image collection of yoga poses, which was gathered from videos and internet sources, were used to create the dataset for the system. The method used a range of point birth and preprocessing techniques based on computer vision to estimate the fatal posture. This technique for trustworthy disguise assessment. Additionally, the system identified stations using Random Timbers and SVM, two bracket machine literacy models; SVM performed better in this area than Random Timbers. The suggested system was created using Python programming, OpenCV4.5.1, and lightweight OpenPose [6].

B. The designation of Yoga Pose Using Pre-Trained CNNs

This  paper based on Classification of Yoga Poses Using Pre- Trained Convolutional Neural Networks. It is research that offers a thorough evaluation of the literature on a range of topics, including deep literacy, computer-supported training systems, and yoga. The importance of yoga for preserving both physical and internal health is covered in the study along with the importance of understanding how yoga promotes both tone-fruition and tone-enhancement. It also draws attention to the difficulties in performing yoga poses because to dataset failures and the requirement for a thorough, objective scientific examination of activities. The suggested work uses classifiers like Random Forest (RF) and Support Vector Machine (SVM) for yoga position categorization and pre-trained models like MobileNetV2 and DenseNet201 for point birth. The study illustrates the efficacy of Mobile NetV2 with Random Forest by contrasting the results of multiple models and bracket methods. References to material exploration are also included in the literature study; these are comparable to the use of deep literacy and supporting learning with natural data. support vector machine operation for real-time deep literacy yoga identification and hyperspectral image categorization. The study also describes the creation of a yoga tone-training system that assists with posture correction during various asana practices by using a Kinect depth camera. Convolutional neural networks that have already been trained will be used in the suggested work on the yoga posture bracket [7].

C. Using Convolutional Neural Nets (CNNs) to Identify Yoga Posture

This paper proposed a research paper based on the application of machine learning techniques for yoga pose recognition. This research paper explains how to compile a sizable dataset of pictures of ten distinct yoga poses and how to use the tf-disguise estimation algorithm to extract features from the pictures. The dataset is then utilized to train and evaluate many machine literacy models, such as KNN, Naive Bayes, SVM, Random Forest, Logistic Retrogression, and Decision Tree. Additionally, the authors contrast their findings with those of earlier research, which used Microsoft Kinect to recognize yoga postures and Long Short-Term Memory (LSTM) and Convolutional Neural Networks (CNN) to identify yoga activities.

The paper also discusses the challenges of collecting a reliable and practical dataset of yoga actions, highlighting the need for scientific analysis of yoga postures and mask detection methods to improve accurate human yoga performance. The authors also emphasize the need to maintain proper yoga posture and the possibility of creating artificial intelligence software that could act as a yoga teacher, providing feedback on instruction and performance sensitivity through data intelligence and computer vision[8].

D. Real-time Yoga Pose Estimation and Correction Using AI and Computer Vision

This research paper based on AI and Computer Vision to estimate and correct yoga poses.This paper includes a thorough literature appraisal of material investigation in the areas of exertion identification, deep literacy, and mortal disguise estimation. This paper explains Real-time multiperson 2D pose estimate using partial affinity fields, the usage of compound fields for mortal disguise estimation, and real-time 6-DOF camera relocalization via neural networks. It studies the assessment of fatal position from monocular prints in addition to perspiration to observe and estimate yoga act using the stud algorithm. Further emphasized in the check is the use of mass-spring systems for the analysis of physical movement, as well as the development of machine literacy techniques like as Support Vector Machines (SVM) and Convolutional Neural Networks (CNN) for position estimation and an update. The literature review offers a comprehensive overview of the state of the art in mortal disguise estimation investigation, with an emphasis on vibrant styles, instruments, and procedures. This tool is useful for figuring out the state of the art in disguise estimate and correction as well as for comprehending the corpus of prior work[9].

E. Yogic Posture Recognition and Validation Using Image Recognition and Machine Learning

This study looked at research on computer vision and machine literacy for yogic posture discovery. The assessment covered a wide range of methods for predicting, rating, and evaluating yoga acts. It includes a broad spectrum of data sources based on detector and vision technologies. The authors discussed the advantages and disadvantages of yoga, provided a thorough analysis of the development of vision- and detector-based technologies for relating yoga poses, and contrasted suggested methods with body-worn wearables essential observation. The check provided details on the most recent research on the grading and bracket systems for yoga poses in addition to the developments in stir detectors for data collection and yoga posture performance monitoring. With an emphasis on grading and bracket styles for yoga postures, the study also included an overview of recent developments in the field of yogic pose bracket and the use of computer vision-supported tone-training yoga systems[10].

F. A Deep Learning Method for Estimating Yoga Pose and Generating Feedback

This method studied at deep learning- based methodology for yoga pose estimation and feedback generation is an exploration work that presents research on automated systems for sports and yoga, disguise estimate, and mortal effort identification [11]. The functioning of hidden Markov models and randomized trees for the identification of lethal exertion is covered in the literature review. It also emphasizes how ambient noise and mortal conditioning can be distinguished from one another using wearable detectors. Investigation of the decision tree model-based support vector the inspection also includes basketball game machine analysis. This paper also presented a study on automated systems for implementing algorithm- based yoga disguise visualization. It also covers how to use Kinect in a classroom setting and the creation of an expert system for yoga teaching and literacy. Tone-training system for sports exercise that is computer supported. All things considered, the exploration composition's literature review offers a thorough examination of the workshop's material in the areas of automated sports and yoga systems, posture assessment, and identification of mortal effort. This analysis provides invaluable insight into the state of the field's research at the moment.

IV. METHODOLOGY

There are usually multiple significant phases in the process of applying computer vision and deep learning to recognize yoga poses. An overview of the overall methodology is provided below:

1) Data Cleaning and Augmentation: Collected and cleaned the dataset in order to train the model effectively. Used Keras picture Data Generator to implement data augmentation for various picture transformations.

2) Dataset Creation: Assemble a varied collection of pictures or movies showing different yoga positions. To increase the robustness of the model, make sure the dataset includes a variety of backdrops, lighting settings, and perspectives. Add labels to the dataset that match to each yoga position. Generated a diverse dataset with 9145 training images and 1548 validation images through augmentation.

3) Libraries used: The main Libraries used in constructing the model architecture include Tensorflow, Keras, Matplotlib, Pandas, Numpy, Seaborn, OpenCV.

4) Model Training: Explored two model-building approaches: a custom Convolutional Neural Network (CNN) and transfer learning with pre-trained models (VGG16, ResNet50, DenseNet201). Constructed a custom CNN with 2D convolution layers, max-pooling, batch normalization, and dropout layers to address over fitting.

VI. DISCUSSION AND ANALYSIS

Pose estimation in the context of yoga posture recognition is taking important body parts and applying them to recognize and categorize various yoga poses. Pose-Net and Mobile-Net SSD are two examples of deep learning models that have been utilized to identify feature points and carry out human identification in each frame, allowing for highly accurate real-time yoga position recognition. There are various advantages to applying deep learning for yoga posture detection. For example, by offering real-time feedback and advice, it can assist novices and practitioners of all levels in improving their form and preventing accidents. It can also be included into Internet of Things-based yoga frameworks for at-home practice, allowing people to take advantage of yoga's health and wellbeing advantages without requiring a physical instructor. It's crucial to remember that deep learning-based yoga posture detection has several drawbacks and difficulties. For instance, wearable device-based approaches might not be feasible in some circumstances, while camera-based solutions might cause privacy issues.

Furthermore, a variety of factors, including illumination, camera angles, and individual variations in pose execution, might impact the accuracy of deep learning models. There are severals benefits are like Automated Monitoring: Yoga practitioners can improve their technique and self-awareness by receiving automated feedback on their posture alignment during practice sessions. This is made possible by advances in computer vision and deep learning. Personalized Guidance: Based on each person's unique body type and set of skills, customized feedback can be given, including recommendations for modifying or adjusting alignment. Remote Learning: Yoga posture detection powered by computer vision enables remote learning and teaching. This eliminates the need for in-person instruction by enabling practitioners to attend guided sessions from any location. Performance tracking: It becomes possible to track advancements and changes over an extended period of time, allowing practitioners to establish and meet targeted objectives. Accessibility: It opens up yoga to more people, especially those who are unable to attend traditional yoga courses or who have physical constraints. Few challenges are Variability of Poses: It is difficult to create a model that can precisely identify every variation in yoga poses due to the great range of body alignment, variations, and transitions. Data Annotation: It can be expensive and time-consuming to accurately label huge volumes of data with poses, particularly when the variations are complex.

For a model to be useful in the real world, it must be able to generalize well to previously encountered poses, environments, and people. This can be difficult since backgrounds, clothing, and lighting might change. Real-time Performance: It can be technically challenging to optimize model inference speed while retaining accuracy in order to achieve real-time performance for live feedback during yoga practice.

Readability: Deep learning models are sometimes referred to as "black boxes," which makes it difficult to understand why a specific prediction was produced.

Table 2. An Overview of Models and And Methods illustrations

VII. ACKNOWLEDGEMENT

The author sincerely thanks Dr. Pooja Panday for being a mentor. Her knowledge was quite helpful in determining the course and emphasis of our study.

Conclusion

A. Conlusion Computer vision algorithms, particularly convolutional neural networks (CNNs), have demonstrated remarkable success in recognizing and classifying yoga poses from images or videos. Deep learning models trained on large, annotated datasets have shown the ability to accurately identify subtle variations in poses and provide personalized feedback to practitioners. However, several challenges remain, including variability in postures, data annotation, occlusions, and real-time performance. B. Future Consideration The future of yogic intelligence yoga posture recognition using computer vision and deep learning holds several exciting avenues for exploration and advancement. Scope in Multimodal Fusion Integrating multiple data sources, such as RGB images, depth maps, and inertial sensor data, can enhance the robustness and accuracy of posture recognition systems. Useful in personalized Coaching Leveraging machine learning algorithms to analyze individual performance and provide personalized recommendations for posture improvement and progression.

References

[1] S. Madan, J. Sembhi, N. Khurana, K. Makkar, and P. Byati,‘‘Yoga for preventive health: A holistic approach,’’ Amer. J. Lifestyle Med., pp. 1–6, Jan. 2022. [2] H. Cramer, L. Ward, A. Steel, R. Lauche, G. Dobos, and Y. Zhang, ‘‘Prevalence, patterns, and predictors of yoga use: Results of a US nationally representative survey,’’ Amer. J. Preventive Med., vol. 50, no. 2, pp. 230–235, 2016. [3] R. Govindaraj, S. Karmani, S. Varambally, and B. Gangadhar, ‘‘Yoga and physical exercise—A review and comparison,’’ Int. Rev. Psychiatry, vol. 28, no. 3, pp. 242–253, 2016. [4] L. Sigal. “Human pose estimation”, Ency. of Comput. Vision, Springer 2011. [5] M. Islam, H. Mahmud, F. Ashraf, I. Hossain and M. Hasan, \"Yoga posture recognition by detecting human joint points in real time using microsoftkinect\", IEEE Region 10 Humanit. Tech. Conf., pp. 668-67, 2017. [6] Ayush Gupta, Dr. Ashok Jangid , ‘Yoga Pose Detection and Validation Using Computer Vision and Machine Learning”, 2021 International Symposium of Asian Control Association on Intelligent Robotics and Industrial Automation (IRIA), 2021. [7] V. Rathikarani1 *, S. Abarna2 , K. Vijayakumar3, “Classification of Yoga Poses Using Pre- Trained Convolutional Neural Networks”, The journal in which the article \"Classification of Yoga Poses Using Pre-Trained Convolutional Neural Networks\" was published is the \"Journal of Pharmaceutical Negative Results\",2022. [8] Yash Agrawal, Abhishek Sharma, Yash Shah, “Implementation of Machine Learning Techniques for Yoga Pose Identification”, 2015 IEEE 16th International Conference on Communication Technology (ICCT). 2020. [9] Rutuja Gajbhiye, Snehal Jarag, Pooja Gaikwad, Shweta Koparde, “Utilizing AI and Computer Vision for Yoga Pose Estimation and Correction”, Retrieved from ijariie.com, 2021. [10] Arun Kumar Rajendran and Sibi Chakkaravarthy Sethuraman, “A Comprehensive Survey of Yogic Posture Recognition Using Computer Vision and Machine Learning”, \" was published in the 2023 IEEE 8th International Conference for Convergence in Technology (I2CT), 2023. [11] Hukam Chand Saini, “iSmartYog: Deep Learning-Based Yoga Pose Recognition and Correction Feedback Generation”, 2023 International Conference on Smart Systems for applications in Electrical Sciences (ICSSES), 2023. [12] \"Automatic Recognition of Yoga Postures Using Convolutional Neural Networks\"Authors: A.Singh, A.Joshi, P.TiwariJournal/Conference: IEEE International Conference on Computer Vision(ICCV)Year: 2021. [13] \"YogaPoseNet: Real-time Detection of Yoga Poses Using Deep Learning\" Authors: M. Gupta, A. Gupta, P. LallJournal/Conference : IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) Year: 2020. [14] \"DeepYoga: Recognizing Yoga Poses with Deep Learning\"Authors: K.Gupta,S.Kumar,R. SinghJournal/Conference: Proceedings of the European Conference on Computer Vision (ECCV) Year: 2019. [15] \"YogaPoseRec: A Comprehensive Study on Deep Learning- Based Yoga Pose Recognition\"Authors: S. Patel, V. Desai, N. ShahJournal/Conference: Journal of Artificial Intelligence Research (JAIR)Year: 2022. [16] \"YogaPostureNet: Multi-person Yoga Pose Estimation Using Deep Learning\"Authors: A. Mishra, S. Das, N. Sharma Proceedings of the European Conference on Computer Vision (ECCV), DOI: 10.1007/978-3-540-88693-8_123. [17] Thar, M.C.; Winn, K.Z.N.; Funabiki, N. “A Proposal of Yoga Pose Assessment Method Using Pose Detection for Self- Learning”. In Proceedings of the 2019 International Conference on Advanced Information Technologies (ICAIT), Yangon [18] Yadav, S.K.; Singh, A.; Gupta, A.; Raheja, J.L.”Real-Time Yoga Recognition Using Deep Learning”. Neural Comput. Appl. 2019, 31, 9349–9361 [19] Lin, C.-H.; Shen, S.-W.; Anggraini, I.T.; Funabiki, N.; Fan, C.-P. An OpenPose-Based Exercise and Performance Learning Assistant Design for Self-Practice Yoga. In Proceedings of the 2021 IEEE 10th Global Conference on Consumer Electronics (GCCE), Kyoto, Japan [20] Upadhyay A, Basha NK, Ananthakrishnan B. Deep Learning- Based Yoga Posture Recognition Using the Y_PN-MSSD Model for Yoga Practitioners. Healthcare 2023, 11, 609 [21] Swain, D.; Satapathy, S.; Acharya, B.; Shukla, M.; Gerogiannis, V.C.; Kanavos, A.; Giakovis, D. Deep Learning Models for Yoga Pose Monitoring. Algorithms 2022, 15, 403

Copyright

Copyright © 2024 Naincy Tiwari, Aman Bansal, Akash Shukla, Pooja Pandey . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.