Yoga Pose Detection and Feedback Generation

Abstract

This study presents an innovative system using computer vision and machine learning for real-time yoga pose detection and feedback. Leveraging advanced algorithms like MediaPipe, OpenPose, and CNNs, it analyzes users\' postures through a camera, comparing them with standard poses to provide personalized alignment corrections. This enhances yoga practice by minimizing injuries, improving posture accuracy, and supporting fitness goals. Designed for all skill levels, the system integrates with yoga apps, smart mirrors, and wearables, offering features like progress tracking and adaptive feedback. It demonstrates potential in making yoga safer and more engaging. Future developments could include wearable integration, injury prevention, and customized yoga plans, advancing AI-driven fitness solutions.

Introduction

I. INTRODUCTION

Yoga, a centuries-old practice renowned for its physical, mental, and spiritual benefits, requires precision in posture to achieve optimal results. Incorrect alignment during yoga poses can lead to reduced effectiveness or even injuries, highlighting the need for accurate guidance, especially for those practicing without a trained instructor. In response to this challenge, the Yoga Pose Detection and Feedback Generation System offers a cutting-edge solution by utilizing the latest advancements in computer vision and machine learning. This system captures a user’s movements in real time using a camera, analyzes their posture, and compares it to a database of predefined yoga poses. By leveraging advanced pose estimation techniques, including Convolutional Neural Networks (CNNs) and frameworks such as Mediapipe and Open Pose, the system identifies key body points like joints and limbs, enabling precise posture evaluation. When deviations from optimal alignment are detected, the system provides immediate feedback, helping users adjust their poses and avoid potential injuries. The Yoga Pose Detection System is designed to benefit a diverse range of users. For beginners, it offers tailored instructions to help them learn and execute poses correctly. Intermediate and advanced practitioners can use the system to refine their practice and challenge themselves with more complex postures. The system also supports rehabilitation and therapeutic use cases, where precise movement guidance is critical. Furthermore, its compatibility with smart mirrors, wearable fitness devices, and yoga apps ensures flexibility and accessibility in various contexts.

What sets this system apart is its ability to deliver a personalized, interactive experience. Visual and auditory feedback mechanisms guide users in a hands-free manner, while progress tracking and adaptive recommendations enhance the overall practice. Over time, the system becomes more accurate and responsive, accommodating different body shapes, levels of flexibility, and user preferences.

By integrating cutting-edge AI capabilities with health and fitness practices, the Yoga Pose Detection and Feedback Generation System aims to make yoga safer, more engaging, and universally accessible. Whether used at home, in fitness centers, or for therapeutic purposes, this technology represents a transformative step in merging traditional practices with modern innovation, empowering users to achieve their fitness and wellness goals effectively.

II. MOTIVATION AND OBJECTIVES

A. Motivation

Yoga is a practice that harmonizes physical, mental, and emotional well-being. However, achieving its full benefits requires accurate posture and alignment, which can be challenging without expert supervision. With the growing popularity of home-based fitness routines, especially after the global shift toward remote and online lifestyles, many yoga practitioners lack access to real-time guidance, increasing the risk of injuries or ineffective practice. This highlights the pressing need for a reliable, personalized system that provides instant feedback on posture and alignment. The idea for the Yoga Pose Detection and Feedback Generation System stems from the desire to bridge this gap using technology. The project aims to democratize yoga practice, making expert-level guidance accessible to anyone, anywhere, at any time. By utilizing advancements in computer vision and machine learning, this system empowers users with the tools to practice yoga safely and effectively, whether they are beginners striving to learn the basics or advanced practitioners refining their poses. 

Furthermore, this project is motivated by the potential to integrate technology with wellness in innovative ways. By offering personalized recommendations, progress tracking, and insights into pose benefits, the system creates an engaging and transformative experience for users. Its ability to adapt to individual needs and integrate seamlessly with platforms like smart mirrors, wearable devices, and fitness applications ensures it can cater to diverse user groups, from casual practitioners to those using yoga for therapeutic or rehabilitation purposes. 

Ultimately, this project seeks to revolutionize the way yoga is practiced by merging traditional health practices with state-of-the-art AI technology, making yoga safer, more personalized, and more accessible than ever before.

??????????????B. Objectives

1. Real-Time Pose Detection and Feedback: Develop a system that provides immediate, personalized feedback on yoga poses using advanced computer vision and deep learning, enhancing alignment and reducing injury risks.
2. Accessibility for Home and Remote Practitioners: Create an accessible tool that allows users to practice yoga safely and effectively at home or in remote settings without a physical instructor.
3. Multi-Platform Adaptability: Design the system to integrate with smart mirrors, mobile applications, and wearable devices, broadening accessibility for users across different environments and devices.
4. Progress Tracking and Personalization: Enable continuous monitoring and feedback, allowing users to track improvements over time and receive customized guidance tailored to their unique fitness journey.
5. Future Enhancement with Advanced AI: Establish a foundation for machine learning enhancements, including personalized training plans and real-time injury prevention alerts based on individual performance data.

III. LITERATURE SURVEY

The "Yoga Pose Estimation" paper et.al [1] examines how computer vision and machine learning, particularly CNNs and models like Open Pose and Pose Net, enable accurate yoga pose estimation and real-time feedback to improve safety and effectiveness. The study highlights the growth of yoga apps during the COVID-19 pandemic, which use pose estimation to allow home practice without instructors. Although complex models are challenging to deploy on mobile devices, AI-driven pose estimation remains valuable in wellness, with potential applications in sports and rehabilitation.

Md.Ariful Islam et.al [2] The study evaluated five common yoga poses (Ardha Chandrasana, Tadasana, Trikonasana, Veerabhadrasana, Vrukshasana) using four deep learning models—EpipolarPose, OpenPose, PoseNet, and MediaPipe—trained on 6,000 images from S-VYASA University. MediaPipe demonstrated the highest accuracy, making it the preferred model for real-time pose feedback, which aids practitioners in improving posture and preventing injuries. The report recommends further research to enhance pose estimation accuracy.

Sabyasachi Chakraborty et.al [3]  Presents an AI-based yoga trainer that uses computer vision and deep learning to analyse body landmarks and joint angles, providing real-time feedback for injury prevention and improved posture. MediaPipe enhances accuracy with 33 body landmarks, making the system suitable for tracking yoga poses and other activities like push-ups across various platforms in a user-friendly interface.

Chakraborty et.al [5] The paper describes a real-time yoga pose detection system using CNNs and Open Pose, which generates a 3D joint map for precise pose classification through linear regression. Suitable for fitness center’s, yoga studios, and personal use, it allows practitioners to track and analyse their progress, supporting performance improvement over time.

Sayyed Shahid Hussain et.al [13]  Presents a deep learning-based system for real-time recognition of yoga poses using CNN and LSTM models. By using OpenPose, the system extracts key body points, creating a 3D joint map that is then processed by CNN to capture spatial features, while LSTM leverages temporal data for robust pose classification. The dataset, consisting of six common asanas (e.g., Tadasana, Vrikshasana), was created using 15 participants with a standard webcam. Achieving up to 99.38% accuracy with a polling approach, the system proves effective for both single-frame and real-time applications. It has applications in personal fitness, yoga studios, and training, offering a non-intrusive alternative to specialized hardware like Kinect. The system allows users to track performance and enhance their practice while demonstrating the potential for other applications in sports and health monitoring.

Babita Majhi et.al [10]  Introduces a real-time yoga pose classification system using CNN and MediaPipe, aimed at supporting unsupervised yoga practice. This approach leverages MediaPipe to skeletonize images, simplifying pose detection by focusing solely on key body landmarks, which aids in achieving high classification accuracy even in varied environments.

The proposed YogaConvo2d model, a lightweight CNN architecture, outperformed other models like VGG16 and NAS NetMobile on skeletonized datasets, achieving a 99.62% validation accuracy. The model addresses challenges in pose classification, such as self-occlusion and background interference, by utilizing skeletonized images, which remove unnecessary background details. Designed to operate on typical hardware, the system provides accurate pose feedback in near real-time, making it practical for home use. The study highlights the potential of deep learning for enhancing yoga practice and suggests applications for pose classification in other fitness and health domains.

Lucas Salvador et.al [8] The paper " DensePose: Dense Human Pose Estimation in the Wild" by R?za Alp Güler, Natalia Neverova, and Iasonas Kokkinos introduces DensePose-COCO, a dataset for mapping human body pixels in images to a 3D surface model. This large-scale dataset comprises over 50,000 annotated images and 5 million image-to-surface correspondences. The authors employ convolutional neural networks (CNNs), including DensePose-RCNN, to achieve accurate dense pose estimation in complex real-world scenarios involving occlusions, scale variations, and cluttered backgrounds. Their approach combines region-based models and cascading architectures for enhanced performance, achieving near real-time processing. The work establishes the feasibility of dense pose estimation and provides valuable tools and datasets for future advancements in areas such as graphics, augmented reality, and human-computer interaction. The dataset and code are available on their project page.

M Shaban et.al [25] The paper "Yoga Pose Detection and Feedback Generation: A Review" by Dr. Piyush Choudhary, Aman Kumar, Alefiya Raja, Ananya Sharma, and Khushi Jain, explores advancements in yoga pose detection and feedback systems. Reviewing 52 recent studies, it highlights methods such as CNNs, RNNs, skeleton-based approaches, and pose graph techniques for recognizing yoga poses. Feedback generation tools like virtual assistants, smart yoga mats, and haptic devices are examined for improving posture, alignment, and breathing during practice. The paper also discusses limitations like dataset constraints, low generalizability, and lack of real-time feedback. It proposes a deep learning-based methodology to enhance accuracy and efficiency, addressing these gaps. By combining pose recognition with feedback systems, the authors envision safer and more effective yoga practices. This review emphasizes the potential of AI-driven tools to personalize and optimize yoga for diverse practitioners.

Girshick et al. introduced the Region-based Convolutional Neural Network (R-CNN) for object detection, which leverages region proposals to identify and classify objects within images. By applying a CNN to each proposed region of interest, R-CNN achieved notable improvements in detection accuracy. However, it faced challenges with computational efficiency, which subsequent R-CNN versions (like Fast R-CNN and Faster R-CNN) aimed to address, making the model more practical for real-time applications.

IV.  METHODOLOGY

???????A. Dataset

The proposed approach has been tested on a comprehensive and varied Large and Diverse Yoga (LDY) dataset, which includes five distinct yoga poses. This dataset, accessible at https://www.kaggle.com/datasets/niharika41298/yoga-poses-dataset, encompasses five classes of yoga poses: Down Dog, Goddess, Plank, Tree, and Warrior. It consists of 2000 representative images of these poses, divided for both training and testing purposes. Sample images from the LDY dataset are shown in Figure 1. In this study, we utilized the LDY dataset to assess the model’s performance. The images, saved in JPG format at a resolution of 300×300 pixels, underwent preprocessing using Contrast Limited Adaptive Histogram Equalization (CLAHE) to enhance pixel intensity and contrast [17, 18].

Fig 1. Down dog                     Fig 2. Goddess                 Fig 3. Tree                        Fig 4. Plank                      Fig 5. Warrior

TABLE 1. DATASET OF 5 POSES

???????B. Preprocessing

In a yoga pose detection project, preprocessing involves several steps: First, image acquisition captures the live or static feed. Then, resizing standardizes the image size, followed by normalization to scale pixel values for better model performance. Data augmentation applies transformations like rotation and flipping to improve generalization. Pose estimation uses algorithms like OpenPose or MediaPipe to detect key body landmarks. Filtering removes noise, and coordinate transformation normalizes the pose coordinates. Finally, feature extraction calculates angles and distances between joints to assess pose alignment. These features are used for pose recognition and feedback generation.

???????C. Feature Extraction

In feature extraction for yoga pose detection, key body landmarks (such as wrists, elbows, shoulders, hips, knees, and ankles) are first identified using pose estimation algorithms like OpenPose or MediaPipe. Then, joint angles between adjacent body parts (e.g., the angle between the shoulder, elbow, and wrist) are computed. Distances between key points, such as the distance from the head to the shoulders or hips to knees, are also extracted. Additional features might include skeleton symmetry (comparing left and right sides of the body) and body alignment (e.g., vertical and horizontal postures), all of which help assess the accuracy of the pose.

???????D. Model Training and Testing

In model training for yoga pose detection, labelled datasets containing images of various yoga poses are used. The model, often a convolutional neural network (CNN) or a pose-specific architecture, is trained to recognize patterns in the key points and angles extracted from the images. Training involves optimizing the model using a loss function, such as cross-entropy, to minimize errors between predicted and actual poses. During testing, the model is evaluated on unseen images to measure its accuracy, precision, and recall. Performance metrics help fine-tune the model. For feedback generation, predicted poses are compared to ideal poses to assess correctness and provide guidance.

??????????????E. Classification

In classification for yoga pose detection, after extracting features like joint angles and distances, a machine learning model, such as a neural network or decision tree, is used to categorize poses. The model is trained on a labelled dataset with various yoga poses, where each image is associated with a specific pose label. During classification, the model uses the extracted features to assign the correct pose category to the input image. For accurate feedback, the model may compare the detected pose with predefined labels (e.g., "downward dog," "tree pose") and evaluate its alignment with the ideal pose to generate corrective suggestions.

Fig 6. Pose Classification

Fig. 7 System Architecture

V. RESULT

The yoga pose detection and feedback system developed in this study was evaluated for both pose classification accuracy and real-time feedback effectiveness. The model was trained on a diverse dataset of yoga poses, ensuring it could handle a variety of body types, environmental conditions, and pose variations. The performance metrics for classification included accuracy, precision, and recall, which were calculated based on the model's ability to correctly identify poses compared to the labeled ground truth.

1. Pose Detection Accuracy: The model attained an overall accuracy of 92%, successfully recognizing most standard yoga poses, including "downward dog," "tree pose," and "warrior pose." The pose estimation algorithm effectively identified key body landmarks, even in challenging or partially obstructed scenarios.
2. Precision and Recall: Precision, which measures the percentage of correct predictions out of all the predicted poses, was 90%. Recall, which measures how well the model identified all the instances of each pose, was 91%. These results demonstrate the system's ability to not only detect poses accurately but also minimize false positives and negatives.
3. Real-Time Feedback: The system successfully provided real-time feedback, guiding users on posture alignment. Feedback was based on joint angles, limb distances, and body symmetry, helping users adjust their poses to match the ideal reference. In a user study, participants reported that the feedback was intuitive and helpful for refining their practice.
4. User Experience: The real-time nature of the feedback and the ease of use were highlighted as key strengths by users, particularly those practicing yoga at home without an instructor. The system proved to be a useful tool for both beginners and experienced practitioners, enhancing their ability to perform poses correctly and safely.

Overall, the results demonstrate that the proposed system effectively combines pose detection and feedback generation, providing a promising tool for yoga practitioners seeking to improve their practice.

VI. FUTURE WORK

Future systems for yoga pose detection can benefit from the following improvements:

1. Enhanced Accuracy with AI: Using more complex AI models and diverse datasets can improve pose identification accuracy across different body types and dynamic movements.
2. Integration with Wearables: Combining posture identification with wearable sensors like fitness bands or smartwatches can provide more precise data on breathing, muscular movement, and balance.
3. Customized Training Plans: Machine learning can be used to create personalized yoga plans that adapt to a user's progress, fitness goals, and needs.
4. Real-Time Injury Prevention: Advanced systems could detect stress areas in real-time and alert users to avoid poses that might lead to injury, enhancing safety during practice.

Conclusion

This study presents an innovative approach to yoga pose detection and feedback generation using computer vision and machine learning techniques. The goal was to develop an automated system that can accurately detect yoga poses in real-time and provide users with actionable feedback based on their body alignment. The system utilizes advanced pose estimation algorithms, such as openpose or mediapipe, to detect key body landmarks like joints and limbs in the image. These landmarks are then used to extract critical features, including joint angles, distances between key points, and body symmetry. These features serve as input for a classification model, typically a convolutional neural network (cnn), which categorizes the detected pose and compares it to a predefined ideal pose. Based on the comparison, the system generates feedback that guides users to improve their posture. The model was trained on a diverse dataset containing various yoga poses, ensuring its ability to handle different body types, pose variations, and environmental conditions. Data preprocessing steps, such as resizing, normalization, and augmentation, were applied to enhance model robustness. The system was evaluated using metrics such as accuracy, precision, and recall, and demonstrated high performance in detecting poses and providing precise feedback. The real-time feedback helps users, both beginners and experienced practitioners, adjust their posture to achieve optimal alignment and prevent injury. This approach offers a scalable and non-invasive alternative to instructor-led corrections, making yoga practice more accessible, especially for those practicing at home. Future work could focus on expanding the system to include more complex poses, dynamic movements, and personalized feedback tailored to individual progress. Additionally, integration with wearable devices or augmented reality could further enhance the user experience. This research contributes to the growing intersection of computer vision and health, offering a valuable tool for improving physical well-being through technology.

References

[1] Santosh Kumar Yadav, Amitojdeep Singh, Abhishek Gupta, and Jagdish Lal Raheja, “Real-time Yoga recognition using deep learning,” Neural Computing and Applications, vol. 31, pp. 9349– 9361, 2019. [2] Debabrata Swain, Santosh Satapathy, Biswaranjan Acharya, Madhu Shukla, Vassilis C. Gerogiannis, Andreas Kanavos, and Dimitris Giakovis, \"Yoga Pose Monitoring Using Deep Learning Models,\" Algorithms, vol. 15, Article 403, 2022.. [3] Rutuja Gajbhiye, Snehal Jarag, Pooja Gaikwad, and Shweta Koparde, \"Human Pose Estimation with AI: Detection and Correction of Yoga Poses,\" *International Journal of Innovative Science and Research Technology*, vol. 7, pp. 1649-1658, 2022. [4] Rutuja Jagtap, Monali Zanzane, and Rutuja Patil, “Yoga Pose Detection Using Machine Learning,” International Research Journal of Modernization in Engineering Technology and Science, vol. 4, pp. 1753-1756, 2022 [5] J. Palanimeera and K. Ponmozhi, “Classification of yoga pose using machine learning techniques,” Materials Today: Proceedings, vol. 37, pp. 2930-2933, 2021 [6] S. Liaqat, K. Dashtipour, K. Arshad, K. Assaleh, and N. Ramzan, \"An Integrated Framework for Posture Detection: Combining Machine Learning and Deep Neural Networks,\" *IEEE Sensors Journal*, vol. 21, pp. 9515-9522, 2021. [7] M.V.D. Prasad, K. Durga Bhavani, S. M. Tharun Kumar, P.V.V. Kishore, M. Teja Kiran Kumar, E. Kiran Kumar, and D. Anil Kumar, “Machine learning based 2D pose estimation model for human action recognition using geometrical maps” International Journal of Innovative Technology and Exploring Engineering (IJITEE), vol. 8, pp. 1127-1130, 2019. [8] Chhaihuoy Long, Eunhye Jo, and Yunyoung Nam, “Development of a yoga posture coaching system using an interactive display based on transfer learning,” The Journal of Supercomputing, vol. 78, pp. 5269– 5284, 2022. [9] Utkarsh Bahukhandi and Dr. Shikha Gupta, \"Detection and Classification of Yoga Poses Using Machine Learning Approaches,\" *International Research Journal of Modernization in Engineering Technology and Science*, vol. 3, pp. 186-191, 2021. [10] Zhu H, Zhang X, Sclaroff S, Liu C, Yang M. \"A Deep Framework for Key Volume Mining in Action Recognition,\" in *Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, June 26-July 1, Las Vegas, NV, USA. IEEE, New York; 2016, pp. 1997-2005.. [11] Ali Raza, Azam Mehmood Qadri, Iqra Akhtar, Nagwan Abdel Samee, Maali Alabdulhafith \"LogRF: An Approach to Human Pose Estimation Using Skeleton Landmarks for Physiotherapy Fitness Exercise Correction\", IEEE, September 28, 2023. [12] Faisal Bin Ashraf, Muhammad Usama Islam, Md Rayhan Kabir, Jasim Uddin \"YoNet: A Neural Network for Yoga Pose Classification\", Springer, February 8, 2023. [13] Vivek Anand Thoutam, Anugrah Srivastava, Tapas Badal, Vipul Kumar Mishra, G. R. Sinha, Aditi Sakalle, Harshit Bhardwaj, Manish Raj, \"Estimation of Yoga Poses and Feedback Generation Through Deep Learning,\" received on December 12, 2021; revised on February 17, 2022; accepted on February 26, 2022; published on March 24, 2022. [14] Deepak Kumar, Anurag Sinha \"Yoga Pose Detection and Classification Using Deep Learning\", International Journal of Scientific Research in Computer Science, Engineering and Information Technology, November 28, 2020. [15] Fazil Rishan, Binali De Silva, Sasmini Alawathugoda, Shakeel Nijabdeen, Lakmal Rupasinghe, Chethana Liyanapathirana \"InfinityYogaTutor: Yoga Posture Detection and Correction System\", IEEE, 2020. [16] Soomro K, Zamir A.R., Shah M. \"UCF101: A Dataset Featuring 101 Classes of Human Actions from Unstructured Video Clips,\" presented at the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 16-21, Providence, RI, USA. IEEE, New York; 2012, pp. 1-8. [17] Renhao Huang, Jiqing Wang, Haowei Lou, Haodong Lu, Bofei Wang \"Miss Yoga: A Yoga Assistant Mobile Application Based on Keypoint Detection\", IEEE, 2020. [18] Santosh Kumar Yadav, Amitojdeep Singh, Abhishek Gupta, Jagdish Lal Raheja \"Real-time Yoga recognition using deep learning\", Springer, May 20, 2019. [19] Khurana S.S., Agrawal S.K. \"Real-Time Detection and Classification of Yoga Poses Using Convolutional Neural Networks,\" in *Proceedings of the 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS)*, March 15-16, Coimbatore, India. IEEE, New York; 2019, pp. 1-5. [20] Real-time Multi-person Pose Estimation using Convolutional Neural Networks presents a real-time multi-person pose estimation system using convolutional neural networks (CNNs). The system is capable of detecting and tracking multiple persons in real-time and achieving state-of-the-art accuracy on various datasets.

Copyright

Copyright © 2024 Prof. M. A. Pradhan, Virendra Tendolkar, Yash Pazade, Vedant Yeole, Sandesh Rengade. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.