Introduction: Video-based assessment of exercise technique can support coaching, injury prevention, and remote rehabilitation, yet many methods stop at action recognition or require laboratory motion capture. To develop and evaluate a deep-learning framework that classifies execution quality as correct vs. incorrect form from instructional videos.
Methods: We compiled 270 YouTube coaching videos spanning ten exercises (~20,500 labeled frames after removing ~15% non-movement content). Clip-level technique labels (correct/incorrect) were assigned based on coaching/biomechanical criteria and propagated to sampled frames. A marker-less pose-estimation model (name/version reported) produced 2D key points and kinematic descriptors (e.g., joint angles and velocities). These signals were encoded with a ResNet-50 backbone, and a multi-head attention Transformer modeled temporal dependencies. Training used a video-stratified 80/20 split, with 5-fold cross-validation on the training portion only for model selection; class-weighted loss mitigated imbalance, and evaluation reports per-class precision/recall/F1 and confusion matrices. We also define a biomechanical complexity index (0–1) combining joints engaged, angular sensitivity, range of motion, and balance demand to relate movement difficulty to performance.
Results: The full CNN–attention–Transformer achieved 97.56% accuracy and F1=0.956 on the held-out test set. Per-exercise performance remained high for low-to-intermediate complexity movements, while the most challenging exercises showed reduced F1—for example, deadlift and crunch exhibited the lowest scores (≈0.88–0.90), yielding an overall F1 range of 0.875–0.945 across exercises.
Conclusion: This framework enables multi-exercise, video-based technique assessment (correct vs. incorrect) from consumer footage. Attention-weight patterns and error-case analyses provide practical insight into model decisions for intelligent coaching and tele-rehabilitation. |
- Penichet-Tomas A. Applied biomechanics in sports performance, injury prevention, and rehabilitation. Appl Sci. 2024;14(24):11623.
- Zhao F. The application of sports biomechanics in sports injury prevention and rehabilitation. Front Sport Res. 2024 May 20;6(3):142-7.
- Pardos A, Tziomaka M, Menychtas A, Maglogiannis I. Automated posture analysis for the assessment of sports exercises. In: Proceedings of the 12th Hellenic Conference on Artificial Intelligence; 2022 Sep 7; (pp. 1-9).
- Jayaweerage I, Rajamanthri K, Kothalawala A, Gunawardana N, Induranga A, Weerakkody P, et al. Motion capturing in cricket with bare minimum hardware and optimised software: a comparison of MediaPipe and OpenPose. In: 2024 First International Conference on Software, Systems and Information Technology (SSITCON); 2024 Oct 18; (pp. 1-8).
- Niu Z. A lightweight two-stream fusion deep neural network based on ResNet model for sports motion image recognition. Sens Imaging. 2021;22(1):26.
- Cust EE, Sweeting AJ, Ball K, Robertson S. Machine and deep learning for sport-specific movement recognition: a systematic review of model development and performance. J Sports Sci. 2019;37(5):568-600.
- Ma J, Ma L, Ruan W, Chen H, Feng J. A wushu posture recognition system based on MediaPipe. In: 2022 2nd International Conference on Information Technology and Contemporary Sports (TCS); 2022 Jun 24; (pp. 10-13).
- Ashok A, Surendiran B, Babu S. Yoga pose estimation using MoveNet deep learning models. Sadhana. 2025;50(3):125.
- Dayapoglu M, Savirdi E, Akcaba I, Alp S, Kucuk S. 3D skeleton-based sport action recognition system via deep learning. In: 2024 Innovations in Intelligent Systems and Applications Conference (ASYU); 2024 Oct 16; (pp. 1-6).
- Pagnon D, Domalain M, Reveret L. Pose2Sim: an open-source Python package for multiview markerless kinematics. J Open Source Softw. 2022;7(77):4362.
- Assa A, Janabi-Sharifi F, Plataniotis KN. Sample-based adaptive Kalman filtering for accurate camera pose tracking. Neurocomputing. 2019;333:307-18.
- Ziche L, Lenzig B, Bieck R, Neumuth T, Schoenfelder S. Image-based deep learning of finite element simulations for fast surrogate biomechanical organ deformations. In: Current Directions in Biomedical Engineering. 2022;8(2):528-31.
- Boyle A, Ross GB, Graham RB. Machine learning and deep neural network architectures for 3D motion capture datasets. In: 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC); 2020 Jul 20; (pp. 4827-30).
- Pan X, Luo Z, Zhou L. Comprehensive survey of state-of-the-art convolutional neural network architectures and their applications in image classification. Innov Appl Eng Technol. 2022 Mar 11;:1-6.
- Wu Z. Application of CNN classic model in modern image processing. J Adv Eng Technol. 2024 Nov 13;1(3):1-6.
- Ding G, Plummer A, Georgilas I. Deep learning with an attention mechanism for continuous biomechanical motion estimation across varied activities. Front Bioeng Biotechnol. 2022;10:1021505.
- Yang J, Ismail AW. A review: deep learning for 3D reconstruction of human motion detection. Int J Innov Comput. 2022;12(1):65-71.
- Liu R, Chen Y, Gai F, Liu Y, Miao Q, Wu S. Local and global spatial-temporal transformer for skeleton-based action recognition. Neurocomputing. 2025;634:129820.
- Ahmad T, Wu J, Alwageed HS, Khan F, Khan J, Lee Y. Human activity recognition based on deep-temporal learning using convolution neural networks features and bidirectional gated recurrent unit with features selection. IEEE Access. 2023;11:33148-59.
- Zhao X. Research on athlete behavior recognition technology in sports teaching video based on deep neural network. Comput Intell Neurosci. 2022;2022(1):7260894.
- Pandey S, Kumar N. Enhancing human action recognition in high-resolution videos using ConvLSTM and LRCN model. In: 2023 4th International Conference on Communication, Computing and Industry 6.0 (C2I6); 2023 Dec 15; (pp. 1-5).
- Al-Timemy AH, Castellini C, Escudero J, Khushaba R, Muceli S. Current trends in deep learning for movement analysis and prosthesis control. Front Neurosci. 2022;16:889202.
- Bazarevsky V, Grishchenko I, Raveendran K, Zhu T, Zhang F, Grundmann M. BlazePose: on-device real-time body pose tracking. arXiv [preprint]. 2020 Jun 17:arXiv:2006.10204.
- Cheng SH, Sarwar MA, Daraghmi YA, Ik TU, Li YL. Periodic physical activity information segmentation, counting and recognition from video. IEEE Access. 2023;11:23019-31.
- Rangari T, Kumar S, Roy PP, Dogra DP, Kim BG. Video based exercise recognition and correct pose detection. Multimed Tools Appl. 2022;81(21):30267-82.
- Li Y, He H, Zhang Z. Human motion quality assessment toward sophisticated sports scenes based on deeply-learned 3D CNN model. J Vis Commun Image Represent. 2020;71:102702.
- Zhang M, Li Y, Cui Y. The use of deep learning in intelligent athlete motion recognition: integrating biological mechanisms. Molecular & Cellular Biomechanics. 2025;22(1):670.
- Sadr MM, Khani M, Tootkaleh SM. Predicting athletic injuries with deep learning: evaluating CNNs and RNNs for enhanced performance and safety. Biomed Signal Process Control. 2025;105:107692.
- Chen S, Yang RR. Pose Trainer: correcting exercise posture using pose estimation. arXiv [preprint]. 2020:arXiv:2006.11718.
- Hsu GSJ, Wu JS, Huang YKD, Chiu CC, Kang JH. Automatic detect incorrect lifting posture with the pose estimation model. Life (Basel). 2025;15(3):358.
|
