The coordinate ingestion pipeline described below — MediaPipe extracting 33 landmarks per frame — running on your own device via WebAssembly. Nothing is uploaded, recorded, or sent anywhere; inference happens entirely in this browser tab.
Runs entirely in your browser tab — nothing is uploaded or recorded. Camera permission is optional.
Requires HTTPS (works once deployed — e.g. via GitHub Pages). Best in Chrome or Edge; camera access varies on Safari/iOS.
Section 01
Executive Summary & Objective
State-of-the-art (SOTA) human pose estimation (HPE) models — Google MediaPipe and YOLOv8-Pose — deliver strong accuracy on standard benchmark datasets featuring form-fitting apparel. Prior work has separately studied martial arts pose estimation (in standard apparel), loose-clothing occlusion (in winter sports), and Kalman-filtered biomechanical tracking (in sprinting) — but not the specific combination this project targets: rigid, heavyweight canvas fabric (e.g. a 14oz Karate Gi) folding under high-velocity kinetic transitions. This is a mechanically distinct occlusion pattern from loose drape-type fabric, and it has not been benchmarked or corrected for in the literature reviewed to date.
This project addresses that gap in two phases:
Quantify the degradation of standard coordinate-prediction models (MediaPipe, YOLOv8-Pose) under rigid-fabric occlusion and high-velocity transitions (Paper 1).
Develop and validate an event-triggered, physics-informed correction filter — an Extended Kalman Filter (EKF) that activates on confidence collapse rather than smoothing continuously — to reconstruct true joint trajectories and compute Center of Gravity (CoG) (Paper 2).
Section 02
Introduction & Problem Statement
SOTA computer-vision HPE frameworks suffer measurable performance degradation when exposed to two concurrent real-world variables: (1) high-velocity kinetic transitions, which introduce localized motion blur, and (2) complex material occlusion, where rigid canvas fabric folds and alters body silhouettes, collapses visual joint markers, and introduces false edge boundaries. This differs mechanically from the loose-fabric draping studied in winter-sports HPE literature: rigid canvas creates hard, false edges rather than soft occlusion. This research formulates a physics-informed post-processing pipeline that stabilizes tracking and produces reliable athletic metrics under these conditions — using consumer-grade hardware only.
Section 03
Related Work & Positioning
A review of adjacent literature shows this project sits at an intersection that has not been directly addressed, though each neighboring piece is well established individually.
Prior Work
What It Covers
What It Doesn't Cover
Echeverria & Santos (2021), Sensors
Pose estimation (OpenPose) applied to karate kumite to model psychomotor performance.
No garment variable; assumes standard training apparel.
Zhang et al. (2017) — MADS dataset
Multi-view + stereo-depth benchmark including Karate and Tai-chi performed by martial arts masters.
Targets general self-occlusion and fast articulation, not garment-specific occlusion.
Dael et al. (2026), ICAAIC
Direct MediaPipe vs. YOLOv8 comparison on scenarios including martial techniques.
Measures FPS / CPU / memory — not accuracy degradation from clothing.
Handball Kalman-smoothing study (2022), J. Imaging
Per-axis Kalman smoothing applied to noisy pose-estimation keypoints in sport.
Continuous smoothing for general noise, not an event-triggered response to occlusion-driven confidence collapse.
Sprinting CoM study (2021), Sensors
The closest direct ancestor: Kalman-smoothed pose keypoints → de Leva segment model → center-of-mass, validated against marker-based ground truth.
Applied to sprinting biomechanics, not to garment occlusion or martial arts.
Nakai et al. (2022), loose-clothing ground truth
HFR camera + blinking LEDs under loose clothing to obtain verified ground truth for skiing/snowboarding apparel.
Addresses loose drape-type occlusion, not rigid canvas folding; not applied to combat sports.
Positioning Statement
This project does not claim to invent martial-arts pose estimation, Kalman-filtered biomechanical tracking, or clothing-occlusion research — each already exists. The contribution is narrower and more defensible: characterizing the specific compounding failure of rigid-canvas occlusion under high-velocity kinetics, and correcting it with an event-triggered (confidence-gated) filter rather than the continuous smoothing used in prior sports-HPE Kalman applications.
Section 04
Research Objectives
01
Establish a controlled dual-garment dataset isolating rigid canvas-fabric occlusion from high-velocity martial arts kinetics, extending existing karate pose-estimation work (which assumes standard apparel) and existing loose-clothing occlusion work (which addresses draping, not folding).
02
Quantify degradation using confidence-score volatility and spatial jitter, with a pre-registered statistical comparison (Set A vs. Set B).
03
Construct a biomechanical framework (de Leva segment model) to compute live Center of Gravity and Stance Depth, following the validated approach used in markerless sprinting biomechanics.
04
Design and validate an event-triggered Extended Kalman Filter that rejects low-confidence coordinates and projects joint location via momentum — distinct from the continuous smoothing used in prior sports-HPE Kalman applications — benchmarked against a linear-interpolation baseline.
05
Publish two manuscripts — a benchmarking study and an algorithmic optimization methodology — with an honest discussion of dataset scale as a limitation.
Section 05
Methodology & System Architecture
5.1 Data Capture Protocol
A controlled dataset will be captured with an iPhone 15 Pro Max on a fixed tripod, using two settings: 4K at 60 FPS for standard tracking and inference testing, and 1080p at 240 FPS as a slow-motion baseline for visual cross-checking during blur.
Two matched conditions will be filmed performing foundational Shotokan Katas (Oi-zuki, Gedan-barai, Zenkutsu-dachi transitions): Set A is the control — tight, lightweight athletic apparel — and Set B is the experimental, occlusion condition — a heavyweight 14oz canvas Gi.
Dataset Scale
Where feasible, footage will be collected across multiple sessions and, ideally, more than one practitioner. A single-subject, single-session dataset is treated explicitly as a pilot/feasibility scope in Section 8 (Limitations), not disguised as a generalizable result.
5.2 Strengthening Ground Truth
Tight clothing (Set A) is a proxy for ground truth, not verified ground truth — MediaPipe/YOLO can still err on unoccluded footage. Following the approach used in loose-clothing HPE ground-truth work, a small number of high-contrast markers (e.g. colored tape or small reflective tags) will be placed at key joints for a subsample of takes in both conditions, enabling manual frame-by-frame verification independent of any pose-estimation model. This gives Paper 1 a semi-independent accuracy check beyond "Set A looks smooth."
5.3 Coordinate Ingestion Pipeline
A Python pipeline (OpenCV, MediaPipe, Ultralytics YOLOv8) streams both video sets frame-by-frame and logs structural joint arrays into structured DataFrames: MediaPipe contributes 33 landmarks per frame, each with (x, y, z) coordinates and a visibility/confidence score (v); YOLOv8-Pose contributes 17 keypoints per frame with bounding-box coordinates and a detection confidence score (c). All frames are indexed against key joints — ankle, knee, hip, shoulder, elbow, wrist.
5.4 Phase 1 — Material Occlusion Benchmark (Paper 1)
Two degradation vectors are isolated and quantified: confidence score volatility, mapping drops in detection confidence (v or c) for the hip and knee joints during deep-stance transitions, and spatial jitter, the frame-to-frame Euclidean displacement of each tracked coordinate.
Jitter = √[ (xt − xt−1)² + (yt − yt−1)² ]
5.5 Statistical Analysis Plan
Defined before data collection, not after. The primary comparison is jitter magnitude, Set A vs. Set B, per joint, using a Wilcoxon signed-rank test (paired, non-parametric; jitter distributions are expected to be right-skewed). Effect size is reported alongside significance (rank-biserial correlation or Cohen's d on log-transformed jitter). Confidence-score volatility is compared using the same paired non-parametric approach. Phase 2 evaluation compares EKF-corrected trajectories against both raw Set B output and a linear-interpolation baseline — not EKF against nothing.
Biomechanical CoG mapping: the human body is modeled as a linked-segment system using de Leva's (1996) anatomical mass percentages (e.g. Torso ≈ 43.46%, Thigh ≈ 14.16%, Lower Leg ≈ 4.33%), following the same approach validated for markerless center-of-mass tracking in sprinting biomechanics. Global Center of Gravity for a given frame is:
XCoG = Σ(xi · wi) YCoG = Σ(yi · wi)
where xi, yi is the midpoint of body segment i and wi is its relative anatomical mass weight.
Event-triggered state correction: unlike prior sports-HPE applications that apply Kalman smoothing continuously across every frame, this filter is designed to activate specifically on occlusion events. While confidence remains high, the filter updates its velocity model from the raw AI output. When a joint's confidence score drops below an empirical threshold (T < 0.50), the filter rejects the raw coordinate and instead projects the joint's location forward using its established momentum vector. Threshold T will be tuned on a held-out portion of the dataset, not fixed a priori. This is evaluated against a linear-interpolation baseline to demonstrate the momentum-based approach is doing more than simple gap-filling.
Simulated data — for illustration
Raw output (MediaPipe / YOLOv8)EKF-corrected trajectoryJoint confidence score
Conceptual sketch of Sections 5.4–5.6: raw coordinates jitter and confidence collapses during rigid-fabric occlusion; the filter detects the confidence drop and switches to a momentum-projected estimate until confidence recovers. Synthetic values, not experimental results.
Section 06
Hardware & Software Infrastructure
The entire development loop runs on existing consumer hardware, requiring no additional budget or university compute infrastructure.
Hardware
iPhone 15 Pro Max
Data capture — 60 / 240 FPS
ASUS ROG G513 (Ryzen 7 6800H, RTX 3060 6GB)
Local ML training & inference — CUDA-accelerated YOLOv8 pipelines
Lightweight consumer-grade models (MediaPipe, YOLOv8) are used deliberately, not because heavier transformer-based pose models are unknown to the field — they are, and tend to outperform on accuracy in adjacent sports benchmarks. The choice reflects a real dojo-deployment use case (no GPU cluster, no motion-capture lab) and is stated as a design decision in the manuscript, not left implicit.
Section 07
Expected Academic Contributions
Manuscript 1 · System Evaluation
A benchmark quantifying pose-model degradation under rigid-fabric occlusion combined with high-velocity martial arts kinetics — a compounding failure mode not isolated in prior karate-HPE or clothing-occlusion literature.
Manuscript 2 · Algorithmic Solution
An event-triggered EKF correction method, benchmarked against a linear-interpolation baseline, extending the Kalman + de Leva pipeline validated in sprinting biomechanics to a garment-occlusion setting.
Venue placement will depend on peer-review outcomes and is not fixed at proposal stage.
Section 08
Limitations & Threats to Validity
Dataset scale — a single-subject dataset (if that is the final scope) limits generalizability; this will be stated explicitly as a pilot/feasibility study, not implied to be population-level.
Ground truth — Set A is a proxy, not verified ground truth; the marker-assisted subsample (Section 5.2) partially, not fully, mitigates this.
Model scope — only MediaPipe and YOLOv8-Pose are benchmarked; transformer-based models (e.g. ViTPose) are known to outperform both in adjacent sports domains and are noted as future work rather than omitted silently.
Threshold tuning — the T < 0.50 confidence threshold will be empirically tuned rather than assumed correct from the outset.
Section 09
Project Timeline
01
Foundations & Environment Setup
Months 1–3
Master Python data handling; stand up the video/coordinate pipeline (Pandas, OpenCV, MediaPipe); finish literature review.
Deliverable — Working extraction script; annotated bibliography
02
Video Acquisition & Baseline Dataset
Months 4–6
Capture the controlled dual-garment dataset across multiple sessions (and additional practitioners if available); apply joint-marker ground-truth aid.
Deliverable — Complete Set A / Set B video corpus + marker-verified subsample
03
Paper 1 — Benchmarking Analysis & Write-up
Months 7–12
Run extraction on both sets; run pre-registered statistical tests; draft manuscript.
Deliverable — Manuscript 1 drafted
04
Biomechanical Integration
Months 13–16
Implement Center-of-Gravity and Stance Depth tracking on the clean Set A baseline; build linear-interpolation baseline filter.
Deliverable — Working CoG overlay + baseline filter
05
Physics-Informed Filter & Paper 2
Months 17–22
Build and validate the Extended Kalman Filter on occluded Set B data; benchmark against the baseline; draft manuscript.
Deliverable — Manuscript 2 drafted
06
Review, Revision & Submission
Months 23–24
Incorporate supervisor/peer feedback; finalize both manuscripts for submission.
Deliverable — Both manuscripts submitted
Section 10
References
de Leva, P. (1996). Adjustments to Zatsiorsky-Seluyanov's segment inertia parameters. Journal of Biomechanics, 29(9), 1223–1230.
Echeverria, J., & Santos, O. C. (2021). Toward Modeling Psychomotor Performance in Karate Combats Using Computer Vision Pose Estimation. Sensors, 21(24), 8378.
Zhang, W., Liu, Z., Zhou, L., Leung, H., & Chan, A. B. (2017). Martial Arts, Dancing and Sports dataset: A challenging stereo and multi-view dataset for 3D human pose estimation. Image and Vision Computing, 61, 22–39.
Dael, F. A., et al. (2026). Comparative Evaluation of MediaPipe and YOLOv8 for Real-Time Pose Estimation. Proceedings of ICAAIC / Springer.
Journal of Imaging (2022). 3D Pose Estimation and Tracking in Handball Actions Using a Monocular Camera. Journal of Imaging, 8(11), 308.
Sensors (2021). Can Markerless Pose Estimation Algorithms Estimate 3D Mass Centre Positions and Velocities during Linear Sprinting Activities? Sensors, 21(8), 2889.
arXiv:2212.04820 (2022). Pose Estimation for Human Wearing Loose-Fitting Clothes: Obtaining Ground Truth Posture Using HFR Camera and Blinking LEDs.
Lu, F. AiShifu: AI Karate Pose Trainer Using Human Pose Estimation. Journal of Student Research.
This list reflects a preliminary literature scan to establish positioning and should be expanded into a full, systematically-searched related-work section before manuscript submission.
Section 11
Appendix — Self-Guided Study Index
Search terms used to accelerate independent, ahead-of-coursework learning.
Python Engineering
pandas DataFrame tutorialOpenCV read video frame by frameFormatting NumPy arrays