You will be provided with a reference and some statements. Please determine whether each statement is 'supported', 'unsupported', or 'unknown' with respect to the reference. Please note:
First, assess whether the reference contains any valid content. If the reference contains no valid information, such as a 'page not found' message, then all statements should be considered 'unknown'.
If the reference is valid, for a given statement: if the facts or data it contains can be found entirely or partially within the reference, it is considered 'supported' (data accepts rounding); if all facts and data in the statement cannot be found in the reference, it is considered 'unsupported'.

You should return the result in a JSON list format, where each item in the list contains the statement's index and the judgment result, for example:
[
    {
        "idx": 1,
        "result": "supported"
    },
    {
        "idx": 2,
        "result": "unsupported"
    }
]

Below are the reference and statements:
<reference>
This CVPR paper is the Open Access version, provided by the Computer Vision Foundation.
Except for this watermark, it is identical to the accepted version;
the final published version of the proceedings is available on IEEE Xplore.

From 3D Pose to Prose: Biomechanics-Grounded Vision–Language Coaching
Yuyang Ji1 Yixuan Shen1 Shengjie Zhu2 Yu Kong2 Feng Liu1B
1
Department of Computer Science, Drexel University
2
Department of Computer Science and Engineering, Michigan State University
Streaming
video

0s

Exercise example: Jumping Jack

Existing VLM methods

30s
Vision
Module

...

...

...

Looking good

Pay attention to the
range of motion in
your arms

Move your arms
and legs together

Pay attention to the
range of motion in
your arms

Land softly; keep
knees tracking toe;
arms overhead >
160 degree

Raise arms to at least
160–170° shoulder
flexion; keep elbows
<15° bend

Maintain hip–knee–
ankle alignment
within 5° during
landing

Left–right arm
elevation mismatch
>15°, synchronize
shoulder flexion

...
LLM

Visual Stream

...

...

...

...

Vision
Module
LLM

3D Skeletal Kinematics

Biomechanics
Module

BioCoach (Ours)

Figure 1. Comparison with existing methods. Top: prior pixel-only VLM methods provide generic, loosely timed comments. Bottom:
BioCoach fuses visual features with 3D skeletal kinematics and a biomechanics module to produce phase-aligned, anatomy-specific,
quantitative cues (e.g., shoulder flexion 160◦ –170◦ ), yielding more precise and biomechanics-grounded feedback along the same timeline.

1. Introduction

Abstract

Live fitness coaching through streaming video is increasingly important, with a broad impact on at-home workouts [44], injury prevention [15, 31], and data-driven rehabilitation [1, 5, 38]. Traditionally, fitness coaching has
relied on in-person trainers who provide immediate feedback to correct form deviations and reduce risks; however,
expert coaching is expensive and often inaccessible. Advances in computer vision now enable automated form assessment from video [13, 36], yet existing approaches face
three critical challenges that are also the field’s key requirements: (1) Timing and interaction, detecting coachable moments and delivering feedback at the right time rather than
post hoc; (2) Biomechanical grounding, reasoning over 3D
poses, joint angles, ranges of motion, and exercise phases
instead of high-level appearance; and (3) Personalization
and explainability, producing traceable, rule-level evidence.
Recent vision-language models have revitalized multimodal understanding [4, 42, 48] and can generate fluent,
instruction-like feedback [8, 27]; streaming VLMs such as
Stream-VLM [32] apply these capabilities to live fitness
coaching. However, they remain largely prompt-driven and
struggle to autonomously surface coachable moments; they
lack explicit morphometric context for personalization; and
they do not integrate symbolic biomechanical constraints,
which can yield generic or mistimed advice (see Fig. 1).

We present BioCoach, a biomechanics-grounded vision–
language framework for fitness coaching from streaming
video. BioCoach fuses visual appearance and 3D skeletal kinematics, through a novel three-stage pipeline: an
exercise-specific degree-of-freedom selector that focuses
analysis on salient joints; a structured biomechanical context that pairs individualized morphometrics with cycle
and constraint analysis; and a vision–biomechanics conditioned feedback module that applies cross-attention to generate precise, actionable text. Using parameter-efficient
training that freezes the vision and language backbones,
BioCoach yields transparent, personalized reasoning rather
than pattern matching. To enable learning and fair evaluation, we augment QEVD-fit-coach with biomechanicsoriented feedback to create QEVD-bio-fit-coach, and we
introduce a biomechanics-aware LLM judge metric. BioCoach delivers clear gains on QEVD-bio-fit-coach across
lexical and judgment metrics while maintaining temporal
triggering; on the original QEVD-fit-coach, it improves text
quality and correctness with near-parity timing, demonstrating that explicit kinematics and constraints are key to
accurate, phase-aware coaching. Project
B fl397@drexel.edu

23506

To address these limitations, we propose BioCoach,
a biomechanics-grounded vision-language framework for
streaming fitness coaching that bridges biomechanical analysis and multimodal understanding. Our core insight is
to construct explicit, interpretable intermediate representations that expose kinematic properties to the language
model, enabling transparent reasoning about exercise form
while preserving end-to-end trainability. Rather than treating visual appearance and 3D pose as disjoint streams or relying solely on pattern learning, BioCoach organizes them
into a structured pipeline that grounds feedback in biomechanical principles and individual morphology (see Fig. 1).
Specifically, BioCoach extracts two complementary signals from streaming video: a visual stream that captures
appearance and context, and a 3D kinematic stream that
captures skeletal pose and body shape. The system then
proceeds through three coordinated steps, each introducing a novel design. First, an exercise-specific joint attention mechanism prioritizes the degrees of freedom relevant to the current exercise, focusing analysis on anatomically salient regions instead of treating all joints uniformly.
Second, a structured biomechanical context pairs morphometric information for personalization with cycle analysis
against curated references to yield explicit, rule-based form
cues, which replace implicit visual heuristics with verifiable
evidence. Third, a vision-biomechanics conditioning mechanism fuses visual evidence with this context to produce
coaching that is temporally targeted, interpretable through
intermediate representations, and grounded in biomechanical principles rather than relying solely on pattern matching. Moreover, to support learning and evaluation, we create
QEVD-bio-fit-coach by re-annotating the QEVD-fit-coach
dataset [32] with fine-grained biomechanical annotations.
In summary, the contributions of this work include:
⋄ We propose BioCoach, a new biomechanics-grounded
vision-language framework for interpretable, personalized
coaching from streaming video.
⋄ We devise a three-stage pipeline unifying exercisespecific joint attention, biomechanical context for personalized rule-level cues, and vision–biomechanics conditioned
feedback for targeted, auditable coaching.
⋄ We create QEVD-bio-fit-coach by re-annotating
QEVD-fit-coach with fine-grained biomechanical labels.
⋄ Extensive experiments show that BioCoach achieves
superior text quality, timing accuracy, LLM-graded correctness, and biomechanics-grounded action quality scores.

tamped coaching feedback. Recent vision-language models—including Flamingo [2], InstructBLIP [8], LLaVA
variants [23, 51], and LLaMA-VID [21]—demonstrate
strong video understanding via gated cross-attention and
token-level fusion. Stream-VLM [32], a strong QEVD
baseline, introduced asynchronous interaction with action
tokens to enable unprompted feedback. However, these
models operate primarily on pixel-level features without explicit biomechanical constraints, leading to feedback that
can be generic or poorly timed. BioCoach addresses this by
extracting and integrating 3D skeletal kinematics as a structured modality, enabling biomechanics-grounded coaching.
Pose, Shape Estimation, and Biomechanical Analysis.
Advances in 3D human pose and mesh recovery span classic HMR [17], PARE [19], and CLIFF [22], and extend to
recent SMPL-X–based methods that improve temporal stability and controllability, including PromptHMR [43] and
joint-feature–guided HMR [47], and CLIP-guided shape
learning [26], with contemporary surveys summarizing the
state of the art [30, 39]. Complementary efforts push for
biomechanically faithful modeling, such as SKEL [18] and
HSMR [45], as well as gait models like Generative GaitNet [33] and Bidirectional GaitNet [34] that condition on
anatomical factors. Toolchains like OpenSim [9] and OpenCap [41] estimate kinematics and even kinetics from video,
enabling musculoskeletal analyses outside the lab. Prior fitness/AQA systems [6, 11, 35, 37] still tend to output scores
or templates, analyze frames rather than full cycles, and ignore morphology [25].
Structured Representations for Language Grounding.
Structured interfaces for grounding language in perception span marker/slot prompts that tie text to localized evidence [46], layout- or schema-driven planners that impose
explicit spatial structure before generation [12], and visual
in-context prompting that conditions models on exemplar
prompts across tasks [20]. Recent “visual chain-of-thought”
work equips multimodal LMs with sketchpads to externalize intermediate spatial reasoning [14]. In human motion, motion–language models discretize continuous kinematics into tokens to align motion and text within a single
generative interface [16]. Together, these directions show
that explicit, compact structure can improve interpretability, controllability, and citation of evidence—key properties
for corrective feedback on fine-grained physical behaviors.

3. Method
3.1. Overview

2. Related Work

The proposed BioCoach framework converts streaming fitness videos into biomechanically-grounded coaching feedback by creating explicit, interpretable intermediate representations that bridge kinematic data and language generation. This design enables transparent reasoning about ex-

Interactive Fitness Coaching and Vision-Language
Models.
Interactive fitness coaching from video has
emerged as a key research direction with benchmarks
like QEVD [32] that pair long-range videos with times-

23507

Visual Appearance Backbone
Input Video

0s

3D CNN
Visual Tokens

Sec. 3.5 Vision-Biomechanics Conditioned Feedback Generation
Vision-Morphometric Cross-Attention
Q

x

Prompt
K

V

30s

0s

+

LLM Integration with
Biomechanical Instruction.

Keep elbows
smaller 15°

3D Skeletal Kinematics Backbone
SKEL-based
Pose Estimator

Feedback

LLM Backbone
i.e., LLaMA

Sec. 3.3 Exercise-Specific DoF
Selection

Top K Joints

Biomechanical Joint Angles
& Shape Coefficients

Sec. 3.4 Structured Biomechanical
Context Generation
Individual
Morphometric
Context Module

Motion Quality
Context Module

30s

Figure 2. BioCoach overview. Streaming video is encoded by two backbones: a 3D CNN for visual tokens and a pose extractor for 3D
skeletal kinematics. The pipeline has three components: (1) Exercise-Specific DoF Selection uses a lightweight attention head to select the
top K biomechanically salient joints; (2) Structured Biomechanical Context builds two representations (individual morphometric context
and motion quality context) capturing body measurements, cycles, ranges of motion, and constraint checks; (3) Vision–Biomechanics
Conditioned Feedback fuses visual tokens with the morphometric context via cross-attention and prepends the motion-quality context as
structured instruction to the LLM. This yields feedback grounded in explicit kinematic evidence rather than pattern matching alone.

extractor, V[t−τ :t] denotes a sliding window of τ consecu∈ RNv ×d contains Nv visual feature
tive frames, and Fvis
t
tokens with an embedding dimension of d. The architecture
combines 2D and 3D convolutional layers: 2D layers capture spatial appearance cues within frames, while 3D layers
encode temporal motion dynamics across the window. Crucially, all convolutions employ causal masking to ensure the
model operates in a true streaming setting, generating predictions solely from past and current observations without
access to future information.

ercise form, personalization to individual body geometry,
and feedback grounded in biomechanical principles rather
than learned patterns. As illustrated in Fig. 2, the framework extracts two complementary modalities from streaming video: visual appearance and 3D skeletal kinematics.
These flow through a three-stage pipeline: (1) ExerciseSpecific Degree-of-Freedom Selection Module (Sec. 3.3)
identifies anatomically salient joints; (2) Structured Biomechanical Context Generation Module (Sec. 3.4) analyzes
motion quality while accounting for individual body geometry; (3) Vision-Biomechanics Conditioned Feedback Generation (Sec. 3.5) generates feedback grounded in explicit
biomechanical analysis.

3.2. Dual-Modality Feature Extraction

3D Skeletal Kinematic Backbone.
Complementary
to visual appearance features, we extract explicit 3D
skeletal kinematics to capture biomechanically-grounded
body motion.
We employ HSMR [45] built upon
SKEL [18], a frame-level method that produces pose estimates in biomechanically-grounded representations. Following SKEL [18], we represent skeletal pose using 46dimensional Euler-angle representations with joint-specific
biomechanical constraints. To extract temporal kinematic
information, we apply the pose extractor P to each frame in
the temporal window V[t−τ :t] :

Our framework extracts two complementary modalities
from streaming fitness videos. Both modalities originate
from the video input V ∈ RT ×H×W ×3 but capture different aspects of motion: visual appearance and 3D skeletal kinematics. To extract these modalities, we employ two
specialized processing backbones, described below.
Visual Appearance Backbone. Following [32], we employ a pre-trained 3D CNN that captures fine-grained motion patterns essential for fitness coaching (see Supp for
details). At each timestep t, the model processes a temporal window of video frames and extracts motion-aware
features:
  \mathbf {F}_t^{vis} = \mathcal {F}(\mathbf {V}_{[t-\tau :t]}), 
(1)

  \{\mathbf {q}_i\}_{i=1}^{\tau }, \{\boldsymbol {\beta }_i\}_{i=1}^{\tau } = \mathcal {P}\left (\mathbf {V}_{[t-\tau :t]}\right ), 

(2)

where F : Rτ ×H×W ×3 → RNv ×d is the 3D CNN feature

where qi ∈ R46 represents biomechanical joint angles for
frame i. To obtain a stable body shape representation across
the temporal window, we aggregate per-frame
shape coeffiPτ
cients via average pooling: β̄ = τ1 i=1 β i , expressed in
SMPL-style parameterization. Thus, the kinematic
 modalτ
ity output at time t is: Pskel
=
{q
}
,
β̄
, where
i i=1
t

23508

β̄ ∈ R10 encodes individual body shape. Unlike appearance
features that conflate body geometry with motion quality,
skeletal kinematics provide normalized, biomechanicallyaware representations that separate body shape from movement patterns.

Motion Quality Context Module

3D Skeletal Kinematics

(a) Cycle Detection

3.3. Exercise-Specific Degree-of-Freedom Selection
The biomechanical relevance of each joint depends on
the exercise being performed. A squat requires lowerbody joints (hips, knees, ankles), while a push-up demands
upper-body joints (shoulders, elbows, wrists). This module
identifies exercise-specific salient joints via visual context.
We leverage visual features Fvis
to infer exerciset
dependent joint relevance using a lightweight attention network Aθ :
  \mathbf {s}^t = \mathcal {A}_\theta (\mathbf {F}_t^{\text {vis}}), 
(3)

Gray: Golden Standard

Feedback
Point

Blue: User

(c) Biomechanical Constraint Evaluation
Dynamic Joints
✓

Match reference motion
𝑟𝑟𝑟𝑟𝑟𝑟
|𝜃𝜃𝑗𝑗 − 𝜃𝜃𝑗𝑗 | < 𝛿𝛿

×

Figure 3. Motion-Quality Context module. Given the selected
joint set and the 3D skeletal kinematics, the module (a) detects
repetition cycles and anchors the feedback moment; (b) timenormalizes each cycle and aligns it to a curated reference trajectory; and (c) evaluates biomechanical constraints: stability for
static joints and deviation to reference for dynamic joints. Gray
curves denote the reference, blue curves denote the user.

Euclidean distance between anatomical landmarks [7]. This
grounds shape in physical quantities that are semantically
meaningful for exercise assessment. The extracted measurements form the morphometric context Cmorph which is
formatted as human-readable descriptors:

Skeleton joints correspond to multiple degrees of freedom in the 46-dimensional Euler angle representation q
(e.g., the shoulder has 3 degrees of freedom (DoFs): flexion/extension, abduction/adduction, and internal/external
rotation). When a joint is selected in J ∗ , all its associated
DoFs are automatically included in downstream analysis,
ensuring coherent joint-level reasoning. The focused joint
set J ∗ remains fixed for the entire exercise session, mimicking how human coaches maintain consistent attention to
relevant body regions throughout a movement.

User body: height 1.78 m, mass 73.22 kg,
chest 1.00 m, waist 0.83 m, hips 0.98 m.

By providing explicit, quantitative anchors grounded in
individual body geometry, the morphometric context enables personalized and biomechanically-aware feedback.
This representation is fused with visual features via crossattention, allowing the language model to generate coaching
feedback that accounts for individual differences.

3.4. Structured Biomechanical Context Generation

3.4.2. Motion Quality Context Module

This module generates subject-specific structured biomechanical contexts from kinematic data to enable personalized and biomechanically-grounded coaching. It consists of
two submodules that, together, produce the individualized
context required for the language module.

Kinematic data must be contextualized within individual
body geometry. This module extracts user-specific anthropometric measurements from the shape parameters β̄
of the kinematic modality Pskel
. Raw shape parameters
t
(i.e., SMPL [28] body shape coefficients β̄) are abstract
and difficult for language models to interpret. To address
this, we leverage Virtual Measurements [7] to extract interpretable anthropometric measurements directly from the
fitted SMPL mesh: mass, height, chest, waist, and
hip circumference. Circumference measurements are derived via plane intersection, while length measurements use

Match

Cycle

Should remain stable
𝑉𝑉𝑉𝑉𝑉𝑉 𝜃𝜃𝑗𝑗𝑡𝑡 < 𝜖𝜖

(4)

3.4.1. Individual Morphometric Context Module

Match

Static Joints

where Aθ is an MLP with learnable parameters θ that outputs importance scores for each joint. Each element stj ∈
[0, 1] represents the biomechanical relevance of joint j. We
select the top-K joints by importance:
  \mathcal {J}^* = \{j : \mathbf {s}_j^t \in \text {TopK}(\mathbf {s}^t, K)\}. 

(b) Reference Alignment

As illustrated in Fig. 3, this module evaluates whether the
user’s motion satisfies biomechanical constraints for the
given exercise, using the kinematic trajectory {qi }τi=1 and
the selected joint set J ∗ from Sec.3.3, and produces structured descriptions of motion quality and form deviations.
Cycle detection. Motion cycles are identified by analyzing
key joints in J ∗ . At each frame i in the session, we have
the full angle vector qi ∈ R46 ; we denote the primary DoF
∗
∗
angle of joint j ∈ J ∗ at frame i as qj,i
, where qj,i
= qd(j),i
and d(j) maps joint j to its primary DoF index in qi (e.g.,
d(knee) = 7 for knee flexion). For repetitive exercises
(e.g., squats, push-ups), we select a representative joint j
and extract its angle trajectory {qj,i }N
i=1 over the entire session. After gaussian smoothing, we identify cycle peaks and
valleys using prominence-based peak detection, producing
candidate cycle boundaries (is , ie ). Spurious detections are
filtered by excluding cycles that are shorter than τmin or

23509

vviolations = {“jointj deviation: δj (bound: [lj , uj ])” |
violationj = 1}. These are formatted as natural language:

longer than τmax . For exercise-specific variants: alternating movements (e.g., jumping jacks) use zero-crossing detection within the detected cycle. WePcompute the cycleie
q and detect
specific mean angle q̄j,cycle = ie −i1s +1 i=i
s j,i
zero-crossings of the normalized signal qj,i − q̄j,cycle . Static
holds (e.g., planks) identify low-variance regions where
Var({qj,i }i∈[is ,ie ] ) < ϵ, indicating that the joint remains
relatively stable within the detected cycle.
Reference Alignment. Each detected cycle (is , ie ) is temporally normalized by resampling to match a curated reference trajectory R via linear interpolation. For joint j, we
interpolate sequences:
  \tilde {q}_{j,k} = q_{j, \lfloor \phi (k) \rfloor } + (\phi (k) - \lfloor \phi (k) \rfloor )(q_{j, \lceil \phi (k) \rceil } - q_{j, \lfloor \phi (k) \rfloor }), 
(5)
where ϕ(k) maps the reference index k ∈ [1, Nref ] to the
user’s cycle timeline i ∈ [is , ie ], and Nref is the reference
cycle length. Function ϕ rescales cycles to align with the
reference length, ensuring that both trajectories map to a
common temporal grid. The subscript j indicates the scalar
angle trajectory for joint j, not the full 46-dimensional vector. We compute similarity scores by comparing the interpolated user cycle {q̃j,k } with the reference trajectory, which
includes (1) cosine similarity on normalized angle profiles,
(2) joint-specific Pearson correlations, (3) temporal derivative alignment measuring velocity consistency, and (4) amplitude comparison of the range of motion. These metrics
yield a cycle quality score scycle ∈ [0, 1].
Biomechanical Constraint Evaluation.
For joint
j ∈ J ∗ , we evaluate exercise-specific biomechanical constraints based on the detected cycle (is , ie ). Each joint
is classified as static (should remain stable) or dynamic
(should exhibit specific motion patterns), as determined by
domain experts or LLM-assisted annotation (see Supp for
details). For static joints, we measure stability via variance: δjstatic = Var({qj,i }i∈[is ,ie ] ) For dynamic joints, we
measure deviation from the reference trajectory at critical
frames: δjdynamic = |qjuser (ikey ) − qjref (ikey )|, where ikey denotes critical frames (e.g., the deepest point of a squat). Violations are detected by checking against exercise-specific
acceptable bounds [lj , uj ]:
  \text {violation}_j = \begin {cases} 1, & \text {if } \delta _j < l_j \text { or } \delta _j > u_j \\ 0, & \text {otherwise}. \end {cases} 

(6)

When detected (violationj = 1), we generate descriptions
quantifying deviations and corrections.
Structured Prompt Generation. At each frame, the module produces two outputs. Pose state captures selected joint
configurations:
  \mathbf {p}_{\text {state}}^{(i)} = \{\texttt {joint}_{j}\text { angle } \lfloor q_{j,i} \rfloor ^{\circ } \mid j \in \mathcal {J}^*\}, 

(7)

For joints where violations are detected (violationj =
1), we generate descriptions quantifying deviations:

Current pose: right knee 85◦ , left knee
88◦ , hip 75◦ .
Form issues: Right knee flexion
insufficient (85◦ detected, 90◦ required)

Motion quality context concatenates pose state and violations:
  \mathcal {C}_{\text {motion}} = [\mathbf {p}_{\text {state}}^{(i)}; \mathbf {v}_{\text {violations}}], 
(8)
which enables precise feedback grounded in biomechanical
analysis.

3.5. Vision-Biomechanics Conditioned Feedback
Generation
This module synthesizes coaching feedback by conditioning a language model (i.e., LLaMA-2-7B [40]) on visual
features Fvis
t , morphometric context Cmorph , and motion
quality context Cmotion .
Structured Multimodal Context Encoding. Both morphometric and motion contexts are encoded into token embeddings through the language model’s embedding layer:
  \mathbf {m}_t = \text {Embed}(\mathcal {C}_{\text {morph}}), \quad \mathbf {c}_t = \text {Embed}(\mathcal {C}_{\text {motion}}), 

(9)

where mt encodes body measurements and pose, and ct
encodes motion quality and constraint violations.
Vision-Morphometric Cross-Attention.
Visual feaare
grounded
in
morphometric
tokens
m
tures Fvis
t through
t
cross-attention:
  \mathbf {z}_t = \mathbf {F}_t^{\text {vis}} + \text {CrossAttn}(\mathbf {F}_t^{\text {vis}}, \mathbf {m}_t, \mathbf {m}_t), 

(10)

where
  \resizebox {\linewidth }{!}{$ \operatorname {CrossAttn}\!\left (\mathbf {F}_t^{\text {vis}}, \mathbf {m}_t, \mathbf {m}_t\right ) = \operatorname {Softmax}\!\left ( \frac {\mathbf {F}_t^{\text {vis}} W_Q (\mathbf {m}_t W_K)^\top }{\sqrt {d}} \right )\cdot (\mathbf {m}_t W_V) $}, \label {eq:crossattn} 
(11)
with WQ , WK , WV being learnable projection matrices.
This produces fused visual-morphometric features zt that
align visual observations with individual body geometry.
LLM Integration with Biomechanical Instruction. Motion quality context Cmotion encapsulates rich biomechanical
analysis, including cycle-based motion quality, constraint
violations, and quantified form deviations. This context is
prepended as structured instruction to directly guide language generation with explicit biomechanical constraints:
  \text {Prompt} = [\text {Embed}(\mathcal {C}_{\text {motion}}), \text {language\_tokens}]. 

(12)

The LLM then generates feedback conditioned on this explicit biomechanical analysis:

23510

  \text {Feedback}_t = \text {LLM}(\text {Prompt}. \{\mathbf {z}_t\}). 

(13)

By injecting structured biomechanical constraints directly
into the prompt rather than relying solely on learned patterns, this design ensures that feedback is grounded in
explicit, exercise-specific biomechanical principles. (See
Supp for mathematical details.)

technical. This controlled augmentation isolates the effect
of explicit biomechanical terminology and enables evaluation of whether kinematic grounding improves coaching
quality.

3.8. Implementation Details

3.6. Training Objective
We employ a parameter-efficient fine-tuning strategy that
freezes both the 3D CNN visual backbone and the LLaMA2-7B language model, updating only the cross-attention fusion layers and the DoF selection network Aθ . This design
preserves pre-trained linguistic priors while adapting multimodal fusion to align visual appearance, skeletal kinematics, and coaching feedback.
Exercise-Specific DoF Selection Training. The attention
network Aθ is trained with annotated exercise-specific joint
relevance labels from domain experts and LLM-assisted annotation. We use binary cross-entropy loss to align predicted importance scores st with ground-truth salient joint
sets J gt :
  \mathcal {L}_{\text {DoF}} = -\sum _{j=1}^{J} \left [ y_j \log (\mathbf {s}_j^t) + (1-y_j) \log (1-\mathbf {s}_j^t) \right ],  (14)
where yj = 1 for biomechanically-relevant joints and yj =
0 otherwise.
Cross-Attention Fusion. Training mirrors the streaming
inference regime using autoregressive cross-entropy loss
with selective down-weighting to prevent excessive deferral of feedback:

Following Stream-VLM [32], we adopt LLaMA-2-7B [40]
as the language backbone and a pre-trained 3D CNN for
visual features. Skeletal kinematics are represented by 46dimensional Euler angles with a temporal window of 3 seconds (τ = 12) of motion history to detect exercise cycles
and evaluate biomechanical constraints. The DoF-selection
network Aθ is a 3-layer MLP (ReLU activations, sigmoid
output) that produces per-joint importance scores st ∈
[0, 1]J ; we select the top K = 12 salient joints per exercise.
We train two variants: one on QEVD-fit-coach (original
feedback) and one on QEVD-bio-fit-coach (biomechanicsgrounded feedback). Both fine-tune only the cross-attention
layers and Aθ using AdamW (learning rate 2 × 10−5 ),
a batch size of 8, and action-token down-weighting α =
0.1. DoF selection is supervised using a balanced binary
cross-entropy loss. For biomechanical analysis, cycle durations are constrained to [τmin , τmax ] = [0.8, 5] s. Constraint thresholds are exercise-specific: static-joint variance
< 5◦ ; dynamic-joint deviation tolerance ±5◦ to ±10◦ at
key frames. Videos are resized to H = 224, W = 160 with
standard augmentations. (See Supp for additional implementation details.)

4. Experiments
4.1. Experimental Setup

  \mathcal {L}_{\text {CE}} = -\sum _{t=1}^{N-1} w_{x_{t+1}} \log P(x_{t+1} \mid x_{\le t}), 

(15)

where wxt+1 = α < 1 for continuation tokens (<next>
action tokens) and wxt+1 = 1 for feedback content. This
down-weighting encourages timely feedback generation
rather than indefinite observation windows.

Dataset. We evaluate on QEVD-fit-coach [32], a streaming
fitness-coaching benchmark with 74 test videos across 23
exercises and ∼2,484 timestamped feedback instances. To
better assess biomechanical understanding, we also report
results on the QEVD-bio-fit-coach.
Baselines: We compare BioCoach against state-of-the-art
vision–language models in two settings:
• Zero-shot models: InstructBLIP [8], Video-LLaVA [23],
Video-ChatGPT [29], Video-LLaMA [49], LLaMAVID [21], and LLaVA-NeXT [51]. These models are
evaluated without any fitness-domain tuning and are periodically prompted to generate feedback from the video
stream.
• Fine-tuned models: Methods trained on the QEVD-fitcoach benchmark [32]:
– Socratic-LLaMA-2-7B [32]: A text-only baseline that
feeds activity descriptors from a fine-tuned 3D CNN
into LLaMA-2-7B to produce feedback.
– Video-ChatGPT [29] and LLaMA-VID [21]: Turnbased VLMs fine-tuned on QEVD-FIT-COACH that

3.7. QEVD-bio-fit-coach Dataset
We augment QEVD-fit-coach [32] (149 training videos, 74
test videos, 23 exercises, ∼2,484 timestamped feedback) to
create QEVD-bio-fit-coach. The enhanced set adds biomechanically grounded feedback annotations. Using these
signals, we systematically rewrite colloquial feedback into
anatomically precise language: generic cues like “lower
your body more” become quantified targets (“increase elbow flexion to 90◦ at the bottom”), and vague instructions
become explicit alignments (“maintain a neutral spine, head
to heels”). We also add brief biomechanical rationales (e.g.,
“increase hip/knee flexion to distribute load”). The temporal boundaries of all feedback instances are preserved exactly as in the original; only the content is rendered more

23511

Table 1. Evaluation on the QEVD-bio-fit-coach, a newly created benchmark with fine-grained biomechanical ground-truth feedback annotations. Second line in each cell shows % improvement vs. Stream-VLM [32]. LLM-Bio-Acc. is our LLM-as-judge metric tailored to
biomechanics, assessing the biomechanical correctness and specificity of generated feedback.
Method
Stream-VLM [32]
(NeurIPS ’24)
BioCoach

METEOR ↑

ROUGE-L ↑

BERTScore ↑

LLM-Acc. ↑

LLM-Bio-Acc. ↑

T-F-Score ↑

0.086

0.108

0.852

1.86

1.72

0.530

0.312
(+262.8%)

0.302
(+179.6%)

0.877
(+2.9%)

3.12
(+67.7%)

3.26
(+89.5%)

0.544
(+2.6%)

Table 2. Performance on the QEVD-fit-coach with original feedback annotations. Parenthesized values are % change vs. the best baseline
(Stream-VLM [32]). † zero-shot without fine-tuning.

Method
Zero-shot Models
InstructBLIP† [8]
Video-LLaVA† [23]
Video-ChatGPT† [29]
Video-LLaMA† [49]
LLaMA-VID† [21]
LLaVA-NeXT† [51]
Fine-tuned Models
Socratic-LLaMA-2-7B [40]
Video-ChatGPT [29]
LLaMA-VID [21]
Stream-VLM [32] (NeurIPS ’24)
BioCoach

METEOR ↑

ROUGE-L ↑

BERTScore ↑

LLM-Acc. ↑

T-F-Score ↑

0.047
0.057
0.098
0.101
0.100
0.104

0.040
0.025
0.078
0.077
0.079
0.078

0.839
0.847
0.850
0.859
0.859
0.858

1.56
2.16
1.91
1.29
2.20
2.27

-

0.094
0.108
0.106
0.127
0.129 (+1.6%)

0.071
0.093
0.090
0.112
0.122 (+8.9%)

0.860
0.863
0.860
0.863
0.864 (+0.1%)

2.17
2.33
2.30
2.45
2.56 (+4.5%)

0.50
0.50
0.50
0.56
0.544 (-2.9%)

human preferences than purely lexical metrics.
• Temporal F-Score (T-F-Score) [32]: time-alignment precision/recall F1 indicating whether feedback is issued at
the correct moments.
In addition, to evaluate biomechanical fidelity on QEVDbio-fit-coach, we introduce LLM-Bio-Accuracy, an LLMas-judge metric specialized for biomechanics. Using
the same LLaMA-3-70B-Instruct with a domain-specific
prompt, it scores (1–5) factual biomechanical correctness and relevance against the reference context (implicitly reflecting per-frame kinematics, phase labels, and constraint annotations), emphasizing precise anatomy (joint
names/sides), angles, range of motion, and phase-aware reasoning (see Supp for more details).

use CLIP/ViT visual encoders and require prompting
at fixed intervals.
– Stream-VLM [32]: The original streaming baseline on
QEVD-FIT-COACH, combining a 3D CNN vision encoder with a LLaMA-2 language model and special
<next>/<feedback> tokens for interactive, timely
responses. This is the strongest, fine-tuned baseline tailored for fitness coaching.
Metrics. Following Stream-VLM [32], we report:
• METEOR [3]: lexical overlap with stemming, synonym,
and paraphrase matching. Rewards correct technical
terms and entities (e.g., if the reference says “arm not
moving,” predictions should mention the arm).
• ROUGE-L [24]: longest–common–subsequence overlap,
capturing phrase ordering and multiword expression similarity.
• BERTScore [50]: semantic similarity via contextual embeddings and cosine similarity, robust to paraphrasing beyond exact word overlap.
• LLM-Accuracy [32]: LLaMA-3-70B-Instruct [10] acts as
an automatic judge to holistically rate each feedback on
a 1–5 scale (higher is better), considering context, correctness, and appropriateness; this correlates better with

4.2. Results on QEVD-bio-fit-coach

23512

To assess biomechanics-grounded feedback on streaming
video, we evaluate BioCoach on the new QEVD-bio-fitcoach benchmark and compare it to the strongest streaming baseline, Stream-VLM [32], which has been fine-tuned
on the same annotations for fairness (Tab. 1). This benchmark stresses anatomy/kinematics-centric language, quantitative ranges of motion, and phase-aware cues—precisely
what safe, actionable coaching requires.

Table 3. Ablation study on QEVD-Bio-Fit-Coach. Each row removes or modifies one component while keeping others fixed.
METEOR ↑

ROUGE-L ↑

BERTScore ↑

LLM-Acc. ↑

LLM-Bio-Acc. ↑

T-F-Score ↑

Full Model (τ = 3 s )

0.312

0.302

0.877

3.12

3.26

0.544

w/o Exercise-Specific DoF Selection

0.305

0.296

0.873

3.06

3.14

0.543

w/o Motion Quality Context Module

0.133

0.127

0.864

2.63

2.04

0.544

w/o Individual Morph Context Module

0.284

0.273

0.871

2.91

3.07

0.535

Window size τ = 2 s

0.311

0.303

0.876

3.10

3.22

0.416

Model Variant

BioCoach markedly outperforms Stream-VLM across all
metrics, with the largest relative gains in lexical measures
(METEOR +262.8% and ROUGE-L +179.6%), reflecting improved alignment with expert references. Judgebased evaluations also rise sharply: LLM-Bio-Acc. shows
the strongest improvement (+89.5%) among judge metrics,
indicating biomechanics-grounded, anatomy- and phaseaccurate feedback. Temporal behavior remains strong (T-FScore), showing that richer biomechanical grounding does
not come at the expense of timely triggering. These gains
stem from exercise-focused joint selection, personalized
morphometrics, and cycle-aligned analysis that turn raw
kinematics into verifiable cues, shifting the system from
pattern matching to principled, auditable reasoning.
For a fair comparison of Stream-VLM [32], we re-train
BioCoach on QEVD-fit-coach (with the original annotations), i.e., without any of our biomechanical labels, and
evaluate it under the official protocol. This experiment
serves two purposes: (i) it isolates architectural gains from
annotation gains, showing that our biomechanics-grounded
design helps even when trained with non-biomechanical labels; and (ii) it demonstrates backward compatibility, indicating BioCoach can be deployed on datasets that lack
fine-grained biomechanical supervision. Under this setting
(Tab. 2), BioCoach still surpasses the strongest fine-tuned
baseline in text/semantic quality and judged correctness
(METEOR 0.129, ROUGE-L 0.122, BERTScore 0.864,
LLM-Acc. 2.56), while remaining near-parity in timing (TF-Score 0.544 vs. 0.56). Together with the large improvements in QEVD-bio-fit-coach, these results show that (a)
BioCoach improves quality under legacy annotations and
(b) adding biomechanical supervision further unlocks its
full potential for precise, anatomy-specific feedback.

provides modest gains (−3.7% LLM-Bio-Acc without it),
filtering irrelevant kinematics. Reducing the temporal window to 2 s maintains text quality but drops the T-F-Score
∼24%, confirming that 3 s offers better stability for cycle
detection. The three modules divide labor: motion quality
supplies what to say, morphometrics supply for whom, and
DoF selection supplies where to look—together enabling interpretable reasoning without sacrificing responsiveness.

4.4. Limitations and Future Work
Our approach depends on the quality of 3D skeletons and
shapes, as occlusions, loose clothing, and extreme viewpoints can distort kinematics. Additionally, it relies on
curated reference trajectories that may miss exercise variants and adaptive forms. Future work will explore multisensor fusion and broader reference coverage to mitigate
these issues. Beyond kinematics, we plan to extend BioCoach toward kinetic reasoning, estimating joint reaction
forces and muscle activation patterns from video to detect compensatory movement strategies that are invisible to
angle-only analysis. Integrating musculoskeletal simulation
could further enable load-aware coaching that flags injuryrisk forces, not just postural deviations

5. Conclusion

4.3. Ablation Study

We present BioCoach, a biomechanics-grounded vision–
language framework for streaming fitness coaching that exposes 3D kinematics, morphometrics, and constraint analyses to the language model. With an exercise-specific
DoF selector, a morphometric and motion-quality context, and a vision–biomechanics conditioned feedback module, BioCoach delivers interpretable coaching rather than
appearance-level heuristics. BioCoach improves lexical, semantic, and judge metrics while maintaining competitive
timing, demonstrating that explicit kinematics and structured biomechanical context are key to accurate, phaseaware, personalized coaching. This formulation opens a
path to coaching systems whose behavior can be inspected,
trusted, and aligned with expert musculoskeletal reasoning.

Ablations on QEVD-bio-fit-coach (Tab. 3) reveal a clear
component hierarchy. Removing Motion Quality Context
causes ∼57% METEOR drop and LLM-Bio-Acc. decline
(3.26 → 2.04), showing it is the core biomechanical driver.
Removing Morphometric Context causes moderate degradation (−9% METEOR, −6% LLM-Bio-Acc), indicating
that personalization improves specificity. DoF Selection

23513

Acknowledgments This work was supported in part by the
National Science Foundation under Grant FMitF-2319242.

References
[1] Ali Abedi, Tracey JF Colella, Maureen Pakosh, and
Shehroz S Khan. Artificial intelligence-driven virtual rehabilitation for people living in the community: A scoping
review. NPJ Digital Medicine, 7(1):25, 2024. 1
[2] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine
Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch,
Katherine Millican, Malcolm Reynolds, et al. Flamingo: a
visual language model for few-shot learning. NeurIPS, 2022.
2
[3] Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for mt evaluation with improved correlation
with human judgments. In Proceedings of the acl workshop
on intrinsic and extrinsic evaluation measures for machine
translation and/or summarization, 2005. 7
[4] Davide Caffagni, Federico Bianchi, Silvia Terragni, et al.
The revolution of multimodal large language models. In
ACL, 2024. 1
[5] Erdong Chen, Yuyang Ji, Jacob K Greenberg, Benjamin
Steel, Faraz Arkam, Abigail Lewis, Pranay Singh, and
Feng Liu. BioGait-VLM: A tri-modal vision-languagebiomechanics framework for interpretable clinical gait assessment. arXiv preprint arXiv:2603.08564, 2026. 1
[6] Steven Chen and Richard R Yang. Pose trainer: Correcting exercise posture using pose estimation. arXiv preprint
arXiv:2006.11718, 2020. 2
[7] Vasileios Choutas, Lea Müller, Chun-Hao P Huang, Siyu
Tang, Dimitrios Tzionas, and Michael J Black. Accurate 3D
body shape regression using metric and semantic attributes.
In CVPR, 2022. 4
[8] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat
Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale
Fung, and Steven Hoi. Instructblip: Towards generalpurpose vision-language models with instruction tuning.
NeurIPS, 2023. 1, 2, 6, 7
[9] Scott L. Delp, Frank C. Anderson, Allison S. Arnold, Peter
Loan, Ayman Habib, Chand T. John, Eran Guendelman, and
Darryl G. Thelen. OpenSim: Open-source software to create and analyze dynamic simulations of movement. IEEE
TBME, 54(11):1940–1950, 2007. 2
[10] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil
Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The
Llama 3 herd of models. arXiv preprint arXiv:2407.21783,
2024. 7
[11] Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre
Sermanet, and Andrew Zisserman. Counting out time: Class
agnostic video repetition counting in the wild. In CVPR,
2020. 2
[12] Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and
William Yang Wang. Layoutgpt: Compositional visual planning and generation with large language models. NeurIPS,
2023. 2
[13] Mihai Fieraru, Mihai Zanfir, Silviu Constantin Pirlea, Vlad
Olaru, and Cristian Sminchisescu. AIFit: Automatic 3D

human-interpretable feedback models for fitness training. In
CVPR, 2021. 1
[14] Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna.
Visual sketchpad: Sketching as a visual chain of thought for
multimodal language models. In NeurIPS, 2024. 2
[15] Jenny Jacobsson, Jan Kowalski, Toomas Timpka, Per-Olof
Hansson, Armin Spreco, and Orjan Dahlstrom. Universal
prevention through a digital health platform reduces injury
incidence in youth athletics (track and field): a cluster randomised controlled trial. British Journal of Sports Medicine,
57(6):364–371, 2023. 1
[16] Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and
Tao Chen. Motiongpt: Human motion as a foreign language.
NeurIPS, 2023. 2
[17] Angjoo Kanazawa, Michael J Black, David W Jacobs, and
Jitendra Malik. End-to-end recovery of human shape and
pose. In CVPR, 2018. 2
[18] Marilyn Keller, Keenon Werling, Soyong Shin, Scott Delp,
Sergi Pujades, C Karen Liu, and Michael J Black. From skin
to skeleton: Towards biomechanically accurate 3d digital humans. TOG, 2023. 2, 3
[19] Muhammed Kocabas, Chun-Hao P Huang, Otmar Hilliges,
and Michael J Black. Pare: Part attention regressor for 3d
human body estimation. In ICCV, 2021. 2
[20] Feng Li, Qing Jiang, Hao Zhang, Tianhe Ren, Shilong Liu,
Xueyan Zou, Huaizhe Xu, Hongyang Li, Chunyuan Li, Jianwei Yang, Lei Zhang, and Jianfeng Gao. Visual in-context
prompting. In CVPR, 2024. 2
[21] Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An
image is worth 2 tokens in large language models. In ECCV,
2024. 2, 6, 7
[22] Zhihao Li, Jianzhuang Liu, Zhensong Zhang, Songcen Xu,
and Youliang Yan. Cliff: Carrying location information in
full frames into human pose and shape estimation. In ECCV,
2022. 2
[23] Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng
Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. In EMNLP, 2024.
2, 6, 7
[24] Chin-Yew Lin. Rouge: A package for automatic evaluation
of summaries. In Text summarization branches out, 2004. 7
[25] Feng Liu, Minchul Kim, ZiAng Gu, Anil Jain, and Xiaoming
Liu. Learning clothing and pose invariant 3d shape representation for long-term person re-identification. In ICCV, 2023.
2
[26] Feng Liu, Minchul Kim, Zhiyuan Ren, and Xiaoming Liu.
Distilling clip with dual guidance for learning discriminative
human body shape representation. In CVPR, 2024. 2
[27] Haotian Liu, Chunyuan Li, et al. Visual instruction tuning.
In NeurIPS, 2023. 1
[28] Matthew Loper, Naureen Mahmood, Javier Romero, Gerard
Pons-Moll, and Michael J. Black. SMPL: A skinned multiperson linear model. TOG, 2015. 4
[29] Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. In ACL, 2024.
6, 7

23514

[30] Rama Bastola Neupane, Kan Li, and Tesfaye Fenta Boka. A
survey on deep 3d human pose estimation. Artificial Intelligence Review, 58(1):24, 2024. 2
[31] Patcharin Nilmart, Mantana Vongsirinavarat, Pimpaka
Khawsuwan, Kiatrada Chumthong, Russareeta Tadein, and
Dwi Rosella Komalasari. Impact of telehealth-based therapeutic exercise on pain, functional performance and dynamic
knee valgus in young adult females with patellofemoral pain:
a randomised controlled trial. BMJ Open Sport & Exercise
Medicine, 10(4), 2024. 1
[32] Sunny Panchal, Apratim Bhattacharyya, Guillaume Berger,
Antoine Mercier, Cornelius Böhm, Florian Dietrichkeit,
Reza Pourreza, Xuanlin Li, Pulkit Madan, Mingu Lee, et al.
What to say and when to say it: Live fitness coaching as a
testbed for situated interaction. NeurIPS, 2024. 1, 2, 3, 6, 7,
8
[33] Jungnam Park, Moon Seok Park, Jehee Lee, and Jungdam
Won. Generative gaitnet. TOG, 2022. 2
[34] Jungnam Park, Moon Seok Park, Jehee Lee, and Jungdam
Won. Bidirectional gaitnet: A bidirectional prediction model
of human gait and anatomical conditions. In SIGGRAPH,
2023. 2
[35] Paritosh Parmar and Brendan Tran Morris. What and how
well you performed? a multitask learning approach to action
quality assessment. CVPR, 2019. 2
[36] Paritosh Parmar, Amol Gharat, and Helge Rhodin. Domain knowledge-informed self-supervised representations
for workout form assessment. In ECCV, 2022. 1
[37] Hamed Pirsiavash, Carl Vondrick, and Antonio Torralba. Assessing the quality of actions. In ECCV, 2014. 2
[38] Jennifer Sumner, Hui Wen Lim, Lin Siew Chong, Anjali
Bundele, Amartya Mukhopadhyay, and Geetha Kayambu.
Artificial intelligence in physical rehabilitation: A systematic review. Artificial Intelligence in Medicine, 146:102693,
2023. 1
[39] Yating Tian, Hongwen Zhang, Yebin Liu, and Limin Wang.
Recovering 3D human mesh from monocular images: A survey. IEEE TPAMI, 45(12):15406–15425, 2023. 2
[40] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert,
Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov,
Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al.
Llama 2: Open foundation and fine-tuned chat models. arXiv
preprint arXiv:2307.09288, 2023. 5, 6, 7
[41] Scott D. Uhlrich, Antoine Falisse, Łukasz Kidziński, Julie
Muccini, Michael Ko, Akshay S. Chaudhari, Jennifer L.
Hicks, and Scott L. Delp. OpenCap: Human movement dynamics from smartphone videos. PLOS Computational Biology, 19(10), 2023. 2
[42] Jiaqi Wang, Hanqi Jiang, Yiheng Liu, et al. A comprehensive review of multimodal large language models: Performance and challenges across different tasks. arXiv preprint
arXiv:2408.01319, 2024. 1
[43] Yufu Wang, Yu Sun, Priyanka Patel, Kostas Daniilidis,
Michael J. Black, and Muhammed Kocabas. PromptHMR:
Promptable human mesh recovery. In CVPR, 2025. 2
[44] Jan Wilke, Lisa Mohr, Gustavo Yuki, Adelle Kemlall Bhundoo, David Jiménez-Pavón, Fernando Laiño, Niamh Murphy, Bernhard Novak, Stefano Nuccio, Sonia Ortega-Gómez,

et al. Train at home, but not alone: a randomised controlled
multicentre trial assessing the effects of live-streamed teleexercise during covid-19-related lockdowns. British journal
of sports medicine, 56(12):667–675, 2022. 1
[45] Yan Xia, Xiaowei Zhou, Etienne Vouga, Qixing Huang, and
Georgios Pavlakos. Reconstructing humans with a biomechanically accurate skeleton. In CVPR, 2025. 2, 3
[46] Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan
Li, and Jianfeng Gao. Set-of-mark prompting unleashes
extraordinary visual grounding in gpt-4v. arXiv preprint
arXiv:2310.11441, 2023. 2
[47] Xinhui Yao, Junyi He, Chang Li, Yang Xie, Haotian Luo,
Wei Zheng, Hongxing Qin, and Yiqun Wang. JFG-HMR:
3D joint feature-guided human mesh recovery with global–
local feature fusion. Computers & Graphics, 132:104339,
2025. 2
[48] Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun,
Tong Xu, and Enhong Chen. A survey on multimodal large
language models. National Science Review, 11(12), 2024. 1
[49] Hang Zhang, Xin Li, and Lidong Bing. Video-LLaMA: An
instruction-tuned audio-visual language model for video understanding. In EMNLP, 2023. 6, 7
[50] Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. BERTScore: Evaluating text generation with BERT. In ICLR, 2020. 7
[51] Yuanhan Zhang, Bo Li, Haotian Liu, Yong Jae Lee, Liangke
Gui, Di Fu, Jiashi Feng, Ziwei Liu, and Chunyuan Li.
LLaVA-NeXT: A strong zero-shot video understanding
model, 2024. Online. Available: https://llava-vl.
github . io / blog / 2024 - 04 - 30 - llava - next video/. 2, 6, 7

23515
</reference>

<statements>
1. and sports-specific evidence remains short-term, school-based, vendor-reported or benchmark-only
2. BioCoach is a biomechanics-grounded streaming fitness-coaching framework that fuses appearance with 3D skeletal kinematics through an exercise-specific degree-of-freedom selector, morphometric context, motion-quality context and vision–biomechanics cross-attention
3. On its newly created QEVD-bio-fit-coach benchmark, BioCoach reports METEOR 0.312, ROUGE-L 0.302, BERTScore 0.877, LLM-Accuracy 3.12, LLM-Bio-Accuracy 3.26 and T-F-Score 0.544, compared with Stream-VLM at 0.086, 0.108, 0.852, 1.86, 1.72 and 0.530
4. The largest judge gain is LLM-Bio-Accuracy at +89.5%
5. Ablations show that removing Motion Quality Context collapses LLM-Bio-Accuracy from 3.26 to 2.04, while removing DoF selection or morphometric context causes smaller declines
6. Reducing the temporal window from 3 s to 2 s maintains text quality but drops T-F-Score from 0.544 to 0.416
7. On the original QEVD-fit-coach benchmark, trained without its biomechanical labels, BioCoach reports METEOR 0.129, ROUGE-L 0.122, BERTScore 0.864 and LLM-Accuracy 2.56 versus Stream-VLM’s 0.127, 0.112, 0.863 and 2.45, but its T-F-Score is 0.544 versus Stream-VLM’s 0.56
8. The paper’s stated limitations are equally important: performance depends on 3D skeleton and shape quality, occlusions, loose clothing and extreme viewpoints can distort kinematics, and curated reference trajectories may miss exercise variants or adaptive forms
9. Future work includes multi-sensor fusion and kinetic reasoning over joint reaction forces, muscle activation and injury-risk forces
10. Fusion improves benchmark action recognition and can generate coaching text in research settings
11. But the evidence does not show that benchmark coaching models have been integrated into validated school tutoring loops, because the coaching papers report dataset-level results, while the school systems report operational deployment metrics
12. BioCoach generates phase-aligned, anatomy-specific cues grounded in kinematic constraints
13. The BioCoach paper frames streaming fitness coaching as relevant to at-home workouts, injury prevention and data-driven rehabilitation
14. The effectiveness picture must be separated into general ITS evidence, K-12 AI-ITS evidence, sports-specific PE evidence and technical benchmark evidence
15. Coaching VLM papers | ExpertAF BLEU-4 44.9; BioCoach LLM-Bio-Accuracy 3.26 | benchmark feedback quality, not classroom learning gains
16. The available evidence does not establish durable learning or health gains at scale, because the cited evidence base lacks longitudinal sports ITS efficacy studies and relies heavily on short-term, quasi-experimental, vendor or benchmark reports
17. BioCoach notes dependence on 3D skeleton quality, occlusions, loose clothing and extreme viewpoints
18. BioCoach relies on curated reference trajectories and does not yet model kinetic forces or muscle activation
19. Whether coaching-generation models can move from benchmark datasets to classroom loops with acceptable latency, privacy protection and teacher oversight remains open, given the separation between benchmark metrics and deployment metrics
20. Frontier coaching-generation evidence exists for multimodal video-plus-pose models and biomechanics-grounded feedback models
21. Coaching VLM papers report strong benchmark metrics
22. A defensible next step is to combine the cloud–edge–end architecture, the digital-base and platform layers, deployed gateway stacks and coaching-generation models
</statements>

Begin the assessment now. Output only the JSON list, without any conversational text or explanations.