You will be provided with a research report. The body of the report will contain some citations to references.

Citations in the main text may appear in the following forms:
1. A segment of text + space + number, for example: "Li Qiang constructed a socioeconomic status index (SES) based on income, education, and occupation, dividing society into 7 levels 15"
2. A segment of text + [number], for example: "Li Qiang constructed a socioeconomic status index (SES) based on income, education, and occupation, dividing society into 7 levels[15]"
3. A segment of text + [number†(some line numbers, etc.)], for example: "Li Qiang constructed a socioeconomic status index (SES) based on income, education, and occupation, dividing society into 7 levels[15†L10][5L23][7†summary]"
4. [Citation Source](Citation Link), for example: "According to [ChinaFile: A Guide to Social Class in Modern China](https://www.chinafile.com/reporting-opinion/media/guide-social-class-modern-china)'s classification, Chinese society can be divided into nine strata"

Please identify **all** instances where references are cited in the main text, and extract (fact, ref_idx, url) triplets. When extracting, pay attention to the following:
1. Since these facts will need to be verified later, you may need to look for some context before and after the citation to ensure that the fact is complete and understandable, rather than just a simple phrase or short expression.
2. If a fact cites multiple references, then it should correspond to two triplets: (fact, ref_idx_1, url_1) and (fact, ref_idx_2, url_2).
3. For the third form of citation (i.e., where the citation source and link appear directly in the text), the ref_idx should be uniformly set to 0.
4. If the main text does not specify the exact location of the citation (for example, only the reference list is listed at the end of the article, without specifying the citation point in the text), please return an empty list.

You should return a JSON list format, where each item in the list is a triplet, for example:
[
    {
        "fact": "Text segment from the original document. Note that Chinese quotation marks should use full-width marks. And add a single backslash before the English quotation mark to make it a readable for python json module.",
        "ref_idx": "The index of the cited reference in the reference list for this text segment.",
        "url": "The URL of the cited reference for this text segment (extracted from the reference list at the end of the research report or from the parentheses at the citation point)."
    }
]

Here is the main text of the research report:
# Construction and Application of a Sports Intelligent Tutoring and Learning Guidance System Driven by Multimodal Data Fusion

## Executive overview

Sports intelligent tutoring and learning guidance systems increasingly rely on multimodal data fusion to deliver real-time feedback, personalized interventions, and safer training experiences. Recent work in physical education, sports training, and intelligent tutoring demonstrates that combining heterogeneous data streams (video, audio, wearable sensors, textual instructions, and knowledge graphs) with deep learning and reinforcement learning yields higher recognition accuracy, more precise guidance, and measurable improvements in learning and health outcomes compared with single-modality or rule-based systems. This report synthesizes state-of-the-art approaches and proposes an integrated design for a sports intelligent tutoring and guidance system driven by multimodal data fusion.[1][2][3][4][5][6]

## Conceptual foundations

### Intelligent tutoring in education and sports

Intelligent Tutoring Systems (ITS) aim to simulate aspects of human teaching—diagnosing learner state, selecting appropriate content, and providing adaptive feedback—based on models of domain knowledge, student knowledge, and pedagogy. Traditional ITS depend on manually curated knowledge bases and structured input (text, quizzes), limiting their applicability in sports, where learning is embodied and heavily multimodal (movement, physiology, environment). Recent ITS research introduces automatic construction of multimodal knowledge graphs from course videos and textbooks and Retrieval-Augmented Generation (RAG) to ground large language models, improving factuality and explainability of tutoring dialogue.[7][1]

In sports contexts, intelligent coaching and guidance systems extend ITS ideas by incorporating motion capture, wearable sensors, and video analysis to deliver feedback on technique, workload, and recovery. Systems such as CAM-Vtrans use multimodal visual–text fusion to provide real-time action feedback, while others use multimodal deep learning to analyze sports teachers’ behavior and recommend continuing education resources. Together, these works motivate an integrated sports intelligent tutoring system that can reason over both symbolic knowledge and sensor-derived signals.[2][3][5][7]

### Multimodal data fusion

Multimodal data fusion integrates information from multiple sensor or content modalities—visual, audio, textual, kinematic, and physiological—into a unified representation for downstream analysis and decision-making. Educational and sports applications commonly fuse:[3][7]

- Video and motion capture for posture and action recognition.[5][8]
- Audio and text for instruction clarity and feedback analysis.[7]
- Wearable signals such as steps, heart rate, acceleration, and galvanic skin response for workload, fatigue, and anomaly detection.[6][3]
- Structured knowledge from textbooks and lecture videos into knowledge graphs for query answering and tutoring.[1]

Fusion strategies range from early fusion (concatenating raw or low-level features), late fusion (combining model outputs), to attention-based or hierarchical fusion mechanisms that dynamically weight modalities according to task relevance. Empirical studies in physical education and sports show that multimodal fusion consistently outperforms single-modality approaches for behavior classification, anomaly detection, and recommendation accuracy.[3][7]

## Representative systems and evidence

### Wearable-based physical education evaluation with Transformer fusion

Dong (IJDST 2023) proposes an intelligent physical education teaching evaluation system that uses wearable devices (e.g., Xiaomi bracelets) to collect time-series step counts and heart rate data in school PE classes. A Transformer-based deep network analyzes step sequences to classify motion performance levels, while a hierarchical fusion network based on Transformer encoders integrates steps and heart rate for anomaly detection during teaching activities.[3]

The system builds a sports database including step and heart rate sequences plus learner attributes (age, sex, weight) and trains sequence models (FC, RNN, GRU, LSTM, Transformer) for performance prediction. Transformer-based step analysis achieves the highest accuracy (93.54%) compared with RNN, GRU, and LSTM (75.87–92.02%). For anomaly detection, hierarchical multimodal fusion (steps + heart rate) yields recognition accuracy of 98.15%, outperforming single-modality and serial fusion baselines (82.85–97.24%). These results show that multimodal fusion significantly improves detection of excessive fatigue and unsafe exercise states, enabling fine-grained guidance to avoid injuries.[3]

### Multimodal sports teaching behavior analysis and personalized recommendations

Chen (Sci Rep 2025) develops a multimodal deep learning framework to analyze sports teachers’ classroom behaviors using synchronized video, audio, wearable inertial measurement units (IMUs), and infrared motion tracking. The system defines a hierarchical classification of teaching behaviors (instructional, feedback, organizational, motivational) and a multimodal representation model capturing visual, audio, and motion features.[7]

A hierarchical attention-based fusion mechanism dynamically weights modalities for different behavior types—visual features dominate demonstration behaviors, while audio features dominate instructional explanations. Ablation experiments show that attention-based fusion achieves an F1 score of 0.85 and behavior classification accuracy of 88.3%, surpassing early fusion (F1 0.79, accuracy 83.1%) and late fusion (F1 0.82, accuracy 85.7%). On top of this, a personalized recommendation engine uses a multi-objective optimization algorithm and multimodal behavior analysis to recommend continuing education resources, achieving F1 0.85 and high teacher satisfaction. This supports the idea that multimodal analysis can drive intelligent tutoring of coaches and instructors themselves.[7]

### Table Tennis Tutor: multimodal stroke classification and real-time feedback

Sanusi et al. (Sensors 2021) introduce the Table Tennis Tutor (T3), an intelligent tutoring system for table tennis forehand stroke classification using multimodal sensor data. T3 combines a smartphone’s accelerometer and gyroscope (placed in the player’s pocket) with Microsoft Kinect depth camera tracking upper-body joints; data streams are synchronized via the Multimodal Learning Analytics (MMLA) pipeline.[5]

Manual annotation of 510 strokes (260 correct short strokes, 250 incorrect long strokes) enables training of an LSTM-based classifier with three input configurations: smartphone-only, Kinect-only, and combined. Classification accuracies are 51.0% (smartphone only), 63.9% (Kinect only), and 62.1% (combined), with combined input achieving the highest precision (72.9%). While accuracy remains modest due to small dataset size and suboptimal sensor placement, the study demonstrates feasibility of multimodal mistake detection and shows that multi-sensor fusion improves precision over single devices. Interviews with coaches indicate positive expectations about real-time feedback systems to complement human mentoring.[5]

### CAM-Vtrans: cross-attention multimodal visual transformer for sports coaching

LinLin et al. (Frontiers in Neurorobotics 2024) propose CAM-Vtrans, a multimodal robotic system for real-time sports training that fuses image and text data via a Vision Transformer (ViT), CLIP, and cross-attention. Video streams from multiple cameras capture athlete movements, while textual instructions (coach input) are embedded and processed by a Transformer text encoder. Cross-attention aligns visual and textual representations, and a multi-scale transformer encoder refines fused features for tasks such as classification and regression.[9][2]

CAM-Vtrans is evaluated on OpenImages, Objects365, MSCOCO, and VG-Gap datasets, outperforming baselines in accuracy, recall, F1 score, and AUC, while achieving real-time inference at approximately 19.2 ms per frame (~52 FPS), further boosted to near 60 FPS with pruning and distillation. Ablation experiments indicate that removing cross-attention degrades performance, confirming its critical role in multimodal integration. Although evaluated on generic vision benchmarks rather than specific sports datasets, the architecture is directly applicable to sports tutoring scenarios where feedback depends on both movement and textual coaching cues.[2]

### Multimodal human action recognition and personalized sports health promotion

Xi et al. (Frontiers in Neurorobotics 2026) present a comprehensive multimodal human action recognition and personalized sports health framework using wearable sensor fusion, a Transformer–GCN hybrid model, federated learning, and reinforcement learning-based exercise prescription. A sensing layer captures 12-dimensional data: three-axis acceleration, heart rate variability, galvanic skin response, temperature, humidity, and other environmental parameters via wearable sensors at wrist, chest, and ankle.[6]

An adaptive preprocessing pipeline performs resampling, wavelet denoising, LSTM-based imputation for missing data, dynamic time warping alignment, sliding-window segmentation, and normalization, producing a unified time–space feature matrix. The core analysis layer employs an 8-head Transformer encoder and a graph convolutional network (GCN) to model long-range temporal dependencies and biomechanical relationships based on human skeleton topology, achieving an average action recognition accuracy of 95.16% across running, jumping, squatting, stretching, and bending. A federated learning protocol allows personalized model tuning without sharing raw data.[6]

The intervention module uses a Proximal Policy Optimization (PPO) reinforcement learning algorithm to generate personalized exercise prescriptions (type, intensity, duration, rest intervals) based on health goals and real-time fatigue indices. In a 12-week randomized controlled trial with 40 adults, the intervention group using the system achieved a 20.1% increase in VO2max and 99.3% improvement in muscular endurance, while maintaining a sports injury rate below 15%; improvements were significantly greater than in a control group following fixed plans. Deployment tests show that TensorRT-optimized Transformer–GCN inference on smartphones yields 8.7 ms latency and 115 FPS, satisfying real-time feedback constraints. These results demonstrate that multimodal fusion plus adaptive control can form the backbone of a sports intelligent tutoring and health guidance system.[6]

### Multimodal knowledge graph and RAG-based intelligent tutoring system

Deng and Yuan (Frontiers in Computer Science 2026) propose an ITS that automatically constructs multimodal knowledge graphs from course videos and textbook PDFs and uses hybrid graph + vector Retrieval-Augmented Generation for tutoring. FFmpeg and Whisper extract and transcribe audio from instructional videos; PDF parsing, OCR, and layout analysis extract structured text, diagrams, and formulas from textbooks. BERT + CRF models perform entity recognition, and BERT-based classifiers perform relation extraction, followed by cross-modal entity disambiguation and hierarchical clustering-based entity fusion.[1]

Knowledge is stored both in a Neo4j graph database and a vector database (HNSW index), supporting hybrid retrieval that combines Cypher graph queries with dense vector similarity. Retrieved subgraphs and text chunks ground an LLM via RAG to produce accurate, explainable answers, with a fusion score balancing graph provenance and semantic similarity. Case studies in a “Stacks and Queues” data structures course yield a knowledge graph with 152 entities and 243 relations; entity extraction achieves F1 88.1%, relation extraction F1 83.6%, and Q&A evaluation shows higher accuracy, relevance, and interpretability than text-only RAG and manual KG baselines. While non-sports, this architecture offers a pattern for the symbolic tutoring component of sports guidance systems.[1]

## Common architectures and fusion patterns

### Layered system architectures

Across studies, sports-intelligent tutoring and guidance systems tend to adopt layered architectures:

- **Sensing layer**: wearable devices, cameras, microphones, and environmental sensors capture raw multimodal data.[5][3][6][7]
- **Preprocessing layer**: synchronization, resampling, denoising, segmentation, and feature extraction transform raw signals into structured representations.[3][5][6]
- **Core analysis layer**: deep learning models (CNN, LSTM, Transformer, GCN, cross-attention, hierarchical attention) perform action recognition, anomaly detection, behavior classification, and state estimation.[2][5][6][7][3]
- **Knowledge and tutoring layer**: knowledge graphs, recommendation engines, RAG-based LLMs, and rule-based logic perform tutoring, explanation, and resource recommendations.[1][7]
- **Application layer**: user interfaces deliver feedback—visual overlays, textual explanations, audio prompts, and vibration alerts—and record interactions for system refinement.[2][5][6][7]

This layered design supports modularity, scalability, and edge-cloud collaboration: latency-sensitive inference (e.g., motion feedback) can run at the edge, while heavier knowledge graph processing and long-term model training run in the cloud.[6][1]

### Multimodal fusion mechanisms

Key fusion mechanisms observed in the literature include:

- **Sequence-level fusion with Transformers**: Dong uses dual Transformer encoders for steps and heart rate, with hierarchical cross-fusion to improve anomaly detection accuracy to 98.15%.[3]
- **Hierarchical attention fusion**: Chen dynamically weights visual, audio, and motion modalities according to behavior categories, yielding higher F1 and accuracy than early/late fusion baselines.[7]
- **Cross-attention image–text fusion**: CAM-Vtrans uses cross-attention between ViT image embeddings and CLIP text embeddings to fuse robot multimodal data for sports training feedback.[2]
- **Hybrid spatiotemporal fusion**: Xi’s Transformer–GCN hybrid model fuses temporal patterns and spatial joint dependencies, achieving >95% action recognition accuracy.[6]
- **Graph + vector retrieval fusion**: Deng’s hybrid Graph RAG uses weighted fusion of Neo4j graph retrieval and HNSW vector retrieval to ground LLM answers with both logical and semantic evidence.[1]

Empirical ablations consistently show that attention-based and hybrid fusion strategies outperform simple concatenation and late fusion, particularly in complex tasks like anomaly detection, behavior analysis, and multimodal tutoring.[2][7][1][3][6]

## Toward an integrated sports intelligent tutoring and guidance system

### Target use cases

Based on surveyed systems, a sports intelligent tutoring and learning guidance system driven by multimodal data fusion can address several use cases:

- **Student-focused physical education guidance**: monitoring steps, heart rate, and movement quality during PE classes; detecting abnormal states; providing immediate feedback and adjusting activity intensity.[3]
- **Athlete technical tutoring**: classifying technique errors (e.g., stroke errors in table tennis), providing corrective cues, and tracking improvement over time.[5][2]
- **Coach/teacher professional development**: analyzing teaching behaviors, recommending targeted resources, and tracking teaching quality improvements.[7]
- **Sports health promotion**: generating personalized exercise plans, dynamically adjusting intensity and rest, and preventing overtraining and injuries.[6]

### Proposed architecture

An integrated system can combine the strengths of the above works in a unified framework:

1. **Multimodal acquisition**
   - Wearable devices (bracelets, IMUs, chest straps) collect steps, heart rate/HRV, acceleration, galvanic skin response, and possibly EMG for select sports.[3][6]
   - Cameras (RGB or depth) and motion capture track body posture and movement; microphones capture verbal instructions and feedback.[8][5][7]
   - Course videos, training manuals, and sports textbooks provide symbolic domain knowledge.[10][1]

2. **Preprocessing and alignment**
   - Standardize sampling rates across sensors via resampling; perform wavelet-based denoising and LSTM imputation for missing data as in Xi.[6]
   - Synchronize video, sensor, and audio streams using timestamp alignment and dynamic time warping.[5][7][6]
   - Segment continuous data into windows (e.g., 1–2 s for action recognition) with overlap to preserve context.[6]

3. **Core multimodal analysis**
   - Use Transformer-based encoders for time-series wearable data (steps, heart rate) with hierarchical cross-fusion to detect anomalies and classify workload states, building on Dong’s design.[3]
   - Employ CNN/ViT + GCN for video-based action recognition, mapping 2D/3D keypoints to skeletal graphs for biomechanical insight, as in athlete motion recognition systems and Xi’s Transformer–GCN hybrid.[8][6]
   - Integrate cross-attention-based fusion of visual and textual instruction (ViT + CLIP) for context-aware feedback similar to CAM-Vtrans.[2]
   - Apply hierarchical attention fusion to combine video, audio, and motion for teaching behavior analysis and quality assessment for sports educators, following Chen.[7]

4. **Knowledge graph and tutoring layer**
   - Automatically construct a sports knowledge graph from multimodal educational resources (videos of drills, coaching lectures, and textbooks) using entity and relation extraction; include entities for skills, drills, anatomical structures, training principles, and common mistakes, following Deng and Li’s sports knowledge graph work.[10][1]
   - Store knowledge in Neo4j and encode supporting text in a vector database; use hybrid graph + vector retrieval to ground a tutoring LLM via RAG for both conceptual questions (e.g., “Why is stride frequency important?”) and procedural advice (“How to adjust running form?”).[10][1]
   - Condition tutoring responses on real-time sensor-derived state (e.g., fatigue level, detected errors) to generate personalized feedback.

5. **Guidance and intervention module**
   - Use reinforcement learning (PPO) to optimize training intensity, duration, and rest intervals based on health goals, performance trajectories, and safe thresholds, following Xi’s intervention module.[6]
   - Incorporate rule-based safety constraints (e.g., heart rate >90% of age-predicted max triggers mandatory rest) to ensure robust safety during policy exploration.[3][6]

6. **User interfaces and experience**
   - Provide athletes and students with mobile or web interfaces showing motion feedback (e.g., skeleton overlays, heatmaps), textual or audio coaching tips, and progress dashboards.[5][2][6]
   - Offer teachers and coaches behavior analytics, quality indicators, and personalized professional development recommendations.[7]

### Expected benefits and empirical support

From the surveyed literature, a multimodal, integrated sports tutoring and guidance system is expected to deliver:

- **Improved recognition and diagnosis**: Action recognition accuracies above 95% and anomaly detection accuracies around 98% are achievable with Transformer-based and hybrid fusion models, enabling fine-grained detection of technique errors and unsafe workload patterns.[8][3][6]
- **Higher recommendation and tutoring quality**: Multimodal analysis and multi-objective optimization (knowledge graphs + behavioral data) can yield recommendation F1 scores around 0.85 and user satisfaction scores above 4.5/5 in teacher development contexts.[1][7]
- **Health and performance gains**: Personalized sports guidance systems with multimodal sensor fusion and RL-based prescriptions can produce substantial VO2max and muscular endurance improvements (>20% and ~100%, respectively) with controlled injury rates (<15%) in 12-week interventions.[6]
- **Real-time operation**: Optimized inference pipelines (TensorRT, edge-cloud collaboration) can maintain sub-10–20 ms latency and >60–100 FPS throughput on edge devices, meeting real-time feedback requirements for sports coaching.[2][6]

## Design and implementation considerations

### Data, privacy, and ethics

Implementing such systems requires large labeled datasets of multimodal sports activity, teaching behavior, and outcomes data. Existing studies often rely on tens to hundreds of participants and thousands of annotated segments, but scaling to broader deployment will demand semi-supervised learning, active learning, and expert-in-the-loop annotation tools. Federated learning and on-device processing, as used by Xi, can mitigate privacy concerns by keeping raw sensor data local while sharing only model updates. Ethical considerations include informed consent, safe training policies, and transparent explanation of AI-driven guidance, especially for youth and novice athletes.[5][7][6]

### Robustness and generalization

Sensor placement, device quality, and environment can strongly affect data quality, as evidenced by T3’s lower accuracy when using smartphone sensors in the pocket versus Kinect skeletal data. Systems must be robust to sensor failures, noise, and domain shifts across sports and populations. Techniques such as adaptive preprocessing, domain adaptation, and robust fusion are critical. Cross-sport generalization may require sport-specific models with shared backbones and sport-specific heads.[5][3][6]

### Human–AI collaboration

Studies of sports teachers and coaches emphasize that AI systems should complement, not replace, human expertise, providing objective measurements, pattern detection, and personalized recommendations while leaving final decisions and nuanced motivational strategies to humans. Interfaces should be designed to support rapid inspection, override, and reconfiguration, and to incorporate user feedback into continuous learning loops, similar to Deng’s human-in-the-loop knowledge graph review.[1][7][5]

## Conclusion

Current research demonstrates that multimodal data fusion—combining wearable sensors, video and audio, and symbolic knowledge—can effectively drive sports intelligent tutoring and learning guidance systems. Transformer-based sequence models, attention-based fusion mechanisms, hybrid Transformer–GCN architectures, multimodal knowledge graphs, and reinforcement learning-based intervention modules each contribute critical capabilities: accurate recognition, robust anomaly detection, explainable guidance, and adaptive training plan optimization. An integrated layered architecture that unifies these approaches promises measurable improvements in performance, safety, and learning quality for athletes, students, and sports educators. Future work will need to expand datasets, refine fusion strategies, and deepen human–AI collaboration, but the empirical evidence already supports practical deployment of multimodal sports intelligent tutoring and guidance systems in real-world settings.[1][2][7][5][3][6]

## References

[1] https://www.informatica.si/index.php/informatica/article/view/8605
[2] https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2024.1453571/pdf
[3] https://pdfs.semanticscholar.org/3f0a/7012898230517f4a7a23dfafc3a2467d2c16.pdf
[4] https://www.informatica.si/index.php/informatica/article/view/8485
[5] https://d-nb.info/1248561813/34
[6] https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2026.1785114/full
[7] https://pmc.ncbi.nlm.nih.gov/articles/PMC12800030/
[8] https://combinatorialpress.com/article/jcmcc/Volume%20127/Volume%20127a/knowledge-graph-construction-for-intelligent-physical-education-and-its-application-in-training.pdf
[9] https://www.spiedigitallibrary.org/conference-proceedings-of-spie/14128/141284X/Intelligent-decision-support-system-for-sports-training-based-on-multimodal/10.1117/12.3106604.short
[10] https://www.sciencedirect.com/science/article/pii/S1110016825006702
[11] https://informatica.si/index.php/informatica/article/download/10174/6031
[12] https://rsisinternational.org/journals/ijrsi/uploads/vol13-iss3-pg2807-2840-202604_pdf.pdf
[13] https://www.frontiersin.org/journals/computer-science/articles/10.3389/fcomp.2026.1777749/full
[14] https://www.informatica.si/index.php/informatica/article/download/9179/4758
[15] https://www.frontiersin.org/journals/neurorobotics/articles/10.3389/fnbot.2024.1453571/full


Please begin the extraction now. Output only the JSON list directly, without any chitchat or explanations.