Analyzing K-12 AI education: A large language model study of classroom instruction on learning theories, pedagogy, tools, and AI literacy

Faculty of Artificial Intelligence in Education, normal university

Wuhan, China

Researchers building LLM analysis pipeline and correlation study

• In-school (K–12)

Secondary analysis of recorded AI instruction videos

Corpus of 98 classroom videos (urban central China)

98 lessons / classrooms as units of analysis

Varied AI teaching tools as captured on video

• Geographic concentration in central Chinese cities
• Video may miss off-camera activity
• LLM coding drift requires ongoing validation
• Ethics coverage rarity is descriptive not causal proof of absence in all moments

K-12 as represented in video corpus

Varies by lesson level

Varies by lesson design

• Quantify pedagogy, theory, tools, and literacy levels in real AI lessons
• Validate LLM coding against human analysts
• Identify instructional profiles (conceptual/heuristic/experimental)

• Test correlations between pedagogy mixes and higher-order literacy
• Flag severe underrepresentation of explicit ethics segments

• Not a student RCT
• Not exhaustive national census of all AI classes
• Not fine-grained discourse analysis without LLM layer

LLM as research instrument for coding classroom video transcripts/segments

• Evaluator
• Automation tool

Chinese instructional discourse

• Automated tagging of instructional events and literacy level indicators
• Human-in-the-loop validation for agreement metrics

• Protect identifiable students/teachers in recordings per ethics approval
• Guard against over-trust in LLM labels—maintain audit samples
• Transparent prompt and rubric documentation for reproducibility
• Avoid high-stakes teacher evaluation from automated scores alone

• Curate 98 AI education videos
• Define coding scheme for theories, pedagogies, tools, literacy
• Run LLM analysis and benchmark to manual coding
• Compute distributions and correlations (e.g., PBL+collaboration vs evaluate/create)

• Researchers validate and interpret LLM outputs; LLM accelerates large-scale annotation

• Use findings to coach teachers toward combinations supporting advanced literacy

• Low prevalence of explicit ethics instruction segments
• Higher-order literacy tasks relatively uncommon
• Scaling manual coding impractical without LLM assistance

• Novel LLM-assisted framework for large-scale classroom AI-education analytics

• Shows feasible >90% consistency with manual analysis in reported checks

• Pedagogy composition correlates with advanced literacy indicators
• Most lessons skew conceptual; ethics segments sparse (5.1%)

Supports combining PBL with collaborative methods to elevate evaluate/create AI competencies and calls for more ethics time.

• Video research ethics and consent
• Potential misuse of automated classroom ratings for accountability
• Bias in LLM toward certain instructional styles/languages
• Data security for large video corpora

• Learning analytics
• Activity documentation
• Practitioner observation

• Expand corpus multi-region; track ethics instruction over years
• Teacher-facing dashboards from validated coding

• AI classroom research
• Computational education research methods
• AI literacy assessment in vivo

• Completed

K-12 (video corpus)

Urban China AI lessons

Instruction analysis + pedagogy mapping

LLM-assisted coding

Medium (method misuse)

High (video)