LLM for Music Education
Objective: Intelligent Music Learning and Knowledge Support with Large Language Models
We study how large language models can support music learning through personalized assessment, practical guidance, and reliable access to music knowledge. Our current topics cover singing assessment and coaching, and knowledge-grounded music question answering.
Table of Contents:
Topic 1: Singing Assessment and Coaching
Topic 2: Music Question Answering
Topic 1
Intelligent Singing Assessment and Coaching
Main Contributors: Hayeon Bang
We develop end-to-end multimodal large language models for singing assessment and vocal coaching. Our research focuses on generating interpretable and personalized feedback from real-world student performances.
Our system goes beyond score-based evaluation by providing evidence-based explanations and concrete practice guidance. At the same time, it adapts its evaluation criteria to different learner groups, such as beginners, amateurs, and professional trainees.
Research Focus
Our work centers on three core directions:
Explainable Assessment
Generating natural-language feedback that combines diagnosis of vocal issues with concrete prescriptions for improvement.
Target-Aware Evaluation
Adapting evaluation standards and feedback strategies according to learner profiles, goals, and training levels.
Real-World Data Modeling
Leveraging large-scale student performance datasets together with authentic educational annotations and instructor feedback.
Dataset Examples
Community-based Singing Feedback Dataset
Q: “My voice cracks when I sing high notes. What should I do?”
A: “This usually happens due to insufficient breath support and throat tension. Practice gentle sirens and lip trills.”
Real-World Singing Assessment Dataset (Excerpt)
Student ID: S_013
Overall Grade: High (86/100)
Phrase (16/20)
“The phrases are connected smoothly, resulting in a stable and coherent musical flow.”
Voice Production (16/20)
“Stable vocal production is maintained throughout the performance.”
Pronunciation (16/20)
“Clear and accurate diction allows the lyrics to be delivered comfortably and effectively.”
Musical Attitude (16/20)
“The performer conveys the character of the piece well and demonstrates an active musical attitude.”
Rhythm (11/20)
“Rhythm is generally maintained, though slight instability appears in certain sections.”
Pitch (11/20)
“Overall pitch is stable, with minor fluctuations in a few passages.”
Topic 2
Music Question Answering
Main Contributors: Daeyong Kwon
Task definition: Answer factual and contextual questions about music by combining an LLM with relevant evidence retrieved from a music-domain knowledge base.
Goal: Provide accurate, culturally diverse, and evidence-grounded music information while reducing reliance on knowledge stored only in model parameters.
ArtistMus: Retrieval-Augmented Music Question Answering
Task Evaluate factual recall and contextual reasoning for artist-centric music questions.
Goal Ground music question answering in specialized knowledge and evaluate it across globally diverse artists.
Method The paper releases MusWikiDB, a music-specific retrieval database with 3.2 million passages from 144,000 Wikipedia pages, and ArtistMus, a benchmark of 1,000 validated questions covering 500 artists from 163 countries and regions. Retrieval, reranking, and retrieval-aware fine-tuning are evaluated to improve both factual and contextual question answering.
Domain-specific retrieval for factual and contextual artist knowledge.
Related Publications
-
ArtistMus: A Globally Diverse, Artist-Centric Benchmark for Retrieval-Augmented Music Question Answering
Daeyong Kwon, Seungheon Doh, and Juhan Nam
Proceedings of the 15th biennial Language Resources and Evaluation Conference (LREC), 2026
[paper] [code & data]