Music and Audio Computing Lab

Main menu: Home | People | Research | Publications |

Piano Performance Analysis and Synthesis

Classical music often contains complicate musical structure and text compared to other genres of music. In this research subject, we search for methods that can help listeners to understand and enjoy the classical music. One of the representative examples is web-based listening interface that provides a score of the piece with its various recordings, which are synchronized with the score. We also work on analyzing performer’s interpretation of the music, so that we can explain the differences among performances in a quantitative way.


PerformScore: Web-based Piano Score and Performance Analysis Visualization System


We have worked on several sub-topics including polyphonic note transcription, audio-to-score alignment, note intensity estimation, and performance generation. Here are the lists of categorized publications.


Polyphonic Note Transcription and Audio-to-Score Alignment

  • Polyphonic Piano Transcription Using Autoregressive Multi-Note-State Model
    Taegyun Kwon, Dasaem Jeong, and Juhan Nam
    Proceedings of the 21st International Society for Music Information Retrieval Conference (ISMIR), 2020 (accepted)
  • PerformScore: Toward Performance Visualization With the Score on the Web Browser
    Dasaem Jeong, Taegyun Kwon, Chaelin Park, and Juhan Nam
    Late Breaking Demo in the 18th International Society for Musical Information Retrieval Conference (ISMIR), 2017 [pdf]
  • Audio-to-Score Alignment Of Piano Music Using RNN-based Automatic Music Transcription
    Taegyun Kwon, Dasaem Jeong and Juhan Nam
    Proceedings of the 14th Sound and Music Computing Conference (SMC), 2017 [pdf] [demo]

Note Intensity Estimation

  • Note Intensity Estimation of Piano Recordings Using Coarsely-aligned MIDI Score
    Dasaem Jeong, Taegyun Kwon, and Juhan Nam
    Journal of the Audio Engineering Society, 2020 [pdf]
  • A Timbre-based Approach to Estimate Key Velocity from Polyphonic Piano Recordings
    Dasaem Jeong, Taegyun Kwon and Juhan Nam
    Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), 2018 [pdf]
  • Note Intensity Estimation of Piano Recordings by Score-informed NMF
    Dasaem Jeong and Juhan Nam
    Proceedings of the Audio Engineering Society Conference on Semantic Audio (AES), 2017 [pdf]

Expressive Performance Generation

  • A Hierarchical RNN-based System for Modeling Expressive Piano Performance
    Dasaem Jeong, Taegyun Kwon, Yoojin Kim, and Juhan Nam
    Proceedings of the 20th International Society for Music Information Retrieval Conference (ISMIR), 2019 [pdf]
  • Graph Neural Network for Music Score Data and Modeling Expressive Piano Performance
    Dasaem Jeong, Taegyun Kwon, Yoojin Kim, and Juhan Nam
    Proceedings of the 36th International Conference on Machine Learning (ICML), 2019 [pdf]
  • Score and Performance Features for Rendering Expressive Music Performances
    Dasaem Jeong, Taegyun Kwon, Yoojin Kim, and Juhan Nam
    Proceedings of the Music Encoding Conference, 2019 [pdf]
  • VirtuosoNet: A Hierarchical Attention RNN for Generating Expressive Piano Performance from Music Score
    Dasaem Jeong, Taegyun Kwon and Juhan Nam
    Workshop on Machine Learning for Creativity and Design, Neural Information Processing Systems (NeurIPS), 2018 [pdf]

Funding

We have received the following funds to support this research.

  • Samsung Research Funding, 2017-2020
  • 중소기업기술정보진흥원 이공계창업꿈나무과제, 2016