Piano Performance Analysis

Main Contributors: Taegyun Kwon, Dasaem Jeong

In classifical piano music, pianists play a piece of music with different styles and interpretations. This topic explores machine learning methods to extract performance information such as note onset, duration and velocity from piano music recordings and analyzes different nuances among pianists.

Polyphonic Piano Transcription

Polyphonic piano transcription is a task that predicts a score-based representation from from piano music recordings. We have proposed effective and efficient deep neural network models. In the following demos, three examples show "reperformance" using our own Diskalvier piano and the last one shows real-time piano transcription.

Related Publications

• Polyphonic Piano Transcription Using Autoregressive Multi-Note-State Model
  Taegyun Kwon, Dasaem Jeong, and Juhan Nam
  Proceedings of the 21st International Society for Music Information Retrieval Conference (ISMIR), 2020 [pdf] [demo]
• A Classification-Based Polyphonic Piano Transcription Approach Using Learned Feature Representations
  Juhan Nam, Jiquan Ngiam, Honglak Lee, and Malcolm Slaney
  Proceedings of the 12th International Conference for Music Information Retrieval Conference (ISMIR), 2011 [pdf]

Audio-to-Score Alignment and Performance Visualization

In classifical piano music, music scores are usually available on the player's side. Aligning the music score with different renditions of performances allows us to compare different play styles in terms of tempo, dynamics, phrasing and articulation. We proposed a method for the audio-to-score alignment using a polyphonic piano transcription algorithm, and also designed a web-based visualization system to intuitively show different renditions of the same piano piece.

Related Publications

• Audio-to-Score Alignment Of Piano Music Using RNN-based Automatic Music Transcription
  Taegyun Kwon, Dasaem Jeong and Juhan Nam
  Proceedings of the 14th Sound and Music Computing Conference (SMC), 2017 [pdf] [demo]
• PerformScore: Toward Performance Visualization With the Score on the Web Browser
  Dasaem Jeong, Taegyun Kwon, Chaelin Park, and Juhan Nam
  Late Breaking Demo in the 18th International Society for Musical Information Retrieval Conference (ISMIR), 2017 [pdf]

Note Intensity Estimation

Note intensity provides dynamics information of piano performance. Estimating the note intensity is challenging due to the mixture of harmonics notes and volumn change in audio recordings. We proposed several methods to estimate note intensity and velocity using non-negative matrix factorization and deep neural networks.

Related Publications

• Note Intensity Estimation of Piano Recordings Using Coarsely-aligned MIDI Score
  Dasaem Jeong, Taegyun Kwon, and Juhan Nam
  Journal of the Audio Engineering Society, 2020 [pdf]
• A Timbre-based Approach to Estimate Key Velocity from Polyphonic Piano Recordings
  Dasaem Jeong, Taegyun Kwon and Juhan Nam
  Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), 2018 [pdf]
• Note Intensity Estimation of Piano Recordings by Score-informed NMF
  Dasaem Jeong and Juhan Nam
  Proceedings of the Audio Engineering Society Conference on Semantic Audio (AES), 2017 [pdf]