Piano Music Transcription

Main Contributors: Taegyun Kwon, Dasaem Jeong

In classifical piano music, pianists play a piece of music with different styles and interpretations. This topic explores machine learning methods to extract performance information such as note onset, duration and velocity from piano music recordings and analyzes different nuances among pianists.

Polyphonic Piano Note Transcription

Polyphonic piano note transcription is a task that predicts individual notes from piano music recordings, resulting in MIDI files. We have proposed effective and efficient neural network models for the audio-to-MIDI conversion. In the left video, we chose a piano performance video by pianist Sung-Jin Cho on YouTube (Chopin Scherzo no. 2) and extracted the MIDI file from the audio part using our piano transcription model. We then reproduced the performance with the MIDI file on Diskalvier piano. In the right video, we implemented the piano transcription model in a real-time version. The MIDI notes extracted on the fly are shown in the form of piano roll.

Related Publications

• Polyphonic Piano Transcription Using Autoregressive Multi-Note-State Model
  Taegyun Kwon, Dasaem Jeong, and Juhan Nam
  Proceedings of the 21st International Society for Music Information Retrieval Conference (ISMIR), 2020 [pdf] [demo]
• A Classification-Based Polyphonic Piano Transcription Approach Using Learned Feature Representations
  Juhan Nam, Jiquan Ngiam, Honglak Lee, and Malcolm Slaney
  Proceedings of the 12th International Conference for Music Information Retrieval Conference (ISMIR), 2011 [pdf]

Note-level Intensity Estimation

Note-level intensity provides dynamics information of notes in piano performances. Estimating the intensity of individual notes is challenging due to the mixture of harmonics notes and volumn change in audio recordings. We proposed several methods to estimate note-level intensity using non-negative matrix factorization and deep neural networks.

Related Publications

• Note Intensity Estimation of Piano Recordings Using Coarsely-aligned MIDI Score
  Dasaem Jeong, Taegyun Kwon, and Juhan Nam
  Journal of the Audio Engineering Society, 2020 [pdf]
• A Timbre-based Approach to Estimate Key Velocity from Polyphonic Piano Recordings
  Dasaem Jeong, Taegyun Kwon and Juhan Nam
  Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), 2018 [pdf]
• Note Intensity Estimation of Piano Recordings by Score-informed NMF
  Dasaem Jeong and Juhan Nam
  Proceedings of the Audio Engineering Society Conference on Semantic Audio (AES), 2017 [pdf]