Expressive Piano Performance Rendering

Main Contributors: Dasaem Jeong, Taegyun Kwon

"VirtuosoNet" is a neural netwrok model that can generate human-like piano performances given a music score. The model is builted with a conditional variational auto-encoder with RNN modules. The score encoder represents a musicXML-format input using hierarchical attention network or a graph neural network. The encoded score is taken as a condition to generate performance features composed of tempo, note onset deviation, velocity, pedal parameters and so on. The following video shows demo examples of VirtuosoNet.

Pianist Jong Hwa Park and VirtuosoNet played a piano duet arrangement of Beethoven's Symphony No. 5 in the 50th anniversary ceremony of KAIST.

Related Publications

• A Hierarchical RNN-based System for Modeling Expressive Piano Performance
  Dasaem Jeong, Taegyun Kwon, Yoojin Kim, and Juhan Nam
  Proceedings of the 20th International Society for Music Information Retrieval Conference (ISMIR), 2019 [pdf]
• Graph Neural Network for Music Score Data and Modeling Expressive Piano Performance
  Dasaem Jeong, Taegyun Kwon, Yoojin Kim, and Juhan Nam
  Proceedings of the 36th International Conference on Machine Learning (ICML), 2019 [pdf]
• Score and Performance Features for Rendering Expressive Music Performances
  Dasaem Jeong, Taegyun Kwon, Yoojin Kim, and Juhan Nam
  Proceedings of the Music Encoding Conference, 2019 [pdf]
• VirtuosoNet: A Hierarchical Attention RNN for Generating Expressive Piano Performance from Music Score
  Dasaem Jeong, Taegyun Kwon and Juhan Nam
  Workshop on Machine Learning for Creativity and Design, Neural Information Processing Systems (NeurIPS), 2018 [pdf]