Music and Audio Computing Lab

Main menu: Home | People | Research | Publications |

Content-based Music Retrieval

In the recent past, music has become ubiquitous as digital data. The scale of music content in online music and video sharing services has significantly increased and we can readily access to them. This posed challenges in terms of efficient and effective content organization, search and recommendation. In this project, we explore machine learning algorithms, particularly unsupervised feature learning and deep learning, to automatically annotate music in terms of genre, mood, instruments, artist and other music descriptions. We are applying some of our research outcomes to real-world music recommedation systems, actively collaborating with the NAVER Music team.

Music Galaxy HitchHiker: 3D web music navigation through audio feature space learned with tag and artist labels


NAVER VIBE Music App


Publications

  • Deep Learning for Audio-based Music Classification and Tagging
    Juhan Nam, Keunwoo Choi, Jongpil Lee, Szu-Yu Chou, and Yi-Hsuan Yang
    IEEE Signal Processing Magazine, 2019 (accepted for publication)
  • Representation Learning of Music Using Artist Labels
    Jiyoung Park, Jongpil Lee, Jangyeon Park, Jung-Woo Ha and Juhan Nam
    Proceedings of the 19th International Society for Music Information Retrieval Conference (ISMIR), 2018 [pdf]
  • Sample-level CNN Architectures for Music Auto-tagging Using Raw Waveforms
    Taejun Kim, Jongpil Lee, and Juhan Nam
    Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018 [pdf]
  • SampleCNN: End-to-End Deep Convolutional Neural Networks Using Very Small Filters for Music Classification
    Jongpil Lee, Jiyoung Park, Keunhyoung Luke Kim and Juhan Nam
    Applied Sciences, 2018 [pdf]
  • Raw Waveform-based Audio Classification Using Sample-level CNN Architectures
    Jongpil Lee, Taejun Kim, Jiyoung Park and Juhan Nam
    Machine Learning for Audio Signal Processing Workshop, Neural Information Processing Systems (NIPS), 2017 [pdf]
  • Cross-cultural Transfer Learning Using Sample-level Deep Convolutional Neural Networks
    Jongpil Lee, Jiyoung Park, Chanju Kim, Adrian Kim, Jangyeon Park, Jung-Woo Ha and Juhan Nam
    Music Information Retrieval Evaluation eXchange (MIREX) in the 18th International Society for Musical Information Retrieval Conference (ISMIR), 2017 [pdf]
    *** The 1st place in the four K-POP tasks across all algorithms submitted so far ***
  • Representation Learning Using Artist labels for Audio Classification Tasks
    Jiyoung Park, Jongpil Lee, Jangyeon Park, Jung-Woo Ha and Juhan Nam
    Music Information Retrieval Evaluation eXchange (MIREX) in the 18th International Society for Musical Information Retrieval Conference (ISMIR), 2017 [pdf]
    *** The 1st place in the Music Mood Classification task across all algorithms submitted so far ***
  • Music Galaxy Hitchhiter: 3D Web Music Navigation Through Audio Space Trained with Tag and Artist Labels
    Dongwoo Suh, Kyungyun Lee, Jongpil Lee, Jiyoung Park and Juhan Nam
    Late Breaking Demo in the 18th International Society for Musical Information Retrieval Conference (ISMIR), 2017 [pdf]
  • Multi-Level and Multi-Scale Feature Aggregation Using Sample-level Deep Convolutional Neural Networks for Music Classification
    Jongpil Lee and Juhan Nam
    Machine Learning for Music Discovery Workshop, International Conference on Machine Learning (ICML), 2017 [pdf]
  • Sample-level Deep Convolutional Neural Networks for Music Auto-Tagging Using Raw Waveforms
    Jongpil Lee, Jiyoung Park, Keunhyoung Luke Kim and Juhan Nam
    Proceedings of the 14th Sound and Music Computing Conference, 2017 [pdf]
  • Multi-Level and Multi-Scale Feature Aggregation Using Pre-trained Convolutional Neural Networks for Music Auto-Tagging
    Jongpil Lee and Juhan Nam
    IEEE Signal Processing Letters, 2017 [pdf]
  • A Deep Bag-of-Features Model for Music Auto-Tagging
    Juhan Nam, Jorge Herrera, Kyogu Lee
    arXiv preprint arXiv:1508.04999, 2015 [pdf]

Participants

Jongpil Lee, Jiyoung Park, Keunhyoung Kim, Chaelin Park, Sangeun Kum, Kyungyun Lee and Juhan Nam


Funding

  • NAVER - industry research fund, 2017-2019
  • National Research Foundation of Korea, 2015-2018
  • KAIST - research start-up fund for new faculty, 2014-2017