Course Description

This course aims to provide an understanding of machine learning with applications to music and audio. The topics of this course include music and audio classification, music representation learning, automatic music transcription, source separation, sound synthesis, and music and audio generation. Students will gain hands-on experiences with audio processing and machine learning libraries through the assignments and gain experience in the full cycle of research through the final research project.

Textbooks

• Fundamentals of Music Processing (FMP), Meinard Müller, Springer, 2015
( Python Notebooks for Fundamentals of Music Processing )
• Pattern Recognition And Machine Learning (PRML), C. Bishop, Springer, 2006
• Deep Learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville (DL), MIT Press, 2017

KLMS             Classum             GitHub (Code)             Resources

Schedule

Week	Topics
1	Course Introduction [pdf] Audio Data Representation [pdf] [practice] 01. audio representations.ipynb Suggested Readings The FMP book (Chapter 1: Music Representations) [python notebook] The FMP book (Chapter 2: Fourier Analysis of Signals) [python notebook]
2	[Mar-3] No Class (Holiday) Music Classification Overview and Audio Features [pdf] Suggested Readings Signal Processing for Music Analysis (2011)
3	Music Classification Overview and Audio Features (Cont'd) [practice] 02. audio features.ipynb [Homework #1] Musical Instrument Recognition with Traditional Machine Learning (Due Mar 23, 11:59 PM) [link] (10% of the total score)
4	Music Classification: Traditional Machine Learning [pdf] [practice] 03. unsupervised learning.ipynb Suggested Readings Musical Genre Classification of Audio Signals (2002)
5	Music Classification: Deep Learning [pdf] Suggested Readings ISMIR 2021 Tutorial - Music Classification: Beyond Supervised Learning, Towards Real-world Applications [Homework #2] Music Tagging with Deep Learning (Due Apr 9, 11:59 PM) [link] (15% of the total score)
6	Music Representation Learning [pdf] Suggested Readings ISMIR 2020 Tutorial - Metric Learning for Music Information Retrieval
7	Automatic Music Transcription: monophonic [pdf] Automatic Music Transcription: polyphonic [pdf] Suggested Readings Automatic Music Transcription: An Overview (2018) [Homework #3] Polyphonic Piano Transcription (Due Apr 20, 11:59 PM) (15% of the total score)
8	No Class (Midterm)
9	Symbolic Music Generation [Homework #4] Symbolic Music Generation (Due May 4, 11:59 PM) (10% of the total score) Suggested Readings ISMIR 2023 Tutorial - Transformer-based Symbolic Music Generation (Part 1) ISMIR 2023 Tutorial - Transformer-based Symbolic Music Generation (Part 2)
10	Music Source Separation Suggested Readings ISMIR 2020 Tutorial - Open Source Tools & Data for Music Source Separation
11	[May-5] No Class (Holiday) Neural Sound Synthesis Suggested Readings ISMIR 2023 Tutorial - Introduction to Differentiable Audio Synthesiser Programming
12	Audio-Level Music Generation
13	Student Project Meetings
14	Final Project: Proposal Presentation (Oral)
15	TBD
16	Final Project: Poster Presentation