Course Description

This course aims to provide an understanding of machine learning with applications to music and audio. Specifically, this course covers various tasks in the topics of music and audio classification, automatic music transcription, source separation, sound synthesis, and music and audio generation. Students will gain hands-on experiences with audio processing and machine learning libraries through the assignments and gain experience in the full cycle of research through the final research project.

Textbooks

• Fundamentals of Music Processing (FMP), Meinard Müller, Springer, 2015
( Python Notebooks for Fundamentals of Music Processing )
• Pattern Recognition And Machine Learning (PRML), C. Bishop, Springer, 2006
• Deep Learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville (DL), MIT Press, 2017

KLMS             Classum             GitHub (Code)             Resources

Schedule

Week	Topics
1	Course Introduction [slides] Audio Data Representation [slides] [practice] 01. audio representations.ipynb Suggested Readings The FMP book (Chapter 1: Music Representations) [python notebook] The FMP book (Chapter 2: Fourier Analysis of Signals) [python notebook]
2	Music Classification: Overview and Audio Features [slides] Music Classification: Traditional Machine Learning [slides] [practice] 02. audio features.ipynb [practice] 03. unsupervised learning.ipynb Suggested Readings Signal Processing for Music Analysis (2011) Musical Genre Classification of Audio Signals (2002)
3	Music Classification: Traditional Machine Learning (Cont'd) Music Classification: Deep Learning [slides] Suggested Readings Deep Learning for Audio-Based Music Classification and Tagging: Teaching Computers to Distinguish Rock from Bach (2018) ISMIR 2021 Tutorial - Music Classification: Beyond Supervised Learning, Towards Real-world Applications [Homework #1] Music Classification (Due Mar 31, 11:59 PM) [link] (20% of the total score)
4	Music Classification: Deep Learning (Cont'd) Music Representation Learning [slides] Suggested Readings ISMIR 2020 Tutorial - Metric Learning for Music Information Retrieval
5	Music Representation Learning (Cont'd) Automatic Music Transcription: monophonic [slides] Suggested Readings Automatic Music Transcription: An Overview (2018)
6	Automatic Music Transcription: polyphonic [slides] Audio-to-Score Alignment [slides] [practice] NMF-Based Spectrogram Factorization [practice] Music Synchronization Suggested Readings The FMP book (Chapter 3: Music Synchronization) The FMP book (Chapter 8: Musically Informed Audio Decomposition)
7	Chord Recognition [slides] [Apr-10] No Class (Election Day) Suggested Readings The FMP book (Chapter 5: Chord Recognition) [Homework #2] Polyphonic Piano Transcription (Due Apr 21, 11:59 PM) [link] (15% of the total score)
8	No Class (Midterm)
9	Rhythm Transcription [slides] Music Generation Overview [slides] Suggested Readings The FMP book (Chapter 6: Tempo and Beat Tracking) ISMIR 2021 Tutorial - Tempo, Beat, and Downbeat Estimation
10	Symbolic Music Generation - Basics [slides] Symbolic Music Generation - Advanced Models [slides] Suggested Readings ISMIR 2019 Tutorial - Generating Music with GANs—An Overview and Case Studies ISMIR 2023 Tutorial - Transformer-based Symbolic Music Generation (Part 1) ISMIR 2023 Tutorial - Transformer-based Symbolic Music Generation (Part 2) [Final Project] Intruction [link]
11	[May-6] No Class (Holiday Break) Music Source Separation (Video) [slides] Neural Audio Synthesis [slides] Suggested Readings ISMIR 2020 Tutorial - Open Source Tools & Data for Music Source Separation ISMIR 2023 Tutorial - Introduction to Differentiable Audio Synthesiser Programming [Homework #3] Symbolic Music Generation (Due May 23, 11:59 PM) [link] (15% of the total score)
12	Audio-Level Music Generation [slides] [May-15] No Class (Holiday Break) Final Project: Team Meeting with Professor
13	[Invited Talk] Dr. Minz Won (Senior Research Scientist, Suno): Music Representation Learning [slides] [May-22] No Class (Admission Interview) Final Project: Team Meeting with Professor
14	Final Project: Proposal Presentation (Oral)
15	[Invited Talk] Seungheon Doh (PhD Student, KAIST): LLM-Powered Music Annotation and Retrieval [Jun-5] No Class (Final Project Preparation)
16	Final Project: Poster Presentation [link]