Course Description

This course aims to provide an understanding of machine learning with applications to music and audio. Specifically, this course covers various tasks in the topics of music and audio classification, automatic music transcription, source separation, sound synthesis, and music and audio generation. Students will gain hands-on experiences with audio processing and machine learning libraries through the assignments and gain experience in the full cycle of research through the final research project.

Textbooks

• Fundamentals of Music Processing (FMP), Meinard Müller, Springer, 2015
( Python Notebooks for Fundamentals of Music Processing )
• Pattern Recognition And Machine Learning (PRML), C. Bishop, Springer, 2006
• Deep Learning, Ian Goodfellow and Yoshua Bengio and Aaron Courville (DL), MIT Press, 2017

KLMS             Classum             GitHub (Code)             Resources

Schedule

Week	Topics
1	Course Introduction [slides] Audio Data Representation [slides] [practice] 01. audio representations.ipynb Suggested Readings The FMP book (Chapter 1: Music Representations) [python notebook] The FMP book (Chapter 2: Fourier Analysis of Signals) [python notebook]
2	Music Classification: Overview and Audio Features [slides] Music Classification: Traditional Machine Learning [slides] [practice] 02. audio features.ipynb [practice] 03. unsupervised learning.ipynb Suggested Readings Signal Processing for Music Analysis (2011) Musical Genre Classification of Audio Signals (2002)
3	Music Classification: Traditional Machine Learning (Cont'd) Music Classification: Deep Learning [slides] Suggested Readings Deep Learning for Audio-Based Music Classification and Tagging: Teaching Computers to Distinguish Rock from Bach (2018) ISMIR 2021 Tutorial - Music Classification: Beyond Supervised Learning, Towards Real-world Applications [Homework #1] Music Classification (Due Mar 31, 11:59 PM) [link] (20% of the total score)
4	Music Classification: Deep Learning (Cont'd) Music Representation Learning [slides] Suggested Readings ISMIR 2020 Tutorial - Metric Learning for Music Information Retrieval
5	Music Representation Learning (Cont'd) Automatic Music Transcription: monophonic [slides] Suggested Readings Automatic Music Transcription: An Overview (2018)
6	Automatic Music Transcription: polyphonic [slides] Audio-to-Score Alignment [slides] [practice] NMF-Based Spectrogram Factorization [practice] Music Synchronization Suggested Readings The FMP book (Chapter 3: Music Synchronization) The FMP book (Chapter 8: Musically Informed Audio Decomposition)
7	Chord Recognition [slides] [Apr-10] No Class (Election Day) Suggested Readings The FMP book (Chapter 5: Chord Recognition) [Homework #2] Polyphonic Piano Transcription (Due Apr 21, 11:59 PM) [link] (15% of the total score)
8	No Class (Midterm)
9	Rhythm Transcription [slides] Symbolic Music Representation Suggested Readings The FMP book (Chapter 6: Tempo and Beat Tracking) ISMIR 2021 Tutorial - Tempo, Beat, and Downbeat Estimation
10	Symbolic Music Generation Music Source Separation Suggested Readings ISMIR 2020 Tutorial - Open Source Tools & Data for Music Source Separation [Homework #3] Symbolic Music Generation
11	[May-6] No Class (Holiday Break) Neural Audio Synthesis Final Project: Team Meeting with Professor
12	Audio-Level Music Generation [May-15] No Class (Holiday Break) Final Project: Team Meeting with Professor
13	Final Project: Proposal Presentation (Oral)
14	Invited talk or Special Topic (TBD)
15	Invited talk or Special Topic (TBD)
16	Final Project: Poster Presentation