GCT634 Fall 2022

Musical Applications of Machine Learning

Course Description

This course aims to learn the new technology to learn machine learning with applications to the music and audio domains. Specificially, we handle various tasks in the topics of music and audio classification, automatic music transcription, source separation, sound synthesis, and music generation. Student will have hands-on experiences using audio processing and machine learning libraries through the assignments and gain experience of the full cycle of research through the final research project.

General Information

  • Instructor: Juhan Nam (남주한)
  • TAs: Seungheon Doh(도승헌), Jaekwon Im(임재권), Houn Su Kim(김현수)
  • Time: Mon/Wed 14:30 - 16:00
  • Place: N25#3229 Paik Nam June Hall (백남준홀)
  • Course Format: online or hybrid (onsite + online)

Grading Policy

  • Assignments: 50 %
  • Research Project: 50%
    • Paper review
    • Presentation
    • Report

Textbooks

                       

Schedule

Week Topics
1
  • [Aug-29, online] Course Introduction [slides]
  • [Aug-31, online] Audio Data Representations [slides]
  • [practice] audio_representations.ipynb
  • Suggested Readings
    • The FMP book (Chapter 1: Music Representations)
    • The FMP book (Chapter 2: Fourier Analysis of Signals)
2
3
  • [Sep-12] No Class (Chuseok Holiday)
  • [Sep-14, online] Music Classification - Traditional Machine Learning [slides]
  • [practice] Unsupervised Learning.ipynb
  • [Homework #1] Musical Instrument Recognition (Due Sep 25, 11:59 PM) [link]
  • Suggested Readings
    • The PRML book (Chapter 3: Linear Models for Regression)
    • The PRML book (Chapter 4: Linear Models for Classification)
    • The PRML book (Chapter 5: Neural Networks)
    • The PRML book (Chapter 6: Kernel Methods)
    • The PRML book (Chapter 9: Mixture Models and EM)
    • The PRML book (Chapter 12: Continuous Latent Variables)
    • Musical Genre Classification of Audio Signals, Tzanetakis and Cook (2002)
4
5
6
  • [Oct-4, online] Automatic Music Transcription - Overview and Pitch Estimation
  • [Oct-5, online] Automatic Music Transcription - Note Transcription
  • [Homework #2] Music Auto-Tagging (Due Oct 16, 11:59 PM) [link]
  • Suggested Readings
7
  • [Oct-10] No Class (Hangeul Holiday)
  • [Oct-12, online] Audio-to-Score Alignment
  • Suggested Readings
    • The FMP book (Chapter 3: Music Synchronization)
8
  • Midterm Break
9
  • [Oct-24, online] Automatic Music Transcription - Tempo Estimation, Beat Tracking, and Chord Recognition
  • [Oct-26, online] Music Source Separation: Auto-Encoder, U-Net
  • Suggested Readings
    • The FMP book (Chapter 5: Chord Recognition)
    • The FMP book (Chapter 6: Tempo and Beat Tracking)
    • The DL book (Chapter 14: Auto-Encoder)
10
  • [Oct-31, online] Symbolic Music Generation - Overview and RNN Models
  • [Nov-1, online] Symbolic Music Generation - VAE and Transformer Models
11
  • [Nov-7, online] Audio Generation
12
  • Individual Project Meeting with Professor
13
  • Student: Project Idea Presentation
14
  • TBD
15
  • TBD
16
  • Student: Project Presentation