GCT634 (AI613) Fall 2020

Musical Applications of Machine Learning

Course Description

This course aims to understand machine learning and learn the applications in the music domain, for example, music search and recommendation, automatic music transcription, music generation, source separation and sound synthesis. Student will have hands-on experiences using audio processing and machine learning libraries through the assignments, and also gain experience of the full cycle of research through the final research project.

General Information

  • Instructor: Juhan Nam (남주한)
  • TA: Taegyun Kwon (권태균), Taejun Kim (김태준), Wonil Kim (김원일), Seunghyun Lee (이승현)
  • Course Format
    • Pre-recorded video lectures: KLMS
    • Online session: Thur 14:30 - 15: 45 (Zoom meeting)

Grading Policy

  • Assignments: 50 %
  • Research Project: 50%
    • Paper review
    • Presentation
    • Report

Textbooks

                                   

Schedule

Week Topics
1
2
  • [Video] Audio Data Representations [Slides]
    • Sampling and Quantization
    • DFT, STFT and Spectrogram
    • Human Pitch Perception, Mel-Spectrogram and Constant-Q transform
  • [Sep 10, Zoom Meeting] Audio Data Representations Using Librosa
  • Suggested Readings
3
4
  • [Video] Traditional Machine Learning: Unsupervised Learning [Slides]
    • Principal Component Analysis (PCA)
    • K-Means Clustering
    • Gaussian Mixture Model (GMM)
  • [Sep 24, Zoom Meeting] Dimensionality Reduction, Data Compression, Vector Quantization and Classification Using SciKit-Learn
  • [Homework #1] Musical Instrument Recognition (Due Oct 4, 11:59 PM) --> [Leaderboard]
  • Suggested Readings
    • The PRML book (Chapter 9: Mixture Models and EM, Chapter 12: Continuous Latent Variables)
5
  • [Video] Traditional Machine Learning: Supervised Learning [Slides]
    • K-Nearest Neighbor (K-NN)
    • Support Vector Machine (SVM)
    • Logistric Regression
    • Multi-Layer Perceptron
  • Suggested Readings
    • The PRML book (Chapter 3: Linear Models for Regression)
    • The PRML book (Chapter 4: Linear Models for Classification)
    • The PRML book (Chapter 5: Neural Networks)
6
  • [Video] Deep Learning: Introduction [Slides]
    • Multi-Layer Perceptron
    • Error Back-propagation
  • [Oct 8, Zoom Meeting (cancelled)]
  • Suggested Readings
    • The DL book (Chapter 6: Deep Feedforward Networks)
7
  • [Video] Deep Learning: Training Models [Slides]
    • Initialization, Normalization, Optimization, and Regularization
  • [Oct 15, Zoom Meeting] HW1 review
  • Suggested Readings
    • The DL book (Chapter 7: Regularization for Deep Learning)
    • The DL book (Chapter 8: Optimization for Training Deep Models)
8
9
  • [Video] Convolutional Neural Network (CNN) and Musical Applications (Cont'd)
  • [Oct 29, Zoom Meeting]
10
  • [Video] Recurrent Neural Network (RNN) and Musical Applications [Slides]
  • Suggested Readings
    • The DL book (Chapter 10: Sequence Modeling: Recurrent and Recursive Nets)
11
12
  • [Video] Encoder-Decoder CNN and Musical Applications [Slides]
  • [Nov 19, Zoom Meeting] Final Project Guide
  • Suggested Readings
    • The DL book (Chapter 14: Auto-Encoder)
13
  • [No Zoom Meeting] project team meetings
14
15
  • Invited Talk: Dr. Yi-Hsuan Yang from Academia Sinica and Taiwan AI Lab
16