Course Description

Musical sounds contain a variety of information, for example, melody, rhythm, harmony, genre and mood. Extracting such information by computers can provide intelligent solutions in various musical activities, for example, finding songs that satisfy users' tastes and contexts among numerous choices or assisting musical instrument learning. Machine learning is a powerful means that enables computers to understand the musical sounds and assist human to find music or learn to play instrument more easily. Machine learning can be also applied to generate new music content. This course introduces a variety of problems that computers can do in music and how to solve them using machine learning.

General Information

Instructor: Juhan Nam (남주한)
TA: Sangeon Yong (용상언)
TA: Soonbeom Choi (최순범)
Lecture: Tue/Thu 14:30-16:00
Room: N25#3229 Paik Nam June Hall (백남준홀)

Grading Policy

  • Assignments: 50 %
  • Research Project (50%)
    • Paper review presentation
    • Poster presentation
    • Final report
                       

Schedule

Week Date Topics
1 Feb 26 Course Introduction [pdf]
Feb 28 Audio Representaions [pdf]
Suggested Readings:
Homework #0
2 Mar 5 / 7 Music Classification and Tagging Overview [pdf]
Suggested Readings:
3 Mar 12 / 14 Machine Learning Basics : Supervised Learning [pdf]
Suggested Readings:
  • The PRML book (Chapter 4: Linear Models for Classification)
  • The DL book (Chapter 5: Machine Learning Basics)
4 Mar 19 / 21 Machine Learning Basics: Unsupervised Learning [pdf]
Suggested Readings:
  • The PRML book (Chapter 9: Mixture Models and EM, Chapter 12: Continuous Latent Variables)
Due Apr 4 Homework #1 [Leaderboard]
TA Session: Mar 26 (Classroom, Tue 7pm) [pdf]
5 Mar 26 / 28 Introduction to Deep Learning and Multi-Layer Perceptron [pdf]
Suggested Readings:
  • The PRML book (Chapter 5: Neural Networks)
  • The DL book (Chapter 6: Deep Feedforward Networks)
6 Apr 2 / 4 Training Deep Neural Networks [pdf]
Suggested Readings:
  • The DL book (Chapter 7: Regularization for Deep Learning)
  • The DL book (Chapter 8: Optimization for Training Deep Models)
7 Apr 9 / 11 Convolutional Neural Networks [pdf]
Suggested Readings:
  • The DL book (Chapter 9: Convolutional Networks)
Due Apr 28 Homework #2 [Leaderboard]
TA Session: Apr 16 and 18 (Classroom, Tue 7pm) [pdf]
8 Apr 16 / 18 Midterm Break
9 Apr 23 Convolutional Neural Networks for Music Classification and Tagging [pdf]
Suggested Readings:
Apr 25 Visualization and Style Transfer [pdf]
10 Apr 30 / May 2 Automatic Music Transcription (Part1) [pdf]
- Audio-to-Score Alignment (Music Synchonization)
- Onset Detection
- Rhythm Transcription
Suggested Readings:
  • The FMP book (Chapter 3: Music Synchonization)
  • The FMP book (Chapter 6: Tempo and Beat Tracking)
11 May 7 / 9 Automatic Music Transcription (Part2) [pdf]
- Chord Recognition
- Multi-Pitch Estimation
Suggested Readings:
12 May 14 No Class
May 16 Recurrent Neural Networks and Deep Learning for AMT [pdf]
Suggested Readings:
  • The DL book (Chapter 10: Sequence Modeling)
Due May 30 Homework #3
TA Session: May 23 (Classroom, Thur 7pm) [pdf]
13 May 21 Deep Unsupervised Learning for Music and Audio [pdf]
May 23 No Class (CT Scape)
14 May 28 / 30 Student Presentations: Project Introduction + Previous Work Review + Progress
15 Jun 4 Invited Talk: Jongpil Lee (Music and Audio Computing Lab Ph.D. Student, KAIST)
Jun 6 National Holiday
16 Jun 11 No Class
16 Jun 13 *** Final Presentation (poster) ***

Resources

Software Libraries

References

Courses

* Last year's course webpage is found at this link.