Mini Project #1: Audio Visualization
The goal of this project is visualzing sound taken from audio files or microphone in real-time. In each of the following assignments, you are going to extraxt specific features of the sounds and display them using visual components, or transform the basic representations.
1) Loudness Mapping (20 points)
Your job is computing "amplitude envelope" as a loudness measure and mapping it to the radius of a circle so that the result renders a simple imitation of sound propagation. Check out the start-up file below and fill out with your code following the instruction inside the file.
2) Pitch Tracking (30 points)
The second one is real-time pitch tracking using microphone input. The pitch detection method is based on comb-filtering in the frequency domain explained in the class. (note that the algorithm is implemented in the Week 4 Matlab example files). Check out the start-up file below and fill out with your code following the instruction inside the file.
3) MIDI-Note-Scaled Spectrum (20 points)
Scaling the frequency axis of spectrum from linear to MIDI note number (i.e. a log scale) can provide more musical representation of sound. In this mission, you are going to implement it by summarizing the magnitude power at frequency bins neighboring the MIDI note frequencies. You can verify if your code is working correctly using the sine tone with diatonic scale.
4) Your Own Audio Visualization (30 points)
Using audio features or spectrum above (but not limited to them), make your own creative (and artistic) audio visualization. If you find some cool visualizations on Internet (or you can refer to examples below), you are allowed to adopt the javascript code as part of your submission. However, you must report the origin (e.g. URLs) and clearly describe your contribution.
Some ideas
- Mapping loudness to geometric parameters or color of images (e.g. size, position)
- Mapping pitch (from real-time micro input) to geometric parameters or color of images
- Mapping discrete pitches (from real-time micro input) to the movement of different images
- Multi-band sound level meter (e.g. 10 or 20 bands)
- Oscilloscope
- ...
Some interesting (but quite advanced) examples
- http://mattdesl.github.io/spins/
- http://srchea.com/apps/sound-visualizer-web-audio-api-webgl/
- http://mdn.github.io/violent-theremin/
- http://preziotte.com/partymode/
- http://tybenz.com/post/visualizr/
- http://airtightinteractive.com/demos/js/reactive/
- http://chromium.googlecode.com/svn/trunk/samples/audio/visualizer-gl.html