Unheard: Quick Visual Assessment of Sounds in Ad Hoc Musical Software Environments
This research takes a practical approach to the visualization of sound: the extension of the ubiquitous waveform figure with textures, lines and colors results in an approximating and compact representation of sound samples within a musical context. Auditive properties can thus be 'read' to a certain extent, making it possible to estimate the musical possibilities of the sounds by sight, without necessarily having listened to them. Perceptual sound features are being analyzed and their values mapped to visual features using mostly intuitive and non-technical audio-visual correspondences based partly on empirical perception studies. This system was put to the test in a live performance in which a piece of music was composed on the spot out of previously unheard sounds.
Maarten Wesselius, "Unheard: Quick Visual Assessment of Sounds in Ad Hoc Musical Software Environments", Master's Thesis for the Media Technology programme, Leiden University (The Netherlands), 2010