Media Technology MSc

Course Material “Perceptualization”

Spring 2016
by Edwin van der Heide & Maarten Lamers

Media Technology MSc program, Leiden University

The term “perceptualization“ was coined specifically for this course. It describes the translation of signals and information to modalities that appeal to any of the human senses. As such, it generalizes the terms “visualization“ and “sonification“ to include all other senses. We study such perceptualizations, with particular focus on how properties of a perception systems can be used to optimally convey information. Theory, practice, history and examples of information perceptualization are studied and discussed. Lectures are combined with reading homework, student presentations and a student project. Full attendance is compulsory.

Students from outside the Media Technology MSc program who wish to participate should contact the program coordinator.

Lecturers Maarten Lamers and Edwin van der Heide
Contact via perceptualization@liacs.leidenuniv.nl
Level/credits level 400 (specialized course), 3 EC
Lecture dates see the Media Technology calendar
Location room 413 of the Snellius building
Communication via the Media Technology forum under "course announcements". Check it regularly!
Course vault the course vault is password protected
Grading Final grade is composed of homework writing assignment 3 (40%) and final project (60%), rounded to the nearest grade accepted by Leiden University's grade adminstration.

 

 

Course Schedule

Lecture Date Topic Homework due
1 Jan 6, 2016 Course intro & introducing the senses I -
2 Jan 20, 2016 Introducing the senses II Assignment 1
3 Jan 27, 2016 Translation & interpretation I Assignment 2
4 Feb 3, 2016 Translation & interpretation II -
5 Feb 10, 2016 Examples of perceptualization systems Assignment 3
6 Feb 24, 2016
9:30-14:00
Project presentations Final project

 

Assignment 1

Read:

Assignment 2

Read:

Assignment 3

Find an existing perceptualization system. It should make a signal or stream of information perceivable and interpretable, in a way other than visualization. It can be a scientific or artistic work.

Write a paper of max 2 pages about your finding, in which you describe the system and create a debate around it. Dedicate one-third of the paper to describing the system, and two-thirds to the debate and reflection. Be critical, and include at least some critique. Main aspects to consider are the translation and interpretation of information. What do you think of the choices made in the system? What did they neglect? Discuss your findings in a broader context then just the system itself (compare it to alternative systems or current theories).

This assignment must be done individually. Send your paper to perceptualization@liacs.leidenuniv.nl before lecture 5. Use PDF format if possible. Also, prepare a short presentation (under 5 minutes) about it to present during lecture 5.

Final Project

Create a perceptualization system. It should make a signal or stream of information perceivable and interpretable, in a way other than projecting it onto a display screen. It should map dimensions (or qualities) of the input signal/data onto dimensions (or qualities) of the human sense(s). How these dimensions are mapped onto each other is most important in the evaluation/grading of the work.

Demonstrate your system to the class in under 5 minutes, in such a way that others can experience it. Provide at least 2 different input streams or data sets for the demonstration. Explain the mapping/translation that you made, and give your system a name.

In 1 page, describe your system, the mapping, and language/interpretation issues such as strengths and weaknesses. Send your paper to perceptualization@liacs.leidenuniv.nl before the presentation session. Use PDF format if possible. This assignment must be done in teams of 2 students.

Bibliography

For reasons of copyright protection, some of the articles can be downloaded from within the university's network only.

[Hermann 2011] T Hermann, A Hunt, and JG Neuhoff (Eds.), The Sonification Handbook, Logos Verlag Berlin GmbH, 2011
[Cain] WS Cain, R de Wijk, C Lulejian, F Schiet, and L-C See (1998), Odor Identification: Perceptual and Semantic Dimensions, Chemical Senses Vol 23, pp 309-326
[Salisbury] K Salisbury, F Conti, and F Barbagli (2004), Haptic Rendering: Introductory Concepts, IEEE Computer Graphics and Applications, March/April 2004, pp 24-32
[Chouvardas] VG Chouvardas, AN Miliou, and MK Hatalis (2008), Tactile Displays: Overview and Recent Advances. Displays Vol 29 Num 3, pp 185-194
[xSonify] http://spdf.gsfc.nasa.gov/research/sonification/documents/Chapter1.pdf">Introduction of Sonification, Chapter 1 of documentation for xSonify sonification software, NASA
[Sturm] Bob L. Sturm (2002), http://imi.aau.dk/~bst/publications/Sturm2002.pdf">Surf Music: Sonification of Ocean Buoy Spectral Data, Proceedings of the International Conference for Auditory Display, Kyoto (Japan), July 2002
[ICAD] International Community for Auditory Display, www.icad.org
[Hermann 1999] T. Hermann and H. Ritter (1999), Listen to your Data: Model-Based Sonification for Data Analysis, In Advances in Intelligent Computing and Multimedia Systems, G.E. Lasker (Editor), International Institute for Advanced Studies in System Research and Cybernetics, pp. 189–194
[Gilbert] Avery N. Gilbert, Robyn Martin, and Sarah E. Kemp (1996), Cross-modal Correspondence Between Vision and Olfaction: The Color of Smells, The American Journal of Psychology, Vol 109 Num 3, pp. 335-351
[Bach-y-Rita] Paul Bach-y-Rita and Stephen W. Kercel (2003), Sensory Substitution and the Human-Machine Interface, Trends in Cognitive Sciences, Vol 7 Num 12, December 2003, pp. 541-546
[Robles-De-La-Torre] Gabriel Robles-De-La-Torre (2006), The Importance of the Sense of Touch in Virtual and Real Environments, IEEE Multimedia, July-September 2006, pp 24-30
[Brewster] S.A. Brewster and L.M. Brown (2004), Tactons: structured tactile messages for non-visual information display, Australasian User Interface Conference, 18-22 January 2004, ACS Conferences in Research and Practice in Information Technology, Vol 28, pp. 15-23
[Benali-Khoudja] Mohamed Benali-Khoudja, Moustapha Hafez, Jean-Marc Alexandre, and Abderrahmane Kheddar (2004), Tactile Interfaces: a State-of-the-Art Survey, 35th International Symposium on Robotics, Paris, March 2004
[Elsenaar] Arthur Elsenaar and Remko Scha (1997), Arthur & The Solenoids, video on youtube.com
[Ando] Hideyuki Ando, Tomofumi Yoshida, Taro Maeda, and Junji Watanabe (2007), Save YourSelf: Galvanic Vestibular Stimulation Interface, website
[Kaye] Joseph Kaye (2004), Making Scents: Aromatic Output for HCI, Interactions, Vol 11 Issue 1, Jan/Feb 2004
[Washburn] Donald A. Washburn and Lauriann M. Jones (2004), Could Olfactory Displays Improve Data Visualization?, Computing in Science and Engineering, Vol 6 Num 6, pp. 80-83, Nov/Dec 2004
[Turin] Luca Turin (2005), Luca Turin on the Science of Scent, talk on TED.com, February 2005
[Loftin] R. Bowen Loftin (2003), Multisensory Perception: Beyond the Visual in Visualization, Computing in Science and Engineering, Vol. 5 Num 4, pp. 56-58, Jul/Aug 2003
[Irving] Lucy Irving, Sensory Substitution project, www.sensorysubstitution.co.uk
[Fisher] Madeline Fisher (2007), Balancing Act (about the work of Paul Bach-y-Rita and Mitch Tyler), On Wisconsin magazine, Spring 2007
[Warwick 2004] Kevin Warwick and Mark Gasson (2004), Practical Interface Experiments with Implant Technology, in "Computer vision in human-computer interaction: ECCV 2004 Workshop on HCI", Nicu Sebe, Michael Lew, Thomas Huang (eds), LNCS 3058, May 2004, pp. 7-16.
[Warwick 2005] Kevin Warwick, Mark Gasson, B Hutt, and I Goodhew (2005), An attempt to extend human sensory capabilities by means of implant technology, IEEE Int Conf on Systems, Man and Cybernetics, October 2005, pp. 1663-1668
[Warwick 2008] Kevin Warwick (2008), Upgrading Humans via Implants - Why Not?, 19: Interdisciplinary Studies in the Long Nineteenth Century, Issue 7 "Mind, Body, Machines", October 2008
[Heimbecker] Steve Heimbecker (2003), Wind Array Cascade Machine
[Haptic Radar] Alvaro Cassinelli, Carson Reynolds, and Masatoshi Ishikawa (2006), The Haptic Radar; Extended Skin Project
[Infosthetics] Information Aesthetics website, infosthetics.com