Media Technology MSc

What You Touch Is What You Get


Often when interacting with a user interafce (UI), we may find ourselves not knowing what to do next, exactly. This could be something as simple as: "Where is that key on my keyboard again?" Or it could be something as relatively high-level as: "Which terms for this web search will get me to the relevant results?" Trying our luck, we may end up with pressing the wrong key or entering the wrong search terms. We then have to stop, undo our mistake, and start over again. This interrupts the main flow of interaction. One way to address this is to design exploratory interactions, to replace what are de facto trial-and-error interactions. Analogous to the well-known "What You See Is What You Get" (WYSIWYG) paradigm, WYTIWYG uses touch input to preview the results of a possible interaction in a way closely matching the actual results.

This is one of two Media Technology entries for the UIST 2009 Student Innovation Contest. The goal of the contest is to develop new interactions on unique hardware. Student groups were supplied with a prototype, pressure-sensitive keyboard designed by Microsoft Hardware. Each key is capable of reporting its press force (8-bit resolution). Within exactly one month, students had to develop a useful or fun application, implement its prototype and demonstrate it.


Staas de Jong, Jeroen Jillissen, Dünya Kirkali, Alwin de Rooij, and Arnout Terpstra, WYTIWYG: What You Touch Is What You Get. Poster and prototype presented at the ACM Symposium on User Interface Software and Technology (UIST 2009) Student Innovation Contest, October 5-6 2009, Victoria (Canada)