A Game to Crowdsource the Labeling of Affective Facial Expressions is Comparable to Expert Ratings
This paper demonstrates the use of a crowdsourced human computation game to accumulate annotations from non-experts as a means to provide labels for an affective facial expression database. To do so, a human computation game is played, in which players are encouraged to ask each other related facial expression questions. These questions are based on the Facial Action Coding System. Emphasis is placed on the participant’s overall understanding of the task and on the ease-of-use of the game so that labeling accuracy is reinforced. Additional game mechanics can be used in future work to encourage players to keep playing the game. This crowdsourced labeling of an affective facial expressions database is important because the manual labeling of an affective database can be relatively expensive and time consuming. Our game shows that non-experts are comparable in labeling our affective database based on the ground truth.
Barry Borsboom, "A Game to Crowdsource the Labeling of Affective Facial Expressions is Comparable to Expert Ratings", Master's Thesis for the Media Technology programme, Leiden University (The Netherlands), 2012