Motivation Research has shown to us, that people are lying often, at least once a day. These lies occur not only when dealing with other humans but also when interacting with systems like questionnaires. Lot of questionnaires nowadays are performed online and the results of the questionnaires shall be accurate. Therefore we need to make sure, that the answers are truthful. Today we deal with issue of non honest answers most often via questioning more people than necessary which should “smooth out” our results in a way, that if in our test group there are some objects that are not honest, quantity of honest subjects will have stronger effect on results. This has however its downsides. First of all, we need more subjects than necessary and acquiring subjects is often not easy and not cheap. Second of all, if the questions are very personal or emotional, more of our subjects may choose to be not honest. In that case, even more test subjects does not help us. If we could differentiate honest answers from non honest automatically, we could reduce number of subjects required and also be able to choose only honest answers for our purposes. To do this, we can focus on implicit feedback from subjects such as eye movements or response times that can serve as indicators for non honest answers. Evaluation We have performed several smaller and two major experiments at UX group lab at the Research Centre of User Experience and Interaction at FIIT STU Bratislava with more than 120 subjects altogether.UX group lab at the Research Centre of User Experience and Interaction at FIIT STU Bratislava After several pilot experiments during which we have improved our methodology the final experiment was attended by 60 subjects. Each subject answered 60 different questions, 30 in honest condition and 30 in non honest condition. For each subject we have collected 60 gaze points per second, pupil diameter and all actions that subjects have performed interacting with questionnaires.Equipment used for experiment Based on our hypotheses we have created five different metrics that should indicate non honest answers - pupil dilation, number of fixations, longest fixation, average duration of fixation and the first fixation for particular answer. With strongest metrics we have achieved precision of 61/67 percent with recall 51/75 percent for honest and non honest answers respectively. We have also evaluated impact of each metric on accuracy and have found out, that the first fixation was very strong indicator of non honest behavior.Publications
|
|
||||||||||||||||||||||||||||||||||||||||||||||
|
|