Multimodal Software For Affective Education: User Interaction Design And Evaluation

Multimodal systems allow user interaction through various methods, or modalities, such as gesture, gaze, speech or head and body movements. There have been many frameworks proposed for the design of multimodal systems, but they focus principally on issues of technical architecture. Some applications for multimodal systems, including education, however, involve issues relating to affective (emotional) user experience.The thesis presents our experience with creating and applying a framework for multimodal software design and evaluation, with a focus on affect (i.e. emotion) in education. Our goal is to support the system designer, and indirectly also teachers and students. We outline our conceptual basis, adaptation of interaction design and evaluation techniques, and our experience over several case studies. In particular, we describe how our techniques were refined after each case study, and how the evaluation techniques led us to revisit our design techniques. We also gained new insight about the role of the teacher, leading us to develop a new affect visualization dashboard for teachers.