Designing a Usability Assessment Process for Adaptive Intelligent Tutoring Systems: A Case Study

Abstract

This study aims at examining the usability evaluation process of ArtiBos, an adaptive intelligent tutoring system specifically designed for teaching of problem solving, and proposing usability evaluation recommendations accordingly. The study group consists of 90 students selected from 3 different secondary education institutions and 30 experts from the fields of Computer Education and Instructional Technology and Mathematics Teaching. The usability of ArtiBos is evaluated according to the criteria of effectiveness, efficiency, and satisfaction included in the definition of usability by the International Standards Organization (ISO). In this study, quantitative and qualitative data were used for triangulation. Study data included users’ screenshots during performance of the instructed tasks in ArtiBos, sound recordings taken from the think aloud technique, interviews with students, and opinions of experts. The study elaborates the procedure of usability tests of ArtiBos and makes suggestions for usability evaluation processes of similar systems. The results are expected to guide researchers planning to carry out similar systems’ usability studies.