The Thermos project developed a feedback instrument (THERMOS dashboard) that provides students with insight and actionable feedback on their study behavior. The developed dashboard was piloted in six study programs in 20/21, across different faculties of Utrecht University (UU), including those located at University Medical Center Utrecht (UMCU). The follow-up project ‘Thermos for UU’ starts in 21/22 and will further focus on the implementation process in collaboration with more study programs across UU and the UMCU.

Go to the project website for more videos 

Background

Students learn best when they have insight in their own study behavior and motivation and know how to self-regulate. The Thermos project gives students insight in their study behavior and motivation by means of Learning Analytics. Learning Analytics revolves around gathering data about learners and their learning processes to better understanding and where possible improve them. It is often defined as “the measurement, collection, analysis and reporting of data, about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (Siemens & Baker, 2012). Presenting students with data about their study motivation and study behavior may help them improve on aspects they choose. However, simply gathering and visualizing data may not necessarily work, as interpreting and acting upon data (visualizations) is not always easy (Jivet et al., 2020). Implementing a Learning Analytics Dashboard (LAD) into a meaningful educational context can increase the value for learners (Wise et al., 2016).

Project description

From academic year 18/19 to 20/21, the Thermos project developed a Learning Analytics Dashboard (LAD) to support students in their studies. This dashboard gives insight in several aspects of study behavior (e.g. motivation and group work skills) and study progress (e.g. ECTS). It also provides users with actionable feedback to engage with these aspects such as individual exercises, and additional referrals to (for example) Skills lab workshops or an online studycoach.

Figure 1. Screen shot of THERMOS dashboard

In academic year 20/21, the dashboard was piloted in 6 study programs from different faculties of UU and the UMCU. In this pilot we explored the dashboard’s usability and usefulness for students, and ways to implement it in a study program.

From 21/22, the follow-up project ‘Thermos for UU’ starts, which focusses more on the process of implementing the dashboard into tutorprograms. Within this follow-up project, the dashboard will be implemented in more study programs across both UU and UMCU.

Goals

The overall goal is to support students in improving aspects of study behavior by: helping them to assess important aspects of study behavior, to communicate students’ study behavior profile to provide them with insight, and to help them develop through actionable feedback.

Results

The dashboard was implemented in the tutorprogram of six studyprograms across UU and UMCU in academic year 20/21. This pilot evaluated the dashboard’s usability, usefulness, and the way in which it was implemented across study programs. Overall, the dashboard’s usability was already good, students understood and were able to interpret the different parts. As it was suggested that usability could be improved upon by developing a Dutch version of the dashboard, this is will be available at the start of the Thermos for UU project. The dashboard’s experienced usefulness varied, with both positive (e.g. remarks such as “I learned more about my study behavior and some useful tips”) and less positive evaluations (“At this moment I’m not sure in which way the dashboard could help my study behavior”). One of the goals for the Thermos for UU project is further investigation of the dashboard’s usefulness and potential improvements. The implementation process varied across study programs, as well as the reasons for these differences. Based on interviews and a focusgroup, a concept implementation format was developed, aimed to help study programs make deliberate choices about the manner of implementation. Further investigation of the implementation process is the main aim for the Thermos for UU project.

Further reading

• See the Thermos website for more information
• Log-in to the THERMOS dashboard [Log in with SolisID]
• EU OFLA project (Onwards from Learning Analytics)
• STELA (Successful Transition in Education using Learning Analytics) from the KU Leuven

References

• Jivet, I., Scheffel, M., Schmitz, M., Robbers, S., Specht, M., & Drachsler, H. (2020). From students with love: An empirical study on learner goals, self-regulated learning and sense-making of learning analytics in higher education. The Internet and Higher Education, 47, 100758.
• Lang, C., Siemens, G., Wise, A., & Gasevic, D. (Eds.). (2017). Handbook of learning analytics. SOLAR, Society for Learning Analytics and Research. DOI: 10.18608/hla17
• Wise, A. F., Vytasek, J. M., Hausknecht, S., & Zhao, Y. (2016). Developing Learning Analytics Design Knowledge in the” Middle Space”: The Student Tuning Model and Align Design Framework for Learning Analytics Use. Online Learning, 20(2), 155-182.
• Siemens, G., & Baker, R. S. D. (2012, April). Learning analytics and educational data mining: towards communication and collaboration.
  In Proceedings of the 2nd international conference on learning analytics and knowledge (pp. 252-254).