Log-Based Multidimensional Measurement of CT Acquisition

Abstract

Computational thinking (CT) has been proven challenging to conceptualize and assess. When assessing CT using problem-solving tasks, it is commonly measured based on achievements, that is, in a unidimensional summative way. However, this traditional measurement neglects to consider vital components of the learning progress, which may produce a richer, formative assessment. Using the log files drawn from an online learning platform for CT (Kodetu), we suggest a nuanced evaluation of CT acquisition which consists of four variables: number of attempts to solve a problem; time to solution; application of newly presented CT concept; and solution originality. The research population included 189 middle-school students who participated in a workshop aimed at promoting CT and creativity. Using a learning analytics approach, we analyzed data from a log file documenting 1478 student-task pairs. Findings suggest that these variables share some common features that make them suitable for assessing CT acquisition. Furthermore, the variables grasp different aspects of the learning progress; hence, taken together, they allow for a richer evaluation of CT acquisition. These results shed light on the importance of using diverse metrics to examine CT and contribute to the proliferation of assessment practices.