The Right Kind of Help: Evaluating the Effectiveness of Intervention Methods in Elementary-Level Visual Programming
Authors
Ahana Ghosh
Liina Malva
Alkis Gotovos
Danial Hooshyar
Adish Singla
Abstract
Prior work has explored various intervention methods for elementary programming. However, the relative impact of these methods during the learning and post-learning phases remains unclear. In this work, we present a large-scale study comparing the effectiveness of various intervention methods in elementary programming both during learning and on novel tasks post-learning. Specifically, we compare three intervention methods: code-edit recommendations (Code-Rec), quizzes based on code edits (Code-Quiz), and quizzes based on metacognitive strategies (Plan-Quiz), along with a no-intervention control (group None). A total of 398 students (across grades 4-7) participated in a two-phase study: learning phase comprising write-code tasks from the Hour of Code: Maze Challenge with the intervention, followed by a post-learning phase comprising more advanced write-code tasks without any intervention. All intervention methods significantly improved learning performance over the control group while preserving students' problem-solving skills in the post-learning phase. Quiz-based methods further improved performance on novel post-learning tasks. Students in intervention groups also reported greater engagement and perceived skill growth.
