Immediate Data-Driven Positive Feedback Increases Engagement on Programming Homework for Novices

Link to full Workshop Paper

Abstract: Learning theories and psychological research show that positive feedback during practice can increase learners’ motivation, and correlates with their learning. In our prior work, we built a system that provides immediate positive feedback using expert-authored features, and found a promising impact on students’ performance and engagement with the system. However, scaling this expert-feedback system to new programming tasks requires extensive human effort. In this paper, we present a system that provides automated, data-driven, immediate positive feedback (DD-IPF) to students while programming. This system uses a data-driven feature detector that automatically detects feature completion in the current student’s code based on features learned from historical student data. To explore the impact of DDIPF on students’ programming behavior, we performed a quasi-experimental study across two semesters in a blockbased programming class. Our results showed that students with DD-IPF were more engaged, as measured by time spent on the programming task, and also showed marginal improvement in their grades, compared to students in a prior semester solving the same task without feedback. This suggests that positive feedback based on data-driven feature detection can provide benefits in student engagement and performance. We conclude with design recommendations for data-driven programming feedback systems.