The Impact of Data-driven Positive Programming Feedback: When it Helps, What Happens if it Goes Wrong, and How Students Respond

Link to full Workshop Paper

Abstract: This paper uses a case-based approach to investigate the impact of data-driven positive feedback on students’ behaviour when integrated into a block-based programming environment. We embedded data-driven feature detectors to provide students with immediate positive feedback on completed objectives during programming. We deployed the system in one programming homework in a non-majors CS class. We conducted an expert analysis to determine when data-driven detectors were correct or incorrect, and investigated the impact of the system on student behavior on the homework, specifically in terms of time they spent in the system. Our results highlight when data-driven positive feedback helps students, what happens when it goes wrong, and how this impacted students’ programming behavior. Results from these case studies can shed light on the design of future data-driven systems to provide novices with the positive feedback that can help them persist while learning to program.