Keywords
self-regulated learning
Abstract
This study investigated the impact of automated feedback granularity and adaptive sequencing on novice programmers' conceptual understanding, engagement, and self-efficacy in introductory CS courses. We employed a mixed-methods quasi-experimental design comparing three feedback conditions: coarse-grained non-adaptive, fine-grained non-adaptive, and fine-grained adaptive. Quantitative analysis of 150 students' performance, engagement, and self-efficacy measures over a 15-week semester revealed significant advantages for fine-grained, adaptive feedback. This condition showed higher post-test scores, sustained engagement, and increased self-efficacy compared to other conditions. Qualitative analysis of student interviews illuminated the mechanisms behind these effects, highlighting the importance of targeted, responsive feedback in scaffolding understanding and maintaining motivation. Our findings contribute to the design of effective automated feedback systems in CS education and offer insights into addressing retention issues in introductory programming courses.
References
Agathe Adam and Jean-Pierre Laurent. 2014. AutoTeach: automatically generating and tailoring automated feedback for programming assignments. In 2014 IEEE 14th International Conference on Advanced Learning Technologies. IEEE, 144–148.
Satabdi Basu, Gautam Biswas, and John S Kinnebrew. 2017. Detecting student misconceptions about programming in introductory computer science courses. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education. ACM, 381–386.
Jens Bennedsen and Michael E Caspersen. 2007. Failure rates in introductory programming. ACM SIGCSE Bulletin 39, 2 (2007), 32–36.
Jennifer M Blaney and Jane G Stout. 2018. Examining the relationship between introductory computing course experiences, self-efficacy, and belonging among first-generation college women. Proceedings of the 49th ACM Technical Symposium on Computer Science Education (2018), 421–426.
Alex J Bowers and Matthew Berland. 2018. The evolution of computing: Implications for data science and the computing curriculum. Technology, Knowledge and Learning 23, 1 (2018), 1–16.
Lori Carter. 2011. Factors influencing students' choice of computer science as a major. In Proceedings of the 42nd ACM technical symposium on Computer science education. ACM, 413–418.
Benjamin Clement, Didier Roy, Pierre-Yves Oudeyer, and Manuel Lopes. 2015. Multi-armed bandits for intelligent tutoring systems. Journal of Educational Data Mining 7, 2 (2015), 20–48.
Sarah Heckman and Jason King. 2018. The impact of automated grading on learning and course outcomes. Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE (2018), 147–152.
Petri Ihantola, Tuukka Ahoniemi, Ville Karavirta, and Otto Sepp"al"a. 2010. Review of recent systems for automatic assessment of programming assignments. Proceedings of the 10th Koli calling international conference on computing education research (2010), 86–93.
Hieke Keuning, Bastiaan Heeren, and Johan Jeuring. 2019. Code quality issues in student programs. Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (2019), 507–513.
Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2018. A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education (TOCE) 19, 1 (2018), 1–43.
Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2018. A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education (TOCE) 19, 1 (2018), 3.
Suin Kim, Jae Won Kim, Jungkook Park, and Alice Oh. 2016. Automatic generation of programming feedback: A data-driven approach. In Proceedings of the 2016 ACM Conference on International Computing Education Research. ACM, 23–32.
P"aivi Kinnunen and Beth Simon. 2012. Failing a programming course: The students' perspective. Learning and Teaching in Computing and Engineering (LaTiCE), 2012 (2012), 79–86.
Amruth N Kumar and Soham Singhal. 2019. Personalized learning in introductory programming: A longitudinal study. ACM Transactions on Computing Education (TOCE) 19, 2 (2019), 1–29.
Jae Young Lee and Emma Brunskill. 2021. Adaptive feedback for programming assignments: A comparative study. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 1082–1088.
Michael J Lee and Andrew J Ko. 2015. Personifying programming tool feedback improves novice programmers' learning. In Proceedings of the eleventh annual International Conference on International Computing Education Research. ACM, 109–116.
Alex Lishinski, Aman Yadav, Jonathon Good, and Richard Enbody. 2016. Learning to program: Gender differences and interactive effects of students' motivation, goals, and self-efficacy on performance. Proceedings of the 2016 ACM Conference on International Computing Education Research (2016), 211–220.
Dastyni Loksa, Andrew J Ko, Will Jernigan, Alannah Oleson, Christopher J Mendez, and Margaret M Burnett. 2016. Programming, problem solving, and self-awareness: effects of explicit guidance. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1449–1461.
Samiha Marwan, Joseph Jay Williams, and Thomas W Price. 2019. An evaluation of the impact of automated programming hints on performance and learning. In Proceedings of the 2019 ACM Conference on International Computing Education Research. 61–70.
Andy Nguyen, Chris Piech, Jonathan Huang, and Leonidas Guibas. 2016. Towards effective programming feedback for introductory computer science. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education. 462–467.
Clayton Ott, Anthony Robins, and Kerry Shephard. 2018. Learning analytics in a teacher-led design of a mobile learning app. British Journal of Educational Technology 49, 2 (2018), 236–251.
Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. In Advances in neural information processing systems. 505–513.
Yizhou Qian and James Lehman. 2017. A systematic literature review of novice programming misconceptions. Journal of Computer Assisted Learning 33, 5 (2017), 482–496.
Vennila Ramalingam, Deborah LaBelle, and Susan Wiedenbeck. 2004. Self-efficacy and mental models in learning to program. ACM SIGCSE Bulletin 36, 3 (2004), 171–175.
Kelly Rivers and Kenneth R Koedinger. 2017. Data-driven hint generation in vast solution spaces: a self-improving python programming tutor. International Journal of Artificial Intelligence in Education 27, 1 (2017), 37–64.
Richard M Ryan and Edward L Deci. 2000. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist.
Valerie J Shute. 2008. Focus on formative feedback. Review of educational research 78, 1 (2008), 153–189.
Rishabh Singh, Sumit Gulwani, and Armando Solar-Lezama. 2013. Automated feedback generation for introductory programming assignments. In Proceedings of the 34th ACM SIGPLAN conference on Programming language design and implementation. ACM, 15–26.
Allison Elliott Tew and Mark Guzdial. 2011. The FCS1: a language independent assessment of CS1 knowledge. Proceedings of the 42nd ACM technical symposium on Computer science education (2011), 111–116.
Kurt VanLehn. 2006. The behavior of tutoring systems. International journal of artificial intelligence in education 16, 3 (2006), 227–265.
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
Copyright (c) 2024 Yu Ma, Chao Wang (Author)