ISEDJ

Abstract: Generative AI has recently gained the ability to generate computer code. This development is bound to affect how computer programming is taught in higher education. We used past programming assignments and solutions for textbook exercises in our introductory programming class to analyze how accurately one of the leading models, ChatGPT, generates solutions. We selected the ChatGPT-4 available through the Bing search engine for our testing. We used a one-tailed test to calculate success percentage of the textbook versus ChatGPT solutions to determine if there was a statistically significant difference. Neither the book nor ChatGPT provided perfect solutions. Analysis of the results showed that the generated code does not always meet the programming requirements, but also that instructions for generative AI coding and for traditional programming can be improved. We conclude with recommendations for incorporating generative AI in programming classes.

Download this article: ISEDJ - V23 N1 Page 4.pdf

Recommended Citation: Bekkering, T.E., Harrington, P., (2025). A Comparison of Generative AI Solutions and Textbook Solutions in an Introductory Programming Course Information Systems Education Journal 23(1) pp 4-22. https://doi.org/10.62273/YQWP1758