ISEDJ

Abstract: This study investigates the effectiveness of a Retrieval-Augmented Generation (RAG) chatbot to enhance learning and engagement in a self-paced, asynchronous online R programming course. To contextualize the development and potential of RAG chatbots, we conducted a literature review on existing approaches and their use in educational settings. Following this, a chatbot powered by generative artificial intelligence (GenAI) was designed to provide tailored conceptual explanations and code examples based on course materials, addressing a range of student inquiries. To evaluate its effectiveness, the study analyzed chatbot interaction logs and survey responses collected at the end of the course. Results showed that students with greater prior knowledge of the subject matter were more likely to engage with the chatbot, primarily seeking help on advanced topics not covered in the course lectures. Overall, students expressed high satisfaction with the chatbot, particularly valuing its ability to provide helpful explanations that are based on the course materials. This study highlights the potential of GenAI, and RAG chatbots specifically, to enhance online education and provides practical insights for future implementations.

Download this article: ISEDJ - V23 N2 Page 4.pdf

Recommended Citation: Lang, G., Gurpinar, T., (2025). AI-Powered Learning Support: A Study of Retrieval-Augmented Generation (RAG) Chatbot Effectiveness in an Online Course. Information Systems Education Journal 23(2) pp 4-13. https://doi.org/10.62273/ZKLK5988