Student Reactions to Online Course Delivery – a Contrast Between Fulltime and Part-time Students Kenneth A. Grant kagrant@ryerson.ca Franklyn Prescod fprescod@gwemail.ryerson.ca Herman Ho (Student) h2ho@ryerson.ca School of Information Technology Management Faculty of Business, Ryerson University Toronto, Ontario M5B 2K3, Canada Abstract Evaluation of the delivery of the same course in a distance format to two differing classes of students, one of fulltime degree students, the other of part time students in a variety of programs, produced strong support for the delivery method. While concerns have been raised about the suitability of distance delivery techniques for younger, less mature students and about different learning styles, this comparison found few significant differences between the participation, performance and satisfaction of such students when compared to a more mature group of part-time students for whom distance education was the method of choice. Further, both groups rated the experience equal or superior to conventional in-class delivery. Keywords: e-learning, distance education, teaching methodologies, learning styles, eBusiness 1. INTRODUCTION This paper describes the results of an assessment of student experience in an Internet-delivered distance education course. The investigation looks at the experience of two distinct groups – a class of distance education students (normally expected to be part time and more mature) and a class of fulltime undergraduate students, who had registered for a “normal” daytime classroom-based course, but were switched to a distance delivery mode because of a scheduling difficulty for the course professor. In addition, the professor’s School had a mandate to investigate alternate methods of course delivery and this was seen as an appropriate (albeit opportunistic) initiative. For the remainder of this paper, these two groups will be referred to as the Distance Class and the Day Class. All the students were taking either an undergraduate business degree (full or part-time) or a continuing education department certificate. 2. RESEARCH OBJECTIVES The primary purpose of this research was to obtain a clearer understanding of the experiences of students studying in a distance education mode, as well as to obtain feedback that would help improve the design of the course. In addition, the delivery of the course in parallel to two different types of student allowed the comparison of the attitudes and experience of fulltime undergraduate day students, who normally study in the traditional lecture/lab mode, with that of students who chose to take the course at a distance and were likely to be older, part-time and in a wider range of academic programs. (It should be noted that five members of the Distance Class were, in fact, full-time day students who had already chosen to study in this mode, perhaps because of scheduling conflicts in their other day classes). 3. METHODOLOGY At the end of the course, all students were invited to complete an online questionnaire, which examined their experience and solicited their opinions on the course and its method of delivery. To reduce the possibility of bias in response and protect the students privacy rights, these data were collected using a method approved by the university research ethics committee and were not made available to the professor until the students’ grades had been submitted. The 50-point questionnaire used consisted mainly of closed questions using a 5-point Likert scale (1 being low and 5 being high) along with some demographic and open response questions. The questionnaire was of a similar nature to that used by the University for course/instructor evaluation. After the course was completed, the student responses were combined with data on their course activities and course performance and examined further. This paper presents the results of this analysis. 22 out of 26 students (85%) responded from the Distance Class and 36 of 51 (71%) from the Day Class. Five of the Distance Class responses were from full-time students, who had already chosen the distance delivery offering. In addition, the Day Class, as part of the full-time degree program was asked to fill out the University’s standard course assessment (covering both the course and the instructor) and these results have also been included in this paper. No equivalent assessment is done for the Distance programs. This paper very much reflects an action research perspective. The primary researcher is the course instructor, the second researcher teaches the same course in the classic manner, and is completing his doctoral research in computing technology in education at Nova Southeastern University, Florida. Both subscribe to the belief in learning by doing and in using the feedback from the research to positively influence the project outcome. 4. ABOUT THE COURSE The Course is a 2nd/3rd year business elective, “Concepts of eBusiness,” designed to be taken as a standalone course or as the first course in a 6-course eBusiness Minor, within any of the 4-year full and part-time Bachelor of Commerce (BCom) degrees offered by the University; or in an 8-course Continuing Education Certificate in eBusiness. It has been offered in the traditional classroom mode for about 5 years (with regular content updates) and as a Distance Education class for part-time students for 4 years. The Concept of eBusiness course is one of the most popular of the general electives offered, taken by 200-300 students each year, with about 1/3 doing so at a distance. The original development of the course was reported in Grant et al (2001). The Distance Education version of the course was developed with a number of guiding principles, drawn from both the literature review (for example, Mitchell, C., Dipetta, T., and Kerr, J., 2001;) and practical experience: > As far as possible, all course material would be available online > Wherever possible, existing Internet sources would be used (do not “reinvent” the wheel”). > Student work is structured into weekly modules, with each week requiring student action, (not simply reading). This was done both to improve the learning experience and to provide a structured framework that, while allowing them flexibility within each week on when they might access the material, required students to do the work during the weekly period (thus avoiding falling behind – a key problem in self learning situations). > Learning events included modularized reading, interactive “click-throughs, hands-on labs and exercises and weekly discussion activities. > The grading system reflected the various expected activities, with “rewards” for active participation (in essence, students who complete each week’s work steadily build up their class mark each week and are virtually assured of passing the course with an acceptable grade. On the other hand, high grades were only possible for steady and high quality effort throughout. > Regular access (electronically) to the course professor, normally on a daily basis. The Distance Education version is delivered through the Blackboard™ Course Management System and is structured as follows: > Thirteen 1-week modules of course content > Six short weekly labs/assignments > Two 3-week Projects (one to build an simple eBusiness website and one to write a research paper on an assigned eBusiness). > A midterm and a final exam (both are set online and are open book. The midterm is done from home, the final is supervised in a university lab or by a proctor) to ensure that a significant part of the student’s grade comes from a supervised activity. > A series of discussion boards (One for graded discussion, another three for in-course administration, project work issues and assignment/exam questions and feedback). > A private message feature, which allows private email communication between students and professor within the course. The weekly material consists of assigned readings (normally a chapter from a text book and a series of click-through links to relevant sites, as well as a set of Weekly Notes, including relevant click-throughs (a “lecture replacement”). All essay assignments and the essay parts of the two exams are submitted to the plagiarism site of Turnitin.com – and are marked directly from the electronic image submitted there. Grades and feedback are provided in the Blackboard™ Gradebook. In this case, the Distance Class was delivered in the normal manner over the Internet, the Day Class received the same Internet material (with very slight administrative modifications) and also received two face-to-face lectures – one at the beginning of the course to explain the change in delivery method and one at its midpoint, intended as a tutorial. 5. RELATED LITERATURE REVIEW The use of the Internet in the delivery of education has grown significantly in recent years (Beyth-Marom, Chajut, Roccas, and Sagiv, 2003). This method is used both as the primary vehicle of delivery for some distance education institutions and as a support for more traditional course delivery, often using course management software such as that provided by Blackboard™ or WebCT™. Research has shown that there is a lack of emphasis on the pedagogical effectiveness of this mechanism (Mitchell et al., 2001; Lu, J., Yu, C., & Liu, C., 2003;). Davies (2003) supports the notion that with the advent of Internet technology, understanding the online learner becomes increasingly important program success. While it has not seen quite the growth or success that was postulated by many (for example, Mottle, 2000) during the dot.com bubble, the growth of Internet-enabled education has continued and matured (BBC News, 2005). In addition to a small number of distance-education focused universities, such as the UK’s Open University and the US-based University of Phoenix, many conventional universities offer distance-delivery courses as an extension to predominantly in-class delivery methods, usually to part-time mature students, however there has been limited experience of its use in full-time undergraduate programs. Some authors have argued for distributed learning models combining classroom and Internet learning, sometimes described as “hybrid” (Cukier, 2000). Others propose the extensive use of information technology to support a new teaching/learning environment called the “flexible learning mode” (Campbell, 2000, p.353). 6. RESEARCH FINDINGS The Two Groups The profile of the two classes is shown in Exhibit 1. On average, the Distance Class was somewhat older, had more experience with distance learning prior to taking the course (50% vs. 30%), was less likely to engage in online chat, but equally likely to use email. About half of each group were students in business IT programs, the remainder in more general business. One unexpected element was that the day class included an international exchange student, who expressed some dissatisfaction with the delivery method. In the Distance Class, 64% were female, compared to some 33% in the Day Class. It was interesting that, with relatively few exceptions, the responses for the two classes were very similar. It might have been expected that the Day Class students would have responded differently, given that they had not chosen this method of delivery and also that there might be demographic differences. The study findings are discussed below, highlighting where differences between the two groups were identified. Distance Class % of Responses Day Class % of Responses Total Students in class 26 51 Total survey responses 22 100% 36 100% Response Rate 85% 71% Full time 5 19% 51 100% Part time 21 81% 0 0% Under 20 0 0% 2 6% 20-30 13 59% 33 92% 31-40 5 23% 1 3% Over 40 4 18% 0 0% Male 8 36% 24 67% Female 14 64% 12 33% Exhibit 1: Class Profiles 7. Student Activity and Performance Most students worked with the course material regularly, accessing the system several times each week. A significant proportion seemed to do so in a clustered manner, concentrating their efforts on one or two days in the week (often on the weekend – when assignments were due). When asked whether they would have accessed the course on a weekly basis if weekly work was not required, virtually all of the Distance Class said they would have done, while almost half of the Day Class indicated otherwise. When asked to compare the work effort for this course to others taken at the University, the overall view was that it took more time, with a majority of respondents (54%) replying more or much more time and only 14% saying it took less time. On the other hand, while professors in the University would suggest anecdotally that students are expected to spend 7-10 hours per week on each class, including lectures, the students’ responses clustered in the 3-7 hour range when asked how long they actually spent. One required element of the course was active participation on the discussion boards (worth 10% of the overall grade). The posting expectations were not onerous (described to the students as “regular postings are expected, but not necessarily every week”). Overall, the Distance classes found this expectation reasonable and useful, while the Day Class assessed it somewhat lower (2.8 vs. 3.4/5). Very few students (only 7%) felt the discussion activity not to be useful, and, when asked why they did not participate as much, students suggested that the biggest issue was available time (especially in the Day Class). In terms of actual participation, the majority of student reported posting at least 8 times during the course. Actual analysis of their posts supports this, with an average of 14 posts by the Day Class and 13 by the Distance Class (with 7-8 posts in the forum for which participation marks were awarded). A few students were very active posters (30—40 during the course), but overall the postings followed a fairly normal distribution with less than 10% of the students in either class failing to participate in any meaningful way. 8. Assessment and Grading Overall, students expressed reasonable satisfaction with the level of assessment and marks awarded. In both classes a significant majority felt that the assignments and project were useful and only a small proportion (16%) felt that the level of effort required should be reduced. Interestingly, students expressed the most dissatisfaction with the marks awarded to the weekly assignments, although these activities received the highest average marks of any element of their assessment. Despite the earlier concerns expressed by the students on the time needed to participate in discussions, a very high proportion found the feature useful (an overall score of 4.1/5), with almost 70% rating it very or extremely useful. One element of the course, which was a new experience to many students, was the use of online exams and electronic submission of assignments. The use of Turnitin.com as the method of submitting all essay work (including exam questions) was very highly rated. When compared to traditional methods of submission (handing in during class, delivering to professor, etc.) 67% viewed this as a more effective method and almost all the remainder found it equally effective. In general, students were very confident that the work had been accepted and was available to be marked, with more than 80% expressing strong confidence in the system (only 2 Day students expressed a lack of confidence here). The use of the Blackboard™ online quiz feature was also quite highly rated, though there were a few more concerns. All the Distance students found this to an effective method and equal or superior to in-class paper tests, while the Day students were somewhat less satisfied, with 33% finding it a little less effective. This lower assessment was consistent with a slightly lower level of trust in the submission, when compared to the essay submission to Turnitin.com. There were no significant differences in grade performance between the two classes. 9. Interaction with Faculty Both groups rated this element of the course quite highly. When asked to compare the level of communications to that experienced in other more traditional courses, about half rated it as more or much more effective and only some 10% found it a little less effective (all but one of these students were in the Day Class). Highest ratings were given to the use of the Discussion Boards for communication with the professor (for both class announcements and questions), with 66% describing it as extremely useful and only 5% finding it not very useful. The ability to send private emails to the professor within the system was also used by most students and rated as an effective communications tool (except for a design limitation within the system which did not flag the availability of new messages). The responsiveness of the course team (professor and marker) was also rated highly with an average of 3.9/5 and no student rating the team below 3/5. This was also supported for the Day Class, by their assessment of the course communication done in the University’s standard course evaluation, where such areas as responding clearly to students’ questions, course organization and general effectiveness were all rated highly (in each case, these ratings were above the average for the instructor’s Faculty and for the University as a whole). In this survey, the only rating that was consistent with the wider averages was the rating of the faculty member’s availability during posted office hours (which did not really exist in this course since the course was taught and supported at a distance). One explanation for such disparity in the rating is the fact that students expect faculty to be available on-demand. In order to manage students’ expectation, Lieblein (2000) suggests that online faculty “define teacher presence to avoid student frustration” (p.164). 10. Overall Response to the Experience The course had 5 major objectives and, overall, these were viewed as having been met (4.07/5 average score) with just one objective (related to discussion of integrated supply chain issues) rated slightly lower by both groups – and subsequent examination of the course material supported this criticism. The Day Class were also very positive to the experience with 100% expressing satisfaction with the delivery of the course, although about 28% indicating that they would still have preferred a face to face class. Over 90% felt that the school should offer more classes in this format to full-time students. 11. Course Team’s Observations and Comments From an instructional viewpoint there are also some lessons to be learned. Creation of an online course requires a considerable effort and content maintenance has also been an issue (compounded in this case by the fast-changing subject matter). The distance format allows considerable flexibility in class delivery – both in time and in place (these two Canadian classes were taught while the instructor was largely living and researching in Europe (the UK, Austria and Bulgaria). However, it requires frequent access by the instructor to keep in touch and monitor activity. This was clearly expected by the students – anything more than a day or so of instructor silence began to raise issues (demonstrated by reminders of previous emails sent or dialogue on a discussion board on some concern or issue). The format allowed for close involvement of two teaching assistants (one for each course) who made necessary maintenance changes to the course content and marked the weekly assignments and projects. The electronic nature of both the submission and grade responses allowed the course team to interact around student work using the electronic records and tools. There were virtually no problems about “missing” assignments or in marks, with less than a dozen questions of any kind being raised regarding over 500 assignments. The teaching assistants spent about 6 hours per week each on the two courses and the instructor between 3 and 5 hours (excluding exam marking) per week on each course. The use of Turnitin.com was also of interest. From a positive perspective, it proved a useful tool to assess the originality and depth of students’ work, including validation of sources cited -- thus allowing an assessment of original vs. (legitimately) copied elements of answers. It provided a secure method of assignment submission (students received a digital receipt as proof of submission) that was easily read and marked, including feedback and comment on each assignment. It also highlighted student sloppiness in the use of quotation marks and citations (which were used as learning experiences for the students involved) as well as three substantiated cases of plagiarism -- which were handled as formal cases of academic misconduct with appropriate penalty. 12. CONCLUSIONS This evaluation has demonstrated rather conclusively that a well-structured online course can deliver at least equivalent levels of student satisfaction and performance as a more conventional face-to-face course. In almost every area of assessment, the vast majority of students indicated satisfaction with the course, both with content and with the method of delivery. Student performance was consistent with that of students taking a regular face-to-face class. What is of significant interest is the comparison of one group (the Distance Class) who had already committed to taking the course in distance format, with another (the Day Class) who had registered for a conventional class and were faced at the beginning of the term with a “fait accompli” that the course would be delivered in distance format. It has been argued (e.g. Beyth-Marom et al, 2003) that students’ performance in an Internet based course cannot be compared with its traditional counterpart if participation in the course is optional. In this study, students were faced with a situation that eliminated the flexibility of an option, thus legitimizing such comparison, in essence, using the Distance Class as a form of control group. In almost every area, the Day Class shared the positive view of the Distance Class (who, it might be expected would be generally more positive than the Day Class, given that they had chosen this method of delivery). There are, however, some worthwhile conclusions that night be drawn from the slight differences that were observed. It was notable that a higher proportion of Day students felt that they would not have worked on a regular weekly basis if it had not been forced on them by the course structure – which might be seen as a maturity issue. Also, it was clear that a small proportion of the Day Class would have preferred not to have done the course in a distance mode and, it might be assumed, had learning styles that would respond better in a face-to-face mode. Many of the distinctive features of an online course received very positive responses. Discussion boards and email were seen to be a very effective form of communication, rated by most as equal or better than conventional classes. Students were also very positive and confident about online assignment submission and exams, again with a few reservations from some of the Day students. While not the primary focus of this paper, the use of the plagiarism detection tool provided by Turinitin.com was also of interest. It was an excellent and efficient tool for both the submission and marking of student assignments. It was an essential element to ensure the academic integrity of the course (where the majority of the work is done at a distance and every element of assessment is, in essence “open book” and several years of similar assignments exist in electronic form widely dispersed in the student population). Finally, the experience also demonstrates that delivery of a course in this form requires a significant effort from faculty. Excluding the time needed to develop and maintain the course material within the course management system, instructors should expect to spend a similar level of time to that needed for a more traditional lecture-based course. Further, the need to do this on a regular (almost daily) basis could be seen as a challenge for instructors with other priorities, such as research, while giving more flexibility to choose when and how they interact with their students. 13. REFERENCES BBC News (2005), “Universities 'expand e-learning'” (online) 18th March, 2005. Found at: http://news.bbc.co.uk/2/hi/uk_news/education/4361711.stm Beyth-Marom, R., Chajut, E., Roccas, S., and Sagiv, L. (2003). “Internet-assisted versus traditional distance learning environments: factors affecting students’ preferences. Computers & Education, 41, 65-76. Campbell, J. A. (200 ). Using Internet technology to support flexible learning in business education. Information Technology and Management, 1, 351-362. Cukier, Wendy and Ward Eagen, Distributed Learning Strategy for Ontario's Colleges of Applied Arts and Technology 1, 2, ACAATO, 2000. Davies, R. S. (2003). Learner Intent and Online Courses. The Journal of Interactive Online Learning, 2(1). Grant, K.A., Cukier, C. and Prescod, F. (2001), “The Use of Internet-based Tools to Support the Delivery of an eBusiness Course”, ISECON, October 2001. Lieblein, E. (2000). Critical factors for successful delivery of online programs. The Internet and Higher Education, 3, 161-174. Lu, J., Yu, C., & Liu, C. (2003). Learning style, learning patterns, and learning performance in a WebCT-based MIS course. Information and Management, 40, 497-507. Mitchell, C., Dipetta, T., & Kerr, T. (2001). The Frontier of Web-based Instruction. Education and Information Technologies, 6 (2), 105-121. Mottle, J. N. (2000). “Learn At a Distance, Online Learning is Poised to Become the New Standard”, Business Week on e-Learning.