Pilot Studies Introducing Collaborative and Distance Learning Paradigms in a Residential Environment Michael Battig – mbattig@smcvt.edu Greta Pangborn – gpangborn@smcvt.edu Computer Science and Information Systems, Saint Michael’s College Colchester, VT 05439, USA Abstract With the advent of the Internet, several alternative educational paradigms have emerged. Although much has been written to support distance learning and collaborative learning, contextualizing research for the residential, liberal arts college requires first-hand experience. With this premise, several studies were funded by Saint Michael’s College to explore the appropriateness of e-learning paradigms in the context of one particular residential environment. Empirical data is presented along with anecdotal and subjective observations. Three learning paradigms are investigated in this paper: pure-distance, partial-distance, and collaborative learning. Keywords: distance learning, collaborative learning, asynchronous learning, information systems curricula 1. INTRODUCTION Each fall semester the Educational Technology Committee of Saint Michael's College issues a call for proposals. During the fall of 2002, the Committee limited the call to experiments that would “assess the effectiveness of different models of technology assisted course delivery in the context of Saint Michael’s educational values and philosophy.” The Committee offered to fund up to three experiments and offered modest stipends and/or course reductions for the successful proposals. Specifically, the call for proposals suggested three models. Two of the three models were targeted at summer school applications while the third could be summer but in all likelihood could also be taught during the traditional academic year. The three models were generally: 1 - A pure distance course designed to capture lost revenue from courses taken at other institutions during the summer. The idea here is to choose a course that in the collective wisdom of the committees and academic affairs staff would have a reasonable enrollment, is traditionally taught on campus, and has a faculty member interested in committing to teach the course for two summers. If we teach the course twice ( or more ) and we have separate experience with the course in a traditional delivery model we ought to be able to say some things about what works and what does not work and under what circumstances, if any, courses based on this model ought to be part of our offerings. 2 - A “limited meeting” course designed to attract students who could come to campus occasionally (once or twice a week in a summer semester ) but who interact most of time with faculty and other students on line. The idea here is to capture those students who are constrained by distance, work or other obligations but still have some opportunity to come to campus for group work or other face to face interaction. The target could be a course similar to the type in model 1, although the committee wanted to maintain as much flexibility as possible in choosing which proposals to support. 3 - An “other people’s content” course designed to give us experience with delivering or supervising the delivery of other people’s content. For example, materials for a wide variety of courses are available online from MIT’s Open Courseware Project (MIT 2003), and there are a number of sites offering high quality journalism resources (Grabowicz 2004, Lynch 2004). These materials may permit us to offer courses that we currently are unable to staff consistently or at all. Ultimately, our students will want to use material developed elsewhere or take courses elsewhere while enrolled here for a variety of reasons; this model explores ways we might be able to offer our students greater variety while exercising control over the quality of the experience. The Educational Technology Committee reviewed proposals and ultimately funded four experiments: three experiments involving distance learning (models 1 and 2 above) and one involving collaborative learning. No proposals were received of type 3. The experiments involved courses from a variety of disciplines: two in Information Systems, one in Accounting, and one writing course for English/Journalism. The remainder of this paper will present the results and conclusions of the funded proposals. 2. PARTIAL-DISTANCE EXPERIMENTS Two of the funded proposals offered courses that involved limited face to face meetings with the majority of the learning occurring through our course management system (eCollege). The first course was “e-Commerce/Web Programming” and the second was “Introduction to Writing.” Due to the large volume of data from four different studies, we will focus our discussion on what we feel are the most significant observations and findings from the studies. The interested reader will find far more detailed discussion in the technical reports generated from these studies (Saint Michael’s College Educational Technology Committee 2004). The e-Commerce course had an enrollment of six students who were all upperclassmen in either Computer Science or Information Systems. The instructor felt the professor / student relationship was similar to that in a traditional course (likely because the class was small and met each week). The students seemed to agree; four of six felt they had a similar amount of interaction with the professor to what they do in a regular course, one found they had significantly more, and one significantly less. All six claimed that the quality of interaction with the professor was similar to what they had during the semester. The negative outcome was with respect to peer interaction, where half found they had significantly less interaction with peers and half claimed to have similar interactions. Five of six students claimed that their number of serious conversations with peers on topics outside of the scope of the class was similar to a standard class; this would seem to indicate that such conversations occur less than we hope during a standard term. One clear benefit of this model was the fact that is was convenient for the majority of the students. In response to the surveys statement “The limited meeting time model made access to the class more convenient than would ordinarily be the case for me,” three students indicated strong agreement, one indicated agreement, one was neutral, and one disagreed. One of the students who selected the strong agreement response indicated some additional enthusiasm by putting three checkmarks in that particular box. Their responses followed a similar pattern on the question “I would consider taking another course with limited meeting times” (three strongly agreed, one agreed, one was neutral, one strongly disagreed). The writing course had an enrollment of 13 students from a variety of majors and classes (sophomore through senior). This study produced at least three major lessons learned. First, students at a traditional, residential college may not be adequately prepared for the rigors of online learning. The same could probably be said for the professors as well! Students, for their part, were astonished at the demands of the course; they truly expected it to be “Internet fun” and an easy three credits. As a result, there was a huge adjustment of expectations that occurred during the first week of the course (one student didn’t log on for the first week, discovered he was hopelessly behind, and dropped). Second, our experiences in this course (as well as the e-Commerce course) lead us to agree with conclusions that real-life interaction is necessary (DeLacey 2002). Students should be required to come to campus at least twice during a four-week session; the online dynamic changed dramatically after the class had had the opportunity to meet one another in real life. We are physical beings; we need physical contexts. Some students really need the kind of ‘face time’ and direct discussion unique to classroom learning. Our seven-hour sessions combined group discussion, peer coaching, and formal assessments; by the time they were over, the class had made connections and had common experience, and the students were no longer isolated learners. It made an extraordinary difference in their commitment to and engagement with the course. Third, the performance of these students was compared to a control group of 21 students who covered the same material during a regular 15-week semester. At the end of the course, there was a substantial difference between the groups. While the control group had improved a scant 8 percent over the 15-week semester, the performance of the online summer class jumped nearly 25 points, to just under 70 percent. Whether those results would or could be replicated and validated in a more rigorous setting remains to be seen, but initial outcomes suggest what common sense would confirm: students who are required to work through more than 400 online grammar questions inevitably improve their basic understandings of the language. Thus, we conclude that the grammar modules are an effective tool for teaching basic writing skills; they represent precisely the kind of drill-and-practice exercises for which the Web is uniquely well suited. 3. PURE DISTANCE EXPERIMENT Only one of the four funded proposals involved a pure distance (i.e., no face to face component involved) course. Financial Accounting was offered during a four week summer term in a similar manner as the courses described above but with the added advantage that students could take the course from remote locations. The results of the survey taken at the end of this course along with the anecdotal evidence from student interactions with the instructor via email indicate that the lessons learned from this experimental course offering are very consistent with what we learned as a residential college in offering partial distance courses. To generalize, students like the idea of the convenience of the distance course offerings but have unrealistic expectations about the work involved. Furthermore, students who completed the course and put forth the required effort report a positive learning experience. Although seven students completed this course and registered results in the post-course survey, more students dropped this course than any other course in our study. We believe the reason for this is that student expectations were not in line with reality. For instance, one student who dropped indicated that she “didn’t feel like there was a teacher for this course.” This kind of feedback can be discouraging to a professor who is putting forth a Herculean effort to build the substantial electronic materials and engage in the regular dialog required for distance learning. In addition, some students are so convinced of the ease of online learning that they were discovered later to have ignored technology-related prerequisites that were clearly articulated during the enrollment period (e.g., they did not possess the required computer software). Consider the results of our survey that asked students if they worked and/or attempted other courses while taking this one. Six of seven students were working during the intensive four-week course and two out of seven took another college-level course at the same time (we don’t know if these classes were at our institution or not). These results are particularly perplexing given that students were advised not to work or attempt other college courses during the intensive summer courses presented in this study. Perhaps one observation we can draw from these studies is that students will not reliably “follow the rules” in the context of summer school. Although we have no specific evidence to back our intuition, perhaps some of these students are taking summer classes because they simply need X number of credits. Thus, the fact that our experiments were conducted during the summer term in the context of a traditional, residential college should be taken into consideration as a potential limitation. To summarize, our most significant lesson regarding pure distance courses has clearly been that we will need to somehow better manage student expectations if we are to offer courses in the future that don’t have any face to face meetings. 4. COLLABORATIVE LEARNING The Davis Foundation funded the installation of a collaborative learning lab on the Saint Michael's College campus during the 2001/2002 academic year. This specialized teaching lab includes an instructor’s podium with a computer and touch pad system that controls 2 ceiling-mounted LCD projectors, VCR, DVD, campus cable, FM radio and sound system.  A laptop may also be connected at the podium.  In addition to the instructor’s podium, the lab contains 5 pods of 4 networked computers configured in order to allow for collaboration and small group work.  There are two additional computers available to capture various forms of multimedia, including digital images and video. The lab has been utilized to teach courses in Biology, Chemistry, Journalism, Computer Science, and Information Systems. Although all of the faculty members utilizing the lab have reported that it is a successful addition to our campus, no empirical data had been collected prior to this study. The course in this study is our Introduction to Computing (the required first course for Information Systems majors and a course taken as an elective by many students). Much of the course content, especially the labs, has been online for many years (http://academics.smcvt.edu/compsci_courses/cs101/). Introduction to Computing is a 4-credit course that in a traditional semester requires 3 hours per week of lecture and a 2-hour closed lab session. During summer semester, the course is offered during an intensive 3-week session, which has resulted in higher summer term enrollments. Our goal was to study the effectiveness of the collaborative learning lab on educational outcomes in the lab portion of this course. During the summer of 2002 a Control Group of 9 students completed the course. The Control Group completed all labs on an individual basis. During the summer of 2003 an Experimental Group of 6 students completed all labs on a group basis using the Collaborative Learning Lab (Davis). The detailed results of the surveys are available on the Web (see References). Although the complete report from this study contains many details, we will attempt to highlight some important observations here. First, we noted that the number of subjects in each group was relatively small. Second, we determined (on an anonymous basis via our Registrar) that the summer students in both groups were generally lower performing students than our undergraduate population (e.g., the experimental group had an average GPA of 2.48, whereas our College average is near 3.0). Again, this may say more about the summer school dynamics than anything else. More important to this study is the impact of collaboration on the lab learning. Our post-course surveys asked students to tabulate their influences on learning among three categories (totaling 100%): instructor, classmates, textbooks & web materials. We were particularly interested to see if the “classmates” percentage would be significantly higher among the participants in the experimental group. Our hypothesis in this case was that the collaborative aspect of the course would result in at least a greater perceived learning from classmates. The results are not significantly different. The experimental group felt on average that 22.2% of their learning could be attributed to classmates, whereas the control group held that 24.4% of their learning should be attributed to classmates. Given the great acclaim that this collaborative learning lab has received on our campus by faculty members from various disciplines, we wondered about these results. In our discussions with Biology faculty, we have found some differences in our courses that perhaps are significant. First of all, the Biology courses that employ this lab are comprised of upper class Biology majors, whereas the Introduction to Computing sections studied here were comprised of all non-majors. Second, the collaborative nature of the Biology assignments is indeed the state of the practice in biological lab work in general. Whereas, in computing, the state of the practice is almost exclusively individualistic. For example, programmers always work solo (there is the “Extreme Programming” paradigm where developers work in pairs, but it is still very new and has not been validated as an approach). Thus, the work the students were being asked to do in our computing lab is something that is traditionally done solo whereas the Biology work is something that is traditionally done in a team context with a “divide and conquer” approach. Anecdotally, we also observed that students had unrealistic expectations of the amount of work required for this intensive summer course. 5. CONCLUSION The Educational Technology Committee of Saint Michael's College determined that although a plethora of research exists in the area of distance learning (Sloan 2004), the College needed to investigate the topic in the context of our own mission. In retrospect, this has turned out to be a very wise endeavor for several reasons. Primarily, we have learned that a substantial amount of preparation is required for both faculty and students in order to successfully utilize e-learning paradigms in our residential context. Student expectations clearly will need to be managed. The Committee has recognized through these studies that the work required for faculty to develop the materials necessary for distance or partial distance courses is substantial. Therefore, we will need to consider some significant administrative assistance for faculty members engaging in these delivery methods. A primary motivator for the College was to recoup summer tuition revenue that current students are probably spending at other institutions. Although we would rather have our students taking courses with us during the summer, we will need to determine some guidelines that will direct the students in managing their time during the summer months (clearly many students in these studies greatly overestimate what they can accomplish during the summer). Finally, we have learned that there is no “one size fits all” approach to collaborative and distance learning. What works well for a Biology lab, may not work satisfactorily for an introductory Information Systems course. In retrospect, the $10,000 the College spent for these studies was a bargain given the lessons learned by the faculty members. This investment was further extended through the seminars that we hosted during the Spring 2004 semester to share the lessons with the entire faculty. 6. ACKNOWLEDGEMENTS The authors wish to thank Dianne Lynch for her helpful suggestions and encouragement. 7. REFERENCES DeLacey, Brian E. and Leonard, Dorothy A., “Case study on technology and distance education at the Harvard Business School,” 2002, Educational Technology and Society, vol. 5, no. 2, pg 13-28. Grabowicz, Paul (editor), Five Steps to Multimedia Reporting, 2004, http://journalism.berkeley.edu /multimedia Lynch, Dianne (project manager), Mutilmedia Journalism Training Series, 2004, http://www.journalists. org/training/project_index.html MIT OpenCourseWare Home Page, 2003, http://ocw.mit.edu/index.html. Saint Michael’s College Educational Technology Committee, Proposals and Reports, 2004, http://academics.smcvt.edu/edtech/Proposals.htm. The Sloan Consortium, Computing Curricula Effective Practices for Online Education, 2004, http://www. computer.org/education/cc1991.