An Action Learning Model to Increase Critical Thinking Skills in an ALN Masters Information Systems Capstone Course Alan T. Burns1 School of Computer Science, Telecommunications and Information Systems (CTI), DePaul University Chicago, Illinois 60604, USA and Thomas N. Janicki2 Information Systems / Operations Management Department University of North Carolina at Wilmington Wilmington, North Carolina 28403, USA Abstract Critical thinking and problem solving skills are essential for Information Systems (IS) graduate students, but these skills can be difficult to teach, particular in asynchronous learning network (ALN) environments. Emphasis to teach or reinforce these skills is usually concentrated in an IS capstone course. The capstone is commonly taught with the case method using interactive, argumentative case discussions in class to develop critical thinking skills. However, at one university a new challenge is introduced: how to offer the capstone course in an asynchronous delivery mode, where interactivity is difficult to achieve. This paper details efforts to deliver the capstone via an alternative approach drawing on principles from action learning theory. The course emphasizes analysis and research to integrate theoretical IS concepts with practice. Preliminary data on student perception of learning is presented comparing the method delivered in synchronous and asynchronous modes. Keywords: action learning, asynchronous learning networks, distance learning, information systems, IS capstone course 1. INTRODUCTION The demands of Information Systems (IS) educators are ever increasing, as the field is one of constant change. New technologies evolve, as do new theories of how to best utilize IS assets for competitive advantage. In addition to these technical skills, educators must also teach a number of non-technical skills specified in the MSIS 2000 curriculum model, such as communication, interpersonal, teaming, analytical and critical thinking skills. Many universities are turning to an asynchronous learning network (ALN) delivery mode for delivering course content in order to increase or maintain enrollments. An ALN is an Internet-based learning system that provides instructional content and material in a distance-learning (DL) environment (Hall, 2002). ALNs present a challenge to IS educators since they require an alternate set of teaching skills for effective delivery. Within the ALN realm, the so-called “soft skills” of critical thinking and decision making in IS can be particularly difficult to teach. Frequently the development of those skills requires rich communication between student and instructor that is difficult to achieve in an asynchronous environment. Unless there is a substantial amount of effort on the part of both parties, DL students may feel isolated and disconnected without instructor interaction (Knight et al., 2002). This paper will have three major focuses. The first goal is to review the literature in the area of action learning. Second, based on prior research we discuss the development and implementation of a model for teaching critical thinking skills in ALN environments. Based on principles of action learning, the model uses a research paper assignment with peer feedback to increase learner participation. The last goal of the paper is to compare student perception of learning between a traditional section and an ALN section of the same course. The overall goal of our research is to increase student critical thinking skills in ALN environments by raising the level of dialog and feedback to a level comparable to traditional classroom environments. This objective is consistent with the call by Robin & McNeil (1997) for new and innovative modules of production, presentation, and delivery that leverage the Internet’s capability for emphasizing learner participation. 2. LITERATURE REVIEW Critical and analytical thinking skills are emphasized in several graduate IS courses, but typically they are concentrated in an integrating capstone component built around business policy and strategy. The capstone occurs at the end of the program since it demands the integration of all prior coursework. The objectives of this course are numerous, as students should learn emerging IS concepts, techniques and technologies, synthesize theoretical IS concepts with practical considerations and develop analytical and critical thinking skills (Gupta and Wachter, 1998; MSIS 2000 Model Curriculum). A sample capstone course description appears in Appendix A. The IS capstone course is frequently taught as a mix of lecture and case study discussions. Case studies serve as a proxy to practical experience. They are also a way for students to exercise their analytical skills by applying creativity to solve real-world, practical problems. The case method relies on openly interactive and sometimes argumentative student input to orchestrate discussion of management situations in which there are frequently no definitive correct answers (Thompson, 2000). The case method’s required interactivity creates a challenge for asynchronous delivery of course content. Threaded discussion groups are one approach to solving the problem. For example, students can post answers to case questions on an electronic discussion forums moderated by instructors. The problem with this approach is inefficiency, both of students who must carefully articulate their position to avoid ambiguity and the instructor who must painstakingly moderate the board to achieve learning outcomes. Also, instructors cannot interpret student problems easily since verbal and non-verbal cues are absent (Knight et al., 2002). Moore (1993) states that transactional distance interjects new problems of understanding and perceptions between learners and instructors. He recommends that learning modules incorporate dialogue (communication between learner and instructor), structure (how the module is designed) and learner autonomy (self-learning or self-direction). Saba (1988) reports the effect of transactional distance decreases when dialogue increases and structure decreases. Improving ALN technology is one approach to solving the problem, but certain limitations remain despite technological advances in web-based education. Courses that rely on small group work or classroom interaction still require significant redesign. Using technology to make the instructor more efficient in delivering the case method in an ALN environment is another approach. For example, videoconferencing or voice recognition software could improve the speed at which the instructor’s knowledge is made explicit. The problem with this approach is that instructors already carry a large burden developing and maintaining courses in a DL environment (Knight et al., 2002). Adding work to the instructor, even with adequate training, will likely compound this problem. Perhaps what is needed is a pedagogical, not technical, approach to solving the problem. One potential solution to the problem of course delivery and presentation is offered by Jonassen et al. (1995) and Hemphill & Hoyet (2000). Jonassen et al. propose the need to move from a teacher-centered to a learner-centered approach. The role of the teacher is no longer the sole expert and distributor of knowledge, and students are no longer the passive recipients of knowledge. They view that the web and other computer technologies are in an excellent position to facilitate this move from teacher-centered to learner-centered instructional styles. They report that in a traditional classroom up to 80% of the verbal exchange is provided by the teacher. However this drops significantly in computer mediated situations. They summarize that the new computer and web-based technologies force a change on the instructor and "Mediated instruction moves the teacher from the podium to the sideline, from leader to coach, from purveyor of knowledge to facilitator of personal meaning making." (Jonassen et. al. 1995, p.8). Table 1. Traditional vs. Resource-based Learning (Rakes, 1996). Traditional learning Resource based Teacher as export model Teacher as facilitator/guide Textbook as primary source Variety of sources/media Facts as primary Questions as primary Information is packaged Information is discovered Emphasis on product Emphasis on process Assessment is quantitative Assessment is qualitative / quantitative The resource view of learning also depicts a changing role for the instructor. This theory states that the teacher is a resource and a guide to the student. Rakes (1996) and Suchman (1962) view a change from traditional learning to one based on a multitude of resources being available to a student. Other concepts include a variety of sources being available to the learner, an emphasis on information being discovered versus pre-packaged in a linear manner and stressing the learning process not the learning product. Rakes (1996) provides a summary of the differences between traditional learning and resource-based learning as shown in Table 1. Rakes proposes that computer-based or web-based tools can assist in the transformation from traditional to resource based learning theories. Action learning theory similarly moves the burden of learning away from the instructor toward the student. Action learning encompasses several variants, but essentially it is a form of learning where emphasis is placed on action through experience or "by doing". Pioneered by Reg Revans, it is a participant-centered process that seeks to solve real, systemic and pending organizational problems (Vat, 2002). Barrows (1980) concurs and states that learning is not just the acquisition of facts. Rather it should emphasize an active role for the student in determining what he or she needs to know. Smith and O’Neil (2003) propose action learning programs are typically based on the following tenets: • Participants tackle real problems without a "right" answer in real time; • Participants meet in small, stable learning groups that hold intermittent meetings over a fixed program cycle; • Problems are relevant to a participant's own workplace realities; • A supportive collaborative learning process is the group; • Process is based on reflection, questioning, conjecture and refutation; and • Participants take action between meetings to resolve their problem. Models proposed by Dilworth (1998) and Vat (2002) represent movement toward the learner-centered and resource-based models discussed previously. Dilworth (1998) adapted prior research to create a static model of learning. He defines learning in terms of the formula L = P + Q + R, where Learning (L) equals Programmed Instruction (P) plus Questioning (Q) and Reflection (R). Programmed Instruction includes lectures, cases, instructions, etc. Questioning is the process of seeking new insight into what is not yet known. Reflection means rethinking, taking apart, putting together and making sense of the problem. Action learning emphasizes Q compared to the traditional method of classroom education placing emphasis on P. Compared to Dilworth, Vat (2002) defines learning in terms of a more dynamic, process oriented model. He adapts action learning principles to the activities performed by IS students as they explore a problem. He identified four iterative stages of activities: analysis, research, reporting and feedback. During analysis, students organize their ideas and knowledge prior to the study of the problem. Students are encouraged to devise a specific statement of the problem to address what they do and do not know about it. Research requires collection of necessary information relevant to the problem, where students teach themselves as they research the issue they face. Information may include journal articles, textbooks, periodicals, interviews with knowledgeable authorities, etc. At the reporting stage, students become “experts” who report their findings to one another, an activity which should help focus their investigation, clarify their knowledge and refine their problem-solving strategies. Feedback is introduced from the instructor and fellow students at the end of each stage. Similar to the action learning concept is the theory of learning proposed by Kearsley and Shneiderman (1988). They described engaged learning as "all student activities involving cognitive processes such as creating, problem-solving, reasoning, decision making and evaluation." The activities of engagement are relate (team efforts involving communications), create (define a specific project) and donate (stress the importance of a useful need for the learning). In essence the engagement theory proposes hands-on problem solving with real world implementations. Building on these foundations, we propose an alternative approach to the case study method is needed for delivering the capstone course in an ALN mode. The case method is pervasive and has many benefits, but other teaching strategies are available. For example, action learning theory has been shown to be successful when employed in other areas such as accountancy (Spiceland and Hawkins, 2002). The next section discusses a new method of delivery for the course that teaches critical thinking skills using concepts derived from action learning theory. 3. A MODEL FOR IMPLEMENTING ACTION LEARNING IN THE IS CAPSTONE A basic objective of the IS capstone course is to expose students to fundamental theory and provide opportunities for them to apply it to real world problems. The case method uses case descriptions and lectures about theory as the primary vehicle for learning. Methods of delivery vary, but one approach of sequencing material can be depicted using Dilworth’s static variables as follows: Plecture --> Reflection --> Pcase --> Qcase --> Reflection When lecture is preceded by a case, the burden of making connections between theory and practice is mostly on the student as they reflect on the questions about the case. The class discussion is critical to clarifying student understanding of theoretical concepts and their application. Another common approach is to present case material before lecture, which would follow a scheme such as this: Pcase--> Qcase --> Reflection --> Plecture With this approach, the instructor assumes more of the burden for connecting theory to practice since they can refer to the case during the lecture. Using the case method, the capstone course is usually taught as repetitions of one or both of the above sequences. When done properly, the case method manages to connect theory to practice, but the method seems counter to at least one objective of the capstone. The course is supposed to build problem solving and critical thinking skills, yet much of the analysis stage described by Vat (2002) has been done for the student. Lecture materials, case and case questions are prepared or preselected by the instructor and fed to the student. Research required by the student is centered mostly on given materials—the need for external research is usually minimal. If the objectives of the capstone are to build these skills, the case method may not be the most appropriate approach. The action learning delivery model we are proposing has two objectives. First, the method places greater emphasis on the development of critical thinking skills when compared to the case method. Second, it seeks to overcome problems of delivering the case method in an asynchronous environment. In the new model the burden of early stages of analysis and research is shifted to the student, who must construct the problem they seek to solve. The sequencing for the action-learning model is shown below: Qproject --> Plecture --> Reflection The Question part of the process appears first in this scheme and is introduced with a semester-long project (paper and presentation) requiring students to adapt theoretical frameworks in IS to a problem in the information technology field. In an early assignment, students organize their knowledge and ideas prior to the actual study of the problem. Throughout the semester additional IS theories are introduced in the form of lectures and a few case discussions. Students receive feedback from the instructor and fellow students in a variety of ways. As feedback is introduced, students are expected to reflect and refine their prior conceptions about the technology and IS theory. The course is delivered as successive repetitions of the above sequence, where new IS concepts are introduced in each sequence. Our learning model more closely follows stages suggested by Vat (2002). While the assignments are not necessarily new, they do represent a less-commonly approach to delivering the capstone which historically has used the case method. Our model incorporates Vat’s model but adds two unique feedback components. Feedback is received not only from the instructor but also from fellow students. It can be seen as: Qproject --> FFeedback - Instructor --> Plecture --> Ffeedback- Students --> Reflection 4. IMPLEMENTATION AND REVISION OF THE ACTIVE LEARNING MODEL The model was introduced in two sections of the capstone course taught during summer of 2003. Both sections had over 30 students enrolled. One section is a synchronous (“live”) classroom section, whereas the other section is asynchronous (distance learning). Distance learning students viewed a recording of the lecture given to the classroom section using technology described by Knight et al. (2002). The recording was available to distance learning students within 24 hours of when class actually occurred. Students in both sections worked on identical assignments; incrementally on the semester-long project by completing a number of exercises intended to develop critical thinking skills. A list of assignments for the course is provided in Appendix B. Students first submitted a one-page topic statement to frame their understanding of an area of interest to them. This assignment represented the analysis stage described by Vat (2002). Direct instructor feedback was provided so the project’s scope would reflect an appropriate amount of work for the individual learner. The next stage of the process, research, required the student to cycle through analysis, research and reflection to refine their preconceived view of the topic at hand. This stage was followed by reporting, where students reported their findings to one another to focus their investigation, clarify their knowledge and refine their problem-solving strategies. Two assignments were created for these stages. For the research stage, students were assigned the task of creating a detailed outline of their topic (four to five sections with at least three sublevels per section). For the reporting stage, students critiqued outlines developed by 3 other students. To facilitate effective reviews, students (as authors) included with their outline a list of three to five important sources (references) for their research. Students (as reviewers) used these sources to draft a critical analysis of the outline. Placing the burden of feedback at this stage on the student achieved a few objectives. First, each peer review represented an opportunity for students to refine their critical thinking skills. Second, students were able to investigate 3 other areas of information technology, albeit on a smaller scale compared to their main paper topic. Third, it alleviated the instructor as a bottleneck in the feedback process. While this approach may not be appropriate for earlier common body of knowledge (CBK) courses, we believe this expectation was reasonable and appropriate for the capstone. The list of assignments for the semester is provided in Appendix B. The assignments culminated in a paper and presentation delivered at the end of the semester. Assignments were identical for both sections with a few exceptions. “Live” classroom students gave presentations on their paper topics at the end of the semester, whereas DL students did not. The class participation component was determined by in-class participation for “live” classroom students. For DL students, it was determined by their activity on electronic discussion forums set up for the class. Technology for both sections included email access to fellow students and instructor, discussion forums and chat rooms. To facilitate the peer review process, an information system was constructed to post all student assignments and comments to the Internet. 5. SURVEY RESULTS A survey was conducted for the synchronous and asynchronous sections of the IS capstone course. The survey included items assessing student perception of learning effectiveness using a Likert scale as originally presented by Hiltz et al. (2000). Our exploratory research indicates that efforts to increase critical thinking skills for the ALN section through an approach of active learning has resulted in similar increases between two sections of the same course (one section ALN and on section traditional). T-tests were conducted to see if there were any significant differences between the two sections on the questions in Table 2. Our tests considered: H0: ?Traditional = ?ALN H1: ?Traditional < > ?ALN At an ?= .05 and 51 d.f. reject H0 if t > 2.00 For the 10 survey questions below, t never exceeded 2.0, thus we can accept H0, that the student perceptions of learning is the same for both the ALN and traditional courses. Table 2. Differences in Student Responses between Synchronous and Asynchronous Sections. Scale: 5=Significant, 4= Above average, 3= Average, 2 = Below average, 1= Well below average. Trad. Class Mean ALN Mean Subjects 28 25 Q(Project) increased critical thinking skills 4.29 4.42 Q(Project) increased critical thinking of written materials 4.32 4.32 P(Lecture) increased critical thinking skills 4.29 4.23 P(Lecture) increased critical thinking of written materials 3.78 4.12 P(Case study discussion) increased my critical thinking skills 4.23 4.04 P(Case study discussion) increased my critical thinking of written materials 4.04 4.12 F(Feedback to others) increased my critical thinking skills 4.18 3.92 F(Feedback to others) increased my critical thinking of written materials 4.04 4.00 F(Feedback from others) increased my critical thinking skills 3.68 3.44 F(Feedback from others) increased my critical thinking of written materials 3.44 3.48 It was our goal to provide a means for ALN students to gain critical thinking skills that were normally learned through dialog of case studies found in traditional classes. The early results indicate the proposed model (projects, lectures, peer feedback, instructor feedback, and reflection) may be a suitable substitute for in-class case discussion. 6. CONCLUSIONS AND FUTURE RESEARCH We believe the new model provides us reason for additional study (our sample size was small, as well as a need for a control group). It may be a viable alternative to the case method of delivery for the capstone course. In addition, it may represent an opportunity to improve the DL experience. Sonner (1999) reports that DL students may have certain characteristics that are more closely aligned with this approach when compared to the case method. Distance learning frequently requires students to work independently and apply creative solutions to overcome problems with DL technology and format. If this is the case, DL students may prefer the proposed method since it provides considerably more freedom in the problem statement and resolution. As indicated by the survey results, no significant difference between the ALN and traditional student sections were observed on the questions related to increases in critical thinking skills. These early results may indicate that the introduction of an active learning model helped to increase the skill set for ALN students who are not part of the class case discussions due to the nature of distant learning. Future research will include contrasting sections of ALN and non-ALN students that do not use the active learning model proposed to determine if there is a difference in perceived learning of critical thinking skills where the enhanced feedback, reflection and peer feedback is not evident. This control group will provide us a better understanding of the effects of the model presented. In additional we are researching a method to ‘measure’ the actually critical thinking skills gained versus the student perception of their gain in critical thinking skills. 7. REFERENCES Barrows, H., 1980, Problem based learning: An approach to medical education. New York: Springer Publishing. Dilworth, R.L., 1998, “Action Learning—Setting the Stage”. Performance Improvement Quarterly, 11 (1), 5-8. Friedman, W.H., 1995, “A New Model for Case Analysis: Iterative Triadic Thinking”. Journal of Education for Business, 70 (4), 228-232. Gupta, J.N.D. and Wachter, R.M., 1998, “A Capstone Course in the Information Systems Curriculum”. International Journal of Information Management, 18 (6), 427-441. Hall, O P., 2002, “Developments in Learning Networks”, Proceedings of ISECON 2002, 323b. Hemphill, H. and H. Hoyet, 2000, “Language of Instruction: Assessing the Instructional Syntax of Technology Based Training”. Educational Technology. 40(4), 53 -56. Hiltz, S.R., Benbunan-Fich, R., Coppola, N., Rotter, N., and Turoff, M., 2000, “Measuring the Importance of Collaborative Learning for the Effectiveness of ALN: A Multi-Measure, Multi-Method Approach”. Journal of Asynchronous Learning Networks, 4, (2), 103-125. Jonassen, D.; Davidson, M., Collins, M., Campbell, J., and Haag, B., 1995, “Constructivism and Computer Mediated Communication in Distance Education”. American Journal of Distance Education, 9 (2), 7-25. Kearsley, G. and Shneiderman, B., 1998, “Engagement Theory: A Framework for Technology-Based Teaching and Learning”. Educational Technology, 38 (10), 20-23. Knight, L. V., Steinbach, T.A., and White, J.D., 2002, “An Alternative Approach to Web-based Education: Technology-intensive, Not Labor-Intensive”, Proceedings of ISECON 2002, 223b. Moore, M., 1993) Theory of transactional distance. Theoretical principles of distance education. Routledge, NY, D. Keegan (ed), 22-38. Petrides, L.A., 2002, “Web-Based Technologies for Distributed (or Distance) Learning: Creating Learning-Centered Educational Experiences in the Higher Education Classroom”. International Journal of Instructional Media, 29 (1), 69-77. Rakes, G., 1996, “Using the Internet as a tool in a resource based learning environment”. Educational Technology, 6 (2), 52-29. Robin, B. and McNeil, S., 1997, “Creating a course-based Web site in a university environment”. Computer & Geosciences, 23 (5), 563-572. Saba, F, 1988), “Integrated telecommunication systems and instructional transaction”. The American Journal of Distance Education, 2 (3) 17-24. Smith, P.A.C. and O’Neil, J., 2003, “A Review of Action Learning Literature 1994-2000: Part 1”. Journal of Workplace Learning, 15 (2), 63-69. Sonner, B., 1999, “Success in the Capstone Business Course—Assessing the Effectiveness of Distance Learning”. Journal of Education for Business; 74 (4), 243-247. Spiceland, J.D. and Hawkins, C.P., 2002, “The Impact on Learning of an Asynchronous Active Learning Course Format”. Journal of Asynchronous Learning Networks, 6 (1), 68-75. Suchman, J., 1962, The elementary school training program in scientific inquiry. Report to the U.S. Office of Education. Project Title VII. Urbana, IL: University of Illinois. Thompson, E.R., 2000, “International Perspective: Are Teaching Cases Appropriate in a Mainland Chinese Context? Evidence from Beijing MBA Students”. Journal of Education for Business; 76 (2), 108-112. Vat, K H., 2002, “On the Importance of Organization Modeling for IS Education”, Proceedings of ISECON 2002, 244b. Web Resources: MSIS 2000: Model Curriculum and Guidelines for Graduate Degree Programs in Information Systems”. Association for Computing Machinery and Association for Information Systems. Accessed at http://www.aisnet.org/Curriculum/msis2000.pdf, on July 18, 2003. APPENDIX A. SAMPLE DESCRIPTION OF CAPSTONE COURSE This course is designed for students and managers who desire an understanding of how an information technology (IT) enables organizations to conduct business more effectively and differently in rapidly changing business environment. The changes, in many cases threatened not just a firm’s competitiveness but also its survival. Executives bear an enormous burden as they attempt to understand the challenges, maintain the business operations, and make intelligent decisions that will keep them ahead of competitors. After completing this course students and managers should have: • A better understanding of the variety of issues facing the various stakeholders involved in IS deployment in organizations. • A better understanding of the importance of harnessing the power of new technologies to enable them to make better decisions and more effectively manage the firms in which they work. • A better understanding of the importance of linking business strategy with IT strategy. APPENDIX B. List of Assignments * Topic Proposal. One page description of the topic for your research paper/project.  Include the following: Your name Working Title for Paper Why you chose the topic What courses and coursework will help you in this paper What you hope to achieve This will be very speculative at this point. * Detailed Outline. This should be a detailed outline of your paper.  It should be about 2 to 4 pages in length, single space, 11 pt. font.  Try to make your outline about 3 levels deep (this is actually quite deep). You should have 4 or 5 main sections.  Some of the main sections might be: INTRODUCTION, HOW THE TECHNOLOGY WORKS, MARKET SEGMENTATION, CRITICAL SUCCESS FACTORS, MAJOR PLAYERS IN INDUSTRY, PREDICTION OF THE FUTURE, etc. Obviously, you will not have all of these; it will depend on the thrust of your paper. You must include a list of the 3 to 5 BEST sources you used to create the outline.  These will be used by your fellow students when they critique your outline. * Peer Review. You will be required to critique the work of 3 fellow students.  Include questions or points that will help the student tailor their paper to the interests of the class.  Keep your criticism constructive.  Offer additional sources of information where possible. * Presentation. Adapt your paper outline into a presentation.  Submit a PowerPoint file with 20 to 25 slides.  Most importantly, use the Notes feature of PowerPoint to write the text you intend to say during the presentation.  (You must overcome the ambiguity of using bullets with short description--your Notes should provide clarity.) * Paper. This should be about an 8 to 10 page paper (11 pt. font, one and a half spacing) which essentially is the same as the presentation with minor refinements.  Include the references at the end. * Class Participation. Your participation will be determined based on several factors, depending on whether you are in the “live” classroom section or distance learning section. 1 aburns@cs.depaul.edu 2 janickit@uncw.edu