Best Practices: A Cure for the New Prep Headache? Alan T. Burns alan.t.burns@gmail.com Argosy University Chicago, Illinois 60601 USA Theresa A. Steinbach tsteinbach@cti.depaul.edu DePaul University School of Computer Science, Telecommunications and Information Systems Chicago, Illinois 60604 USA Abstract The teaching of a new course is colloquially known among faculty as a “new prep.” New preps are often time-consuming and laborious for instructors. They can be particularly frustrating when this effort does not yield results in the classroom. This research explores how a best practice approach can make the transfer of new preps across faculty less burdensome and increase student learning. Best practices are desirable because they inject substantial, relevant and validated knowledge directly to the task at hand. The new prep transfer process is modeled in terms of communication theory, where the transfer occurs between an experienced instructor (source) and an instructor new to the course (recipient). A template has been developed to facilitate the transfer of best practices for a course. Some features of the template are discussed as are the transfer barriers they seek to overcome. Although one benefit of the best practices approach presented is reduced faculty workload, it is believed that the quality of instruction and student learning should increase as well. Keywords: best practices, IS education, knowledge transfer 1. INTRODUCTION Educational research has four goals. It can increase effectiveness and/or efficiency in learning by students, as well as teaching by instructors. Effectiveness is defined as the ratio of the actual outcome to the possible or ideal outcome, i.e. “doing the right things; efficiency is the ratio of output to input or “doing things the right way (Cowan, 1985). Appendix A summarizes the four goals. Most educational research focuses on improving the effectiveness of student learning. While admirable, this goal is not singular. Increasing the efficiency of instructor teaching is also important, especially to Information Systems educators who spend considerable effort remaining current with new technology. This research focuses on how the transfer of best practices can deliver proven teaching materials and methods to faculty, hence making them more productive. Perhaps the most time-consuming task for faculty is the preparation of a course they have never taught, colloquially known as the “new prep.” New preps can either be courses already developed by one instructor where materials are subsequently transferred to another instructor, or the most difficult of new preps, the course that has never been offered. Once taught a few times, the course tends to run smoothly, but the learning curve for teaching a course can be long and painful. Inefficiencies abound in the early stages, whether related to learning new material, sequencing topics for students, creating meaningful assessment methods (quizzes, homework, etc.) with corresponding metrics, or crafting lectures which engage students. Harvesting the fruits of faculty labor may take several iterations, spanning months or even years. “New preps” can be nightmares for teachers, to say nothing for the students who must endure them. One would think a process as common as a new prep would be supported by substantial research findings; however, research in this area is scant or nonexistent to the best of our knowledge. The purpose of this paper is to begin filling this void in understanding about new prep transfers, by modeling the process as a form of knowledge transfer. Specifically, proposed is a configuration of a best practice template which overcomes many of the challenges in transferring course preps across faculty. Faculty who teach in an area as dynamic and ever-changing as Information Systems should benefit from a systematic study of how course configurations are transferred to one another, since courses and their content can turn over so quickly. While one benefit of the approach presented is reduced faculty workload, it is believed that student learning should increase as well, since faculty who accelerate up the learning curve will offer effective teaching more quickly, thereby improving the quality and rate of student learning. 2. SOME PROBLEMS WITH NEW PREPS Great teaching is as much an art as a science. Within the classroom, effective course delivery requires the instructor to know the material completely. The instructor must augment the course’s principles with metaphors and examples that are meaningful to students. Ideally, the instructor will anticipate student questions and respond quickly with explanations to remove the “bugs” from their thinking. Effective instruction requires the mastery of many aspects of course delivery beyond the classroom, however. Designing assignments and appropriate rubrics for assessing student learning is critical. Matching the appropriate assignment format (exam, research paper, quiz, project, etc.) to the subject matter is a frequent challenge, usually solved by trial and error. Along with these complexities, the learning process by instructors for course delivery can be laden with challenges. Time pressures to deliver material which builds upon itself increases the difficulty. Feedback from students may not be useful, as they frequently cannot articulate the source of confusion. In the worst case scenario, a teacher may be assigned a course at the last minute, providing little time to understand the material, let alone deliver it to students. All too often, the process works only by endless hours of faculty effort, experimentation and learning. 3. WHAT ARE BEST PRACTICES? The term “best practice” refers to a superior internal practice within the organization that provides better results than any known alternatives (Szulanski, 2000). The concept can be traced back at least as far as Frederick Taylor, who proclaimed for each element of a trade “there is always one method and one implement which is quicker and better than any of the rest” (Taylor, 1919). Taylor was referring to manufacturing operations involving physical labor, but the idea has gained recent, renewed popularity with the emergence of the Internet (O’Dell & Grayson, 1998). The scope of a “best practice” can range from a one-sentence suggestion, e.g. “To improve classroom interaction, try to learn your students’ names as early in the semester as possible,” to a list of guidelines, e.g. “Best practices for presenting online,” to a detailed plan with accompanying explanation of why a practice is successful, e.g. a collection of manuals concerning “How to build and operate a new franchise store.” The American Productivity & Quality Center (APQC) defines best practices as meeting at least some of the following criteria (O’Dell et al., 2004). Best practices 1) are discovered through assessments and audits or self-nomination in operating entities of the organization, 2) are recognized by internal and external experts or sources, 3) can easily be measured, and/or 4) result in a measurable improvement for the organizations. The term is used to describe a practice to sufficiently complete a discrete but nontrivial task; in this case the task is a new prep transfer. The appeal of best practices is in their replication. As a good, knowledge is particularly treasured because its value increases with usage, in contrast to many other assets such as raw materials or equipment (Szulanski, 2000). With the articulation of a best practice by a subject matter expert, the organization receives an injection of validated knowledge at a central decision point. Identification and transfer of the knowledge required for successfully demonstrated practices or processes among units or groups in an organization creates a value proposition that closes performance gaps and bring all similar organizational units up to the same level of high performance (O’Dell et al., 2004). Best practices are typically modeled in terms of communication theory (Shannon & Weaver, 1949) as a form of knowledge transfer. The transfer occurs between a knowledgeable source and a willing recipient who seeks to duplicate or adapt a validated practice. The word “transfer” is used instead of “diffusion” to emphasize that the movement of knowledge is purposeful and a distinct experience, not a gradual process of dissemination. A successful knowledge transfer occurs when the organization recreates and maintains a complex, causally ambiguous set of routines in a new setting (Szulanski, 2000). Figure 1 depicts the overall transfer process, beginning with the Source, who may be a course mentor, instructor familiar with teaching the course, or a committee of people who collaborate on the best practice. Lessons learned are compiled and articulated (encoded) into Best Practice documents. The documents are then delivered to a Recipient, who may be an experienced instructor new to the course or a new instructor. The Recipient then interprets (decodes) the Best Practice document to deliver a course to the end recipients, Students. Modeling best practice replication with communication theory is beneficial because it identifies a fairly comprehensive list of areas which inhibit successful transfer. In his study of best practice transfers in several large corporations, Szulanski (1996) has identified a multitude of possible problems with the source, channel/message, recipient and organizational context. Szulanski’s barriers can be applied to the new prep transfer process. For example, a source instructor with employment security concerns may not be motivated to share knowledge. This lack of motivation at the source represents a barrier to knowledge transfer. Another barrier may occur with the channel or message, e.g. when a source uses technical jargon or acronyms the recipient does not understand. Recipient barriers can include a lack of motivation to learn, or an inability to receive or retain the knowledge provided by the source. Finally, transfer occurs in an organizational context. If the organizational environment is not conducive to transfer, such as excessive competition among faculty, then transfer may not occur. Appendix B provides a list of common impediments in the new prep transfer process. Best practice initiatives generally require iterations across six stages: 1) identify, 2) create, 3) document, 4) validate, 5) publish, and 6) adopt (APQC, 2004). The nature and frequency of the transfer problems described above vary depending on the stage of the transfer process. No stage is particularly easy; in fact the entire process is laden with barriers to adoption. Part of the problem is the misleading adjective “best.” Best practices are not so defined that their practice can be programmed; nor can they be completely understood or explained. They are context-dependent, i.e. an accumulation of what an organization believes to be the best way to do something in a given situation; however, the practice may not work elsewhere since every situation is different. The bottom line is the definition, capture, documentation, validation and particularly replication of what constitutes a best practice can be very difficult. The core of the best practice need not be fully understood before it is replicated. Winter and Szulanski (2001) discuss what is transferred as an idealized “Arrow core,” i.e. a set of attributes, a configuration or information about a routine, business model, organizational form, etc. Organizations acquire understanding about the best practice through learning, e.g. through experimentation in another context-dependent setting. Such learning is reflected in the creation of a central organization that has the dynamic capabilities needed to transfer the Arrow core to new outlets (Winter & Szulanski, 2001). These dynamic capabilities themselves improve as a result of learning from experience, thus creating an iterative method of continuous improvement. In summary, the transfer of best practices is worthwhile to organizations. Not only does the organization benefit from the injection of expertise into its processes; it provides a formal mechanism by which learning can occur in several stages. However, the process of knowledge transfer is far from trivial and laden with barriers to adoption at every stage. Research has explored the use of best practices in some areas of education, such as Internet-supported learning (Abel, 2005), preparing a syllabus (Slattery & Carlson, 2005), university administrative systems (Dodd, 2006), and audience/student response systems (Caldwell, 2007). To our knowledge, no work has studied the application of best practices to the new prep transfer process. 4. INCORPORATING BEST PRACTICES IN THE NEW PREP PROCESS Over the past year a process has been developed for creating and transferring new preps. The process was refined based on the transfer of three separate courses. The process presented includes several activities, including a) creating a best practice template which summarizes the goals, procedures, schedule and content for a successful course configuration; b) modularizing course content to provide flexibility in delivery for the recipient instructor; c) delivering the configuration to recipient instructors and fine tuning it based on their feedback, and d) integration of the configuration into the normal practice of course delivery, including a mechanism for maintaining the practices. This section discusses two components of a template configuration which embodies the lessons learned from this project. The first component, a Best Practice course document, formalizes and centralizes information about the configuration of the course, including course topics, proposed weekly schedule, sample assignments, etc. The second component, a repository of course modules, is organized in folders which the recipient instructor can use and adapt to their own delivery. The discussion includes features of both components and the adoption barriers the features seek to overcome. The section concludes with a discussion of dilemmas associated with managing the overall process and some practical strategies to remedy them. 4.1 The Best Practice Course Document The first component in the new prep transfer method is a Best Practice Course Document that serves as a single, central point of information about a course. This document is created by a source instructor, which can be a course mentor, an instructor who has taught a course successfully on multiple occasions, or a committee that oversees a course. The creation of this document provides an organizational mechanism for materials already gathered or developed. A screen shot of the cover page of this document is shown in Appendix C. A sample page is shown in Appendix D. Below is a list of some of the Best Practice (BP) document’s features, along with the rationale for inclusion. Formality The BP document appears more formal than most documents within our school. Graphics and styles were added to “gold plate” the document, since it should reflect the best attempt to deliver teaching excellence in the classroom. Rationale: A formal approach improves transfer by increasing the credibility of the source document and author. Disclaimer Best practices vary in terms of whether they are mandatory or optional; faculty are more receptive to practices if they are suggested, not required. To this end, a disclaimer is included on the first page of the BP document that states the practices within the document are not mandatory but rather a summary of lessons learned by past instructors. Rationale: The disclaimer serves a few purposes. First, it tempers the recipient’s expectations, so they understand early on that some learning and experimentation will be required on their part. Second, it promotes continuous improvement by encouraging instructor to modify the course template to suit their needs and report their experiences after the course has concluded. Third, it leaves the instructor in control of the course configuration. Course overview This section describes basic facts about the course, including course description, prerequisites for the course (especially hidden ones), courses for which this course is a prerequisite, and what curriculum requirements the course fulfills. Rationale: Communication theory suggests increasing source credibility increases the likelihood of successful knowledge transfer. Articulating, centralizing and maintaining these elemental course facts can improve the credibility of the BP document and the new instructor. For example, students often bring in questions about a course during the first week of class to clarify (mis)information they may have heard secondhand from other students. Knowing whether a course is a prereq, what requirements it fulfills, etc. gives the new instructor credibility to answer these questions with the most current information available. By leveraging these potential gains in instructor credibility, the Best Practice document may increase the probability of successful transfer. Weekly schedule of topics Included is a schedule of weekly topics for a course and some alternative sequences. Also included is a description of why topics should be covered in a certain order. Rationale: One of the “sticky” areas of course delivery knowledge is the pacing and sequencing of material. The source (experienced instructor) knows implicitly why topics are ordered a particularly way. This section encourages the source to make this sticky knowledge explicit. For example, the source can articulate a few alternative sequences and explain their rationale. When the knowledge becomes less sticky, it flows more easily to recipients accelerating them up the learning curve. Classroom questions Where appropriate, a bank of provocative class discussion questions organized by course topic can be included. Audience response systems (ARS) enable students to respond to questions via small, hand-held, remote keypads. Use of this technology spices up the standard lecture classes with periodic breaks and in situ assessment of student opinions or understanding, increasing the degree of interactivity in large classrooms (Caldwell, 2007). Answers can be immediately tallied and displayed on a classroom projection screen where both students and instructor can see and discuss them. Rationale: One of the “stickiest” areas of course delivery knowledge is how to lead class discussions to reinforce and integrate course learning goals. It would be difficult for an experienced instructor to articulate this in a sufficient level of detail to guide the recipient, e.g. “if students mention this idea, then discuss this concept, else discuss other concept, etc.” Yet, rich class discussions are rewarding for students and instructors. The bank of discussion questions reaches a middle ground by specifying the major points of discussion; it is left to the recipient instructor how to proceed in the classroom. Course Repository reference The BP document contains a link to the repository and a description of the modular content available. Excerpts from the modules are embedded within the Best Practice document. Rationale: Following the tenets of best practices, source instructors create and validate modules of course content, which are made available in a repository for recipient instructors (see next section). 4.2 Course Content Repository The second component of the template is a repository of course content modules, created and validated by knowledgeable sources, such as experienced instructors or a committee overseeing courses or curriculum. Appendix E contains some screen shots of excerpts from the repository. The ideal transfer creates a successful knowledge transfer of the new prep while minimizing the burden on faculty for absorption and subsequent delivery. Toward this end, the phrase “Drag, Drop and Deliver” was coined to describe this idealized approach, where the recipient instructor simply: 1) Drags a module (folder) of content from the source repository, 2) Drops it onto the their desktop, and then 3) Delivers it in class. Drag-Drop-Deliver is an obvious oversimplification, but it represents a worthy goal, especially in the eyes of recipient instructors. Instructor learning and experimentation are still required, but at least the instructor is at a starting point where content is validated and proven to be effective for student learning. Some of the features of the repository are discussed here, with the rationale for their inclusion. Modularity Course content is organized in terms of modules that require about 60 to 90 minutes of class time. A module’s content is stored in one folder. The module usually contains a theoretical component (e.g. a PowerPoint file) and a “learning-by-doing” component (e.g. an assignment description file which supports the theory). Modules are self-contained and independent of one another. This produces a “buffet” approach, where the recipient instructor is free to choose whichever modules they like in any combination. Rationale: A repository of best practices is more likely to be adopted if the recipient instructor has flexibility in choosing content. To that end, the modules are created with high cohesion, where the content supports a few well-defined, discrete learning goals. Modules also have low cohesion; it is necessary only to specify the basic prerequisites for a module. This allows the instructor creativity in sequencing material. Instructor Notes Each module folder contains an instructor notes document with suggested strategies for delivering the module’s content (about 2-3 pages). A screen shot of one such file is in Appendix E. Instructor notes include: * Duration of the module * A list of the learning goals the module supports * Prerequisite knowledge, if any * A description of the files required for the module (the content of the module folder) * A procedure for delivering the module * Representative comments from past students regarding their interaction with the module * Problem areas for the content, and how to mitigate them Rationale: Some aspects of teaching knowledge are inherently sticky. A good example is how to manage a particular case or class discussion. The instructor notes document captures this knowledge regarding what works in the classroom (hence, the list above). Instructor notes are written in a conversational tone and easy to read—the plight of the recipient instructor is difficult enough as is! Lessons learned reside in one location, within the module. Assignment templates/solutions. Modules should contain a “learning-by-doing” component to provide opportunities for the student to master the module’s learning goals. Thus, wherever possible a module will contain an assignment description and answer template for students, as well as a scoring rubric and solution key for instructors. The recipient instructor should need only to make the assignment description and answer template available to students. The solution key and scoring rubric provide the instructor with the means for grading. Rationale: Some assignments contain sticky knowledge in terms of purpose, procedure and assessment. The goal is to remove the guesswork from assignments. The answer template helps the student focus on the learning goals and not other superficial elements (such as what font type to use for a research paper, how a particular question should be answered, etc.) The solution key shows the intention of the source (initial author of the assignment). The scoring rubric provides the instructor a starting point. In this way, knowledge is made less sticky and more easily transferred. 4.3 Managing the Process Whether a best practice approach is worthwhile for a university to pursue depends on several factors, such as the number of courses taught within the school, availability of source instructors, number of new instructors, the frequency of new prep transfers and the need for quality improvement. As is the case with all knowledge management initiatives, best practice approaches are prone to failure unless certain barriers are addressed. First, the bottleneck in the transfer process is generally the encoding of the message by the source. It is often difficult for a source to articulate what he or she knows. Therefore, an initial investment is required for setting up a process. The template approach described here mitigates this problem to a degree, since templates allow for reusability. With each successive effort, the source can reuse certain materials such as general guidelines for teaching any course. Effort should also be reduced as the source learns about the overall process. Source instructors can take heart in the fact that their investment will pay dividends in the long term, especially if they become recipients of other instructors’ practices. Perhaps the chief dilemma of any knowledge management effort is maintenance. Knowledge, lessons learned and best practices decay over time; therefore, some schedule of maintenance is required. Maintenance is made easier if practices are institutionalized. For example, a standing committee that oversees popular or common core courses can be responsible for ensuring consistency of course delivery across all sections of a course. Best practices make the committee’s job easier, since it is a move toward standardization. The best practices document becomes a living document maintained by a committee, which uses it as a primary vehicle for communication across faculty who teach several sections of a course. 5. SUMMARY This research explores how best practices can improve the transfer of new course preps to instructors in order to improve teaching and learning. A best practice approach injects a rapid influx of validated knowledge directly to a decision point or task, thereby improving performance of novices or actors new to the task. Best practices are typically modeled as a form of knowledge transfer between a source and recipient. During the past year various methods have been explored for documenting best practices for teaching a course. The lessons learned are embedded into a template with two main components: a best practice document for the course, and a repository of course content. This paper outlined some features of both components and explained the rationale for their inclusion in terms of the transfer barriers they seek to overcome. The efficacy of templates is in their reuse, and the amount of effort saved compared to starting from scratch. The template outlined here can reduce the work of other instructors contemplating similar best practice initiatives. Future work which builds on the template through replication should create a pathway by which many best practices in IS education can be shared. 6. FUTURE DIRECTIONS This preliminary investigation has framed the transfer process for new preps from instructor to instructor. Many research questions have been unearthed, requiring additional qualitative and quantitative study. Below are some questions of potential interest to IS educators: * Which parts of the Best Practice document and repository are perceived as being most useful to recipient instructors? Which parts improve the rate and quality of knowledge transfer? * What quantitative measures of benefits are associated with a best practice approach? For example, how much instructor time and effort is saved compared to the traditional method of new prep transfer? * How does the application of best practices extend to the end recipients, students? Is learning made more efficient? * Do best practices translate into higher quality and consistency? * Is a modular approach to course design more transferrable compared to more traditional approaches of course design? Is it more effective in the long term in terms of teaching and learning? * What barriers exist in the new prep transfer process, and how might they be overcome? * What are the barriers and issues associated with transfer of new preps across faculty from different universities? 7. REFERENCES Abel, R. (2005). Implementing Best Practices in Online Learning, EDUCAUSE Quarterly, 3:5, 75-77. O’Dell C., Dixon, R. & Szulanski, G. (2004). Facilitated Transfer of Best Practices, Leavitt and Skogstad (eds.), APQC Publications. Caldwell, J.E. (2007). Clickers in the Large Classroom: Current Research and Best-Practice Tips, Life Sciences Education, Vol. 6, Spring: 9-20. Cowan, J. (1985). Effectiveness and Efficiency in Higher Education, Higher Education, 14:3, 235-240. Dodds, T. (2006). Changing Assumptions in Best Practice, EDUCAUSE Review, July/August: 76-77. O’Dell, C. & Grayson, C.J. (1998). If Only We Knew What We Know: The Transfer of Internal Knowledge and Best Practice, Free Press. Shannon C. E. & Weaver W. (1949). The Mathematical Theory of Communication. Chicago University Press, Chicago, IL. Slattery, J.M. & Carlson, J.F. (2005). Preparing an Effective Syllabus: Current Best Practices, College Teaching, 53 (4): 159-164. Szulanski G. (1996). Exploring internal stickiness: Impediments to the transfer of best practice within the firm. Strategic Management Journal, 17 (Winter Special Issue): 27-43. Szulanski, G. (2000). The process of knowledge transfer: A diachronic analysis of stickiness. Organizational Behavior & Human Decision Processes, 82 (1): 9-27. Taylor, F. (1919). The Principles of Scientific Management. New York: Harper & Brothers Publishers. Winter, S. & Szulanski, G. (2001). Replication as strategy. Organization Science, 12 (6): 730-743. APPENDIX A. Goals of educational research. FOCUS Learning (students) Teaching (instructors) GOAL Effectiveness (Doing the right things) Increase learning effectiveness e.g. Action learning Increase teaching effectiveness e.g. Using multimedia Efficiency (Doing things the right way) Increase learner efficiency e.g. Automated submission of assignments Increase teaching efficiency e.g. Transfer of best practices APPENDIX B. Some Impediments to the New Prep Transfer Process. Location Transfer Impediment Description Source Source lacks motivation. The source may lack motivation to support the sharing of knowledge, e.g. out of job security concerns. Source lacks credibility. A successful transfer may not occur if the recipient perceives the source lacks credibility, e.g. if the source instructor receives poor teaching reviews. Recipient Recipient lacks motivation. Recipients may lack motivation to support the sharing of knowledge, e.g. if they perceive the activity as unnecessary. Recipient lacks absorptive capacity. Even if a recipient is willing to receive knowledge, transfer may not occur if the recipient does not have the background to absorb the message. Recipient lacks retentive capacity. Similarly, a recipient may not have the capacity to retain the knowledge, e.g. if it is not incorporated into a business process. Message Causal ambiguity. When it is not clearly understood why certain practices are effective, transfer of those practices is bound to be problematic. Knowledge stickiness. When the knowledge does not flow easily across participants, e.g. if it is not easily articulated, it is considered “sticky”. Transfers of sticky knowledge are difficult to resolve and create barriers to adoption. Context Barren organizational context. Sometimes knowledge is not transferred because the organizational culture or climate does not support it. For example, competitive incentive programs may inhibit people’s willingness to share knowledge. Arduous relationships. Some organizational relationships may not be conducive to knowledge transfer, e.g. if a university utilizes many adjunct faculty in remote facilities; the lack of “thick” communication early in the transfer process will inhibit transfer. APPENDIX C. Cover Page of a Best Practice Course document. APPENDIX D. Sample Page from a Best Practice Course document APPENDIX E. Screen shots from the Course Repository. Screen shot 1. Module folders for the repository / Contents of a module folder. Screen shot 2. Instructor notes for a module. Screen shot 3. Excerpt from assignment for students.