Less is More When Developing PowerPoint Animations Stephen Mahar mahars@uncw.edu Ulku Yaylacicegi yaylacicegiu@uncw.edu Thomas N. Janicki janickit@uncw.edu University of North Carolina Wilmington Wilmington NC 28403 Abstract Presentation software has the potential to complement traditional teaching methods in the classroom. Over the last decade, packages like PowerPoint have gained wide acceptance in academia for their ability to incorporate additional stimuli in traditional presentations via animation options. Recent research has considered the benefit of using such custom animation to incrementally introduce concepts in the classroom (for the entrance, exit, and emphasis of text and figures) and suggests that static slides allow students to retain more information than their dynamic counterparts. This research extends those findings by assessing the impact of custom animation on student learning across various demographic and performance characteristics (i.e., gender, class year, and academic performance). Computational results show significant differences between overall student performance after viewing non-animated and animated PowerPoint slides, independent of most student attributes, and suggest the robustness of past findings on the impact of custom animation on student learning. Keywords: pedagogy, computer based training, instructional technology 1. INTRODUCTION AND RELATED LITERATURE As universities enable their classrooms with technology, instructors may feel under pressure to use all of the “bells and whistles” that the new equipment provides in their classroom presentations. Further, incoming university students accustomed to multimedia may expect entertainment as well as education during class sessions. These trends have led to increased academic focus on how new stimuli and technology enabled teaching methods may be incorporated into the classroom. One popular and commonly used method of leveraging the technology-enabled classroom and adding stimuli to a presentation is PowerPoint. Since the inception of PowerPoint, its potential benefits for both traditional and web presentations has been considered by many researchers. Past studies have focused on student preference of the medium and its ability to enhance student learning. Recently research has also provided some insight into the value of using “custom animation” for the entrance, exit, and emphasis of text and figures to guide concept development by incrementally introducing concepts. Despite the apparent benefits of using custom animation in PowerPoint presentations, the growing preference for its use is still based on little more than intuition (Lowe, 2003). Many educators and students intuitively feel that integrating the computer’s interactive capabilities into a classroom experience enhances learning and the student’s ability to apply knowledge and skills (Alavi, 1994). This belief has been challenged in the literature. Moreno & Mayer (2000) and Mayer (2001) considered the impact of presentation complexities and find that irrelevant sounds or pictures in a lecture can reduce student comprehension. Bartsch and Cobern (2003) also showed that students perform worse on quizzes when PowerPoint presentations include non-text items such as pictures. More recently, Mahar et al. (2009) found that incrementally introducing information on PowerPoint slides via custom animation decreases student learning versus having all information shown on the slide at the same time. Although these studies provide valuable insight to instructors, it is unclear how custom animation impacts student learning across various demographic and performance characteristics. Along these lines, James et al. (2006) argue that there needs to be more research on the impact of PowerPoint animation based on discipline and majors. Unfortunately, research in this area has been relatively limited and inconclusive. Annetta et al. (2007) found that there was a gender difference in learning in the sciences, and suggested that the integration of graphics in PowerPoint can be a more effective approach to learning science for females than for males. Conversely, a study of accounting students by Nouri and Clinton, (2005) found that gender played no role in the effectiveness of the learning and overall perception of students when it came to PowerPoint animations. Age may also influence the effectiveness of training systems. Koons (2008) reported differences between media packaging when the subjects were under twenty, between twenty and thirty and above thirty. In general Koons found the under thirdly subjects were much more receptive to enhanced visual and simulation presentations. In the medical sciences, Carmichael and Pawlina (2000) engage student learners and have them help create animated PowerPoint slides for their anatomy course. Post course evaluations show increased favorable evaluations by the students. This study fills a gap in the literature by extending the work of Mahar et. al (2009), to investigate whether custom animation can create a disadvantage for students’ learning experience, independent of student academic achievements, gender differences, advancement in student’s degree and major choice. The knowledge gained from this study provides valuable insight for instructors, particularly those teaching online web-based courses. Section 2 states the research hypothesis. Section 3 details the experimental testing and methods. Data is then presented and analyzed in section 4. Sections 5 and 6 present our results and discuss implications, limitations, and direction for future research. 2. HYPOTHESES Mahar et. al (2009) show that incrementally introducing information on PowerPoint slides via custom animation decreases student learning over having all information shown on the slide at the same time. This study tests the idea that learning traits with animation vs. with no animation would be disparate for various student attributes, such as gender, varying academic success levels, academic progress, and academic disciplines. Specifically, we focus on how the added complexity of custom animation impacts student learning (or recall) of information depending on different student characteristics. For clarity we keep the same definition of “animation” as Mahar et al. (2009). That is, the incremental inclusion of information on PowerPoint slides. The objective of this research is to determine how custom animation impacts learning of a new conceptual topic considering differences in academic success, gender, academic year and discipline. Extending the ideas presented in Mahar et al. (2009), Bartsch and Cobern (2003) and Lowe (2003), we hypothesize that average student recall of information presented in non-animated presentations exceeds average student recall of information presented in animated presentations independent of academic achievements, gender differences, advancement in student’s degree and student’s major choice. Specifically, 3. EXPERIMENT The experiment was conducted with 93 students taking a large introductory section Management of Information Systems class at a midsized public university. This course was required for all students in the business school. The class had separate lecture and lab meetings and students were divided into separate lab subsections at the beginning of the semester. Each week the entire class met once in a large auditorium for group lecture. In addition to the group lecture, each week each subsection met separately in a computer lab. The environment provided a controlled setting where all students received identical conceptual instruction throughout the course of the semester. The lab sessions were used to test our hypotheses. Each lab classroom was arranged identically. No significant difference was discovered when comparing the average test scores of students in the sections. Two versions of a PowerPoint lecture on the information security topic were developed using Camtasia Studio software. The software enabled the addition of a “voice over” to explain the concepts presented via PowerPoint. The ‘voice over’ narration was exactly the same for both sets of PowerPoints. The only difference between the two presentations was the presence of custom animation to incrementally present information. Students were shown either the animated or non-animated recordings depending on their lab section via a projection screen located in the front of the classroom. After viewing the presentation, students were asked multiple choice survey questions to gauge their recall of the material presented. The average number of animations per slide in the animated recording was 3.4. 3.1. Methods and Procedures Five weeks prior to the experiment all students were given a pre-test in the lecture class to assess their understanding of information security and privacy issues. Data was collected using Interactive Student Response Pads from Turning Point (i.e., “clickers”). These clickers were used regularly throughout the course to allow the instructor to take attendance, give pop quizzes, as well as collect anonymous feedback on the class’ understanding of a topic. Prior to the experiment students were divided into lab groups (according to the lab section they were enrolled in). For the experiment, one of these groups was treated as the test group and the other was treated as the control group. All groups were shown a PowerPoint lecture (with sound) dealing with the information security and privacy component of the course. The “control” group was shown the custom animated slides (dynamic case). In these slides, words/bullets/images entered the slide at different times to introduce concepts incrementally. The “treatment” or “test” group was shown the non-animated slides (static case). In these slides, all words/bullets/images entered the slide immediately together to introduce concepts simultaneously. Care was taken to control for aspects of the presentation not dealing with custom animation. Both treatment and control lectures were developed in Techsmith’s Camtasia Studio to ensure that both presentations progressed at the same pace and contained the same script of concepts from the same voice. Camtasia is currently one of the most widely used animated screen capture tools that allows instructors to capture visual activity occurring on the screen while narrating in the background (Gill, 2007). Both presentations had duration of 17:30 and used the same sound track. Practically, the control (animated) PowerPoint presentation was developed and recorded first. The treatment (non-animated) slides were then created by removing all custom animation from the animated presentation. The same dialogue (.mp3) track (i.e. script) was used in both the control and treatment Camtasia recordings. At the end of each presentation (control or treatment) students were presented with multiple choice questions (built into the end of the presentation slides to collect test results immediately and anonymously using student response clickers). The questions were the same multiple choice questions covered in the pretest (see Appendix A). Results were used to assess the impact of the custom animation treatment on student learning. This study measured the learning experience using animated PowerPoint slides for various student attributes. A set of multiple choice questions were prepared to assess the knowledge gained on concepts delivered with the experiment. Five weeks before the experiment was conducted, the students’ apriori knowledge of the material was tested using the prepared questions. On the day of the experiment, both test and treatment group students were subjected to the same set of questions. The impact of custom animation on student learning considering specific student characteristics, i.e. the learning disparity within clusters was evaluated by comparing performance of students who were shownlectures using animated and non-animated slide sets. 4. DATA AND ANALYSIS Student understanding of the information security concepts in the experiment was assessed earlier in the semester by asking nine survey questions using clickers. Each question was designed with four different answer choices. If a student chose to skip or not to respond to a question, the response was recorded as an unsuccessful attempt for both pre-experiment and post-experiment tests. The average pre-experiment test score for all the subjects was 38.39%. The percentage for correct answers varied roughly between 4 to 78%. The student responses for the same set of questions were collected immediately after the recorded lectures were shown. The percentage of correct answers was 81.98% for the test group and 71.43% for the control group, which indicated statistically significant (p<0.001) learning took place. For this study the comparison of means for student test scores was conducted using a two-tailed t-test. The tests measured the learning improvements for various student attributes; academic achievement, gender, academic year and major. Based on the average scores students obtained throughout the semester, students were grouped into four quartiles differentiating their academic achievements. The division of the quartiles is shown in table 1 found in Appendix B. The test investigated whether there are significant learning differences within the same quartile for the students shown animated slides vs. non-animated slides. Students are also grouped based on their gender, academic year and major. The same test is conducted to examine the differences within each group. 5. RESULTS The test results on academic achievement, gender, academic year and major differences are given respectively in tables 2, 3, 4 and 5 (see Appendix B). Table 2 shows the results for the differences in learning for different performance groups. Based on students’ academic performance during the semester, they are divided into four groups. The results indicate that on average, students that saw the non-animated presentation performed better on the survey than those that saw the animated presentation; regardless of the quartile their scores fell. Table 3 presents the results for the learning differences within a specific gender. While the average score for female students shown the non-animated lecture was 83.33%, the female students shown animated lecture scored approximately 20 points less in average. A similar significant trend is observed for male students with slightly less disparity in between the average scores (~13%). The results demonstrate that on average, students that were shown non-animated presentation performed significantly better on the test than those that were shown the animated presentation, regardless of the gender. Our results support the findings of Nouri and Clinton (2005). The differences of learning traits for academic progress are displayed in table 4. Introduction to Management Information Systems course is a course business students generally take in their sophomore or junior years. For this reason, we had insufficient sample size for seniors. Even though there was an 11 points difference between the average scores between test and treatment freshmen groups, this difference was statistically insignificant. For sophomores and juniors, the results were in agreement with the overall results for the whole sample. On average, students shown non-animated presentation performed significantly better than the students shown animated presentation, regardless of academic progress for sophomores and juniors. Finally, table 5 shows the results of learning differences within group of students that chose the same major. The results on average for each major appear to be in the same direction as the other factors, i.e. students shown non-animated slides appeared to retain the information better than the students shown animated slides. However, this trend is slightly different only for one group, students chosen “Information Systems” as their major. This observation may be due to the Information Systems students’ higher familiarity with the concept tested compared to students preferring other majors. Lowe (2003) suggests that if custom animation is used to deliver a topic known to the subjects, the animation has a positive impact in student learning. 6. DISCUSSION, IMPLICATIONS, LIMITATIONS, AND FUTURE RESEARCH This paper extends the exploration of PowerPoint animations by assessing the impact of custom animation on student learning across various demographic and performance characteristics. Even though students appear to like use of animations in lectures delivered using PowerPoint, there is evidence from literature suggesting that the learning benefits of using animations over static diagrams are limited (Mahar et al., 2009; Tversky et al., 2002). The results of this study show that conveying concepts in lectures via custom animation can create a disadvantage for students’ learning experience independent of student attributes. Even though the use of custom animation allows the introduction of new information incrementally the technique can adversely impact student learning experience when factual information is conveyed in the presentation. Subjects shown the static slides had better recall of graphics and text on the slides due to prolonged exposure to the information. The incremental introduction of concepts in dynamic slides’ was designed to prevent student exhaustion caused by visually presenting all concepts at once. However, the dynamic slides lead to excessive processing demands and limited exposure time. In this study, we observed that static slides allowed better knowledge retention for male students, female students, students with academic excellence, students with poor academic performance, students in different academic years and students from different disciplines. The results are consistent with the aggregate results presented in Mahar et al. (2009) and Lowe (2003) who suggested that diminishing expected benefits of animations may be caused by excessive processing demands on learners and a reduced learner engagement. 7. REFERENCES Alavi, M. (1994). Computer mediated collaborative learning an empirical evaluation, MIS Quarterly, V. 18, N. 2, pp. 159 - 174. Annetta, L., Slykhuis, D., and Wiebe, E. (2007) Evaluating Gender Differences of Attitudes and Perceptions Toward PowerPointTM for Preservice Science Teachers. Eurasia Journal of Mathematics, Science & Technology Education, 2007, 3(4), 297-304 Bartsch. R.A., & Cobern, K.M. (2003). Effectiveness of PowerPoint presentations in lectures. Computers & Education, 41, 77-86. Carmichael, S. and Pawlina, W. (2000), Animated PowerPoint as a tool to teach anatomy. The Anatomical Record, 261(2), 83-88. Gill, T.G. (2007). Quick and Dirty Multimedia. Decision Sciences Journal of Innovative Education, 5(1), 197-206. James, K., Burke, L., and Hutchins, H. . (2006), Powerful or Pointless? Faculty versus Student Perceptions of PowerPoint use in Business Education. 69(4), 374-396. Koons, W. (2008), Effects of Age and Media Packaging on Designing Training Systems. Proceedings of the Interservice/Industry Training, Simulation & Education Conference (I/ITSEC), 2006. Lowe, R.K. (2003). Animation and learning: Selective processing of information in dynamic graphics. Learning and Instruction. 13, 157-176. Mahar, S., Yaylacicegi, Y., & Janicki T. (2009). The dark side of custom animation. International Journal of Innovation and Learning, forthcoming. Mayer, R.E. (2001). Multimedia learning. New York: Cambridge University Press. Moreno, R., & Mayer, R.E. (2000). A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. Journal of Educational Psychology, 92, 117-125. Nouri, H., and Clinton, B. (2005), The Effects of Gender, Mental Imagery and Powerpoint Presentation on Student Attitudes. Advances. Accounting Education, Teaching and Curriculum Innovations , Vol. 7, 253-275 Tversky, B., and Morrison, J.B. (2002). Animation: Can it facilitate, Int. J. Human-Computer Studies, 57, 247-262 APPENDIX A – SURVEY QUESTIONS 1. Criminals frequently can talk a computer password out of an individual; a practice known as: a. Biometrics b. Password Sniffer c. Dumpster Diving d. Social Engineering 2. What indicates that a web page is secure to permit you to safely transmit your credit card information? a. https: - in the address line b. Your firewall is active c. Your virus scan is up to date d. All of the above 3. A Dept of Defense agency frequently auctions off computers to the public; later the agency learns that sensitive information is still on the hard drive, this is known as: a. Residual data b. Unauthorized access c. Compromising emanations d. Malicious code 4. You open an email attachment from an unknown source; later you discover all files with a .doc ext are unreadable; this is a. Unauthorized access b. Damage c. Theft d. Malicious codes 5. A disgruntled employee secretly installs a program that will allow him to access sensitive information at home, what security problem does this illustrate? a. Theft b. Unauthorized access c. Residual data d. Malicious code 6. Which of the following limits your exposure to a computer virus? a. Having your own flash drives for multiple machines b. Give it a “flu” patch c. Disconnect from the Internet d. Download music 7. Most computer crime is attempted by: a. Competitors b. Employees c. Outside Hackers d. Foreign Governments 8. Which of the following is NOT a commonly used means for access control? a. Auditing b. Locks c. Passwords d. Fingerprints 9. Encryption: a. Would permit all users to read your documents b. Would permit NO users to read your documents c. Turns a document into a series of letters and numbers d. Writes your documents to your hard drive APPENDIX B: TABLES Q1 90.00 Q2 95.00 Q3 97.33 Q4 100.00 Table 1. Division of quartiles based on average score throughout the semester. Quartile based on average quiz scores Average score / without animation Average score / with animation Grand Total t value of Difference p-value 1 75.31% 59.60% 66.67% -2.29 0.022 2 82.54% 72.65% 76.11% -1.57 0.117 3 84.44% 63.33% 73.89% -3.32 0.001 4 83.33% 66.05% 69.19% -2.39 0.017 Grand Total 81.11% 65.81% 71.41% -4.73 0.000 Table 2. Results based on the differences on the average quiz scores. Gender Average score / without animation Average score / with animation Grand Total t value of Difference p-value Female 83.33% 63.70% 72.43% -3.58 0.000 Male 79.63% 66.67% 70.91% -3.17 0.002 Grand Total 81.11% 65.81% 71.41% -4.73 0.000 Table 3. Results based on the gender differences. Year in college Average score / without animation Average score / with animation Grand Total t value of Difference p-value Freshman 77.78% 66.67% 74.07% -0.60 0.549 Sophomore 88.89% 69.92% 72.69% -4.10 0.000 Junior 80.39% 48.89% 68.72% -5.11 0.000 Senior 72.22%   72.22%     Grand Total 81.11% 65.81% 71.41% -4.73 0.000 Table 4. Results based on the differences in academic progress. Student major Average score / without animation Average score / with animation Grand Total t value of Difference p-value Accounting 77.78% 65.08% 68.89% -1.27 0.204 Finance 72.22% 66.67% 68.25% -0.44 0.661 Human Resources 88.89% 88.89% 88.89% 0.00 1.000 International Business 100.00% 55.56% 66.67% -4.65 0.000 Information Systems 61.11% 81.48% 73.33% 1.49 0.137 Management 82.22% 61.54% 67.28% -2.85 0.004 Marketing 82.41% 65.93% 73.25% -3.01 0.003 Operations Management 77.78% 61.11% 66.67% -0.93 0.355 Other 88.89% 66.67% 80.00% -1.76 0.079 Grand Total 81.11% 65.81% 71.41% -4.73 0.000 Table 5. Results based on the differences between major choice.