Will Handheld Computers Succeed in College? David W. Johnson and Derek Rudd School of Business, Utah Valley State College 800 W. University Parkway Orem, Utah 84058 Abstract Handheld computers have been used successfully in K-12 education based primarily on grants from hardware and software manufacturers. Several higher educational institutions have begun using handheld computers in the classroom as well. In fact, several universities are now requiring their use by all students. This paper describes a faculty-student research project aimed at understanding the willingness and ability of college students to use handhelds as an educational support tool. Results from the research show that, while handhelds work fine for their domain specific functions such as calendaring and to-do-list, their use as an educational support tool for college students may be limited. Keywords: PDA, handheld computers, higher education, computer assisted education 1. INTRODUCTION During the Spring 2003 term, students in a senior level IS class on Computer Supported Cooperative Work were given a Dell handheld computer (PDA) with wireless network card to experiment with as a potential aid in their college course work. Lectures were given on the successful use of PDAs in K-12 education and their current use in colleges as summarized in Johnson (2002) and Jones (2002). These papers suggest that while PDAs have had moderate success in K-12 education, their success in higher education was yet to be demonstrated. In addition, students were assigned a group research project to investigate the use of PDAs in education. Students were presented with an overview of the research process and each group was asked to write a Research Proposal for their project. After several iterations of the proposals and the approval of the institution’s IRB, the student researchers began their projects. Projects included “An Investigation of the Extent of PDA usage in Local Schools,” “A study of the use of Networked PDAs,” “The Use of PDAs in Medical Education,” “An Investigation of the use of PDA Software to Support Learning,” and “A Study of the Use of E-content on PDAs and the Retention of Learning.” This paper provides a description and results of one of the group research projects and summarizes the experiences of the students in using a PDA to help with their college course work during the term. The next section provides details of the student research project on the use of e-content on PDAs. This includes a literature review of the usage of PDAs in education which is followed by the approach and findings of the students’ research and their conclusions. Finally, the results of a survey of the experience of all the students in the college course regarding their use of the PDA to support their learning during the term are presented followed by an overall summary of the paper. 2. STUDENT RESEARCH PROJECT A group of five students, including the student author of this paper, worked together to define and implement a research study on the use of e-content on PDAs in education. What follows is an edited version of their final research report. Research Overview Our objective was to explore the effectiveness of using e-text on handheld computers for learning. Our hypothesis was that there would be no difference in reading comprehension/retention between using e-text on a PDA and reading the same material on paper. Our experiment was to randomly assign half of the students in a class to read certain course material using e-text on a PDA and the other half to read the same material on paper. We then administered a brief exam to test all students reading comprehension and retention. Following the exam we also administered a PDA Usage Survey to those students who had used a PDA in the experiment. As a laboratory for this experiment we utilized three middle school classes and one high school class, whose teachers we knew, in addition to our college class. After performing the experiments, we analyzed our data and reached our conclusions. Before going into the details of our research, we first summarize the literature in the area. Literature Review Many K-12 schools have been able to utilize PDAs in their classrooms in a variety of ways. One of the best sites for viewing such information can be found at Palm.com. This site includes various stories of successful use of PDAs in the classroom. Teachers and students have been able to use the most popular features of the PDA like the calendar, reminders, and contacts. Others have been able to use the PDAs to aid in data collection. Students with disabilities have also been able to perform better in school with the help of PDAs. North Star Middle School in Eau Claire, WI, was part of a study that took place in 1995 when PDAs were first gaining popularity. Utilizing the infrared port and basic productivity applications that came with their Palm Pilots, students were able to download assignments and keep a running calendar of their due dates. One of the advantages that the school had was the ability for the PDAs to interface with both Macintosh and Windows environments thus making the use of the PDAs possible in every classroom (Palm Inc., 2003). PDAs have also been used for more than just appointments, assignments, and reminders. At the Consolidated High School District 230 in Wisconsin, students have been able to track their nutritional and physical fitness information using the PDA, thus allowing them to record their progress toward their goals. Biology students have also used their PDAs to collect ecological data. Finally, English students have used PDAs for writing, sharing ideas, and for journal entries (pdaED, 2001). Bev Schottler, a resource teacher at Marysville High School in Marysville, KS, utilizes PDAs to help students with disabilities. Some students are not able to take legible notes to aid in studying. Some students have a hard time remembering their locker combinations. The PDAs have been a great help to these students. With reminders, students don’t forget their assignments. PDAs with keyboards help students who can’t write legibly allowing then to review their notes and understand them. For the students who have a hard time remembering their combinations, the PDA again comes to their rescue. The ironic thing is that the PDA is able to replace much of the materials that would be stored in the locker (Palm Inc., 2003). With the benefits of PDAs being realized there are many third party vendors writing software that will increase academic productivity and help students. One example is Kaplan’s software to help prepare for the SAT and ACT. Applications can be found that facilitate the tracking of grades and assignments. The Internet is full of success stories in K-12 education. Many educators want this information spread to all teachers so that their students can see the benefits of this technology. Some of the best sites include palm.com/education, pdaed.com, handheldeduca-tion.com, and educators.palm.com. When PDAs were first introduced they were priced beyond the limits of most students. Because of this, many handheld computer manufacturers made large grants to K-12 schools that have resulted in the success stories found in the literature. With the advance in technology and steadily dropping prices, PDAs have moved from being a corporate toy to an everyday tool. Along with the move to digitize planners came the idea to do the same with books. As technology has advanced, e-books have emerged as a less expensive alternative to paper books. A few companies like Wizeup and Metatext are working with publishers to bring big name books to the digital world. Experts believe that the first place in which e-books will have a substantial impact is education. Digital textbooks are becoming more and more popular among students and teachers. Imagine having textbooks that talked about events that happened only a month or two ago. Now imagine that they talked about a major event that happened yesterday. This is one of the greatest benefits to digital textbooks. Instead of having a five or even ten year old history book, students could be learning about current events and discussing them. With the widespread availability of the Internet, these books could also contain links to other sources of information such as magazine or newspaper articles. Instead of just reading about Martin Luther King Jr., the things he did and the speeches he made, students could listen to or even watch one of his speeches. They could watch a video of Neil Armstrong taking the first steps on the moon or witness the events that took place on September 11, 2001; and all this in the palm of their hand. A student that was sick for a few days could download the lessons that were missed so as not to get behind. “For many professors, the flexibility of e-books is the quality mentioned most. The electronic version can be updated more frequently than printed editions, and can also be linked to newspaper articles and other supplementary texts, as well as audio and visual aids. “The text is no longer frozen on the page,” says Dr. Sanford Berg of the University of Florida. He adds: “E-books will change the way classes are taught because students will have so much information. They make professors more a guide than a pontificator, which professors should never be anyway.” (An eBook in Every Schoolbag?) Financial Feasibility of PDAs and E-Books in K-12 The educational benefits of this kind of technology are astounding and something that we believe any student or teacher would love to take part in. The problem lies on the financial side. How do we go about getting PDAs in the hands of all the students and teachers and how are things like digital textbooks, lessons, etc. distributed and maintained? Fist off is the initial cost of the PDA. The price of handhelds ranges from about $100 to $700. As previously indicated, a few years ago several large companies in this industry, such as Palm and Handspring, issued grants or giveaways to schools willing to put this technology to use. The closer the idea of using PDAs in education came to becoming reality, the less generous these companies have been. In fact, as of recent, Palm and Handspring are no longer offering grants. Palm does have an Education Purchase Program where schools can receive as many as 72 units for free. In order to receive the 72 free devices the school must purchase 500 of them at the retail price. The other option is to require that parents purchase the PDAs. If schools were to purchase all of the units, then obviously every student would have one. Probably the most important benefit would be compatibility. Every device would be of the same make with the same specifications. This would help not only with ordering software like digital textbooks but also with their distribution. There are companies that offer a multiple cradle setup so that you can sync several PDAs at the same time. One of the drawbacks of having PDAs provided by schools is, of course, the amount of money that would be drained from the schools budget. In purchasing such large quantities it would also be nearly impossible to keep the equipment up-to-date in order to run the latest software. One the other hand, if parents were to purchase the handhelds for their children, the PDAs could be used for more than just schoolwork. This would allow the children to really get to know how to use their handheld which in turn would help them feel more comfortable using it to its full potential to do schoolwork. We believe that if students owned the PDA, they would also take better care of it which would eliminate the repair or replacement costs that schools would face. There are two drawbacks that we have found for placing the responsibility of providing handhelds on the parents. The most obvious one is the expense. Parents with several children in K-12 may not be able to afford devices for each of them. The second and probably the biggest disadvantage is the lack of compatibility that would unavoidably occur. It is much more difficult and costly to supply software to ten different types of PDAs than to just one. Once every student has access to a PDA there is still the problem of purchasing the digital textbooks. The implementation of digital textbooks is something that is still being worked out between publishers and online providers. As of right now the cost of an e-book is about the same as the used price of that book. As we mentioned earlier, one of the biggest advantages of having a digital textbook is having up-to-date material. Publishers have not figured out how to go about keeping books current other than just requiring the user to purchase the newer edition when it comes out. Obviously this does not take full advantage of having a digitized book. The type of infrastructure needed to implement this type of system really depends on who owns the PDAs. There are companies like K12Handhelds that will provide a classroom of up to 33 students with everything but the e-books. The kit includes Palm PDAs for every student and one for the teacher as well as a Secure Digital expansion card and software that allows every handheld to be setup with learning material using that card. The entire kit is $6,300. They also offer other useful products such as a multi-unit cradle that allows the simultaneous charging and syncing of up to six handhelds. The cost of the cradle is $699 plus an additional $499 for the management software. If the PDAs are owned by the students there is a much different and less expensive route to take. This method has been put to work in at least seven different dental schools around the country. “Each incoming student will be asked to purchase a DVD containing the entire curriculum -- textbooks, manuals and lecture slides -- for all four years of dental school. Each semester, students will trade the old DVD for an updated version. Creators of the technology estimate that DVDs, each weighing less than an ounce, will replace more than 2 million pages, thousands of images and more than 400 pounds of books and manuals (Guernsey).” The cost of the DVD is equal to the cost of buying all of the books it contains and is paid for over time. However, the DVD includes extra material including audio and video and the ability to do things like search a textbook or several textbooks for a word or phrase. Since this has only been done in higher education it is hard to say how well a system like this would work in K-12. Obviously it would be on a much smaller scale such as one semester worth of material which could easily be stored on a PDA. Handheld devices and digital textbooks have come a long way in the past few years. The technology is there for use in the K-12 arena and we believe that they would be very beneficial. However, there are still several things that must be worked out before any school could expect the use of PDAs in every classroom. Most importantly, publishers must devise a way to distribute digital textbooks and update them regularly. Also, because there is little to no overhead to print a digital text, the initial cost needs to come down. PDAs are much less expensive than they have been in the past but there must be a significant additional drop in price before we will see them abundantly at the K-12 level. In addition to funding, additional questions remained in our minds about the effective use of PDAs in education. The following section describes these questions and indicates our approach to obtaining their answers. Research Study: Questions, Methods, and Data Collection Mechanisms The literature review above has shown that PDAs have been used successfully in K-12 education. We were, however, unable to find any research on the effectiveness of using e-books. This suggested the following research questions for our further study. Is there any difference in learning retention and comprehension between traditional printed text and e-text? Are there differences between various educational levels (Jr. High, High School, College) in the use of e-text? And finally, what are students’ perceptions of the effectiveness of PDAs in education and how do they differ between levels of education? To help find answers to these questions, we utilized quantitative research methods, both experimental and descriptive. We performed experiments and collected data by administering tests and surveys. As our laboratory we selected classes, based on our acquaintance with teachers, at a Middle School and a Sr. High School in the area, and our College class. Students in each class were randomly assigned to one of two groups (independent variable), namely PDA readers and paper readers. Each group was given a set time to read a particular article selected by the teacher. We then tested their retention rates by administering a test (dependent variable), again written by the teacher, to the two groups. In addition to collecting data by administering a test, a follow-up survey (Appendix A) was given to all PDA users. The intent of the survey was to allow us to make judgments about how the students felt about using the PDA. The survey was the same for all PDA groups in all classes. The three middle school classes were all history classes, and their article dealt with Lewis & Clark. The high school class was a finance class, and their article dealt with inappropriate activities by Enron executives. The college class was an information technology class, and their article dealt with an educational technology called MUDs and MOOs. The reading of the articles and the testing of their retention were performed at the students’ classroom, and in their normal seat. All the teachers were present and helped set-up and perform the experimentation. Human Observations All the students in both the middle school and the high school were very excited to be able to use and learn about PDAs. Those that were randomly selected to read the article on paper were extremely disappointed that they were not chosen to use the PDA. One student in the high school class kept asking if we could set up his entire class to use PDAs for a year as a school experiment. Many of the middle school students did not want to give the PDA back. They also wanted us to set up their whole class with PDAs. We asked all the students how they thought that PDAs could be useful in their education. We were very surprised at how creative they were in coming up with ideas on how using PDAs could improve education. Some students thought that the use of PDAs would save the school district money. Others felt that they would save lots of time and paper (One middle school student even said that he thought using PDAs would save millions of trees). As we were walking out to our car, a student approached us and asked us again what he could do to keep one of the PDAs. He was carrying a baseball mitt. We told him “Bet you wouldn’t give us your mitt for a PDA.” He looked at the mitt for about three seconds and then said “Yeah, I’ll trade the mitt for a PDA.” Results of our Experiment Collecting and gathering data for the test scores and the surveys for the three schools was the easy part. The difficult part was organizing the data in a way that would prove or disprove our hypothesis. To accomplish this we utilized Excel for data storage and analysis. Table 1 summarizes the basic statistics for the exam scores from the three schools. Overall, there was very little difference in the mean scores between the PDA readers and the paper readers. The college class was the only one where the mean score for PDA readers was higher than that for paper readers, although the medians were equal. To test for statistically significant differences we ran several t-tests assuming equal variances and again assuming unequal variances. In all the tests there were no statistically significant differences indicated at the alpha=.05 level. These results supported our hypothesis that there is no difference in reading comprehension and retention between using e-text on a PDA and using a traditional paper book. School Statistic PDA Readers Paper Readers Middle School (three classes combined) Mean 88.3 88.8 Median 91 91 Variance 266.7 156.8 Skewness -3.05 -.98 Count 40 32 Senior High Mean 67.1 69.2 Median 65 70 Variance 268.1 153.8 Skewness .41 .52 Count 14 12 College Mean 84.2 76.7 Median 80 80 Variance 171.9 500 Skewness .22 1.10 Count 12 9 Table 1: Basic Statistics for Exam Scores Survey Findings In addition to the exam scores, we also collected data using a survey (Appendix A) for PDA readers at the three schools. The purpose of the survey was to help us learn more about the students’ feelings and experience while using a PDA and to assess any differences between the various educational levels. The questions dealt with the ease of using and learning with the PDA, the likelihood of PDA problems in education, and the value of various PDA software options. The following are summaries of several of the questions that show the most interesting results. First, we looked at the survey question, “Easy of reading the document.” Figure 1 shows a comparison across the educational levels of students responding that using e-text on a PDA was either “easy” or “very easy.” Notice that the middle school at 95 percent and high school at 92 percent both agreed that documents was easy to read on the PDA. On the other hand, only 40 percent of the college students felt that it was easy to read the PDA documents and 60 percent split evenly between difficult and neither. Figure 1 – Ease of Reading E-text on a PDA We next looked at the questions, “Ease of Remembering and Understanding what you read.” Figure 2 shows the percentages of students answering “easy” or “very easy” to the two questions. Middle school and high school students responded approximately the same. Once again the college students felt less sure of there understanding and remembering what they read on the PDA. This is undoubtedly caused by their difficulties in reading the e-text as indicated in Figure 1. Figure 2: Ease of Understanding and Remembering E-text on a PDA Following this we turned our attention to the questions “How likely can you learn as well with a PDA as with a textbook?” and “How likely will the PDA replace textbooks in education?” Figure 3 shows the percentage of students responding either likely or very likely to each of these questions. Notice that both middle school at 96 percent and high school at 92 percent feel that they could learn as well using a PDA as with textbooks. The college students, however, disagree with only 30% favoring the PDA. The contrast is even greater for the thought of PDAs replacing textbooks. The middle school at 93 percent and high school at 85 percent both agree that in the future there is a place for PDAs in the school system. The college students felt drastically different with only 10 percent of college students believing that PDAs will replace textbooks. Figure 3: E-text on PDA vs. Textbooks Finally, we turned to the issue of PDAs impeding the learning process. Figure 4 shows the percentage of “likely” and “very likely” responses to the question “How likely are games and the Internet on a PDA to impede your learning?” While only 30 percent of middle and high school students perceived a problem, 80 percent of the college class did. It should be noted, however, that the college class had been using PDAs for several months and could comment first-hand on the potential distraction of games and Internet browsing. This may also be due to the more highly structured environment in K-12. Figure 4: Likelihood of PDA Games and Internet Impeding Learning Conclusions of Student Research Project After performing our experiments and statistical analysis of the data, we discovered there is no difference in retention rates in reading from a PDA compared to reading from paper across all educational levels studied. These results would indicate that a PDA with e-books can be a useful tool in education. However, we also believe that in order for PDAs to be widely utilized in education, the price of the PDA must decrease. If PDAs follow the same course as all other computer devices, their price will decrease, and they will find their way into becoming an efficient and helpful tool in educational environments. On the other hand, the results of our survey data showed major roadblocks to the successful use of PDAs and e-books in college. College students found it difficult to read text on a PDA’s small screen and were less confident in their ability to comprehend and remember content read form a PDA. We postulate that this might be due to deteriorating eyesight or the fact that college students are too accustomed to reading from larger computer screens. In addition, the survey data suggests that college students are more prone to distractions from schoolwork such as game playing or Internet surfing on the PDA. These factors suggest that the successful use of PDAs by college students will be limited. Also, we realize that middle school and high school students are more excited about technology and may be more willing to adapt to change while college students are more set is their ways and have more difficulties with change. 3. COLLEGE STUDENTS’ EXPERIENCES USING A PDA A class of 25 IS students were given a Dell PDA with wireless network card to use during the Spring-2003 term. A limited number of keyboards were also available for students to share. Following a discussion of current PDA educational usage reported in the literature, students were asked to use the equipment during the term to help with their course work as well as personal use. For the most part students were left to experiment with the device on their own with few required PDA assignments. At the end of the term, students were given a brief survey to help quantify their experiences. This section reports the results of the survey. Overall, only 36 percent of the students agreed or strongly agreed that the PDA was a valuable tool in helping with their coursework. Forty percent were neutral. Those that disagreed were students that had only minimally used the equipment or were already using a laptop computer. The typical PDA functions of scheduling, contact lists, and to do lists were sited as most beneficial. Only two of the students used the PDA for taking notes. Four students indicated that the downloading of course content and its availability electronically was an important factor. The biggest drawback cited (36 percent) was the small screen size. This was followed by difficulties in input using the stylist (20 percent). Students were divided on the value of the wireless network. Fifty-six percent indicated that wireless access was important; however a nearly equal number (44 percent) indicated that the wireless network had little value to them. Problems cited included web browsing on the small screen (most web sites are not PDA enabled), and problems with email. While portability of the device was cited as important (25 percent), students also indicated that carrying and caring for the equipment was a concern (20 percent). Most of the students did enjoy experimenting with the PDA (Agree: 72 percent; Strongly Agree: 16 percent, Neutral: 12 percent), however when asked if they planned to use a PDA in their future schooling or work only 52 percent agreed or strongly agreed and 48 percent disagreed or were neutral. Finally, when asked if all faculty and students at the college should be required to use a PDA 32 percent disagreed or strongly disagreed, 52 percent were neutral and only 16 percent agreed. 4. SUMMARY This paper has explored the use handheld computers or PDAs in education. The literature abounds in anecdotal examples of the successful use of PDAs in K-12 education, mostly generated by the donation of equipment by the PDA vendors. Financial concerns may limit the wide-spread duplication of these success stories in K-12 education. The success of PDAs in higher education, however, is yet to be determined. Although data sets were small and generalizations may be limited, faculty and student research described in this paper suggest that there are serious roadblocks to a successful college career for PDAs. While experiments show that there is no statistical difference in the comprehension and retention of content read from paper documents vs. from e-text on a PDA across educational levels, survey data suggests that college students have a harder time reading e-content on a PDA than do middle or high schools students. Furthermore, college students are more prone to be distracted from schoolwork by the use of PDA games and Internet surfing than K-12 students. Overall, college students are much more pessimistic about the successful use of PDAs than are K-12 students. While the limitations of this research are recognized, the results do suggest barriers to the successful use of PDAs at the college level. These, in turn, generate more questions and open additional areas for more detailed research. 5. REFERENCES An eBook in Every Schoolbag? Retrieved March 16, 2003, from http://www.dclab.com/ebooktextbooks.asp District 230 Students Make Use of Handheld Computers. Retrieved March 28, 2003 from http://www.pdaed.com/features/district230.xml by Darrell Walery Ebooks in the Classroom. Retrieved March 15, 2003, from http://www.publishing.about.com/library/ weekly/aa072200a.htm Guernsey, Lisa. (2000). Bookbag of the Future. The New York Times. Retrieved March 5, 2003, from http://www.indiana.edu/~aainfo/FurtherReading/InfoGeneral/NYT/Bookbag%20of%20the%20Future.htm Johnson, D.W., C.G. Jones, and S.J. Cold (2002). Handheld computers: Ready for prime-time in the college classroom? In the refereed Proceedings of the 2002 Americas Conference on Information Systems (AMCIS), Dallas, TX. Association for Information Systems. Jones, C.G., D.W. Johnson, and S.J. Cold (2002). M-education: Mobile computing enters the classroom. Issues in Information Systems, 3, 309-315. Marysville High School. Retrieved March 28, 2003 from http://www.palm.com/education/studies/ study3.html Mayfield, Kendra. (2001, Jan.). E-Book Forecast: Cloudy. Wired News. Retrieved from http://www.wired.com/news/culture/0,1284,40984,00.html NorthStar Middle School. Retrieved March 28, 2003 from http://www.palm.com/education/studies/ study1.html Print-On-Demand & Digital Textbooks Will Thrive. Retrieved March 15, 2003, from http://retailindustry.about.com/library/weekly/aa001222a.htm Ruley, John D. (2002, Oct.). PDAs in Education. Mobile & Wireless Solutions. Retrieved from http://www.mobile-and-wireless.com/Articles/ Index.cfm?ArticleID=26881 Textbook Publishers: Who’s Afraid of eBooks? Retrieved March 16, 2003, from http://www.dclab.com/textbooks.asp Appendix A Please complete the following questionnaire regarding the use of the PDA (Personal Digital Assistant) in education. Place an X in your column of choice. Rate the ease of these activities Very Easy Easy Neither Difficult Very Difficult Ease of accessing the document           Easy of reading the document           Ease of scrolling up and down the document           Ease of understanding the material           Ease of remembering what you read                                   Rate the likeliness of these occurrences Very Likely Likely Neither Unlikely Very Unlikely How likely will PDA's replace textbooks in education?           How likely can you learn as well from a PDA as with a textbook?           How likely could the limited battery life impede your education?           How likely are you to drop and break your PDA?           How likely are games and Internet on PDA to impede your learning?                                   Which tools on PDA do you think will help in your education? Very Likely Likely Neither Unlikely Very Unlikely Accessing E-mail           Beaming to team members           Playing games           Internet access           Word processing           Spreadsheets           Downloading from the Internet                                   Which tools on PDA do you think will impede your education? Very Likely Likely Neither Unlikely Very Unlikely Accessing E-mail           Beaming to team members           Playing games           Internet access           Word processing           Spreadsheets           Downloading from the Internet