Service-Learning Impact on IS Students in a Web Development Course Ruth A. Guthrie California State Polytechnic University Pomona Pomona, CA 91768, USA and Carlos J. Navarrete California State Polytechnic University Pomona Pomona, CA 91768, USA Abstract This paper describes the impact of Service-Learning (S-L) on the students of a Web development course. S-L requires that students perform a community-based project that allows them to apply the knowledge learned in the course to a real world situation. Proponents of experiential learning claim that deeper contextualized learning takes place because students can practice skills that are hard to simulate in a classroom. However, S-L projects can also be time-consuming and add more work for instructors and students. The findings of this paper indicate that after the S-L project, Computer Information Systems students’ attitudes changed regarding perceptions of clients and enjoyment of the project. Students also demonstrated that they developed communication skills and knowledge of the systems development life cycle. Keywords: service-learning, active learning, IS education, Web design 1. INTRODUCTION Service-Learning (S-L) requires students to apply course knowledge to solve a problem or provide a service in their local community. S-L does not require that students learn extra or less material for the course. It requires that courses be modified to involve field-type assignments and projects in unison with the already established course content. Professors who incorporate S-L into their classes report many benefits for students taking these courses. In many disciplines, an increased academic emphasis is being placed on student learning in the community of which the university is a part. Not only are universities trying to impart to students that they are integral members of society and should altruistically participate in their communities, but universities also seek to graduate students with field experience in their major. For Computer Information Systems (CIS) students this experience is highly beneficial. Information Systems is a field plagued with complaints about poor service and professionals who are poor communicators and non-responsive to user needs. Often classroom focus is on the details of solving a technical problem and not on managing the political problems one might encounter in a development environment. Problems that CIS students solve in the classroom have fictitious, or in-house clients, that already have a lot of computer knowledge. Typically, CIS graduates have little skill or experience working with a real client and display immature communication skills in their professional work. S-L gives students the opportunity to develop critical communication skills in the context of learning practical and theoretical details about technology. This paper discusses the impact of adopting a S-L strategy in an introductory web design course. Students developed web pages for elementary school teachers over a ten-week class. The data was collected from pre-course and post-course student essays and analyzed according to a specific rating system. However, the analysis in this paper is purely qualitative. More direct measures and controls are needed to perform a quantitative study. 2. LITERATURE REVIEW Several aspects are important to the adoption of S-L projects in higher education programs. This section presents S-L definitions reported in non-IS and IS literature, necessary criteria that courses taught with S-L should fulfill, examples of courses from different disciplines taught under S-L, previous experiences of S-L in information systems courses, and S-L benefits for information systems students. There are several entities embedded in service learning projects: The community, students, teachers, and the academic program. From an educational perspective, Bringle and Hatcher (1996) define S-L as a form of education that promotes learning experiences for students. During these experiences students develop a project for a not-for-profit organization. A strong S-L condition for this project is that it should be directly related to the learning objectives of the course. Similarly, Jacoby (1996) states “service-learning is a form of experiential education in which students engage in activities that address human and community needs together with structured opportunities intentionally designed to promote student learning and development(p. 5)”. More oriented to the embedded process, Furco’s (1996) definition looks at S-L programs as educational experiences promoted by faculty and a community entity that acts as client/sponsor. The service provided under the program enhances student learning, and the student course learning enhances the service provided. Different community based activities can be misinterpreted as S-L projects. According to the Michigan Journal of Community Service Learning (MJCSL), S-L is different from student community service, student internships, or an addition of community service to a traditional course. These forms of service miss two important components of S-L: A learning dimension linked to an academic course and purposeful civic learning, which prepares students to adopt an active civic role. According to the MJCSL, S-L should fulfill three criteria: “Relevant and meaningful service to the community,” “enhanced academic learning,” and “purposeful civic learning (p. 12).” Multidisciplinary adoption examples include an MBA business policy course at Bringham Young University, where students apply general management skills to the challenges of a local food bank in systematizing its policies for food collection (Godfrey, 1999). In a seminar course on the psychology of organizational effectiveness and change, organizational psychology Ph.D. students and human resources students worked with small businesses and not-for-profit organizations. The purpose of this seminar was to study barriers to organizational morale and inter-organizational relationships. Specifically, the students assessed limitations to organizational effectiveness and designed an intervention plan to enhance the productivity of the organizations that took part in the project (Thomas and Landau, 2002). In an MBA leadership course, students developed leadership skills by participating in several community service projects that developed leadership skills (Godfrey, 1999). In the case of information systems programs, several authors report the potential for S-L in enhancing IS student learning experiences. Ruppei and Ruppei (2002) describe how information systems students worked with the community on a Local Area Network and a web design project. The LAN project included writing a grant proposal, wiring a building, and setting an Ethernet LAN using Category 5 unshielded twisted pair. In the case of the web design project, students developed a web page for their school. The main point of the project was to produce a web page for faculty that they could maintain by themselves (Lazar and Lidtke, 2002). Adopting S-L strategies benefits students in several ways. In general, S-L helps students to become responsible citizens and to enhance their technical capabilities (Godfrey, 1999). In the case of technical capabilities, IS students can find S-L settings unique for acquiring critical professional skills. In S-L studies done in IS courses, students were found to appreciate being part of a project that helped others and liked being treated with respect by the clients (Rupei and Rupei, 2002). Another IS related study (Lazar and Lidtke, 2002) reported that students gain better understanding of user issues working in an S-L environment, get exposure to ethical and political issues that occur in a work environment, gain experience with the systems development life cycle and an appreciation for other activities (not just coding) of systems development. 3. RESEARCH QUESTIONS AND METHODOLOGY Several broad research questions were posed for this study: Q1. What is the impact of S-L on the perceptions of students about working with a client? Q2. Do students develop “soft” skills necessary for working with clients? Q3. Do students develop an understanding of issues related to the System Development Life Cycle? Q4. Can students draw analogies from this experience to future work? Q5. What is the impact of S-L on the perceptions of students about working in the community and their greater role in society? Project description The elementary school used in this study was selected because the principal had an interest in exposing her teachers to Web technology and in working with Cal Poly Pomona. The principal asked teachers to volunteer to work with a student in developing a home page for their class. Teachers represented all grade levels, including special instruction such speech and language. The teachers had a range of technological ability, some knowing very little about the Internet and how to use it in the classroom and others knowing a great deal. Twenty-two students in a web design course were assigned to assist eighteen teachers at a local elementary school in developing web sites for their classrooms. Since students outnumbered teachers, eight students worked in self-selected groups of two so that each teacher had one or two students randomly assigned to him or her. The web design course is a junior level course, with a ten-week duration, covering HTML, JavaScript and some ASP. Students in this course have already taken a survey course in IS, programming, systems analysis and design and a database course. This class had 16 men and 6 women. Twelve of the men and 4 of the women had prior community service experience. During the term, students had deliverables associated with their project: * An essay on perception of community service. * An initial client-student meeting to gather requirements. * The requirements document and schedule. * A web page prototype and client feedback. * The final web page. * A teacher training session. * An essay on perception of course experience with Service Learning. The web page was a graded assignment for the course. The course varied from the traditional course in that students did the S-L assignment instead of developing their own home page. As part of the course, students were required to write pre and post essays about their perceptions of service learning. The pre-essay included any prior experience they may have had and the post essay required them to reflect about their experience in this course. A measurement instrument, shown in Table 1, was developed with consideration as to what learning outcomes are desirable from an S-L based course. Two knowledgeable IT professionals tested the instrument. The graders read the before and after essays by four different students and evaluated them using the form. Results were coded on a scale of 1 to 5, 1 being negative. A few changes were made to clarify specific questions where interpretation was unclear. Table 1 Service Learning CIS Measurement Instrument Before/After SL Project Question Negative Somewhat Negative N/A Somewhat Positive Positive 1. Perception of Service Learning 2. Perception of Client 3. Learning Value of Experience 4. Enjoyment of Experience (Fun) 5. Shows Evidence of ‘soft’ skills for an IT professional 6. Shows Evidence of technical skill 7. Shows Evidence of SDLC Knowledge 8. Demonstrates Problem Solving Capability 9. Draws analogy to future work 10. Shows awareness of one’s greater role in the community At the conclusion of the course, the “before” and “after” essays were evaluated by the two researchers. When a disagreement occurred, the difference was discussed and a mutually acceptable rating was agreed upon. Aggregate and individual ratings were placed in an Excel spreadsheet for comparison. While the instrument gives us a way to quantify the essays, this is still a qualitative study. Interpreting student essays is a subjective, broad way to analyze the results. The quantitative values presented in the paper are intended to provide guidance for areas of future study. 4. ANALYSIS OF RESULTS Table 2 shows the Mean Rating of the student essays before and after the S-L project. Perception of Clients Q1: What is the impact of S-L on the perceptions of students about working with a client? Results from the before and after essays indicated that students’ perceptions of the client (mean rating: 3.95 before, 2.77 after) and enjoyment (mean rating: 4.05 before, 3.09 after) of the experience declined after taking the course. Though most students had previous rewarding experiences with Table 2 Mean Rating Before and After Service Learning Question Mean Response Before S-L Mean Response After S-L 1. Perception of Service Learning 4.57 4.36 2. Perception of Client 3.95 2.77 3. Learning Value of Experience 4.38 4.18 4. Enjoyment of Experience (Fun) 4.05 3.09 5. Shows Evidence of ‘soft’ skills for an IT professional 3.38 4.32 6. Shows Evidence of technical skill 3.10 3.41 7. Shows Evidence of SDLC Knowledge 3.05 4.00 8. Demonstrates Problem Solving Capability 3.76 3.73 9. Draws analogy to future work 4.24 4.68 10. Shows awareness of one’s greater role in the community 4.57 3.86 community service, in developing a web page for a client their perception was more negative, and their enjoyment of the project was less. The S-L project was different than their community service projects of the past. If you repair a house for someone or work to donate food for the homeless, it does not require that you actually work with a person. You can feel fulfillment without ever meeting the person you are helping. The S-L project required that students talk to, define and implement a client’s requirements. Students were very good at repeating what they had learned in previous course work, about listening to the client and working with people without technological experience. However, when they had to apply what they had talked about, they were ill equipped to do so. Table 3 shows some comments students made before and after the S-L project. Before, they had a willingness to work with the client and a respect for the role of teachers in society. After working with the teachers, who knew little about Web technology and were very busy, students descriptions indicated a perception of the client as ‘incompetent’, ‘not meeting their deadlines’ and ‘uncooperative’. What better lesson for an upcoming IT professional? In a sense, the comments of these students reflect what many would say are the problems of IT professionals in industry. Soft-Skills Q2: Do students develop ‘soft’ skills necessary for working with clients? The before and after rating of ‘soft’ skills for an IT professional show an increase in this skill (mean rating: before 3.38, after 4.32). After going through the process of contacting, interviewing and exchanging a prototype with the teacher, students indicated greater awareness about communicating with clients with little technical ability and undefined project goals. Students were faced with the age-old client requirement of “I don’t know what I like but, I’ll know it when I see it.” Students also indicated the frustration of setting up meetings and transaction costs associated with product development, a large part of their future careers. One student reflected, “It teaches you that you’re not the only one that has things to do. This assignment should have been called time schedule learning.” Life-Cycle Skills Q3: Do students develop an understanding of issues related to the System Development Life Cycle? Three areas on the rating form addressed skills associated with system development skills. In the evidence of technical skills (mean rating: before 3.10, after 3.41) and the demonstration of problem solving ability (mean rating: before 3.76, after 3.73), little change was recorded. In the essays however, students were not asked to specifically talk about their technical skills and most reflected on their social rather Table 3 Selected Student Comments Before After “…service can become important community contacts that can help you later in your career…used in your resume...” “I expect to gain …seeing the look of joy on the children’s faces when they get to see their pictures on the Internet and have something they can share with their friends and family.” “I expect to gain an even greater respect for elementary school teachers. I know it takes a lot of patience to handle children. I expect to see things through the teacher’s perspective from teaching techniques to creativity.” “To bring the user closer to technology without having to go into technological details.” “…enhance my creativity, communication skills and knowledge of customer’s expectations.” “…benefited by having school class information available at their fingertips day or night.” “…help children have a better learning experience, and have their parents become more active in their child’s daily lives and experiences at school.” “..Learning how to encourage children and be able to teach things beneficial.” “The only complaint I had about this project was the fact that my client did not cooperate with me throughout the mission in completing this project." “A Web site is something that every elementary school teacher needs in order for them and their students to become aware of technology.” “..I don’t think I gained any important community contacts that could help later in my career. I would never want to be hired by a school district to do Web design. I did find the work fun and interesting and emotionally rewarding.” “..regardless of how hard you try, it is often hard to get cooperation from the client.” “The first deadline that I set for Mrs. Joiner came and went without a response…Maybe we should all just throw the professionalism out the window and show up in front of their classroom door.” “My client was not helpful with feedback and was difficult to get in contact with….overall it was a good experience. I gained knowledge of how to deal with unwilling clients, which I am positive I will meet in the real world.” “I learned that as a developer you have to schedule around your clients.” “I learned that you must actively seek additional information from the client. …clients do not know what they really want until after you start on the project and they can be quite indecisive when figuring out what they want.” “Mrs. D was technologically challenged.” “I found out that it is very hard to work with clients in the real world….people who were incompetent with computers.” “It teaches you that you’re not the only one that has things to do. This assignment should have been called time schedule learning.” “Unfortunately, because we were assigned such a busy and incompetent teacher, doing this project did not meet my expectations.” “Instead of the praise that I thought I would be receiving, all that I got was negative feedback on how the Website was not colorful enough.” than technical experience. In their description of problem solving capability, both before and after essays were the same. Though, in previous community service the solution to the problem, while they participated in implementing it, it was not developed by them. For the S-L project, they actively had to solve technical and communication problems. The rating of SDLC knowledge (mean rating: before 3.05, after 4.00) indicates growth in student awareness of SDLC relationships and processes. Though, this measure is misleading because students were not specifically asked to discuss the SDLC in the first essay. The results seem strong for all these indicators, though a less open-ended instrument needs to be developed to make the comparison clear. Future Work in Industry and Society Q4: Can students draw analogies from this experience to future work? In relating this experience to future work, only a small increase was recorded (mean rating: before 4.24, after 4.68). Though, both before and after measures are high, indicating that community service and S-L both have a positive experiential value to the students’ future job. Student perception of their greater role in society was rated as decreasing (mean rating: before 4.57, after 3.86). This may be due to the nature of the project. Building a home for someone or working in a soup line can directly have an impact on someone’s well being. Development of a Web page, while it may have long range implications for technology in the classroom or lessening the digital divide, is not a necessity to survival. As such, students did not talk about the humbling or fulfilling experience that previous community service gave them. Learning Experience Q5: What is the impact of S-L on the perceptions of students about working in the community and their greater role in society? Ratings indicate student descriptions about perceptions of S-L (mean rating: before 4.57, after 4.36) and learning value (mean rating: before 4.38, after 4.18) of the experience remain similar. Both before and after measures indicate very high. Prior to the project, students indicated the benefit of community service as being very positive. After the S-L project, though with some frustration, students still perceived S-L and the learning value as somewhat positive. Other Outcomes Three students failed the course because they could not meet project deadlines, largely because of failing to meet with their client. This is somewhat alarming, considering they are far along in their choice of major and may not be suited to an IS career. Time-management was an important learning experience for students. Students are very well trained to read a book or a case, solve a concise, well defined problem that takes a finite amount of time and requires little decision making. The S-L project was highly undefined, required lot of decision making on the part of the student and did not have answers scripted in the review section of the text book. The process of developing the Web page for a busy client was much different and much more realistic than developing canned Web assignments isolated to the classroom. Ethics, particularly copyright law, became an issue in four of the eighteen Web sites developed. On one site, the teacher wanted the student to convert Elvis Presley’s classic, “You Ain’t Nothing But a Hound Dog” to an MP3 and have it play continuously on his site. The student, aware of music piracy issues, negotiated with the teacher until he agreed it was illegal admitting, “OK, I wouldn’t want Cal Poly to get into trouble.” In another instance, a speech teacher wanted to use Looney Tune characters on her site because so many of them have speech impediments. The students were caught between the professor saying “no copied materials” and the teacher saying, “I use this in my class all the time.” Students showed the fair use policy on the Warner Brothers Web site to the teacher and even emailed Warner Brothers to ask permission. The teacher countered with a URL of a site in Europe with Warner Brothers characters and a statement saying copy anything you like. They did not, ending up with a dissatisfied client and a wealth of practical negotiating experience. Elementary school teachers were very enamored with bells and whistles, regardless of functionality. Over the term, design rules like alignment, proximity, and simplicity were taught to the students. Once they interviewed teachers and prototyped pages, they realized that animated gifs and cartoonish clip art was very well received. Once teachers saw/heard a page with continuously playing sound (e.g. the alphabet song), all of them wanted sound. Overall they like busy, brightly colored pages that did not necessarily adhere to the design rules advocated in the classroom. Students had to balance their knowledge of design with client expectations. 5. CONCLUSION This study is a beginning investigation of the benefits of S-L to IS education. To measure the impact of S-L on student learning, an instrument was developed, tested, and used to evaluate student learning. This instrument specifically focused on assessing student experience with the software development life cycle, communication skills, technical skills, and community services awareness. The S-L experience gave students an opportunity to work with clients with little technical ability. Students successfully worked with a client, applying prototyping techniques and the SDLC process. Overall, students perceived S-L and its learning value as somewhat positive. These results have implications for students, teachers, and IS programs. Most students have the technical ability to solve problems, but they miss “soft” skills critical to succeed as an IS professionals. Imagine a student at his/her first job. That person is likely to make mistakes and struggle with clients who speak an entirely different domain language than IT. Now imagine a student who has participated in S-L projects with several real clients as part of his/her curriculum. This person enters industry as a wiser graduate. He or she has had experience negotiating and understanding another’s perspective, as opposed to simply reading a written perspective or listening to the perspective of a peer pretending to be a client. Another benefit of S-L projects is that they help teachers to implement an active learning strategy. With S-L, the learning focus is taken away from technical problem solving to context problem solving. While the technical solution still needs to be developed, the student has the opportunity to develop a solution in a real setting with a real client. Context problem solving allows the teacher to guide instead of present learning experiences in lectures. Lastly, IS programs can find in S-L projects an alternative to enhance students quality, to partner with community entities, and to get feedback on the quality of the program. Ideally, to measure the impact of S-L, it would be preferable to have a control group, developing pages in a class setting, that you could compare a S-L group to directly. Then, a concrete assessment of the costs and gains of S-L classrooms can truly be made. Another weakness of this study was the indirect measure of student attitudes towards clients and artifacts of the S-L process. In a study planned for the Fall of 2003, students will be asked to directly report on the research questions posed in this paper through a series of online journal entries. Instead of having an interpretation of what a student experienced, students will directly report on controlled topics related to their experience. As with other S-L studies, this project is limited by a small sample size. Twenty-two students’ experiences do not support the generalizability of the study’s findings. A larger sample of students would be beneficial. However, managing the students’ projects and the partner relationships limits the number of students and sections that the teacher can handle. Increasing the number of students in the S-L project would increase the workload for the teacher, which is a strong limitation for adopting S-L strategies. Teacher overload is an opportunity for future research on adopting S-L in IS courses. Specifically, it is important to find settings that facilitate achieving the S-L benefits without overworking the professor. Another proposed research project is to study two sections of the same course, investigating the levels of expertise of the non-S-L and S-L groups. Lastly, it would be informative to investigate if the students’ evaluations of the professor are higher in non-S-L or S-L sections. The use of technology in S-L is very different from traditional community service. Instead of feeding or helping a person in need of life-sustaining services, a web page is something teachers can live without. However, creating web pages for teachers influences them to think about how they might use the web to teach, communicate information or celebrate what students do in the classroom. Even if they use technology at a basic level, teachers and elementary school students need to be exposed to many types of technology. In the future, parents may receive school notices and permission slips over cellular phones. Students may attend literary discussions of books in classroom chat sessions. There is a lot of pressure for everyone to be proficient with technology. Not doing anything means no exposure to technology. Exposure at early ages means not being left behind. 6. REFERENCES Bringle, R. G., and J. A. Hatcher, (1996). “Implementing Service-Learning in Higher Education.” Journal of Higher Education. No. 67, pp 221-223. Furco, A., (1996). “Service-Learning: A Balance Approach to Experiential Education.” In the Corporation for National Service’s (Eds.) Expanding Boundaries: Service and Learning. p. 2-6. Godfrey, P. C., (1999).“Service-Learning and Management Education A Call to Action” Journal of Management Inquiry, Vol 8. No. 4. December, pp. 363-377. Howard, J., Ed., (2001). “Service-Learning Course Design Workbook.” Michigan Journal of Community Service Learning. University of Michigan. Summer. Jacoby, B., (1996). “Service-Learning in Today’s higher education.” In Jacoby & Associates (Eds.), Service-Learning in Higher Education. pp 3-25. Lazar, J. and D. Lidtke, (2002). “Service-Learning Partnerships in the Information Systems Curriculum.” In Lazar J. (Ed.), Managing IT/Community Partnerships in the 21st Century. (pp.1-16). Ruppei, D and C. Ruppei, (2002). “A University/Community Partnership to Build a K-8 School Network Infrastructure.” In Lazar J. (Ed.), Managing IT/Community Partnerships in the 21st Century. pp.52-68. Thomas, Kecia M. and Harriet Landau, (2002). “Organizational Development Students as Engaged Learner and in Teaching OD.” Organizational Development Journal. Vol. 20. No. 3. Fall, pp. 88-99.