IS 2002.10 - Project Management and Practice: Making It Work Louise L. Soe Computer Information Systems, Cal Poly Pomona Pomona, CA 91768, USA llsoe@csupomona.edu Abstract This paper discusses the critical success factors necessary for successful student projects in a class that is an instance of the Project Management and Practice course, which is part of the IS 2002 Model Curriculum. It is based on several years experience teaching the course. Keywords: IS20002.10, project management, practice, teamwork 1. INTRODUCTION This paper discusses a number of critical success factors that ensure the successful completion of student projects in a course that is an instance of the IS 2002 Model Curriculum: IS 2002.10 Project Management and Practice. The course is the required capstone class taught in a Computer Information Systems Department at a state university in California. The official description of the course reads: Application of software engineering concepts, principles, and practices in the development of computer-base information systems. The application of technical, managerial, communications, and interpersonal skills to a realistic business project in a team environment. Organizations that hire graduates of this program value the senior project experience, which is consistent with research findings (Weistroffer & Roland-Gasen, 1995). Students bring different levels and kinds of experience into the course. Students in the program specialize in four different career tracks: systems analysis and design; programming; Internet programming and security; and telecommunications. Students are eligible to take the course after they have completed three of the seven courses in their career track. Students typically enroll during the last or second-last quarter of their coursework. Most students have had an internship in IT or some other kind of work experience. A few students have had years of experience in the IT industry. However, many have not had to deal with a “real” customer in the development of an IT project. Some students enroll in the course together with several peers with whom they have already completed a team project. Others may not know anyone in the class. Student teams may be heterogeneous or homogenous, including students in one or several career tracks, same sex (usually male), all male with one woman, or more evenly divided. Table 1 shows the breakdown of enrollment statistics for the major, college, and university for Fall 2000. Almost half the students in the program are Asian, and 35% are women. Although students enrolled in this required course bring different skill sets, attitudes, and experiences, the course goal is to have all the students learn how to work on a team to successfully design and implement an IT project for a real customer. Table 1. Undergraduate Enrollment: Fall 2000 CIS College University Hispanic 11% 16% 23% African American 2% 5% 3% Asian 47% 37% 27% Pacific Islander 1% 0% 0% Caucasian 15% 24% 25% Filipino 8% 5% 2% Native American 0% 0% 0% Unknown 7% 12% 9% Women 35% 48% 45% Teams are responsible for discovering the project requirements, designing a solution, using project management software to monitor project progress, developing a solution, testing it, and integrating it into the customer’s environment. They also provide documentation and training to the customer. Customers include large corporations, school districts, police departments, university administrators and professors, and startups. Frequently students end up doing a project with a technology with which they have no experience, and learn new technologies and programming languages during the course of the project. Some projects are feasibility studies in which students research multiple solutions to a problem, but these always include a project deliverable with at least a prototype of the most feasible solution. When a new technology appears on the scene (e.g., Active Server Pages or XML), customers may request a project that demonstrates the potential of the technology for the customer In some cases, CIS faculty request projects demonstrating untried technologies that they may adopt in a class. Obviously, since students bring such different backgrounds into the class, and since all the students must pass the class to graduate, many potential problems can block the successful completion of the project. This class is fraught with uncertainties, for the students, the customers, and the instructor. What better preparation for a career in IT? 2. PATHWAY TO SUCCESS This rest of this paper discusses practical ways deal with these uncertainties when you organize a group project class. The author has taught the class for several years and developed a methodology that results in success for the students and the customer. The paper lays out typical problems and solutions that occur throughout the term, including the necessary pre-work, getting projects jump-started, the evolutionary prototyping development methodology, problems that occur during the course of the project, and ways to ensure a successful conclusion. It also discusses assessment, grading, customer feedback, and student peer evaluations. 3. PRE-TERM PREPARATIONS The students have 11 weeks from the beginning of the term to finish what may be a complex project. Thus, pre-term preparation is essential. The instructor needs to line up potential customers with projects of interest to students that are challenging but possible to complete during the term. If the project seems appropriate in size and scope, and the customer is interested and agrees to the terms of the class, I invite them to present their projects. There is no predicting student tastes for projects, and ones that look quite unappealing to me can be exciting to a particular group of students. Finding Projects How does one locate projects for student teams if one has never taught the course before? Frequently companies have interesting projects that they would like to have done, but that never make it to the top of the priority list for the employees in the company. The trick is to find them. Some projects walk in the front door through inquiries from the public. Alumni are an extremely good source of senior projects, since they understand the educational requirements of the program. They can be extremely good customers, especially if they had the senior project course as students. Faculty members provide projects, especially regarding technologies for courses, or applications for their own businesses. Administrators within the university have excellent projects. The students themselves may bring in projects from somewhere they work or someone they know. However, someone other than the student enrolled in the class has to be the project customer. Local business incubators also have projects. Other faculty members who have taught the course may have suggestions for sources of projects. Once the course is in full production, projects come in through word-of-mouth, and customers request additional projects. Once the projects are successful, you will end up with a glut of projects. Preparing the Customer One of the critical requirements for a successful project is a customer who is willing to understand, respect, and fulfill the needs of the students. The purpose of the class is to have the students learn, and it is the instructor’s responsibility to see that the customers understand the limits of working with a student team. Customers need to meet with the students regularly, give them feedback, information, and other resources they need, and respect the time constraints under which the projects take place. Once the quarter is over, the students often are finished with school and leave. This time requirement from the customer is one reason Ellen and West (2003) advise against using actual customers for this type of course. The customer is in fact one of the major risks. An effective way to keep customers on track is to remind them that they will have to pay the students for their work once the term is over. The instructor has to spell out clearly the contributions that the customer must make to the project. The customer is expected to provide any technology that the students might not normally have (e.g., special hardware or software), as well as information that the students need to complete the project. If the customer fails to respond to the student needs, the instructor has to step in and remind the customer of his or her obligations. Our program does not charge customers for projects for a number of reasons, including the institutional bureaucracy and overhead involved in doing so. However, customers are encouraged to donate to the department, and to pay for student costs, such as mileage for any trips they make to the customer site, and meals or snacks. Scoping the Projects Initial project scope is up to the instructor. The project has to be challenging enough to occupy a team of students for a quarter, but not so challenging that the students will not be able to complete it. Large projects may be divided into modules, each of which is completed by a team during the quarter. The author’s experience with projects that span two quarters and two teams has not been good. Typically, the students during the second quarter throw out the first team’s work and start over again. Project scoping continues to occur throughout the project, with the student team involved in the process. However, the instructor must continue to monitor project scope so that it does not grow either too large or too small to ensure the student team delivers a project that works. Preparing the Students Emailing the students enrolled in an up-and-coming senior project class gives them information about the course requirements (which are on the Web), and the proposed projects. Project descriptions together with customer contact information are posted on the Internet before the beginning of class. Students are encouraged to contact customers to inquire about their projects. Inviting the students enrolled in next term’s senior project course to this term’s final senior project presentations is helpful, because they can see the project results and sometimes meet potential customers. 4. EARLY IN THE COURSE One of the critical elements in finishing a project is to get momentum going right at the beginning of the project. Students are encouraged to pace themselves throughout the entire quarter so that they do not end up with an overwhelming amount of work during the last week. Forming Student Teams Usually the first class session is occupied with a discussion of class organization and structure, schedules, possible projects, team formation, team roles, and the instructor’s expectations. The instructor needs to be enthusiastic and confident about the results at the end of the quarter. Each student introduces him- or herself, and the session ends with a “cocktail party without the drinks,” in which students mingle and get to know one another. Students are encouraged to prepare resumes and bring them to the next class for their meetings with the customers. Senior project teams have a maximum of six members and usually a minimum of four members. The coordination costs for teams of more than six members are too high. Usually when a team has seven or more members, one of the members is a free rider. A team of fewer than four members may not be able to complete much work. Students are also encouraged to contact customers about their projects and to go after a project they really want. Customers are usually impressed with this type of assertiveness. The students are also warned that the project description the customer has prepared does not necessarily represent the project they will do if they sign up for it. In the best circumstances, students form teams around a project that interests them. Negotiations continue as teams try to attract members from unattached students or even from students on other loosely bonded teams. Matching Customers and Teams The customers are invited to the second class session in order to pitch their projects. Some time is left at the end of the class for customers and students to mingle together and interview one another. This system of letting the customers and teams choose one another is quite successful, since customers seem to hold teams in higher esteem if they have chosen them. It also prevents later complaints from the customer that they really wanted a better team. If the customer’s favorite team is unavailable, they can make a second choice, wait for another term, or visit another professor’s senior project class. Negotiations continue and usually result in customers and teams choosing one another by the end of this meeting. The students learn how to negotiate, and to deal with uncertainty. The team/customer matches made this way are quite successful. Team Formation Once the customers and teams have chosen one another, it is important to get them working on the project immediately, while enthusiasm is high. The student teams choose a name, design a logo, organize themselves, and develop business cards. They are encouraged to prepare a set of team norms or rules of behavior to which everyone on the team subscribes. One of the norms is “No free riding,” since the success of the project depends on participation by everyone on the team. The team members decide team roles for each team member. Each team needs a leader and a project manager to create and maintain the project plan, usually in Microsoft Project. All team members participate in weekly project planning and monitoring sessions, however, and usually team leaders and project managers work on the technical part of the project as well. Each team has to complete and sign a contract defining 10 hours per week that everyone is available to meet. Teams do not necessarily have to meet during those times, but if they have trouble with a member who does not attend meetings when they schedule them, they can fall back on holding meetings during their agreed-upon meeting time. JAD Sessions As the team organizes itself, its members prepare for their first requirements gathering or Joint Application Development (JAD) session with the customer. The students schedule this session with the customer and the instructor. The first JAD session is held on the customer site, which gives better information about the project environment and allows the instructor to evaluate the customer site, should the students be expected to work there. Students prepare an agenda and a series of questions for this meeting. They introduce themselves, make eye contact with the customer, and establish a relationship of trust. Each student is required to speak at every meeting and presentation. The agenda helps the students manage the meeting, keeps them focused on their task, and can be used to refocus a customer who digresses. One student takes notes, prepares a summary of the discussion, and reiterates decisions and open issues at the end of the session. This gives the customer an opportunity to affirm or modify any decisions that have been made. Some teams send an email of this summary to their customer (and instructor), and/or post it on their project Web site. Once the students have completed this initial JAD session, they prepare a report on the project requirements as they understand them, and submit it to the instructor. The instructor meets with the team and gives them extensive feedback on the report and on their JAD session. Feedback sessions allow the instructor to shape the performance of the team. Since 70% of the grade for the project is based on team effort, group members usually respond to reduced points that are the results of one person’s performance, and place pressure on that individual to improve. The students then revise their reports, and meet with the customer for a second JAD session to further refine and develop the project requirements. They submit a second report on that JAD session and again receive feedback. They may have additional meetings with the customer before they present their first prototype of the project about midway through the quarter 5. PROJECT DEVELOPMENT METHODOLOGY The CIS Department has adopted many of the procedures and the methodology from its Rapid Application Development class (including JAD sessions and prototyping), into senior projects. Evolutionary Prototyping The students present their first project prototype to the customer and the instructor during the sixth week of the quarter. The contents of the prototype vary, since some projects include Web applications or software development and others include systems security or configuration of telecommunications networks. The first prototype for a software project typically includes the Graphical User Interface (GUI) and sample report layouts. Frequently students give the customer options for different GUI layouts and colors. Internal functionality may or may not work at this time. The team receives feedback and questions from the customer and the instructor. Again, one student keeps notes on the decisions and outstanding issues, and reiterates these at the end of the session. The project scope may be modified at this point if the project appears too large and difficult or too small and easy. The team prepares another iteration of its report and again receives instructor feedback. Two weeks later the students present their second prototype. By this time, the functionality of the project should be partially completed. For example, for a programming project, the students should be able to show that information is being written to a database. One student again takes notes about the decisions and outstanding issues, and reiterates them to the group at the end of the meeting. This summation includes a list of the final deliverables for the project. If all of the customer’s requests cannot be delivered, the customer is asked to identify items that must be completed, and prioritize items that are not essential but that would enhance the project. The students then know how to plan their work for the rest of the project, since they must also test, document, and implement the project within a 3 week period. The students email the list of deliverables to both the customer and the instructor after the session, which ensures that all the stakeholders know what will be delivered at the end of the project. There should be no surprises. Project Management Each team has a project manager who is responsible for tracking the project tasks in a project management program, usually Microsoft Project. The project manager maintains task lists and prepares GANTT charts that the team uses to plan and monitor its project’s progress. Teams meet weekly to monitor their progress, and prepare reports approximately every two weeks for the instructor to review. The students are also required to prepare timesheets that indicate the tasks each person is expected to perform, the estimated hours each task will take, and the actual hours they spend. This requirement has two purposes. It helps the students understand how much they over- or under-estimate the amount of time they will need to do a task, so that they can refine their estimation abilities when they take a job. Since the timesheets must be turned in with the reports, it can also give the instructor information about the progress of the project. Students are expected to work 10 hours a week on their senior projects. When team members work only a few hours during a week, it may just mean that they have a lot of exams and other projects. Alternatively, it may mean that they are waiting for feedback or information from the customer and cannot proceed until they receive it. Sometimes students may be reluctant to complain to the instructor about negligent customers, even though the instructor has emphasized that it is her job to keep the customers under control. Monitoring Project Progress The instructor must continually monitor the progress of the projects. While the customers are primarily concerned with the product they will receive at the end, the instructor has to be concerned about the processes the students engage in to deliver that product. The class works best if the instructor meets with the team once a week, either in a meeting with the customer, or in a feedback meeting. Team reports after each major milestone in the project (2 JAD sessions, 2 prototype sessions, and the final report) provide information about how the project is progressing. Even though the project is in an area in which the instructor does not teach, it is still necessary to read the reports carefully and understand what the students are doing. All team members are required to attend feedback meetings. The feedback should be very specific and relate to the requirements for the project stage. Deducting points for failure to fulfill specific requirements helps the instructor gets the team’s attention. The instructor can view the interchange among the team members, and as probing questions if it appears stressed and unfriendly. Frequently the simple question of “How are things going?” uncovers information that might not come out otherwise. 6. COMMON PROBLEMS The same types of problems that occur in the development of information systems in industry occur during a senior project. The students are frequently quite adept at solving technical problems, but have more trouble with the “soft” problems, such as human relationships among themselves and with the customer. Most of the significant problems that arise are process problems rather than technical problems. Feature Creep Adding requirements to the project is normal. Just as in industry, team members overestimate their own abilities to perform work and make promises that they cannot keep. During meetings that I attend with the team and the customer, I may take an active role in project scope, since teams may promise too much or too little. During feedback sessions with students, I coach them on how to negotiate requirements with customers, and refer them to chapters on managing project scope in McConnell’s book on Rapid Development (1996). Managing Customer Expectations Managing customer expectations is one of the most important skills that students learn in this class. The process begins with the establishment of trust between customer and team members at the beginning of the quarter and continues throughout the project. I warn students in our initial meeting that the project descriptions the customers present at the beginning of the term are usually very different from the actual projects that fulfill the customer requirements. Some customers are quite savvy and know exactly what they want (particularly if they work in IT careers), while others are unaware of the capabilities of the technologies available to solve their problems. If the customers trust the student team, then the team will be able to steer the customer toward a solution that will satisfy the customer and perhaps do much more than the customer envisions. Student teams are frequently so “customer friendly” that they agree to add features that they cannot possibly deliver. Occasionally a team of “hot shots” may overpromise what they can deliver. Once overpromising occurs, students have to find a way to adjust the schedule to what is feasible. They learn to ask the customer to prioritize their requirements and decide what is most important to them, and they emphasize the importance of time for testing the final product. Once they feel overwhelmed because they overpromised, they learn to say, “I will get back to you on that.” (McDonnell, 1996). Team Organization Managing peers is another one of the most important skills that students learn in this class. Members of a team have very little power over one another other than the power that they give one another. Part of this power comes from group consensus about team norms and ways of behavior. I remind students that the team leader is no one’s boss and is expected to “lead by example.” That means the team leader should be first to arrive and last to leave, and should participate in the development of the project just like other team members. The most successful teams organize very quickly and start working on the project early in the quarter. Occasionally teams fail to organize until well into the quarter, and need the prospect of a failing grade to pull themselves together. Free Riding. The most frequent cause of team problems is free riding. “No free riding” is the motto of the class for good reason. Free riding has a much wider effect than just the lost work of the free rider. The rest of the team engages in complaining sessions, becomes distracted, loses morale, and completes less work. Thus, teams are able to deter free riding by threatening to divorce a team member who is not contributing to the project. The divorced team member receives an automatic grade of “F” in the course. However, student teams must follow procedures similar to those in industry. All members of the team other than the free rider must agree on the need to divorce the student. They must document the failure of the team member to participate, give the team member written notice of the impending divorce, and provide a way for the team member to avoid it by attending meetings, completing tasks, etc. The team must then schedule meetings during the hours in their team contract, which are hours the free rider agreed were available. If the free rider does not perform, the team can then prepare a memo divorcing the student from the team. Divorce is rare, primarily because of team norms and the threat of failure. Sometimes divorce is prevented because one team member will not agree to it, or because the problem surfaces too late in the quarter to follow the procedure of written notification. In that case, the team members can punish the scofflaw by rating him or her lowest on their peer evaluations. Although most of the class points stem from team effort and even free riders receive team points, a student’s grade is usually significantly lower if it turns out that every member of the team ranks the student at the bottom on every item on the peer evaluation. That is usually a very good indication that either the student did nothing, or occasionally that he or she was such a pain that it deterred the effectiveness of the team. Team Diversity. Because all students are required to take the class, a single team may have students of widely divergent skills, abilities, and backgrounds. Thus one or two students frequently take the technical lead on a project and coach the less experienced students with their tasks. Each student is expected to contribute to the project efforts, although the results will vary from one student to another. While research in Australia (Caspersz, Wu, & Skene, 2003) has shown that students from different ethnicities may experience lower team member satisfaction, ethnicity is not usually a source of cultural problems in our teams. Instead, primarily male team may have a man who denigrates the work of the lone woman on the team, or more experienced older team members have trouble with an inexperienced, younger student. When students bring in such problems, I discuss the problem with the student(s), and ask them for ways in which they might resolve the problem themselves. I give the team the opportunity to resolve the problem, with the assurance that I will step in if necessary. In every case, student teams have been able to settle these problems on their own. One word of advice that I regularly dispense to student teams is “Be kind to one another.” Coordination and Integration. Disorganized teams, which also fail frequently fail to develop effective team norms, may experience problems integrating their final projects. Interestingly this usually happens to teams made up of students with heavy outside experience, who underestimate the project tasks and overestimate their own abilities. These students change the database and overwrite one another’s code as they try to integrate the different parts of the project at the last minute. 7. ENSURING A SUCCESSFUL CONCLUSION This capstone class should be one of the best courses that the students take during their career at the university. At the end of the term, the students should be proud of their work, the projects should all work, be tied up and delivered, and the customers should be satisfied and happy with the results. The big problem is how to assure that this happens. There are a number of ways to achieve this goal. Student Teams Grading and feedback sessions are the best way to motivate student teams. Sometimes it is necessary to “lay down the law” and be strict with student teams that do not perform. Occasional teams do not organize themselves, and do not coordinate their work. The instructor’s one recourse is to lower their grades, usually with the reassurance that if they improve their performance they can receive a very good grade at the end of the course. Customers also play a strong role by providing regular feedback and encouragement to the team. The customer’s time is the price he or she pays for the project deliverables. This fact needs to be made clear to the customer at the beginning of the project and sometimes during the course of the term. List of Deliverables This list becomes binding near the end of the project. It gives the students a roadmap to follow and keeps them from embellishing the project with bells and whistles while ignoring the core product. It also prepares the customer for what he or she will receive. Testing, Documentation, Training Students are encouraged to finish their projects at least a week before it is delivered, and to spend time testing it and fixing bugs. The customer is encouraged to test it as well to make sure that it works. Project documentation is developed throughout the quarter in the team reports. Appropriate training of end users is expected for every project. Customer issues Since the purpose of the course is to teach students how to manage and deliver an IT project to a real customer, the customer’s role is central to the project success. Training customers and being clear in expectations of their time is critical and must be something that the instructor undertakes before the project begins. Customer obligations are reiterated during the first JAD session as well. The worst case scenario is the customer who leaves for a two-week vacation in the Bahamas during the third week of the project, leaving behind a gifted team with nothing to do. The team then loses heart and does not usually build up enthusiasm again when the customer returns. If the customer must provide some technology (e.g., Web server, computer, IP address) before the student team can integrate the project into the customer’s environment, the customer must be told early in the quarter that they need to arrange for this technology. Student teams may research possible solutions and make recommendations for the customer. If this part of the project is ignored, the team may deliver their project on a CD-ROM disk, and the customer may not know how to implement it. Rarely customers become demanding and want students to do menial clerical work. It is important to emphasize the nature of the work the students can be expected to do before the class starts. In addition, when the customer is demanding, the students have to learn how to handle the problem. Designating a single tough-minded person as team communicator and copying the instructor on messages to the customer can be effective in curbing customer demands. Our university recently adopted a service learning program. Service learning senior projects can be very satisfying for the students. 8. ASSESSMENT Assessment is an important tool to measure the success of senior projects. It occurs on several levels. Team Reports and Sessions Feedback sessions provide an opportunity to assess student work and mold appropriate behavior and work. The importance of meeting with students face-to-face regularly cannot be overemphasized. Managing the project is the students’ responsibility. However, the instructor needs to give regular feedback to be sure they are on track for success. Peer Evaluations Each team member prepares a peer evaluation that rank orders team members on a number of criteria, for example “Quality of work,” “Quantity of work,” etc. There are nine categories, some of them specific (e.g. preparation of reports, research) so that each team member should be able to rank first in one of the categories. This grade is part of the 30% individual score for the course. Peer evaluations for the team are averaged, and students on a well-functioning team usually receive approximately the same scores. Peer evaluation ratings really only influence the grades of the outliers at both the high and low ends of the scale. Peer evaluations probably have questionable value in grading (Lejk & Wyvill, 2001) since they are subjective and can be political. Peer evaluations also give team members leverage to punish a colleague’s performance and therefore serve as a deterrent to free riding. Customer Evaluations Customers are given an evaluation sheet at the end of the project. Customers provide a grade for the entire project, as well as a grade for each student. Both of these grades are factored into the student’s final grade even though customers’ grading standards vary widely. The fact that students know that customers grade them encourages them to interact with the customer and build customer trust. Customers also answer open-ended questions about their experience and provide suggestions for improving that experience. Grading Student grades are calculated so that 70% comes from team effort and 30% from individual effort. Team members who contribute extra work may receive bonus points. After the course is finished, the instructor prepares a final feedback form for each student that includes their grade, plus any comments from the customer regarding the project and the individual student’s work. These are sent to the student’s home address. 9. CONCLUSION This course can be one of the most satisfying or frustrating courses that one teaches, depending on the student teams and the customers. Sometimes all of the projects are local or within close proximity. Other times they may require driving all over Southern California and. Usually customers are required to come to campus for their two-prototyping sessions. However, telecommunications and security projects may be done on-site and require visits to the customer site in order to view student progress. This course provides a view of the success of the undergraduate program through which the students have traversed. If there are obvious gaps in student knowledge, they show up here. The areas of strength in the program are also quite visible. Constantly monitoring the results of the projects provides important assessment information back into the program and the curriculum. 10. REFERENCES Caspersz, D., M. Wu, and J. Skene, 2003, “Factors influencing effective performance of university student teams.” Herds a Annual Conference–Christ-church,NZ. http://surveys.canterbury.ac.nz/herdsa03/pdfsref/Y1025.pdf). Ellen, Nicky and John West, 2003, “Classroom management of project management: A review of approaches to managing a student's information system project development.” Journal of American Academy of Business, September, 3(1), p.93. IS 2002 Model Curriculum, http://www.is2000.org/front.html Lejk, M. and M. Wyvill, 2001, “The effect of the inclusion of self-assessment with peer assessment of contributions to a group project: A quantitative study of secret and agreed assessments.” Assessment & Education in Higher Education, 26 (6), p. 551. McConnell, Steve. 1996. Rapid Development: Taming Wild Software Schedules, Microsoft Press. Weistroffer, H. and J. Roland-Gasen, 1995, “Assessment of information systems programs: A life cycle approach,” Journal of Education for Business, pp. 70, 258.