The ABET CAC Accreditation Experience – Intent and Reality— The Information Systems Perspective Frederick G. Kohun Kohun@rmu.edu and David F. Wood wood@rmu.edu Robert Morris University 6001 University Blvd Moon Township, PA 15108, USA Abstract This paper addresses unresolved issues associated with the Accreditation Board for Engineering and Technology, Computing Accreditation Commission (ABET CAC) accreditation process for Information Systems (IS) programs. To approach these issues, we will discuss the history of IS accreditation, the standards, the ABET accreditation process, and conclude with some of our observations and recommendations for prospective programs. Robert Morris University in Pittsburgh, PA was among the first group of programs evaluated by ABET CAC. This paper is based on Robert Morris University experience with the accreditation process and visit conducted in the Fall of 2002. Keywords: accreditation, ABET, information systems education 1. HISTORY The history of IS accreditation is relatively long and complex. The complexity lies in the interdisciplinary roots of IS—business and computer science. The twenty year process of finding a home for the inherent applications focus of IS started with discussion in the academic computer science community, then to the applied professional associations, and finally into a home of its own in Accreditation Board for Engineering and Technology, Computing Accreditation Commission (ABET CAC). A brief chronological summary of events include (but is not limited to) the following: • In 1985, the Computer Sciences Accreditation Board was formed • In 1986 at the Fall Joint Computer Conference, the first IS accreditation workshop was held. Representatives from ACM, IEEE-CS, and DPMA (now AITP) drafted criteria for accreditation. • In 1987 the DPMA Educational Foundations Information Systems Education Conference (ISECON) hosted public presentations on the draft criteria. • Later, NSF provided a three-year grant to examine the feasibility of accreditation programs in Computer Science/ Systems/ Technology. The study provided for the development of a set of criteria and procedures considered for IS program accreditation. The NSF project brought together representation from all identified stakeholders. 1. Leading Information / Computing Societies * AIS (Association for Information Systems) * ACM (Association for Computing Machinery) * IEEE-CS (IEEE – Computer Society) * AITP (Association for Information Technology Professionals) * Recent Curriculum Efforts * IS97 * ICC99 2. NSF funded Curriculum Development Efforts 3. CSAB and representatives from industry and academia (Gorgone 2000) • Fall 2001, ABET CAC (Computing Accreditation Commission) becomes the Accrediting body for Information Systems • December 2001, IS Accreditation Criteria Finalized, Accreditation Criteria 2. ACCREDITATION STANDARDS There are 54 ABET-CAC accreditation stands contained in the accreditation categories. Every accreditation category has an “Intent” which indicates a broad overview of what would make an accredited program. A program must meet the intent statement of every category. Within each category are precise “standards” which, if satisfied, constitute evidence of meeting the intent. Programs are permitted to present rationale and justification for an alternative implementation meeting the intent. The following narrative summarizes the categories, Intents, and Standards: (ABET 2003b) Objectives and Assessments Category The first category consists of objectives and assessments. The intent of the category consists of the following statements: “The program has documented educational objectives that are consistent with the mission of the institution. The program has in place processes to regularly assess its progress against its objectives and uses the results of the assessments to identify program improvements and to modify the program’s objectives.” Criteria: To meet this intent, the following standards must be satisfied: 1. The program must have documented educational objectives. 2. The program’s objectives must include expected outcomes for graduating students. 3. Mechanisms must be in place to periodically review the program and the courses. 4. The results of the program’s assessment must be used to help identify and implement program improvement. 5. The results of the program’s review and the actions taken must be documented. Comments: The planning and assessment process cannot be informal. The detailed process used must be enumerated thoroughly, including data gathered, reports generated, analysis of that data and actions taken. The results and process must be documented. In general, the objectives are not only for the department, but for each course as well. A track record of evaluation and improvement in the planning and delivery of each course must be presented. Students Category The intent reads “Students can complete the program in a reasonable amount of time. Students have ample opportunity to interact with their instructors and are offered timely guidance and advice about the program’s requirements and their career alternatives. Students who graduate the program meet all program requirements.” Criteria: To meet this intent, the following standards must be satisfied: 1. Courses must be offered with sufficient frequency for students to complete the program in a timely manner. 2. Information systems programs must be structured to ensure effective interaction between teaching faculty and students. 3. Advising on program completion, course selection and career opportunities must be available to all students. 4. There must be established standards and procedures to ensure that graduates meet the requirements of the program. Comments: Student transcripts are audited, including courses transferred from previous institutions. There should be a process insuring graduates are audited for meeting the program standards, including those transferring or asking for special permissions. Students receiving exceptions can cause problems. Faculty Category Faculty members’ responsibility is addressed in two categories, the Faculty category, and the Program Delivery Category. The intent in the Faculty category reads “Faculty members are current and active in the discipline and have the necessary technical breadth and depth to support a modern information systems program.” Criteria: To meet this intent, the following standards must be satisfied: 1. The interests, qualifications, and scholarly contributions of the faculty members must be sufficient to teach the courses, plan and modify the courses and curriculum, and to remain abreast of current developments in information systems. 2. All faculty members must have a level of competence that would normally be obtained through graduate work in information systems. 3. A majority of the faculty members should hold terminal degrees. Some full-time faculty members must have a Ph.D. in information systems or a closely related area. 4. All faculty members must remain current in the discipline. Comments: All faculty members must be current and active in the discipline. Currency must be documented. Currency is demonstrated by publication, attendance at courses and workshops, new course development, presentations, and consulting, although it is presumed that the consulting would lead to publication. The standard says all faculty members need to be current. There can be no one left out. All of faculty activities need to be documented before the self-study document is created. Curriculum Category The intent of the curriculum category is “The curriculum combines professional requirements with general education requirements and electives to prepare students for a professional career in the information systems field, for further study in information systems, and for functioning in modern society. The professional requirements include coverage of basic and advanced topics in information systems as well as an emphasis on an IS environment. Curricula are consistent with widely recognized models and standards.” Criteria: To meet this intent, the following standards must be satisfied: 1. The curriculum must include at least 30 semester-hours of study in information systems topics. 2. The curriculum must contain at least 15 semester-hours of study in an information systems environment, such as business. 3. The curriculum must include at least 9 semester-hours of study in quantitative analysis as specified below under quantitative analysis. 4. The curriculum must include at least 30 semester-hours of study in general education broaden the background of the student. 5. All students must take a broad-based core of fundamental information systems material consisting of at least 12 semester hours. 6. The core materials must provide basic coverage of the hardware and software, a modern programming language, data management, networking and telecommunications, analysis and design, and role of IS in organizations. 7. Theoretical foundations, analysis, and design must be stressed throughout the program. 8. Students must be exposed to a variety of information and computing systems and must become proficient in one modern programming language. 9. All students must take at least 12 semester hours of advanced course work in information systems that provides breadth and builds on the IS core to provide depth. 10. The 15 semester hours must be a cohesive body of knowledge to prepare the student to function effectively as an IS professional in the IS environment. 11. The curriculum must include at least 9 semester-hours of quantitative analysis beyond pre-calculus. 12. Statistics must be included. 13. Calculus or discrete mathematics must be included. 14. The oral and written communications skills of the student must be developed and applied in the program. 15. There must be sufficient coverage of global, economic, social and ethical implications of computing to give students an understanding of a broad range of issues in these areas. 16. Collaborative skills must be developed and applied in the program. Comments: Double counting of credits in different categories is not permitted. The wording of the standards is very precise. All issues must be covered. We treated the discrete mathematics topic in two ways. First, we developed our own course, “Quantitative Analysis for Information Professionals” in collaboration with Mathematics faculty members. Secondly, we surveyed all our existing courses and identified which assignments are related to discrete math, and could be used to satisfy the math requirements. We could do this because we had more than the required number of courses (30 credits) in Information Systems topics. Even so, this approach was problematic, because our documentation of the individual assignments and their grading was sketchy. Technology Infrastructure Category The intent of this category includes “Computer resources are available, accessible, and adequately supported to enable students to complete their course work and to support faculty teaching needs and scholarly activity.” Criteria: To meet this intent, the following standards must be satisfied: 1. Each student must have adequate and reasonable access to the systems needed for each course. 2. Documentation for hardware and software must be readily accessible to faculty and students. 3. All faculty members must have access to adequate computing resources for class preparation and for scholarly activities. 4. There must be adequate support personnel to install and maintain computing resources. 5. Instructional assistance must be provided for the computing resources. Institutional Support and Financial Resources Category The institution’s support for the program and the financial resources available to the program are sufficient to provide an environment in which the program can achieve its objectives. Support and resources are sufficient to provide assurance that an accredited program will retain its strength throughout the period of accreditation. Criteria: To meet this intent, the following standards must be satisfied: 1. Support for faculty must be sufficient to enable the program to attract and retain high-quality faculty capable of supporting the program’s objectives. 2. There must be sufficient support and financial resources to allow faculty members to attend national technical meetings with sufficient frequency to maintain competence as teachers and scholars. 3. There must be support and recognition of scholarly activities. 4. There must be office support consistent with the type of program, level of scholarly activity, and needs of the faculty members. 5. Adequate time must be assigned for the administration of the program. 6. Upper levels of administration must provide the program with the resources and atmosphere to function effectively with the rest of the institution. 7. Resources must be provided to acquire and maintain laboratory facilities that meet the needs of the program. 8. Resources must be provided to support library and related information retrieval facilities that meet the needs of the program. 9. There must be evidence of continuity of institutional support and financial resources. Program Delivery Category The second faculty category is called Program Delivery. Its intent is There are enough faculty members to cover the curriculum reasonably and to allow an appropriate mix of teaching and scholarly activity.” Criteria: To meet this intent, the following standards must be satisfied: 1. There must be enough full-time faculty members with primary commitment to the program to provide continuity and stability. 2. Full-time faculty members must oversee all course work. 3. Full-time faculty members must cover most of the total classroom instruction. 4. Faculty members must remain current in the discipline. 5. All full-time faculty members must have sufficient time for scholarly activities and professional development. 6. Advising duties must be a recognized part of faculty members’ workloads. Comments: Students are interviewed about access to faculty. Advising and access to faculty help is a serious responsibility. It is not sufficient to merely assign advisors. Students must have personal access to academic advising and help with courses. Institutional Facilities Category The intent of this category states “Institutional facilities including the library, other electronic information retrieval systems, computer networks, classrooms, and offices are adequate to support the objectives of the program.” Criteria: To meet this intent, the following standards must be satisfied: 1. The library that serves the information systems program must be adequately staffed with professional librarians and support personnel. 2. The library’s technical collection must include up-to-date textbooks, reference works, and publications of professional and research organizations. 3. Systems for locating and obtaining electronic information must be available. 4. Classrooms must be adequately equipped for the courses taught in them. 5. Faculty offices must be adequate to enable faculty members to meet their responsibilities to students and for their professional needs. 3. THE ABET CAC ACCREDITATION PROCESS – FORMAL PROCESS AND ACTUALIZATION While the following sections provide an overview of the accreditation process and the major lessons learned from our experience, the narrative is intentionally limited. The purpose is to provide a brief overview of the process and what was either emphasized or needs to be reinforced. Self Study The self study is key to the accreditation process. Robert Morris used this opportunity to truly discover, in an objective way as possible, its strengths and its weaknesses. Using the standards and their intent allowed the faculty a depth of perspective never attained before. Through this self-study process many lessons were learned. Among the most important, each standard should be taken literally, and the document should show direct evidence of how the standard is met. If standards aren’t satisfied, there must be elaborate rationalization and presentation of how the intent is met. It is far easier in general to design the program and the procedures to satisfy the standards than it is to justify an alternate way of meeting the intent. There will be no presumption or coaching by the team to make an effective case. Another important lesson was that a program should be careful about the use of words to describe topics in syllabi. Each course should specify in detail the coverage of relevant standards—in as literal way as possible. One final lesson not to be taken for granted was that faculty currency should be documented in detail and in multiple dimensions. The intent states that “Faculty members are current and active in the discipline and have the necessary technical breadth and depth to support a modern information systems program.” Our faculty submitted essentially Vitae describing their professional activities. They omitted many other activities regarding consulting, self-studies, and curricular revision, which demonstrated their currency that had a direct impact on the courses taught.. Curriculum and How it Maps to the Model The IS program at each school should be mapped to the IS2002 curriculum and the ABET standards (Gorgone 2002). The curriculum standards are specific, and it helps the evaluation team if an academic department can translate and map its courses into the standards. Courses cannot be used to count for two different standards simultaneously. For example, if the standard requires coverage of global economic, social, and ethical implications of computing, the course that provides this cannot normally be used to satisfy other standards. It is possible to split a standard across courses, but the remaining part of the course would not be counted as 3 credits. To demonstrate how standards related to the quantitative analysis requirement were met, we constructed an elaborate matrix to allocate quantitative analysis to a number of different courses, recognizing at the same time that our total coverage of information systems concepts, in turn involved more than the minimum number of concept courses required in the model. (The Robert Morris IS curriculum involves a greater number of IS courses than is required by the IS2002 Model curriculum.) Team Formulation The team consisted of one MIS evaluator, one IS evaluator, and one Computer Science evaluator. Together they examined the data applicable to all of the standards, including personal interviews with everyone involved in the self-study report. It was useful to have multiple perspectives on our program. The team recognized the importance of IS and MIS representation. Current ABET training materials emphasize Computer Science, assuming that IS represents a minor deviation. It was extremely useful for the team to form an ad hoc definition of what an Information System program consists. The dynamic nature of Information Systems as a discipline requires flexibility not needed in a more stable discipline such as Computer Science. The Computer Science standards are written differently from those of Information Systems. It is easy for an evaluator to assume they are the same, but the detailed wording is important. Words like “Most”, “All”, “Some” are interpreted quite precisely. ABET July Workshop ABET, in its national July meeting. Offers a CAC Institutional Representative workshop to institutions about to undertake a self study process for subsequent ABET review. Because we were in the first round, there was no workshop the previous year. As a result we learned a great deal about the self study process—after we had already completed it, written a report, and submitted the report to ABET. It is important to send faculty members and administrative leaders to the ABET workshops in July. This should happen before the self study is written. The study will be a much more productive document if it is written from the perspective of how it is to be used by the team. Course Displays Must be Detailed Second in importance to the self study report are course displays. The ABET-CAC process requires course display that includes syllabi, exams, assignments, and most important student work highlighting evidence of the standards being implemented. We had not collected sufficient student data to show good assignments, bad assignments and exams. Every course needs to be conducted with ABET standards in mind. The syllabi have to be detailed in the content covered. There should be a log containing assessments and improvements added each time the course is offered. Student assignments need to be collected with comments. During Visit Informal conversations with the team were quite important. The course displays didn’t completely and accurately reflect what was in the courses. Faculty discussion and explanation of the displays provided context that would have been difficult to ascertain from the displays alone. It is a multiple year task to make sure all course displays are current, clear, and consistent with the standards. Exit Interview – Completely Covers Conclusions It is important to decide who needs to participate in the exit interview. The response to the accrediting team’s suggestions often requires a significant commitment by the University. Those responsible for approving this commitment need to be involved. In some cases, the Department Head, Dean, and President all have separate meetings. In other cases, the results of the visit are presented to groups of constituents—each group getting progressively larger with new constituents entering each group. This is suggested because the core University decision makers gets to hear the same comments repeatedly—minimizing ambiguity and interpretation. Unlike most exit interviews, the ABET-CAC exit interview is more of a collegial discussion than the often hurried and awkward presentation and exit of other accrediting bodies. It is important for the officers of the institution to hear exactly the exit visit comments. Those comments are presented in a straightforward format going through each standard and relating the team’s determination whether the standard is met, is a concern, is a weakness, or is a deficiency. 1 week After Visit According to accreditation process, an institution has up to one week to respond immediately to those concerns which could be dealt with immediately. If a situation can be changed, the institution is encouraged to do so. Official Letter There should be no surprises in the official letter (received in March of the next year.) A program has sufficient time to address all but the most serious infractions of the standards. 4. CONCLUSIONS • The process was consistent and uniform. It represented a predicable way to apply standards to a chaotic world. • Outcomes assessment is important and must be taken seriously. It must be documented, but not necessarily quantitative. • ABET values Faculty Currency as well as scholarship. Very few academic organizations recognize the amount of effort it takes to maintain currency in Information Systems and Computing. • Not having a doctorate in Information Systems on the faculty was a concern. There is a requirement to have at least one doctorate in Information Systems on the faculty. This is a non-negotiable new requirement as part of the accreditation standard. • Information Systems is different from Computer Science. The accreditation process at this point is dominated by Computer Science Evaluators, bringing a Computer Science perspective. Programs need to be prepared to make the case for the definition of their discipline. We were fortunate to have a team willing to respect our interpretation of the intent statements of the IS categories. Institutions must recognize their responsibility to argue that they meet the intent of the criteria. This intent must be documented in the self study and supporting evidence, not merely oral and anecdotal. • This accreditation is evolving. As more programs experience the accreditation cycle and more IS evaluators are qualified, we expect there to be adjustments and flexibility in the standards. • While ABET-CAC has become the accrediting body for IS programs, it is apparent that the criteria and standards still need to evolve beyond their roots in computer science. 5. REFERENCES ABET (2003a) “ABET 2003 Program Evaluator Training Guide”, ABET ABET (2003b) “Accreditation Criteria”, http://www.abet.org/images/Criteria, ABET Gorgone, John T., 2000, “Information Systems Accreditation: Revisited”, White Paper, http://cis.bentley.edu/ISA/pages/documents/acrd_sigcsecol4v4.pdf Gorgone, John T., Gordon B. Davis, Joseph S. Valacich, Heikki Topi, David L. Feinstein, Herbert E. Longenecker Jr. (2002) “IS2002 Model Curriculum and Guidelines for Undergraduate Degree Programs in Information Systems”, Association for Information Systems, Atlanta, GA, http://192.245.222.212:8009/IS2002Doc/Main_Frame.htm 6