The Development of a Comprehensive Assessment Plan: One Campus’ Experience Bruce White Bruce.white@quinnipiac.edu Richard McCarthy Richard.mccarthy@quinnipiac.edu Abstract “For society to work […] we must be accountable for what we do and what we say.” (Dowdell, 2007). Assessment continues to grow in importance. Assessment, accountability and feedback are the cornerstones of the concept of ‘constant and continuous improvement’. As educators, we strive to deliver quality instruction that fits within a unified curriculum. We present the development and implementation experience of a comprehensive Information Systems assessment program. The use of the CCER IS Assessment Test and other assessment processes are discussed. Keywords: Assessment, Assessment Plan, Curriculum, Accreditation 1. 1. INTRODUCTION Grant Wiggins stated: “no person can succeed unless he or she is held accountable”. Ken Blanchard stated “feedback is the breakfast of champions”. To be accountable, we must obtain feedback and assess our programs, learning objectives and individual teaching abilities. In education, our stakeholders are demanding more information and accountability. Such questions as “Are we teaching what we say we are?”; “Are students learning?”; “How can we be more effective in our instruction” need to be answered. There are many methods that can be used to assess a program. Assessment should answer the following questions (Acharya, 2003): 1) What do we want students to learn? 2) Why do we want them to learn it? 3) How can we help them to learn it? 4) How do we know what they have learned? (Stemler, Chamblin, 2006) There are many possible methods for assessment. Some of the most common types include: * Archival Records – where past student records are analyzed * Behavioral Observations – where student activities, and processes are seen * Exit Interviews – where graduating seniors are interviewed and asked for feedback on their academic experiences * External Examiner – where an expert (or experts) is invited to campus to review activities and academic (such examinations may be part of accreditation visits) * Focus Groups – selected students gather to give feedback and opinions * Locally Developed Exams – tests developed by faculty to measure learning * Oral Exams – interactive questions to determine knowledge and skills * Performance Appraisal – combining scores from students across the curriculum * Portfolios – students document their accomplishments (frequently electronically) * Simulations – students compete in real life scenarios to solve problems * Surveys and Questionnaires – getting feedback using both open and close ended questions * Standardized Tests – such as the Center for Computing Education Research’s IS Assessment test (White, McCarthy, 2006) Each of these assessment processes has value and merit. Some may be more appropriate for assessing one particular aspect of learning. Each require time and effort to implement, so many programs must pick and choose which assessment method(s) are most appropriate to meet their needs. 2. LEARNING GOALS To develop an assessment plan for an Information Systems academic department, one must determine the learning goals. In short, ‘what do we want our students to learn’? As a general foundation, most American programs in Information Systems are built on the IS2002 model curriculum (currently) – which is the successor to previous such curriculum models. IS2002 (see http://is2002.org) has a wealth of reports detailing the suggested learning goals and skills. Using the IS2002 curriculum model as a foundation, each program can develop learning goals that they wish to emphasize. Such learning goals can vary from program to program. For example, if an information systems academic program emphasizes the business integration of information technology their learning objectives may differ from a program that emphasizes the application development process. Other aspects of learning goals might depend upon the program’s organization and structure. For example, information systems academic programs within a school of business might have different requirements and structure as compared to programs within a school of computing. 3. Quinnipiac University Case Study: The Development of Learning 3. Objectives in Information Systems Management The Information Systems faculty members at Quinnipiac University, a medium sized university in southern Connecticut initiated their learning objective development by first developing the following program mission statement: “The Information Systems Management program maintains a focus of enabling students to manage and work with information systems which meet business or organization requirements effectively; this means that students must understand the need for fault tolerant systems which are within the requirements of budgetary constraints, incorporate ethical and legal considerations, and meet specific enterprise goals, including quality requirements for customer service.” After formulating this mission statement, the faculty developed six specific learning objectives. In developing these specific learning goals, the faculty had to consider where the program had been, where it was going and what they saw as the primary areas of focus. Specifically, the learning objectives are: 1) Analysis and design of information systems which meet enterprise needs. 2) Use and experience with multiple design methodologies. 3) Experience in the use of multiple programming languages. 4) Development of hardware, software and networking skills. 5) Understanding of data management. 6) Understand the role of IS in Organizations. These learning objectives were determined by the Information Systems faculty. In addition to following the IS2002 Model Curriculum guidelines, the faculty solicited input from alumni, advisory board members, and companies that hire there is graduates and interns. In their deliberations, the faculty wanted to give graduates a solid exposure to these learning concepts. After formulating the learning objectives for the program, the faculty set about to find appropriate assessment processes. Assessment processes For each learning objective, we examined each of the core and elective courses and determined if there was no coverage, light coverage, moderate or extensive coverage of that learning objective in that course. For example, in ISM 110 Object Oriented Programming course, we determined that it would have moderate coverage of basic analysis and design techniques (learning objective 1), but extensive coverage of programming languages (learning objective 3) and light coverage of data management (objective 5). (See appendix B for program coverage matrix.) The matrix was developed to ensure that each of the objectives were being met and that each objectives was integrated throughout the program (Not necessarily in every course). Once the matrix of learning objectives and courses was created, the faculty faced the next issue of how to evaluate the learning objective through the curriculum. Since the Information Systems curriculum at Quinnipiac University is based on the IS2002 Model Curriculum (Gorgone, et. al, 2002), the faculty first looked at the IS assessment test based on the IS2002 curriculum and developed and promoted by the Center for Computing Education Research, a Division of Institute for Certification of Computer Professionals Educational Foundation. As this test lines up very closely with the IS2002 curriculum it seemed like an excellent match. (See Appendix A for the IS2002 Body of Knowledge areas). The CCER IS Assessment test was first developed by a group of information systems professors representing 17 institutions that met at the University of South Alabama in February 2003. (McKell, et. al, 2006). It is significant to note that Dr. Herbert (Bart) Longenecker and Dr. David Feinstein from the University of South Alabama were both involved in the development of the IS model curriculum (for many years) and also with the CCER IS assessment test. (Aside: Both Dr. Longenecker and Dr. Feinstein have been recognized as IS Educators of the Year by the ISECON / EDSIG organization). Thus a strong overlap exists between the IS2002 model curriculum and the CCER IS Assessment test. “The assessment examination serves [..] to provide institutional feedback on programmatic preparation of graduate consistent with the IS 2002 Model Curriculum.” (McKell, et. al. 2006) For the objective “Analysis and Design of Information Systems which meet enterprise needs”, the faculty are using the organizational systems development and project management sub-scores from the CCER IS Assessment Test (3.1.1 Strategic Utilization; 3.1.2 IS Planning; 3.1.3 IT and Organizational Systems; 3.1.4 IS analysis and design; 3.1.5 Decision making; 3.1.6 Systems Concepts; 3.1.7 Systems Theory; 3.2.1 Team Leading; 3.2.2 Monitor resources and activities; 3.2.3 Coordinate live cycle scheduling; 3.2.4 Continuous improvement; 3.2.5 Project Scheduling and Tracking). The CCER IS Assessment test gives feedback on each of these areas and the IS faculty felt that such a national standardized test would be beneficial to assessing this area. In addition, the faculty added watching external input from conferences, advisory board, from changes in the model curriculum and other sources. The second program objective “Use and experience with multiple design methodologies” utilized the CCER IS Assessment test (3.1.1 Strategic Utilization; 3.1.2 IS Planning; 3.1.3 IT and Organizational Systems; 3.1.4 IS analysis and design; 3.1.5 Decision making; 3.1.6 Systems Concepts; 3.1.7 Systems Theory; 3.2.1 Team Leading; 3.2.2 Monitor resources and activities; 3.2.3 Coordinate live cycle scheduling; 3.2.4 Continuous improvement; 3.2.5 Project Scheduling and Tracking). The third program objective is: “Experience in the use of multiple programming languages”. Again, the IS faculty choose the IS Assessment test with a focus on these subsections: (3.1.1 Strategic Utilization; 3.1.2 IS Planning; 3.1.3 IT and Organizational Systems; 3.1.4 IS analysis and design; 3.1.5 Decision making; 3.1.6 Systems Concepts; 3.1.7 Systems Theory; 3.2.1 Team Leading; 3.2.2 Monitor resources and activities; 3.2.3 Coordinate live cycle scheduling; 3.2.4 Continuous improvement; 3.2.5 Project Scheduling and Tracking The fourth program objective is: “Development of hardware, software and networking skills”. From the IS Assessment test, the sub-skills are: 1.4.1 Computer System Hardware; 1.4.2 Networking and Telecommunications; 1.4.3 Operating Systems Management; 1.4.4 Computer Systems Software; 1.4.5 LAN/WAN; 1.4.6 Systems Configuration. The fifth program objective is: “Understanding of data management”. Again, using the IS Assessment test, we are looking at these sub-skills: 1.3.1 Modeling and design; 1.3.2 triggers, stored procedures; 1.3.3 administration. The final program objective for the IS program at Quinnipiac University is “Understand the role of IS in Organizations”. For this goal, the focus is on the IS Assessment test 2.0 organizational skills area (2.1 General Organization Theory, 2.2 Information Systems Management, 2.3 Decision Theory, 2.4 Organizational Behavior, 2.7 Managing the Process of Change, 2.8 Legal and Ethical Aspects of IS, 2.9 Professionalism, 2.10 Interpersonal Skills). Additional Assessment factors The ICCP IS Assessment test has been the main overall assessment factor for the Information Systems Program at Quinnipiac University. The program has used this assessment for four years and is now developing the metrics and baseline to be used in the future. Additional factors have also been used as part of the assessment process. Senior Exit Survey Graduating seniors have been surveyed since spring 2006. This survey is a voluntary (and anonymous) mechanism to gain the perspective of graduates. In particular, the students are asked to evaluate their skills in each of the six program objective areas listed above, plus two additional areas: Ethics in IS/IT and Global Aspects of IS/IT. Data from this survey is covered in the results section. Other factors The Information Systems Department has an annual meeting with a professional advisory board (as well as ongoing informal interactions with members of that board). Ideas for curricular change and program changes are part of the annual agenda. This group includes mid to upper level managers in regional companies that hire graduates – many of whom are also alumni of the program. Assessment also has at its base the question of “What do we want students to learn?” The information systems field is dynamic. The faculty need to be able to deliver skills and education that is appropriate to the field (for example, programming languages have changed over the years). The faculty have added external factors to the learning objectives / assessment plan that include: conference attendance (to see and here of topics that could be beneficial to the curriculum); literature (mainly curriculum models like IS2002); external changes such as School of Business changes and campus academic changes. 4. RESULTS Data is now collected from the assessment processes. The table (below) summarizes some of the skills from the IS Assessment test for the 2004 to 2007 test years: Skill Set 3.0 Strategic Org. Systems Develop. 04 05 06 07 Avg 3.1 Organizational Systems Development 3.1.1 Strategic Utilization of Information Technology 37 40 46 39 40.5 3.1.2 IS Planning 37 31 47 14 32.3 3.1.3 IT and Org. Systems 34 33 50 29 36.5 3.1.4 Information Systems Analysis & Design 47 42 53 44 46.5 3.1.5 Decision Making 23 22 22 17 21 3.1.6 Systems Concepts, Use of IT, Cust. Service 43 35 45 38 40.3 3.1.7 Systems Theory and Quality Concepts 43 38 43 20 36 3.2 Project Management 3.2.1 Team Leading, Project Goal Setting 49 42 61 36 47 3.2.2 Monitor and Direct Resources and Activities 35 41 50 50 44 3.2.3 Coordinate Life Cycle Scheduling and Planning 55 64 74 45 59.5 3.2.4 Apply concepts of continuous improvement 47 44 37 42 42.5 3.2.5 Project Schedule and Tracking 45 37 57 43 45.5 Number of Students Taking Test 29 24 11 9 (declining) It is a challenge to make sense of such data. It is also interesting for this campus to track the number of individuals taking the test. Like many IS programs, our enrollment has been dropping. In 2004, 29 graduating seniors took the test; in 2005 there were 24 students taking the test; in 2006 there were 11 students taking the test; and in 2007 there were only 9 students taking the test. With fewer students taking the test, there can be a greater variance. It is also of some interest to note that in 2007 there were three seniors who did not take the test due to time conflicts – would or could have that significantly changed the results? IS Core Area 04 05 06 07 Avg Undergraduates only Hardware and Software 39.6 38.7 48.0 31.1 39.35 Modern Programming Language 38.1 37.3 42.2 32.4 37.5 Data Management 41.2 40.9 53.9 41.4 44.35 Networking and Telecommunications 39.4 35.1 53.6 46.3 43.6 Analysis and Design 43.1 43.0 53.6 40.9 45.15 Role of IS in Organizations 50.1 44.7 58.7 45.5 49.75 Likewise here is a comparison for the past four years. In this view, it might look like the results from 2006 are a bit of an outlier as the other three years (2004/ 2005/ 2007) tended to be similar. The close alignment of the IS assessment test with the IS2002 model curricula – and with the goals and learning objectives of the Information Systems program at Quinnipiac University seem to be demonstrated. Senior Exit Survey A second feedback mechanism is from a survey of graduating seniors. The survey ask these graduates about their skills in the six learning objective plus two additional areas: Ethics in IS / IT and Global Aspects of IS / IT. The results from the 2006 senior surveys are presented here: Learning Objective 2006 scores 2007 scores Systems Analysis (including project management 4.4 4.5 Alternative Design Methodologies 3.7 3.5 Programming Languages 4.0 3.0 Hardware and Software 4.2 4.5 Networking 3.9 3.5 Data management 4.2 4.2 IS in Organizations 4.2 4.4 Ethics in IS / IT 4.2 3.8 Global aspects of IS / IT 3.6 3.8 It is not a goal of this paper to thoroughly discuss the results, but to present how the assessment process was developed and implemented. Anecdotally however, the authors suggest that the learning objectives were generally reached or surpassed. Some concerns about where and how alternative design methodologies and global aspects of IS / IT can be strengthened were identified as curriculum objectives to be addressed by the faculty for the 2007-2008 academic year. Setting Standards Now that the campus has four years of data on the IS Assessment Test as well as two years of senior exit survey data, it is time for the department to set our standards. With assessment, setting a standard is setting a metric that can be measured and the question becomes: Did the students reach the standard? If the answer was ‘they reached the standard’, the department will continue in the same vein of instruction. If the answer is ‘they did not reach the standard’, then the department will need to carefully review the results and make appropriate changes. A standard might be to use the average of the IS Assessment test and an average of the Senior Exit Survey for future students. If the standard was not reached, the faculty will need to review why it was not reached. Was this a poorer class of graduating students as compared to previous classes? Did we have poor instruction in one or more classes? What could be done to change the score? Was the textbook selected particularly weak when students took that class (or classes)? 5.CONCLUSION This paper has described an action-based research experience of the development of an assessment plan for the Information Systems Management department at Quinnipiac University. The faculty worked closely to develop the learning objectives. Then the faculty worked to find appropriate tools to assess the learning. The authors suggest that the model has worked well for feedback and program assessment. The authors’ experiences indicate that it is critical to establish the specific measurements a program will utilize and develop a process to enable those measurements to drive ongoing review and possible program change. A rigorous review program must then be followed to measure the results and determine if adjustments to the program are warranted. Additional details of the specifics are listed in the appendices. The development of the assessment plan has been completed – now is the time to set the standards and make the assessment process a reality. Gloria Rogers, the ABET director of assessment, stated “We must establish a culture of assessment, not a climate of assessment.” (Rogers, 2005) Assessment must be an integral part of our community of learning in order to strive for constant and continuous improvement. 6. REFERENCES Aasheim, C, JA Gowan, H. Reichgelt, “Establishing an Assessment Process for a Computing Program”, Information Systems Education Journal (ISEDJ), 2007. Acharya, C, “Outcome-based Education (OBE): A new Paradigm for Learning”, CDTLink, November 2003, retrieved from: http://www.cdtl.nus.edu.sg/link/nov2003/obe.htm, June 27, 2007. Brennan, R L. Educational Measurement, jointly sponsored by National Council on Measurement in Education and American Council on Education, 2006. Crooks, T, “The Validity of Formative Assessment”, Paper presented to the British Educational Research Association Annual Conference, 2001 Dowdell, B, “Quick Take”, April 9, 2007 Gorgone, J, G. Davis, J. Valacich, H. Topi, D. Feinstein, H. Longenecker, “IS 2002 Model Curriculum and Guidelines for Undergraduate Degree Programs in Information Systems, ACM, New York, NY, AIS and AITP, 2002 IS 2002 Model Curriculum, http://is2002.org, retrieved June 27, 2007 IS Education, http://iseducation.org, retrieved June 24, 2007. Maki, P, “Developing an Assessment Plan to Learn about Student Learning”, The Journal of Academic Librarianship, January-February 2002 McKell, L, J. Reynolds, H. Longenecker, J. Landry, H. Pardue, “The Center for Computing Education Research (CCER): A Nexus for IS Institutional and Individual Assessment”, Information Systems Education Journal (ISEDJ), 2006. Palomba, C., T. Banta, (1999) Assessment Essentials, Jossey Bass, San Francisco Rogers, G, “Do Grades make the Grade for Program Assessment”, ABET Quarterly News Source, Fall / Winter 2003. Rogers, G, Presentation handouts on assessment, given to Quinnipiac University faculty, August 2005. Stemler, L, C. Chamblin, “The Role of Assessment in Accreditation: A Case Study for an MIS Department”, Information Systems Education Journal (ISEDJ), 2006. Valacich, J., Accreditation in the Information Systems Academic Discipline, White paper submit to the AIS Executive Committee, November 13, 2001. White, B., R. McCarthy, “What gets Measured Gets Done: Evaluating the Assessment Results of Accredited IS Programs”, Journal of Informatics Education Research, April 2006. Wiggins, G., Educative Assessment, Jossey-Bass Publishers, 1998 Appendix A: IS2002 Body of Information Systems Knowledge (main topic areas). Note that each of the units (like 1.3 Programming Languages is broken down into 1.3.1; 1.3.2; etc.). Body of Information Systems Knowledge 1.0 Information Technology 1.1 Computer Architectures 1.2 Algorithms and Data Structures 1.3 Programming Languages 1.4 Operating Systems 1.5 Telecommunications 1.6 Database 1.7 Artificial Intelligence 2.0 Organizational and Management Concepts 2.1 General Organization Theory 2.2 Information Systems Management 2.3 Decision Theory 2.4 Organizational Behavior 2.7 Managing the Process of Change 2.8 Legal and Ethical Aspects of IS 2.9 Professionalism 2.10 Interpersonal Skills 3.0 Theory and Development of Systems 3.1 Systems and Information Concepts 3.2 Approaches to Systems Development 3.3 Systems Development Concepts and Methodologies 3.4 Systems Development Tools and Techniques 3.5 Application Planning 3.6 Risk Management 3.7 Project Management 3.8 Information and Business Analysis 3.9 Information Systems Design 3.10 Systems Implementation and Testing Strategies 3.11 Systems Operation and Maintenance 3.12 Systems Development for Specific Types of Information Appendix B: Information Systems Management Program Coverage Matrix Courses/ Objectives Analysis and Design: Develop compre-hendsion in the systems development life cycle, including planning, analysis, data gathering, data and process modeling, design options, construction, imple-mentation and maintenance. Also acquire basis skills in project management techniques, controls and process. Multiple Design Method-ologies:. Use and experience with multiple design metho-dologies (such as the System Develop-ment Life Cycle, Agile Develop-ment and Joint Application Develop-ment), and multiple system models (procedural, enterprise, data oriented and object-oriented models). Program-ming Languages: As tools for system construction and modification, with an under-standing of appro-priateness for an application and the capabilities and limitations of a language Hardware, Software and Net-working: Acquire skills in hardware and software, including different computing platforms and operating systems. This also included an under-standing of networking concepts and applications. Data Manage-ment: Develop insight and knowledge of data manage-ment, including SQL structures and techniques; entity-relation diagrams (ERD); normal-ization and efficiency in data optimization Role of IS in Organi-zations: Achieve appreciation for the role of information systems in organi-zations, including IT for competitive advantage, value chain, enterprise resource planning (ERP); electronic business and electronic commerce; supply-chain manage-ment; and more. ISM 110 Object-Oriented Programming Develop an initial under-standing of the systems development process (Moderate) Develop-ment of algorithmic concepts and program flow, including objects, classes and funda-mental operations (Extensive) Develop initial use of databases in application develop-ment (light) An introduction to business development (Nominal) ISM 210 Advanced Object-Oriented Programming Gain additional insights to analysis and design, user interface design, construction, testing and implement-ation (Extensive) Introduction to multiple design method-ologies (Light) Continued develop-ment of systems, including database applications (Extensive) Additional knowledge and experience, including SQL statements and database. (Moderate) Additional insight into IS for business functions (Nominal) ISM 260 Advanced Excel and ERP Systems (Elective) Gain additional under-standing of the importance of business analytics within an organization (Nominal) Discussion of ERP systems with JAD and SDLC approached (Light) Develop-ment of macro coding for spread-sheet efficiencies. Introduction to some hardware concepts for ERP systems (Light) Develop insights into data quality issues (Nominal) Introduction to the role of ERP systems within an organization (Moderate) ISM 270 E-Business Systems Development of architectures for e-business platforms (Extensive) Under-standing of the hardware and software require-ments to develop e-business architect-ures (Moderate) Under-standing of the data for conducting e-business; including security (Moderate) Insight into the role of e-business within the overall information systems strategy (Moderate) ISM 301 Hardware and Software Some under-standing of the SDLC in acquiring hardware and software (nominal) Understand processes for develop-ment of applications on alternative platforms (Perl on Linux) (Nominal) Understand processes for develop-ment of applications on alternative platforms (Perl on Linux) (Nominal) Extensive under-standing of hardware and software platforms, including processors, system software and more. (Extensive) Develop an under-standing of how various hardware and software platforms suppose business goals for competitive advantage (nominal) ISM 330 Networking and Telecom-munications Develop an understanding of the SDLC in acquiring and modifying networking (moderate) Analysis of networking hardware and software, including routers, bridges, switches, TCP/IP and other protocols (Extensive) Develop an under-standing of how networking aids businesses in commun-ication (nominal) ISM 335 Accounting Information Systems (Elective) Some under-standing of using SQL to query (Nominal) Extensive under-standing of data controls (Extensive) Under-standing of the role of ERP systems within an organization (Nominal) ISM 351 Database Applications Study of the SDLC as it applies to database and systems (nominal) Extensive study of database appli-cations and develop-ment (Extensive) Under-standing of how databases aid modern business functions (nominal) ISM 370 Systems Analysis and Design Complete and compre-hensive study of systems analysis and design from idea development through implement-ation and maintenance. (Extensive) Understand multiple design method-ologies (JAD, Agile, SDLC) (Extensive) Minor analysis of hardware / software options (nominal) ERD / DFD diagrams for analysis and design (logical and actual). (Moderate) Understanding the business value of projects (cost/benefit) and business / technology fit (moderate) ISM 381 Web Development (Elective) Exposure to agile programming and rapid prototyping (Nominal) Develop-ment of Web based applications using JAVA (Extensive) ISM 400 Emerging Technologies (Elective) Topics vary, but include analysis of current and emerging technologies and their role in an organization (Extensive) ISM 411 Information Systems Security (Elective) Analysis of IT security policies and procedures (Nominal) Analysis of hardware and software network security tools and techniques (Extensive) ISM 427 Design and Implement-ation of Information Systems in Emerging Environments (Elective) Under-standing of UML and object-oriented design (Extensive) Under-standing of Information Systems Architecture and change management policies and procedures (Extensive) ISM 440 Project management Extensive under-standing of project management as it applies to the SDLC (Extensive) Understand managing projects with multiple design method-ologies (moderate) Understand managing application / system development (moderate) Integrating database concepts into projects (nominal) Understanding business / technology fit and standard operating procedures (Moderate) ISM 484 ISM Internship Can vary Can vary Can vary Can vary Can vary Understanding of how Information Technology supports the organization (topics vary) (Moderate) Note: Nominal Coverage = 1 to 2 weeks Moderate Coverage = 3 to 4 weeks Extensive Coverage = 5 or more weeks Appendix C: Information Systems Management Program Objectives and Assessment Measures Objectives Assessment Measures Analysis and design of information systems which meet enterprise needs. This includes developing a comprehensive understanding in the systems development life cycle, including planning, analysis, data gathering, and acquiring basic project management, data and process modeling, design options, construction, implementation and maintenance skills. Students also acquire basic skills in project management and project control. Methodology: Use of the IS Assessment test with emphasis on 3.1 Organizational Systems Development and 3.2 Project Management scores Timing: at the end of a student’s senior year Individuals Responsible: ISM Chair, Assessment Coordinator, all ISM faculty, ISM Advisory Board Action Items: Make changes to courses based on the organizational systems development and project management sub-scores (3.1.1 Strategic Utilization; 3.1.2 IS Planning; 3.1.3 IT and Organizational Systems; 3.1.4 IS analysis and design; 3.1.5 Decision making; 3.1.6 Systems Concepts; 3.1.7 Systems Theory; 3.2.1 Team Leading; 3.2.2 Monitor resources and activities; 3.2.3 Coordinate live cycle scheduling; 3.2.4 Continuous improvement; 3.2.5 Project Scheduling and Tracking External Action Items: Make changes to courses based on external input, such as changes in the IS model curriculum; input from the advisory board; input from conferences and academic sources; changes based on campus changes (such as core curriculum changes) or state changes and mandates. Use and experience with multiple design methodologies (such as the System Development Life Cycle, Agile Development and Joint Application Development), and multiple system models (procedural, enterprise, data oriented and object-oriented models). Methodology: Use of the IS Assessment test with emphasis on 3.1 Organizational Systems Development and 3.2 Project Management scores Timing: at the end of a student’s senior year Individuals Responsible: ISM Chair, Assessment Coordinator, all ISM faculty, ISM Advisory Board Action Items: Make changes to courses based on the organizational systems development and project management sub-scores (3.1.1 Strategic Utilization; 3.1.2 IS Planning; 3.1.3 IT and Organizational Systems; 3.1.4 IS analysis and design; 3.1.5 Decision making; 3.1.6 Systems Concepts; 3.1.7 Systems Theory; 3.2.1 Team Leading; 3.2.2 Monitor resources and activities; 3.2.3 Coordinate live cycle scheduling; 3.2.4 Continuous improvement; 3.2.5 Project Scheduling and Tracking External Action Items: Make changes to courses based on external input, such as changes in the IS model curriculum; input from the advisory board; input from conferences and academic sources; changes based on campus changes (such as core curriculum changes) or state changes and mandates. Experience in the use of multiple programming languages to be used as tools for system construction and modification, with an understanding of appropriateness for an application and the capabilities and limitations of a language. Methodology: Use of the IS Assessment test, with emphasis on 1.1 Software Development scores. Timing: at the end of a student’s senior year Individuals Responsible: ISM Chair, Assessment Coordinator, all ISM faculty, plus report and feedback from ISM advisory board Action Items: Make changes to courses based on the software development items ((3.1.1 Strategic Utilization; 3.1.2 IS Planning; 3.1.3 IT and Organizational Systems; 3.1.4 IS analysis and design; 3.1.5 Decision making; 3.1.6 Systems Concepts; 3.1.7 Systems Theory; 3.2.1 Team Leading; 3.2.2 Monitor resources and activities; 3.2.3 Coordinate live cycle scheduling; 3.2.4 Continuous improvement; 3.2.5 Project Scheduling and Tracking . External Action Items: Make changes to courses based on external input, such as changes in the IS model curriculum; input from the advisory board; input from conferences and academic sources; changes based on campus changes (such as core curriculum changes) or state changes and mandates. Development of hardware, software and networking skills, including different computing platforms and operating environments. This also includes understanding networking concepts and applications. Methodology: Use of the IS Assessment test with emphasis on 1.4 Systems Integration scores Timing: at the end of a student’s senior year Individuals Responsible: ISM Chair, Assessment Coordinator, all ISM faculty, ISM Advisory Board Action Items: Make changes to courses based on the systems integration sub-scores (1.4.1 Computer System Hardware; 1.4.2 Networking and Telecommunications; 1.4.3 Operating Systems Management; 1.4.4 Computer Systems Software; 1.4.5 LAN/WAN; 1.4.6 Systems Configuration. External Action Items: Make changes to courses based on external input, such as changes in the IS model curriculum; input from the advisory board; input from conferences and academic sources; changes based on campus changes (such as core curriculum changes) or state changes and mandates. Understanding of data management, including SQL structures and techniques; entity-relation diagrams (ERD); normalization and data optimization. Methodology: Use of the IS Assessment test with emphasis on 1.3 Database scores Timing: at the end of a student’s senior year Individuals Responsible: ISM Chair, Assessment Coordinator, all ISM faculty, ISM Advisory Board Action Items: Make changes to courses based on the database sub-scores (1.3.1 Modeling and design; 1.3.2 triggers, stored procedures; 1.3.3 administration. External Action Items: Make changes to courses based on external input, such as changes in the IS model curriculum; input from the advisory board; input from conferences and academic sources; changes based on campus changes (such as core curriculum changes) or state changes and mandates. Understand the role of IS in Organizations, including IT for competitive advantage, value chain, enterprise resource planning (ERP); electronic business and electronic commerce; and supply-chain management; Methodology: Use of the IS Assessment test with emphasis on 2.0 Organizational Skills Timing: at the end of a student’s senior year Individuals Responsible: ISM Chair, Assessment Coordinator, all ISM faculty, ISM Advisory Board Action Items: Make changes to courses based on the 2.0 Organizational and Professional Skills areas (10 sub-scores / sub-areas). External Action Items: Make changes to courses based on external input, such as changes in the IS model curriculum; input from the advisory board; input from conferences and academic sources; changes based on campus changes (such as core curriculum changes) or state changes and mandates.