Five Years of Success: Some Outcomes of the Carnegie Mellon Information Assurance Capacity Building Program Carol Sledge - cas@sei.cmu.edu Software Engineering Institute Carnegie Mellon University Pittsburgh, PA 15213 USA Daniel P. Manson - dmanson@csupomona.edu Computer Information Systems Department California State Polytechnic University, Pomona Pomona, CA 91768 USA Anna Maria Berta - Carnegie Mellon University Dena Haritos Tsamitis - Carnegie Mellon University Abstract In July 2007 Carnegie Mellon University (Carnegie Mellon®) hosted its sixth annual offering of the Information Assurance Capacity Building Program (IACBP). The goal of the intensive in-residence summer program is to help build Information Assurance education and research capacity at minority-serving universities. Since 2002, in the first 5 editions, over 45 faculty have participated in the month-long intensive program, which has resulted in the development of a number of new offerings at partner schools, including updates to existing courses, new courses, certificate programs and degree programs. Carnegie Mellon has also supported regional information assurance symposia and mini-boot camps, additional information assurance grants, and a number of research projects and publications involving faculty and students at partner schools, as well as multi-institution collaborative centers. Throughout the planning and implementation the Carnegie Mellon IACBP goal has been on choosing faculty and university programs that are committed to information assurance curriculum development, teaching and research. As a result, a high level of success has been achieved by faculty and their programs. One example of this success is California State Polytechnic University, Pomona (Cal Poly Pomona), which became a designated Center of Academic Excellence (CAE) in Information Assurance in 2005. The purpose of this paper is to highlight the success of the IACBP and the experience of Cal Poly Pomona and subsequent designation as a CAE. A symposium recently brought back graduates of the IACBP to share outcomes. A recent survey was also conducted of IACBP attendees to document IA education, training and outreach activities. The results provide strong evidence of Carnegie Mellon IACBP successes. Keywords: Faculty, Information Assurance, Information Systems Security, Information Systems Curriculum, Curriculum Design, Curriculum Development, Capacity Building 1. INTRODUCTION In response to organizational needs for graduates with expertise in information security, many universities have launched degree programs in information technology systems (Buckler, 2005; Kim, et. al., 2005). The goal of the National Science Foundation Federal Cyber Service: Scholarship for Service (SFS) program is to increase the number of qualified students entering the information assurance field and to increase higher education capacity to produce professionals in these fields (National Science Foundation, 2006). Centers of Academic Excellence in Information Assurance Education (CAE/IAE) institutions have held month-long Information Assurance Capacity Building Programs (IACBP). These information assurance faculty training “boot camps” provide a combination of Information Assurance theory and hands-on experiences. Significant government funding is needed to provide these boot camps. Carnegie Mellon received a $400,000 NSF grant in 2002 to develop their program, targeting faculty from Minority-Serving Institutions (MSI), and provide initial boot camps through CyLab, a university-wide, multidisciplinary initiative involving more than 200 faculty, students, and staff at Carnegie Mellon. In 2006 Carnegie Mellon received a third two-year grant from NSF for the amount of $360,000 (CyLab, 2007). Information on applying to the IACBP is available by contacting Dr. Carol Sledge, co-PI, at cas@sei.cmu.edu. Table 1 shows the IACBP Schedule. Date Topic Week 1 Security Engineering, Survivability and Information Assurance, Virtual Training Environment, Curriculum Development Week 2 Security Engineering, CISCO boot camp Week 3 Curriculum Development, Computer Ethics, Usable privacy and security, Assessing of Quality of a Business Process Week 4 Curriculum Development, Participant Curriculum and Research Presentations Table 1: 2007 IACBP Schedule: http://www.cylab.cmu.edu/default.aspx?id=2289 The goals of the Carnegie Mellon capacity building program are to: • Help build new capacity or expand existing capacity of minority-serving institutions to offer Information Assurance courses and programs at institutions not currently designated as Centers of Academic Excellence in Information Assurance • Expand the number of institutions that are Centers of Academic Excellence in Information Assurance • Expand the number of Ph.D.-level researchers in Information Assurance (Cylab, 2007) 2. PARTICIPANTS A significant time and financial commitment is involved in attending a month-long in-residence boot camp. Faculty must spend four weeks in a training environment away from home. While funding has been provided by the NSF SFS program, Carnegie Mellon, CyLab, Software Engineering Institute (SEI) and CERT faculty and staff all invest their own resources into the program. Two members of the SEI, including Dr. Carol Sledge and the Manager of Diversity and Outreach, help select faculty who attend the Carnegie Mellon IACBP. Demonstration of support (including letters of support) by the department and campus administration are required for all faculty attendees, to provide evidence of their commitment to the outcomes of the IACBP. Participant goals include enhancing faculty expertise in the area of Information Assurance (IA), helping home institutions start programs in IA or build upon existing programs, and developing new modules, courses, or even degree program curricula in the area of IA. Information assurance is an umbrella term developed by government that encompasses security and privacy. The National Security Agency defines Information Assurance as “the set of measures intended to protect and defend information and information systems by ensuring their availability, integrity, authentication, confidentiality, and non-repudiation” (National Security Agency, 2006). For several institutions, Carnegie Mellon’s IACBP has leveraged another strategy. For each institution, the IACBP has supported multiple faculty members from that institution over a period of two to three years. This has helped build a critical mass at that institution’s department to better ensure the completion of its short- and long-term plans for the incorporation of information security into the curriculum. Another strategy to build critical mass and to leverage existing efforts, involves working with the California State University (CSU) Discipline Council, composed of the department chairs from the departments of computer science, software engineering, information systems, etc. at the 23 CSU campuses. Currently, of the 12 CSU campuses that have attained Hispanic Serving Institution (HSI) status, nine, plus a tenth CSU campus designated as an MSI, have attended the Carnegie Mellon IACBP, adding an additional layer of critical mass and leverage. Beginning in 2004 the IACBP added faculty from business departments that have strong information systems components and thus an interest in enhancing the information assurance coverage in their curricula (Sledge, 2006). Schools that have participated in the Carnegie Mellon IACBP from 2002-2007 are listed below: • Alabama A&M University [2006] • California State Polytechnic University, Pomona [2004-05] • California State University, Dominguez Hills [2005-06] • California State University, Fresno [2005] • California State University, Fullerton [2006] • California State University, Long Beach [2007] • California State University, Los Angeles [2004-06] • California State University, Northridge [2005-07] • California State University, San Bernardino [2003] • California State University, Stanislaus [2007] • Elizabeth City State University (NC) [2005] • Grambling State University (LA) [2007] • Hampton University (VA) [2003-05] • Howard University (DC) [2002-03] • Langston University (OK) [2007] • Morgan State University (MD) [2002-03] • Mt. San Antonio College (CA) [2004-05] • Oakwood College (AL) [2004-06] • Prairie View A&M University (TX) [2007] • San Jose State University (CA) [2005] • Spelman College (GA) [2006] • Texas A&M University, Corpus Christi [2003-04] • University of the District of Columbia [2007] • University of Texas at El Paso [2002-03] 3. CAL POLY POMONA IACBP OUTCOMES In July 2004 Dr. Dan Manson from Cal Poly Pomona attended the IACBP at Carnegie Mellon University. During the program, initial plans for a 2+2 degree in Information Assurance between Cal Poly Pomona and Mt. San Antonio College were developed. The IACBP also provided Dr. Dan Manson with partnership support from Carnegie Mellon needed to apply for designation as a National Center of Academic Excellence (CAE) in Information Assurance Education. Criteria required to become a CAE include the following • Provide evidence of partnerships in IA education with minority colleges and universities, 2-year community colleges and technical schools. • The academic program demonstrates how the university encourages the practice of IA, not merely that IA is taught. • The academic program encourages research in IA. • The IA curriculum reaches beyond the normal geographic borders of the university. The following paragraphs show how working with Carnegie Mellon University helped enable Cal Poly Pomona achieve each of these criteria. 4. PARTNERSHIPS AND REACHING BEYOND NORMAL GEOGRAPHIC BORDERS On December 11, 2004, in partnership with Carnegie Mellon University Software Engineering Institute, Cal Poly Pomona hosted an Information Assurance Symposium forum. The forum served in building Information Assurance Capacity, with a special focus on improving infrastructure at Minority Serving Institutions (MSIs). The event also served as a model for other regional workshops at MSIs. Attendees included faculty and students from throughout the California State University system. Featured speakers included Hun S. Kim, Deputy Director, Strategic Initiatives at the Department of Homeland Security (DHS), National Cyber Security Division (NCSD) who spoke on DHS/NCSD strategic goals and initiatives. Other speakers included Richard D. Pethia, Director, CERT® Centers, Software Engineering Institute, Carnegie Mellon University who spoke on “Computers Under Attack What Can We Do?” and Noopur Davis of the Software Engineering Institute who spoke on “Developing Secure Software”. The conference – the first of its kind between Carnegie Mellon University and the CSU – was the genesis of collaboration between Carnegie Mellon, the College of Business Administration and the Computer Information Systems Department. In June 2005, The Software Engineering Institute (SEI), with the support of California State Polytechnic University, Pomona, presented a summer faculty workshop designed to help prepare faculty to teach a software project course. The workshop was based on many years of experience in teaching such courses, and on previous SEI workshops on the Personal Software Process (PSPSM) and the Team Software Process (TSPSM) developed by Watts Humphrey. The TSP, itself supported by the PSP, was designed for industrial software teams; it has been introduced in many software development settings and is proving to be very successful. The TSPi was developed for academic use, with student teams in a software project course. Workshop attendees included faculty from across the U.S. and Taiwan. For more information on PSPSM and TSPSM, a good starting point is http://www.sei.cmu.edu/tsp/ . From June 17 to July 1, 2005, Cal Poly Pomona and Cal State Los Angeles co-hosted a 2-week “mini-boot camp” based on the Carnegie Mellon IACBP model. The mini-boot camp covered a broad range of information assurance topics, including government information security training standards, operating system security, security certifications, risk assessment, and computer forensics. Faculty learned from other faculty, with peer-to-peer knowledge exchange. The Cal Poly Pomona / Cal State Los Angeles mini-boot camp was held with 35 attendees from the following institutions. Cal Poly Pomona Cal State Chico Cal State Humboldt Cal State Northridge Cal State Sacramento Cal State San Bernardino Idaho State University Mt. San Antonio College Oxnard College Pasadena City College West Los Angeles College Harrisburg Area Community College Community College of Allegheny College Table 2: List of Cal Poly Pomona Mini-Boot camp schools represented 5. PRACTICING IA In the summer of 2003, Cal Poly Pomona established a Security Office for the campus. From 2003 to 2004, and again in 2006, Dr. Dan Manson served as the campus security officer. In 2003, Dr. Manson organized an Incident Response Team with representatives from various campus units who need to work in a coordinated fashion to identify and response. In 2005, Carnegie Mellon University’s Software Engineering InstituteSM initiated a cooperative research and development relationship with the CSU Office of the Chancellor to improve the handling of network and system security incidents at each of the 23 campus in the CSU system (CSU: Information Security Management/Incident Response, 2005). 6. IA RESEARCH The partnership developed in Information Assurance Education by Carnegie Mellon and Cal Poly Pomona has resulted in several papers and presentations over the past few years. These have provided both schools with the opportunity to demonstrate broader impacts of their work in Information Assurance teaching and research. These include a presentation at the 2006 Frontiers in Education Conference on “Learning Modules for Security, Privacy and Information Assurance In Undergraduate Engineering Education” (Manson, et al, 2006) and papers for the Software Engineering Institute (Sledge, 2005, 2006). In March 2005, The Department of Homeland Security and National Security Agency designated California State Polytechnic University, Pomona as a National Center of Excellence in Information Assurance Education for academic years 2005-2008 Centers of Academic Excellence, (2007). 7. SYMPOSIUM For the 2006-2007 NSF capacity-building grant, the Carnegie Mellon IACBP partnered with the Computer Engineering Department in the College of Engineering at San Jose State University, a minority serving CSU campus, to add a two-day symposium. This symposium is approximately one year after each summer program, with participation of the members of that year’s cohort and with an invitation for IACBP participants from previous cohorts to contribute as well. Here too, the IACBP leverages the existing relationships, expertise and work by Carnegie Mellon and San Jose State University, as they are both members of The Team for Research in Ubiquitous Secure Technology (TRUST)i, an NSF Science and Technology Center started in 2006 (TRUST: Education and Outreach, 2007). On June 14th and 15th, 2007, San Jose State University hosted a symposium of Carnegie Mellon IACBP graduates with the following goals. * Showcase the participants' achievements after the IACBP * Further update their expertise * Bring them into closer touch with industry Industry speakers and topics at the symposium included the following: * “Control-Flow Integrity and Security Vulnerability Mitigation in Windows” by Ulfar Erlingsson, Microsoft Research * “Introduction to Tamper Resistance and Power Analysis Attacks” by Josh Jaffe, Cryptography Research, Inc. * “What is Your Data Was Always Secure?” by Chris Parkerson, RSA * “Phishing Attack Trends Through 2006” by Zulfikar Ramzan, Symantec * “The Ghost In The Browser Analysis of Web-based Malware” by Niels Provos, Google The following academic participants attended the Carnegie Mellon-San Jose State University IACBP Symposium and gave presentations on their IA/IS course and curriculum development: • California State Polytechnic University, Pomona • California State University, Fullerton • California State University, Los Angeles • California State University, Northridge [2] • Carnegie Mellon University (PA) • Elizabeth City State University (VA) • Hampton University (VA) • Mt. San Antonio College (CA) • San Jose State University (CA) [4] • Spelman College (GA) [2] 8. PROGRAM ASSESSMENTS Executive Summary The Information Assurance Capacity Building Program (IACBP) welcomed its sixth group of participants in July 2007. It is hoped these participants will build up on the success of its past participants whose results are reported in this document. In brief, the IACBP has delivered the following results to date: * According to self-assessments, the extent of participants' backgrounds in information assurance (IA) varies, but those who report having stronger IA background also rate the program higher in terms of effectiveness and satisfaction. * The Cisco Boot Camp receives high ratings from participants. * Participants have implemented new coursework and supplemented existing coursework at both undergraduate and graduate levels. * Overall, 5700 undergraduate students and 1080 graduate students have benefited from the IA coursework developed by participants since attending the IACBP program. In addition, the IACBP participants have contributed to symposia and conferences, clusters, research papers, and community outreach events related to IA since attending the program. The details of these findings follow. Pre-program Self-assessment Since 2004, at the beginning of every edition of the IACBP, participants anonymously completed a brief questionnaire describing their background skills and knowledge in areas relevant to the program. The topics covered by the questionnaire were selected by the principal program instructors. The assessment instrument appears to capture considerable variation in participants’ self-rated knowledge of familiarity and expertise with different concepts. This was good, since it is likely that the background and strengths of program participants differ substantially. Therefore, we have some confidence that the questionnaire was actually capturing some of this variation. Of course, self-reported assessments must always be accompanied by the concern that responses contain some bias, possibly due to a desire to appear more competent or knowledgeable. This concern is addressed in two ways. First, participants were told that their responses would be completely anonymous. The only identifying information they included on the questionnaire was a private PIN, which the respondents themselves generated and did not share with anyone. Second, there are many areas in which participants report almost no familiarity. If responses indicated a bias towards improving one’s stated knowledge and experience, it seems unlikely that we would observe as many low levels of rated familiarity. We are therefore confident that, to at least some extent, the questionnaire captures participants’ background skills and knowledge accurately. In 2006 and 2005 the results of this pre-assessment are also quite similar to those of the 2004 IACBP pre-assessment. While some of the details differ, the general patterns are very similar for all years so far. Post-program Self-assessment Similar to the pre-program self-assessment survey, at the conclusion of the IACBP, all participants anonymously completed a questionnaire in which they reported how much they felt they had learned in various areas covered by the program. The topics covered by the questionnaire were drawn from the pre-program assessments that had been administered to participants at the beginning of the program. Participants also reported their perceptions of the overall effectiveness of the program on several dimensions. Interestingly, there always appears to be a relationship between participants’ self-rated background knowledge (measured by the composite score from the pre-program assessment) and their ratings of how much they learned in the program and how satisfied they were with it. In general, participants with a stronger background rate the program as better. The general patterns of responses of 2005 and 2006 are similar to those for the 2004 post-program assessment. The overall ratings of satisfaction with the program are higher in 2005 than they were in 2004, perhaps reflecting the slightly stronger self-rated background and familiarity of 2005 participants. Cisco Boot Camp Assessment For all the editions since 2004, we also offer a Cisco evaluation in the form of a pre- and a post-boot camp survey. The post-program assessment of the Cisco Boot Camp always found, on average, participants report having learned a considerable amount on the general topics covered in the program. While participants also report relatively high levels of knowledge and familiarity with important specific concepts, there is considerable variability in these ratings. Some participants report a very low ability to teach and work with most of the concepts covered in the program. Nevertheless, most participants report high levels of familiarity, on average. In ratings of overall satisfaction with the program and the instruction, the responses are very high. None of the responses were lower than 8, which is the second-highest possible response. Thus, while there is considerable variation in how much participants feel they learned in the program, they appear to be very satisfied with the effectiveness of the program. 9. OTHER ACCOMPLISHMENTS In addition to Cal Poly Pomona’s accomplishments and recognition as a Center for Academic Excellence (CAE), other significant results of the IACBP during the first five editions include the following: * California State University Dominguez Hills (CSUDH), CA, for the introduction of a new course on Wireless Networking and Security, a General Education course in Computer Education including topics in IA and Ethics, a proposal for new graduate level Security Courses, and for establishing a Network Security Lab. In addition, for their Master of Science program (approval of the MS program allows Computer Science faculty to start working on “Information Security Track”), and the new undergraduate course on Security Engineering. * Oakwood College, Huntsville, AL, for a number of additions to existing courses in their Information Technology Department, and a significant impact on outreach activities and K-12 initiatives * California State University Los Angeles, CA, for a new certificate program (the focus of the certificate program is to manage information security technologies on the one hand and to satisfy some key security certifications (Security +, CISSP, and CCSP) requirements on the other) * California State University Northridge, CA, for existing courses’ enrichment, and a new “Special Topics in Information Systems and Assurance,” offered as an upper-division undergraduate elective for IS majors. (This course was added to the Fall semester immediately following attendance at the IACBP!) * Spelman College, Atlanta, GA, for the addition of topics such as Computer Security, Ethics, and Privacy in a computer literacy course, as well as an enriched First Year Seminar for Computer Science. 10. SURVEY AND METHODOLOGY In spring 2007, a survey was conducted to document IA education, training and outreach activities from all the alumni of the IACBP during the past five editions. The survey results provide additional evidence of successful outcomes of the IACBP program. Since the goal of the survey was to collect quantitative information about all the initiatives planned and designed during the IACBP, all the questions have been designed and phrased to collect this kind of information. Furthermore, since for many of the questions the alumni had to do a little bit of investigation, at the moment we contacted them, we also provided an overview of the data that we were planning to receive. In this way, they were able to have all the information at hand before actually starting to answer the survey itself. A total of 30 questions were asked, divided in four categories: courses, academic collaborations, community outreach, and other. 11. SURVEY HIGHLIGHTS The survey has been an excellent way to re-connect with the IACBP alumni and get important feedback on longer term results and accomplishments. The data collected is presented below. In the first five editions, since the program started in 2002, there have been 45 attending faculty from minority institutions Their attendance at the program allowed them to implement the following: • Undergraduate programs * 29 new undergraduate courses that include IA o More than 1,400 students enrolled and attended those courses * 65 existing undergraduate courses have added IA topics o more than 4,300 students enrolled and attended * Graduate programs * 31 new graduate courses that include IA o More than 630 students enrolled and attending those courses * 18 existing graduate courses have added IA topics o More than 450 students enrolled and attending those courses * Symposia and conferences * 26 faculty have been involved with symposia and conferences o In those conferences, a total of more than 3,000 attendees * 35 seminars and lectures on IA o In those seminars, a total of more than 850 participants * Research Papers * At least 61 research papers related to IA * Cluster and collaboration centers * 16 faculty involved with local IA clusters * Community Outreach and K-12 initiatives * 10 faculty used knowledge and material from the IACBP to reach the local community and teach cybersecurity * 23 outreach events organized (~800 attendees) * 24 cybersecurity events in schools and/or specific classes * 13 presentations on cybersecurity topics (~800 attendees) 12. CONCLUSIONS The Carnegie Mellon IACBP has increased the capacity of institutions of higher education to offer information assurance (IA) and information security (IS) courses. In addition, the IACBP has promoted educational partners to adapt, adopt, and expand IA and IS education to educational institutions within their various regions. The IACBP has also leveraged other initiatives. The TRUST initiative provides Carnegie Mellon and its partners with a program that provides education and knowledge to undergraduate colleges, educational institutions serving under-represented populations, and the K-12 Community. The IACBP has also increased the number of schools designated and working toward designation as Centers of Academic Excellence in Information Assurance Education (CAE/IAE). In addition to Cal Poly Pomona, Cal State Sacramento became a CAE/IAE in 2007. Cal State Los Angeles and Cal State San Bernardino have mapped their curriculum to government information assurance training standards, meeting a pre-requisite for applying for CAE/IAE status. Institutional sharing and collaborating on information assurance faculty education is not a zero sum game. Carnegie Mellon and Cal Poly Pomona working together have helped both institutions improve their ability to serve as regional centers of IA expertise. The IACBP program has enabled a larger group of faculty to develop and enhance information assurance courses and curriculum at their own institutions. 13. REFERENCES Buckler, Grant (2005) “Master of the IT security universe,” Computing Canada, Vol. 31, Iss. 9, page 26. Centers of Academic Excellence, (2007). Accessed July 25, 2007, from National Security Agency Central Security Service: www.nsa.gov/ia/academia/caeiae.cfm CSU: Information Security Management/Incident Response, (2005). Accessed from http://www.calstate.edu /info_security_mgmt/incident_response.shtml on July 25, 2007. CyLab (2007). 2007 Capacity Building Workshop. Accessed from http://www.cylab.cmu.edu/default.aspx?id=2146 on July 25, 2007. Kim, H., Han, Y., Kim, S., and Choi, M. (2005), “A Curriculum Design for E-Commerce Security,” Journal of Information Systems Education, Vol. 16, Iss. 1, pp-55-65. Manson, D., Meldal, S., Sledge, C., Maurer, S., Mitchell, J., Spengler, E., Szitpanovits, J., and Torner, J. (2006), “Panel - Learning Modules for Security, Privacy and Information Assurance In Undergraduate Engineering Education, 36th Annual Frontiers in Education Conference, San Diego, CA, October 29, 2006. Available at http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=4116978. National Science Foundation (2006). Division of Undergraduate Education. Retrieved from Federal Cyber Service: Scholarship for Service (SFS). Accessed from http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5228&org=NSF&sel_org=NSF&from=fund on July 25, 2007. National IA Education & Training Program (2006). National IA Education & Training Program. Retrieved September 21, 2007, from National Security Agency Central Security Service: http://www.nsa.gov/ia /iaFAQ.cfm. Sledge, Carol (2005). “Building Information Assurance Educational Capacity: Pilot Efforts to Date”, Carnegie Mellon Software Engineering Institute Special Report CMU/SEI-2005-SR-009. Sledge, Carol (2006). “Information Assurance: Building Educational Capacity”, Carnegie Mellon Software Engineering Institute Special Report CMU/SEI-2006-SR-007. TRUST: Education and Outreach (2007). Team for Research in Ubiquitous Secure Technology. Accessed from http://www.truststc.org/education/ on July 25, 2007. i The TRUST initiative provides Carnegie Mellon and its partners with an additional program that provides education and knowledge to undergraduate colleges, educational institutions serving under-represented populations, and the K-12 community. --------------- ------------------------------------------------------------ --------------- ------------------------------------------------------------