The Balance between Teaching and Research: The Development of a Survey Instrument to Assess Factors that Affect MIS Research Richard V. McCarthy Department of Computer Information Systems Quinnipiac University, Hamden, Connecticut, USA E-mail: richard.mccarthy@quinnipiac.edu George Claffey MIS Department, Central Connecticut State University New Britain, Connecticut, USA E-mail: Claffey@ccsu.edu Bruce White Department of Computer Information Systems Quinnipiac University, Hamden, Connecticut, USA E-mail: Bruce.White@quinnipiac.edu Abstract Management information systems are a comprehensive discipline. Over the years there have been many areas of research covering topics such as IT research, strategy, technology, and implementation in a variety of environments. Numerous studies have examined factors related to various areas of research as well as rankings of various kinds. Prolific authors were surveyed using a qualitative questionnaire for the purpose of obtaining data on the factors that motivate their research. This study reports upon the completion of a pilot study in the development of a quantitative research instrument to evaluate the factors that affect MIS research. Keywords: MIS research, academic writing 1.INTRODUCTION A career in academia requires a balance between teaching, research and service requirements. There is no universal standard across colleges and universities as to what the precise balance should be, although at a minimum there is at least a subjective assessment for promotion and tenure. Academic research in MIS is a broad topic covering areas such as information systems strategy, the adoption of technology, user perception and usage of technology, and technical evaluations of effectiveness. MIS researchers have made tremendous progress over the past thirty years. What motivates MIS research? This study presents the development of a survey instrument to provide a quantitative assessment of the factors that affect research in management information systems. The development of a quantitative instrument is important because it provides the basis for empirically validating factors that affect MIS research. The survey instrument was developed after a comprehensive review of the prior research dealing with research motivational factors in other disciplines, and a pilot evaluation that surveyed prolific authors in the field of MIS. The results of the pilot study that led to the development of the survey instrument are also presented. 2. LITERATURE REVIEW There is a certain attraction within the American culture for being ranked. Sports teams (e.g. baseball, football, basketball), athletes (e.g. tennis and golf) are ranked and so too beauty contestants. Students are ranked upon graduation. Everyone seems captivated in knowing who the best is. Once a team or individual is ranked, then the focus is on becoming number one, or at least moving up. There have been several research studies that have evaluated journal rankings, author productivity, and school rankings within the field of management information systems. Many of the most current articles tend to extend the findings of previous studies while also increasing the breadth of review. Mylonopoulos and Theoharakis (2001) used an on-line questionnaire sent to members of the IS World mailing list and the IS Faculty Directory on www.isworld.org to determine the ten top journals as well as a second ten listing from a list of 87 journals compiled from previous ranking studies. Participants were asked to list the five journals they most frequently read. Both journal rankings and readership are depicted by world, North America, Europe, and Australasia (Asia, New Zealand, and Australia). With the exception of Europe, the top three ranked journals were the same in all regions: MIS Quarterly, Communications of the ACM¸ and Information Systems Research. In Europe the third ranked journal was the European Journal of Information Systems. The readership findings for the top three journals were the same as the ranked journals by all categories except for Europe that placed Harvard Business Review in the top three instead of Information Systems Research. Respondent profile information was also collected. Boudreau, Gefen, and Straub (2001), while interested in validation techniques currently used in reported MIS research, relied on earlier studies of journal rankings by Nord and Nord (1995), Hardgrave and Walstrom (1997), and Walstrom, et al (1995) to determine appropriate journals to survey. The five journals included: Information & Management, Information Systems Research,. MIS Quarterly, Journal of Information Systems, and Management Science. Gillenson and Stutz (1991) conducted an earlier study by mail that included journal rankings and counting books in the tenure and promotion process. Participants represented nearly half of all AACSB accredited business schools. The study ranked Management Science, MIS Quarterly, and Communications of the ACM as “A+ or most favored outlets” and a list of ten other journals as highly regarded journals. Athey and Plotnicki (2000) examined the quantity of research articles by individual IS faculty and their university affiliations using previous ranking studies as the journal basis for the period 1992-1996. Alphabetically the ten “premier” journals reviewed were: Communications of the ACM, Decision Sciences, Harvard Business Review, IEEE Transactions on Software Engineering, Information and Management, Information Systems Research, Journal of Management Information Systems, Management Science, MIS Quarterly, and Sloan Management Review. The findings indicate that 73% of the researchers who published in the premier journals on average published less than one “adjusted count article” in five years. The authors provide detailed information regarding frequency of authorship by individual universities as well. No aggregated data is provided for all authors and universities. 3. RESEARCH METHODOLOGY Survey Development Blaszczynski’s (2000) study of business education researchers using a 17-item questionnaire covering topics such as writing habits, writing fulfillment, writing strategies, and advice to researchers was the initial basis for identifying factors that affect MIS research. Based upon the literature, it was felt that the topics were broad enough to have applicability to the field of management information systems. The open-ended questions, which are qualitative in nature, were modified to be applied to the MIS field of research. Open-ended questions provide a variety of responses and richness in wording not available using other methods. These questions were intended to generate a sufficient pool of data to look for common factors that apply to the process of MIS research. Five journals were then selected based on previous journal and citation studies in MIS (Gillenson and Stutz, 1991; Holsapple, Johnson, Manakyan and Tanner, 1993; Holsapple and Johnson, 1994; Nord and Nord, 1995; Walstrom, Hardgrave, and Wilson, 1995; and Hardgrave and Walstrom, 1997) as well as the most recent research completed by Boudreau, Gefen, and Straub (1991). They studied validation techniques in quantitative MIS research. The study examined 394 articles appearing in MIS Quarterly, Information & Management, Journal of Management Information Systems, Management Science, and Information Systems Research from 1997 to 1999. For our pilot study, we selected the same journals, but extended the period of time selected to 1996-2001, shown in Table 1. Table 1. Journals Surveyed Information & Management Jan 1996 - Oct 2001 Vol. 30-38 Information Systems Research Mar 1996 - Dec 2001 Vol. 7-12 Journal of Management Information Systems Sum 1996 - Spr 2001 Vol. 13-17 Management Science Jan 1996 - Dec 2001 Vol. 42-47 MIS Quarterly Mar 1996 - Dec 2001 Vol. 20-25 Electronic and hard copy indexes of the five journals were initially captured into a database. Each journal was reviewed to ensure that all article authors were accounted for in the database. Table 2 summarizes the number of articles in the selected journals. Editorials, notes, comments, and erratum were not included. Table 2. Number of Articles and Authors 1996-2001 Journal Articles Authors Information & Management 258 567 Information Systems Research 139 313 Journal of Management Information Systems 185 432 Management Science 747 1,634 MIS Quarterly 129 296 Total 1,458 3,242 There were 213 authors who published three or more times in the journals reviewed. It is interesting to note that 71 of these authors came from 13 universities (Table 3). Table 3. School Currently Employed Four or more individuals employed at same school (N=213) Wharton School 13 Columbia University 6 University of Texas – Austin 6 Carnegie Mellon University 5 Georgia Institute of Technology (Dupree) 5 Harvard University 5 Hong Kong University of Science and Technology 5 University of California – Irvine 5 University of Georgia 5 Georgia State University 4 Massachusetts Institute of Technology 4 Stanford University 4 University of Maryland 4 The 213 authors were selected as the sample population for the qualitative analysis based upon their authorship in these top-ranked MIS journals. The questionnaire was distributed via email, with a follow-up request for non-participants. This resulted in 38 responses (a response rate of 18%). The qualitative data within the thirty-eight responses was then analyzed to determine the factors that would be used to analyze MIS research. 4. RESULTS The demographics of the sample population included 30 males and 8 females. There were 23 professors, 9 associate professors, 4 assistant professors and 2 who did not indicate a rank. Twelve participants indicated that they held endowed chairs. The preponderance of senior faculty members was not unexpected based upon the difficulty and relatively low acceptance rates of the selected journals. The respondents included sufficiently detailed descriptions of their research factors to be able to cluster their responses into categories. The first two questions asked for frequency and location of their research. The results of the frequency question (Table 4) indicate that the majority of prolific authors work on their research on a regular basis. Table 4. Research Frequency How Often Responses Daily 13 Weekly 11 Every 2 weeks 5 Monthly 2 Other 7 Preferred research location however was spilt, with 17 indicating their home, 13 their office, 3 both, and 5 other. An almost equal number of respondents indicated that they felt it was important to serve as an editor, or were currently serving as a dissertation advisor. There were 27 respondents who indicated it was important to serve as an editor, 9 who responded negatively, and 2 who expressed no opinion. Twenty-seven respondents were serving as a dissertation advisor, 10 were not, and 1 failed to respond. Additionally, 20 respondents indicated that they mentored others, this includes both doctoral students and junior faculty, 9 indicated that they only mentor doctoral students, 6 indicated that do not provide any mentoring, and 3 did not respond. The question of whether the respondents felt that it was important to be specialized in their research yielded an unexpected 16 negative responses, 8 affirmative and 8 other responses. It is also interesting to note that 21 of the 38 respondents indicated that they had been recognized for their teaching at either the school or university level. Is teaching load a factor? The respondents were asked to indicate the number of courses they teach per academic year (excluding summer courses). Summer courses were excluded because a review of the curriculum schedules of the respondents indicated that the vast majority of them are on a traditional fall/spring semester schedule and do not require students to be enrolled during the summer. Table 5 reflects the teaching loads of the respondents. The majority of the respondents (58%) teach two or fewer courses per semester, with one respondent who indicated that he was a full-time research faculty member. Table 5. Teaching Load Number of Courses/Year Responses 0 1 1 0 2 2 3 3 4 16 5 1 6 8 7 0 8 5 There is enough variation in the small sample population to warrant further study. Finally, experience was looked at to determine if it is a potential factor for MIS research. One of the questions asked the respondents to indicate the number of years they had been at their current school. The authors recognized the potential limitation it presented for faculty who had recently changed universities, but felt it would give enough of an indication as to whether to pursue experience as a potential moderating variable in a future study. It was also felt that number of years in the current school would be a stronger indicator than age (potentially biased if there is a significant number of career change faculty) or total teaching experience (as it was felt that respondents would be inclined to include graduate assistantships). The years at the current school ranged from 1 to 35. Twenty had 10 or more years at the same school. Fifteen respondents indicated 6 or fewer years at their current school; 7 were at the rank of professor—including 3 first-year professors. 5. CONCLUSION Every respondent included detail comments that served to explain their perspective on the questions. When asked specifically what motivates their research; responses varied from promotion and tenure to the more intrinsic motivators such as the need for recognition, personal fulfillment, and the acquisition of new knowledge. Not surprisingly, most of the respondents indicated that they use their research to supplement their teaching by bringing their research findings into the classroom in the form of lectures and case studies. Responses to how new research ideas are generated varied greatly. Some indicated that they get ideas from doctoral students (one indicated, “We follow our stars”), co-authors, and practitioners; while others indicated their ideas typically came from journals and conferences. Based upon our analysis of the responses we have identified the following factors that affect MIS research: 1. experience 2. teaching load 3. teaching recognition 4. serving as an editor 5. serving as a dissertation advisor/committee member 6. frequency of research 7. service requirements 8. location These factors can be grouped into three categories: 1. Factors that motivate research topic selection 2. Factors that motivate faculty to engage in the research process 3. Factors that motivate research success We will construct a quantitative survey based upon these factors to test their relationship to MIS research productivity. In addition, the survey will also include a continuation of the research into the area of idea generation. 6. LIMITATIONS OF THE STUDY This study was not intended to create a ranking system; however, we did rely upon prolific authorship as the basis for our sample population. As a result, one limitation of our study is that it does not pretend that the quality of a program or one’s education is a function of an individual or department’s quantity of publications in a given set of journals over a specified period of time. Nor does it pretend to draw any conclusions regarding the quality of an individual or educational institution based on publication in a select set of journals. The population was restricted to a select group of journals. Other individuals who write textbooks, conduct grant research, or perform other scholarly activities were not reviewed. Within the journals surveyed, authors of editorial notes or comments were not included. 7. FUTURE RESEARCH This paper presents the initial step in a research study intended to empirically identify factors that affect MIS research. The authors will develop a research model based upon the results of the pilot survey and validate that model by surveying a wide audience of MIS authors. The survey will consist of research factors identified as a result of this pilot and codified on a Likert scale. Multiple questions per factor will be utilized and statistical tests of reliability and validity will be performed on the results to ensure that the appropriateness of the instrument. The authors intend to present their final quantitative survey at the conference. 8. REFERENCES Athey, S., and Plotnicki, J. (2000) “An Evaluation of Research Productivity in Academic IT,” Communication of the Association for Information Systems (3)7, March, 1-20. Blaszczynski, C. (2000). “Insights from Prolific Business Education Researchers,” The Delta Pi Epsilon Journal, 43(3), 113-122. Boudreau, M., D. Gefen, and D.W. Straub. (2001). Validation in Information Systems Research: A State-of-the-Art Assessment, MIS Quarterly, 25(1), 1-16. Gillenson, M.L., and J. Stutz (1991) “Academic Issues in MIS: Journals and Books,” MIS Quarterly, (15)4, December, pp. 447-452. Hardgrave, B.C. and K.A. Walstrom (1997). Forums for MIS Scholars, Communications of the ACM, 40(11), 119-124. Holsapple, C.W. and L.E. Johnson (1994). Business computing research journals: A normalized citation analysis, Journal of Management Information Systems, 11(1), 131-141. Holsapple, C.W., L.E. Johnson, H. Manakyan, and J. Tanner, (1993). A citation analysis of business computing research journals, Information & Management, 25, 231-244. Mylonopoulos, N.A., and V. Theoharakis. (2001). “On Site Global Perceptions of IS Journals,” Communications of the ACM (44)9, September, pp. 29-33. Nord, J.H. and G.D. Nord, (1995). MIS research: Journal status assessment and analysis, Information & Management, 29(1), 29-42. Walstrom, K.A., B.C. Hardgrave, and R.L. Wilson, (1995). Forums for Management Information Systems Scholars, Communications of the ACM, 38(3), 93-107.