Assessment of Student Online Course Participation Dr. Adnan Omar aomar@suno.edu Department of MIS, Southern University at New Orleans New Orleans, LA 70126, USA Dr. M. Khurrum S. Bhutta bhutta@ohio.edu Department of Management Systems, Ohio University Athens OH 45701, USA Mr. Daff Kalulu dkalulu.004@hotmail.com Department of MIS, Southern University at New Orleans New Orleans, LA 70126, USA Abstract Most universities view e-learning as a step to the future. In past years, universities regarded e-learning as a strategy to increase their student enrollment and retention. However, after Hurricane Katrina, several Gulf Coast universities resorted to e-learning as a means of providing basic education to their students. In fact, Southern University at New Orleans (SUNO) has chosen e-learning as an element of its quality enhancement program (QEP) to enhance the quality of education and instruction especially for first year freshmen. However, despite the administration of pre-mastery tests at the beginning of every semester along with an extensive workshop by the e-learning department, students still do not get motivated in their daily performance in a timely fashion. The reason behind this phenomenon is most likely the lack of self motivation and of e-learning techniques, a deficiency in mentoring from K-12, and social-economic constraints on the students’ time. As a result, administrators and scholars at the university have spent countless hours and resources addressing reasons for this lack of student participation. Part of the process included conducting pre- and post-tests to measure students’ learning outcomes and recommends implementation of new software such as Web 2.0. Data was collected to analyze the lack of student involvement. This study provides faculty members with ways of structuring their online courses. Keywords: assessment, E-Learning, innovation, process, participation, student, faculty 1. INTRODUCTION Distance learning has evolved and grown in popularity. New communication technology and new media have enhanced the student learning experience. The latest educational research (Soloman & Schrum, 2007, and Reynard, 2008) indicates that a university can achieve its educational objectives through the use of e-learning as effectively as it does through traditional classroom instruction. According to such research, the subject matter of most university courses can be successfully conveyed to students through the implementation of e-learning tools. Not only can e-learning convey knowledge, but it can also enhance interactivity between student and teacher, which is a hallmark of higher learning. Furthermore, some theorists (Siemens, 2004) even claim that e-learning offers advantages over classroom instruction, namely: greater convenience, improved pacing, and higher levels of communication between instructor and learners, instruction and instructors, and student and student (Soloman & Schrum, 2007, and Reynard, 2008). Students and faculty are increasingly turning to online education internet to supplement or even replace traditional approaches to classroom teaching and learning (Alavi and Leidner, 2001; Altbach, Gumport, and Johnstone, 2001; Hanna, Glowacki-Dudka, and Conceicao-Runlee, 2000; Palloff and Pratt, 2001). Advancements in computer and communications technologies, the internet, and online education are attractive and powerful new tools for teaching and learning. Some scholars even argue that these technologies have the potential to revolutionize higher education with increased access to educational services for students and a wider reach in the educational marketplace for academic institutions (Hollenbeck, Zinkhan, and French, 2005; Medlin, Vannoy, and Dave, 2004). Wireless networks, course management systems, multimedia, and other technologies add new dimensions of richness and complexity to the learning experience. While technology offers a wide range of learning possibilities, it also presents a new set of challenges. To use e-learning effectively, institutions must adapt their pedagogy, enhance the technical proficiency of users, and develop a reliable and robust technology infrastructure (Arabasz and Baker, 2003). Despite the unquestionable benefits of e-learning technologies, the number of special education teachers who are trained to use technology in classrooms remains low. While the availability of computers, internet, and various types of assistive technologies has continued to grow, most school personnel find themselves limited in the understanding of how to operate, utilize, and implement the functions of the available hardware (Birnbaun, 2000). The presence of complex sets of factors makes motivating students a difficult challenge for faculty. Lack of participation on the first day of school is one factor that the university must address. This paper discusses the process of teaching online, including teacher skills in course organization and planning, teaching guidelines as well as new software tools, implementation of new ideas, mentoring relationships, means of student motivation, and measurement of outcome, and focuses on assessment of student performance and course evaluation. 2. LITERATURE REVIEW The rapid growth of distant learning and the increasing pace of technological innovation is a challenge to course designers. With all the research dedicated to e-learning, it is clear that more efforts are needed from both the teacher and the student. However, not much is being discussed about how these kinds of changes should affect the recognition and assessment of academic value of the skills that are being developed in the learning process. Additionally, a review of literature on e-learning and student motivation suggests that self motivation from the student as well as the instructor is critical for success in online classes (Cheng, 2008; Reynard, 2008). The intensive use of technology challenges students’ participation in online classes. Although many students believe that their success in the online orientation proves their online communication skills, however some do not have sufficient technology experience to use communication technologies such as accessing course materials on the Blackboard Software, sending and receiving emails, browsing the internet or searching for information online. Students lacking computer skills cannot concentrate on the learning activities. Instead, they spend their time fearing how they would successfully communicate using a computer (Lee, 2000). Fear, lack of confidence, and low self-esteem usually undermine online students’ participation and performance. Thus, the burden of motivating online students in order to increase their participation and reduce the drop-out rates rests on the shoulders of the instructors. As information and communication technology advances, colleges and universities are increasingly offering online classes worldwide. However, this phenomenon is accompanied by a high drop-out of online students compared to the traditional classroom students. A survey conducted on 35 students who had taken online classes showed that 90% confirmed that lack of self discipline and inadequate skill in new technology were the main problems students encounter in online classes. Many students do not set aside specific and adequate time for studying and writing assignments. Without frequent interaction with other online students or an instructor, online students may easily lose their interest and motivation mid or late in the online course of study (Roper, 2007). A syllabus or webpage consisting of a detailed course description, prerequisites, learning objectives, work assignments as well as estimated time it will take to complete course work would help students to set aside adequate time for studying, writing and submitting assignments in order to meet expectations (Hofmann, 2003). However, it can be argued that even if a detailed syllabus or website is published, students may be reluctant to fully participate in online classes if they have inadequate computer skills. In this case, instructors should be prepared to spend time during the first week of an online class helping students to access and navigate the course management tools (e.g. Blackboard) thus helping students to get hands-on training with the technology before instruction begins. 3. METHODOLOGY Online learning programs continue to grow in popularity, due mainly to the increasing number of adults who aspire to earn a college degree but are unable to do so because their full-time jobs or other personal/professional commitments prevent them from attending on-campus classes. Online courses are fast becoming both economical and practical, because the technological infrastructure needed to address the growing interest in online education is readily available (Totaro, Tanner, Noser, Fitzgerald, & Birch, 2005). SUNO established the department of e-learning in January 2006 and has set policies and procedures concerning faculty support, standards, course approval and coordination, faculty training, course development and ownership, teaching load, enrollment cap, student services, and student tuition and fees. The e-learning department is responsible for meeting standards set forth by the Board of Regents, Southern Region Education Board, and the Western Interstate Commission for Higher Education. With students displaced from New Orleans and scattered across the nation post-Katrina, implementing e-learning on a full scale directly helped SUNO retain and graduate many of its students. Recently, students in California, Georgia, Mississippi, and Texas completed degree work through online curricula, an accomplishment that was impossible pre-Katrina. 4. SAMPLE AND COLLECTION OF DATA SUNO is an open admission institution with a predominantly African American student body; the vast majority of the students come from economically-challenged homes in the Greater New Orleans area. SUNO services approximately 2,600 students per semester in all degree areas with approximately 50% employed full-time. Furthermore, the Departments of Criminal Justice, Early Childhood Education, and General Studies currently offer on-line undergraduate degree programs. An on-line Master’s Degree Program in Museum Studies is also available. For the purpose of this study grades from the Personal Productivity with Technology  (MIS 166) course were used. This course is is designed to improve students’ skills as knowledge workers through effective and efficient use of the Internet and advanced software packages.  Topics covered include: organizing data and information, software functionality to support personal and group productivity, selecting computer solutions, designing and implementing user interfaces, Internet business fundamentals, and developing computer solutions using advanced application software packages and the Internet. Table 1 depicts a summary of the grade distribution for the past 5 semesters plus the pre- and post-test statistics (Appendix). 5. COURSE ASSESSMENT As an important component of modern teaching and learning processes in face-to-face courses as well as in e-learning environments, assessment provides valuable feedback to teachers and students, which facilitates the revision and adaptation of teaching and learning activities. Furthermore, assessment activities and results can also be utilized for building and strengthening meta-cognitive skills (Osika, 2006). Focusing on the assessment of a MIS 166 the concept can be further distinguished in summative assessment, performed at the end of a learning module. Research studies that focus on learning use students’ test score improvements to measure their learning performance. In this study, students taking the course (MIS 166) were subjected to a pre- and post-test. The outcome of these tests is depicted in Table 2 (Appendix). These grades were used to measure the score difference between the two tests. To further analyze the matter, the grades were coded and analyzed using appropriate statistical techniques. Table 3 (Appendix) shows the coding scheme used. Based on the coding system, descriptive statistics are presented in Table 4 (Appendix). To study group variance, the authors tested the following hypothesis; H0 = student performance in online classes will be the same across semesters i.e.; Ho = µ1=µ2=µ3=µ4=µ5 Where: µ = Mean Grade in semester A single Factor ANOVA (Table 5) was conducted to test the null hypothesis. F-test indicated that there is not enough evidence (0.05 confidence level) to reject the null hypothesis, i.e., the mean grades across the semesters are equal. This led the authors to deduce that the grades across semesters are in fact different. 6. NEW APPROACH Distance learning ( e-learning ) has become a significant method of delivering higher education and is positively impacting the content of materials delivered in courses, the methods of delivering, and the ways in which students learn the materials. Faculty is now challenged to match the delivery of instruction to the learning styles of students. In fact, research shows that online learning modules that are static provide little interactively for learners (Cheng, 2008). E-learning is a dynamic environment that is no longer limited by traditional pedagogical system, and the tools available to faculty to work in this new environment have expand dramatically. For example, Camtasis studio software has allowed instructors to become more involved in “teaching” distant courses (Creighton, Kilcoyne, and McDonald, 2008). Software such as Adobe Breeze Presenter, Microsoft PowerPoint and Adobe Captivate 2 empower faculty to create effective and engaging presentations through voice and animations delivered on the web ( Wyrostek, 2008). It is evident that there is a lack of significant improvement in participation and passing rates from one semester to the next (Table 1). Thus, new and innovative directions /approaches are necessary to insure improvement in learning outcome. Despite the short term success with e-learning, the university should ensure the ongoing educational improvement process by requiring seminar/workshop prior to enrollment for all students who wish to participate in an e-course. This seminar/workshop should address: networking, managing time, academic skill, study habits, peer group influence, family responsibility, financial problems, support services and extra-curricular activities. The instructor must notify the Recruitment and Retention Office if the student does not participate and/or contact the professor within the first 2 weeks of school. Finally, a book voucher should be issued to students in the form of a debit card in the first week so that they can purchase books based on their allotted financial aid. Educational institutions offering online courses are responsible for providing a quality education. E-learning is having a great impact on higher education. Review modification is planning to implement alternative models of teaching and learning by installing advanced software and hardware and creating multimedia based learning modules in order to enhance e-learning as well as onsite learning outcomes. In addition, instructional techniques and strategies for promoting interactivity should be adapted to address students’ varied needs and styles and enhance student success (Omar, Liu & Koong, 2008). Also, tools such as Wiki and web 2.0 should be adapted and utilized to encourage student collaboration, innovation and participation. The characteristics of web 2.0 are a rich user experience, participations, dynamic contents, metadata, web standard, scalability and shared intelligence by way of user participation (Anderson, 2007). Another powerful tool to increase online participation is student-to-student interaction. It has been observed that students who communicate with each other regarding class activities become part of the academic group, which lessens their feeling of isolation. Furthermore, Lee (2000) states that when a learning task is accomplished, students who participate in teamwork get higher self esteem than those who work individually. Therefore, it can be argued that online student interaction with each other, minimizes the chance of drop-out, and results in increased levels of motivation. In addition, he explains that communication through online threaded discussions enables online students to know each other by recognizing the writing style and expression of thoughts and ideas rather than by physical attributes. As a result, many online students develop meaningful connections with each other which may result in enhanced career networking opportunities in years to come (Lee, 2000 & Roper, 2007). Instructors can motivate online students by awarding points to the processes online students use in order to arrive at the final answer. Such processes include thinking, interaction, collaboration, communication, and application (Reynard, 2008). Instructors should encourage all online students to show innovation and demonstrate critical thinking and application. Online students’ efforts and skills to perform on a higher level than answering multiple choice questions should gain points towards the final course grade. Instructors should reward online students based on each student’s learning process. Instructors offering online courses or face-to-face traditional classes can motivate students’ participation and enhance the learning outcome by supporting and facilitating the learning process in Figure 1 as shown in the Appendix. Figure 1 illustrates future developments for assessing students’ learning processes with the help of an online instructor as a motivator to enhance an outcome. In the “Instructor” column, the instructor enhances online learning by implementing new software in order to redesign the delivery of online courses (1A), by creating effective presentations with voice and animations (1B), and by learning how to use new tools to organize, prepare, teach and monitor the online class (1C). These processes enable the instructor to establish and encourage online students’ learning outcomes through innovation, collaboration and implementation of new ideas. In the “Online Student” column, assessment is based on the student’s demonstration of an innovative method (2A), an illustration of collaborative effort (2B), and the implementation of new ideas (2C). A student who follows these learning processes should be able to write required information and add new information (3A), follow required format and implement new designs (3B), show required learning application and new suggestions (3C), and demonstrate learning ability that is different from other students (3D). In the “Outcome” column, the student benefits from enhanced learning and is graded accordingly. This process should be replicated in such a way that both students and faculty advance their intellectual learning skills. Implementing such a technique should improve the student’s learning outcome. 7. CONCLUSION It can be argued that without the physical presence of an instructor and face-to-face interaction between student and instructor and student and student, online students may lose interest and motivation. This may be particularly true of students whose motivation and management skills are inadequate, and an instructor’s best efforts to motivate these students may not succeed in an online environment. Thus, as technology advances, it becomes incumbent on the instructor to develop and possess excellent course management skills, such as recording and posting lectures on the board using Interactive Java Applet, so that online students can access lectures and answer questions following the lecture. Instructors should be motivated to introduce tools such as web 2.0 to encourage student’s innovation and participation. Such tools will bring together more information to users who are curious and eager for knowledge and will be useful to students before and after graduation. The knowledge gained from this study provides faculty members with insights to further explore innovative use of advanced technology to address students’ learning styles, preferences, and outcomes. The outcome of this study shows that although there is a very good improvement in pre-test and post-test, there is no significant difference between the semesters. 8. REFERENCES Alavi, M., and Leudner, D. E. (2001), “Research Commentary: Technology- Mediated Learning—A Call for Greater Depth and Breadth of Research.” Information Systems Research, Vol. 12(1), pp. Altbach, P. G., Gumport, P. J., and Johnstone, D. B. (2001), In Defense of American Higher Education. Baltiore: Johns Hopkins University Press. Anderson, p. (2007) “What is web 2.0? Ideas, technologies and implication for education” JISC technology and Standards Watch Retrieve October 3, 2008 from: http://www.jisc.ac.uk/media/documents/techwatch/tsw0701b.pdf Arabasz, P. and Baker, B. (2003). Evolving Courses. Educause Center for Applied Research. Retrieved June 3, 2008, from http://net.educause.edu/ir/library/pdf/ecarso/ers/ers0303/ekf0303.pdf Cheng, J. (2008). Virtual Tutor For The 21st Cyberclassroom. University of Central Oklahoma. The IABPAD Conference Proceedings. Creighton, W., Kilcoyne, M., & McDonald, J. (2008). Integrating Multi-media Technology into Business Courses. Association of Business Information Systems. Federation of Business Disciplines Conference. Houston, TX. Hanna, D. E., Glowacki-Dudka, M., and Conceicao-Runlee, S. (2000), 147 Practical Tips for Teaching Online Groups: Essentials of Web-based Education. Madison, WI: Atwood Publishing. Hofmann, J. (2003). Motivating Online Learners. Learning Circuits. ASTD’s Online Magazine All About E-Learning. Retrieved May 9, 2008, from http://www.learningcircuits.org/2003/aug2003/hofmann.htm Hollenbeck, C. R., Zinkhan, G., and French, W. (2005), “Distance Learning Trends and Benchmarks: Lessons from an Online MBA Program.” Marketing Education Review Vol. 15 (2), pp. 39-52. Lee, C. (2000). Student Motivation in the Online Learning Environment. Journal of Educational Media & Library Sciences, Vol. 37, Iss. 4; p. 367-375. Retrieved May 13, 2008, from http://homepages.findlay.edu/clee/research _interests/motivation_online.htm Medlin, B. D., Vannoy, S. A., and Dave, D. S. (2004), “An Internet-based Approach to the Teaching of Information Technology: A Study of Student Attitudes in the United States.” International Journal of Management, Vol. 21 (4), pp. 427-434. Omar, A. Liu L, & Koong K. (2008). From disaster recovery to mobile learning: A Case Study. Int. J. Mobile Learning and Organization, Vol. X, No. X. PP 1-14. Osika, E. R. (2006). Assessing Student Learning Online: It’s More Than Multiple Choice. Online Cl@ssroom. Ideas for Effective Online Instruction. Retrieved May 29, 2008, from http://www.magnapubs .com/products/0606ocff.html Palloff, R. and Pratt, K. (2001),Lessons from the Cyberspace Classroom: The Realities of Online Teaching. San Francisco: Jossey-Bass. Reynard, R. (2008). 21st Century Teaching and Learning: Assessing New Knowledge. T.H.E Journal. Retrieved May 1, 2008, from http://www.thejournal.com /articles/22531 Roper, A. R. (2007). How Students Develop Online Learning Skills. EQ Educause Quarterly. The IT Practitioner’s Journal, Vol. 30, Issue 1. Retrieved May 13, 2008, from http://connect.educause.edu/Library/EDUCAUSE+Quarterly/HowStudentsDevelopOnlineLearningSkill.htm Siemens, G. (2004, Dec. 12). Connectivism: A learning theory for the digital age. Soloman, G; Schrum, L. (2007).Web 2.0: New tools, new schools. Eugene, OR: International Society for Technology in Education. Totaro, M., Tanner, T., Fitzgerald, J., &Birch, R. (2005). Faculty perceptions of distance education courses: a survey. Journal of College Teaching & Learning, 2 (7), 13-20. Wyrostek, F. (2008). Bridging The Onsite/Online Gap For Students/Faculty: Integrating Adobe Breeze Software To Teach Managerial Finance. Association of Business Information Systems. Federation of Business Disciplines Conference. Houston, TX. APPENDIX TABLE 1: ONLINE COURSE ASSESSMENT (MIS 166) Grades Distribution Avg. Pre-Test Score A B C D *F **FX W Avg. Post-Test Score Spring, 06 40 6 3 11 4 4 14 14 68 Fall, 06 42 6 7 11 - 3 10 5 70 Spring, 07 40 5 2 7 5 2 7 4 69 Fall, 07 45 5 8 7 5 3 7 10 68 Spring, 08 48 3 3 10 1 3 10 9 71 *F: Academically Fail *FX: Excessive Absence TABLE 2: AVERAGE ONLINE COURSE GRADES (MIS166) Grades Distribution Avg. Pre-Test Scores Avg. Post-Test Scores Percentage Improvement Spring, 06 40 68 28% Fall, 06 42 70 28% Spring, 07 40 69 29% Fall, 07 45 68 23% Spring, 08 48 71 23% TABLE 3: CODING SCHEME OF GRADES. Grade A B C D F/FX Code 5 4 3 2 1 TABLE 4: DESCRIPTIVE STATISTICS Spring, 06 Fall, 06 Spring, 07 Fall, 07 Spring, 08 Mean 2.053571 2.627907 2.71875 2.4 2.076923 Standard Error 0.187983 0.233001 0.277643 0.221108 0.218523 Standard Deviation 1.406732 1.527888 1.570584 1.48324 1.364679 Sample Variance 1.978896 2.334441 2.466734 2.2 1.862348 Count 56 43 32 45 39 TABLE 5: ANOVA: SINGLE FACTOR Groups Count Sum Average Variance Spring, 06 56 115 2.053571 1.978896 Fall, 06 43 113 2.627907 2.334441 Spring, 07 32 87 2.71875 2.466734 Fall, 07 45 108 2.4 2.2 Spring, 08 39 81 2.076923 1.862348 ANOVA Source of Variation SS df MS F F crit Between Groups 15.60645 4 3.901614 1.817023 2.414642 Within Groups 450.9238 210 2.147256 Total 466.5302 214