Computer-managed Instruction: Evaluation of Alternate Methods of Technology Integration in Higher Education

Educational Technology & Society 3(1) 2000
ISSN 1436-4522
Resumo nos Portugueses
Computer-managed Instruction: Evaluation of Alternate Methods of Technology Integration in Higher Education
Padma G. Anand
Slippery Rock University, Slippery Rock, PA, USA
Tel: +1 724 738 2271
Fax: +1 724 738 2280
Padma.Anand@sru.edu
Adi Zaimi
Project Consultant
Psychology Department
Rutgers University
Newark, NJ 07102 USA
ABSTRACT
The purpose of the present project was to compare the impact of using alternate methods of technology in under-graduate and graduate classes. Despite the demand for the integration of technology in higher education, the authors observed that some students could show resistance to the adoption of this approach. Thus, it is recommended that teachers apply the concept of Usability Testing during the design and implementation phase of technology. Pilot studies have indicated that thoughtful planning and evaluation of technology integration during the implementation phase can reduce the resistance towards the use of computers by students, and often results in higher achievement and more favorable attitudes. The current project focused on reducing barriers to technology integration by obtaining on-going user feedback about the technology integration and by continuously updating the instructional resources provided via technology. The feedback obtained from students revealed very clearly that for the successful integration of technology, the differential needs of the traditional as well as adult learners should be addressed.
Keywords: Computer-managed instruction, World Wide Web, Software Evaluation, Adult learner
Introduction
In order to enable students to function successfully in the twenty-first century, there is an immediate urgency to integrate technology into instruction. Teachers are constantly urged by the various accreditation agencies to integrate technology in higher education. The ever-changing work environments have resulted in an increasing number of adult learners attending higher education classes to improve their skills (Meade, 1998). Various learning theorists actually support the use of technology to enhance the learning and motivation of students (Bigge & Shermis, 1999). Further, rapid changes brought on by technological developments have transcended the barriers of time and space, thereby opening the doors for teachers to try innovative methods of teaching and learning. However, such changes have also raised several issues relating to teaching and learning. For example: How do we know that innovative and alternate methods of teaching are actually effective? If they are effective, are there some sub-groups of students who would benefit either more or less by the same tool? Issues like this require that technology integration is gradually incorporated, and that student learning and teaching effectiveness is continually monitored. An opportunity to address an important issue came about in the 1998 academic year when CMIE (Computer Managed Instruction & Evaluation) courseware was developed and simultaneously field-tested during the developmental phases. Another opportunity came by way of a university grant to promote the development of a Web-based instructional package. Web-based self-instructional materials were developed for two of the courses (one undergraduate and one graduate) taught by the author.
Field-testing results relating to students‘ experience with the use of technology revealed positive as well as negative student reactions. The instructor inferred that integrating technology, with the end-user in view appeared to be an appropriate strategy to overcome user resistance. Several writers have stressed the need to test technology resources during the design phase by collecting formative evaluation data about Usability Testing (Gould & Lewis, 1985; Reisman & Carr, 1991).
Technology Integration Methodology
The primary goal of the instructor was to create positive student attitudes towards the use of technology. The secondary goal was to empower the student to take charge of learning and make appropriate decisions. Adopting the concept of Usability Testing, an attempt was made to follow a systematic method of assessing student needs and perspectives during technology integration. Undergraduate students enrolled in the Teacher Education program and graduate students in counsellor training programs participated in this project. Interviews with pilot students had revealed that computer anxiety due to lack of prior use (computer illiteracy) and lack of access to computer resources were the major barriers to using technology. Hence, during the initial phases of integration, the focus was on adopting user-friendly ways of integrating technology. The technology resources were provided after a brief period of instruction and orientation relating to the instructional unit. Excessive lecturing was avoided, and a collaborative setting was arranged. Students were provided with hands-on training and other resources via handouts and videotapes throughout the semester. Working closely with students helped integrate technology from a student’s perspective. Interview questions focused on "How would you want your teacher to integrate technology in your classroom? If your suggestion is feasible, accommodations will be made to meet it." Based on student feedback, technology integration occurred gradually. In addition to modeling technology via whole-class instruction, the instructor began requiring the use of e-mail to submit assignments and collect handouts and also the Internet to obtain course materials and information about grades. The students began to receive immediate feedback via technology, and reacted very positively to these changes. While continuing to integrate these resources, we also began developing custom-designed software, which focused on incorporating special self-paced instruction and self-evaluation materials as alternates for some of the materials in the course. Students were constantly involved in contributing suggestions for the software development.
Technology Resources Used
Two major and several minor technology resources were made available for students.
The custom-designed Computer Managed Instruction and Evaluation (CMIE): CMIE provided opportunities to the student to review content, evaluate learning, and a variety of other options. CMIE essentially focused on helping the student to evaluate his/her learning, with a view to provide specific feedback to overcome the weaknesses. The CMIE also served as a tool for the instructor to obtain on-going feedback about student performance. This information was incorporated in instruction by either addressing these topics in classrooms again, or by providing remediation to the students who needed them. Thus, the CMIE technology actually assisted the teacher in addressing the diverse needs of the learners on an ongoing basis.
Web-based instruction module: A special web site was developed to provide self-instruction & evaluation resources pertaining to a major content area.
Additionally, students also had access to other resources via the instructor‘s homepage, the university networks, and e-mail. Most of the course materials were provided via these resources. Also, an additional incentive to use technology was created by posting grades within 12-24 hours after the completion of major tests and assignments.
Formative evaluation, and follow-up accommodations to address the differential needs of traditional & adult learners
Technology integration involved conducting regular formative evaluations to obtain feedback about the software usability and other aspects. The impact of such a student-centered approach was essential to develop resources that addressed the unique needs of the learner. We involved the students in the field testing process and motivated them by providing immediate changes in the program. In the following couple of weeks, students could actually see that their ideas were being incorporated and appropriate changes made. Students reacted very positively to these accommodations. For example, in one of the field testing sessions in spring 1998, after the students observed that their ideas were being accommodated, they felt very satisfied to see their feedback being incorporated in the CMIE. On all of the 4 evaluation items (relating to immediacy of feedback, helpfulness in comprehending course learning material, etc.) 100% of the students (N=20) responded positively. Also, the most positive feature mentioned in open-ended responses in the questionnaire as well as student evaluation of the instructor was the integration of CMIE in the course. Being a student-driven project, it is not at all surprising that 100% of the students in all of the four courses that I have taught this semester have recommended that I continue the use of the technology resources.
During the implementation of the custom-designed software (CMIE and WBI) the instructor also began to observe that several adult and a few traditional students actually liked these technology resources, but some of them became very frustrated that they could not make use of all the resources that I provided. The non-traditional students began to communicate to me in person as well as in the evaluations that lack of access to technology resources was a major problem for them. The graduate students (almost 75%) who were able to use the Internet were very happy that the material could be accessed anywhere, at anytime, and that they could work in a personalized setting at home. The remaining students with no access to the Internet, and also a few of those with the access frequently mentioned time as one the major barriers. In order to address this need, some of the technology-aided instruction sessions took place either in the class or in the lab where students were given on-line computer time to work on the instructional materials. This also resulted in creating learning opportunities and providing technology resources to all of the students.
Overall, keeping the students’ perspectives in mind, three specific steps were followed. First, special arrangements were made to provide course materials via the local campus network so that the students could access the resources outside of the classes at their own convenience. Several adult students appreciated this opportunity. Second, course materials and other learning resources were made accessible via my Homepage on the World Wide Web, thereby enabling off-campus students to access the resources. Third, the classroom assignments required the students to use technology in varied collaborative settings. This approach resulted in more positive reactions than all of the previous approaches.
Results
During the initial implementation phase, majority of the traditional students indicated negative reactions (especially to Web-based resources) and preferred to go back to traditional mode of instruction via lecture. The majority of adult students however reacted positively to being able to access the resources via the Internet. Traditional students began to express the reactions that I was avoiding my teaching responsibilities by delegating instructional activities to the computer! At one point, the use of technology had to be terminated. Some students felt suspicious that they were being experimented upon as laboratory subjects. The open-end comments of the learning log also revealed that several students would rather attend classes and listen to lectures, than study for themselves. This was an unexpected observation in that students typically view lectures negatively. The negative reactions of the students could be attributed to the fact that the course content has been traditionally perceived as abstract and difficult to learn. The lesson learned here was that technology cannot replace teachers, and traditional students need to be interacting more closely with teachers while using technology. Luchini has also addressed this issue (1998). She has rightly suggested that formal research must be conducted to identify the type of students to whom technology works well.
On a positive note, students accessed technology more frequently than before. The homepage counter readings (one of the evaluation measures) indicated that before the project implementation, in a period of seventeen months, the homepage was visited less than 500 times. After project implementation, in a period of one month, the homepage was visited over 500 times. The most striking feature observed during the implementation phase was the difference in attitudes between that of the traditional learner and the adult learner. It appears that developmental differences do affect the motivation and other features involved in the use of technology. The instructor observed that the typical traditional student is not used to working independently, and requires some amount of initial teacher-centered structured learning environment. Teacher-student interaction is important to all students, but that it is more critical for the traditional student. Adult learners, on the other hand, were constantly requesting me to put the resources on the Internet for independent and easy access. It appears that the greatest positive impact of adaptive technology occurred among adult learners, thereby indicating that technology is a great resource to attract adult students to participate in the life-long learning process. Thus, addressing equity and excellence in the learning environment requires that the needs of all the students, both regular as well as non-traditional, be addressed. It is interesting to note that at this point due to lack of time and money, there is an inability to respond to CMIE revision requests. The next phase of implementation is planned (through grants) in two stages, namely, development of Web-based resources for at least two more units in spring 2000, and conversion of CMIE to adapt to the Internet.
Recommendations
Technology has unlimited potential to enhance instruction, and it is here to stay. Hence, technology-related activities should become an integral part of higher education, as they tend to provide opportunities for students to interact with resources that they might decide to use later in future. Such experiences also help the students understand the limitations of using technology, besides conveying the message to them that computers cannot replace teachers! Teachers should combine traditional instruction with technology-related experiences so that students view technology with a more positive attitude. Additionally, student feedback should be continually sought while designing and implementing technology, hopefully resulting in a better appreciation of the purpose for which technology was being used. Most important, adult learners tend to benefit most from technology. Hence, in order to ensure equity of learning opportunities, and to address the diverse needs of the learners, every attempt must be made to integrate instructional technology in student-centered and teacher-supported collaborative learning environments.
References
Bigge, M. L. & Shermis, S. (1999). Learning theories for teachers, NewYork: Longman.
Gould, J. D. & Lewis, C. (1985). Designing for usability: Key principles and what designers think. Communications of the ACM, 28 (3), 300-311.
Luchini, K. (1998). Problems and potentials in web-based instruction, with particular focus on distance learning. Educational Technology & Society, 1 (1),
http:/ifets.ieee.org/periodical/vol_1_98/informal_summary_katy_luchini.html
Mead, D. G. (1998). The information revolution could create jobs. In D. L. Bender (Ed.) The information revolution: Opposing viewpoints, San Diego, CA: Greenhaven Press, 121-130.
Reisman, S. & Carr, W. A. (1991). Perspectives on multimedia systems. IBM Systems Journal, 30 (3), 280-295.
Skinner, B. F. (1968). The technology of teaching, New York: Appleton-Century Crofts.

_xyz