A Comparative Study of the Standard Cloze Test and the Multiple Choice Cloze Test

我爱英语网  http://www.52en.com
A Comparative Study of the Standard Cloze Test and the Multiple Choice Cloze Test
Bai Ruixue
0.Introduction
Cloze tests are prose passages, usually a paragraph or more in length, from which certain words have been deleted. In such tests, testees are required to supply the missing words, using “linguistic knowledge, textual knowledge and knowledge of the world” (Cohen 1980:97).
This paper aims at a brief comparison between the standard cloze test and one of its alternate forms, the multiple-choice cloze test. The first part is intended for an introduction of the birth and development of cloze as a relatively new type of testing. The remaining part, the main body of this paper, is centered on a comparison between the two alternative forms mentioned above, revealing their own advantages and disadvantages respectively.
This paper is mainly based on library sources, supplemented by my own experiment data.
1.Background
1.1 Theoretic Background of Cloze Tests
Initially, the cloze tests, which were introduced by Taylor (1953), were used only to assess the readability of texts in the reader’s native language. It originates from Gestalt psychology, which holds that integrity of human consciousness is the most intrinsic feature of psychology. Therefore, it is a sub-consciously-motivated tendency to fill up the empty when one is observing an object or picture. After Taylor, the cloze tests came to be used in L2 testing, but it did not get popular until 1970s when overall competence was gaining attention.
The idea of overall competence can be traced back to the Unitary Competence hypothesis proposed by Oller (1979); Before that time tests were devised to measure performance or recognition of separate sounds, specific grammatical features, or vocabulary items. What Oller said, in brief, was that language proficiency is indivisible, that tests only differ in their effectiveness at measuring this one factor, and that the elaborate apparatus of dimensions and tests used by the psychometrists could be replaced by one test which would directly tap the single indivisible faculty of overall proficiency, one of the most well-known being cloze tests, in which every nth word has been deleted. Among the deleted words, lexical ones take up 65 percent to 75 percent and functional ones, 25 percent to 35 percent.
1.2 Forms of Cloze Tests
According to Hughes (1989), “there was time when the cloze procedure seemed to be presented almost as a language testing panacea”, because it is integrative – that is, it requires students to process the components of language simultaneously, much like what happens when people communicate. An integrative method, it was also quite easy to prepare, administer and rather easy to score.
However, proficiency tests such as the cloze do have some limitations. For one thing, they are not sensitive tests of short-term gains. A good achievement test could show big improvement on question tags studied over a two- to three-week period. But a proficiency test generally would not show much if any improvement. What’s more, controversy arises in scoring. There are two possible ways to score cloze tests. One is to give credit for only the exact word from the story, called the exact word method. Another is to allow full credit for equivalent words as well, namely, the acceptable word method. The first method is quite economical, but might discourage the productive skills of students. And, although the second one is generally considered to be more “fair”, it is difficult for teachers who are non-native English speakers to choose acceptable equivalent words.
Problems in scoring give rise to multiple-choice cloze tests, in which optional items are provided as clues. The following parts will be focused on the effects of the standard cloze tests (S cloze) and the multiple-choice cloze tests (M-C cloze), in which an experiment is conducted.
2. Experiment[1]
2.1 Methods and Materials
In my experiment, I choose sixteen first-year students from my university as testees. All of these students are top or nearly top ones in their classes, divided into two groups on the basis of an equal average level. As the first step, two short passages with multiple-choices are selected as testing materials (materials A) for Group 1 and Group 2 respectively.[2] Next, the two groups exchange their materials with the optional items canceled out as standard cloze tests (materials B). Testees are required to complete the two tasks within half an hour. The two chosen passages are given below:
Passage 1
Our aeroplane was just beside the airport building. It did not look too strong to me, but I decided not to think about such things. We saw luggage going out 1____ it on trolleys and being loaded from 2____ the aircraft. Next, three men and three girls, all in uniforms, went over to the plane and 3____ it. Over the loudspeakers we were 4____ the plane was ready to leave and were asked to walk 5____ to it. Everybody moved quickly in order to 6____ the seats they wanted. I was 7____ to get a seat near the tail, but the plane looked 8____ inside than it had from outside. I fastened my seat 9____ before we took off and tried to 10____ my nervousness.
1.a.at b.for c.to d.above
2.a.inside b.beside c.behind d.under
3.a.reached b.flew c.climbed d.entered
4.a.asked b.noticed c.announced d.told
5. a.out b.along c.in d.by
6.a.fetch b.hold c.keep d.get
7.a.impossible b.incapable c.unable d,unsuitable
8.a.prettier b.stronger c.steadier d.smaller
9 a.belt b.string c.band d.ribbon
10.a.smooth b.decrease c.forget d.relax
Passage 2
Traffic is a perennial problem in Hong Kong. Over the years many suggestions have been put forward to ease transport difficulties. These 1____ from constructing wider roads and an underground railway 2____ to staggering hours for schools to open and close. One official report pressed 3____ for working hours to be 4____ out to relieve congestion 5____ peak periods. In England and other countries this system is called “staggering working hours” 6____ many business centers use it. It is 7____ that greater working efficiency can be 8____ , employees are happier because they can arrange convenient personal 9____ hours, buses and trains can be dispatched at suitable 10____, and so on.
1. a. differ b.range c.include d.change
2. a.system b.station c.terminal d.terminus
3. a.correctly b.immediately c.sympathetically d.strongly
4.a.taken b.spreaded c.spaced d. stretched
5. a.at b. by c.on d.for
6.a. because b.and c.by d.in
7. a.considered b.predicted c.claimed d.forecasted
8.a.reduced b.approved c.added d.achieved
9.a.departing b.evening c.working d.appointed
10.a.intervals b.sessions c.periods d.interims
2.2 Results
Results of the experiment are shown in the following diagram, with the full score being ten points. (To be appropriate for testees as freshmen, acceptable word method is adopted in S cloze tests.)
Group 1 (8 testees)
Group 2 (8 testees)
Type of material
A1(M-C cloze)
B2(S cloze)
A2 (M-C cloze)
B1 (S cloze)
Average score
7.25
3.75
5.75
2.75
3. Analysis
3.1 materials
Before analyzing the experiment results, a brief review of the material itself is in order. As a matter of fact, these two passages well point to cloze as an integrative method . Just take passage 2 (M-C cloze) as an example. Blank No.1 measures the mastery of vocabulary (“range”), No.2 involves a noun (“system”) that re-appears in later lines and can be predicted according to discourse knowledge, No.5 tests preposition collocation, and so on. In brief, the twenty empties in the two passages cover a wide range of knowledge ---- vocabulary, discourse, logical, collocation, etc. In this sense we can say cloze has its validity as an overall proficiency test.
3.2 S-cloze vs. M-C cloze
From the diagram we can see that students get higher marks in M-C cloze than in standard cloze. It seems to suggest that the latter is more difficult to handle. In reviewing the M-C cloze tests, several conclusions can be drawn:
The most obvious advantage of M-C cloze tests, of course, lies in the economy in scoring, which needs no further explanations. M-C cloze tests are creditable also in the sense that although optional items can be clues, they can also be confusing distracters and testees have to rule out distracters before making up their own choices. Some “correct” but not the “best” items are a major source of distracters. For instance, only two out of the eight examinees get the right answer to the last blank of passage 2: At first sight item C (“periods”) seems OK, both semantically and grammatically. However, although correct, it is not the best or the most suitable: In consideration of the sentence subject “buses and trains”, “intervals” is the best word. New words may also become distracters. Still in passage 2, words like “sympathetically”, “departing” might be new for freshmen and thus become obstacles in comprehension.
Of course, M-C cloze tests are in no way panacea. Its greatest drawback is that this technique tests only receptive skills rather than productive ones and thus performance on a M-C cloze test may give a quite inaccurate picture of candidates’ real ability. The person who can identify the correct items may not able to produce or use the correct forms when speaking or writing. Another problem is that guessing may have a considerate but unknowable effect on test scores. In the above experiment, 13 out of the 18 testees admit at least one answer comes out by guessing. Consequently, the scoring, although economical, has been weakened in validity since it does not make much difference between guessing or not. In addition to all this, the simple form of M-C cloze may facilitate cheating in tests.
On the other hand, so far as the standard cloze tests are concerned, both advantages and disadvantages are summed up here as well.
The primary strength lies in its effectiveness in measuring one’s ability to use language. Without items to give any clues, testees cannot come up with a right answer unless he has a good understanding of its usage. It is encouraging that among eight testees, three fill in the correct word “range” in the first blank of passage 2 (S cloze), an indicator that they can not only identify the word, but also freely use it. Apart from this, S cloze tests help to build up in one’s mind a notion of discourse unity: He has to have a careful reading of the passage in search of textual clues; Factors like lexical chain and logical unity may stand out in his effect of processing information.
As is already pointed out, there are problems concerned with scoring of S cloze tests. Besides, this type of test may be too hard for beginners. One writer illustrates that although he has never studied Italian, he can get much of the meaning from simple articles in Italian newspapers. But, he says, if any words were deleted form an article written in Italian, he would be unable to predict what to put in those blanks. In short, while he has some passive understanding of Italian, he lacks the productive skill that would be required in such tests. Realizing that many beginning students have a similar problem with English, he suggests using a M-C cloze format so that students might rely on their recognition of the needed word.
4.Conclusions
To wind up the loose ends, two conclusions can be reached here:
A. Generally speaking, cloze, as an integrative method, turns out to be an effective test with validity and reliability;
B. Both the traditional standard cloze test and its recent development, the multiple-choice cloze tests have advantages and disadvantages. They should be used for different purposes as testing methods. As training methods, they are to be conducted as complementary to each other.
Bibliography
1. Bachman, L.F. 1990, Fundamental Considerations in Language Testing, Oxford and New York: Oxford University press;
2. Baker, David. 1989, Language Testing: A Critical Survey and Practical Guide, Edward Arnold;
3. Madsen, H.S. 1983, Techniques in Testing, New York: Oxford University Press;
4. Hughes, Arthur. 1989, Testing for Language Teachers, Cambridge: Cambridge University Press;
5. 孙建东等，1996，改错与完形填充，南京：南京大学出版社。