English is currently spoken by more nonnative speakers than
native speakers (Crystal, 2004). Therefore, in many target language use
(TLU) domains, English use is not limited to native speakers. In higher
education in the United States, students encounter varieties of English
spoken by international students, international teaching assistants
(TAs), and faculty (Gorsuch, 2003; Major, Fitzmaurice, Bunta, &
Balasubramanium, 2005). The existence of different varieties of English
leads to questions about the usefulness (Bachman & Palmer, 1996)
of listening comprehension (LC) tests based only on a native speaker
variety. This lack of correspondence between TLU tasks and test tasks
limits the generalizability of tests and the validity of inferences made
about test takers’ listening ability.
Major, Fitzmaurice, Bunta, and Balasubramanium (2002) pointed
out that the relationship between accent and performance on LC tests is
unclear. Nonnative varieties have been marked as the factor most
contributing to comprehension difficulty (Goh, 1999). Flowerdew (1994,
p. 24) furnished a summary of studies, some of which demonstrated
listener advantage when listeners were familiar with the accent of the
speaker, and some showed that accent familiarity provides no advantage
While research has looked at the effect of question preview
(Sherman, 1997) and the influence of reading (Friedman & Ansley,
1990) on LC tests, the effect of accents (Ortmeyer & Boyle,
1985; Wilcox, 1978) has received limited attention. Most recently,
Harding’s (2008) study found no significant difference in test takers’
LC performance when accented speech was used.
Another factor that confounds the use of nonnative varieties in
listening comprehension is the attitudes toward accented speech
(Lippi-Green, 1997). Nonnative speakers are often categorized as
learners or uneducated or deficient speakers and stereotyped solely on
their accents (Brennan & Brennan, 1981; Cargile, 1997). Harding
(2008), however, found that listeners’ views on accented speech differed
according to the purpose of the listening activity.
The following research questions guided this study:
- Does the use of nonnative varieties of English reduce comprehensibility and affect test performance?
- Is there any interaction between test takers’ native
language and the variety of English used in test input?
- What are test takers’ perceptions and attitudes toward nonnative varieties of speech in LC tests?
Data for this study were obtained from 110 test takers: the
Koreans and Brazilians represent English as a foreign language (EFL)
situations, and the Sri Lankans represent an English as a second
language (ESL) situation, typical of students entering U.S.
universities. Students were taking courses in various disciplines and
were simultaneously enrolled in English classes at universities in their
respective countries. Based on the performance on placement tests and
in English classes, the Korean (N = 36) and Brazilian
(N = 33) test takers were identified as having
high-intermediate English proficiency, and Sri Lankan test takers
(N = 38) as having low-intermediate proficiency.
The LC test consisted of eight listening texts, each followed
by three or four comprehension questions for a total of 31
multiple-choice items. These listening passages were retired versions of
the TOEFL exam (TOEFL Institutional Testing Program), with reliability
estimates of 0.9 (TOEFL Test and Score Manual).
The speakers were TAs at leading universities in the United
States: two native speakers of Chinese, one Korean, one Sri Lankan, and
four native speakers of American English. The nonnative-English-speaking
TAs had a Test of Spoken English (TSE) score of 50 or higher and had
passed the Spoken Proficiency English Assessment (SPEAK) at their
Test takers were randomly assigned to two groups based on their
availability to take the test. Test takers in both groups heard Text 1,
which was about American history and was read by an American TA. This
was to help students familiarize themselves with the test. For the other
passages the speakers were counterbalanced: Group 1 listened to passage
3 read by a Chinese speaker while Group 2 listened to the same text
read by a Korean speaker.
Other instruments employed in the study included a background
questionnaire administered at the beginning of the test to gather
demographic information. Then, after each listening test, participants
were asked to evaluate speaker comprehensibility using a 7-point Likert
scale ranging from “strongly agree” to “strongly disagree.” After
completing all eight LC tests, a final perception questionnaire was
given to learn about test takers’ attitude toward the use of nonnative
speaker input in LC testing.
A one-way analysis of variance (ANOVA) was conducted with the
students’ performance as the dependent variable to answer research
question 1. The difference in the mean performance across the two groups
was not significant F (1, 0.565) = .452, p = 0.095, indicating that the use of nonnative
speaker input did not make a difference in the participants’
Research question 2 asked if test takers would perform better
when the speaker shared their first language (L1). A two-way ANOVA was
conducted, where students’ performance was the dependent variable, and
the input language and test takers’ L1 were independent variables. The
interaction between the speaker’s L1 and the test takers’ L1 was not
significant F (6, 0.103) = 0.996, p = 0.095, suggesting that, for example, Sri Lankan students did
not perform significantly better than expected when the speaker was
from Sri Lanka.
For research question 3, responses to the comprehensibility
questionnaire did not provide any clear indication of perceptions of
nonnative speech as test takers were not consistent in their judgment
and were oftentimes unable to identify the cause of difficulty in
comprehensibility. The attitude questionnaire, however, revealed that
more than 62 percent of test takers preferred a native variety. While
test takers seemed generally favorable of nonnative varieties (46
percent saying Yes because in real life they also hear nonnative
varieties), when asked if using nonnative varieties in listening tests
made a difference in test performance, 48 percent seemed certain that
performance is better with native English. When asked if the type of
speech input may cause test bias, almost 60 percent said that they
thought Japanese test takers would better understand Japanese-accented
English than would test takers from other countries. Finally, test
takers were asked if they would prefer a native variety on a listening
test even if a nonnative variety were easy to understand. They
overwhelmingly responded Yes (65%).
Test performance data in this study suggest that nonnative
varieties of English can be used as listening test input while attitude
questionnaire results go counter. Although some test takers acknowledged
the presence of nonnative or accented speech in their TLU domains,
overwhelmingly they did not have a favorable attitude toward nonnative
speech in tests. These findings support most of the previous research in
Even though the test performance findings suggest the
possibility of using nonnative varieties of English as test input,
further research is necessary because of a number of limitations. First,
the study was limited to three nonnative varieties of English. Second,
although questions in each listening test had almost the same average
difficulty (0.54 to 0.57 item difficulty), it was not possible to
isolate the speaker input (nonnative English/accented English) as the
only within-subjects factor. Another limitation of the study was the
test takers’ level of language proficiency. In this study, all three
groups were fairly similar, but different language proficiency levels
may cause test takers to perform differently.
Until research shows that the use of nonnative English as input
is irrelevant to the construct of listening, we may be unable to use
such input in testing, especially in high-stakes tests such as TOEFL and
IELTS. In order to address some of the issues of test usefulness,
validity, and authenticity in particular, we should perhaps use
nonnative varieties/accented English in low-stakes tests such as
placement, diagnostic, or achievement tests in classrooms. Furthermore,
introducing nonnative varieties of English in teaching listening and
speaking skills will expose learners to a wider range of linguistic
models and may help move learners away from the attraction of standard
varieties. As nonnative speech gains recognition among language users
and test takers, we hope the attitudes and perceptions toward the use of
these varieties of English will change.
Bachman, L., & Palmer, A. (1996). Language
testing in practice. Oxford, England: Oxford University
Brennan, E., & Brennan, J. S. (1981). Measurements of
accent and attitude toward Mexican-American Speech. Journal of
Psycholinguistic Research, 10, 487-501.
Cargile, Aaron C. (1997). Attitudes toward Chinese-accented
speech: an investigation in two contexts. Journal of Language
and Social Psychology, 16, 434-443.
Crystal, D. (2004). The language revolution. Cambridge, England: Polity.
Flowerdew, John. (1994) Academic listening: research
perspectives. New York: Cambridge University Press.
Friedman, S. J., & Ansley, T. N. (1990). The influence
of reading on listening test scores. The Journal of
Experimental Education, 58(4), 301-310.
Goh, C. (1999). How much do learners know about the factors
that influence their listening comprehension? Hong Kong Journal
of Applied Linguistics, 4(1),
Gorsuch, G. (2003). The educational cultures of international
teaching assistants and U.S. universities. TESL-EJ, 7(3), A-1.
Harding, L. (2008) Accent and academic listening assessment: A
study of test-taker perceptions. Melbourne Papers in Language
Testing, 13(1), 1-33.
Lippi-Green, R. (1997). English with an accent. London: Routledge.
Major, C. R., Fitzmaurice, S. F., Bunta, F., &
Balasubramanium, C. (2002). The effects of nonnative accents on
listening comprehension: Implications for assessment. TESOL
Quarterly, 36(2), 173-190.
Major, C. R., Fitzmaurice, S. F., Bunta, F., &
Balasubramanium C. (2005). Testing the effects of regional, ethnic, and
international dialects of English on listening comprehension. Language Learning, 55(1),
Ortmeyer, C., & Boyle, J. P. (1985). The effect of
accent differences on comprehension. RELC Journal, 16(2), 48-53.
Sherman, J. (1997). The effect of question preview in listening
comprehension tests. Language Testing, 14(2), 185-213.
Wilcox, G. K. (1978). The effect of accent on listening
comprehension: A Singapore study. English Language Teaching
Journal, 25, 239-248.
A native of Sri Lanka, Priyanvada Abeywickrama teaches
courses in TESOL methodology and curriculum and assessment in the MA
TESOL Program at San Francisco State University. Her research in
language assessment specifically examines issues of