Quick Tip: WARP V—5 Principles of Language Assessment
by Baburhan Uzum
Audience/Level: ESL/EFL teachers and teacher trainers of all levels
Assessment is often a fearsome topic for students and teachers alike. In most English as a second language and English as a foreign language contexts, the assessment of students’ second language skills is usually designated to a unit or testing center, and teachers do not have to worry about creating major tests in their daily activities. However, this does not mean that teachers never have to assess students’ language skills and never have to prepare their own assessment materials—they do, and they must adjust their instruction accordingly.
A common practice is to find online quizzes for in-class diagnostic purposes or as instructional activities. With the advent of technology, there is no shortage of assessment materials, and one can easily find many free online quizzes at various lengths in a given topic, but the real challenge is to choose valid and reliable ones in the abundance of such materials.
In this Quick Tip, I present an easy-to-remember framework so that teachers can evaluate the materials they find online by quickly looking for specific qualities. I have named this framework "WARP V," inspired by the well-known TV series Star-Trek. The activity name stands for the five major qualities of second language assessment: washback, authenticity, reliability, practicality, and validity (Brown & Abeywickrama, 2010). WARP V is an escalated speed in which a ship improves its performance to its peak, but not every ship can handle the WARP V speed—the weaker ships fall apart. Similarly, when teachers design or find a test, they can put it across the WARP V analysis and see how the test is doing across these benchmarks. If the test survives the WARP V analysis, it’s an effective test.
Benchmark |
Definition |
Criterion |
Washback |
Washback is the learning from the assessment, either while preparing for it or remembering certain assessment experiences (e.g., going over the answers after the test). Positive washback refers to expected test effects. Negative washback refers to the unexpected or harmful consequences of a test. |
Are students learning from this test by taking it or preparing for it?
Does the test have any potential detrimental effects? |
Authenticity |
Authenticity may be achieved by making the test more natural and meaningful. |
Is the language in the test as natural as possible?
Are items contextualized rather than isolated?
Are the topics meaningful (relevant, interesting) for learners?
Do the tasks approximate real-world tasks? |
Reliability |
A reliable test is consistent and dependable. If you give the same test to the same student or matched students on two different occasions, the test would yield similar results. |
Would the students score the same if they took the test again?
Are there any outside factors such as an illness, street noise, technology troubles, or other physical conditions that might impede students’ performance? |
Practicality |
An effective test is practical. |
Is the test affordable?
Does it stay within appropriate time constraints?
Is it easy to administer?
Does it have a scoring/evaluation procedure that is specific and efficient? |
Validity |
Test validity refers to the extent to which inferences made from assessment results are appropriate, meaningful, and useful in terms of the purpose of the assessment. |
Is the test measuring what it is intended for or something else?
Is the test measuring the intended language skill or construct? |
The WARP V framework provides teachers a method to quickly evaluate assessment instruments before adoption and gives them a common language to talk about and reflect on their observations on a given assessment medium. For an example of the WARP V framework in use, see Appendix A (.pdf), and for a blank WARP V template, see Appendix B (.docx).
Reference
Brown, H. D., & Abeywickrama, P. (2010). Language assessment: Principles and classroom practices. White Plains, NY: Pearson Education.
Baburhan Uzum is an assistant professor in the Department of Language, Literacy, and Special Populations at Sam Houston State University, Huntsville, Texas, USA. His research interests include multicultural education and second and foreign language teacher education.
TESOL Blogs
Interested in writing a blog for TESOL?
Read the submission guidelines and send us your post!
Check out a few recent TESOL Blog posts:
TESOL President’s Blog: SETESOL Conference, by Ester De Jong
One partnership that TESOL has with its 100+ affiliates is the Affiliate Speaker Program. This program encourages TESOL’s affiliate partners to select a TESOL board member to come and speak at their conference. As part of this program, I had the pleasure of visiting Argentina when I was on the board of directors. Luciana de Oliveira, our current president-elect, wrote a wonderful reflection on how powerful these experiences are for both the affiliate and the board member. Read more.
|
Challenging Leadership Stereotypes Through Discourse, by Kevin Knight
Have you considered how the ESP Project Leader Profiles (36 published to date with projects on six continents) might be viewed as research data that can give us a deeper understanding of leadership itself? Please keep this question in mind as I share with you in this TESOL Blog post a new book that explores leadership using discourse analytical approaches. Read more.
|
The Betty Azar Travel Grants: Gobinda Puri at TESOL 2017, by Sherry Blok
Did you know Betty Azar generously supports TESOLers in their professional development annually? This year, we are very proud to have renamed the award the Betty Azar Travel Grant for Practicing ESL ESL/EFL Teachers because she deserves that recognition. This is the second year I’m coordinating the award and I can attest that volunteering on the TESOL Awards Council is an excellent way to connect with Betty, but also the award recipients all the committed volunteer reviewers. One special recipient of the award is Gobinda Puri, an educator from Nepal. Betty’s generous donation helps educators like Gobinda make a difference with his learners and community. Please read on as Gobinda shares his experience from the 2017 TESOL International Convention and Expo in Seattle. Read more.
|
TESOL Bookstore
Featured Resources from TESOL Press
Teaching English to Chinese Learners in U.S. Colleges
Ke Xu and Liglan Cao
Gain an in-depth understanding of the Chinese learner to improve your teaching practices and help your students succeed. Take a step back from familiar contexts and examine how institutional structures, the relationship between the U.S. and a student's country of origin, and the student's motivations and attitudes all play a role in their English learning.
TESOL Voices: Adult Education
Tim Stewart, Editor
Follow the triumph and trials of real-life students as they learn English, whether they're pursuing it for their vocation, as immigrants to a new country, for travel, or for their very survival.
More Than A Native Speaker: An Introduction to Teaching English Abroad, Third Edition
Don Snow and Maxi-Ann Campbell
In this newly updated third edition, learn step-by-step how to effectively teach English abroad. Gain valuable tips and resources for teaching in an unfamiliar educational system, working with students of varying ages and skill levels, and adapting to life in a different culture. A rich array of online resources and activities included.