Introduction
The central part of this feature is a lecture entitled Motivation, Identity and Learning by Dr Martin Lamb of the University of Leeds. Firstly, in order to explain the relevance of the topic to langauge testing, I introduce the notion of test taker characteristics, and explain why motivation is a characteristic in which language testers are interested. Dr Lamb then brings us up to date with motivation theory and research, which has changed greatly in the last few years. There are two handouts to download before you watch the lecture.
Test Taker Characteristics
Davies et al. (1999: 208) define test taker characteristics as:
"A wide range of variables, any of which may significantly influence test performance, hence producing measurement error and affecting the validity of the assessment. These may inlcude language background, age, sex, educational background, background knowledge, [and] affective reactions to test taking...."
Kunnan (1995: 1) also includes
"...cognitive, psychological and social characteristics, such as learning strategies and styles, attitude and motivation, aptitude and intelligence, field dependence and independence, extroversion and introversion, and anxiety, personality, and risk taking."
Everything in the list is a "characteristic" or "quality" of the test taker. Some of the characteristics are likely to fluctuate over time, while others (such as personality traits) may be stable. However, the language tester has no control over these characteristics, either generally, or at the time when a person takes a test. Yet, we are interested in these characteristics because they can affect test scores in predictable ways.
For example, Berry (1994) looked at the effect of the personality traits of introversion and extroversion on paired speaking test performance when introverts were placed with other introverts, and with extroverts. She discovered that introverts performed worse when paired with extroverts, but always performed better in homogenous pairs, and in tests with examiner interlocutors. Berry (1997) went on to question whether paired tests are fair to introverts. This research from the 1990s is highly relevant today, as the paired format is becoming very common in speaking tests. This is because of the perceived usefulness of the format to measure complex constructs such as 'interactive competence' (see, for example, the special issue of Language Testing in 2009 on the paired format). What we need to ask ourselves as language testers is whether we should use the paired test with all test takers, arrange the pairs deliberately to avoid bias against introverts, or whether we adjust the scores of some test takers to make up for adverse test conditions.
These are not easy issues to address. While pairing test takers with others of similar personality types seems the most equitable solution, the practicality of actually doing it is questionable. But the reason why we're concerned with test taker characteristics is that we try, as hard as possible, not to put anyone at a disadvantage when we can be sure that a particular characteristic effects test performance. It's a matter of avoiding potential bias. So where does motivation fit into this?
Motivation as a Test Taker Characteristic
All users of test scores make the basic assumption that the score reflects the ability of the test taker on the construct(s) that the test is designed to measure. We have seen with the example of introversion that this assumption may be threatened. The assumption is also threatened if the test taker is not motivated to do well on the test. If the test taker doesn't really care about the outcome the score will not reflect their ability.
Again, there are some things that are beyond our control, and for which language testers can take no responsibility. For example, the test taker may have been forced to take the test by an employer (or parent!). There just isn't much that we can do about this. However, if we discover that an identifiable group of test takers have reacted negatively to test content or format, and that this reaction is related to their motivation in learning the language and taking the test, it may be something that is potentially within our control.
Henning (1987: 96) referred to this as response validity. He explained it in this way (ibid., 92):
"If the examinees do not approach the testing situation in the expected manner, the results may prove to be invalid. This might occur if the examinees are insincere, misinformed, or hostile with regard to the test or the testing situation."
What Henning saw clearly is that test designers should do everything within their power to harness motivation; the language tester wants the test taker to do the best they can on the test. We should therefore try in as far as we can to reduce the possibility of a 'hostile' or negative reaction to the content and format.
When motivation has been studied as a test taker characteristic it has been treated as 'instrumental' or 'integrative' (eg., Johnson and Krug, 1980; Kunnan, 1995: 8), following the classic definition by Gardner and Lambert (1972). The research questions investigated have related to what kind of motivation (within this theoretical framework) leads to purposeful engagement with the test and/or higher scores. The general claim has been that integrative motivation is more advantageous, as summarized in this web page from the University of Michigan.
Over the last few years however, motivation has been reconceptualised. It is now argued that 'integrative motivation' makes little sense in a globalizing world in which English is the medium of communication between speakers of many languages, from many cultures, for many purposes (Lamb, 2004). The desire to 'integrate' with the first language community hardly makes sense any more. Yet, we have hardly begun to explore what this means for how test takers react to test content, or task types, especially where these reflect cultures or norms of communication that do not fit easily with the language learning motivation of identifiable groups of test takers.
The more recent approaches to studying motivation (e.g., Dornyei and Ushioda, 2009) focus upon the learner's sense of identity, and concept of their 'second-language-self', which will affect their attitude toward language learning, and test taking. If we wish to achieve 'response validity' we need to reasses motivation as a test taker characteristic in order to understand how and why some test takers may have negative reactions to our tests.
I do not wish to start defining 'identity' or 'L2 self', as this is best left to Dr Lamb. As you follow the lecture I hope that you will think about the implications of the new concepts in motivation theory for language testing. What research is needed into 'response validity' in a globalized world? You may wish to make notes so that after listening to the lecture you can think about potential research projects in this area with colleagues.
The Lecture
I am very grateful to Dr Martin Lamb for allowing me to record this lecture, and to make it available through this web site.
Before playing the video, please download these documents:
Berry, V. (1994). The Assessment of Spoken Language under Varying Interactional Conditions. Washington D.C.: ERIC Document ED386065. Available online: http://eric.ed.gov/PDFS/ED386065.pdf.
Berry, V. (1997). Ethical Considerations when Assessing Oral Proficiency in Pairs. In Huhta, A., Kohnen, V., Kurki-Suonio, and Luoma, S. (Eds.) Current Developments and Alternatives in language Assessment. Jyvaskyla: University of Jyvaskyla and University of Tampere Press, 107 - 123.
Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., and McNamara, T. (1999). Dictionary of Language Testing. Cambridge: Cambridge University Press.
Dornyei, Z., and Ushioda, E. (Eds.) (2009). Motivation, Language Identity and the L2 Self. Bristol: Multilingual Matters.
Gardner, R. C. and Lambert, W. E. (1972). Attitudes and motivation in second language learning. Rowley, Mass: Newbury House.
Johnson, T. R., and Krug, K. (1980). Integrative and Instrumental Motivation: In search of a measure. In Oller, J. and Perkins, K. (Eds.) Research in Language Testing. Rowley, Mass: Newbury House, 241 - 249.
Kunnan, A. J. (1995). Test taker characteristics and test performance. A structural modeling approach. Cambridge: Cambridge University Press.
Lamb, M. (2004). Integrative motivation in a globalizing world. System 32(1), 3 - 19.
