It's that time of year again! Inspired by the TV and Radio shows that take a long hard look at the events of the last year, I turn my attention to the big stories of testing and assessment that hit the headlines in 2011. And I'm always surprised by how many there are, and how people get so excited and upset. Not to mention the fact that many of the issues keep coming back year after year. This is, inevitably, a personal selection. And on some of the questions I hold particular views. Although I would claim that these are informed, I know there will be other professionals who disagree with me for a whole range of reasons. And you may be one of them. But the key thing to remember is that we only make progress by expressing our views openly and engaging in debate, as this quotation from Mill (1859) makes clear: There must be discussion, to show how experience is to be interpreted.
Wrong opinions and practices gradually yield to fact and argument: but
facts and arguments, to produce any effect on the mind,must be brought
before it. Very few facts are able to tell their own story, without comments
to bring out their meaning."
Satellite Industries The year got off to a worrying start. In the last couple of reviews I've been commenting on the sham marriage industry in India, driven by the use of IELTS scores for immigration in Australia. Basically, legal firms have a side-line in matching girls who can pass IELTS with boys from rich families. She gets her education paid for, he gets a work visa. In this story from the Times of India the two families behind a sham marriage agreed that the couple would stay together until they got permanent residence permits, but the girl filed for divorce soon after arrival in Australia. When this was granted the boys family sued for fraud. This is a despicable satellite industry fuelled by poor government policy and a failure of testing agencies to issue warnings that their tests were not designed for making such decisions. This is one of the reasons why I have written about the need for sound test retrofit practices, which would help to eliminate such misuse and adverse consequences.
Technology & Biometrics Also in January was the first of two fascinating stories about Cambridge ESOL's technology adventures. It was reported in New Zealand that they had formed a partnership with Vital English to deliver online preparation courses for some of their examinations. This has always been highly contentious as there is always a potential conflict of interests when the people who write the tests also deliver the teaching or indeed the teacher training (see the discussion below). Less contentious was the announcement on PRS Newswire that Cambridge had teamed up with Aware Inc. to put biometric identification procedures into place for its tests. Given the growth in cheating worldwide, and the use of biometrics in the administration of the Pearson PTE, it was not unexpected to hear that other large players would follow suit.
Cramming This follows on nicely to the problems of "cramming" - not really learning the language or a subject, but simply preparing to take a high stakes test. This has been in the news all year, starting with the news in The Nation that the Thai government was going to tax cram schools to try to make them less popular! The statistics in this story, if true, are quite amazing. While in the United States the use of high-stakes tests to drive the agenda of educational improvement was blamed for the growing phenomenon of teaching to the test and actually damaging the educational opportunities of learners. Whether we like it or not, test preparation is a practice that is big business. Just take a look at these growth figures for the top test preparation company in China, which is floated on the stock exchange.
Cheating And of course cheating was back in the news as well. This year it all kicked off in Japan, where a student was discovered to have cheated on a University entrance test, and was reported to be the most wanted person in Japan. Reuters were reporting on the Japan frenzy over the incident. Japanese websites that promote the art of cheating were soon being discussed in the blogosphere. See the example on the left from YouTube. I'm not going to dwell too much on cheating this year as I've produced this extended analysis as a feature. But the largest cheating scandal in Australia came to a head with the conviction of staff from Curtin University who had taken bribes and falsified IELTS documentation. In most countries around the world, cheating on tests is a criminal offence and can carry serious penalties.
The problems in Australia led to a call from other test providers to be allowed into the market, which had until 2011 been an IELTS monopoly. Now Educational Testing Service and Pearson have tests recognized alongside IELTS. Perhaps the most interesting part of this story is the new Cambridge emphasis on the Certificate of Advanced English (CAE) as an alternative to IELTS. If you haven't been following this story, I strongly recommend this article. It will be interesting to see how this plays out in the coming years in terms of IELTS popularity and growth, particularly given the new marketing hype around CAE and its mythical link to the CEFR, as evidenced in this article published in The Australian in August.
High Stakes Testing in China Every year the Goakao or National Higher Education Entrance Examination gets into the news. It's inevitable because it is the single largest testing system in the world. This year almost 10 million students took this crucial test in China. The results dictate whether test takers get into college or university, and which one they go to. The whole experience is more a rite of passage than taking a test. This link to a Chinese news agency report shows just how seriously these tests, which include a compulsory English language test, are taken. Notice the emphasis on 'fairness' in this report, and the approaches taken to try to restrict cheating. These include electronic measures and CCTV surveillance. Other issues include traffic control to make sure that test takers can get to test centres, ensuring a comfortable noise free environment, and even providing psychological counselling to prepare students.
Poorly Constructed Tests At least there weren't endless stories from China about mistakes in the examination papers - which are precisely what we got this year in the UK. These included multiple choice items with no key (a correct response), integrated tasks with missing information, and items with multiple keys. Test takers complained bitterly. In the US this could have turned into a class action. But in the UK it fizzled out with a number of smarmy suits defending the examination boards. It really is time that these examination boards employed some assessment experts - and that goes for the national regulator OFQUAL as well - rather than just administrators and the party faithful respectively. But at least it isn't as bad as deliberately writing poor test questions as political propaganda. This is, of course, a very old practice. But one revised this year by the Russian government, according to this story from Business Insider. On this language test given on the day of the anti-government protests in December, one of the essay titles apparently read "By following the law we serve the nation." Be warned - just like the European Union, other countries use language tests for political ends. And if you want to read more, download my paper on this topic in PDF. However, you may also wish to download this Kindle version. This is a personal bit - I got an Amazon Kindle for Christmas, which I think is brilliant. Stop printing out paper and read in e-ink! When you've downloaded my paper, just send it to your @free.kindle.com address.
Confounding Testing, Training and Teaching There is a related story that compounds the disasters associated with British examination boards this year. In December the Daily Telegraph reported that senior examiners were tipping off teachers about which questions would come up on exam papers. They filmed a secret video of exam training courses for history teachers to support their case. It was all over the press in a matter of hours. The Guardian reported on the growing exam board scandal, and the Independent covered the start of an enquiry into what they claimed was basically cheating. The examiner in question was suspended, and later pulled before a panel of MPs to explain himself. The excuse? Well, click on that last link to see the video: he blamed it all on league tables and the need to improve scores. The following radio 4 item was broadcast on 9th December 2011.
Notice in this programme the question of whether test providers should also be training teachers and producing test preparation material - because of their insider status. Precisely what most of the international testing agencies are now doing, as discussed elsewhere in this year's review.
The suspended examiner may well have a point here, even though the examiners are clearly playing the system in this case. Evidence emerged from the investigations that other examiners were boasting that their tests were easier than all the others in order to get more business. Simple really: if you have the easiest test which requires very little learning, the teachers have an easier job, and the scores are likely to rise. Perhaps this is the clearest evidence so far of a "race to the bottom". It will be interesting to see how this is dealt with in our new educational consumer society, driven by testing the offically sanctioned accountability regimes that we have created in recent years.
Language Testing for Health Professionals An on-going story in the United Kingdom has been the use of language tests for medical practitioners coming to work in the National Health Service from Europe. If you remember my review for 2009 a German (Nigerian born) doctor caused the death of a patient on his first day as a locum. Stories like these reported in the Daily Telegraph had people up in arms, and the General Medical Council saying that action should be taken. The BBC reported on potential dangers to patients, and laid the blame firmly at the door of European Union bureaucratic
directives. Eventually the government changed the law to make language tests compulsory.
In October this issue got into the mainstream media, primarily focusing on the distressing stories of patients. Listen to this 5 Live Radio programme that highlights the need for language testing for health care staff (note - this programme is 45 minutes long).
Immigration Issues Much more controversial was the new policy to curb UK immigration that focused its attention on the spouses of UK citizens who cannot pass English Language Tests. Previously, those married to UK citizens could enter the UK as long as there was evidence that the family could support itself; the new requirement is that the spouse can speak English as well. This was almost immediately branded as racist and challenged in the courts, as explained by this story in the BBC. There are two key issues here: firstly, it seems to impact primarily on UK citizens with family ties in India, as explained in the video. Secondly, it appears to contravene the Human Rights Act, which guarantees the right to family life. The New Statesman made the issue of human dignity central to its commentary. However, in December a court ruled that the language tests were fair, and the government welcomed the decision, saying that it would help cut the costs of employing interpreters in government services like the NHS. Whatever testing specialists feel about the ethics of this use of tests, the remaining question is whether the language tests they use are suitable for these purposes - and that may be another legal challenge. The default option is to specify a CEFR level, which all tests produced in Europe now claim to reflect. So any test is potentially usable - including those whose intended purpose is for University entrance. But the needs of new immigrants are very different from those of 1st year University students! This is potentially extremely unfair and unethical - but a practice that is encouraged by the use of the CEFR in setting policy. Policy makers really need to consult language assessment experts in these cases in order to avoid unnecessary harm and discrimination.
I don't want to give the impression that it's just the United Kingdom that is doing this. It's a global phenomenon, but has taken a particular hold in Europe. Let's face it, Europe's a tiny place. You can fly over it in less time than it takes to get from New York to Los Angeles. Yet for some reason, in recent years, Europe has turned itself into a fortress in the belief that its single market will somehow provide economic protection from the rest of the world. This video shows how Italy is using language testing as part of its policy to reduce immigration as part of its response to an economic crisis. But this report from USA Today argues correctly that it is in fact the entire European Union that is suffering from this malaise. The use of language testing as part of an immigration policy, driven by current economic needs, is always going to be a sensitive area for language testers who take consequential validity seriously. But it does seem to be a phenomenon that is here to stay, at least while Europe is in economic meltdown. Perhaps our most important role can be in mitigating the most unfair decisions by recommending or even designing better tests for this purpose, rather than watching current construct and content invalid uses of tests which were designed for other purposes.
Expansion Every year I'm amazed by the sheer expansion of language testing activity around the world. This year is no exception. One interesting story was about the extension of language testing to all taxi drivers in Australia, irrespective of their first language background. I guess this is to avoid claims of discrimination, but Australian-born taxi drivers were up-in-arms over the test fees and the imposition. And perhaps the review of the year wouldn't be complete without a mention of the continued expansion of Pearson Education. Rather like the Borg in Star Trek, they seem to be assimilating everything in their path, this year buying up test preparation companies in China in order to strengthen test sales. If there is an an ethical issue with having companies both designing and selling tests, and preparing students to take the test (as described above in relation to the UK), this is where the two appear to be completely merged into a single business model. This is an area of commercial exploitation that we need to keep an eye on in future years.
If I've left out your favourite story of 2011, my apologies. But I hope you've enjoyed reading about those I've included; and I wish you all the best for 2012.
It's that time of year again!
