Language Testing Review of the Year 2010

This site designed and maintained by
Prof. Glenn Fulcher



In December and January each year, news channels around the world reflect on the stories and events that hit the headlines to discover just what kind of a year has come to a close. And so on this website we take a look back at the language testing stories of the year. These are organized both thematically and chronologically. Before we begin with the serious stuff I'm going to start with one of the more bizarre stories of the year, which comes from Taiwan. Published in the Worldwide Faith News, the headline was First ever nationwide Taiwanese certificate draws large crowds. This reported on the first administration of the Taiwanese language test at Cheng-Kung University, which was open to the public. Large numbers of people flocked to the test site to watch approximately 1000 people.....well.....take a test. You have to admit, it must have been gripping stuff. Entertainment at its very best. Having said that, look at what has happened to the Who Wants to Be a Millionaire franchise. I guess that's little more than watching a string of people answer multiple choice questions. Perhaps we should start televising test administrations for fun, with commentaries from expert testers and teachers?

But to our first big story of the year - and perhaps a big story of every year. Cheating. This started in the UK in January, when two men, described as James Bond-esque spies were convicted of transmitting answers for a citizenship test to test takers. High tech cheating has become a nightmare for test administration over the last few years, and now affects many school examinations in countries like the UK. Of course, this becomes material for the satirists, as in this clip from the News Quiz, from January this year.

No less serious is paying someone to take the test for you, which was reported in a similar citizenship test later in the year. This practice is quite sophisticated. In April the New Zealand press reported on stories of cheats using photoshop to morph pictures of the true candidate and their 'stand in' on identity documents. When it comes to international tests, passing questions across international time zones for a fee is also on the increase, as we saw in a report from Korea in stories like SAT Instructor Busted for Helping Studies to Cheat.

Of course, cheating is always a danger when the stakes are very high, as we see in this report from the biggest test of them all: the 2010 Gaokao in China. In recent years China has taken security measures ranging from personal electronic checks, surveillance, and radio signal jamming patrol vehicles, to stop these practices. We can all see why cheating has to be tackled: it is unplanned score variation that brings score meaning into question. Those who do study for tests and justifiably get the grades they need are rightly upset that others should get the same grade for less effort. That's meritocracy!I'll close this theme with one final report from India, mainly because I never cease to enjoy the language of news reporting there. The one that caught my eye this year was Cops plan to get inputs from UK, Oz High Commissions, as they bust a fake IELTS certificate racket. The language may sound like '30s Chicago, but the money to be made from forging test certificates is probably just as large as trading bootleg spirits in prohibition America.

So let's turn to the topic that never goes away, the Gaokao. This year I won't dwell on it, especially since you've already seen the video above; but the issue of its fairness as a college entrance test is endlessly debated year on year. But this year, we had the chance to judge some of it for ourselves, as the essay questions were widely leaked (after the test!) and published on the internet. English versions are available on Danwei. The question that has drawn the most attention is "Why chase mice when there are fish to eat?". This has captured the attention of the world media, speculating on its true intention and what kind of an answer the examiners might wish to see, as in this discussion from the Telegraph.

Remaining in China for the moment, in August this wonderful piece was published in the Global Times about Gong Yuan Hutong where an examination compound was built in 1415. This report is well worth reading, not only for the history, but for the pictures of the compound in the early 20th century, and how the buildings have now become houses but are in danger of disappearing. Photos like these bring alive the stories of the great Imperial Examinations of the past, along with the pomp and ceremony that would have accompanied them.

Tests for Medical Staff have been in the news for most of the year, following a number of high profile cases like these, where patients have been put in danger or died because of language problems. Earlier in the year this also inspired me to produce a medical language testing scenario for use in language testing courses. But it was the case of this doctor that brought the matter to a head, when a patient died. There were immediate calls for language testing of overseas doctors, especially when it was discovered he had failed his English test but still been allowed to practice. This caused an immediate problem with the European Union, which says that if a medical practitioner is registered in one EU country he can be registered in any other EU country. Panic ensured and EU bureaucracy once again became the target of the British press for putting lives at risk. But as in most EU affairs there was a quick fudge: doctors from other EU countries could be registered in the UK, but this did not mean they could practice unless they were deemed to be competent to do so. This story from the Scotsman records the outcome of this year-long saga.

Immigration was also a hot topic in 2010, as the use of language tests to control the movement of people around the globe continues to rise. In the United Kingdom additional restrictions were placed on students entering the country, forcing them to demonstrate a particular level of English on a number of 'approved' tests before a visa could be issued. The Guardian Weekly produced this story in August, reporting that it was a reaction to government concerns that some unqualified students were applying to bogus colleges and using the visas to enter the country and remain illegally. In this video Alan Johnson (then home secretary, and at the time of writing the shadow Chancellor) explains his thinking, and you can download the approved tests document issued by the government at the time. The reason for the selection, as I explain in the Guardian, is primarily that these test providers offer the required security at test taking sites and in general test administration. In the video you will note that Johnson says the required level of English to get a student visa is 'intermediate' or 'GCSE' (General Certificate of Education - the qualification taken in the UK at the age of 16), which was stated to be B1 on the CEFR. After this policy was announced, on 23rd July the EL Gazette issued an electronic announcement with the title "Stop Press! Britain Closes Door to Foreign Students". Of course, this would mean that no students would be able to apply to UK language schools to study English at a level below the level required; English UK was up in arms, warning of the millions that would be lost to the UK economy. One of the interesting revelations in this article was Brian North (a main author of the CEFR) wrote to the United Kingdom Border Agency in February 2010 to tell them that GCSE was a "low A2, two school years below the B1 level". No evidence presented on either side for these claims, of course. But that is only to be expected. This is politics, not language testing. Language tests are merely used as tools in a policy game, by governments and the European Union.

One of the reasons we know this to be the case is that the matter did not stop with students. Language tests were then required for the wives or husbands of visa holders, and the spouses of UK citizens who had married non-UK citizens in order to reduce immigration. The government claimed that this was also designed to stop sham marriages taking place in India, where men who wished to work in the UK would marry a girl who could pass the IELTS but did not have enough money to study overseas. The man's family would then pay her expenses. This is indeed a problem, as there is a huge marriage industry in India to get into countries like Australia, where scores on academic language tests have been used for immigration as well as university entrance. I covered this in my 2009 review.

Where there are stories like these, there are always claims of discrimination. Rather bizarrely one group of people who claim that these new laws are unfair is the league of ugly rich men, who say they should be able to marry beautiful non-communicative girls who want them for their money. Hmmmmmm.

While the UK was in the news a great deal, immigration stories were big in other countries as well. Japan introduced language tests to keep foreign workers out mid-year, and Indonesia has put in place plans to do the same from January 2011. So it's not just Europe, North America and Australia. Although Canada did overstep the mark when it introduced English language tests for native speakers of English as part of its immigration procedures this year. It is stories like this, of a Professor of English literature hired by a Canadian University having to take an English language test that reveals the true purpose as purely political, rather than being genuinely concerned about the linguistic abilities of immigrants to be able to work and integrate in society. We can see exactly the same occurring in Australia where the IELTS is used for immigration and work purposes, when in August the government decided to increase the score needed for nurses to work from 6.5 to 7.0, not because there was any validation evidence that a 7.0 was linked to safe practice in hospitals (see above), but because they just didn't want any more nurses from India.

Are Tests Getting Easier? This question is asked every single year when pass rates go up. Are the kids getting brighter? Is the teaching getting better? Or are the tests getting easier? In this first clip from 5 Live Investigates (first broadcast on 7th November, 2010) all the difficult issues are raised. Do teachers change examination provider because some are easier than others? Do the examination boards engage in 'insider trading' - they provide the tests and they produce the teaching materials for teachers to prepare for the test. Should the two activities be strictly separated? There's also a short interview with Mick Waters who claims that examination boards are 'testing the tolerance to the limit' by conspiring to ensure that everyone does well. He has recently published claims that the examination system in the UK is diseased, and you can check out the book that makes the claims on Amazon. I'm not keen on the presenter's style, but the discussion is good.

In this second clip the Today team report on a small experiment to discover if students of English literature find today's tests easier than those P.D. James would have taken in the inter-war years.

Test Wars: Recognition and Expansion. We all know that language testing is big business, and so the constant struggle for recognition and market share goes on. Many test providers think that the IELTS monopoly in Australia is unfair, and so we have seen ETS make significant efforts to have TOEFL recognized. Reports of significant market expansion and the attempts of the "big three" to grow at each other's expense have been around all year.

As the testing industry grows, so does the test preparation industry, particularly in China. Test preparation is now so big that the companies are trading on the stock exchange, with share prices growing significantly.

But perhaps the biggest 'shock' news came in September, when we learned that Cambridge ESOL and the University of Michigan were going to create a new joint company to market their tests together under the heading Cambridge - Michigan Language Assessments (CAMLA). No one knows how this will pan out, but there may be another report on this at the end of 2011.

It's not often that Reliability gets into the news, but this year it did, when under the delightful title Strictly Come Marking it was revealed that as many as 1 in 3 students taking the national SATS in the UK got "the wrong grade". Basically, the exams watchdog commissioned and published this report from the National Foundation for Educational Research on reliability that discussed things like the standard error of measurement. This was written in 2009, but only made available this year. Politicians and the press were outraged that there should be such a thing! Well, what can I say. And these are the guys who shape public opinion and make education policy.

Kangaroo Meat is the final topic for this year's review. So what's all this about? A NAPLAN test from Australia contained a reading passage that compared beef to Kangaroo meet, which claimed that beef production generated more methane than the coal industry. Conclusion? Well, it's better to eat a Kangaroo! The beef industry was up in arms, arguing that by giving this to so many kids to read could seriously damage the Australian beef industry. What I want to know is why representatives of the beef industry spend their time scrutinizing texts in reading tests? Perhaps it's a paid appointment? Key requirement: an ability to write meaty reports! (Okay, but it is the end of the year....)

I hope you've enjoyed this short review, that you have a great Christmas, and a very happy start to 2011.

Glenn Fulcher
December 2010