Each year since I started doing the yearly reviews I've always been able to begin and end with something highly amusing. And perhaps include some very light hearted pieces between the serious news items. This year I struggled a little. 2015 seems to have been a fairly dark and serious year. A bit like watching one of the early episodes of Dr. Who with Peter Capaldi after the geniality of Matt Smith. Takes a bit of getting used to. The year has been dominated by quite vitriolic debates around standardized testing, league tables and accountability, with a heavy dose of immigration issues, particularly for health professionals. But as you read on, you will see that there have been some lighter moments as well. At least the satirical Now Show returned to the topic of accountability, and there was John Oliver's unforgettable performance on Last Week Tonight. I do hope that you enjoy my selection of stories for the 2015 language testing review of the year.
Standardized Testing. One theme that ran through the testing year was the seemingly endless conflicts that rage over standardized testing. As you'll see below, this emerges in the big issues of finance, cheating, and international migration. So it's not surprising that important figures in linguistics get involved; or that testing becomes a topic for prime-time television. In this YouTube video Noam Chomsky gives his views on the detrimental impact of over-testing in Schools and its impact on learning and teaching. The basic message is that testing is not the same as learning and teaching, and that it can impede (if not destroy) creativity. This is actually an argument that most language testers (but not necessarily psychometricians) have agreed with for many years. The study of washback in particular has taught us that if we are going to use tests in learning contexts we need to design for intended impact, or engage in what Fred Davidson and I call effect-driven testing. So I'm rather ambivalent about the simplicity of such attacks on standardized testing; but when testing is done badly - as it so often is - it is impossible to defend against this kind of criticism.
The Miami Herald published two articles on standardized testing that were a cut above what I normally expect to see in the popular press. Laurie Futterman published Beyond the Classroom: The history and challenges of standardized testing on 4th March, and Beyond the Classroom: Perspectives on and the future of standardized testing on 11th March. These highly intelligent pieces draw on research and assessment literature to critique the overuse of standardized testing in the United States. But the lessons are relevant well beyond those shores. She covers the history and economics of testing, unaccountability of private testing companies, effect on instruction, cheating, cultural bias, and data use, among other topics. So much is achieved in two short articles. Great food for thought. The video covers some of the same ground. This is John Oliver's performance on Last Week Tonight, aired on the US channel HBO in May. As the Huffington Post put it, this was a comedian pointing out everything that is wrong with standardized testing; even taking the risk of litigation by naming some of the most powerful testing service providers. Within days a Pearson spokesman responded with a defence of the company in the Washington Post, which was discussed endlessly on social media. This programme is amusing, but also very serious. It didn't pull any punches in raising the issues of financial gain and monopolies, particularly when the same company produces the tests, the test preparation materials, and may even run training courses for teachers who prepare learners for the test. Oliver also refers to the quality of test materials, and illustrates his point with reference to the Pineapple text. You may recall that I discussed the Pineapple incident in the Review of 2012, and you may wish to go back there, download the questions, and see the related video. Finally, you'll note Oliver's criticism that no one is allowed to comment on test questions, on the grounds of "security". This year Pearson was also criticised for taking "security" to new levels, by "snooping" around social media and reporting children who were commenting on their tests to the Department of Education. As the Huffington Post pointed out, there's a fine line between trying to maintain test security, spying, and just protecting your reputation when poor items have been used in live testing. And of course, there's always the right to protest too, like the sabotage undertaken by opponents of the Common Core Tests by deliberately revealing questions on Facebook! What subversion! It could even bring down Western civilization as we know it.
Keeping the Lawyers in Business. So here we develop the theme raised in the last section. As Oliver pointed out, there is a lot of money involved in high-stakes testing. We learned in May that California had awarded the state testing programme to ETS, which was contested by Pearson. The contract was worth a staggering $240,000,000. Education Week reported that Pearson complained the award of the contract was "invalid and illegal". Then came the news that Pearson had lost both the $500 million contract to provide testing services to Texas, and a New York contract. The sums of money involved, with all the collateral sales of textbooks and training services, does raise the issue of the primary purpose of the testing companies, which Oliver is correct to question.
Big Business. Two fascinating financial stories hit the press in 2015. The first, and arguably the most interesting, was the decision of IDP Australia to float shares on the Stock Market. IELTS is jointly owned by Cambridge Assessment, the British Council, and IDP. IDP owns 50% of its share, with the other 50% owned by Australian Universities. But 50% of IDP is owned by SEEK, an online job agency. SEEK is now planning to sell off its share in IDP to concentrate on its employment business. Confused? Just like the last story, take a look at the profits in this story from the Sydney Herald. The other story worth telling is the rapid expansion of test prep start-up companies, many of which are launching apps for learners to practice test questions on their mobile devices. The value of these companies is quite astonishing. Over the years I've made a study of the satellite industries created by high-stakes testing, from the marriage agencies in India (select your IELTS girl) to online preparation. The expansion of uncontrolled testing related business never seems to slow down.
Internet Use? Regular readers may remember that a few years ago I wrote a feature on internet access during tests. Denmark is still the only country to allow this in high stakes tests because of the fear of what test takers will get up to if they can access information outside the test engine. This year the Oxford and Cambridge Examination Board suggested that access should be considered in the UK. There was an immediate reaction against the idea. In this extract from the Today Programme you can see just how animated some people can become about the prospect of "dumbing down" standards. Note also the comparison between countries (using test data from PISA) to object to such an innovation. So many assumptions, so little thought! What surprises me is that there has been no published research (that I am aware of) into learners' use of the internet in a testing context.
League Tables and Averages. The reason for very sad stories like that from Atlanta is in fact quite simple. If you start evaluating teacher performance by learner achievement on tests, and link evaluations to pay and career prospects (including being fired), there comes a point where it's better to help test takers cheat than face the consequences. Add to that institutional accountability by test scores alone, and there's a real incentive for poor behaviour. And in many countries individual institutions are named and shamed using the ubiquitous league table. In the following audio extract from The Now Show the UK government is subjected to much deserved satire following stupid pronouncements like "all schools should be above average." Again, this is driven by international economic comparisons using yet more tests, like PISA. Anyway, this is absolutely brilliant satire. Enjoy!
Button Poetry. As so much of the year has been about standardized testing, its use and misuse, I decided to end my review with a YouTube video of Olivia Fantini reading her poem On Standardized Testing. This is very moving, and when I first heard it, it reminded me of the girl in tears on the John Oliver video above. You can get the words to the poem by following the link. And this allows me to finish with a message that I've used before at the end of my annual review. But it's one that should be repeated again and again until it gets through to the politicians and all those who think that education is improved through the imposition of dumb accountability systems. Poor tests and testing policy harm good educational practice; and useful testing should not be done so frequently that it interferes with quality learning and teaching. It's really the same point Chomsky makes in his YouTube video. With the caveat that I believe language testing is too important to do badly.