2013 - A good one for the bizarre Selecting the stories for this review is always tough. I don't want to write so much that web readers click away. But I still want to give a flavour of the testing and assessment stories of the year. And in this year's selection there's just more than a usual hint of the bizarre. We have meditation and singing to prepare for tests, asking girls to remove their bras before taking tests, and strange case of all 25,000 test takers failing the exam. So here we go. The 2013 Review of the Year.
100% Failure Rate
My first selection illustrates the bizarre extremely well. In August we were treated to the news that 25,000 students sat for the University entrance examinations in Liberia, but no one passed. The Guardian speculates on why this might have happened, and it is suggested that the country's schools are at fault. With particular criticism being aimed at English language ability. But this can't possibly be true. Even if the test was deliberately made extremely difficult for the intended population, there would be some who would pass either by chance, or because of ability. This is stands out as a clearly political act, which is extremely sad for the school leavers concerned. Bizarre, but outrageous. If you wish to read more, here's a view from inside Africa on the difficulties facing young people.
Cheating Again
In April 35 teachers and administrators from Atlanta in the United States were indicted for cheating. This story was in the news last year, but was brought to a conclusion only this year. It has attracted a lot of attention because of the widespread view that if authorities use test scores to judge teachers and schools in a high stakes system, it encourages cheating. David Callahan, author of The Cheating Culture, wrote this article in the Huffington post to argue precisely this case. HuffPost Live also aired a discussion on the topic, and I've imported their video on the left; so you can play it from their website or here. Hosted by Nancy Redd, a group of teachers and researchers explore the issues. Of course, there's nothing new about cheating. So you may also wish to see my own take on this, which I wrote in 2011.
Are Canned Responses Cheating?
First of all, what is a "canned response"? It's a response to a question that has been pre-prepared and memorized. This can happen when speaking or writing prompts are in the public domain and occur frequently on test papers. This can happen with questions like "Describe a person who has had a significant impact on your life". So here is one, brought to my attention by Fred Davidson. Just click here to listen to the student response. It is taken from a 1936 College Entrance Test, and is a response to "What do you want to do when you finish College?" Absolutely brilliant, and amazing that it's survived.
Gaokao
No review these days is complete without an update on what is going on in China. This year there were two big stories surrounding the annual Gaokao, which is held in June each year. The first concerned the generation of satellite industries. We've been here in these reviews before - third parties finding ways to make money out of the desire of learners to get higher scores. A couple of years back we looked at the marriage industry for "IELTS girls" in India. This video on the Gaokao economy documents some of the business that flourishes around examination time. But it's not all bad of course. There are also taxi drivers who provide free lifts, and even stock stress medicines for the journey!
The second story is about - yes, you guessed it - cheating! Security has become a major problem in these exams, and now it's common for test takers to be checked with metal detectors on entrance to the exam hall, to make sure they are not carrying secreted communications devices. It transpires that bras with metal in them can confuse the detectors, and so the newspapers were full of banned bras headlines. Of course, it was only bras with metal in them, but what great copy for the media! Another priceless gem is the story of an invigilator who discovered hidden phones and confiscated them from the test takers. The parents attempted to bribe the invigilator, and when he refused a crowed of angry parents attacked him after the examination. This even went into some news outlets as students and families fight for the right to cheat, claiming that without cheating there can be no fairness! Now that's a very novel take.
Sacred Right
And speaking of rights, I found this article from Reuters in May, reporting on the Greek government's invocation of anti-strike legislation to stop teachers going on strike during the University entrance examination period. The Education Minister declared that students had a "sacred right to sit examinations without disruption". This has to be the first time I've heard God invoked in defence of tests. Funnily enough, I can't find a reference to the sacredness of tests in the New Testament - not even in the koine edition. But I did find this in St.Paul's letter to the Galatians 6:4, "All must test their own work; then that work, rather than their neighbor's work, will become a cause for pride." So perhaps that could also be taken as a sermon against cheating? All rather far fetched, I have to say.
College and Career Readiness
In the US this year the news has focused on the introduction of new "common core" testing. There is an excellent introduction to the US Common Core here, for those (like myself) who do not live in the US, and frequently find American education and assessment policy very difficult to understand. This clip from PBS News in August seems to be fairly typical of the media coverage. While there is plenty of critical comment, there are also those who believe that testing is pushed in a positive direction when we are asked to evaluate complex constructs and abilities. I'm always worried by phrases like "standards tell us what students should be thinking", and how they should be communicating; but the fact of the matter is that politicians are always going to mess with testing in order to drive their own policy agendas through education. When it comes to English Language Learners, the attempt to get states to agree on common definitions for comparison (and accountability) is also an interesting development.
Stressed Out?
These stories also illustrate another fact: high-stakes tests are nerve racking for test takers. It appears this is so much so that in 2013 New York issued guidance to schools on how to deal with
vomit covered test papers. Yep. Throwing up during the examination seems to be quite common. How to avoid the stress? 2013 has been simply full of advice. So, why not try "beditation" - or "mindfulness" while lying down, which is a form of meditation to relieve exam stress. The other very popular approach to test stress reduction this year has been exam warrior songs to focus the mind before the test. Call me old fashioned, but I when I was a student I always felt much more relaxed when I'd studied the syllabus well and knew that I could answer the questions likely to come up. Still, the times they are a-changin!
Pisa
Pisa is the set of tests used by the OECD to compare the educational performance of member countries, and make predictions about future economic performance based on results. The process assumes, minimally, that the tests are reliable and valid for all populations tested, that there is no differential item functioning, no test bias, and that the constructs measured are relevant to the interpretations being made. And finally, that the scores are predictive of future economic performance. There is very strong evidence that most of these assumptions are false Nevertheless, politicians look at the results and bemoan the performance of children in their country compared with those in Korea and Singapore. Here is the UK Education Secretary condemning the government of Wales (led by the opposition) for causing the country to "go backwards" (whatever that means) on the basis of Pisa data.
Here is the relevant analysis of the Pisa assessment from the BBC's More or Less, first broadcast on 7th December, 2013. This shows the hysteria generated by the test scores.
Oh Dear, Mr. Clegg This video from the Guardian is well worth watching (once the compulsory advertisement has passed). You'd have thought by now that politicians would at least consult an expert advisor before pronouncing on education and testing. This is a double shot in the foot. Firstly, he criticizes criterion-referenced assessments as being difficult for parents to understand. He suggests a new revolutionary approach: compare each child to the others in their year cohort. I've got an idea, Mr. Clegg. Why don't you call it "norm referencing"? Next, he says that the new system wouldn't be about saying which primary schools succeed or fail. Then goes on to say how terrible it is that some schools do not produce pupils who are ready for secondary school, so we have to know which ones they are. After this performance, Mr. Clegg will need to put new shoes on his Christmas wish list!
Who's being tested?
There is a fundamental anomaly in modern accountability-driven testing regimes. Students take tests and get scores. The reason is to see what they know and can do: the skills, knowledge and abilities that they have developed. This information can be used to provide feedback and targeted help. However, the scores are used primarily to judge whether teachers and schools are succeeding or, more usually, failing. It's frequently reported that this is causing excellent teachers to drop out of the profession, and standardized testing on such a large scale may be responsible for curriculum shrinkage. In this vein, my penultimate choice is this apposite song, which appeared on YouTube in 2013. It is surely a wakeup call to policy makers: testing has its place, and it's too important to do too frequently or badly. But in accountability mode, it has to be treated with a great deal of caution.
And so I leave you with my favourite cartoon of 2013, which encapsulates the unfortunate position that teachers now find themselves in. In our new world of accountability all professionals are considered incapable of doing their job responsibly unless they are constantly watched, monitored, and evaluated. The scores are not used to improve learning and teaching in the classroom, but to impose a system of constant teacher surveillance and rank order institutions for national and international league tables. The assumption is that it is market forces that drive up the quality of teaching, rather than investing in professional teacher education and continuing professional development. You may think that here I'm railing against the system in some postmodern frenzy of spleen venting. Far from it. I am simply acknowledging the way tests have been used since the dawn of time by policy makers. If you think this is an over-statement, then listen to my inaugural lecture from January this year. It's up to teachers and assessment professionals to influence the policy makers and improve the system through positive, proactive engagement. A fitting New Year's resolution for a better testing future!
2013 - A good one for the bizarre
The second story is about - yes, you guessed it - cheating! Security has become a major problem in these exams, and now it's common for test takers to be checked with metal detectors on entrance to the exam hall, to make sure they are not carrying secreted communications devices. It transpires that bras with metal in them can confuse the detectors, and so the newspapers were full of 
And so I leave you with my favourite cartoon of 2013, which encapsulates the unfortunate position that teachers now find themselves in. In our new world of accountability all professionals are considered incapable of doing their job responsibly unless they are constantly watched, monitored, and evaluated. The scores are not used to improve learning and teaching in the classroom, but to impose a system of constant teacher surveillance and rank order institutions for national and international league tables. The assumption is that it is market forces that drive up the quality of teaching, rather than investing in professional teacher education and continuing professional development. You may think that here I'm railing against the system in some postmodern frenzy of spleen venting. Far from it. I am simply acknowledging the way tests have been used since the dawn of time by policy makers. If you think this is an over-statement, then 