LT.info Bulletin

Changes to Article Update Page

8 May 2012 11:00:40 GMT

Some time ago I pointed out that the article update page had ceased to function effectively. There is a simple reason for this. Science Direct don't understand RSS feeds. Every single day each feed is updated with today's date, even if the content of the feed is old. In fact, they're keeping up to two year's worth of articles on each feed, for each journal. But it's always date-stamped as current. When my aggregator searchers for papers on language testing it therefore finds all this old stuff and keeps it on the top of the page, forcing all over journals and articles off. This is exceptionally poor web practice. I've been in touch with their web people, and the response is less than encouraging. So today I've removed all Science Direct content. This is very sad, because it's around 5 good journals. But the good news is that another couple of Taylor and Francis journals have fixed their RSS feeds, so I've added those. If Science Direct can get their act together I'll put their journals back on the search list. In the meantime, I'm sorry for the inconvenience.

What's New in May

3 May 2012 07:38:49 GMT

The May update to the Language Testing website sees the first part of a two-part feature on Test Architecture. The second part will be available in September. These two short pieces pose questions that focus on the metaphor of test design as architecture. They are designed to go with the 2009 paper Test Architecture, Test Retrofit published by myself and Fred Davidson in Language Testing. These features started life as part of an inservice training package for Cambridge Assessment, now rewritten and shortened for the website. Usual updates throughout the website on a daily basis.

Handbook of Language Testing

26 Apr 2012 08:16:36 GMT

After a number of delays, it's finally out and shipping. Fulcher, G. and Davidson, F. (2012). The Routledge Handbook of Language Testing. London and New York: Routledge. Here's the blub:

The Routledge Handbook of Language Testing offers a critical and comprehensive overview of language testing and assessment within the fields of applied linguistics and language study.

An understanding of language testing is essential for applied linguistic research, language education, and a growing range of public policy issues. This handbook is an indispensable introduction and reference to the study of the subject. Specially commissioned chapters by leading academics and researchers of language testing address the most important topics facing researchers and practitioners, including:

An overview of the key issues in language testing
Key research methods and techniques in language test validation
The social and ethical aspects of language testing
The philosophical and historical underpinnings of assessment practices
The key literature in the field
Test design and development practices through use of practical examples

The Routledge Handbook of Language Testing is the ideal resource for postgraduate students, language teachers, and those working in the field of applied linguistics.

And here are the reviews so far:

'This thorough and comprehensive introduction to the practical and theoretical dimensions of language testing and assessment is set to become a standard reference. With chapters by leading international authorities in the field, it is both intellectually stimulating and practically useful, and is a must-read for those involved in language testing as test developers, test users, policy makers and researchers.'

Tim McNamara, The University of Melbourne, Australia

'I found this book to be the finest collection of cogent articles on language testing to date. The Handbook solidifies language testing as a discipline in its own right-one that has real-world impacts on society at large. Stimulating and clear, the articles will be a valuable resource for decades to come.'

Paula Winke, Michigan State University, USA

I hope that we've produced a comprehensive text that will do just what the reviews say. Here's the content:

Editorial. Glenn Fulcher and Fred Davidson. Section 1: Validity. 1.Conceptions of Validity. Carol A. Chapelle. 2. Articulating a Validity Argument. Michael Kane. 3. Validity Issues in Designing Accommodations for English Language Learners. Jamal Abedi. Section 2: Classroom Assessment and Washback. 4. Classroom Assessment. Carolyn Turner. 5.Washback. Dianne Wall. 6: Assessing Young Learners. Angela Hasselgreen. 7.Dynamic Assessment. Marta Anton. 8. Diagnostic Assessment in Language Classrooms. Eunice Jang. Section 3: The Social Uses of Language Testing. 9. Designing Language Tests for Specific Social Uses. Carol Moder and Gene Halleck. 10. Language Assessment for Communication Disorders. John Oller. 11.Language Testing for Immigration, Citizenship and Asylum. Antony Kunnan. 12. Social Dimensions of Language Testing. Richard Young. Section 4: Test Specifications. 13. Test Specifications and Criterion Referenced Assessment. Fred Davidson. 14.Evidence-centred Design in Language Testing. Robert Mislevy and Chengbin Yin. 15. Claims, Evidence and Inference in Performance Assessment. Steve Ross. Section 5: Writing Items and Tasks. 16. Item Writing and Writers. Dong-Il Shin. 17. Writing Integrated items. Lia Plakans. 18. Test Taking Strategies and Task Design. Andrew Cohen. Section 6: Prototyping and Field Tests. 19. Prototyping New Item Types. Susan Nissan and Mary Schedl. 20. Pre-operational Testing. Dorry Kenyon and David MacGregor. 21.Piloting Vocabulary Tests. John Read. Section 7: Measurement Theory and Practice. 22. Classical Test Theory. J. D. Brown. 23. Item Response Theory. Gary Ockey. 24. Reliability and Dependability. Neil Jones. 25. The Generalisability of Scores from Language Tests. Rob Schoonen. 26. Scoring Performance Tests. Glenn Fulcher. Section 8: Administration and Training. 27. Quality Management in Test Production and Administration. Nick Saville. 28. Interlocutor and Rater Training. Annie Brown. 29. Technology in Language Testing. Yasuyo Sawaki. 30. Validity and the Automated Scoring of Performance Tests. Xiaoming Xi. Section 9: Ethics and Language Policy. 31. Ethics. Alan Davies. 32.Fairness. Scott Walters. 33. Standards-Based Testing. Thom Hudson. 34. Language Testing and Language Management. Bernard Spolsky.

As always, you can order a copy through the languagetesting.info bookstore. First page of the general tab.

Accent in Listening Assessment

26 Apr 2012 08:12:58 GMT

Issue 29(2) of the journal Language Testing contains a paper entitled "Accent, listening assessment and the potential for a shared-L1 advantage: A DIF perspective", by Luke Harding. In this podcast we explore why it is that most listening tests use a very narrow range of standard accents, rather than the many varieties that we are likely to encounter in real-world communication. This podcast is available from this website, from the SAGE website, and from iTunes.

Problem with Articles Feed

26 Apr 2012 08:07:54 GMT

If you use the webslice to get information on new articles at this page, you'll have noticed over the last week that it's bolding all the time, and older material is appearing. This is not my fault. Two journals - Assessing Writing and Studies in Educational Evaluation are not using their RSS feeds in an appropriate manner. Some web person is updating all the dates attached to articles every time they update their publication lists. This pushes all their content to the top of other web pages that search those RSS feeds. If this doesn't stop within a week I'll pull these two journals from the list. I'll let you know if I do this as you'll then have to check those two journals manually. These silly attempts to get their content more coverage just results in harm to themselves, so I hope they change this practice really soon.

Webinar on Assessing Speaking

17 Apr 2012 08:22:32 GMT

On 19th April I'm leading a webinar on the assessment of speaking, organized by the Language Acquisition Resource Center at San Diego State University, and CALPER at Penn State. I understand that a recording of the webinar will be available after the event for others to watch.

Articles Page Updated

17 Apr 2012 08:26:41 GMT

As I still can't upload the latest podcast due to delays beyond my control, I've just updated the articles page with a lot of new stuff that I've found over the last few weeks during some research I've been doing. I'll let you explore the articles page for yourself, but I will draw your attention to the University of Waikato making available the PhD thesis by Philip Valax on the use of the CEFR within and beyond Europe. It raises a lot of interesting questions, and despite the scope of the project, is probably one of the first critical empirical studies of how the CEFR is used outside Europe. Well worth a read.

Testing too Important to Do Frequently?

9 Apr 2012 10:02:32 GMT

One of the things I've frequently said in my language testing seminars is that testing is too important to do too frequently. If children in schools are subjected to high stakes tests on a regular basis, the quality of the tests probably isn't going to be that great. The scores are therefore not likely to be useful for decision making purposes. In addition, lots of testing takes time away from what is most important: learning and teaching. The AFL movement makes the division between formative and summative uses of assessment extremely clear. We're talking about different paradigms here, as I tried to explain in my 2010 book Practical Language Testing. When high-stakes tests are used - and they will be for decisions like University entrance - it is more likely that the tests will be designed with great care, and the decisions taken on the basis of scores much more sound.

This is why I like this article in today's Guardian about the school system in Finland. They've managed to put tests in their place - which is not driving the entire curriculum, as is happening in standards-based systems that require large degrees of centralised control and harmonization. Note also in this article the other critical component - the high status of the teaching profession, with a requirement that all teachers have masters degrees. These professionals are given the freedom to design and implement local curricula that are responsive to student needs.

The rest of Europe, the UK and the US can learn a great deal from the Finnish system.

What's new for April?

2 Apr 2012 12:21:34 GMT

The main addition to the site this month is a new podcast on accent in listening assessment, in which I interview Luke Harding about his research in this field. However, it won't be up on the podcast page for another week or so, as it has to be released in conjunction with the publication of Language Testing 29(2) in which his article on this topic is due to appear. If you want to be informed when the podcast is uploaded you can go to the podcast page and subscribe to the webslice. The link on your toolbar will then go bold and flash as soon as it's released. Other regular updates throughout the site and, as always, daily updates to news.

One other bit of information this month. For the first time the number of visits to this website during March 2012 went above 13,000. That's an awful lot of interest in language testing! I'll try to keep it interesting enough for you to call back on a regular basis.

Delayed Publication

27 Mar 2012 07:41:36 GMT

I was hoping to announce the publication of my new book with Fred Davidson - the Routledge Handbook of Language Testing. But there has been a delay. It was originally scheduled for 15th March, but there's been a glitch at the printers. Hopefully it'll be on the shelves in a month or so. Watch this space!

Cambridge Guide to Second Language Assessment

10 Mar 2012 08:15:15 GMT

Edited by Christine Coombe and colleagues, this guide has just been published by Cambridge University Press, ISBN 1107677076, with 336 pages. Here is the blurb:

"This collection of original articles provides language teachers with a theoretical background of key issues associated with language testing as well as practical advice on how to improve the effectiveness of the tests they develop and implement. Written by internationally prominent researchers and educators, the chapters are organized into five sections: key issues in the field, assessment purposes and approaches, assessment of second language skills, technology in assessment, and administrative issues. Chapters assume no particular background knowledge and are written in an accessible style."

As I have a chapter in this book (co-written with Fred Davidson) on test specifications, I received an early copy, and so I've already skimmed through and read a number of the chapters myself. As it says on the cover, "chapters assume no particular background knowledge and are written in an accessible style". This is for the most part true, which makes this useful for readers who have no background in assessment. However, some of the chapters do assume awareness of trends in language teaching, or movements in assessment; but they are accessible even without such awareness. All chapters are fairly short too, which means that the treatment of topics is highly selective and introductory. So these should not be taken as 'state-of-the-art' descriptions, but as very selective/personal introductions to a topic before the reader goes on to really explore the field.

Available from the Bookstore in the General Section.

March Updates

1 Mar 2012 09:56:10 GMT

The website continues to be used very widely by teachers of language testing around the world. For some of these uses the British Council has kindly subtitled some of the videos. If you haven't discovered these yet you may wish to check them out. The website is updated on a daily basis with language testing news and other information. And a personal plug this month - my new book is out, co-edited with Fred Davidson. If you haven't seen it yet details are available here. When it is officially published later in the month I'll send out a full description.

Also take a look at the news section on the home page. There are links to new conference announcements, internships, and other opportunities.

Language Testing in Greece

25 Feb 2012 09:54:55 GMT

Just out:

Tsagari, D. & S. Papageorgiou (eds) (2012). Language Testing and Assessment Issues in the Greek Educational Context. Special Issue of Research Papers in Language Teaching and Learning 3 (1). Available online from http://www1.eap.gr/images/stories/issue_03/RPLTL-03-01-fulltext.pdf.

I've added a link to this from the articles page as well.

Confidence Scoring

1 Feb 2012 09:32:40 GMT

In Issue 29(1) of the journal Language Testing, three authors from the Chinese University of Hong Kong have a paper on the application of fuzzy logic to scoring speaking tests. This is termed 'confidence scoring', and the first two authors join us on Language Testing Bytes to explain a little more about their novel approach.

February 2012 Updates

1 Feb 2012 09:29:15 GMT

One of the most used features on this website is the Introduction to AFL, so this month I've decided to produce a second piece on AFL that focuses on effective questioning. Teacher's TV unfortunately had to close last year but all of their videos are still available in various places on the internet, so I've linked in to these excellent resources for this piece. I'll try to do a third AFL feature on feedback later in the year, as my own students find it difficult to think of specific classroom practices that they can implement in these areas. Let's face it, the literature on AFL is good on advice, but not very clear about precisely what you need to do to get AFL to work, as the first video in this new feature shows.

Incentives and Test Based Accountability

10 Jan 2012 14:18:50 GMT

This new book edited by Haut and Elliot has a December 2011 publication date, but it's only just become available. It's from the National Academies Press, ISBN 0309128145, offering 200 pages for $32.00 or £22.00. This is the latest text to deal with the topic of accountability in education and the use of tests to hold both teachers and institutions responsible for the outcomes of learning. Here's the blurb:

"In recent years there have been increasing efforts to use accountability systems based on large-scale tests of students as a mechanism for improving student achievement. The federal No Child Left Behind Act (NCLB) is a prominent example of such an effort, but it is only the continuation of a steady trend toward greater test-based accountability in education that has been going on for decades. Over time, such accountability systems included ever-stronger incentives to motivate school administrators, teachers, and students to perform better. Incentives and Test-Based Accountability in Education reviews and synthesizes relevant research from economics, psychology, education, and related fields about how incentives work in educational accountability systems. The book helps identify circumstances in which test-based incentives may have a positive or a negative impact on student learning and offers recommendations for how to improve current test-based accountability policies. The most important directions for further research are also highlighted. For the first time, research and theory on incentives from the fields of economics, psychology, and educational measurement have all been pulled together and synthesized. Incentives and Test-Based Accountability in Education will inform people about the motivation of educators and students and inform policy discussions about NCLB and state accountability systems. Education researchers, teachers, as well as graduate students studying education policy and educational measurement will use this book to learn more about the motivation of educators and students. Education policy makers at all levels of government will rely on this book to inform policy discussions about NCLB and state accountability systems."

You can order a copy through the bookstore, where you'll find it under the standards-based testing tab.

Happy New Year

2 Jan 2012 09:38:05 GMT

Best wishes to all site users for 2012. I hope you'll find the Language Testing Resources as useful this year as in the last! And just to kick off with some interesting information - during 2011 this site had 117,498 visits, with a total of 130.64GB of downloads (including viewing video). I hope you enjoy the 2011 review of the year, and keep coming back for more updates throughout 2012.

Christmas Feature

19 Dec 2011 12:58:57 GMT

The end of the year once more! Where does all the time go to? This year I thought that the 2011 Review would be rather late due to work commitments, but I have managed to find time to write it due to a few days of annual leave that I hadn't taken during the year. Quite a lot has been happening in the world of language testing in 2011. So there is something on the growth of satellite industries, the increasing use of biometrics, cheating (as always - but not quite so much due to the earlier feature), testing in China, problems with poorly constructed tests, confounding teaching and testing, immigration issues, and language testing for health professionals. Lots of video and a couple of interesting radio programmes as well. I hope you enjoy the Review of the Year, and I would like to wish you a very Merry Christmas and a Happy 2012.

English Language Assessment and the Chinese Learner

17 Dec 2011 11:10:11 GMT

New from Routledge this month is this book by Liying Cheng and Andy Curtis. ISBN: 0415504783, with 288 pages. There is a hardback, paperback, and kindle edition. I'm a kindle fan - so this is good news. It's a pity that so few books in the field are available for the kindle, mostly because of copyright issues with examples from tests and/or images. The foreword by Lyle Bachman says: 'This volume addresses a very timely and important topic, and provides both broad and in-depth coverage of a number of large-scale English tests in China, including Hong Kong and Taiwan, and about the Chinese learner.' And from the blurb:

Building on current theoretical and practical frameworks for English language assessment and testing, this book presents a comprehensive, up-to-date, relevant picture of English language assessment for students in China (Mainland China, Hong Kong and Taiwan) and for Chinese learners of English around the world. Written by well-recognized international scholars in language testing, it covers:

the history of tests and testing systems, issues and challenges, and current research in China
both test-designers’ and test-users’ points of view on test development and test validation within a range of political, economical, social, and financial contexts in China
theoretical/conceptual perspectives on the use of the English language assessment at different levels, including societal, university, and schools
empirical research related specifically to test development, curricular innovation, and test validation

Given the long history of objective testing and its extensive use in Chinese society, and considering the sheer number of students taking various tests in English in China and elsewhere, an understanding of the impact of English language testing is essential for anyone involved in testing and assessment issues in China and elsewhere in the world. This is a must-read volume for testing and assessment policy makers, curriculum designers, researchers, ESL/EFL materials writers, graduate students, and English language teachers/researchers at all levels.

The book can be ordered through the languagetesting.info bookstore under the general books tab, page 5. Remember that when you order your language testing books through this Amazon affiliate bookstore, you are helping to keep this site free!

December Updates

30 Nov 2011 17:08:31 GMT

This month the video page has been given a facelift, and I've finally got around to adding suggested follow-up reading to John Read's new video on assessing vocabulary, and Ardeshir Geranpayeh's contribution on structural equation modeling. The articles page has a few more additions too. Coming around New Year (or a little earlier) is the Language Testing Review of 2011. Best wishes for the Christmas season.

November Updates

1 Nov 2011 08:14:13 GMT

The articles page has been updated this month, with the addition of a number of papers and books that have been discovered online. Language testing researchers are increasingly willing to place papers over three years old on their personal websites just as I am doing here. At the end of last month the new podcast came out that I make for SAGE to accompany Language Testing journal. This one is on measurement models. As usual the website is updated daily, with language testing news, links, and employment opportunities.

A Little Statistical Info

1 Nov 2011 07:54:55 GMT

I'm often asked at conferences why more people don't know about this website. I still don't know how to respond. For a number of years the use of the site has been growing rapidly, and in October this year (for the first time) the number of visits went above 11,000. This bulletin was viewed over 4,000 times alone. After the home page the most visited part of the site is the employment page, which was viewed 1,234 times in October. But here is the most interesting statistic: only 5% of visitors come to this site through links from other web pages, many of these being links in VLEs where the site is being used as part of a language testing course at a University, or on a DELTA course. 29% come via a search engine (usually Google), and 66% have the site in their bookmarks or type the URL into the browser. This means that two thirds of the site users are regular visitors. There is particularly high use of the news, employment and articles sections, all of which use webslicing to keep users updated. The scenarios are also being used more as a focus for discussion in seminars, just as the videos have been for a long time. If you're using the site in this way, please let me know! I'm always eager to explore new ways of teaching my own classes and you may have come up with innovative uses for these materials that I haven't considered.

Research Data Management Training

31 Oct 2011 10:58:27 GMT

This is a free course aimed at postgraduate students and early career researchers in data management. I'm putting a link in here as I think this would be very useful to anyone who is starting to handle test data for the first time, or who hasn't had data management training before. Click here to get going!

New Podcast: Measurement Models

21 Oct 2011 07:03:39 GMT

Issue 7: Mark Wilson on Measurement Models

Mark Wilson delivered the Messick Memorial Lecture at the Language Testing Research Colloquium in Melbourne, 2006, on new developments in measurement models to take into account the complexity of language testing. In Language Testing 28(4) we publish the paper based on this lecture, and Mark joins us on Language Testing Bytes to talk about his work in this area.

More on Item Response Theory

18 Oct 2011 12:02:39 GMT

A new book by Sara Jalali looks at IRT in computer adaptive tests for language testing. Item Response Theory: and its Applications to Differential Item Functioning, Item Banking, and Computer Adaptive Testing in TEFL is from the rather expensive LAP Lambert Academic Publishing house, released on 4th October. Here is the blub:

"There are two major theories of measurement in psychometrics: Classical Test Theory (CTT) and Item-Response Theory (IRT). Despite its widespread and long use, CTT has a number of shortcomings, which make it problematic to be used for practical and theoretical purposes. IRT tries to solve these shortcomings, and provide better and more dependable answers. One of the applications of IRT is the assessment of Differential Item Functioning (DIF). DIF tells the test developer whether the test item functions differently for different groups. Another important use of IRT is in the area of Computer Adaptive Tests (CAT). CAT is based on IRT, and the stepping-stone in preparing a CAT is the preparation of an item bank. Item banking is based on IRT. When IRT is ignored, item banking will not be applicable and consequently there will be no CAT."

Hmmmm. At $91.00 (today's Amazon price) I'm not sure I'll take the risk. But I've still added it to the LT Bookstore. You can find it under the technology tab.

New Feature: Cheating on Tests

11 Oct 2011 15:25:21 GMT

I've just released a new feature on Cheating in Language Testing. The news has been full of cheating stories from around the world, including the Atlanta schools scandal, high-tech counter-cheating activities in China, and University cheating in Japan and Vietnam. Not to mention ghost writing in Hong Kong and the prosectution of testing staff in Australia. What's been going on? Why do they do it? What can be done to reduce it? This is written to appear together with a new article in the Guardian entitled Cheating gives lie to our dependence on language testing.

October Updates

2 Oct 2011 10:30:38 GMT

This month a new feature on Cheating in Language Testing will be released on 15th October in conjunction with an article that I have written for the Guradian newspaper. The news has been full of cheating stories from around the world, including the Atlanta schools scandal, high-tech counter-cheating activities in China, and University cheating in Japan and Vietnam. Not to mention ghost writing in Hong Kong and the prosectution of testing staff in Australia. What's been going on? Why do they do it? What can be done to reduce it? The website also has a new search engine, which was put in place part way through last month. This is much more powerful and useful than the previous system.

Foundations in Language Testing 2011

13 Sep 2011 10:00:16 GMT

This has just been announced on the UK's Language, Linguistics and Area Studies website, which is part of the Higher Education Academy. This is a one day introduction to language testing and assessment for lecturers working in higher education in the United Kingdom. It is run by myself, Dr Julie Norton and Mr Wasyl Cajkler, all members of the School of Education at the University of Leicester. Date: 19th November 2011; Fee: £125. All applications through the LLAS website. Limited to 20 places.

Brand New Search Engine Launched

9 Sep 2011 09:49:49 GMT

The search facility on this website is one of the most frequently used pages. Since 2009 I've used Entropy Search, which is a cgi based search solution. However, since then I've moved rather radically away from static html pages towards dynamic php content. I discovered only last week that Entropy can't index dynamic content. So today I've ditched it and implemented Zoom Search. This indexes not only dynamic content, but also pdf, xls, and many other file types. It is a dynamic page itself so search results are generated on the same page rather than in a new window. Finally, there's a lot more information in the search - including a 'score' for relevance to the search, highlighted search terms, KWIC information, and number of terms matched. It also allows users to search for phrases or individual words, and supports the use of wildcards. It went live about 15 minutes ago and is infinitrely superior to Entropy. I hope that this makes the site even more useful.

Testing for HE Scenario Now Ready

1 Sep 2011 08:33:37 GMT

This month I've finally got around to completing the last of the six planned study scenarios on language testing for higher education. I originally designed these for use with my own students, as a basis for seminar work and for some of their coursework. One or two people have been in touch to tell me that they are also using them - but in very different ways. If you incorporate them into your course do let me know. I may add to these in the future, but for the moment six is enough to give my own students enough choice.

CAE Enters Competition with IELTS

1 Sep 2011 08:29:33 GMT

A great developing story. It's going to be interesting to see how Cambridge plays the market now that CAE is increasingly being used for University admissions and immigration. Watch this space over the next few years!

New Book on Testing Speaking

29 Aug 2011 09:03:23 GMT

Edited by Lynda Taylor, Examining Speaking: Research and Practice in Assessing Second Language Speaking (Studies in Language Testing) is an edited collection on testing speaking from Cambridge ESOL. ISBN; 0521736706, it was released on 25th August at 408 pages. As one would expect from a book that comes from an examination board it very much describes the way Cambridge approaches the assessment of speaking at the expense of all others: the discussion is set firmly within Weir's checklist validity framework. However, it is well worth reading to understand just how this large examination board does things. Here is the blurb:

This edited volume develops a theoretical framework for validating tests of second language ability. The chapter authors show how an understanding and analysis of the framework and its components can assist test developers to operationalise their tests more effectively, especially in relation to the key criteria that differentiate one proficiency level from another. The book provides an up-to-date review of the relevant literature on assessing speaking, an accessible and systematic description of the different proficiency levels in second language speaking and a comprehensive and coherent basis for validating tests of speaking. It will be of considerable interest to examination boards who wish to validate their own speaking tests in a systematic and coherent manner, as well as to academic researchers and graduate students in the field of language assessment more generally.

Available through the bookstore under the testing speaking and the Cambridge SILT series tabs.

Article Update

29 Aug 2011 08:11:45 GMT

Ahead of a general update on Thursday, I've just uploaded new links to free online articles, including a link to the full text of Robert Yerkes' 934 page tome Psychological Examining in the United States Army. Published in 1921 this described in detail the Army Alpha and Beta tests, and their use. This is a very substantial pdf so if you decide to download it, allow plenty of time.

Upgrade to Article Alerts

18 Aug 2011 09:46:58 GMT

If you haven't yet discovered the Language Testing Article Alert, you may wish to take a look now. Some publishers had taken RSS feeds out of service and introduced new ones, and a number of journals that didn't have RSS feeds now provide them. Today I've overhauled all the feeds and tied up the filters,so I hope it will provide a better service. I can't exclude all papers that don't have a testing focus but for the most part the papers do, or have a testing aspect to them. The page uses webslicing to inform you whenever a new article on language testing is published in the literature and lists the 30 most recent publications. Webslicing only works in IE8 and IE9, I'm afraid. My apologies to Firefox users. If you just want to browse articles currently in print in the general applied linguistics literature (excluding specialist testing and assessment journals) you can find these on the aggregation page.

Site Changes

17 Aug 2011 07:56:19 GMT

At the start of the month I introduced an e-card facility, but I'm afraid I've had to take it off the website already. It has been hit rather hard by spammers, which is precisely what happened when I set up a discussion board a couple of years ago. Both of these were protected from spambots, so it's individuals who just seem to spend their time making interactive features of other people's websites unworkable. The only solution to this is to have a password protected interactive area for which individuals must apply for a logon, but alas, I have a day job. So, apologies for the withdrawal of the e-cards. Next month, however, I'm adding the final scenario to the site, which is on the use of language tests for access to higher education.

Cambridge and IELTS Part Company?

9 Aug 2011 11:14:59 GMT

If you missed this article in The Australian Higher Education Section, then it's time to read it. What should we make of this? Well, the first thing is probably that Cambridge is starting to distance itself from IELTS, and it appears that it is going to push one of its main suite examinations as an alternative test for University entry (and therefore tier 4 visa application in the UK). If you now visit the Certificate of Advanced English website you will see that Cambridge are now advertising that it is accepted for immigration in Australia and tier 4 applications. So far so good. This seems to make a lot of commerial sense for Cambridge because they share the IELTS income with the British Council and the Australian IDP.

Notice also the claim in the article, attributed to the Cambridge CEO, that no test is suitable for all purposes. IELTS has recently come under significant criticism for claiming to be just such a test. If you look at its online global recognition system it appears that the at least one of the partners wishes to expand the use of IELTS in any arena they can access, from banking and finance, to a range of industries, the service sector, and leisure. So again, distancing itself from this approach to marketing that ignores validity issues is smart.

What is bizarre, however, is the claim that the Common European Framework of Reference (CEFR) is going to take the place of IELTS. How can an abstract framework that is not in itself a test, nor even a specification for a test, take the place of a test? The fact of the matter is that the CEFR fulfills no specific purpose, and its use goes against the argument being put in this article. Furthermore, anyone can conduct a standard-setting exercise to link their test to the CEFR, thus claiming that the test is relevant for a "B2" or a "C1" purpose, without the inconvenience of having to do any validation research. This final claim is therefore quite dangerous.

It will now be interesting to see how the politics of language testing plays out in the Cambridge game plan over the coming months.

New Podcast

1 Aug 2011 08:59:33 GMT

Issue 6: Craig Deville and Micheline Chalhoub-Deville on Standards-Based Testing

Standards-Based Testing is highly controversial for its social and educational impact on schools and bilingual communities, and the technical aspects that rely to a significant extent on expert judgment.This podcast discusses the issues surrounding Standards-Based Testing in the United States with the guest editors of a special issue of Language Testing on this topic. The collection of papers that they have brought together, along with reviews of recent books on the topic, and test review, constitute a state of the art volume.

The Dictation Test Again

1 Aug 2011 08:48:45 GMT

A few weeks ago the issue of making apologies for the use of the dictation test as part of Australia's immigration policy at the turn of the 20th Century got into the press. Since then the calls of Australia's Chinese population for an apology have got louder, and in the past few days has been reported in the UK Press. This story from the BBC makes interesting reading. It is difficult to see how today's politicians can really apologise for acts of a government so long ago, and raises many ethical issues about practices from the past when standards of practice and test use were so different.

On Continuous Assessment

22 Jul 2011 11:51:23 GMT

Rather pricey at £53 or $84, just out is this 156 page book by Dagne Tiruneh Dinsa from VDM Verlag Dr. Müller, entitled Practices of Continuous Assessment by English as a Foreign Language Instructors, ISBN: 3639365526. This is not unusual for German based publishers, many of which take on very low volume issues. The blurb reads:

This book is aimed at helping instructors to practice continuous assessment appropriately. It will equip teachers with the skills and knowledge of implementing continuous assessment in class. The book will also raise instructors awareness about the what, the why and the how of continuous assessment. It reveals the roles which continuous assessment plays in improving students' learning and teachers' teaching process. Unless instructors know the main intention of continuous assessment, they may not practice it in the right way. The book, thus, fills the gap between the knowledge and the practices of continuous assessment in teaching institutions.

Available from the bookstore under the classroom assessment tab.

Jacqueline Ross TOEFL Dissertation Award 2012

20 Jul 2011 09:17:18 GMT

The TOEFL Program at ETS is pleased to announce it will now begin accepting applications for the Jacqueline Ross TOEFL Dissertation Award to be presented in 2012. The deadline for receipt of applications is September 1, 2011.

A description of the award process is given below. To access application and supervisor recommendation forms please visit the Grants and Awards section of the TOEFL website at http://www.ets.org/toefl/grants/jacqueline_ross_dissertation_award

Purpose

To recognize doctoral dissertation research that makes a significant and original contribution to knowledge about second or foreign language tests and testing and/or the use and development of such tests and testing.

Award Value

$2,500 (U.S. dollars)
Round-trip economy airfare, expenses, and hotel accommodations for three nights at the Language Testing Research Colloquium (LTRC), where the award is presented.

Eligibility

To be considered for the award:

The candidate’s institution must have accepted the dissertation within three years before the date of the award application.
The research must have been completed as part of the requirements for a doctoral degree, or its equivalent, at a university within or outside the United States.
Although the dissertation under consideration must be in English, the research may be related to second- or foreign-language testing of any language.

Note: Candidates who have submitted abstracts for this award in previous years are not eligible.

Important Dates

September 1 –Deadline for receipt of summary and abstract, letter from advisor, and official proof of dissertation acceptance
October 15 – Call for submission of dissertations of finalists
November 1 – Deadline for receipt of dissertations
January 31 – Selection of award recipient

Application Process

Applicants must submit the following:

A typed summary of the dissertation, not to exceed 5000 words, including tables and figures, but excluding references, and a 150-word abstract of the research. The summary should include brief descriptions of theoretical background and rationale, purposes of the research, methods (including data analyses), results, and implications of the results. Submissions exceeding the word limit will not be considered.
Proof of acceptance of the dissertation by the candidate's university.
A completed application form (PDF).
A completed, signed letter from the dissertation advisor or director, using the template form.

Electronic applications are encouraged. The application form is a form-fillable PDF. Save a copy of the form on your computer, fill in the fields, and submit electronically.

Evaluation Criteria

Proposals are evaluated according to these criteria:

Scholarly or professional significance to the field of second or foreign language testing and/or test use and development
Originality and creativity
Technical quality
Quality of presentation.

Where to Send Your Application or Request Further Information

ETS
TOEFL Grants and Awards Program
MS 50-L
Princeton, NJ 08541 U.S.A.
Fax: 609-683-2310
E-mail: TOEFL_awards@ets.org

Interaction in a Paired Speaking Test

19 Jul 2011 12:30:40 GMT

Just published with Peter Lang, this book is based on Lyn May's PhD thesis. ISBN 978-3-631-61723-6. The book synopsis reads:

Paired speaking tests are increasingly used in both low- and high-stakes second language assessment contexts. Until recently, very little was known about the way in which raters interpret and apply descriptors relating to interactional competence to a performance that is co-constructed. This book presents a study which explores the interactional features of a paired speaking test that were salient to raters and the extent to which raters viewed the performances as separable. The study shows that raters use their own frames of reference to interpret descriptors and that they viewed certain features of the performance as mutual accomplishments. The book takes us 'beyond scores', and in doing so, contributes to the growing body of research on paired speaking tests.

Available from the Bookstore under the testing speaking tab on page 2.

New Website for Cambridge-Michigan Assessments

19 Jul 2011 11:05:30 GMT

Last year Cambridge took over the language testing operations of the University of Michigan's English Language Institute. There was a lot of press coverage, like this article from the ELT News in Greece, where Michigan exams are widely used. Now they have launched a new website, which I've added to the links page under the test provider tab.

Aviation English Testing Upgraded

15 Jul 2011 16:40:00 GMT

I have just upgraded the Aviation English Testing Scenario by adding an activity that combines the use of flightradar24.com and live airtraffic control to follow instructions from air traffic to controllers to planes on approach to Schiphol airport while monitoring the aircraft's flight path, altitude, speed, track and position. The image on the left is a screenshot of the zone around Schiphol airport (the blue cross). The instructions for the activity are as follows:

Step 1: Click here to open Air Traffic Control at Schiphol Airport, Amsterdam (AMS). Click on one of the options to listen. Step 2: Click here to open Flightradar24.com. Step 3: In Map Mode centre on Schiphol Airport (see screenshot). Try clicking on planes to see their squawk and flight path (this map updates in real time; on the left you see a picture of the plane, origin, destination, speed, altitude and heading). Step 4: Select a plane on approach to Schiphol and follow its progress while monitoring instructions from ATC. Question: Could you use this to create a task for a test of aviation English?

It may also be possible to actually design a task as a group activity. Of course, the same activity may be carried out at any other airport covered by LiveATC.net and within range of the flightradar ADS-B receivers. I will be trying this out with my students in my Fall language testing class. If you try it before me, or if you come up with interesting alternatives/variations, let me know.

Scenarios Back Online

15 Jul 2011 08:58:42 GMT

Thanks to users who warned me last week that the Language Testing Scenarios had gone offline. These pages are all created on-the-fly so that links always stay current. Basically, once a week a cron job activates a script that checks the links against major search engines and updates the web page. As some of you may have noticed this feature is not working on Bing at the moment, which is why the Scenarios suddenly went blank. Everything is now working again, re-routed through Google and Yahoo.

Open Library

5 Jul 2011 14:12:27 GMT

I've just discovered the Open Library. This site is packed full of free e-books that you can download in a range of formats, or send them directly to your Kindle. For some books you even have a choice of which edition you'd like to download and read. Some of the titles can only be borrowed, for which you'll need Adobe Digital Editions and an account with the e-book distributor Overdrive. You can get some classic texts in education and assessment. For example, try typing "John Dewey" into the search engine, and then scroll down to get your copy of How We Think, or Democracy and Education. This is a great site - well worth spending time to discover what's available.

A Piece of History Resurfaces

4 Jul 2011 15:05:49 GMT

The Australian Dictation Test, which was used to enforce the 1901 immigration act, is often discussed in the language testing literature as a classic (and very clear) case of discriminatory test use. You can also look at some of the passages used in the test in order to restrict immigration. Chinese Australians are now asking for an apology from the Australian Government for the way the Chinese were treated during the days of what is now referred to as the White Australian policy. It will be interesting to see how this story develops, and whether or not there will be a government apology for the use of a language test in this way.

LTRC 2012 Website Available

3 Jul 2011 16:26:48 GMT

The 34th Language Testing Research Colloquium (LTRC) will take place from April 1–5, 2012 at the Hyatt Regency Princeton in Princeton, NJ. This year’s event will be hosted by Educational Testing Service (ETS). The conference theme is:

Assessment and Learning: Bridging Diverse Disciplines and Domains

Interplay amongst classroom teaching, formative assessments and summative assessments
Interface between language assessment, language learning and computer technologies
Critical research issues in language assessment for younger learners
Critical research issues in language assessments in different contexts and domains
Potential applications of theories, innovations and practices in other domains to language assessment

New development in Aviation English Testing

3 Jul 2011 16:17:07 GMT

As a result of ground breaking cooperation between the International Language Testing Association (ILTA) and the International Civil Aviation Organization (ICAO), the Aviation English Language Testing Endorsement Service was launched in 2011. Under this scheme a pool of internationally qualified language testers are drawn upon to make recommendations to the ICAO regarding the validity of tests for the aviation industry. The endorsement service has a new website: Aviation English Language Test Endorsement.

For further informationon language testing in the aviation industry visit the Aviation Language Testing Scenario.

New Video for July 2011

1 Jul 2011 12:06:40 GMT

At LTRC last month I attended a workshop on Structural Equation Modeling (SEM) run by Ardeshir Geranpayeh of Cambridge University, who uses SEM in his research on test validation. I asked if he would make a short video for users of this website to explain just what SEM is, and how it can be used. This is the 19th video in the series, with more to be added shortly. I have also added a couple of new articles to the free lists, and spent some time updating the filters on the employment page to exclude a few rogue items that have been coming through from the world of computing. And of course there are all the regular daily news updates across the site.

A Touch of Technical Stuff

22 Jun 2011 01:11:50 GMT

This website is about language testing. But I also like to keep it technologically up to date. If you've looked at the timeline for this website, I started it in 1995.The internet wasn't up to much then. So from time to time I don't have any worries about adding something that isn't entirely language testing related if I think it's going to be useful. So, for people who are interested in internet and web stuff, here are two new things to investigate. First of all, check out lopesoft, and try their file menu tools. This is freeware, and brilliant. One of the tasks that I often have to do when maintaining the website is right clicking on files and opening with the correct program to edit. This tool allows you to customise your rick clicking. It saves me a load of time. Next, I have a technical page that lists known problems with this website. I've just extended the section on web and internet news, importing third party content of interest. This stuff keeps me up to date with web developments and helps inform how I develop the website. May be of interest to others too.

LTRC 2011 - Downloads

18 Jun 2011 15:14:22 GMT

LTRC 2011 is almost upon us. If you haven't yet downloaded your program, you can do so here. You may notice that on Thursday I'll be taking part in a symposium on assessment literacy. The handout is quite a number of pages, and so in the interest of trees and the weight of my luggage, the handout is available for download. Hope to see many of my website users shortly!

TOEFL iBT Research Insight Series

10 Jun 2011 08:09:03 GMT

This has clearly been around for some time because there are already four publications available from this website, but I've only just discovered them. One of the problems with the ETS website is that they just haven't yet discovered RSS! If they would only set up their own 'research news feed' we wouldn't have to wait until we stumble across information to establish links. That's my whinge for the day. Still, a very useful resource, the purpose of which is to make research accessible to teachers and score users who may not have a strong background in language testing. I've added a link to this collection from the ETS Collections section of the articles page.

EdMash

7 Jun 2011 09:51:26 GMT

One of the rss feeds that I have on this site is a mash up of education news. I really created this for myself to keep up with what's going on the the wider world of education beyond testing and assessment. However, it's become clear from user feedback, and from the way I've started to use the feed, that it isn't fit for purpose. Basically, it combines news that has a mainly North American focus with news that concentrates on the UK. I've therefore separated the two out. You can now decide whether you wish to subscribe to EdMashUS or EdMashUK. Just click on the links to subscribe. Yes, I know. This is still just two countries. But remember - this website is a one man operation and I do have a day job. However, if there's a real demand for another one with a different focus I may be able to find time to have a go. At least now the two aren't mixed up - I have to confess that I just didn't understand some of the US stories because I don't know the people or issues involved. I guess the same is true for US subscribers.

Humour Page - New Video (7)

3 Jun 2011 12:11:31 GMT

Thanks to Kevin Browne for pointing me to this video from the Onion satirical website. This is a great play on the issue of test bias against known subgroups of the test taking population, and DIF, or differential item functioning, where some items are found to discriminate unfairly against at-risk subgroups. I won't tell you any more, other than to say that there is some mildly strong language in this clip, so if you're likely to be offended, don't watch! Otherwise, just follow this trail.

Edited Volume Just Published

3 Jun 2011 11:59:06 GMT

Language Testing: Theories and Practices (2011) edited by Barry O'Sullivan has been listed on Amazon for a long time, and it is now finally in print. A new offering from Palgrave Macmillan, here is the cover blurb:

Language Testing: Theories and Practices offers a coherent overview of some of the key issues currently dominating the language testing profession. By bringing together a large group of authors from across the world who are actively engaged in language testing in different ways, from the practitioner to the academic, the book succeeds in highlighting a number of central ideas, the interpretation of which is to modern testing. These ideas (validation, localization, professionalization, reflexivity and politicization) are central to the chapters, which offer the reader an opportunity to see how language testers envisage the discipline developing in the coming years. The work included here will appeal to the expert and non-expert reader. The authors present their ideas and evidences in an informative yet accessible style and offer a unique glimpse of the broad range of interests, methodologies and concerns that define language testing.

You can order this through the bookstore, under the general tab on page 5.

New Guest Feature for June

1 Jun 2011 08:23:03 GMT

This month sees the first guest feature on this website. John Oller has worked in the field of language testing for many years, and I'm sure that users of my website will be familiar with his work. Perhaps the most widely read book - and certainly one of the first that I read when I started to work in language testing - was Language Tests at School (Longman, 1979); but I also have his other volumes on my shelves, including Research in Language Testing and Issues in Language Testing Research. In recent years he has been researching and teaching in the field of communication disorders, and his recent books include Cases: Introducing Communication Disorders Across the Lifespan, and Autism: The Diagnosis, Treatment, & Etiology of the Undeniable Epidemic. He has also contributed a chapter on language assessment and communication disorders to a book that I am editing with Fred Davidson:

Oller, J. W., Jr. (In press). Language assessment for communication disorders. In G. Fulcher & F. Davidson (eds.), Routledge Handbook of Language Testing, London & New York: Routledge.

This is an area of language testing and assessment research that was certainly off my radar until very recently, and I have enjoyed engaging with it over the last year. I was therefore delighted when the idea came up of John doing a guest feature for the website to explain the issues and concepts relating to communication disorders in an accessible way; and the feature makes an excellent partner for his chapter. I hope you find it as interesting as I have.

Videos with Subtitles

1 Jun 2011 07:57:01 GMT

The videos on this website are widely used for teaching language testing. 12 of them are used by the British Council as part of their blended learning course in language testing for China. In order to make the videos more accessible for this purpose they have subtitled them and we have made them available for others as well. My thanks to the British Council for enhancing the content of my website for all users.

ETS Publish Comparison Table

27 May 2011 09:11:17 GMT

The issue of score comparison between major international language tests has always been a thorny issue. On the one hand, the tests are very different and it is not clear that they have similar construct definitions, so it is hard to say that a score on one 'means' the same thing as a score on another. But test users, particularly in educational settings, want to know what the 'equivalent' cut score might be across tests for admissions decisions. This pulls the testing agencies in two directions. Rocks and hard places come to mind. The situation has been made worse by the European tendency to compare different tests by linkage to the CEFR. In validity terms this is a futile activity; but as the Council of Europe has rather successfully re-defined validity as 'recognition' (thus subverting a hundred years of validity theory) the testing agencies need to take the new political situation into account, or not sell their tests.

And so yesterday there was an announcement in the press (see this one from Reuters) that ETS has released an Official TOEFL iBT Score Comparison Table so that score users can compare scores between TOEFL and IELTS, arguably the two largest volume tests on the market. However, unlike many competitors, ETS has also released the research that supports the creation of the table. Here are the relevant links:

Score Comparison Tool

Supplementary Comparison Tables

Linking TOEFL iBT Scores to IELTS Scores: A Research Report

Linking TOEFL iBT Scores to the CEFR

I'll add links to these two research reports to the updated articles page on 1st June.

Accent and Listening Assessment

15 May 2011 09:15:18 GMT

Just published: Luke Harding (2011). Accent and Listening Assessment: A Validation Study of the Use of Speakers with L2 Accents on an Academic English Languge Test. Peter Lang. Here's the blurb:

Given the linguistically diverse nature of academic institutions in English-speaking contexts, a strong rationale exists for the incorporation of L2 accents of English in academic listening assessment on the grounds of authenticity and construct representation. However large-scale tests have tended to feature only native-speaker varieties in listening test input owing to concerns about the intelligibility of L2 accents, construct validity and acceptability. This book presents a mixed-methods study designed to address these concerns. Versions of the University Test of English as a Second Language (UTESL) featuring Australian English, Japanese and Mandarin Chinese accented speakers were used to explore the potential for a shared-L1 or familiarity advantage, and to investigate test-takers' attitudes towards L2 accents on a listening test. Implications are drawn for test development and for future research.

Available in the the testing listening section of the bookstore.

New Testing Textbook

9 May 2011 16:23:33 GMT

Designing and Analyzing Language Tests by Nathan Carr has just been published by Oxford University Press at $39.95. ISBN-10: 0194422976. The product information is as follows:

Designing and Analyzing Tests with accompanying CD-ROM gives you a solid grounding in language assessment theory, practical guidance on developing and administering tests, and shows you how to use Excel for statistical analysis of procedures. Language teachers often have to develop or choose tests or other assessments for their classes without any training. With this book and CD-ROM you learn how to choose from ready-made tests, or to adapt existing tests to better meet your needs. It gives you the tools you need to write your own tests and to analyze them.

The book is organized in two Parts: Part 1 focuses on language assessment theory, and guidance on developing and administering tests. Part 2 introduces the use of Excel for analyzing language test data.

You can purchase this title through the bookstore in the general category, page 5.

Two Collections Added to Articles Page

6 May 2011 09:27:21 GMT

I've just updated the articles page with a couple of new publications and two collections that have recently been brought to our attention. The first of these is the ETS R&D Connections Series. The publications in this series summarize aspects of ETS research "...written in nontechnical language, each issue highlights a particular area of research and discusses how it impacts the field of education and offers possible real-world applications." A very useful addition to the literature, and particularly useful as introductions to key topics for students of language testing and those who make policy decisions by are not testing experts. The second collection is the IELTS Research Reports. This is a diverse collection of papers of varying quality, as one would expect from funded research from external researchers who differ markedly in their research experience and abilities. However, read judiciously, these volumes give an idea of the kind of research being conducted on this important international English language test. Much of this material has not been easily accessible in the past, so it's very useful to have it all freely available on the web.

May Update

3 May 2011 15:31:19 GMT

The Language Testing videos are one of the most popular parts of the website, and used widely in language testing programmes around the world. One of the obvious gaps in these videos was something on vocabulary testing. Thanks to John Read from the University of Auckland this has now been filled.

The articles page has also been revised again to include a couple of papers that I discovered recently. Other updates to the site occur on a regular basis. If you wish to keep up with news and publications in language testing just visit the relevant page and you can take out a subscription to one of the many webslices or news feeds.

LTRC 2011 Program Available for Download

3 May 2011 15:30:28 GMT

The complete LTRC program is now available from the University of Michigan website.

New Podcast: John Read on Vocabulary

14 Apr 2011 14:40:18 GMT

The journal Language Testing has seen a flurry of articles on vocabulary testing in recent months, and issue 28(2) is no exception, with Marta Fairclough's paper on the lexical recognition task. It seemed like an appropriate moment to conisder why vocabulary is receiving so much attention, and so we turned to Professor John Read of the University of Auckland, New Zealand, to give us an overview of current research and activity within the field.

BALEAP 2011

5 Apr 2011 08:15:20 GMT

This coming weekend is the annual BALEAP conference, which is being held in Portsmouth. It's not one that I've been to before but was tempted this year because the conference dinner is on board HMS Warrior. I will be giving a paper with Jenny Kemp on Performance Decision Trees and their potential application in EAP settings. The programme is available from the conference website.

Interim Article Update

5 Apr 2011 08:02:20 GMT

I've just updated the articles page with a number of new publications that visitors to the site have let me know about. These include articles on vocabulary assessment by John Read, which come highly recommended.

April 2011 Update: New Resources

1 Apr 2011 09:02:49 GMT

This month the website is launching a new resource. William Bonk (Colorado University) has given a set of extremely useful excel spreadsheets to the site. The spreadsheets calculate the basic statistics developed in classical test analysis for closed response items such as multiple choice. These include distractor analysis, item facility, a discrimination index, reliability, and descriptive statistics (mean, standard deviation, and standard error of measurement). The spreadsheet for Cronbach's alpha also handles partial credit data. These will be very valuable for students on testing programmes in particular, and I have already tried them out with my own students. Visit the new page to see what is available by clicking on the 'statistics' link in the resources navigation panel. And our thanks to Bill! Just click here to download the files.

LTRC 2011 is only just around the corner. Registration is now open, and it's eseential to book soon if you want to attend a pre-conference workshop. They get booked up very quickly. Follow the link in the left-hand navigation pane, or below in the conference news section. I've also recently added a rotating links bar to make life easier.

Navigating this site should be fairly easy, but one thing I've never had is a sitemap. A recent visitor suggested it might be useful to have one just in case some people preferred to see a tree structure of the site, so that was added at the end of last month. It also provided me with an opportunity to try my hand at programming a tree-menu, which was great fun.

The next major update to the site is likely to be on 3rd May, which is after the long Easter vacation. But in the meantime there will be daily news updates, and weekly updates on most pages across the website.

Two more articles uploaded

5 Mar 2011 15:57:37 GMT

Today I've uploaded two more of my own articles to the website. These were both published in 2003 in Language Testing: Interface Design in Computer-based Testing, and Task Difficulty in Speaking Tests. Many more of my own publications are available from my personal page on the website. Just scroll down to the section on selected papers. I will try to make more available once they are over three years old, which is when royalties expire on journal articles through the Authors' Licensing and Collection Society.

2011 Summer Internships at ETS

2 Mar 2011 12:28:39 GMT

The English Language Learning (ELL) program at ETS (Educational Testing Service) in Princeton, N.J, is now accepting applications for summer interns.

The ELL summer internship program will begin on July 5 and end on August 12, and interns are expected to work for the entire program. We pay $25/ hour and expect interns to work a 40-hour week, 8:30-5:00, Monday through Friday. There is no overtime and no weekend work. All work is conducted at our Princeton office at the intersection of Carter and Rosedale Roads in Princeton, N.J. Interns must arrange their own housing and transportation.

ELL interns will produce materials for use on large-scale, high-stakes standardized tests of English language proficiency. Such work might include writing questions that test knowledge of grammar, vocabulary, and reading comprehension; creating conversations and talks that test listening comprehension; and developing scenarios and prompts that allow candidates to demonstrate their writing and speaking skills. Hires will work on one section (one skill) of one test: TOEFL, TOEIC, or K-12 Assessments. Four test sections are hiring for summer 2011, and there will be multiple hires for each test section. The work is intellectually challenging and rewarding.

All interns must have excellent writing skills. The work requires verbal precision and sensitivity to nuance, analytic skill, attention to detail, and receptiveness to instruction. Interns must have a very high degree of fluency in English but do not need to be native speakers, nor do they need to be U.S. citizens. TOEFL, TOEIC, and K-12 English language assessments are global measures, so ETS actively seeks candidates who can bring diverse experiences and perspectives to the work.

Applicants are selected on the basis of their performance on work samples that are designed by us. There is a two-step application process. First, you must go to ETS Careers website and complete an online application http://www.ets.org/ell/intern.html. A day or two after submitting the online application, you will receive an e-mail directing you to a second website. This second website contains the work samples and the directions for completing them. Each of the four test sections hiring for summer 2011 is associated with a specific work sample. You may apply for as many of the four positions as you wish but must complete the corresponding work sample for each position of interest. All work samples, as well as an electronic résumé, must be submitted to me (jmalloy @ets.org) by Wednesday, March 23, 2011.

Work samples will be evaluated in April, and you will be notified of your status by April 30th. The ELL summer internship workforce includes undergraduate students, graduate students, and teachers.

Questions welcome.

Contact: Jeanne Malloy, ELL Summer Institute Coordinator
jmalloy@ets.org

TOEFL® COE 2012 Research Program

1 Mar 2011 16:12:13 GMT

TOEFL® Committee of Examiners (COE) research grants are intended to support research on topics related to TOEFL as identified by the Committee of Examiners and Educational Testing Service (ETS®). Studies involving the collection of new empirical data will normally be funded up to US$125,000. This amount will be inclusive of institutional overhead.

Preliminary applications described below should be submitted via e-mail to Toeflrc@ets.org by April 21, 2011.

Full details available on the ETS website.

Performance Decision Trees: A New Feature

1 Mar 2011 08:39:52 GMT

This month there a new feature on scoring speaking tests and Performance Decision Trees (PDTs). This is to complement an article just published in Language Testing (28)1 by myself, Fred Davidson, and Jenny Kemp, entitled Effective rating scale development for speaking tests: Performance decision trees. In the article we analyse a number of discourse features of service encounters, using original and published data to demonstrate how it is possible to create 'thick' descriptions of performance contexts to generate scoring instruments that do not suffer from descriptional inadequacy. We illustrate the poverty of measurement driven scales, such as those contained in the Common European Framework of Reference, and show how Performance Decision Trees can form a much sounder basis for understanding performance in context.

In this feature I explore further some of the issues surrounding PDTs, and make available some of the sound files from which the transcripts in the article were produced. We hope that this will make it possible for others to examine some of the data directly and therefore evaluate our arguments more objectively. But of course, this is an online piece, and so we are able to supplement the feature with some entertaining video clips that illustrate service industry 'scripts'.

March 2011: Articles Update

1 Mar 2011 08:23:39 GMT

This month there is a significant update to the articles page. I frequently come across new materials on the web when I'm searching for articles I need during my own research. This month there are some very exciting new additions, including a number of papers of significant historical interest such as Glass' 1978 Standards and Criteria.

KELTA 2011 Call for Papers

1 Mar 2011 08:21:54 GMT

KELTA (Korea English Language Testing Association) is inviting presentation proposals for its 7^th international conference on Saturday, August 20, 2011.

NB: Hospitality for Overseas Presenters:

KELTA will provide selected overseas presenters with free hotel room and board for two nights and three days, on condition that they submit their full texts before the conference day, which will be considered for inclusion in KELTA’s journal, English Language Assessment.

The details of the conference are as follows:

1. Name: KELTA 2011 International Conference

2. Date: Saturday, August 20, 2011

3. Venue: Korea Institute for Curriculum and Evaluation, Seoul.

4. Theme: Impacts of High Stakes Exams

5. Keynote and Plenary Speakers:

Sara Cushing Weigle (Georgia State University)

Antony Kunnan (California State University)

WonKey Lee (Seoul National University of Education)

6. Proposal Submissions:

(1) Deadline: May 1^st, 2010.

(2) Proposal Form: Please use the form below, or download the Presentation Proposal Form from www.kelta.or.kr

(3) Contact Persons: Submit your presentation proposal to:

Conference Chair: Dr. Dong-Wan Cho (Pohang University of Science and Technology) at dongwanc@postech.ac.kr

Secretary General: Dr. Tae-young Jeong (Korea Military Academy) at tyjeong@kma.ac.kr

MwALT & TSLL 2011 Call for Papers

23 Feb 2011 16:05:45 GMT

Exploring innovation in language assessment

September 16-17, 2011

Iowa State University

http://apling.public.iastate.edu/mwalt/2011/13MwALT_9TSLL_Callforpapers.html

The 13^th annual MwALT conference will be held in conjunction with the 9^th annual conference on Technology for Second Language Learning at Iowa State University. The integration of these two conferences provides an opportunity for examining a range of innovations in language assessments—many of which rely on computer technology—in the process of test conceptualization, development, delivery, scoring, and analysis. Abstracts for papers and posters developing the theme of innovation broadly construed are invited for the conference, but we are particularly interested in papers with implications for innovation in language assessment through computer technology.

PLENARY SPEAKERS:

Margaret Malone, Center for Applied Linguistics

Innovative language tests: What the test-takers say

Sara Weigle, Georgia State University

Automated essay scoring in high stakes language testing

INVITED COLLOQUIUM

Lia Plakans, University of Iowa

Investigating innovative assessment tasks

TYPES OF PRESENTATIONS:

1. Papers: Theory-oriented presentations or reports on completed research related to the theme of the conference. The presenter will have 20 minutes to present, followed by 10 minutes for questions and comments.

2. Posters: Presentations of work in progress and research at the planning phase. All poster presenters will give a 5-minute introduction of their study to the whole audience, and will then host their posters for the rest of the session. This is a great opportunity for graduate students to discuss their research plans.

SUBMISSION DEADLINE:

In order to be considered, proposals must be received via email by Monday, May 16th, 2011. Those submitting proposals will be notified of acceptance or non-acceptance by June 1, 2011.

INSTRUCTIONS FOR SUBMITTING AN ABSTRACT:

All submissions must be sent electronically to Hye-won Lee at hwlee@iastate.edu. Attach a MS Word file to an e-mail with the subject header “MwALT Submission – ‘your name’.”

Page 1: Abstract of proposed contribution with title. Abstracts should be no longer than 250 words for papers and 150 words for posters.

Page 2: Please include the following:

a) Title of presentation (10 words maximum);

b) Type of presentation (paper/poster)

c) Name(s) of the author(s) in the order you would like them to appear in the program, their institutional affiliation, including the city and state of the institution (Underline the name of the author who will be the correspondent)

d) AV equipment required (projector will provided)

LTRC 2011 Registration Open

23 Feb 2011 09:21:31 GMT

The following registration rates apply until May 9, 2011

ILTA Regular Member: $160 (€115)
ILTA Student Member: $100 (€75)
Regular Non-Member: $190 (€140)
Student Non-Member: $130 (€95)

Preconference Workshops

Regular: $145 (€105)
Student: $115 (€85)

Banquet and Presentation of Awards

Saturday, June 25, 2011: $60 (€45)

Registration received after May 9 carries an additional $25 fee.

ILTA Student Travel Award

18 Feb 2011 17:29:30 GMT

Deadline: March 15, 2011

PURPOSE OF THE AWARD
The ILTA Student Travel Award is to provide partial travel funding for students whose papers have been accepted at LTRC to attend the conference. This highly competitive award is given to two presentations.

DETAILS OF THE AWARD
Each award carries an honorarium of US $750 to be used towards travel expenses. If a paper has more than one student co-author, the $750 may be shared. In addition, each awardee will receive a free one-year membership to ILTA. They will also receive a waiver of conference registration fees. All awardees will receive official recognition in the LTRC Conference Program.

CRITERIA FOR APPLICATION
Graduate students may apply for this award if their LTRC 2011proposal was accepted. The proposal should be for a research paper or symposium paper; applications for poster or work-in-progress proposals will not be considered. The student must be listed as an author, but need not be the sole or first author. For co-authored papers, co-authors must be other students rather than supervisors, professors or other senior colleagues.

Graduate students are eligible for consideration if they:
1. were enrolled in a degree bearing program at the time of submission
2. are not receiving other financial support that covers a significant portion of the expenses to attend LTRC
3. do not study or live in the LTRC host site (e.g. students from Michigan would not be eligible in 2011)
4. have not won the ILTA Student Travel Award before.

APPLICATION PROCESS
To be considered for the award applicants must send a 1-page letter containing the information 1-5 below, as well as a copy of the paper proposal (i.e. the abstract).
1. a statement requesting consideration for the travel award
2. the proposal title of the paper proposal
3. a statement of financial need
4. an assurance that the student is enrolled in a doctoral program
5. the name and contact information of the student's advisor (his or her e-mail address).

Both the letter and the abstract should be in one MS Word document or in a PDF file. Send the document/file as an attachment to indiac@umich.edu. The subject line should read: ILTA Student Travel Award. In reaching a decision, the selection committee will consider the content and quality of both the application letter and the abstract.

ILTA GRANT FUNDING FOR WORKSHOPS AND MEETINGS

18 Feb 2011 17:23:40 GMT

ILTA aims to promote high standards of professionalism in language testing and assessment practice. One way of carrying out this mission is to encourage workshops and meetings where language testing experts educate and train teachers, test developers, and others, including policy makers, principals, and even test takers in key skills and knowledge in language testing/assessment.

ILTA offers individual grants of up to US$ 3,000 for carrying out an approved workshop/meeting designed to help diffuse knowledge among a wider group in different parts of the world. The aim of a workshop/meeting should be to promote understanding, familiarity, and knowledge of language testing issues and practice among various groups of test users. It is expected that the award will be used as 'seed' money to organise such a workshop/meeting and, ideally, to establish a permanent local organisation that that will conduct further practical activities in language testing in the future, especially in places in need of language testing expertise.

More than one award of US$ 3,000 may be made available in any one year, subject to sufficient funds being available. The procedures require that an award must be used within 15 months of awards (i.e. proposals approved in 2011 must be carried out by the June 30, 2012)

CRITERIA FOR SUBMISSION:
Those eligible to apply are individual language testers, teachers and others representing various types of institutions such as universities, research institutes, schools and testing organisations. Applicants must be ILTA members or, in the case of joint applications, at least one applicant must be an ILTA member. (In cases of extreme financial hardship the membership costs for ILTA may be included as part of the proposal, but a justification is needed in such a case.)

APPLICATION PROCESS:
Those interested can apply by sending a 3- to 5-page proposal which should include the following:
1. brief background regarding the state of language testing in the specific context (country, region, school system)
2. rationale for the meeting including specific reasons and needs for holding the meeting
3. description of target participant groups and their prior training/knowledge in language teaching and in language testing/assessment
4. plan for the meeting including specific themes to be covered
5. tentative list of speakers/workshop leaders plus information about their expertise
6. background of the organiser(s) in the area of language testing (the CV of at least one main organiser should be included to support this point)
7. duration and location of the workshop/meeting
8. intended audience for the workshop/meeting
9. amount of money needed and a specific description of how it will be used (including budget breakdown)
10. intended results of the workshop/meeting, specifically the short- and long-range impact the event is expected to have
11. a specific explanation of how the workshop/meeting will contribute to the mission and to the establishment of a permanent, local organisation
12. full contact details of the organiser(s)

The application must include a written statement of a commitment to provide ILTA within 3 months of the workshop/meeting, with an evaluative report of the event. This will be posted on the ILTA website and may be distributed to ILTA members.

APPLICATION DEADLINE:
Applications should be sent by e-mail to the ILTA President John Read at ja.read@auckland.ac.nz by March 31 2011. No late applications will be accepted.

SELECTION PROCESS:
Each application for grant funding will be evaluated by a special committee of ILTA Executive Board members, which is chaired by the ILTA President and includes one or more of the current ILTA Members at Large. The committee may also call upon other relevant experts to input to the selection process. In light of this, the committee will determine the likely evaluation/wait time needed before a decision can be made and will notify the applicant(s) accordingly.

CRITERIA FOR SELECTION:
Selection will be based upon the fit between the application and the criteria which are outlined above. Priority will be given to contexts where language testing is not well-established so that the workshop/meeting can contribute significantly to increasing awareness, knowledge and effective practice in language testing/assessment.

Advanced Notice of New Title

18 Feb 2011 14:18:57 GMT

There hasn't been much book news recently, so I thought that I'd let users know about a new title that will be available later this year, or very early next year. Fred Davidson and I are now working on the final edit, and the manuscript should be with the publisher in the not too distant future. I hope that this will be one of the most thorough and innovative books of its kind.

Fulcher, G. and Davidson, F. (forthcoming). The Routledge Handbook of Language Testing. London and New York: Routledge.

This will appear in the new Handbooks in Applied Linguistics Series.

Contents:

Editorial. Glenn Fulcher and Fred Davidson.

Section 1: Validity

Chapter 1: Concepts of validity. Carole Chapelle, Iowa State University.
Chapter 2: How to articulate a validity argument. Michael Kane, Educational Testing Service, New Jersey.
Chapter 3: Validity issues in designing accommodations. Jamal Abedi, University of California.

Section 2: Classroom Assessment and Washback

Chapter 4: Classroom Assessment. Carolyn Turner, McGill University.
Chapter 5: Washback. Dianne Wall. Lancaster University.
Chapter 6: Assessing young learners. Angela Hasselgreen. Bergen University College.
Chapter 7: Dynamic Assessment. Marta Anton, Indiana University-Purdue University at Indianapolis.
Chapter 8: Diagnostic Assessment. Eunice Jang, University of Toronto.

Section 3: The Social Uses of Language Testing

Chapter 9: Language testing for immigration, citizenship and asylum. Antony Kunnan, California State University.
Chapter 10: Designing language tests for specific social uses. Carol Moder and Gene Halleck, Oklahoma State University.
Chapter 11: Language Assessment for Communication Disorders. John Oller, University of Louisiana at Lafayette
Chapter 12: Social Dimensions of Language Testing. Richard Young, University of Wisconsin.

Section 4: Test Specifications

Chapter 13: Test specifications and criterion referenced assessment. Fred Davidson, University of Illinois at Urbana-Champaign.
Chapter 14: Evidence-centred design in language testing. Robert Mislevy, University of Maryland, and Chengbin Yin, Center for Applied Linguistics.
Chapter 15: Claims, evidence and specification design in performance assessment. Steve Ross, University of Maryland.

Section 5: Writing Items and Tasks

Chapter 16: Item writing and writers. Dong-Il Shin, Chung-Ang University, South Korea.
Chapter 17: Writing integrated items. Lia Plakans, University of Iowa.
Chapter 18: Test taker strategies and task design. Andrew Cohen, University of Minnesota.

Section 6: Prototyping and Field Tests

Chapter 19: Pre-operational Testing. Dorry Kenyon and David MacGregor, Center for Applied Linguistics, Washington DC.
Chapter 20: Prototyping new item types. Susan Nissan and Mary Schedl, Educational Testing Service, New Jersey.
Chapter 21: Piloting vocabulary tests. John Read, University of Aukland, New Zealand.

Section 7: Measurement Theory and Practice

Chapter 22: Classical Test Theory. J. D. Brown, University of Hawaii.
Chapter 23: Item Response Theory. Gary Ockey.
Chapter 24: Reliability and Dependability. Neil Jones, University of Cambridge ESOL.
Chapter 25: The generalisability of scores from language tests. Rob Schoonen, University of Amsterdam.
Chapter 26: Rating scales in performance testing. Glenn Fulcher, University of Leicester.

Section 8: Administration and Training

Chapter 27: Quality management in test production and administration. Nick Saville, University of Cambridge ESOL.
Chapter 28: Interlocutor and rater training. Annie Brown, Ministry of Education, United Arab Emirates.
Chapter 29: Technology in Language Testing. Yasuyo Sawaki, Waseda University, Japan.
Chapter 30: Automated Scoring. Xiaoming Xi. Educational Testing Service, USA.

Section 9: Ethics and Language Policy

Chapter 31: Ethics. Alan Davies, University of Edinburgh.
Chapter 32: Fairness. Scott Walters, Queens College, City University of New York.
Chapter 33: Standards-based testing. Thom Hudson, University of Hawaii.
Chapter 34: Tests in language management. Bernard Spolsky, Bar Ilan University, Israel.

As soon as it's available for pre-order on Amazon I'll put an announcement on the bulletin.

Cultural Validity Publication

2 Feb 2011 10:44:21 GMT

New from Routledge this month is Cultural Validity in Assessment: Addressing Linguistic and Cultural Diversity, edited by Maria del Rosario Basterra, Elise Trumbull and Guillermo Solano-Flores, ISBN: 0415999804. Although culture is a factor that is taken into account by language testing researchers, to my knowledge this is the first book to deal specifically with the topic. Here is the blub from the publisher:

What is assessment and how is it a cultural practice? How does failure to account for linguistic and cultural variation among students jeopardize assessment validity? What is required to achieve cultural validity in assessment? This resource for practicing and prospective teachers – as well as others concerned with fair and valid assessment – provides a thorough grounding in relevant theory, research, and practice. The book lays out criteria for culturally valid assessment and recommends specific strategies that teachers can use to design and implement culturally valid classroom assessments.

Assessment plays a powerful role in the process of education in the US and has a disproportionately negative impact on students who do not come from mainstream, middle-class backgrounds. Given the significance of testing in education today, cultural validity in assessment is an urgent issue facing educators. This book is essential reading for addressing this important, relevant topic.

You can order this through the languagetesting.info bookstore. Click on general books, and to go page 5.

February Update

1 Feb 2011 10:27:21 GMT

This month Language Testing (28)1 is published, and with it comes a new podcast that I produce with Cathie Elder for SAGE publishers. This time we look at the use of Think-Aloud protocols in language testing research, with the help of Khaled Barkaoui from York University in Canada, and Melissa Bowles from the University of Illinois at Urbana-Champaign.

The articles page has been updated yet again. The number of freely available publications has been rising quite rapidly recently, so you may have to use the search feature that I added to the page a couple of months ago to find what you want. Pay-for publications can be found in the journals pages, which are updated on a daily basis as new content becomes available.

Do take a look at the coming soon section this week, as there is a new feature on the way, and some Excel files for test analysis, courtesy of William Bonk.

A Satellite Industry

13 Jan 2011 08:43:15 GMT

High stakes testing has always generated satellite industries. The most visible and frequently discussed are test preparation institutions on the teaching side, and the sale of test preparation materials on the publishing side. However, over the last few years another industry has grown very rapidly on the back of language testing: the sham marriage market. Frequently operated by lawyers, marriage/immigration/testing agencies advertise for girls who have passed the language test who wish to study abroad but cannot afford the cost, and match them with boys whose family have the money to fund their studies but who cannot get the grade required for entry to another country. The marriage is arranged, the couple emigrate, and once the boy achieves permanent residency they apply for a divorce. This week the Times of India reported on a case where a couple who had married for this purpose had arrived in Australia and the girl had immediately filed for a divorce, which was granted before the boy had achieved permanent resident status. As a result, the boy's family have claimed breach of contract, and the girl's family have been charged with fraud.

This is a very large industry.

Where does responsibility lie for its creation (if anywhere)? And what can be done to discourage this industry? The UK government thought to address this question (in part - I'm sure there were other motives) by imposing language tests on spouses, but were immediately accused of introducing regulations that would "tear migrant families apart", or stop men from marrying non-UK brides unless their English was at the required level. Perhaps using more language testing to solve a language testing problem isn't the answer.

What is? Or is this beyond the boundary of what we think might be our consequential concerns?

I'm going to post this on the ltest-l discussion list shortly. So if you wish to see any responses, that's where to go.

First language testing book of 2011

7 Jan 2011 15:48:00 GMT

Available now in the UK, but awaiting release in the US in March, is a new book by Bethan Marshall, entitled Testing English: Formative and Summative Approaches to English Assessment from Continuum. ISBN: 1441194266, 168 pages. The blurb says:

This title considers why summative assessment in English is difficult and explores viewing it as an arts subject rather than one which is quantifiable and assessable objectively. "Testing English" considers why English is such a difficult subject to assess summatively and takes the view that English is an arts subject rather than one which is quantifiable and assessable objectively. Bethan Marshall examines the nature of the subject, the battlegrounds of examinations over the last 100 years and considers some of the solutions that have been put in place to overcome the problem both in the UK and abroad. "Testing English" looks at the way English lends itself to formative assessment in that it actively encourages dialogue with the pupils in the absence of 'right answers'. It explores the complex relationship between formative and summative assessment and considers the relationship in the light of the introduction of Assessing Pupil's Progress (APP).

Available from the bookstore under the general books tab.

2010 Site Statistics

1 Jan 2011 17:35:09 GMT

Website statistics may not seem like the most exciting topic on earth, but I thought that regular visitors to the site might like to know something about its use in the last year. I have a couple of programs running in the background that collect information on traffic – nothing ‘personal’ or attributable to individuals – don’t worry. It’s aggregated information only that helps with managing the website, planning content, and designing for particular browsers.

So, in 2010 there were 73,956 separate visits to the Language Testing Resources website, with 816 GB of content downloaded. The home page was the most frequently viewed with 22,928 visits. This was followed by the employment opportunities page at 11,155 visits. In third place was the articles page with 7,902 visits, in fourth place the ‘what is language testing’ page with 7077 visits, and in fifth place the video resources with 4,450 users watching one or more videos.

What about the users? The largest single group of site users are based in the United States, with a total of 56,722 pages downloaded in 2010. This was followed by the United Kingdom at 44,285 pages, China on 13,923 pages, Canada on 10,539 pages, and then other ‘European countries’ (people from providers with a .eu extension) on 10,288. In slots 6 to 15 were (in sequence) visitors from Indonesia, Iran, the Philippines, Australia, Germany, Turkey, Russia, South Korea, Japan and Italy. Perhaps surprisingly, only 25.5% of visitors arrive at the site via a search Engine – primarily Google, and then Bing. Only 5.8% of visitors come to the site from a link on another site. The top feeder page is the ILTA website. However, after this the top feeder pages are inside virtual learning environments (VLEs) like Blackboard or Moodle, where my website is being used as part of a language testing course. I can’t see inside these sites, as they are password restricted. The list is topped by the University of Maryland, followed closely by Yeditepe University (Turkey), Southern Taiwan University, the University of Innsbruck, and then Carleton University in Canada. Most interestingly, 68.3% of users come to the site from a direct address: a bookmark in the web browser, including access via the website’s rss feed, or the webslices. This gives me some idea of how popular these services have proven to be.

A few other bits of interesting information are available as well. 59.8% of users use Internet Explorer, and so can take advantage of the webslicing alert services, while 28.2% prefer Firefox, 5% use Google Chrome, and 3.6% Safari. The average time spent on the site is around 6 minutes per visit, however this hides a very wide range. Some people leave fairly quickly (it isn’t what they’re looking for), while others stay on the site for a long time. In 2010 some 2541 visitors (3.4% of traffic) spent over 1 hour on the site, with 3016 visitors (4.0% of traffic) spending between 30 minutes to 1 hour on the site at a time.

Happy 2011

1 Jan 2011 09:12:49 GMT

Happy 2011! In the first update this year I've added the fifth scenario on testing legal English. Much to my surprise, there isn't as much published in this field as there is for other specific examples of language use, such as medical or aviation language. I think that there are significant opportunities here for research. Links have been added to existing tests of legal English. As with other scenarios the links will be updated on a weekly basis.

There are tweaks throughout the site to make it more user friendly. I have spent a little time during the vacation to update search engines and filters to pick up any testing articles appearing in journals, and to weed out irrelevant content. I'm grateful to site users who let me know of things that I don't know about so that they can be added when it's technically feasible.

I hope you'll find your way back to the site often in 2011 to keep up to date with what's going on in the field of language testing.

Opportunities with Pearson

29 Dec 2010 10:50:08 GMT

Pearson currently has a call for proposals for funded research projects, with a closing date of 28th February, 2011. The company has also announced an open call for internships, suitable for students of language testing to gain some experience working with a testing agency.

Merry Christmas

21 Dec 2010 11:46:32 GMT

I'm now on vacation until the New Year, and there is usually very little going on in the world of language testing over Christmas. However, the news will be updated on the home page if something does happen, as will the employment page. The next major update will be in January, when another scenario will be ready. In the first couple of months of the first year look out for a new podcast on think-aloud protocols in language testing research, and a new feature on rating speaking tests, with an introduction to performance decision trees. Lots more stuff is planned for the rest of the year.

In the meantime, if you haven't already taken a look, I hope you enjoy the 2010 Review of the Year.

Have a very merry Christmas, and a Happy New year.

4th ALTE Conference Call for Papers

17 Dec 2010 12:35:53 GMT

The Impact of Language Frameworks on Assessment, Learning and Teaching, viewed from the perspectives of Policies, Procedures and Challenges

7-9 July 2011 in Kraków, Poland

ALTE welcomes papers in German, English, Spanish, French, Italian and Polish related to this theme such as:

The extent to which frameworks affect language teaching, learning and assessment at individual, institutional, community and societal level
The extent to which frameworks are useful for language teachers and learners
Ways in which language frameworks can inform how particular languages are described by providing information about criterial features to distinguish different levels of proficiency

Proposals can be submitted until 31st January, 2011.

Links and Articles Pages Updated

1 Dec 2010 15:53:18 GMT

From time to time I check every single link on a page. There's nothing more infuriating than clicking on links to pages or resources that no longer exist. This used to be a terribly tedious task, but these days there is software that does the job for you. People who have websites should really keep them up to date - and I try my best. But if you do come across a broken link please let me know, and I'll try to fix it or remove it as quickly as possible.

New Feature for December

1 Dec 2010 15:49:09 GMT

In December and January each year, news channels around the world reflect on the stories and events that hit the headlines in reviews of the year just gone. And so on this website we take a look back at the language testing stories of the year. These are organized both thematically and chronologically. All stories have links to interesting websites, and there's video and audio to illustrate the issues at stake. Needless to say the selection of material is entirely personal and rather idiosyncratic, but I still hope you will find the feature an interesting take on the year.

E-portfolios Book Published

26 Nov 2010 12:24:52 GMT

Cambridge, D. (2010). e-Portfolios for Lifelong Learning and Assessment. Published by Jossey Bass, this month. ISBN 0470503769. Here's the blurb:

This book clearly articulates the foundations of an educational vision that is distinctively supported by eportfolio use, drawing on work in philosophy, sociology, higher and adult education, and elearning research. It is academically rigorous and accessible not only to scholars in a range of disciplines who might study or use eportfolios. It surveys the state–of–the–art of international eportfolio practice and suggests future directions for higher educational institutions in terms of curriculum, assessment, and technology. This resource is written for scholars, support staff, instructional technologists, academic administrators, and policy makers.

This is one that I've already ordered for myself, as I often feel that assessment on my own language testing courses is a little too traditional at times. I'm hoping for inspiration! You can order a copy through the languagetesting.info bookstore, under the alternative assessment tab (last one in the list).

Nominations for Best Article of 2009

20 Nov 2010 15:12:10 GMT

If you haven't yet nominated an article to receive this ILTA award for 2009, you have until 15th December. Just click on the link for instructions.

Diagnostic Assessment and Interventions

11 Nov 2010 10:51:54 GMT

Arwood, E. L. (2010). Language Function: An Introduction fo Pragmatic Assessment and Intervention for Higher Order Thinking and Better Literacy. Jessica Kingsley Publishers. ISBN 1849058008.

Here's the blurb: "Literacy teaching tends to take a structural approach to language, focusing on concepts such as sounds, morphemes, words, sentences, and vocabulary. However, new research suggests that the majority of English speakers actually think and learn in visual terms, and that there is a cultural and linguistic mismatch between language teaching and language learning. This has important implications for educators of students with neurogenic disabilities, such as autism. In her new book, Dr. Ellyn Arwood outlines a revolutionary new four-tiered model of how a learner acquires language, and suggests ways to impose visual language functions onto language in order to assist both neurotypical learners and those with neurogenic disabilities, such as autism spectrum disorders and ADHD. Dr. Arwood provides tried-and-tested intervention strategies that can be used with such individuals at all levels, as well as tools for working effectively with those who think in terms of a visual, spatial, contextual kind of language. This book takes a fresh look at how language and literacy interact, and will be of interest to educators and special educators, speech and language pathologists, occupational therapists, and other professionals who support language learning and development."

Available through the bookstore under the diagnostic testing tab.

New Education Mashup

11 Nov 2010 10:43:51 GMT

As one or two people may have noticed, I've recently upgraded my own personal page on the website. It had always been a bit of a tombstone. Recently there was a great article in the Times Higher Education Supplement on the more creative use of personal information for academics. I didn't actually like any of the "good examples" provided, but it made me realise just how dull my own page was. But to the point. Language Testing and educational assessment generally sits within a larger educational and policy context, and so I decided to inlucde a few 'hot' educational news items on my web page. I've had a few emails to say that this is really useful, and could I make it more easily accessible. So I've created a mash that some people might find useful. Just click on the link and take a look.

2011 Summer Internships at ETS

9 Nov 2010 07:43:48 GMT

Updating the Test Frameworks for TOEFL iBT (12-week internship)

The TOEFL iBT^TMtest was first launched in selected countries in 2005 and then worldwide in subsequent years. The development of the revised test began in mid 1990s with reviews of research about the English language skills needed to participate in English-medium post-secondary education. Subsequently, in 2000, frameworks were developed by language testing experts for the overall test and for each of the four sections, reading, listening, writing and speaking ( Bejar et al., 2000; Butler et al., 2000; Cumming et al., 2000; Enright et al., 2000; Jamieson et al., 2000). The frameworks laid the groundwork for the design of the prototype test items, which were piloted, revised and included in the final test design. However, they are a decade old and are in need of updating to reflect recent conceptual and practical developments in language testing, and to guide future revisions of TOEFL.

We are looking for four interns to conduct critical literature reviews of the current trends and practices of language testing and assessment. The reviews will include recent developments in how the constructs of language abilities are defined with a focus on tests for specific purposes, and new advances in innovative item types and measurement approaches. Each intern is expected to focus on one of the four skills.

The selected interns will be mentored by senior researchers and assessment specialists at ETS, and be invited back in fall 2011 to review and discuss the findings with ETS team members, the TOEFL advisory committee, and a number of leading scholars specializing in the assessment of each of the four skills. Successful interns may have opportunities to continue to work with the teams on updating the test framework papers and co-authoring papers and presentations.

Required qualifications and skills:

Excellent knowledge in second language learning and assessment.

Strong conceptual thinking skills.

Excellent writing and interpersonal skills

New CEFR Title in SILT Series

8 Nov 2010 10:59:16 GMT

Martyniuk, W. (Ed.) Aligning Tests with the CEFR: Reflections on Using the Council of Europe's Draft Manual. Studies in Language Testing 33. Cambridge: Cambridge University Press. ISBN: 0521176840. 296 pages. Here's the blurb:

"The Council of Europe released a preliminary version of the Manual for Relating Language Examinations to the Common European Framework of Reference for Languages (CEFR) in 2003. A variety of institutions and individuals conducted case studies to pilot this draft version, prior to its revision in 2008/9. This volume features selected case studies presented at a colloquium in Cambridge in December 2007, where practitioners and academics shared their experiences of applying the Manual procedures. Projects included the linking of a single test to the CEFR, the linking of suites of examinations at different levels and large-scale national studies undertaken by examination boards and specialist research institutes. As well as describing their approaches and reporting their findings, contributors reflect and comment on their experience of using the draft Manual. An introductory chapter explains the development of the CEFR and the draft Manual, discussing their relevance for the future."

I shall look forward to reading this book, as I have been a long-standing critic of the CEFR, the manual, and the approaches that have been developed around so-called 'alignment' or 'linking'. In my 2010 book (Practical Language Testing), I argue that many of these practices subvert validity theory, with reference to existing published literature. It is therefore excellent that we have more of this in the public domain so that it can be independently critiqued - assuming the authors provide us with enough evidence to do the job. So this volume is a very welcome addition to the literature.

It is now available in the UK, but can be pre-ordered for delivery shortly in the US. Copies can be ordered from the languagetesting.info bookstore, under the Cambridge SILT Series tab.

Improved News Filters

1 Nov 2010 06:18:43 GMT

As you'll probably realise, I don't spend all my time sitting in front of my computer googling or binging stories about language testing. Over the last two years I've developed a search engine to do this for me, along with the filters necessary to weed out stuff that isn't really about testing and assessment. One or two 'rogue' items have slipped through recently, and so I've put some time aside during the last month to tweak the filters to exclude the stuff that's been getting through about film (strong 'language' and 'testing' the market etc.) It's inevitable that some stories will get through that aren't strictly relevant to our subject, but these should be few and far between now. My search engine runs checks on all major news sources every four hours. If you subscribe to the webslice your browser's tool bar will automatically alert you to new items. If you still don't know what a webslice is, check out the technical page for an explanation.

New Scenario for November

1 Nov 2010 06:15:52 GMT

This year I have been using the new scenarios with my own language testing class to look at issues that arise in particular assessment contexts. This month I am adding the fourth in the series of six that I'm developing, this time on the assessment of communication in medical contexts. I hope that you find these useful too, whether you're a student of language testing, or you teach language testing.

More on Technology

24 Oct 2010 08:31:32 GMT

Devi, A. (2010). Computer Assisted Language Learning, Teching and Testing (CALLTT): Computer assisted language teaching and testing with multimedia material - some aspects. LAP LAMBERT Academic Publishing, ISBN: 3838395522.

First of all, I have to say that I haven't read this, and at well over $100 I don't think I'll be investing in a copy. It sounds very much like a publication version of a PhD dissertation, but I don't really know. If the author sees this perhaps she can let us know! However, for those out there who are researching in this area and so desperately need to know what's going on, here is the blurb:

"Computer Assisted Language Learning has created a growing interest among educators, teachers and researchers in developing and using software. Moreover, computers have facilitated the teaching and testing procedures of certain skills like speaking and listening, adopted for training large groups of learners. These skills are no doubt taught in conventional language classrooms but not always tested. The computer with multimedia technology makes it possible for us to test the proficiency levels of these skills also. This is because multimedia can present information effectively in an integrated format using text, graphics, sound, animation and video. The present study aims at finding out the pedagogical significance of the use of computers along with that of tailor made software meant for language teaching and testing. Five null hypotheses were formulated that enabled the researcher to analyse the results obtained from experiments conducted."

If you have some cash to spare, you can order a copy from the bookstore under the technology and testing tab.

Updated Rubrics Book

24 Oct 2010 08:04:29 GMT

The 3rd edition of Kathleen Burke's From Standards to Rubrices in Six Steps: Tools for Assessing Student Learning has just been released in the US. Published by Corwin Press, ISBN 1412987016, it is 236 pages long. This doesn't have a language focus, but does have ideas relevant to language testing. Here is the official blurb:

In this new edition Kay Burke provides an updated, comprehensive six-step walk-through of how to create tasks that promote learning for all students and how to write rubrics linked straight to the requirements of state standards and the No Child Left Behind Act. Burke demonstrates how to build a performance task unit as the ideal curriculum framework for clustering standards and differentiating instruction within a single cohesive unit of study. She shows how to share checklists and rubrics with students for ongoing formative assessment and self assessment.

Teachers and administrators will find everything they need, including:

- Templates for all six steps

- Tools and tips to help teachers build their own tasks, checklists, and rubrics

- Theory, examples, applications, and explanations to help apply the six-step process

- Guidance for differentiating for special needs within standards-linked tasks

- Sample tasks, rubrics, and units

The book can be ordered through the bookstore, under either the classroom assessment or standards-based assessment tabs.

Measurement and Evaluation in Post-Secondary ESL

16 Oct 2010 07:48:45 GMT

Just published by Routledge (ISBN 0805861262), and coming in at 128 pages, this new book by Glayol Ekbatani of St. John's University, Jamaica, is a basic introductory text. The blurb says:

Practical and concise, this introductory text for language teaching professionals is a guide to ESL assessment and to fulfilling the testing component of TESOL programs in the U.S. and around the world. Covering the fundamental descriptive and quantitative facets of effective language testing, it explicates key technical aspects in an accessible, non-technical manner. Each chapter includes relevant practical examples and is augmented by a partnered project that provides practical opportunities for readers to apply the concepts presented in real testing situations.

Available on page 4 under the general books tab in the Bookstore.

ETS R&D Fellowship and Internship Programs

16 Oct 2010 07:48:44 GMT

The goal of the ETS R&D Fellowship and Internship programs is to promote quality and distinction in educational measurement and related fields through support of significant research by early-career scientists and graduate students and exposure to methodologies within the ETS environment. These programs provide opportunities for talented scholars and students from diverse backgrounds – especially traditionally underrepresented groups such as African Americans, Hispanic/Latino Americans, and American Indians – to pursue scientific research under the guidance of ETS senior scientists and psychometricians. These programs encourage research in areas such as educational measurement, psychometrics,validity, natural language processing and computational linguistics, cognitive psychology, learning theory, linguistics, speech recognition and processing, teaching and classroom research, statistics, international large scale assessments, and assessment design and development.

Summer Internship Program in Research for Graduate Students

Selected interns participate in research projects under the guidance of ETS mentors in Princeton, NJ. Graduate students who are currently enrolled in a full-time doctoral program in one of the areas listed above and who have completed a minimum of two years of coursework toward their PhD or EdD prior to the program start date are eligible to apply.

Harold Gulliksen Psychometric Research Fellowship Program

During the academic year selected fellows study at their universities and carry out research under the supervision of an academic mentor and in consultation with an ETS research scientist. During the summer, fellows are invited to participate in the Summer Internship Program for Graduate Students working under the guidance of an ETS researcher. The program is open to applicants who are enrolled in a doctoral program in psychometrics or statistics, have completed their course work toward the PhD, and are at the dissertation stage.

Postdoctoral Fellowship Program

Selected fellows conduct research under the mentorship of ETS senior researchers in Princeton, NJ. The program is open to early-career scholars who hold a PhD or an EdD in one of the areas listed above.

Sylvia Taylor Johnson Minority Fellowship in Educational Measurement

Selected fellows conduct research under the mentorship of ETS senior researchers in Princeton, NJ. The program is open to candidates who have received their PhD or EdD within the past ten years in one of the areas listed above and who are US citizens or permanent residents.

To Apply:

The application process for 2011 will open on November 1, 2010. No applications will be accepted prior to this date. Apply online at the ETS Fellowship and Internship Programs Website:

http://www.ets.org/research/fellowships.html

The deadline for applying for the summer internship program is February 1, 2011.

The deadlines for applying for the Harold Gulliksen program are December 1, 2010 for the preliminary application materials and February 1, 2011 for the final application materials.

The deadlines for applying for the postdoctoral fellowship programs are January 1, 2011 for the preliminary application materials and March 1, 2011 for the final application materials.

Contact

E-mail: internfellowships@ets.org

Phone: (609) 734-5543

October Update

3 Oct 2010 10:05:58 GMT

Language Testing Bytes Issue 3 is released this month, in conjunction with Issue 27(4) of the journal Language Testing from SAGE. In this podcast Jim Purpura is interviewed on the topic of assessing grammar. This and previous podcasts can be downloaded from the podcast page.

The Bookstore has been updated with new titles that might be useful in courses on language testing, including texts on research methods and statistics. It also acts as an information resource on what's currently available on language testing in print. Please let me know if you know of relevant books that I haven't listed.

The articles page has been updated with new material, and I've also added three new applied linguistics journals to the article aggregation page. The article alert page has to be different because of the misuse of time stamps by some journals in their RSS feeds. The article alert page only contains journals that use correct time stamps, otherwise the webslice feature wouldn't work!

Article Update

19 Sep 2010 10:27:08 GMT

This weekend I've been working on a paper in which I had to refer to the literature on linking educational assessments, and needed to refer to the classic 1992 work by Mislevy, Linking Educational Assessments: Concepts, Issues, Methods, and Prospects. In my lit. search I discovered that the entire work is now available online in pdf. Click here to download. I'll add this to the next update to the articles page.

Testing Chapter in New Book Hits Headlines

17 Sep 2010 15:53:34 GMT

Published this week: Bangs, J., MacBeath, J., and M. Galton (2010). Reinventing Schools, Reforming Teaching: From Political Visions to Classroom Realities. London: Routledge.

This edited book looks at the ways politicians see education, and how they frame policy to get the educational system to achieve the society they wish to create. It is about policy in England, but this shouldn't limit its usefulness in analysing the relationship between government and educators. But for people interested in testing and assessment, the key chapter is on the UK's examination system by Mick Waters. There are two basic claims in the chapter. Firstly, that in a commercial environment where it is necessary to drive up test volume, the examination boards 'dumb down' the tests to make them more attractive to schools who wish more students to pass so that they will rise up the league tables. Secondly, that the examination boards make further profit from publishing textbooks that prepare students for the examinations they produce, thus including advice and topics that are very close to what is going to appear in the actual examination. This practice is labelled 'insider dealing'. The discussion raises many serious issues that have been ignored in recent years regarding the extent to which examination boards should also be involved in publishing and teacher training.

This chapter has already hit the newspapers. You can read about it in The Independent and The Guardian. Not surprisingly, the examination boards have already publicly denied the accusations. It is always difficult to know the truth of the matter unless there is empirical evidence to support the claims made, but the chapter neverthless draws our attention to some of the ethical issues that constantly re-emerge in the testing marketplace. The larger theoretical issue at stake is what I call the "separatist vs. integrationist" debate (see Practical Language Testing, pp. 282 ff.). Reinventing Schools, Reforming Teaching is available from Amazon.

Change to the Employment Page

15 Sep 2010 14:50:06 GMT

Recently, you will have noticed that I've stopped truncating the job titles on this page. At the same time I also shortened the date information. A few users have asked me to put the full date and time stamp back in so they know precisely when the job came on the market. I've done this, and made it accurate to the minute! I was happy to oblige, as this page is one of the most visited on my entire site. Please note that the time stamp is for the Central Time Zone, which during the summer is GMT-5, and in the winter GMT-6. Users who read the time stamps on updates to my site should remember that my server which puts these time stamps on pages is physically situated in Chicago, even though I'm in the United Kingdom. So if you're lucky enough to live in Chicago - it's your 'real' time!

Delay to October Update

15 Sep 2010 14:46:09 GMT

From time to time I need a vacation. When this happens I try not to stare at screens or do what I normally do throughout the year. So the next update to the website will take place on 4th October 2010, rather than on the first of the month as usual. However, it will be worth the wait. Next month the next issue of Language Testing Bytes will be released, featuring an interview with Jim Purpura on assessing grammar.

New Book on Classroom Assessment

15 Sep 2010 14:35:36 GMT

McMillan, J. H. (2010). Classroom Assessment: Principles and Practice for Effective Standards-Based Instruction. 5th Edition. Prentice Hall. Here's the blurb for this updated edition:

This text provides prospective and current teachers with a concise, non-technical, and practical guide to conducting a full range of high-quality classroom assessments. The text emphasizes assessment in the context of the realities of teaching and teacher decision-making in an era of standards-based education. Assessment methods are integrated with instruction and presented according to when teachers evaluate students (before, during, and after an instructional unit), the learning targets that are measured, and standards emphasized in state-wide testing. There is considerable emphasis on the nature of learning targets and how different assessments are most appropriate for different targets. For each assessment technique, suggestions for effective practice are presented with examples, case studies, and teacher interviews. This edition includes additional emphasis on formative assessment for student learning.

New Book on ELL Content Area Assessment

9 Sep 2010 13:58:53 GMT

New Publication: Mihai, F. (2010). Assessing English Language Learners in Context Areas: A Research-into-Practice Guide for Educators. University of Michigan Press. Here's the blurb:

Assessing English Language Learners in the Content Areas: A Research-into-Practice Guide for Educators seeks to provide guidance to classroom teachers, staff developers, and test-item designers who want to improve ELL assessment outcomes, particularly in the areas of math, science and social studies. The first two chapters of the book establish the background for the discussion of content-area assessment for ELLs, examining several important characteristics of this rapidly growing student population (as well as critical legislation affecting ELLs) and providing a description of various forms of assessment, including how ELL assessment is different from the assessment of English-proficient students. Important assessment principles that educators should use in their evaluation of tests or other forms of measurement are provided.

Other chapters review ELL test accommodations nationwide (because, surprisingly, most teachers do not know what they can and cannot allow) and the research on the effectiveness of these types of accommodations. The book analyzes the characteristics of alternative assessment; it discusses three popular alternative assessment instruments (performance assessment, curriculum-based measurement, and portfolios) and makes recommendations as to how to increase the validity, reliability, and practicality of alternative assessments. The book proposes fundamental assessment practices to help content area teachers in their evaluation of their ELL progress.

Remaining Work for 9th September

8 Sep 2010 19:46:32 GMT

The repairs to the video page have now been completed. However, other multimedia files have been deleted or corrupted during the server failure. Some of these are in the features, and others in the scenarios section of the website. Work on these will take approximately 6 hours, and will be completed by the end of 9th September. The entire website should then be working as normal. However, if you do come across any further problems please do contact me.

Video Outage (8th September)

8 Sep 2010 11:58:02 GMT

Most of the damage done to languagetesting.info by the hardware failures last weekend have now been corrected, and the new server should be running up to speed by the end of the week. However, kind users of the website have been e-mailing me to let me know about problems that I haven't spotted. The most serious is that the videos have stopped working. It turns out that when the server went down the entire video directory was wiped clean. I have two backups of all the videos, and have just started uploading these to the server. It will take most of the day to get the video page back up and running, but it should be complete by around 8 pm. BST, which would give us 3 pm. EST. I live in the former, but the latter is my server's time zone. Sorry for any inconvenience in the meantime.

Apologies for Outage

5 Sep 2010 19:49:40 GMT

The Lanugage Testing Resources website has been out of operation this weekend (4th - 5th September), due to my server crashing. I apologise for the 48 hour down time if you've been trying to access the website. However, a new server is now in place, and there shouldn't be any more interruptions. A number of updates to web pages and information have been lost, but these will be corrected in the next 48 hours. All subscription services are now back online. If you have any problems, right click on one of the LT tabs in the browser, and select 'refresh all'. All information should then be updated as and when the web pages are updated in the coming hours.

End of Truncated Titles

2 Sep 2010 16:05:26 GMT

On some of the languagetesting.info dynamic web pages you may have noticed that titles longer than around 100 characters were being truncated. This was particularly annoying on the article alert and article aggregation pages, as it was difficult to work out the complete title without clicking on the link. I've finally come up with a way to stop this happening, so from now on you'll get full titles on all dynamic pages.

Fun Stuff

1 Sep 2010 12:49:20 GMT

Long before Mr Bean, Rowan Atkinson made his name in a series called Not the 9 o'clock News, which was broadcast at the same time as the 9 o'clock news during the early 1980s. I was addicted to it at the time, and had forgotten all about the writing examination sketch. I was watching old episodes during a restful weekend in the summer and came across it again. This has now been added to the humour page as clip 6. It will certainly appeal to all the English language teachers who use this site. Other additions have been made to the page as well to remind us that there is a lighter side to testing!

Updates for September

1 Sep 2010 12:34:15 GMT

The first thing to check out this month is the Articles Page. This has had a complete makover, and many more articles added to the list. What appears to be happening at the moment is more and more individuals are making papers available on the internet once the 1-year limitation imposed by publishers expires. This allows researchers to post their papers on university or personal websites. We can expect more of this in the future.

The other major change to the website can't be observed. Hopefully you'll notice that pages load much more quickly than in the past - particularly the home page. This is because of a technical change: all updating is now done offline, and the page is reassmbled on-the-fly when nothing else is happening in the background. This means that your browser is never waiting for information from my server. While this may not sound very exciting, and there isn't anything to see, it reduces the frustration of waiting times quite significantly!

Other updates have been implemented throughout the site, my own personal page having been given the most complete overhaul, with a move away from static html.

2000 Publication Available to Download

29 Aug 2010 15:44:41 GMT

In the September update next week there will be many new editions to the article links. A great deal on language testing has appeared on the web over the summer, and I can only expect that this trend will continue now that individuals can upload their papers to personal websites after a year or so from publication. However, I thought that I would share this one a little early.

Geoff Brindley's edited volume Studies in immigrant English language assessment (2000) was well ahead of its time. The papers are arguably more relevant today than they were when they first appeared. I had no idea this volume was available in its entirity until I was searching for a reference for a paper I'm currently writing and came across it by chance. Here's the full content list:

Chapter 1
Assessment in the Adult Migrant English Program
Geoff Brindley

Chapter 2
Comparing AMEP assessments: A content analysis of three 45 reading assessment procedures
Geoff Brindley

Chapter 3
Issues in the development of oral tasks for competency-based assessments of second language performance
Gillian Wigglesworth

Chapter 4
Task difficulty and task generalisability in competency-based writing assessment
Geoff Brindley

Chapter 5
Rater judgments in the direct assessment of competency-based second language writing ability
David Smith

Chapter 6
Individual differences and learning outcomes in the Certificates in Spoken and Written English
Steve Ross

This volume is 146 pages in length and takes some time to download, so when you click on the link just wait for things to happen.

Dictionary 3

27 Aug 2010 08:47:09 GMT

From time to time there's a website designed for learners that I think is worth commenting on in the language testing bulletin, and this is one of them. I came across it via a review in Jazzou News, which I confess I'd never heard of until my language testing search engine discovered it. Dictionary 3.0 is a very useful web-based dictionary that also includes a thesaurus and a range of other useful tools. In the dictionary, apart from the usual definitions, the user gets relevant extracts from Wikipedia, famous quotations with the word included (if there are any), and lexical sets to which the word belongs. The vocabulary section provides lists of words that appear frequently in major US tests, and in the tools section you'll find some fun applications and a translator. One always has to be careful with online translators, but I tried this one with fairly simply phrases from English to a second language in which I'm fairly fluent, and it was accurate enough to be useful. Along with the tools in the Compleat Lexical Tutor (no, I haven't spelt that incorrectly), we have a set of online vocabulary building resources that can be used creatively in the classroom.

Article Downloads

23 Aug 2010 17:43:34 GMT

Next week I'll be adding a lot of new articles to the website, but as August has been so quiet on the language testing front (apart from the annual hand wringing about whether tests are getting easier in the United Kingdom), I thought that I'd link in a few from now. These are my own papers that we can now make available through a personal website one year after publication. My favourite is a piece published in Educational Philosophy and Theory entitled Tests in Life and Learning that I wrote with Fred Davidson (University of Illinois). We have been concerned for some time that social critiques of language testing have tended to rely too heavily on insights from Foucault, and this is our attempt to redress the balance. I've also uploaded an older paper from 1996 that I think is still valuable, as it addresses the legal framework of testing. This was published in System as I didn't get the grade I need. Where's my solicitor? For those of you not familiar with the UK system, a solicitor is a lawyer. Which is why I always find notices on houses in the US that read "No Solicitors" so amusing. More in early September.

Paperback Copy of Vol.7 Available

14 Aug 2010 14:19:29 GMT

The Springer Encyclopedia of Language and Education has been in publication for a number of years now, but when these multi-volume works are first released they are only sold as a set, therefore limiting sales to larger libraries. After a few years individual volumes are made available in paperback so that mere mortals like you and I can have a copy on our very own shelves. And so it has come to pass that Volume 7: Language Testing and Assessment, 502 pages, ISBN 10-9048191831, edited by Elana Shohamy, has been released for individual purchase. As a contributor, I've had a free copy on my shelves for a couple of years, and it's a volume that I go back to consult regularly, and so I can recommend it if you have some spare cash floating around - as this is still not cheap. Here is the blurb:

This volume addresses the broad theme and specific topics associated with current thinking in the field of language testing and assessment. The volume offers •multiple perspectives on the ‘what’ (the ‘trait’) of languages, and the ‘how’ (‘the method’) of assessment •multiple approaches developed for assessment especially given the multiplicity of languages used by many diverse groups of learners in many different contexts •focus on the societal roles of language testers, their responsibility to be socially accountable and to ensure ethicality and professionalism •focus on language testing in multilingual and diverse contexts This is one of ten volumes of the Encyclopedia of Language and Education published by Springer. The Encyclopedia bears testimony to the dynamism and evolution of the language and education field, as it confronts the ever-burgeoning and irrepressible linguistic diversity and ongoing pressures and expectations placed on education around the world.

The volume is available through the languagetesting.info bookstore, under the 'general books' and the 'dictionaries and research methods' tabs.

New Book on Formative Assessment

14 Aug 2010 14:11:11 GMT

At 160 pages long, Heritage's new book entitled Formative Assessment: Making it Happen in the Classrooom (ISBN 10-1412975042) is the latest in a string of books that looks at how classroom assessment can be used in learning. This book has a particular focus on the use of feedback, and how it is used. The product description from Corwin Press is simple and to the point: Packed with examples from various subjects and grades, this guide walks readers through every step of the formative assessment process, from articulating learning goals to providing quality feedback. I'm sure that Black and Wiliam didn't realise what they were starting when they first published Inside the Black Box.As you would expect, this book is about general classroom assessment, and not the assessment of language in particular. It is available from the languagetesting.info bookstore, in the classroom assessment category.

New Book on Testing Reading

5 Aug 2010 14:11:09 GMT

Published today, Components of L2 Reading: Linguistic and Processing Factors in Reading Test Performances of Japanese EFL Learners by Toshihiko Shiotsu is the the 32nd volume in the Cambridge Studies in Language Testing series. The blurb describes it thus:

"The book investigates the explanatory variables for the passage-reading comprehension performance of Japanese EFL learners and their sentence-reading speed. Following rigorous preliminary studies evaluating the significance of the candidate variables and refining research instruments, a large scale main study was conducted. Regression analyses indicated that syntactic knowledge was the best predictor of passage-reading ability, while vocabulary breadth also contributed to its prediction. Subgroup analyses supported the significance of syntactic knowledge for both higher- and lower-ability readers. Sentence-reading speed was best predicted by the speed of lexical semantic access. The significance of this speed was also consistent across the group division. A subsequent exploratory factor analysis and structural equation modelling of the data indicated that the performances of the participants were best explained in terms of two latent factors: one based on careful text processing power and the other on efficiency of lexical semantic access."

ISBN-10: 0521157277, and 248 pages, you can order a copy through the bookstore under the reading tab, or the Cambridge SILT series tab. However, you will have to wait a while before copies are available in the United States.

More on Language Testing and Immigration

3 Aug 2010 16:59:04 GMT

For those of you who follow the news (on the home page) on a regular basis, you'll know that over the past month the issue of language testing and immigration has dominated discussions in the UK, as the government tries to restrict student immigration to all those who are not at the (so-called) level B1 on the Common European Framework. The association of language schools, English UK, even took the government to the high court as the new regulations mean that students will not be able to come to the UK to study on lower level English courses, costing millions in lost business. The United Kingdom Border Agency has now announced which tests are going to be used to control immigration.

But there is a new and interesting turn in events. Even some of the language testing agencies whose tests have been accepted are now questioning the use of the CEFR for linking tests, as well as questioning the methodologies used by the competition to link their tests to the CEFR, as evidenced in this newspaper article from the Guardian today. The CEFR has been a political tool from the moment of inception, but its use has thus far favoured the large European testing agencies. However, it now looks as if claims can be made and challenged at will, and the small players are also learning how to play the game as well. They can establish a 'link' to the CEFR and then go on to say that scores on their tests are equivalent to scores on the established tests. The internet is starting to become littered with 'equivalence tables'. Even the UK commercial sector that runs its own tests for university entry is making equivalence claims, like this table from INTO, a private building and finance operation to which some UK universities outsource foundation and language courses. (If you don't know about this development, read more here.)

The CEFR game is played with mirrors. Everyone knows what answer they have to get when they link to the CEFR. It's the answer that's necessary to remain competitive and have a test recognized for its intended market. Validation theory has been thoroughly undermined in the process of course, and very few people talk about test purpose any more. The CEFR has encouraged validity chaos. But the real question now is just how long the CEFR will remain in favour with the main players, now that it has become a point of marketing weakness rather than strengh. I suspect that over the next few years we will increasingly see the larger testing agencies question the usefulness of the CEFR. Watching how the shift occurs will be quite entertaining for people like me who have been sceptical from the beginning.

Other Updates for August 2010

1 Aug 2010 07:26:19 GMT

The YouTube page is now linked into the site. This will automatically update on Sunday each week, after the software that runs behind the page searches for new content. Obviously, I don't have the time to do this manually! The page is then created on-the-fly. If there is new video content the webslice will let you know. If you don't use a browser that recognises webslices, you'll have to come back and manually check the page.

I've recently noticed that an increasing number of visitors are coming to the website from Facebook. I've therefore put links to social networking sites on the home page, and links to Facebook on some other pages that appear to be shared quite regularly. I will expand this in the future.

You may have noticed the appearance of the scrolling Most Visited Language Testing Websites This Week on the Home Page. I've been experimenting with this over the last month. This will now be updated every Sunday at midnight (Chicago time - which is where the server for this website is physically situated). The websites featured in this scrolling box are those that appear to have been most visited during the previous week according to data from the search engines Bing and Yahoo.

There are a number of other updates throughout the site, but I'll leave you to discover those for yourselves. And if you are about to go on vacation, I hope you have a very restful August!

New Feature for August 2010

1 Aug 2010 07:22:58 GMT

Open Book - Open Web Tests

Although tests in which we have access to books (and now other resources) have been talked about for well over a hundred years, there have been very strong arguments against their widespread use. But last year Denmark introduced them for national language tests, and new arguments in favour are beginning to emerge. We take a look at some of the issues and questions behind this alternative approach to testing.

Update on YouTube Page

31 Jul 2010 10:42:02 GMT

When I launched the YouTube page a week ago, I hadn't realised that some of the providers on YouTube were rather fickle. Videos have appeared and disappeared faster than rabbits in Tommy Cooper's fez. With a little bit of tinkering to the programs, plus an email to the web folk at Teachers TV, I’ve got the page to settle down. While I was doing this I also came across a pretty good set of podcasts from Cambridge Assessment, so I’ve added those as well. Technically it’s no longer just a YouTube page, but basically what it does is to import third-party content so you don’t have to look for it all over the web on your own. It takes far too much time. Anyway, I hope you’re enjoying the content that my search engines have found so far. More updates tomorrow on the 1^st August.

New Publication on Case Studies

30 Jul 2010 13:54:47 GMT

From time to time I think there's a new research methodology text that might be particularly useful for researchers in the field of languge testing, and this is one of them. We increasingly conduct case study research when addressing consequential issues, particularly in the ever expanding area of washback studies. So this new book by Peter Swanborn may be useful for our field. Entitled Case Study Research: What, Why and How? (ISBN 10:1849206120) from SAGE, the product details are as follows:

How should case studies be selected? Is case study methodology fundamentally different to that of other methods? What, in fact, is a case?

Case Study Research: What, Why and How? is an authoritative and nuanced exploration of the many faces of case-based research methods. As well as the what, how and why, the author also examines the when and which - always with an eye on practical applications to the design, collection, analysis and presentation of the research.

Case study methodology can prove a confusing and fragmented topic. In bringing diverse notions of case study research together in one volume and sensitising the reader to the many varying definitions and perceptions of 'case study', this book equips researchers at all levels with the knowledge to make an informed choice of research strategy.

This book was published in the United Kingdom today, and can be pre-ordered on Amazon in the United States. It can also be ordered through the Language Testing Bookstore, under the dictionaries and research methods tab.

Language Testing on YouTube

24 Jul 2010 17:38:32 GMT

YouTube has a lot of language testing related videos, but their quality is wildly variable. I have spent some time tracking down a list of providers who are putting up material that is not too overtly promotional, has interesting and relevant content, and which I have enjoyed watching. I have been particularly impressed with the amount of material on formative classroom assessment that is directly relevant to the needs of teachers. Just yesterday, for example, a new video was posted by Teachers TV on formative assessment by Dylan Wiliam, one of the leaders in the Assessment for Learning field. And if you haven't yet seen the feature on this topic, just click on the link.

The next step was to develop a new web page that would automatically recognize new video content on YouTube and import it directly into my web page. Combine that with a webslice, and this means that whenever a new video on language testing appears on YouTube, we will be informed from the toolbar of our web browser (only IE8, I'm afraid - Firefox users used to have 'webchunks' but it has been discontinued).

I haven't linked this new YouTube page into the website yet (all the links and an update on the homepage will come in a week's time, on 1st August). However, after piloting it for a couple of weeks I'm very happy with how this page is functionining. So I thought that I'd give subscribers to this bulletin an opportunity to try it out now. If you'd like to see what's available and subscribe to the weblice, just click on this link: Testing and Assessment on YouTube. The page updates itself on the fly twice weekly.

There will be more updates on 1st August, including a new feature. In the meantime, I hope you enjoy watching the YouTube content. Of course, don't forget the video page either.

Coming Soon - Language Testing on YouTube

16 Jul 2010 16:57:03 GMT

There hasn't been very much going on in the language testing world this month (conference in Melbourne aside - although I couldn't get there!), and I suspect that it is going to be fairly quiet over the rest of the summer. However, in the few moments I've had between projects, writing, and teaching, I've been working on a method to capture language testing videos from YouTube, incorporate them into a single web page, and add an alert when new testing and assessment videos are uploaded. This has proved fairly complicated, mainly because search feeds are not recognized as feeds, but as web pages. This means that it has to be done by provider, which is less efficient. However, I will have this up and running by the beginning of August so that you can keep up with language testing on YouTube, most of which is put there by test providers, and offers a different angle on testing and assessment issues.

Updates for July 2010

1 Jul 2010 11:27:53 GMT

This month the home page of the Language Testing Resources website is getting it's long awaited upgrade. For those of you who were regular viewers of the news, this is no longer embedded within an i-frame. This was a rather primitive solution. You will now find the news firmly on on the home page, complete with a subscription service so that you won't miss the news again. You will also see that the free subscription services have now been expanded to include news, articles, jobs in testing and assessment, and podcasts. As long as you're using a browser that can recognize webslices you can take out a subscription, and you'll get an alert whenever there is new content on that page. Go to the home page for more details. If you have any views on the new look homepage, please send them via the 'contact' page.

Language Testing Bytes Issue 2 & Video

1 Jul 2010 11:22:59 GMT

Language Testing 27(3), 2010, is a special issue guest edited by Dr Xiaoming Xi on the automated scoring of writing and speaking tests. In this podcast she talks about why the automated scoring of speaking and writing tests is such a hot topic, and explains the possibilities, limitations and current research issues in the field. Dr Xi has also made a video version of the interview, which is available on the video page.

Another general assessment publication

22 Jun 2010 08:08:40 GMT

Popham, W. J. (2010). Everything School Leaders Need to Know About Assessment. Corwin Press. ISBN: 141297979X . Very much a text on general educational assessment, but unusually written for the particular audience of school leaders. This could be the one to give your head teacher for Christmas this year, just as you put in a budget for test development resources for the future! Here's the product blurb: "With plain language and practical examples, Popham covers key understandings for school leaders, including validity, formative assessment, interpreting test results, instructional sensitivity, and more."

Available under the general tab in the bookstore.

New Book on Diagnostic Measurement

22 Jun 2010 08:08:33 GMT

Rupp, A. A., Templin, J. and Henson, R. A. Diagnostic Measurement (2010). Methodology in the Social Sciences. Guilford Press. ISBN: 1606235273.

There isn't a great deal on diagnostic assessment and testing, as Charles Alderson pointed out in his book Diagnosing Foreign Language Proficiency in 2005. This new book does not focus on language, but measurement in general. For anyone who has struggled with some of the more recent literature in this area (see for example, Eunice Jang's paper "Cognitive diagnostic assessment of L2 reading ability: Validity arguments for Fusion Model application to LanguEdge assessment" in Language Testing 26(1), 31 - 74) this might be a welcome explanatory text. Anyway, here is one review which is promising on the accessibility front.

"The readability of the book is excellent. It offers a good, basic-level exposition of the underpinnings of cognitive diagnostic assessment. It is particularly important for a book like this to be accessible to a broader audience beyond experts in cognitive diagnosis. The book covers cognitive foundations, makes connections with the most salient aspects of assessment validity, and includes detailed derivations and discussion of core cognitive diagnostic models. The examples are especially effective and clear, as is the tabulation of model parameters in the discussion of six core models and their relationship to the log-linear modeling framework. I would recommend this text for use in graduate seminars and expect to quote from it and cite it in professional presentations and papers. There is no other book available with comparable breadth." Lou DiBello, University of Illinois-Chicago.

And here is the product description:

This book provides a comprehensive introduction to the theory and practice of diagnostic classification models (DCMs), which are useful for statistically driven diagnostic decision making. DCMs can be employed in a wide range of disciplines, including educational assessment and clinical psychology. For the first time in a single volume, the authors present the key conceptual underpinnings and methodological foundations for applying these models in practice. Specifically, they discuss a unified approach to DCMs, the mathematical structure of DCMs and their relationship to other latent variable models, and the implementation and estimation of DCMs using Mplus. The book's highly accessible language, real-world applications, numerous examples, and clearly annotated equations will encourage professionals and students to explore the utility and statistical properties of DCMs in their own projects. The companion website (projects.coe.uga.edu/dcm) features data sets, Mplus syntax code, and output.

This book is available from the Languagetesting.info bookstore, under the tab diagnostic testing.

Gaokao 2010 Questions Revealed

9 Jun 2010 07:43:30 GMT

The annual Gaokao is over, and Danwei has just published a selection of essay questions from around China. Click on the link above to see the selection. The first example is about mice and fish, that was based on a cartoon. A translation of the cartoon is available on Shanghaiist, with an explanation of what the test takers were supposed to infer. Or at least what the media think the test takers were supposed to infer! This is as much a rite of passage as it is a test. There is a great image from the Wall Street Journal's photo journal of stories in the news, of students being checked with electronic scanners for 'cheating devices'. I have now added the English editions of the Chinese news media to the language testing news search engine, so stories like this will be picked up and displayed on the home page in the future. For more information on the Gaokao see the Wikipedia entry, and this recent blog from the New Yorker.

TOEFL® COE 2011 RESEARCH PROGRAM

4 Jun 2010 15:22:50 GMT

Just two weeks to go if you wish to submit a bid for the TOEFL Research Program for 2011. Follow the link for details and submission guidelines.

New Book on Think Aloud Protocols

3 Jun 2010 15:22:59 GMT

Language Testing researchers are increasingly using think-aloud or stimulated recall protocol analysis in their research, so this book by Melissa Bowles entitled The Think-aloud Controversy in Second Language Research is a welcome addition to the literature in this area. According to the blub, the book "...aims to answer key questions about the validity and uses of think-alouds, verbal reports completed by research participants while they perform a task."

Also available under the 'research methods' tab in the Bookstore.

First Issue of Language Testing Bytes

1 Jun 2010 08:32:35 GMT

The first issue of Language Testing Bytes is now available. This is the official podcast of the journal Language Testing, and is available from the SAGE website ltj.sagepub.com, from this website, and as part of the SAGE Podcast on iTunes. Issue 1 is related to the articles on fairness in Langugage Testing (27)2. In this podcast I interview with Mike Kane on his approach to validation, and how he sees the notion of test fairness in relation to validity.

Language Testing in the Military

1 Jun 2010 08:21:37 GMT

The scenarios have been designed to illustrate the kinds of settings in which language testing is important, and in a seminar setting could be used to explore the concept of test purpose and effect. In a new scenario for June, we focus on the use of language testing and learning in the military, where it is frequently important for personnel to be able to communicate effectively in one or more second languages. I have put the following quotation from Kaulfers (1944: 137) on the web page, as it remains one of my favourites in this context:

The nature of the individual test items should be such as to provide specific, recognisable evidence of the examinee's readiness to perform in a life-situation, where lack of ability to understand and speak extemporaneously might be a serious handicap to safety and comfort, or to the effective execution of military duty.

Many innovations in language testing have been linked to military needs, and this scenario shows how these needs are still very much present today.

Language Testing Bytes

22 May 2010 11:13:31 GMT

SAGE Publishers will be launching a new podcast in June 2010 to accompany the journal Language Testing. This journal is currently edited by Cathie Elder (University of Melbourne) and myself, Glenn Fulcher (University of Leicester). Language Testing Bytes will be published three or four times per year, in which we discuss topics related to a particular issue of the journal. This may be an interview with a contributor to the journal, or another expert in the field. You will be able to download the podcast from this website, from the journal website, or subscribe to the podcast through iTunes. The first issue of Language Testing Bytes will be available from languagetesting.info in June, featuring an interview with Professor Mike Kane on the topic of validation.

Just out: Testing the Untestable in Language Education

16 May 2010 09:45:48 GMT

Edited by Amos Paran and Lies Sercu, from Multilingual Matters. ISBN-10: 1847692656.

Review

The title of this book is well chosen. Despite the apparent oxymoron, this collection of papers succeeds in addressing important issues of educational policy and theory with the precision born of empirical work combined with discussion of principles. This book will open new options for testers, for teachers and for those who make policy decisions. --Michael Byram, School of Education, University of Durham, UK

Product Description

The testing and assessment of language competence continues to be a much debated issue in foreign language teaching and research. This book is the first one to address the testing of four important dimensions of foreign language education which have been left largely unconsidered: learner autonomy, intercultural competence, literature and literary competence, and the integration of content and language learning. Each area is considered through a theoretical framework, followed by two empirical studies, raising questions of importance to all language teachers: how can one test literary competence; can intercultural competence be measured; what about the integrated assessment of content-and-language in CLIL and teaching; and, is progress in autonomous learning skill gaugeable. The book constitutes essential reading for anyone interested in the testing and assessment of seemingly largely untestable aspects of foreign language competence.

Available through the Language Testing Bookstore on the 4th page under the General Category

IELTS Research Grants 2010–2011

15 May 2010 09:31:40 GMT

IELTS, the International English Language Testing System, is now accepting applications for Research Grants.

Individuals and educational institutions with relevant experience are invited to apply for funding to conduct applied research related to English language proficiency testing, specifically IELTS. U.S. educators and researchers are encouraged to submit relevant proposals.

Funding up to $22,000 (15,000 GBP) per research project will be awarded to a select number of proposals this year.

Identified areas for research are:

• test development and validation

• investigations into the use of IELTS by institutions

• case studies on the impact of IELTS on candidates and users.

Teaching, Learning, Assessing in Second Language Contexts

13 May 2010 11:12:48 GMT

Dates: 2nd-3rd December 2010

Venue: Zanzibar Beach Resort Hotel, Zanzibar, Tanzania

This Symposium aims to share research findings and discuss current issues
and policies related to teaching, learning and assessing through the
medium of a second or foreign language across the globe.

We invite submissions of individual papers and posters on any topic of
relevance to the Symposium theme.

See details at: http://www.bris.ac.uk/spine/symposium/symposiumthemes/

DEADLINE FOR PROPOSALS (papers & posters): July 30th 2010

Special Issue of ELTED - Call for Papers

6 May 2010 13:52:51 GMT

English Language Teacher Education and Development (ELTED) is planning a special winter issue (2010) on testing and assessment. The guest editor, Gerard Sharpling, is particularly interested in receiving manuscripts from research students who would like to see some of their work in print.

Areas of interest include:

- an outline/appraisal of courses involving training in language testing;

- personal accounts of how language testers working within testing organisations have developed/enhanced their skills/knowledge;

- reports/evaluations of how training procedures for rater/assessor training (standardisation, inter-rate/intra-rater reliability, etc) have been implemented;

- approaches to quantative/statistical procedure training, in particular, on teacher education courses;

- development of classroom-based assessment skills (especially formative assessment);

- developing humanistic/learner-centred approaches to language testing/assessment;

- the training of raters in the marking/grading of writing and speaking;

- evaluation of the usefulness of teacher development in terms of language testing and assessment.

- training in the use of technology within the language testing sphere.

See the journal's website for further information and how to submit: http://www.elted.net/

Alternatively, you can contact Gerard Sharpling directly at: Gerard.Sharpling@warwick.ac.uk.

New Publication on testing speaking in China

4 May 2010 09:53:35 GMT

Testing Spoken English Achievement for Non-English Majors in China, by Li Huadong. 180 pages. ISBN-10 1425151418.

Product Description:

This book tries to solve a practical problem, that is, the general lack of the spoken English achievement test (SEAT) in the college English teaching for non-English majors in mainland China. To achieve this purpose, it first reviews theories and techniques relevant to speaking and language testing, then conducts empirical research into four SEAT formats, and finally puts forward a presentation and discussion (P&D) SEAT format. It can make the "impractical" SEAT practical, thus promoting testing, and in turn, teaching and learning of spoken English for non-English majors in Chinese universities in order to meet the social and learners' individual needs for the English speaking skill.

Video Breakthrough for Firefox Users

3 May 2010 09:53:18 GMT

Up until now it has not been possible for Firefox users to select a video to play on the video page. This is because of problems with the flash plugin that has to be downloaded into broswers like Firefox and Safari. I have struggled to solve this problem, as some 23% of the website users are Firefox folk. I have now managed to produce blank thumbnails in the flash menu system so that a video can be selected by number. This is a breakthrough, as previously all Firefox users could do was hit the play button and see the videos in sequence. Not much fun! Note, however, that users of all browsers apart from Internet Explorer still cannot see the thumbnail images or other graphic posters on videos. This isn't my fault! It's the plugins that they use, rather than having an active-X flash player built in, like Internet Explorer 8. If you wish to see more about technical issues around browsers and what you can/can't see or do on this site because of the browser you use, please visit the technical page.

Language Testing Scenarios

1 May 2010 11:02:46 GMT

The new language testing scenarios are being added to the What is Language Testing? page, starting this month. Definitions and key texts are useful to understand a topic. Perhaps even more powerful are scenarios where language testing is an important ingredient in decision making. I am planning six scenarios, each of which provide information on the language testing issues at stake, and presents questions for discussion and exploration. These have been designed to be used in a seminar setting and may be particularly useful to explore the various purposes for which we test. The first two scenarios are launched today, on language testing in the aviation industry, and language testing for call centres. Other scenarios will be added in coming months, on language testing in the military, for medical and legal purposes, and finally for education.

Feedback to help me improve these is most welcome, so if you do use them and can think of content that I've missed, innovative ways of using the material, or improve focus questions, I'd like to hear from you. You can get in touch by clicking on the 'contact' tab at the top of the page.

Lots of New Articles

1 May 2010 10:33:12 GMT

Over the last month I've been catching up with lots of the recent literature on validation, and in the process I inevitably went back to look at what had been written in the past. I discovered, to my amazement, that many classic papers have now been scanned and are available online. So I've added links to work by authors like Embretson and Loevinger, among others. But I'll let you search for the other new links at your leisure. Happy reading!

New Video: Carol Chapelle on Technology in in Language Testing

1 May 2010 10:16:22 GMT

The Language Testing Resources website is 15 years old this month, and so there's a lot of new material to push it foward into what I hope will be the next 15! Firstly, we have a new video by Professor Carol Chapelle on the use of technology in language testing. Her many books, articles and research papers in the field deserve careful study, and the new video provides an excellent introduction to the topic. If you haven't read Assessment Through Computer Technology, it comes highly recommended. This new video is ideal for use in a language testing course to generate discussion and further study. I would, of course, like to thank Carol for sparing time to share her expertise with us.

Almost 15 Years Old

27 Apr 2010 17:17:28 GMT

The Language Testing Resources Website was first launched in May 1995. There are going to be a range of updates to the site this weekend to celebrate its birthday.

News Quiz Item added to the Fun Page

27 Apr 2010 17:12:53 GMT

On 23rd April 2010 testing managed to find its way onto the News Quiz, a popular Radio 4 satirical news show broadcast on a Friday evening. It dealt with the issue of teachers boycotting tests as a means of opposing government policy on accountability. It could have been more searing, but is quite funny nevertheless.

Published: Language Assessment: Principles and Classroom Practices 2nd Edition

22 Apr 2010 18:07:11 GMT

From the Back Cover

Language Assessment provides teachers with a clear presentation of the essentials for assessing second language learning fairly and effectively. This updated second edition includes new research and information on standardized tests, a new chapter on form-focused assessment, and a concise glossary of terms.

With a focus on the most common pedagogical challenge – classroom-based assessment – Language Assessment delivers useful tools for evaluating and designing practical, effective assessment techniques.

Features:

Updated references and new information reflecting recent advances and current challenges in the field.
Classification of assessment techniques ranging from controlled to open-ended item types on a specified continuum of micro- and macroskills of language.
Systematic treatment of assessment for all four language skills (reading, writing, speaking, and listening) as well as grammar and vocabulary.
Discussion of large-scale standardized tests, the ethics of testing, letter grading, and overall evaluation of student performance.
Comprehensive presentation of alternatives in assessment, such as portfolios, journals, conferences, observations, interviews, and self- or peer-assessment.

New Features

Chapter objectives that preview content and goals
A new chapter on form-focused assessment
An appendix of commercially available tests
A concise glossary of assessment terms

This text can be ordered through the bookstore, under the general category.

LTRC 2010 and 2011

17 Apr 2010 19:53:10 GMT

LTRC 2010 in Cambridge was a fantastic event, and all participants are very grateful to Cambridge ESOL for hosting the conference, and particularly Lynda Taylor and her team for the wonderful organization. Participants are now trying to make their way home through the clouds of volcanic ash! At least we all head home refreshed. And no sooner is this year's event concluded, than it is time to look to 2011. The organizers, the University of Michigan, already have a web page up and running, which you can take a look at by following this link. It is far too early for a call for papers, but we will alert you when it happens.

Cambridge Info and Weather for LTRC

12 Apr 2010 12:06:49 GMT

I've added further information on Cambridge and the surrounding area to the Home Page, along with a regularly updated 3-day weather forecast.

Published: Language Assessment in Practice

12 Apr 2010 12:05:02 GMT

The second edition of Bachman and Palmer's Language Assessment in Practice is now available. Copies can be purchased through the bookstore. Product description: Presents an innovative, unified, and easily applied approach to designing and developing language assessments. Language Assessment in Practice is a follow-up to the bestselling Language Testing in Practice. It allows readers to become competent in the design, development, and use of language assessments. The authors discuss concepts and procedures clearly, illustrated with examples.

Published: From Migrant to Citizen: Testing Language, Testing Culture

3 Apr 2010 14:43:02 GMT

Product Description

In this examination of the Australian debate on citizenship tests in its historical and international context, linguists, lawyers and historians, political theorists and philosophers draw out themes of identity and cultural belonging underlying the political rhetoric of testing new citizens knowledge of the language and culture of a nation.

Divided into three parts From Migrant to Citizen

- explores the historical background which Australian citizenship testing shares with other nations of the British colonies.

- provides cross-national perspectives on citizenship by examining the proliferation of new tests for citizenship in Canada, the United States, the United Kingdom, the Netherlands, Germany and in Australia.

- discusses philosophical implications and popular attitudes towards the new testing regimes by discussing debates about identity, values and nation and the implications for Australia and for the wider international community.

This title is now available from Amazon.co.uk, but can be pre-ordered in the US on Amazon.com, or through the languagetesting.info bookstore in the category of special purposes and EAP testing.

Published: Practical Language Testing

2 Apr 2010 14:42:41 GMT

Product Description

Practical Language Testing equips you with the skills, knowledge and principles necessary to understand and construct language tests.

This intensely practical book gives guidelines on the design of assessments within the classroom, and provides the necessary tools to analyse and improve assessments, as well as deal with alignment to externally imposed standards. Testing is situated both within the classroom and within the larger social context, and readers are provided the knowledge necessary to make realistic and fair decisions about the use and implementation of tests. The book explains the normative role of large scale testing and provides alternatives that the reader can adapt to their own context. This fulfils the dual purpose of providing the reader with the knowledge they need to prepare learners for tests, and the practical skills for using assessment for learning.

Practical Language Testing is the ideal introduction for students of applied linguistics, TESOL and modern foreign language teaching as well as practicing teachers required to design or implement language testing programmes.

Conference Travel Update

1 Apr 2010 17:04:39 GMT

The threatened UK rail strike that may have caused problems for LTRC participants travelling to attend pre-conference workshops has just been called off, and all trains will now be running as normal. Great news for everyone. However, there is a message in this episode for language testers and researchers.

Rail Network went to the High Court in London to take out an injunction against the Rail Union to stop them going on strike as planned. The grounds for application were that there were significant errors in the data collection that "invalidate" the ballot, and so the court was asked to force the Union to run a new ballot of members before they could take strike action. The problems with the data and data collection are (and I quote):

11 signal boxes have been balloted by the RMT that do not exist - most have been closed for yearsFor example, RMT balloted members at Chalford signal box, near Gloucester, closed in 1965/6. It also balloted members at East Usk signal box, Newport, that burnt down in April 2009
67 locations have been balloted where the numbers of RMT members exceeds the total number of employees that work there. For example, we have three employees at South Tottenham, the RMT asked 11 members to vote. We have 24 employees at Crewe and the RMT invited 33 members to vote
26 workplaces (such as Rugby signal box) amounting to almost 100 employees, have been completely missed thereby giving RMT members at these locations no opportunity to vote
12 locations were balloted where there are no operations staff at all and thus were ineligible to vote

What a great reminder to plan data collection in advance, and to handle and report it accurately! And here's a useful reference to keep to hand: Davidson, F. (1996) Principles of Statistical Data Handling. London: Sage.

I hope you have a very enjoyable and trouble free journey to Cambridge.

LTRC Final Programme Available

1 Apr 2010 16:04:27 GMT

The LTRC programme is now available on the conference website for download.

Online First Added

1 Apr 2010 16:02:41 GMT

You can now check on papers that are due to appear in future editions of Language Testing using the online first page. This has also been added to the Article Alert Service, so that subscribers will be notified when new papers are uploaded to this, as well as other journals.

Updated Articles

1 Apr 2010 09:14:35 GMT

I have significantly added to the articles links this month, having discovered pdfs of may key assessment texts on the internet. Browse the articles to see what is available, but I will draw your attention to the addition of two important papers from the educational assessment literature:

Embretson, S. (1983). Construct Validity: Construct Representation Versus Nomothetic Span. Psychological Bulletin, 93, 1, 179 - 197.

Loevinger, J. (1957). Objective tests as instruments of psychological theory. Psychological Reports 3, 635 - 694. Southern Universities Press, Monograph Supplement 9.

Happy reading!

New Feature: Assessment for Learning

1 Apr 2010 08:57:01 GMT

Classroom assessment is said to be formative rather than summative. That is, it takes place during the process of learning, rather than at the end of a period of learning. And its purpose is to inform and improve learning, rather than simply to assess whether the learners have mastered the learning objectives. The role of formative assessment is therefore to generate information to help improve learning and teaching. This feature offers a brief introduction to assessment for learning in general, and relates the principles of the assessment for learning movement to language testing and assessment. Links to key resources are provided.

Re-assessing assessment in EAP: Call for Papers

1 Apr 2010 08:56:54 GMT

BALEAP Professional Issues Meeting

RE-ASSESSING ASSESSMENT IN EAP : CHALLENGES AND OPPORTUNITIES
Centre for English Language Education,
Jubilee Campus, The University of Nottingham
Saturday 13 November 2010

Proposal deadline : 21 June 2010

New Edition: Educational Assessment of Students

28 Mar 2010 12:16:02 GMT

Originally scheduled for publication in February, the 6th edition of Educational Assessment of Students by Brookhart and Nitko is now in the bookshops. Product Description: A highly-respected book in its field, The Educational Assessment of Students 6/e is the most thorough discussion of traditional and alternative assessments of any text–explaining, giving practical real-world examples, discussing pros and cons, and showing how to construct virtually all of the choices teachers can make in classroom assessment. Theories and research findings abound; the author examines why, when, and how teachers should use assessment in the classroom. Topics include: the bases for assessment in the classroom, crafting and using classroom assessments, and interpreting and using standardized tests. With its extensive and valuable appendix set, this book is a must-have desk reference for teachers and others involved in the field of education. I have now added this to the Bookstore in the category of classroom assessment.

Cambridge Info for LTRC Participants

27 Mar 2010 17:31:11 GMT

I've just added a new section to the Homepage which provides links to information about Cambridge and gives a constantly updated 3-day weather forecast for the area. This will stay on the website through the conference, and will be removed on 18th April.

Travel Update for Easter Period Conferences

26 Mar 2010 13:39:05 GMT

National Express, the UK's main inter-city coach service, has announced that it's putting on additional services during and around the rail strike. If you are worried about getting from your airport to a conference venue just after Easter, you can consider coach as an alternative to rail. Information and booking for National Express can be done online here.

UK Conference Travel News for April

26 Mar 2010 09:25:00 GMT

If you are travelling to IATEFL in Harrogate, or planning to arrive at LTRC early, and are travelling within the UK by train, note that the rail union has called a national strike for 6th - 9th April. This may put trains out of position for a good few days after the final date of the planned strike. Hopefully, the dispute will be solved in time. If not, you can got to the National Rail Enquiries website to find out what disruptions you might face. In the meantime, it may be useful to follow the transport news on the BBC website in case you need to make alternative arrangements.

JLTA Conference 2010: Call for Papers

21 Mar 2010 16:10:26 GMT

The Japan Language Testing Association (JLTA) invites presentation proposals for its annual
conference.

Date: September 11, 2010 (Saturday) 9:00 to 17:00
Place: Toyohashi University of Technology
Theme: Diagnostic testing in language teaching

Attendance Fee: Student & JLTA Member: \1,000
　　　　　　　　Non-JLTA-member: \3,000

The deadline for submission of presentation proposals: July 3, 2010.
Conference homepage: https://e-learning.ac/jlta.ac/

Scalar Conference: Call for Papers

21 Mar 2010 15:45:52 GMT

The 13th Annual Conference of the Southern California Association for Language Assessment Research (SCALAR) will be held on Saturday, 1 May 2010 at the University of California, Los Angeles. The theme of this year’s conference is, “Formative Assessment”.

New Video Clip (Number 5)

18 Mar 2010 06:57:34 GMT

I was recently reminded of the excellent video clip from Yes, Prime Minister on how to create a survey with leading questions. As interviews and questionnaires are frequently used in language testing to collect data, and I have been teaching this topic recently, I thought I would add it to the humour page. I hope you enjoy this as much as I and my students do!

Articles Update

13 Mar 2010 11:15:02 GMT

I will be adding quite a number of new articles to the website at the beginning of April. However, for I thought I would send this out in advance to feed subscribers. I have just discovered that AILA Review 7 from 1990, entitled Standardization in Language Testing, is now available online in pdf in its entirety. The contents are listed below. You can download the volume by clicking on this link. This, and more new content, will be posted on the articles page on 1st April.

Guest-editor's Preface John H. A. L. DE JONG	3-5
Language Testing in Research and Education: The Need for Standards Peter J. M. GROOT	6-23
The Cambridge-TOEFL Comparability Study : An example of the Cross-National Comparison of Language Tests Fred DAVIDSON & Lyle BACHMAN	24-45
The Australian Second Language Proficiency Ratings (ASLPR) David E. INGRAM	46-61
Cross-National Standards: A Dutch-Swedish Collaborative Effort in National Standardized Testing John H.A.L. DE JONG & Mats OSCARSON	62-78
The Hebrew Speaking Test: An Example of International Cooperation in Test Development and Validation Elana SHOHAMY & Charles W. STANSFIELD	79-90
EUROCERT: An International Standard for Certification of Language Proficiency Alex OLDE KALTER & Paul VOSSEN	91-105
Response to Alex Olde Kalter and Paul Vossen John READ	106-107

MwALT 2010

9 Mar 2010 10:45:29 GMT

Performing assessment through Performance assessment

The 12th Annual Conference will be held
Saturday, September 25th, 2010 at
Wright State University
Dayton, Ohio

Plenary Speaker: April Ginther

Call for Papers

LTRC Early Bird Registration Ending

2 Mar 2010 07:49:23 GMT

If you are planning to attend the Language Testing Research Colloquium in Cambridge this April, it is best to register this week. Early bird registration ends on Sunday 8th March, after which the price goes up. Abstracts of the talks by Michael Kane (Messick lecture) and Elana Shohamy (lifetime achievement award lecture) are both available on the LTRC website.

Competition Winner

1 Mar 2010 03:58:54 GMT

During the winter months you may recollect that we ran a competition to define 'language testing'. This has now been judged by myself and Professor Alan Davies of the University of Edinburgh, and the winner is Priscilla Allen of the University of Washington. Many congratulations to her on winning a competition with lots of very good entries indeed. And thank you to all those who put so much effort into their definitions - especially the entrants from Turkey and Cyprus! We enjoyed reading your work, and wish we could have given a prize to everyone. I have now created a new web page that gives a number of brief definitions of 'language testing', and provides downloads of two of the founding documents of the field.

New Article Alert Service

1 Mar 2010 04:02:55 GMT

Using the new web slice technology that comes with IE8, this new service will alert users to the publication of new articles on language testing in a range of journals from the testing, educational, and applied linguistics fields. The journals included in this service are only those that put an electronic date stamp on individual articles. Some journals do not do this, and so their articles would 'sit' at the top of the page without moving until the journal removed that content. You can see that this happens on the aggregation page. You have to scroll down some way before you reach an article with a publication date. The Article Alert service will therefore tell you when new publications become available, but you should still check the aggregation page and the journals page periodically, so you don't miss anything.

Conference: Assessment for Learners

18 Feb 2010 10:41:55 GMT

FIFTH BIENNIAL NORTHUMBRIA/EARLI SIG ASSESSMENT CONFERENCE

Assessment for Learners

1-3 September 2010
Slaley Hall Hotel, Northumberland, UK

AILA 2011

17 Feb 2010 08:55:34 GMT

AILA: 16th World Congress of Applied Linguistics
Harmony in diversity: language, culture, society
Beijing Foreign Studies University, Beijing
23-28 August 2011

Download the Call for Papers

Published: Classroom Assessment (6th edition)

14 Feb 2010 10:34:36 GMT

Product description: Written with energy and wit, this reader-friendly text discusses practical ways in which teachers can use well-written tests to improve their effectiveness in the classroom. Exploring a wide range of classroom assessments as well as how these assessments can benefit student learning, the thoroughly revised sixth edition of Classroom Assessment retains its humorous tone and unique practice exercises while adding new research, new material, and new features. This edition covers both contemporary classroom assessment issues, including No Child Left Behind, instructor accountability, and formative assessments, while also covering traditional assessment topics such as assessment validity and assessment reliability. Three new chapters include fresh and expanded material on formative assessments, the evaluation of instruction, and properly grading assessments. In addition, a new section at the end of the book provides students with critical thinking exercises that involve real-world assessment issues and that cover central topics from each chapter. Another new feature teaches readers how to best communicate with parents about assessments and assessment results.

Available through the Bookstore in the Classroom Assessment Category.

Published: Developing Report Cards

14 Feb 2010 10:08:07 GMT

Now available ahead of the publication date. Review: Guskey and Bailey clearly articulate the need for reform on one of today’s most pressing issues in education: grading and reporting. Throughout the book, they offer realistic solutions to improve how educators communicate a student’s academic progress to all stakeholders. Their work provides the practitioner with the research, step-by-step guidelines, and reporting templates so a faculty is ready to begin the dialogue to develop a standards-based report card. The research has helped us answer the two main questions: 'What goes into a grade?' and 'How do we report it out?' This work, without a doubt, is a model for schools that want to improve their system of grading and reporting.

Available through the Bookstore in the Classroom Assessment category.

Reliability Paper Published

5 Feb 2010 11:23:53 GMT

Although it has a 2009 publication date, Parallel Universes and Parallel Measures has only just been made available. The publication, commissioned by OFQUAL and written by authors from the UK's National Foundation for Educational Research, explains and describes the different forms of reliability and the commonly used reliability indices.

This and other new freely available publications are listed on the articles web page.

ILTA Funding for Workshops and Meetings

4 Feb 2010 16:06:45 GMT

Call for Proposals 2010

ILTA aims to promote high standards of professionalism in language
testing and assessment practice. One way of carrying out this mission is
to encourage workshops and meetings where language testing experts
educate and train teachers, test developers, and others, including
policy makers, principals, and even test takers in key skills and
knowledge in language testing/assessment.

ILTA offers individual grants of up to US$ 3,000 for carrying out an
approved workshop/meeting designed to help diffuse knowledge among a
wider group in different parts of the world. The aim of a
workshop/meeting should be to promote understanding, familiarity, and
knowledge of language testing issues and practice among various groups
of test users. It is expected that the award will be used as 'seed'
money to organise such a workshop/meeting and, ideally, to establish a
permanent local organisation that that will conduct further practical
activities in language testing in the future, especially in places in
need of language testing expertise.

More than one award of US$ 3,000 may be made available in any one year,
subject to sufficient funds being available. The procedures require that
an award must be used by the end of the calendar year following
application (i.e. proposals approved in 2010 must be carried out by the
end of 2011).

CRITERIA FOR SUBMISSION: Those eligible to apply are individual language
testers, teachers and others representing various types of institutions
such as universities, research institutes, schools and testing
organisations. Applicants must be ILTA members or, in the case of joint
applications, at least one applicant must be an ILTA member. (In cases
of extreme financial hardship the membership costs for ILTA may be
included as part of the proposal, but a justification is needed in such
a case.)

APPLICATION PROCESS: Those interested can apply by sending a 3- to
5-page proposal which should include the following:
1. brief background regarding the state of language testing in the
specific context (country, region, school system)
2. rationale for the meeting including specific reasons and needs for
holding the meeting
3. description of target participant groups and their prior
training/knowledge in language teaching and in language
testing/assessment
4. plan for the meeting including specific themes to be covered
5. tentative list of speakers/workshop leaders plus information about
their expertise 
6. background of the organiser(s) in the area of language testing (the
CV of at least one main organiser should be included to support this
point)
7. duration and location of the workshop/meeting
8. intended audience for the workshop/meeting
9. amount of money needed and a specific description of how it will be
used (including budget breakdown)
10. intended results of the workshop/meeting, specifically the short-
and long-range impact the event is expected to have
11. a specific explanation of how the workshop/meeting will contribute
to the mission and to the establishment of a permanent, local
organisation
12. full contact details of the organiser(s).

The application must include a written statement of a commitment to
provide ILTA within 3 months of the workshop/meeting, with an evaluative
report of the event. This will be posted on the ILTA website and may be
distributed to ILTA members.

APPLICATION DEADLINE: Applications should be sent by e-mail to the ILTA
President Carolyn Turner, carolyn.turner@mcgill.ca, by 1 April 2010.

SELECTION PROCESS: Each application for grant funding will be evaluated
by a special committee of ILTA Executive Board members, which is chaired
by the ILTA President and includes one or more of the current ILTA
Members at Large. The committee may also call upon other relevant
experts to input to the selection process. In light of this, the
committee will determine the likely evaluation/wait time needed before a
decision can be made and will notify the applicant(s) accordingly.

CRITERIA FOR SELECTION: Selection will be based upon the fit between the
application and the criteria which are outlined above. Priority will be
given to contexts where language testing is not well-established so that
the workshop/meeting can contribute significantly to increasing
awareness, knowledge and effective practice in language
testing/assessment.

New Edited Volume

3 Feb 2010 13:54:21 GMT

Edited by Lynda Taylor and Cyril Weir, Language Testing Matters: Investigating the Wider Social and Educational Impact of Assessment (Proceedings of the ALTE Cambridge Conference April 2008) is now available in paperback.

Product description: Language Testing Matters explores the social and educational impact of language testing and assessment at regional, national and international level. It brings together a collection of 20 edited papers based on proceedings of the 2008 ALTE Conference in Cambridge. The selected papers focus on three core strands addressed during the conference: new perspectives on testing for specific purposes; insights on testing policy and practice in the context of language teaching and learning in different parts of the world; reflections on the impact of testing among differing stakeholder groups. With its broad coverage of key issues, combining theoretical insights and practical advice, this volume is a valuable reference work for academics, employers and policy-makers in Europe and beyond. It is also a useful resource for postgraduate students of language testing, for practitioners, and anyone else seeking a better understanding of the social and educational impact of language assessment.

This title is available through the languagetesting.info Bookstore.

LTRC 2010 Provisional Programme

3 Feb 2010 13:48:46 GMT

A provisional programme for this year's Language Testing Research Colloquium is now available to download. This is the first time parallel sessions have been timetabled, reflecting the growth in the number of proposals being submitted.

Competition Closed

3 Feb 2010 11:37:23 GMT

The winter language testing competition is now closed, and the entries will go to the two judges. Thank you to everyone who took part, and we hope to have a definition of 'language testing' that we can use on this site. The winning entry will be announced next month, and the winner will get a copy of Practical Language Testing.

Testing and Politics

3 Feb 2010 11:39:24 GMT

There is a new language testing feature available this month on the use of tests to achieve political harmonization. This is a practice as old as organized society. The earliest existing test for this purpose (actually linked to a textbook - Bede's On the Reckoning of Time) dates from 809 CE. The feature has a copy of the test for you to download, a discussion of the context of its use, and some thoughts about the modern use of tests for overtly political purposes. You can also download a pdf of Test use and political philosophy (2009).

One Week Left to Enter the Competition

27 Jan 2010 15:35:23 GMT

If you haven't entered the competition there is just one week left. Visit the competition web page and submit your definition of 'language testing' before noon (GMT) on 3rd February. The winning entry (should there be one!) will be announced in March.

Testing Conference - 29th January

20 Jan 2010 09:03:24 GMT

Just in case anyone is in Paris on Friday next week, there is a one-day language testing conference at the University of Pierre and Marie Curie - Paris 6. Venue: Amphi Jacques Monod, 4 place Jussieu - F-75252, Paris cedex 05. Places must be reserved in advance from the organizers, so contact the University directly. I can't give emails in this feed when there isn't a conference website, but if you wish to see if there are places available and need an email address, please contact me.

Programme:

Glenn Fulcher. The CEFR: Uses and usefulness.

Christian Puren. An example of 'action oriented' certification prior to the CEFRL: The DCL-Diploma of Competence in Language.

Christine Demaison, Faouzia Benderdouche. The CEFRL: A tool to use to build the beginning and ending of a language policy.

Catherine Carol and Mirielle Prodeuau. The case for a dialogue between didactitions and acquisitionists: thoughts from the European project Slate.

Sabine Lopez: Is it possible to relate CEFRL levels to the different stages of acquisition of French as a Foreign Language?

Innovations in Testing (ATP 2010)

20 Jan 2010 08:13:24 GMT

I have just come across an organization of which I was previously unaware, the Association of Test Publishers. I will be adding this to the links page shortly. In the meantime, you may wish to take a look at their annual conference. As we would expect, the focus is rather different from what we are used to at conferences like LTRC. But I thought it would be of enough interest to bring to your attention.

ALTE 4th International Conference

19 Jan 2010 12:17:30 GMT

The ALTE 4th International Conference will take place from 7-9 July 2011 in Kraków, Poland. The theme of the conference will be The Impact of Language Frameworks on Assessment, Learning and Teaching viewed from the perspectives of policies, procedures and challenges. The first Call for Papers will be announced in April 2010.

Some technical Issues

19 Jan 2010 09:13:49 GMT

Recently I've had one or two users of languagetesting.info contact me to say that they are having problems with some pages. The most frequent is an inability to see the video thumbnails on the video page, which means that they can only hit 'play' and watch them from the beginning, rather than select the video they want. I was puzzled about this, until I discovered that they all have one thing in common: they all use Firefox. I don't have the time to check all pages on this web site in multiple browsers, but I have now taken the trouble to go through the site with Firefox. There are multiple problems, which I have now outlined in a new 'technical issues' page. If you are a Firefox user, please check this out. I've also included information on a couple of other topics that I frequently get questions about.

Jobs Web Slice Subscription Available

15 Jan 2010 10:17:07 GMT

Today I've added a web slice subscription to the testing and assessment jobs page. You can place a tab on your browser's tool bar that will automatically alert you when new jobs are posted on this page. This is a new feature that (sadly) only works with Internet Explorer 8 at the present time. My apologies to the 20% of visitors to this site who use Firefox! Click here to learn more about web slices and how to use them.

KELTA Conference 2010

11 Jan 2010 17:06:04 GMT

KELTA 2010 Call for Presentations KELTA (Korea English Language Testing Association) is going to hold its 6th international conference on Saturday, August 28, 2009, at Seoul National University. The details of the conference are as follows: Name: KELTA 2010 International Conference Date: Saturday, August 28, 2009 Venue: Education Information Hall, Seoul National University Theme: Professionalism in the Policy Making of English Language Tests Keynote and Plenary Speakers: TBA Deadline for topic/abstract submission: April 1st , 2010. Contact Persons: Conference Chair: Dr. Kyung-Ae Jin (Korea Institute for Curriculum and Evaluation) at kajin@kice.re.kr Secretary General: Dr. Tae-young Jeong (Korea Military Academy) at tyjeong@kma.ac.kr or jty1234@yahoo.com

Change in Feed Name

10 Jan 2010 16:47:06 GMT

If you have already subscribed to this feed, the name (but not the source file) has been changed today. There are two reasons for this. The existing name is rather long for the toolbar, and a shorter one was preferable. However, two new feed services are planned for the near future, and the names need to show their content clearly. If you currently subscribe to this feed I suggest that you right click and 'delete', then re-subscribe. You can always mark all items as 'read'. I'm sorry for any inconvenience caused. News of the new feeds will be posted here as they become available.

Language Testing at IATEFL 2010

10 Jan 2010 09:37:16 GMT

IATEFL 2010 is in Harrogate, 7th - 11th April. There is a strong testing and assessment strand in the programme, which is now available for download from the IATEFL web site.

Establishing and Maintaining Standards

9 Jan 2010 16:42:02 GMT

Dubai, UAE, February 11 & 12, 2010 Assessment of language competence has always been an important part of the language teaching environments as it can have a tremendous effect on the lives of many language students, immigrants, employees and teachers. This clearly makes the task of setting meaningful and attainable standards one of the biggest challenges in assessment. On the one hand, establishing and maintaining standards in assessment can be extremely beneficial and can enhance the quality in language teaching.

Language Testing Research Colloquium 2010

9 Jan 2010 15:25:31 GMT

Registration for LTRC is now open. Early registration will end on 7 March. The final day for registration is Wednesday 31 March. Click on the link to go to the Registration Page, where you will also find details of conference fees and UK visa information.

Stephen Toulmin dies at 87

7 Jan 2010 15:59:15 GMT

Stephen Toulmin’s model of argument structure, expounded in his seminal work The Uses of Argument (1958/2003), has been widely used in philosophy, rhetoric, and educational measurement (eg., Kane, 2006). The model has also been used in developing the notion of validity as argument in many language testing books and articles, including Bachman (2005), Chapelle et al. (2006), and Fulcher and Davidson (2007).

Language Testing Bookstore Opened

1 Jan 2010 09:40:00 GMT

During 2009 I created the various pages on this web site that provide information on articles published in language testing. The next obvious step was to try to do the same for books. Amazon provided the answer, with the launch of their new aStore software. The new bookstore on this web site is an Amazon affiliate. I select the titles from my current lists of available books to stock the book store, so that users do not have to spend valuable time searching for relevant titles around the internet. Although this is on the languagetesting.info web site, orders are made directly with Amazon; and Amazon is responsible for all deliveries as if the user were shopping on Amazon.com. The main advantage is that this is a "specialist list" that would be difficult to generate even from multiple Amazon searches; and it also makes purchase of these books a little easier online.

Last chance to enter the Winter Competition

1 Jan 2010 09:09:44 GMT

If you haven't already entered the competition, you now have until 9 am on 3rd February to do so. Clilck on the link to take you directly to the competition page. The aim is to write a definition of 'language testing' suitable for a dictionary entry, and for use on this web site, to tell a complete newcomer what "language testing" is. Its length should be similar to that of any dictionary entry: short, pithy, and informative. Like all competitions, there is a prize for the winning entry.

Feature in EDTECH NEWS

26 Dec 2009 11:12:49 GMT

The Language Testing Resources page has been featured in the December issue of EDTECH NEWS, an electronic publication of the Univeridad Chileno - Britanica de Cultura. The number of visitors to this site from Chile have increased dramatically as a result, with over 400 pages being downloaded this month.

Obituary: Caroline Clapham

23 Dec 2009 09:18:00 GMT

Picture from the Video Testing Reading

The language testing community has recently been informed of Caroline's death on 14th December, after a long illness. Caroline was one of the very first people I met when I began to study language testing, and she remained a friend as well as colleague throughout the years. In 1999 she made one of the original videos for this language testing site, which you can still watch on the video page. However, I am now making an mp4 podcast of this video (136 mb) available for download so that we can remember Caroline as she was. This can be played on your computer, or on an iPOD.

To download this video, right click on this link and then select 'save target as'.

Caroline's major work The Development of IELTS: A Study of the Effect of Background on Reading Comprehension is seminal to the field, and is absolutely essential reading for all students of language testing. Fred Davidson wrote the review, which was published in in Language Testing in 1998, and shows how exceptional Caroline's contribution has been. I do not normally make texts like this available on my web site, but in this case will make an exception.

To download the review, right click on this link and then select 'save target as'.

Caroline will be greatly missed.

New Book on Validity

11 Dec 2009 14:44:07 GMT

The Concept of Validity: Revisions, New Directions and Applications.

Edited by R. W. Lissitz, this volume will be released on 15th December, 2009. The range and variety of the chapters should make it excellent reading. There is a chapter on validity in language testing by Micheline Chalhoub-Deville, and the chapter by Bruno Zumbo is available on the internet (see the 'Articles: Interim Update' message).

Product Description

Validity is widely held to be the most important criterion for an assessment. Nevertheless, assessment professionals have disagreed about the meaning of validity almost from the introduction of the term as applied to testing about 100 years ago. Over the years, the best and brightest people in assessment have contributed their thinking to this problem and the fact that they have not agreed is testimony to the complexity and importance of validity. Even today, ways to define validity are being debated in the published literature in the assessment profession. How can such a fundamental concept be so controversial? This book brings focus to diverse perspectives about validity. Its chapter authors were chosen because of their expertise and because they differ from each other in the ways they think about the validity construct. Its introduction and ten chapters bridge both the theoretical and the practical. Contributors include most prominent names in the field of validity and their perspectives are at once cogent and controversial. From these diverse and well-informed discussions, the reader will gain a deep understanding of the core issues in validity along with directions toward possible resolutions. The debate that exists among these authors is a rich one that will stimulate the reader's own understanding and opinion. Several chapters are oriented more practically. Ways to study validity are presented by professionals who blend current assessment practice with new suggestions for what sort of evidence to develop and how to generate the needed information. In addition they provide examples of some of the options on how to present the validity argument in the most effective ways. The initial chapter by the editor is an effort to orient the reader as well as providing an overview of the book. Bob Lissitz has provided a brief perspective on each of the subsequent chapters as well as presenting a series of questions regarding validation that the reader will want to try to answer for themselves, as he or she reads through this book. This book's topic is fundamental to assessment, its authors are distinguished, and its scope is broad. It deserves to become established as a fundamental reference on validity for years to come.

Link Warning

3 Dec 2009 09:29:16 GMT

Late Wednesday Brett Reynolds sent a message to LTEST-L to warn that the DIALANG test site had potentially been hacked into. I have checked this, and it is an html/framer virus that re-directs your access via an IP tracker; other hidden advertising content is also lodged in the temporary internet file folder. DIALANG itself (the software and test) is not compromised. However, until this is cleared up users should not click on the DIALANG link on the tools page.

New Feature: Review of the Year 2009

1 Dec 2009 00:12:46 GMT

There is a new feature on the langauge testing resources site this month. I've always interested in the 'review of the year' programmes that we get on radio and television towards the end of the year; and so this month I thought that I would try a 'language testing review' for the first time. The review has links to the major language testing stories that have got into the newspapers, radio and television during 2009, along with a short commentary.

Articles Added

1 Dec 2009 00:05:29 GMT

As it's the end of the year I've scoured the free access journals and other sources for updated content. There are a significant number of additional links, including to papers produced in the ETS R&D Connections series, which provide introductions to ETS research in accessible language.

RSS Feeds

1 Dec 2009 00:02:29 GMT

This is the first month that the langauge testing resources site has had an RSS Feed. I hope this is a useful addition to the other tools on the site.

Defining 'Language Testing'

1 Dec 2009 00:11:24 GMT

If you are asked by the person sitting next to you on a plane what 'language testing' is, what do you say without talking for the rest of the flight? If you were asked to produce a definition of 'language testing' that could be used in a dictionary, what would you write? That's what this competition is about.

Entries will be judged on the quality, brevity and precision of the prose, the adequacy and scope of the definition, and appropriacy as a dictionary-style definition. Enter the competition between now and February 3rd, 2010.

Winner of the Prize Draw

1 Dec 2009 00:04:30 GMT

You will remember that earlier this year I was conducting the Practical Language Testing Survey, and each person who responded had the opportunity to enter a prize draw to win a copy of the book. There were approximately 300 entries, and in order to randomly select a winner I used a random number generator. The number selected was response 129, which was Agnieszka Kotula, a language teacher from Poland, and currently a student of applied linguistics. Congratulations to Agnieszka, and a copy of the book will be on its way to you as soon as possible. And thank you once again to all those who took the time to complete the survey, and help with the research. I wish I could afford to send a copy to everyone - but there is another chance to win one by entering the winter competition!

New Book from Dan Douglas

1 Dec 2009 00:01:00 GMT

Dan Douglas' new book, Understanding Language Testing, will be published by Hodder Education on Christmas Day. Pre-ordering is possible on Amazon. The last time I checked the price (30th November), it was £14.99.

Product Description

Understanding Language Testing presents an introduction to language tests and the process of test development that starts at the very beginning. Assuming no knowledge of the field, the book promotes a practical understanding of language testing using examples from a variety of languages.
While grounded on solid theoretical principles, the book focuses on fostering a true understanding of the various uses of language tests and the process of test development, scoring test performance, analyzing and interpreting test results, and above all, using tests as ethically and fairly as possible so that test takers are given every opportunity to do their best, to learn as much as possible, and feel positive about their language learning.
Each chapter includes a summary, suggestions for further reading, and exercises. As such this is the ideal book for both beginning students of linguistics and language education, or anyone in a related discipline looking for a first introduction to language testing.