Glenn Fulcher

Home

About

Contact

Search

Citation

Profile, Research, Publications, Education

Dr Glenn Fulcher

@languagetesting.info

[Profile] [Books] [Papers] [Education]

Stuff About Me, Research, Assessment and Education
My main interests lie in the field of language testing and the philosophy of educational assessment. I have extensive experience of working on test design and development projects, including the design and creation of test specifications, item/task design and prototyping, item/task review procedures, pre-testing, test analysis and assembly, and the design of both paper and computer delivery systems. I have a particular expertise in the design and development of rating instruments for performance assessments, particularly those in the assessment of speaking and communication. I developed the first data-based rating scales for speaking tests between 1987 and 1993, and continue to conduct research in this area. I have worked for a number of major testing companies in a research capacity, or as a consultant, on both large- and small-scale test development projects. I have been involved with alignment studies, and in training item writers and raters for performance assessments. I have also written on the ethical and legal aspects of language testing and assessment, including test mandates, political agendas, and legal cases involving test use and interpretation. Finally, I am interested in teaching language testing; I currently teach two modules each year, both on campus and by distance learning. In both modes I am keen to develop innovative electronic materials to enhance the student experience.

Qualifications
BD (London)
AKC (London)
PGCE (Cambridge)
MTh (London)
MA Applied Linguistics (Birmingham)
PhD (Lancaster)

Selected Download

Get a copy of my 2009 paper on the political use of tests, published in ARAL.

During 2009 I was a Leverhulme Research Fellow. I am grateful to the Leverhulme Trust which awarded me a substantial grant to pursue research leading to the publication of Practical Language Testing.

I am co-editor of Language Testing (Sage) with Cathie Elder (Melbourne University). I am also a member of the editorial board of Assessing Writing (Elsevier).

I served on the Executive Board of the International Language Testing Association (ILTA) from 1998 - 2000. In 2005 I was elected Vice President, served as President in 2006, and then as ex-officio Immediate Past President from 2007 - 2008.

I am Reader in Education (Applied Linguistics & Language Testing) in the School of Education at the University of Leicester in the United Kingdom. I teach language testing and assessment, and research methods, on the MA in Applied Linguistics and TESOL, and doctoral programmes. I undertake research and consultancy work as a member of the University, sometimes through Consult Leicester. Please contact me for further information.

Latest Education News

White British school children 'worst hit' by poverty
3 September, 2010

Bargain netbooks bite back at Apple
3 September, 2010

And now for something completely different, Wolverhampton v-c decides
2 September, 2010

Social class affects white pupils' exam results more than those of ethnic minorities ? study
2 September, 2010

School lotteries fail to help poorer pupils
2 September, 2010

Now just one man aged under 25 works in a state nursery school
2 September, 2010

Maths prodigy, now 15, heads for Cambridge
2 September, 2010

When I'm not working, reading, walking, swimming or practising guitar, I enjoy listening to the radio, usually while cooking! My favourite programmes include comedy from the BBC, philosophy, statistics, and world news. Here is a selection of good stuff. Why not listen while you explore other areas of this page?

FriComedy: Just a Minute 30 Aug 10
Popular panel game in which guests attempt to speak for a minute without hesitation, repetition or deviation. Recorded at the Edinburgh Fringe festival with guests Paul merton, Jenny Eclair, Fred Macaulay and Stephen K. Amos.

Joshua Knobe on Experimental Philosophy

Many people think that the idea of experiments in philosophy is a contradiction. Joshua Knobe disagrees. He is at the forefront of a new movement known as Experimental Philosophy. David Edmonds interviews him in this episode of the Philosophy Bites podcast.

Philosophy Bites is made in association with the Institute of Philosophy.

There is now a Philosophy Bites book published by Oxford University Press - further details are available on www.philosophybites.com

MoreOrLess: 03 Sep 2010
How reliable are life expectancy figures? Can cycling ever be safer than driving? And, what can maths tell us about guerilla insurgencies?

For news, it doesn't get much better than this roundup of the days events from all of the BBC's news services

NewsPod: 2 Sept 10 Middle East peace talks resume
Middle East talks / Hague supporters / Hawking and God / Living with dementia / Labour ballot

Language Testing Books

Fulcher, G. (2010). Practical Language Testing.
London: Hodder Education.
ISBN-10: 0340984481; 342 pages + xiv.

Order from Amazon UK

Order from Amazon US

The blurb from the back cover, the Preface, and contents list

Practical Language Testing equips you with the skills, knowledge and principles necessary to understand and construct language tests.

This intensely practical book situates testing both within the classroom and within a larger social and educational context. It provides the reader with the necessary tools to design and develop assessments, and gives guidelines on how to evaluate and improve them. The book also offers direction on alignment to standards, and how to manage test administration. The role of large-scale normative testing is discussed, and readers are provided with the knowledge required to prepare learners for tests, and to make realistic and fair decisions about the use and implementation of tests in their own context.

Practical Language Testing is the ideal introduction for students of Applied Linguistics, TESOL and Modern Foreign Language teaching as well as for practising teachers who are required to design or implement language tests.

************************************

Preface

This book is about building and using language tests and assessments. It does what it says on the tin: it is a practical approach. However, it does not provide ready-made solutions. Language testing is a complex social phenomenon, and its practice changes lives. The book therefore assumes that you will wish to think carefully about testing and its impact in your own context.

The term 'practical' therefore needs some definition. The book is 'practical' in the sense that it gives guidance on how to do things to build a test. It is also 'practical' in that each chapter will be useful to you when you come to making decisions about when, why, and how to conduct assessments. The book is designed to provide the knowledge you will need to apply, and the skills you will need to practice. However, if we are to build good language tests we have to be aware of the larger social, ethical, and historical context, within which we work. If language testing and assessment is not guided by principles we could end up doing more harm than good. Davies (2008a) has cogently argued that testing and assessment texts that do not embed knowledge and skills in principles, ignore the increasing demand of professionalism and social responsibility.

Language professionals, applied linguists and educational policy makers need an expanded 'assessment literacy' in order to make the right decisions for language learners and institutions (Taylor, 2009). This literacy will be about learning the nuts and bolts of writing better test items (Coniam, 2008), and establishing a core knowledge base (Inbar, 2008); but it is also about appreciating the reasons why we test, why we test the way we do, and how test use can enrich or destroy people's hopes, ambitions, and lives.

Although I am far from being in the 'postmodern' school of language testing and assessment, the view that language testing is a social activity cannot be denied (McNamara, 2001). Nor can the fact that our practices are thoroughly grounded in a long history that has brought us to where we are (Spolsky, 1995). It is partly because of this history that many texts published 'for teachers' focus almost entirely upon the technologies of normative large-scale standardized testing. While it is important that teachers are familiar with these, they are not always directly relevant to the classroom. This book therefore tries to introduce a balance between standardized testing, and classroom assessment.

The structure reflects a conscious decision to place language testing and assessment within context, and to provide the 'practical' guidance on the nuts and bolts of test building. Broadly, the first three chapters survey the language testing landscape upon which we can build. Chapter four is about the material that we can use in construction, and the rest of the book takes the reader through the process of building and implementing a language test.

Chapter 1 considers the purpose of testing in the broadest sense of why societies use tests, and in the narrow sense of how we define the purpose of a particular test. It looks at how tests are used, for good and ill; and the unintended consequences that testing can have on people who are caught up in the need to succeed.  Chapters 2 and 3 deal in turn with large-scale standardized testing, and then with classroom assessment. The stories of both paradigms are set within a historical framework so that you can see where the theories and practices originate.

In chapter 4 we begin the journey through the process of test design, starting with deciding what to test, and why. Chapter 5 begins the test design process in earnest, where we discuss how to create test specifications - the basic design documents that help us to build a test. This is where we learn to become 'test architects', shaping the materials and putting them together in plans that can be used to produce usable test forms. In chapter 6 we look at how to evaluate the test specifications and test items, from initial critical discussions in specification workshops to trying out items and tests with learners.  Chapter 7 contains a discussion of scoring, covering both traditional item types like multiple-choice, as well as performance tests that require human judgment. Frequently, we have to use tests to make decisions that require a 'cut score' - a level on the test above which a test taker is judged to be a 'master', and below which they are still 'novices'. Establishing cut scores and linking these to absolute standards is the subject of chapter 8. Chapter 9 discusses the practicalities of test administration, and why the 'rituals' of testing have grown as they have.

Finally, in chapter 10, we return to the classroom and to the effect that tests have upon learning and teaching, and how we go about preparing learners to take tests.

Throughout the book I have included examples from real tests and assessments. Some of these are good examples that we can emulate. Others are provided for you to critique and improve. Some of them are also drawn from historical sources, as 'distance' is useful for nurturing critical awareness. However, I do not present sets of typical test items and tasks that you could simply select to include in your own tests. There are plenty of books on the market that do this. This book asks you to think about what item or task types would be most useful for your own tests.  We discuss options, but only you can provide the answers and the rationales for the choices you make.

There are activities at the end of every chapter that you can attempt on your own, although many would benefit from team work. Sharing experiences and debating difficult issues is best done in a group. And it's also more fun. The activities have been designed to help you think through issues raised in the chapter, and practice the skills that you have learned. The activities are not exhaustive, and you are encouraged to add to these if you are using the text in a language testing course. Beginning in chapter 4 there is also a Project that you may wish to do as you move from chapter to chapter.

This structure has been shaped not only by my own understanding of what an introductory book to foster 'assessment literacy' might look like, but also by what language teachers and students of applied linguistics have told me that they need to know, and be able to do.  Prior to writing the book I undertook a large-scale internet based survey, funded by the Leverhulme Trust. Almost 300 respondents completed the survey, and I was struck by the sophistication of their awareness of assessment issues.

Here is a selection of typical responses to a question about what teachers and students of applied linguistics most need in a 'practical' language text:

"Evaluating reliability for our in-house tests, and checking questions at each stage in test development."

"I don't understand statistics, but I know they can be useful. I need it explaining conceptually."

"We need to know the jargon, but introduce it step by step."

"Hands-on activities; examples of test specs; a glossary would be useful."

"A book of this type must focus on the basics of item writing and test construction, the basic concepts of validity and reliability, particularly in regards to the assessment of speaking and writing. It must also cover the ethics of test use and test score interpretation."

"Developing classroom tests, performance tests, setting score standards, deciding what to test, preparing learners for test situations."

"Differentiation between classroom assessments, formative assessment, and large scale assessment when discussing key issues."

"Most of the assessment/testing practices are done by teachers; I think that a book should be aimed at 'normal' language teachers more than specialists in testing, they already have other sources of information and training."

"Issues to do with ensuring validity and reliability in language testing. The test writing process from the creation of test specifications through to the trialling, administration and marking of tests."

"Vignettes; glossary; application activities for individuals and groups, including some practice with basic test statistics and approaches to calculating grades."

"Some information on testing as an industry, a multi-billion dollar concern and why we have to fight crap when we see it."

Luckily, many respondents said they realised that it is impossible to include everything in a practical language testing book. This is evidently true, as you will see. I am sure to have left out a topic that you think should have been included.  One respondent understood this all to well: "The book should be well-structured, clearly focused, and however tempted you might be to put everything into one book, you should be selective in order to be comprehensible and user-friendly."

I am not entirely sure that I have achieved this. But if I have got even half way there my time will have been well spent.

As another respondent said, "the learning never ends." In order to sustain you during your journey through the book you may wish to pay regular visits to my website:

http://languagetesting.info

Here you will find a set of online videos that define and explain some of the key concepts and topics in language testing. To help you with additional reading I have links to online articles, and other language testing websites. There are links to useful journals, and regular updates on testing stories that get into the news.

Constructive criticism is always welcome, via the website.

************************************

Book Content

Chapter 1: Testing and Assessment in Context
1. Test Purpose
2. Tests in educational systems
3. Testing rituals
4. Unintended consequences
5. Testing and society
6. Historical Interlude I
7. The politics of language testing
8. Historical Interlude II
9. Professionalising language education and testing
10. Validity
11. Activities

Chapter 2: Standardized Testing 
1. Two paradigms
2. Testing as science
3. What's in a curve?
4. The curve and score meaning
5. Putting it into practice
6. Test scores in a consumer age
7. Testing the test
8. Introducing reliability
9. Calculating reliability
10. Living with uncertainty
11. Reliability and test length
12. Relationships with other measures
13. Measurement
14. Activities

Chapter 3: Classroom Assessment
1. Life at the chalk-face
2. Assessment for Learning
3. Self- and Peer-Assessment
4. Dynamic Assessment
5. Understanding change
6. Assessment and Second Language Acquisition
7. Criterion-Referenced Testing
8. Dependability
9. Some thoughts on theory
10. Activities

Chapter 4: Deciding What to Test
1. The test design cycle
2. Construct definition
3. Where do constructs come from?
4. Models of communicative competence
5. Construct models
6. Performance models
7. From definition to design
8. Activities

Chapter 5: Designing Test Specifications
1. What are test specifications?
2. Specifications for testing and teaching
3. A sample detailed specification for a reading test
4. Granularity
5. Performance Conditions
6. Target language use domain analysis
7. Moving back and forth
8. Activities

Chapter 6: Evaluating, Prototyping and Piloting
1. Investigating usefulness and usability
2. Evaluating items, tasks and specifications
3. Guidelines for multiple-choice items
4. Prototyping
5. Piloting
6. Field Testing
7. Item shells
8. Operational item review and pre-testing
9. Activities

Chapter 7: Scoring Language Tests
1. Scoring items
2. Scorability
3. Scoring constructed response tasks
4. Automated scoring
5. Corrections for guessing
6. Avoiding own goals
7. Activities

Chapter 8: Aligning Tests to Standards
1. It's as old as the hills
2. The definition of 'standards'
3. The uses of standards
4. Unintended consequences revisited
5. Using standards for harmonization and identity
6. How many standards can we afford?
7. Performance Level Descriptors (PLDs) and test scores
8. Some initial decisions
9. Standard setting methodologies
10. Evaluating standard setting
11. Training
12. The special case of the CEFR
13. You can always count on uncertainty
14. Activities

Chapter 9: Test Administration
1. No, no. Not me!
2. Controlling extraneous variables
3. Rituals revisited
4. Standardized conditions and training
5. Planned variation: Accommodation
6. Unplanned variation: Cheating
7. Scoring and moderation
8. Data handling and policy
9. Reporting outcomes to stakeholders
10. The expense of it all
11. Activities

Chapter 10: Testing and Teaching
1. The things we do for tests
2. Washback
3. Washback and content alignment
4. Preparing learners for tests
5. Selecting and using tests
6. The Gold Standard
7. Activities

Epilogue

Glossary

References

Index

The research for this book has kindly been supported by the The Leverhulme Trust, which awarded me a Leverhulme Research Fellowship to fund the research and study leave. I am also grateful to the University of Leicester, which granted me study leave.

Fulcher, G. and Davidson, F. (2007). Language Testing and Assessment
London & New York: Routledge. ISBN: 0415339472. 424 pages.

Order from Amazon (UK)
Order from Amazon (USA)

The blurb from the back cover & the Authors' Preface

Language Testing and Assessment:

�	provides a fresh and thorough review from practical details of test development to matters of controversy and ethical practice

�	develops important themes, chiefly that all testing should be effect-driven and highly pragmatic: during test development and data analysis, the first and best determiner of validity is attention to consequence

�	illustrates its thematic breadth in a series of thought exercises and tasks, such as analysis of test results, study of test revision and change, design of arguments for test validation, and exploration of influences on test creation

�	presents influential and seminal readings in testing and assessment by names such as Michael Canale and Merrill Swain, Michael Kane, Alan Davies, Lee Cronbach and Paul Meehl, and Pamela Moss.

The authors are experienced language test developers and researchers.  They have trained many graduate students in assessment and testing, and their experience and training make Language Testing and Assessment both a valuable shelf reference and a superb source book for language educators, worldwide.

----------------

Authors' Preface (How to Use This Book)

Testing and assessment is part of modern life. School children around the world are constantly assessed, whether to monitor their educational progress, or for governments to evaluate the quality of school systems. Adults are tested to see if they are suitable for a job they have applied for, or if they have the skills necessary for promotion. Entrance to educational establishments, to professions, and even to entire countries, is sometimes controlled by tests. Tests play a fundamental and controversial role in allowing access to the limited resources and opportunities that our world provides. The importance of understanding what we test, how we test, and the impact that the use of tests has on individuals and societies cannot be understated. Testing is more than a technical activity; it is also an ethical enterprise.

The practice of language testing draws upon, and also contributes to, all disciplines within applied linguistics. However, there is something fundamentally different about language testing. Language testing is all about building better tests, researching how to build better tests, and in so doing, understanding better the things that we test.

Sociolinguists do not create 'sociolinguistic things'. Discourse analysts do not create 'discourses'. Phonologists do not create spoken utterances.  Language testing, in contrast, is about doing. It is about creating tests.

In a sense, therefore, each section of this book is about the practical aspects of doing and of creating. And so each section has a research implication; no section is concerned purely with exposition. Research ideas may be made explicit in C: Exploration, but they are implicit throughout the book; put another way, the creative drive of language testing makes it a research enterprise, we think, at all times.

In the text we do not merely reflect the state of the art in language testing and assessment; nor do we simply introduce existing research. Our discussion is set within a new approach that we believe brings together testing practice, theory, ethics and philosophy. At the heart of our new approach is the concept of effect driven testing. This is a view of test validity that is highly pragmatic.  Our emphasis is on the outcome of testing activities. Our concern with test effect informs the order and structure of chapters, and it defines our approach to test design and development.

As test design and development is about doing, creating and researching, we have taken special care over the activities. With Dewey, we believe that through doing we grow as language testers, as applied linguists, and as language teachers.

The book is divided into three sections. A: Introduction consists of 10 Units dealing with the central concepts of Language Testing and Assessment. It contains activities for you to carry out alone or with others if you are studying this book as part of a course. B: Extension provides extracts from articles or books relating to Language Testing and Assessment which give you further insights into the concepts introduced in Section A. Each extract in B: Extension is accompanied by activities to focus your reading and help you to evaluate critically what you have read and understand how it links to a wider discussion of language testing and assessment. C: Exploration builds on the material you will already have found in the book. In this section we provide extended activities that help you to work through practical and theoretical problems that have been posed in the other sections. We also present ideas for individual and group project work, as well as suggestions for research projects.

The organisation of this book means that you can concentrate on particular themes such as Classroom Assessment, or Writing Items and Tasks by reading the relevant units from A: Introduction, B: Extension and C: Exploration consecutively. Alternatively, you may wish to read the whole of A: Introduction before embarking on Sections B and C. In fact, you may decide to read the Sections in any sequence, just as you would read Julio Cortazar�s novel Hopscotch: there is no one right place to start, and each path through the text provides a different experience. Whichever choice you make, the book is extensively cross-referenced and carefully indexed so that you can easily find your way around the material.

At the end of the book we provide a glossary of key terms that are not explained within the text itself. If you come across a term about which you feel uncertain, simply turn to the glossary for an explanation. We also provide an extensive list of references for additional reading.

In addition to the book itself, there is also a web site which will provide very extensive additional reading, activities, links to relevant web sites, and further ideas for projects that you might like to undertake on your own or with colleagues.

A book in the Routledge
Applied Linguistics Series.

Read the Reviews here

Fulcher, G. (2003). Testing Second Language Speaking
London: Longman/Pearson Education.
ISBN: 0582-472709. 312 pages.

Order from Amazon (UK)
Order from Amazon (USA)

What is this book about? 
Find out here.

Testing second language speaking has become extremely important since the Second World War. When it was realised the American service personnel were often ill equipped to communicate, that their inability to speak in a second language might be 'a serious handicap to safety and comfort', the push for tests of second language speaking began in earnest. From the needs of the military, the practice soon spread to colleges and universities. Today, speaking tests are available from all major test producers, for a multitude of purposes.

This book offers a comprehensive treatment of testing speaking in a second language. It will be useful for anyone who has to develop speaking tests in their own institutions, which is becoming an increasingly common requirement placed upon teachers who may have no training in test development. For teachers this book contains a generic programmatic approach from design to implementation that can be adapted to local conditions. Teachers are also asked to evaluate tests for use by their institutions when there is a need to use speaking tests produced by agencies like Educational Testing Service or the University of Cambridge Local Examinations Syndicate. The book provides clear guidance about the kinds of questions teachers may ask, and how to evaluate the answers. But it is never prescriptive.

The book is also intended for students of Applied Linguistics, languages and Teaching English to Speakers of Other Languages (TESOL). It brings together in one volume a summary and critical analysis of research into testing second language speaking. There is a wealth of examples, not only of task types that are commonly used in speaking tests, but of approaches to researching speaking tests, and specific methodologies, that students may have to use in their own projects. The approaches and methodologies are presented with examples from the literature, and are carefully referenced for follow-up study.

It is a 'practical' text for teachers and students. But it is not just a practical text. There is a theoretical framework, drawing on our evolving understanding of validity in language testing, which underpins the discussions and examples. We argue that practical decisions in speaking test development only make sense when we understand why we make those decisions. There is no one 'correct' decision in any testing context. We are faced with many possible choices, and the process of making those choices is a crucial aspect of understanding what the scores from our speaking tests might mean. Establishing what scores from a speaking test might mean is part of constructing, or evaluating, a validity argument (see the discussion at the beginning of Chapter 5). Readers are introduced to how validity arguments are constructed, what they may contain, what evidence can be presented to support them, and how we can evaluate them. Validity arguments are never 'static'. They are dynamic, fallible, endlessly evolving, attempts to investigate test score meaning. Ultimately we judge them by their utility and plausibility.

For teachers and students of language testing alike, this book tries to provide a theoretical backdrop to the issues raised. And so it cannot avoid philosophical matters. There is no running away from questions of meaning and interpretation. Statistical analysis will always play a part in language testing, but the ultimate challenge, as Davidson (2000) so eloquently reminds us, is how we utilise the range of tools at our disposal to address questions that are essentially human and social in essence. The same applies to qualitative approaches to researching second language speaking tests. Investigating how test takers do tasks, their affective reactions, the language they use, are all legitimate areas of enquiry that enhance our understanding of what we are doing when we test second language speaking. The interpretation of qualitative evidence to support a validity argument is just as problematic as the interpretation of quantitative evidence, as we shall see in Chapter 8.

The quality of a validity argument is subject to the evaluation of the community of language testing specialists, those who take the tests, the consumers of test scores, and society at large. Placing a book in this context also makes it relevant, we hope, to language testing researchers, and all those stakeholders who use have an interest in how test scores are used to make critical decisions about the lives of individuals.

Although perhaps better left to the judgement of the reader and reviewers, we would like to think that this book is therefore about language testing in general, as well as the testing of second language speaking in particular. It adopts a pragmatic stance. There is no suggestion that there is one 'right' way to do things. But we do argue that there is a responsibility to be systematic and deliberate in test development, employ a variety of analytical techniques, and present validity arguments that are open to scrutiny. So it is different from Underhill (1987), which is a polemic against statistical analysis, while still managing to provide a useful list of task types for speaking tests. And while we spend many pages evaluating the American Oral Proficiency Interview, this is not the primary focus of the book, as it is for Johnson (2001). Rather, we are concerned about what we can learn from the tradition of the Oral Proficiency Interview to improve the quality of validity arguments.

Just as our approach to language testing cannot exist in some kind of limbo separate from our conception of society and the rights of the individual test taker, nor can it remain isolated from history. Testing second language speaking has a history, albeit a relatively short one. The first large-scale test of speaking was developed and implemented to exclude immigrants from the United States, because there was a loop-hole in the 1924 immigration act. If readers of this book think that such practices are long gone, they need only read Shohamy (2001). And if we reflect on why we have the linear rating scales that we use today, we need only look back to the first rating scales constructed for the military in the 1950s.

All of the considerations outlined above have influenced the structure and content of this book. It is structured so that it can be read from Chapter 1 to Chapter 8. However, some readers may wish to turn directly to a chapter that addresses a particular issue in the testing of second language speaking. The following outline of the book may be useful in deciding upon priorities, which is broken down into four sections. Chapter 1 is essentially historical. Chapters 2 to 5 are concerned with the process of design and development of second language speaking tests. Chapter 6 deals with raters and interlocutors, training, administration and finance. Chapters 7 and 8 look at evaluation and research. This is an artificial compartmentalisation, but may nevertheless be useful.

In Chapter 1 we look in some detail at the history of testing second language speaking. It looks at the political and military impetus behind testing speaking. Not all of the story is particularly pleasant. But for the most part, testing speaking was conducted for the benefit of the test takers, whether it was to ensure their safety in military operations or place them in the right class at school.

In Chapter 2 we turn to the construct. We attempt to define what is meant by the term 'construct', and then ask how we can define the construct of 'speaking', or its constituents. This is particularly important. If we can't define what we think we are testing, the scores from a speaking test are hardly likely to be meaningful. When we have defined what we are looking for, we can go on to consider how we wish to elicit speech from test takers that will provide the basis for giving a score. This is the subject of Chapter 3. The use of tasks in second language pedagogy, second language acquisition research, and second language speaking tests is reviewed. We tackle the thorny problem of 'what makes a task difficult', before presenting a range of task types used in second language speaking tests within a test task framework. Having elicited a performance, a speech sample, we have to score it. Chapter 4 outlines the types of rating scales that can be employed, and describes in some detail four distinctive approaches to the development of rating scales for second language speaking tests. Chapter 5 is designed to draw together everything from Chapter 2 to Chapter 4. We begin by discussing what is meant by a validity argument, and how this encompasses our construct definition, the tasks we use, and how we score the performances on the tasks. These are brought together in the test specifications for a test of second language speaking. But it is not just a matter of sitting down to write test specifications. The process of producing test specifications is characterised as non-linear, and decidedly 'messy'. We provide four examples of test specifications for speaking tests, and a generic template for specifications based on the work of Davidson and Lynch (2002).

Chapter 6 moves us into getting ready for 'live or 'operational' testing. What does it mean for a rater to be 'reliable'? And is it necessary for a rater to be reliable? Rater reliability studies are reviewed, and we consider approaches to rater training and certification. In recent years, just as much attention has also been paid to the interlocutor in the speaking test. We consider the sources of interlocutor variability in second language speaking tests, and how this can be controlled through training. Equally important, although not frequently discussed, are local performance conditions. These conditions are defined as the administration of the speaking test, and the environment in which it takes place. We look at these issues from the perspective of how variations in administration or environment can impact on the score that an individual test taker may get. But we recognise that the administration of the speaking test may be deliberately altered for some test takers with disabilities. We outline the types of 'accommodation' that may be offered test takers with disabilities, such as deafness. We discuss how this impacts upon our definition of the construct, and how we might report scores from the test. Last, but not least, we look at how much it costs to design a test of second language speaking. A working budget is presented that draws on all the processes described in Chapters 2 to 6. The cost of the process is significant, especially for institutionally developed speaking tests where resources may be limited. We look at what is lost in validity claims through cutting corners to reduce costs, and what may be lost through court action if costs are cut and high stakes decisions retained.

Chapter 7 offers a view of how we may evaluate tests of second language speaking. Drawing upon the discussion in Chapter 5 it presents three validity arguments, with a review of the literature associated with each argument. The first argument is the most traditional, and for which there is a wealth of literature. It is the argument for the validity of the Oral Proficiency Interview, which is termed the 'FSI tradition of testing speaking'. The second argument is more recent. We consider the claims made for testing second language learners in pairs or groups, and evaluate the evidence presented for the proponents and opponents of this format. Thirdly, we look at the validity argument for 'indirect' (semi-direct, simulated) tests of second language speaking. In each of the three examples we ask how plausible the arguments are, given the probability that further evidence or rationales may be presented to change the nature of the validity argument in the future. We conclude this chapter by outlining a validity model based upon the work of Messick, which summarises the types of questions we may ask about any validity argument, and the sort of evidence that could be provided to support a particular validity argument.

Chapter 8 considers the type of evidence that may be offered to support or undermine a validity argument, while acknowledging that there is no one-to-one mapping between the source of evidence and the argument. Various approaches to analysis are described and discussed, under the headings of 'quantitative' and 'qualitative' methods. It is argued that this is an artificial distinction. Quantitative methods always involve human interpretation. Qualitative methods usually have a quantitative component, but equally involve human interpretation. The methods that we select to study tests of second language speaking are chosen because of their utility for addressing the validity questions that are most pressing in assuring the quality of our argument, and fairness to those who would take the test.

It is sometimes claimed that testing second language speaking is a much more difficult undertaking than testing other second language 'abilities', 'capacities', 'competencies', 'skills', or whatever other labels we may wish to use. Perhaps because it is the 'ability' that makes us human. Perhaps because speaking is fleeting, temporal, and ephemeral. But the paucity of books on testing second language speaking might suggest that testing speaking is difficult. It is hoped that this book will fill the gap for those who are concerned with testing or assessing this most intrinsically human of capacities.

Read the Reviews here

In preparation

Fulcher, G. and Davidson, F. (Eds.) The Routledge Handbook of Language Testing. London and New York: Routledge

More information will be available on the content and availability of this volume shortly.

Selected Papers

Performance Testing and Scoring

Fulcher, G., Davidson, F. & Kemp, J. (forthcoming). "Effective rating scale development for speaking tests: Performance Decision Trees." Language Testing.

Fulcher, G. (2008). "Assessing Language Quality". In Shohamy, E. (Ed.) Language Testing and Assessment. Vol. 7, Encyclopedia of Language and Education. New York: Springer Publishers, 157 - 176.

Fulcher, G. (2007). "Evaluating quality in second language performance tests." English Language Assessment 1, 3 - 19.

Fulcher, G. (1998). "Testing Speaking". In Clapham, C. (Ed). Language Testing and Assessment. Vol. 7. of the Encyclopaedia of Language and Education, Kluwer Academic Publishers, 75 - 86.

Fulcher, G. (1997). "Assessing Writing". In Fulcher, G. (Ed) Writing in the English Language Classroom. Phoenix ELT/Prentice Hall Macmillan.

Fulcher, G. (1996). "Testing tasks: issues in task design and the group oral." Language Testing 13, 1, 23 - 51.

Fulcher, G. (1996). "Does thick description lead to smart tests? A data-based approach to rating scale construction". Language Testing 13, 2, 208 - 238.

Fulcher, G. (1994). "Some priority areas in oral language testing research." Language Testing Update 15, 39 - 47.

Fulcher, G. (1989). Lexis and reality in oral evaluation. US Department of Education, Office of Educational Research and Improvement, and the Centre for Applied Linguistics. ERIC document number: ED 298759.

Fulcher, G. (1987). "Tests of Oral Performance: the need for data-based criteria." English Language Teaching Journal 41, 4, 287 - 291.

Test and Task Design

Fulcher, G. (forthcoming). "Test Design and Retrofit." In Chapelle, C. A. (Ed.) The Encyclopeidia of Applied Linguistics. London: Blackwell.

Fulcher, G. and Davidson, F. (2009). "Test Architecture, Test Retrofit." Language Testing 26, 1, 123 - 144

Fulcher, G. (2007). "Industrial methods for cottage industries? Alternatives for classroom assessment." In Niakaris, C. I. and Nebel, A. (Eds.) Testing and Teaching: Opportunities for Learning. Athens: Hellenic American Union, 44 - 51

Davidson, F. and Fulcher, G. (2007). "The Common European Framework of Reference (CEFR) and the design of language tests: A Matter of Effect." Language Teaching 40, 3, 231 - 241.

Fulcher, G. (2006). "Test Architecture." Foreign Language Education Research, 9, 1 - 22.

Davidson, F. and Fulcher, G. (2006). "Flexibility is proof of a good 'framework'." Guardian Education, 17 November.

Fulcher, G. and Marquez Reiter, R. (2003). "Task difficulty in speaking tests." Language Testing 20, 3, 321 - 344.

Fulcher, G. (1998). "Widdowson's model of communicative competence and the testing of reading: An exploratory study." System 26, 3, 281 - 302.

Issues in Validity and Reliability

Xing, P. and Fulcher, G. (2007). "Reliability assessment for two versions of the Vocabulary Levels Test." System 35, 2, 182 - 191.

Chalhoub-Deville, M. and Fulcher, G. (2003). "The oral proficiency interview and the ACTFL Guidelines: A research agenda." Foreign Language Annals 36, 4, 498 - 506.

Fulcher, G. (1999). "Assessment in English for Academic Purposes: Putting content validity in its place." Applied Linguistics 20, 2, 221 - 236

Fulcher, G. (1997). "An English Language Placement Test: Issues in reliability and validity." Language Testing 14, 2, 113 - 139.

Fulcher, G. (1996). "Invalidating validity claims for the ACTFL Oral Rating Scale." System 24, 2, 163 - 172.

Fulcher, G. (1996). "The interplay of evidence in the consideration of variable scores and performance across tasks." In J. Field, A. Graham and M. Peacock (Eds) Insights 1. Kent: IATEFL Publications.

Fulcher, G. (1995). "Variable competence and Second Language Acquisition: A problem for research methodology. System 25, 1, 25 - 33.

Fulcher, G. (1988). "Item Response Theory: what is it?" Education Today 28, 4, 44 - 52.

Fulcher, G. (1987). "Measurement or Assessment: a fundamental dichotomy and its educational implications." Education Today 37, 2, 60 - 65.

Pedagogy and Applied Lingistics

Fulcher, G. (2009). The commercialization of language provision at University. In Alderson, J. C. (Ed.) The Politics of Language Education: Individuals and Institutions. London: Multilingual Matters, 125 - 146.

Fulcher, G. (2007). "Universities undermine their own foundations." Guardian Education, 13 April.

Marquez Reiter, R., Rainey, I. and Fulcher, G. (2005). "A comparative study of conventional indirectness in British English and Peninsular Spanish." Applied Linguistics 26, 1, 1 - 31

Fulcher, G. and Locke, D. (1999). "Distance Education: The future of library and information services requirements." Distance Education 20, 2, 313 - 329.

Fulcher, G. (1999). "Library and Information Services Requirements: A Course Developer's Perspective." In D. A. Lock (ed.) Library Services for Distance Learners. Surrey: Proceedings of the first Conference on Library Services for Distance Education. 3 - 8.

Fulcher, G. (1997). "Text difficulty and accessibility: Reading formulae and expert judgement." System 25, 4, 497 - 513

Fulcher, G. (1991). "Conditionals Revisited." English Language Teaching Journal 45, 2, 164 - 168.

Fulcher, G. (1989). "Cohesion and coherence in theory and reading research." Journal of Research in Reading 12, 2, 146 - 163.

Fulcher, G. (1988). "The EFL Classroom as a place for research." Education Today 38, 4, 44 - 52.

Fulcher, G. (1988). "Teaching vocabulary for writing." Modern English Teacher 15, 1, 25 - 30.

Fulcher, G. (1987). "Contextual Hyponymy: a communicative approach to teaching lexis in context." Modern English Teacher 14, 3, 14 - 17.

Social, Political and Ethical Uses of Tests

Fulcher, G. (2010). "The reification of the Common European Framework of Reference (CEFR) and effect-driven testing." In Psyaltou-Joycey, A. and Matthaioudakis, M. (Eds.) Advances in Research on Language Acquisition and Teaching. Greece, Thessaloniki: GALA.

Fulcher, G. (2009). "Test use and political philosophy." Annual Review of Applied Linguistics 29, 3 - 20.

Fulcher, G. and Davidson, F. (2008). "Tests in Life and Learning: A Deathly Dialogue." Educational Philosophy and Theory, 40, 3, 407 - 417.

Fulcher, G. (2008). "Testing Times Ahead?" Liaison Magazine, Issue 1: July, 20 - 24. Published by the UK Subject Centre for Languages, Linguistics and Area Studies, University of Southampton.

Fulcher, G. (2005). "Language Framework Documents and the Hallmark of Political Purpose." In Niakaris, C. I. and Nebel, A. (Eds.) Testing and Evaluation. Athens: Hellenic American Union, 28 - 31.

Fulcher, G. (2004). "Deluded by artifices? The Common European Framework and harmonization." Language Assessment Quarterly, 1, 4, 253 - 266.

Fulcher, G. (2004). "Are Europe's tests being built on an unsafe framework?". Guardian Education, 18 March.

Fulcher, G. (2003). "Few ills cured by setting scores." Guardian Education, 17 April.

Fulcher, G. (1999). "Ethics in Language Testing." TAE SIG Newsletter 1, 1, 1 - 4.

Fulcher, G. and Bamford, R. (1996). "I didn't get the grade I need. Where's my solicitor?" System 24, 4, 437 - 448.

Computers, Technology and Testing

Fulcher, G. (2005). "Better Communications Test will Silence Critics". Guardian Education, 18 November.

Fulcher, G. (2003). "Interface design in computer based language testing." Language Testing 20, 4, 384 - 408.

Fulcher, G. (2001). "Machines get cleverer at testing." Guardian Education, 17th May.

Fulcher, G. (2000). "Computers in language testing." In Brett P. and Motteram, G. (Eds.) A special interest in computers: Learning and teaching with information and communications technologies. Manchester: IATEFL publications, 93 - 107.

Fulcher, G. (1999). "Computerising an English language placement test." English Language Teaching Journal 53, 4, 289 - 299

Fulcher, G. (1998). "Computer-based language testing: The call of the internet." In Coombe, C. A. (Ed.) Current Trends in English Language Testing. UAE: Al Ain University Press. 1 - 14.

Teaching, Learning and Assessment

Fulcher, G. (2003). "Scaling the peak of writing assignments/dissertations." Guardian Education, November 21.

Fulcher, G. (2002). "Managers face the tests." Invited editorial for the EFL Gazette, April, Issue 267.

Fulcher, G. (2001). "Communicative Language Testing." In Coombe, C., Riley, S. and Troudi, S. (Eds.) Bridging the Gap between Teacher and Learner. TESOL Arabia, 52 - 68.

Fulcher, G. (2000). "The 'communicative' legacy in language testing." System 28, 4, 483 - 497

Fulcher, G. (2000). "Assessment in Independent Language Learning." In the Curriculum and Independence for the Learner Network Handbook (CIEL)

Fulcher, G. (1996). "Writing in the Classroom" Modern English Teacher, 5, 3, 45 - 48.

Fulcher, G. (1991). "The role of assessment by teachers in schools." In Caudery, T. (ed) New Thinking in TEFL. (The Dolphin Series No. 21), Denmark: Aarhus University Press, 138 - 158.

Fulcher, G. (1990). A GCE O Level Workbook. Mavromatis Books, Nicosia, Cyprus.

Book and Software Reviews

Fulcher, G. (2005). "Educational Test Analysis." Language Testing Update,36, 152 - 154.

Fulcher, G. (2000). Review of Chalhoub-Deville, M (Ed.) 1999. Issues in computer-adaptive testing of reading proficiency: selected papers. Studies in Language Testing Vol. 10. Cambridge: Cambridge University Press. Language Testing 17, 3, 361 - 367.

Fulcher, G. (1999). Review of Barnwell, D. P. 1996: A History of Foreign Language Testing in the United States: from its beginnings to the present. Tempe, Arizona: Bilingual Press. Language Testing 16, 3, 389 - 394.

Fulcher, G. (1997). "QM Web: A World Wide Web Test Delivery Program." Language Testing Update 21, 45 - 49.

Fulcher, G. (1996). "Brief report on the Language Testing Forum 1995". Language Testing Update, 19, 79 - 80.

Fulcher, G. (1996). "Cyberspace revisited". Language Testing Update, 19, 43 - 51.

Fulcher, G. (1995). "The Language Tester's Guide to Cyberspace." Language Testing Update 17, 62 - 65.

Fulcher, G. (1990). "Review of Testing Spoken Language by Nic Underhill, CUP 1987. English Language Teaching Journal 41, 1, 80 - 82.

Oddities from the Past

Fulcher, G. (1993). "Synonymy RIP." I came across this in a dusty drawer recently. Strange how we sometimes do stuff and forget about it. Anyway, I thought it was worth resurrecting ;-)

Fulcher, G. (1983). "Structuralism." Before Applied Linguistics and Language Testing I studied philosophy and theology. Here's an early publication from my previous life!

Interviews

(June 2010). Glenn Fulcher talks to... ELT News Athens, Greece.

(August 2007). An Interview with Glenn Fulcher Shiken Japan, JALT.

Other Volumes

Fulcher, G. (2000). Expanding perspectives on language testing in the 21st century.

Guest Editor of a special edition of System, Volume 28, Number 4.

Fulcher, G. (1997). Writing in the English Language Classroom
Pearson Education/Phoenix ELT/Prentice Hall Macmillan.
ISBN: 0-13-626417-4

The blurb:

The papers in this book offer an overview of current thinking about writing in the language classroom. They address the key issues facing the classroom teacher, bringing theoretical, research-based and practical approaches to the day-to-day teaching of writing, devising writing programmes and assessment, including portfolio assessment.

Writing in the English Language Classroom also reviews and offers critical comment on the main approaches to teaching writing, including the Process and Genre approaches. It examines a variety of writing styles and traditions and considers the importance of cultural issues such as gender, social class and educational background, and their implications for language planning and literacy development policies.

A consideration of the growing role of electronic written communication and its relevance to the classroom completes a comprehensive  updte which provides the teacher of writing with an essential resource for the preparation of more effective classes.

Education


	1993: University of Lancaster. PhD in Applied Linguistics and Language Testing. Thesis: The Construction and Validation of Rating Scales for Oral Tests in English as a Foreign Language. 1987: University of Birmingham, English Language Research. MA in Applied English Linguistics. Dissertation: Discourse Analysis and the Testing of Reading Comprehension. 1982: University of London, King's College. MTh. Dissertation: Structuralism as a New Exegetical Method. 1981: University of Cambridge: Christ's College. Post-graduate Certificate in Education (PGCE). 1980: University of London, King's College. BD and Associate of King's College (AKC). Major: Textual Analysis of first century Greek texts. 1979: Exchange scholarship in philosophy to Dartmouth College, Hanover, New Hampshire, USA.