This web page lists articles in general applied linguistics and educational research journals that may be related to language testing and assessment, or use language tests as part of their methodology. Over thirty widely used journals are currently scanned for relevant articles. Content of the specialist language testing journals can be found here.

You may also wish to subscribe to the Article Alert Service, which automatically informs you of the publication of new articles on language testing from your browser's toolbar.

This web page is updated daily.

• Professional development of teachers?A prerequisite for AfL to be successfully implemented in the classroom
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Kari Smith
  A prerequisite for AfL to be successfully implemented in the classroom is the teachers? assessment pract... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Kari Smith
  A prerequisite for AfL to be successfully implemented in the classroom is the teachers? assessment practice. In many contexts, including the Norwegian, AfL has not been successfully dealt with during initial teacher education, and there is a need for qualified teachers to engage in professional development in AfL. This article first discusses different perspectives of professional development, not all of which lead to change of attitudes and beliefs resulting in changed practice. Difficulties in implementing AfL beyond the individual teacher and reach the full teaching staff is presented through experiences from a Norwegian research and development project. The next project to be briefly discussed focuses on engaging teachers in developing their own criteria for good classroom assessment. Finally the paper proposes that a major obstacle to teachers? professional development in AfL is the competence of teacher educators in AfL. Do those who are to guide and support professional development in schools hold the essential knowledge and competence themselves? The conclusive argument is that for AfL to be common assessment practice in schools there is a need for development processes at various levels of the education system.
  
  
  (show less)
  
  
• How do Dutch secondary teachers and students conceive the purpose of assessment?
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Mien Segers, Harm Tillema
  How teachers and students deal with assessment practices is largely influenced by the co... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Mien Segers, Harm Tillema
  How teachers and students deal with assessment practices is largely influenced by the conceptions they hold about the purpose of assessment. This means that, given the current plea for a paradigm shift from Assessment of Learning to Assessment for Leaning with a balance between the formative and summative purposes of assessment, the study of teachers? and students? conceptions of the purpose of assessment is of the utmost relevance. In this study Dutch secondary school teachers? (n =351) and students? (n =712) conceptions of the purpose of assessment were measured using a translation of the teachers? conceptions of assessment questionnaire and a translated and shortened version of the students? conceptions of assessment questionnaire. Maximum Likelihood factor analyses resulted in four teacher and five student conceptions of the purpose of assessment. The results indicate that the teachers in our sample did not distinguish between formative and summative purposes of assessment. They did discern classroom assessment (formative as well as summative) from school accountability. In addition, they conceive assessment as being of bad (in terms of measurement errors) or good quality. In contrast with the teachers, students do make a distinction between the formative and summative purpose of assessment. They also refer to school accountability and the affects that assessment triggers in them as further purposes of assessment.
  
  
  (show less)
  
  
• Assessment for learning in the accountability era: Empirical evidence from Scotland
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Carolyn Hutchinson, Myra Young
  In many European languages, the words ?assessment? and ?evaluation? are synonymous,... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Carolyn Hutchinson, Myra Young
  In many European languages, the words ?assessment? and ?evaluation? are synonymous, but in English the two terms are used differently. Educational ?assessment? is learner-focused and part of professional pedagogy, relating directly to the accountability of practitioners and schools to learners and their families. Educational ?evaluation? is understood to be a parallel and linked process, relating to the quality of national policy and of provision and practice in establishments, part of accountability to the wider public. In Scotland, assessment policy has been focused on aligning ?assessment for learning? and ?assessment for accountability? for nearly a decade and policy guidance has been based on the principle that formative and summative uses of information can apply as well to system evaluation as to school-based assessment. Self-evaluation has been increasingly emphasized, promoting better understanding of the role of continuous review in supporting and improving education. In theory, the two processes of assessment and evaluation should align without difficulty as ?intelligent accountability? and together should act as a powerful driver for improved educational outcomes. However, achieving alignment in practice, which requires good professional relationships and mutual trust in different communities of practice, as well as shared understanding and expectations, has proven to be very hard to achieve. This article will show the progress that has been made with alignment; indicate where there have been difficulties and why; and suggest that several important factors need to be acknowledged if alignment is to be achieved.
  
  
  (show less)
  
  
• Assessment for learning in the accountability era: New Zealand
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Terry Crooks
  The requirements for valid assessment for learning and assessment of learning are reviewed and then th... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Terry Crooks
  The requirements for valid assessment for learning and assessment of learning are reviewed and then the tensions between these two purposes of assessment are explored, first in general and then within two areas of the New Zealand education system. First, the nature of assessment for learning in primary schools is discussed and how it is, or might be, affected by four accountability pressures associated with assessment of learning. The second area is assessment for qualifications in the final years of secondary education. Here, some of the assessment of learning occurs at local level, within schools, and the remainder occurs at national level. These two assessment pathways create substantially different conditions affecting assessment for learning.
  
  
  (show less)
  
  
• Assessment for learning in the accountability era: Queensland, Australia
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Val Klenowski
  Developments in school education in Australia over the past decade have witnessed the rise of nationa... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Val Klenowski
  Developments in school education in Australia over the past decade have witnessed the rise of national efforts to reform curriculum, assessment and reporting. Constitutionally the power to decide on curriculum matters still resides with the States. Higher stakes in assessment, brought about by national testing and international comparative analyses of student achievement data, have challenged State efforts to maintain the emphasis on assessment to promote learning while fulfilling accountability demands. In this article lessons from the Queensland experience indicate that it is important to build teachers? assessment capacity and their assessment literacy for the promotion of student learning. It is argued that teacher assessment can be a source of dependable results through moderation practice. The Queensland Studies Authority has recognised and supported the development of teacher assessment and moderation practice in the context of standards-driven, national reform. Recent research findings explain how the focus on learning can be maintained by avoiding an over-interpretation of test results in terms of innate ability and limitations and by encouraging teachers to adopt more tailored diagnosis of assessment data to address equity through a focus on achievement for all. Such efforts are challenged as political pressures related to the Australian government's implementation of national testing and national partnership funding arrangements tied to the performance of students at or below minimum standards become increasingly apparent.
  
  
  (show less)
  
  
• Developing a TPCK-SRL assessment scheme for conceptually advancing technology in education
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2012
  Source:Studies In Educational Evaluation, Volume 38, Issue 1
  Zehavit Kohen, Bracha Kramarski
  The present study aimed to: (a) develop a conceptual TPCK-SRL scheme for assessing... (show all)
  Publication year: 2012
  Source:Studies In Educational Evaluation, Volume 38, Issue 1
  Zehavit Kohen, Bracha Kramarski
  The present study aimed to: (a) develop a conceptual TPCK-SRL scheme for assessing teachers? integration of self-regulated learning (SRL) considerations while infusing technology into a TPCK classroom context (blending K=knowledge about T=technology, P=pedagogy, and C=content), which reflects all three knowledge components? dynamic interactions with SRL and (b) test this scheme's validity and reliability as a practical tool for measuring effects of teacher education. The scheme was used to assess 9 preservice teachers? lesson designs, collected before and after a course on TPCK Teaching and Learning Methods. At posttest, lesson designs revealed specific, qualitative TPCK-SRL descriptions that referenced TPCK components and SRL considerations (what, how, when, and why to infuse technology), thus providing content validity for the scheme. Theoretical, methodological, and practical implications and future research are discussed for using the TPCK-SRL scheme to empower teachers? technological professionalism.
  Highlights ? A TPCK-SRL assessment scheme was developed. ? Integrating technology, pedagogy, content knowledge with self-regulated learning. ? The scheme was validated on preservice teachers? lesson designs. ? Mixed methods were used for content validity for the scheme.
  
  (show less)
  
  
• Nested contexts that shape assessment for learning: School-based professional learning community and classroom culture
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Menucha Birenbaum, Helena Kimron, Hany Shilton
  The study investigated the relationships between assessment for le... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Menucha Birenbaum, Helena Kimron, Hany Shilton
  The study investigated the relationships between assessment for learning (AfL) and attributes of two school-related contexts ? the classroom assessment culture (CAC) in which AfL is embedded, and the larger context in which CAC is nested, namely the school-based professional learning community (SBPLC). The research design comprised two layers ? quantitative and qualitative; the former, using rating-response questionnaires, investigated the association between attributes of SBPLC and AfL in a sample of in-service teachers. The latter, a multiple case study, examined AfL practices of six elementary school teachers, the assessment culture in their classrooms, and characteristics of their SBPLCs. Taken together, the findings of both layers pointed to meaningful relationships among AfL, CAC, and SBPLC highlighting similarities between SBPLC and CAC with regard to cultural attributes and to AfL/inquiry practice. Contextual attribute patterns conducive to proper implementation of AfL and those that impede it were discerned and discussed.
  
  
  (show less)
  
  
• Formative assessment and feedback: Making learning visible
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2012
  Source:Studies In Educational Evaluation, Volume 38, Issue 1
  Anton Havnes, Kari Smith, Olga Dysthe, Kristine Ludvigsen
  The study explores how assessment information is recei... (show all)
  Publication year: 2012
  Source:Studies In Educational Evaluation, Volume 38, Issue 1
  Anton Havnes, Kari Smith, Olga Dysthe, Kristine Ludvigsen
  The study explores how assessment information is received and attended to. The research is linked to a 2-year intervention project involving six Norwegian upper secondary schools, and with a particular focus on vocational training and the three core subjects: English, Norwegian and Mathematics. Survey data was collected from five schools, including both vocationally and academically oriented education. Other sources of data are focus-group interviews in three of the five schools, involving students, teachers and school leaders. Findings show that there are significant differences in how students and teachers perceive feedback practices. There are also significant differences between boys and girls, as well as within the various school subjects. Students experience more feedback in vocational training than in the more traditional academic subjects.
  Highlights ? Teachers rate the quality of their feedback higher than students do. ? Feedback is linked to grading of completed assignments, not to work in progress. ? There is more feedback in vocational training than in academic subjects. ? Classroom-teaching and learning-activities are potentially rich on feedback. ? Interactional and learning situations need to be exploited as feedback settings.
  
  (show less)
  
  
• Do central examinations lead to greater grading comparability? A study of frame-of-reference effects on the University entrance qualification in Germany
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 4
  Marko Neumann, Ulrich Trautwein, Gabriel Nagy
  In Germany, the Abitur grades awarded at the end of upper secondary... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 4
  Marko Neumann, Ulrich Trautwein, Gabriel Nagy
  In Germany, the Abitur grades awarded at the end of upper secondary education are critical in the allocation of sought-after university places. Drawing on a representative sample of 3526 grade 13 Abitur students in the German state of Baden-Württemberg, this article examines whether and to what extent grading is affected by the mean achievement of the school serving as frame of reference (?group-referenced grading?), and to what extent this influence differs for coursework and examination grades in mathematics and English as a foreign language. Overall, the results indicate that the higher level of standardization of the central Abitur examinations makes examination grades less susceptible to frame-of-reference effects than are coursework grades.
  Highlights ? The comparability of teacher-assigned grades is thought to be limited. ? There is evidence for reference group effects in grading. ? Grades from central exams are less susceptible for such frame of reference effects.
  
  (show less)
  
  
• Video portfolios The development and usefulness of a teacher assessment procedure
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issues 2?3
  Mirjam E.J. Bakker, Erik C. Roelofs, Douwe Beijaard, Piet F. Sanders, Dineke E.H. Tigelaar, Nico Verloop
  Th... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issues 2?3
  Mirjam E.J. Bakker, Erik C. Roelofs, Douwe Beijaard, Piet F. Sanders, Dineke E.H. Tigelaar, Nico Verloop
  This study reports on the development and usefulness of an assessment procedure for teachers? coaching competence. We examined the usefulness of the developed procedure with respect to making reliable judgments. Video portfolios were constructed by researchers, which consisted of deliberately planned video recordings of teachers? coaching performance in the classroom and additional data sources providing context information. Six trained assessors scored three video portfolios. Interrater agreement was determined and assessors were interviewed. Judgments across assessors were relatively similar. Both supportive and hindering aspects in the procedure for making reliable judgments could be derived from the interviews. Theoretical insights about assessor scoring are discussed, and implications for optimizing the assessment procedure are described.
  
  
  (show less)
  
  
• Informal formative assessment: The role of instructional dialogues in assessing students? learning
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Maria Araceli Ruiz-Primo
  This paper focuses on an unceremonious type of formative assessment ? informal formative a... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Maria Araceli Ruiz-Primo
  This paper focuses on an unceremonious type of formative assessment ? informal formative assessment ? in which much of what teachers and students do in the classroom can be described as potential assessments that can provide evidence about the students? level of understanding. More specifically, the paper focuses on assessment conversations, or dialogic interactions or exchanges, which continuously happen in the classroom and that are at the center of informal formative assessment. It is argued that assessment conversations make students? thinking explicit in an unobtrusive manner, and when students? thinking is explicit, it can be examined, questioned, and shaped as an active object of constructive learning. The paper conceptualizes informal formative assessment at the center of effective instructional activities with the use of instructional dialogues as assessment conversations, a typical informal formative assessment practice. The paper then presents a discussion about the evidence on the effect of assessment conversations on student learning.
  
  
  (show less)
  
  
• What is assessment for learning?
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Dylan Wiliam
  The idea that assessment is intrinsic to effective instruction is traced from early experiments in the... (show all)
  Publication year: 2011
  Source:Studies In Educational Evaluation, Volume 37, Issue 1
  Dylan Wiliam
  The idea that assessment is intrinsic to effective instruction is traced from early experiments in the individualization of learning through the work of Benjamin Bloom to reviews of the impact of feedback on learners in classrooms. While many of these reviews detailed the adverse impact of assessment on learning, they also indicated that under certain conditions assessment had considerable potential to enhance learning. It is shown that understanding the impact that assessment has on learning requires a broader focus than the feedback intervention itself, particularly the learner's responses to the feedback, and the learning milieu in which the feedback operates. Different definitions of the terms ?formative assessment? and ?assessment for learning? are discussed, and subsumed within a broad definition that focuses on the extent to which instructional decisions are supported by evidence. The paper concludes by exploring some of the consequences of this definition for classroom practice.
  
  
  (show less)
  
  
• Using systemic functional linguistics in academic writing development: An example from film studies
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2012
  Source:Journal of English for Academic Purposes, Volume 11, Issue 1
  James P. Donohue
  On film studies courses, students are asked to treat as objects of study the same films whi... (show all)
  Publication year: 2012
  Source:Journal of English for Academic Purposes, Volume 11, Issue 1
  James P. Donohue
  On film studies courses, students are asked to treat as objects of study the same films which they may more commonly experience as entertainment. To explore the role of academic writing in this, an action research project was carried out on a university film studies course using a systemic functional linguistics approach. This paper presents a key assessment essay genre, referred to as a taxonomic film analysis. This genre was analysed drawing on the work of Halliday and Mathiessen (2004), Martin (1992) and Lemke, (1985, 1990), focussing on three aspects: the genre acts performed in the process of analysing film; the conceptual frameworks of film studies knowledge, or ?thematic formations? (Lemke, 1993) drawn on and re-constituted in the assignment; the particular ways that language is used to perform these acts and build these thematic formations. For EAP to be relevant to film students, it is proposed that EAP specialists need to engage with these three aspects of film study. This application of SFL in film studies EAP is intended as an illustration of how SFL tools can be used for relevant EAP provision across the HE curriculum.
  Highlights ? The key genre identified was taxonomic film analysis. ? Deploying film studies language and genre converts film into an object of study. ? EAP lecturers need to engage with both the language and the meaning making of film students.
  
  (show less)
  
  
• Examiners? reports on theses: Feedback or assessment?
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:Journal of English for Academic Purposes, Volume 10, Issue 4
  Vijay Kumar, Elke Stracke
  Traditionally, examiners? reports on theses at the doctoral and Master?s level co... (show all)
  Publication year: 2011
  Source:Journal of English for Academic Purposes, Volume 10, Issue 4
  Vijay Kumar, Elke Stracke
  Traditionally, examiners? reports on theses at the doctoral and Master?s level consist of two components: firstly, summative assessment where a judgement is made about whether the thesis has met the standards established by the discipline for the award of the degree, and, secondly, the developmental and formative component, where examiners provide feedback to assist the candidate to revise the thesis. Given this dual task of providing assessment and feedback, this paper presents the findings of a small-scale empirical study that aimed to gain insights into the connection or potential disjunction between feedback and assessment in six examiners? reports. The main aim of this study was to identify the nature of examiners? reports on Master?s and doctoral theses: is it primarily assessment or feedback? Our study suggests the crucial role of feedback in postgraduate thesis examination practice. Without feedback, there is little impetus for the candidate to progress, to close the gap between current and desired performance, and to attain the level needed to become a member of the scholarly community. The study concludes with the implications that a stronger focus on feedback might have for all stakeholders involved in the thesis examination process.
  Highlights ? We examine the connection between feedback and assessment in examiners? reports. ? We suggest the crucial role of feedback in postgraduate thesis examination practice. ? Feedback will close the gap between the candidate?s current and desired performance. ? Emphasis on feedback affects examiners, the university, supervisors and candidates.
  
  (show less)
  
  
• The influence of learner strategies on oral presentations: A comparison between group and individual performance
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:English for Specific Purposes, Volume 30, Issue 4
  Mu-hsuan Chou
  Cooperative learning has frequently been used in language classrooms, from in-class task-based group work... (show all)
  Publication year: 2011
  Source:English for Specific Purposes, Volume 30, Issue 4
  Mu-hsuan Chou
  Cooperative learning has frequently been used in language classrooms, from in-class task-based group work to group presentations. Research suggests that cooperative learning provides mutual support, as well as successful and effective learning outcomes of tasks. The present research addressed a number of problems discovered in group oral presentations in an EAP context where cooperative learning is the core of the course. As learning is strongly related to strategies or approaches adopted to facilitate one?s comprehension and production of the target language, the purposes of the present study were to investigate the use and influence of learner strategies in cooperative and individual learning, and the benefits of cooperative learning in improving students? English speaking ability. Fifty-two, third-year French major college students taking the Professional English Course at a college in Taiwan participated in the study. All participants were required to give oral presentations on the topics taught in class. Research methods included questionnaires, interviews, and oral assessments. The findings showed that, first, certain learner strategies had positive impacts on the students? language performance in the two types of presentation and second, that group oral presentation had the greatest influence on improving students? speaking ability, while other forms of cooperative learning activities enhanced their motivation for learning.
  Highlights ? The participants deployed metacognitive, retrieval, and rehearsal strategies more frequently in individual than group presentations. ? Communicative strategies were used more often in group presentations. ? Learner strategies had positive impacts on students? fluency in group presentations, but on organization in individual presentations. ? Cooperative activity involving in-class discussion was considered motivating, while it with oral presentations was regarded as beneficial to increase of speaking ability. ? The participants preferred individual presentations to groups for developing independent thinking, improving organizational skills and English ability.
  
  (show less)
  
  
• A corpus-based approach to online materials development for writing research articles
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:English for Specific Purposes, Volume 30, Issue 3
  Ching-Fen Chang, Chih-Hua Kuo
  There has been increasing interest in the possible applications of corpora to both lingu... (show all)
  Publication year: 2011
  Source:English for Specific Purposes, Volume 30, Issue 3
  Ching-Fen Chang, Chih-Hua Kuo
  There has been increasing interest in the possible applications of corpora to both linguistic research and pedagogy. This study takes a corpus-based, genre-analytic approach to discipline-specific materials development. Combining corpus analysis with genre analysis makes it possible to develop teaching materials that are not only authentic but also research-supported. An RA corpus consisting of 60 research articles from three major journals of computer science was constructed. A word frequency list derived from the corpus was analyzed to develop a vocabulary profile for the genre. Move analysis was also conducted based on a self-developed coding scheme of rhetorical moves in the target genre. The move codes were tagged in the corpus texts so that individual moves and move patterns could be retrieved for the purpose of developing research-based online teaching materials for graduate students of computer science. Numerous examples of specialized vocabulary, grammatical usage, and move structures that showcase and characterize academic computer science discourse were used in not only lessons but also learning tasks, discussion topics, and online writing models. The paper ends with a discussion of the usefulness and effectiveness of the online RA writing materials, based on student feedback and assessments.
  Highlights ? We combine corpus-based with genre-analytic approach to develop research-based online courseware for graduate students of computer science. ? We build a corpus of 60 research articles. ? We analyze word frequency and rhetorical moves in each section from the corpus. ? The online courseware includes examples of specialized vocabulary, grammatical usages, move structures, and learning tasks. ? The usefulness and effectiveness of the online courseware are evaluated by student feedback and assessments.
  
  (show less)
  
  
• Ways of communicating emotional stance in online disagreements
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2012
  Source:Journal of Pragmatics
  Andreas Langlotz, Miriam A. Locher
  Online disagreements constitute a particularly interesting and relevant testing ground to explore different way... (show all)
  Publication year: 2012
  Source:Journal of Pragmatics
  Andreas Langlotz, Miriam A. Locher
  Online disagreements constitute a particularly interesting and relevant testing ground to explore different ways of communicating emotional stance (Mateosian, 2005). Our qualitative and quantitative analysis of 120 English postings from the MailOnline has revealed the notable presence of emotional stance through conceptual implication, explicit expression, and emotional description. While this quantitative survey can neither be regarded as comprehensive nor conclusive, we consider it as a first step towards detecting and categorizing different ways of expressing emotion in online and offline linguistic data. We suggest that a quantitative survey is complemented with a qualitative discussion to account for the complex and dynamic interaction between conceptual, relational and affective meaning. This speaks for a discursive approach for studying emotional stance in conflict and disagreements.
  
  
  (show less)
  
  
• The effect of known-and-unknown word combinations on intentional vocabulary learning
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2011
  Source:System, Volume 39, Issue 4
  Kiwamu Kasahara
  The purpose of this study is to examine whether learning a known-and-unknown word combination is superior in terms of retentio... (show all)
  Publication year: 2011
  Source:System, Volume 39, Issue 4
  Kiwamu Kasahara
  The purpose of this study is to examine whether learning a known-and-unknown word combination is superior in terms of retention and retrieval of meaning to learning a single unknown word. The term ?combination? in this study means a two-word collocation of a familiar word and a word that is new to the participants. Following the results of Kasahara (2010), the study predicted that attaching a known word to a word to be remembered could help learners to retain and retrieve the meaning of the target word. Sixty-six university students took part in the experiment. They were divided into two groups with the same vocabulary size. One group was told to remember the 20 target words; the other was told to remember the 20 two-word collocations (known words + the target words). Each group was asked to define the meaning of the target words or the collocations respectively in immediate and delayed tests. The results of both tests revealed that the collocations allowed significantly better retention and retrieval of the meanings than the single target words.
  
  
  (show less)
  
  
• The effect of mechanical and meaningful production of output on learning English relative clauses
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2012
  Source:System, Volume 40, Issue 1
  Shirin Abadikhah
  The present study examined the effectiveness of mechanical and meaningful production of output on the learning of English rel... (show all)
  Publication year: 2012
  Source:System, Volume 40, Issue 1
  Shirin Abadikhah
  The present study examined the effectiveness of mechanical and meaningful production of output on the learning of English relative clauses by Iranian EFL learners. The study involved two groups of learners (N = 36) who completed a pre-test, three activities and a post-test over an 8-week period. The collaborative dialogue of the participants was tape-recorded and transcribed. The results of the tests showed that both groups made significant progress from pre-test to post-test and they did not differ significantly in their gains on the target linguistic forms, suggesting that both contexts contributed to the learning of English relative clauses. Findings from the tape-recorded data, however, indicated that there was a significant relationship between activity type and number of language-related episodes (LREs): the more meaningful activities elicited significantly more LREs compared to the more mechanical activities. Furthermore, there was a significant relationship between the gains of learners who scored below the average on the pre-test and the number of LREs they produced.
  
  
  (show less)
  
  
• What about me? Individual self-assessment by skill and level of language instruction
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2012
  Source:System, Volume 40, Issue 1
  Cindy Brantmeier, Robert Vanderplank, Michael Strube
  In an investigation with advanced language learners, Brantmeier [Brantmeier, C., 2006. ... (show all)
  Publication year: 2012
  Source:System, Volume 40, Issue 1
  Cindy Brantmeier, Robert Vanderplank, Michael Strube
  In an investigation with advanced language learners, Brantmeier [Brantmeier, C., 2006. Advanced L2 learners and reading placement: self-assessment, computer based testing, and subsequent performance. System 34 (1), 15?35.] reports that self-assessment (SA) of second language (L2) reading ability, when measured with self-rated scales, is not an accurate predictor of subsequent reading performance as measured via multiple choice items. In another experiment with advanced learners that utilizes criterion-referenced SA items, Brantmeier and Vanderplank [Brantmeier, C., Vanderplank, R., 2008. Descriptive and criterion-referenced self assessment with L2 readers. System 36 (3), 456?477] reveal that learners accurately estimate their reading comprehension when it is measured via multiple choice items. For the present study, an SA instrument of language learning achievement was designed according to specific course content to take into consideration the direct experience learners have had in practicing reading, listening, speaking, and writing [Ross, S., 1998. Self-assessment in second language testing: a meta-analysis of experimental factors. Language Testing 15, 1?20.]. With 276 participants, the study examines skill-based SA across beginning, intermediate and advanced levels of language instruction, and it offers evidence to validate the relationship between the SA instrument and achievement on an online abilities test with advanced learners. Findings hold important implications for language learner assessment, especially in terms of advanced students? ability to rate themselves when given specific criteria. A discussion about the value of SA as a complement to other traditional approaches for language program assessment is offered.
  
  
  (show less)
  
  
• The acquisition of the copula be in present simple tense in English by native speakers of Russian
  Posted on 18 May 2012 at 1:36 am
  
  Publication year: 2012
  Source:System
  Elena Antonova Unlu, Çiler Hatipoglu
  The current research investigated the acquisition of the copula be in Present Simple Tense (PST) in English by native speake... (show all)
  Publication year: 2012
  Source:System
  Elena Antonova Unlu, Çiler Hatipoglu
  The current research investigated the acquisition of the copula be in Present Simple Tense (PST) in English by native speakers of Russian. The aim of the study was to determine whether or not Russian students with different levels of English proficiency would encounter any problems while using the copula be in PST in English. The study also identified domains related to the use of the copula be that appear to be problematic for native speakers of Russian. To carry out the research, two diagnostic tests and a taxonomy of the domains of the use of the copula be were developed. The data were collected from three groups of Russian students who were in their first, fourth and eighth years of learning English. Quantitative and qualitative analyses were used in the study. The findings of the study revealed that some of the errors, such as omission or misuse of copula forms, disappeared with further exposure to the language. However, some of the errors were fossilised and were observed in the Russian students' performance even after eight years of learning English. Negative transfer at the morphological level and incomplete understanding and application of the copula be rule were suggested to be the underlying reasons for the fossilised errors made by the Russian learners.
  
  
  (show less)
  
  
• LiyingChengAndyCurtisEnglish Language Assessment and the Chinese Learner2010RoutledgeNew York and Londonxvii+ 282 pp
  Posted on 18 May 2012 at 1:36 am
  Publication year: 2012
  Source:System
  Zhi Li
  
  
  
  
  
• Research agenda: Priorities for future research in second language assessment
  Discussion
  Stephen Stoynoff,
  Language Teaching, Volume 45 Issue 02, pp 234-249
  
  Abstract
  
  
• Questioning linguistic instrumentalism: English, neoliberalism, and language tests in Japan
  Posted on 18 May 2012 at 1:30 am
  
  Publication year: 2011
  Source:Linguistics and Education, Volume 22, Issue 3
  Ryuko Kubota
  Linguistic instrumentalism, which underscores the importance of English skills for work and for achieving indi... (show all)
  Publication year: 2011
  Source:Linguistics and Education, Volume 22, Issue 3
  Ryuko Kubota
  Linguistic instrumentalism, which underscores the importance of English skills for work and for achieving individual economic success, has influenced language education policies and proliferated the language teaching and testing industry in Japan. Linguistic instrumentalism is linked to the notion of human capital (i.e., skills deemed necessary for the knowledge economy) and the unstable employment conditions of neoliberal society. Focusing on the role of learning and using English for work in Japan, this qualitative study investigates the aspirations and experiences of adult workers learning English and the views of managers of manufacturing companies about the role of English and English tests. In-depth interviews revealed that the learners? aspirations and work experiences, and the managers? views do not always confirm linguistic instrumentalism. Rather, career successes are influenced by gender, geography, and other factors. The findings indicate that language tests actually create the demand for learning English through measuring efforts to learn in the knowledge economy.
  Research highlights ? I examine why Japanese workers learn English through a qualitative study. ? Participants do not always engage in learning to seek economic returns. ? Nor did their experiences confirm the idea that English is essential for work. ? Language tests create demands for language learning and sort neoliberal subjects. ? Neoliberalism disadvantages women and marginalized workers.
  
  (show less)
  
  
• Assessment Issues in Languages for Specific Purposes
  Posted on 17 May 2012 at 2:47 pm
  by BARRY O'SULLIVAN
  
  While Grosse and Voght (1991) set out a well-considered overview of LSP and identified areas in need of development, they limited their observations on the topic of assessment to a short section de... (show all)
  While Grosse and Voght (1991) set out a well-considered overview of LSP and identified areas in need of development, they limited their observations on the topic of assessment to a short section devoted to what they called the proficiency movement. While it is true that they really did not have a lot to report on at the time they wrote their review, and that this current review highlights much interesting work done in the area in the intervening years, there remains a considerable emphasis on practice rather than on theory, with published papers attempting to identify solutions to given assessment problems, rather than attempting to build cohesive assessment theories. The primary focus of this contribution is on the latter aspect of language assessment. The article first offers a historical overview of the area over the past twenty years, which moves from a brief discussion of the issues highlighted by Grosse and Voght to the theoretical issues that have emerged, and finishing with a critical review of research on issues around assessment in three specific domains (immigration and citizenship, or work and the professions). The article then highlights current needs and priorities, focusing on issues of test usage and introducing the concept of test localization before presenting the core of the argument: a theory of LSP assessment validation. It concludes with a preliminary attempt to exemplify how a theory-driven research agenda can inform future research, and ultimately, practice in the area of LSP assessment over the coming decades.
  (show less)
  
  
• Languages for Specific Purposes in the United States in a Global Context: Commentary on Grosse and Voght (1991) Revisited
  Posted on 17 May 2012 at 2:47 pm
  by BARBARA A. LAFFORD
  
  This introduction will provide a thematic overview of the major issues raised by Grosse and Voght (1991), ?The Evolution of Languages for Specific Purposes in the United States,? and the contributi... (show all)
  This introduction will provide a thematic overview of the major issues raised by Grosse and Voght (1991), ?The Evolution of Languages for Specific Purposes in the United States,? and the contributions in this Focus Issue that address those issues from a 2011 perspective. It will also explore contemporary (largely praxis-oriented) languages for specific purposes (LSP) in the United States within a global LSP context that has historically evidenced strong connections between theory and practice. Points of comparison between LSP practiced in the United States and abroad include the rationale for LSP, curricula/programs innovations, professional infrastructure, researchers/practitioners, theoretical bases for research, and research topics and outlets for publication. Then follows a discussion of the challenges to LSP research (e.g., establishing norms of various workplace interactions, data collection, creation of assessment tools, lack of LSP practitioners and researchers with second language acquisiton [SLA] training), and future directions for the growth of LSP as an interdiscipline in the United States and for SLA and foreign language pedagogical research that would utilize data from LSP classroom and experiential settings in both domestic and international contexts. The introduction concludes with a discussion of LSP and community service learning as preparation for the workforce and for lifelong language learning.
  (show less)
  
  
• Revisiting the vowels of Brunei English
  Posted on 17 May 2012 at 3:41 am
  by SALBRINA SHARBAWI
  
  ABSTRACT:  This paper presents findings of a phonetic investigation of the 11 monophthongal vowels produced by 18 speakers of Brunei English (BrunE). Data for the current study was also collected f... (show all)
  ABSTRACT:  This paper presents findings of a phonetic investigation of the 11 monophthongal vowels produced by 18 speakers of Brunei English (BrunE). Data for the current study was also collected from 12 Singapore English (SgE) speakers. In order to avoid a prescriptive approach, the description of the vowels was carried out using a modified inventory of vowel contrasts for the monophthongs following Wells? model of standard lexical sets. The findings revealed that BrunE speakers are inconsistent in their vowel productions in which there is indeterminate usage of American English (AmE) and BrE vowels. The Singaporeans, however, show more uniformity across the subjects. It is proposed that the lack of consistency in BrunE vowel pronunciations may be explained in terms of the maturity of the English varieties. Whereas Schneider places SgE in an advanced phase 4 of his dynamic model of the evolution of postcolonial Englishes, the findings of the current study seems to indicate that BrunE belongs in the third phase. Statistical tests, however, do not seem to support the claim that BrunE and SgE are in different stages of development. It is also concluded that use of the lexical keywords may not be feasible in the description of an emerging English variety since the lexical sets are fixed and are based on the pronunciation of an Inner Circle variety. As such, an improvised lexical keyword set has been proposed for the description of BrunE vowels.
  (show less)
  
  
• Testing the Factorial Invariance of the English and Filipino Versions of the Inventory of School Motivation With Bilingual Students in the Philippines
  Posted on 16 May 2012 at 5:24 am
  by Ganotice, F. A., Bernardo, A. B. I., King, R. B.
  

  The study explored the invariance of Filipino and English versions of the Inventory of School Motivation (ISM) for Filipino-English bilingual students. There was invariance in the factor structure ...

  (show all)

  The study explored the invariance of Filipino and English versions of the Inventory of School Motivation (ISM) for Filipino-English bilingual students. There was invariance in the factor structure and factor loadings across the two language versions. Between-network construct validation showed consistent associations between ISM-mastery goals and sense-of-self dimensions.

  
  (show less)
  
  
• The LAOM Multidimensional Anxiety Scale for Measuring Anxiety in Children and Adolescents: Addressing the Psychometric Properties of the Scale
  Posted on 16 May 2012 at 5:24 am
  by Kozina, A.
  

  The article introduces a new anxiety scale, called the LAOM (Lestvica anksioznosti za otroke in mladostnike [The anxiety scale for children and adolescents]) for measuring self-reported multidimens...

  (show all)

  The article introduces a new anxiety scale, called the LAOM (Lestvica anksioznosti za otroke in mladostnike [The anxiety scale for children and adolescents]) for measuring self-reported multidimensional anxiety. The scale has been developed with a special focus on the school setting, using one sample from an elementary school which is representative of fourth- and eighth-grade students in Slovenia (N = 10,427) and two secondary samples from secondary schools (N = 1,406; N = 3,253) which are representative of final-year secondary school students in Slovenia. The exploratory (PCA) and confirmatory analyses (CFA) were performed on separate samples for the elementary and the secondary school students. The CFA confirmed that the items on the scale formed three factors which were related to the higher order factor. The structure was stable over different age groups. The scale demonstrated adequate internal consistency and sensitivity. Validity tests are presented in their preliminary forms and will be elaborated upon in future research.

  
  (show less)
  
  
• Development and Technical Characteristics of a Team Decision- Making Assessment Tool: Decision Observation, Recording, and Analysis (DORA)
  Posted on 16 May 2012 at 5:24 am
  by Algozzine, B., Newton, J. S., Horner, R. H., Todd, A. W., Algozzine, K.
  

  Problem solving is fundamental to psychoeducational assessment practices and generally grounded in activities related to identifying problems, developing and refining hypotheses, generating solutio...

  (show all)

  Problem solving is fundamental to psychoeducational assessment practices and generally grounded in activities related to identifying problems, developing and refining hypotheses, generating solutions, developing and implementing actions, and evaluating outcomes. While the process is central to response-to-intervention practices as well, little research has addressed the form, content, or outcomes of decision-making teams as they operate in schools. One barrier to building a program of research on team problem solving has been the absence of a credible and feasible measure of team performance. We developed the Decision Observation, Recording, and Analysis (DORA) tool to document problem-solving behaviors during team meetings. We were interested in evaluating problem solving during team meetings that focus on academic and behavior concerns in school. We describe the development and preliminary psychometric data for DORA in this article. Our discussion focuses on the implications of DORA for expanding the study of team processes and for improvement of problem-solving practices in schools.

  
  (show less)
  
  
• Constructing and utilizing wordnets using statistical methods
  Posted on 10 May 2012 at 1:07 am
  

  Abstract  Lexical databases following the wordnet paradigm capture information about words, word senses, and their relationships. A large number of existing tools and datasets are based on the ori...

  (show all)

  Abstract  Lexical databases following the wordnet paradigm capture information about words, word senses, and their relationships. A large number of existing tools and datasets are based on the original WordNet, so extending the landscape of resources aligned with WordNet leads to great potential for interoperability and to substantial synergies. Wordnets are being compiled for a considerable number of languages, however most have yet to reach a comparable level of coverage. We propose a method for automatically producing such resources for new languages based on WordNet, and analyse the implications of this approach both from a linguistic perspective as well as by considering natural language processing tasks. Our approach takes advantage of the original WordNet in conjunction with translation dictionaries. A small set of training associations is used to learn a statistical model for predicting associations between terms and senses. The associations are represented using a variety of scores that take into account structural properties as well as semantic relatedness and corpus frequency information. Although the resulting wordnets are imperfect in terms of their quality and coverage of language-specific phenomena, we show that they constitute a cheap and suitable alternative for many applications, both for monolingual tasks as well as for cross-lingual interoperability. Apart from analysing the resources directly, we conducted tests on semantic relatedness assessment and cross-lingual text classification with very promising results.

  Content Type Journal ArticleCategory Original PaperPages 1-25DOI 10.1007/s10579-012-9183-2Authors Gerard de Melo, Max Planck Institute for Informatics, Campus E1 4, 66123 Saarbrücken, GermanyGerhard Weikum, Max Planck Institute for Informatics, Campus E1 4, 66123 Saarbrücken, Germany Journal Language Resources and EvaluationOnline ISSN 1574-0218Print ISSN 1574-020X
  (show less)
  
  
• The need for speed: a critical discourse analysis of the reclassification of English language learners in Arizona
  Posted on 5 May 2012 at 1:01 am
  

  Abstract  Several states, including Arizona, have enacted English-only legislation, within the past decade, impacting the schooling of students who are identified as English language learner (ELLS...

  (show all)

  Abstract  Several states, including Arizona, have enacted English-only legislation, within the past decade, impacting the schooling of students who are identified as English language learner (ELLS). As a result, ELLS in Arizona are assigned to a prescriptive program?apart from their fluent English-speaking peers?for 4 h a day, during a time ?not normally to exceed 1 year.? The ultimate goal is to reclassify ELLs to fluent English proficient (FEP) status, exit them from the program and integrate students into ?mainstream? classrooms, where no additional support services are offered. Since language policies are dictating the instructional policy decisions regarding assessment and reclassification of ELLS, the authors argue that it is necessary to critically examine how English-only policies?especially the discourse of policy making?contribute to the shift in the definition of reclassification, its process and ultimately, the consequences for students. Thus, the paper unpacks the shifts in reclassification policy and process; our critical discourse analysis of the legislators? meetings, based on the work of van Leeuwen (Discourse Commun 1(1):91?112, 2007), show that although Arizona?s ELL Task Force set out to develop an educational policy to prepare ELLS linguistically and academically within one school year, the timeframe (or time limit) took precedence to the academic preparedness of newly reclassified students?a population that is academically vulnerable.

  Content Type Journal ArticleCategory Original PaperPages 1-18DOI 10.1007/s10993-012-9242-yAuthors Alisa G. Leckie, University of Arizona, PO Box 210069, Tucson, AZ 85721-0069, USASuzanne E. Kaplan, University of Arizona, PO Box 210069, Tucson, AZ 85721-0069, USAEliane Rubinstein-Ávila, University of Arizona, PO Box 210069, Tucson, AZ 85721-0069, USA Journal Language PolicyOnline ISSN 1573-1863Print ISSN 1568-4555
  (show less)
  
  
• Second Language Listening and Unfamiliar Proper Names: Comprehension Barrier?
  Posted on 1 May 2012 at 4:09 am
  by Kobeleva, P. P.
  

  This study examines whether unfamiliar proper names affect English as a second language (ESL) learners’ listening comprehension. A total of 110 intermediate to advanced ESL learners participated; c...

  (show all)

  This study examines whether unfamiliar proper names affect English as a second language (ESL) learners’ listening comprehension. A total of 110 intermediate to advanced ESL learners participated; comprehension of a short news text was tested under two conditions, Names Known (all proper names pre-taught in advance) and Names Unknown (all proper names unfamiliar prior to listening). The Names Known group performed significantly better than the Names Unknown group on two measures, true-false-don’t know statements and open-ended questions. The effect of unfamiliar proper names on comprehension remained statistically significant after the listening proficiency variable was controlled. Listeners in the Names Unknown treatment were able to derive 44.8% of information about the referents of proper names from context but they could not identify all the referents successfully. Unfamiliar proper names evidently hinder comprehension, which was also supported by the participants’ task difficulty ratings. When proper names were familiar, participants rated all comprehension tasks as easier to do and self-reported higher comprehension success. In contrast, when proper names were unfamiliar, participants rated all comprehension tasks as harder to do and self-reported a lower amount of comprehension.

  It has long been established that vocabulary knowledge is a prerequisite to successful comprehension. This study confirms that, similarly to common vocabulary, the presence of unfamiliar proper names decreases the ESL learners’ chances of understanding spoken English. Implications of the findings for second language learning and testing are discussed.

  It is recommended that teachers be flexible when setting up listening exercises and use different techniques depending on the depth of understanding the learners are supposed to achieve. If the instructional goal is listening for gist, proper names will unlikely affect the outcomes of the activity. However, if detailed and precise understanding is required, then it is unrealistic to expect ESL listeners to cope with unfamiliar proper names ‘on the spot’ without assistance or preparation. It is also advisable to avoid unfamiliar proper names in a testing situation because they may cause unwarranted anxiety and reflect negatively on the ESL listeners’ performance.

  
  (show less)
  
  
• The SAS Education Value-Added Assessment System (SAS® EVAAS®) in the Houston Independent School District (HISD): Intended and Unintended Consequences
  Posted on 29 Apr 2012 at 7:00 pm
  by Audrey Amrein-Beardsley, Clarin Collins
  

  The SAS Educational Value-Added Assessment System (SAS® EVAAS®) is the most widely used value-added system in the country. It is also self-proclaimed as ?the most robust and reliable? system availa...

  (show all)

  The SAS Educational Value-Added Assessment System (SAS® EVAAS®) is the most widely used value-added system in the country. It is also self-proclaimed as ?the most robust and reliable? system available, with its greatest benefit to help educators improve their teaching practices. This study critically examined the effects of SAS® EVAAS® as experienced by teachers, in one of the largest, high-needs urban school districts in the nation ? the Houston Independent School District (HISD). Using a multiple methods approach, this study critically analyzed retrospective quantitative and qualitative data to better comprehend and understand the evidence collected from four teachers whose contracts were not renewed in the summer of 2011, in part given their low SAS® EVAAS® scores. This study also suggests some intended and unintended effects that seem to be occurring as a result of SAS® EVAAS® implementation in HISD. In addition to issues with reliability, bias, teacher attribution, and validity, high-stakes use of SAS® EVAAS® in this district seems to be exacerbating unintended effects.

   

  
  (show less)
  
  
• The Effects of Sentence Writing on Second Language French and Korean Lexical Retention
  Posted on 27 Apr 2012 at 6:01 pm
  

  This study investigates the effects of sentence writing (SW) on second language (L2) lexical retention by comparing two word-learning conditions: writing new words in sentences and repeated word-p...

  (show all)

  This study investigates the effects of sentence writing (SW) on second language (L2) lexical retention by comparing two word-learning conditions: writing new words in sentences and repeated word-picture viewing. L2 learners of French and Korean attempted to learn 24 new words: 12 words with one condition and 12 words with another condition. Dependent measures were one immediate and two delayed posttests that required participants to produce target word forms. Results for both language groups revealed negative effects for SW, suggesting that SW can impede word-form learning during the initial stages of L2 vocabulary learning. Furthermore, the finding that the Korean learners' scores were much lower than the French learners' scores under the SW condition suggests that SW may result in even less retention when the L2 script is far more distant from one's first language (L1), thereby supporting the impact of L1?L2 orthographic distance on L2 word learning and retention.

  Content Type Journal ArticleCategory ArticlePages 164-189DOI 10.3138/cmlr.68.2.164Authors Wynne WongDanielle O. Pyun Journal Canadian Modern Language Review/ La Revue canadienne des langues vivantesOnline ISSN 1710-1131Print ISSN 0008-4506 Journal Volume Volume 68 Journal Issue Volume 68, Number 2 / May 2012
  (show less)
  
  
• A Stochastic Method for Balancing Item Exposure Rates in Computerized Classification Tests
  Posted on 25 Apr 2012 at 10:54 am
  by Huebner, A., Li, Z.
  

  Computerized classification tests (CCTs) classify examinees into categories such as pass/fail, master/nonmaster, and so on. This article proposes the use of stochastic methods from sequential analy...

  (show all)

  Computerized classification tests (CCTs) classify examinees into categories such as pass/fail, master/nonmaster, and so on. This article proposes the use of stochastic methods from sequential analysis to address item overexposure, a practical concern in operational CCTs. Item overexposure is traditionally dealt with in CCTs by the Sympson-Hetter (SH) method, but this method is unable to restrict the exposure of the most informative items to the desired level. The authors’ new method of stochastic item exposure balance (SIEB) works in conjunction with the SH method and is shown to greatly reduce the number of overexposed items in a pool and improve overall exposure balance while maintaining classification accuracy comparable with using the SH method alone. The method is demonstrated using a simulation study.

  
  (show less)
  
  
• Dynamic Problem Solving: A New Assessment Perspective
  Posted on 25 Apr 2012 at 10:54 am
  by Greiff, S., Wustenberg, S., Funke, J.
  

  This article addresses two unsolved measurement issues in dynamic problem solving (DPS) research: (a) unsystematic construction of DPS tests making a comparison of results obtained in different stu...

  (show all)

  This article addresses two unsolved measurement issues in dynamic problem solving (DPS) research: (a) unsystematic construction of DPS tests making a comparison of results obtained in different studies difficult and (b) use of time-intensive single tasks leading to severe reliability problems. To solve these issues, the MicroDYN approach is presented, which combines (a) the formal framework of linear structural equation models as a systematic way to construct tasks with (b) multiple and independent tasks to increase reliability. Results indicated that the assumed measurement model that comprised three dimensions, information retrieval, model building, and forecasting, fitted the data well (n = 114 students) and could be replicated in another sample (n = 140), showing excellent reliability estimates for all dimensions. Predictive validity of school grades was excellent for model building but nonexistent for the other two MicroDYN dimensions and for an additional measure of DPS. Implications are discussed.

  
  (show less)
  
  
• Proficiency Level--a Fuzzy Variable in Computer Learner Corpora
  Posted on 24 Apr 2012 at 2:23 am
  by Carlsen, C.
  

  This article focuses on the proficiency level of texts in Computer Learner Corpora (CLCs). A claim is made that proficiency levels are often poorly defined in CLC design, and that the methods used ...

  (show all)

  This article focuses on the proficiency level of texts in Computer Learner Corpora (CLCs). A claim is made that proficiency levels are often poorly defined in CLC design, and that the methods used for level assignment of corpus texts are not always adequate. Proficiency level can therefore, best be described as a fuzzy variable in CLCs, representing a potential source of error in CLC-based research. The article starts with a review of some of the most commonly used methods of proficiency-level assignment of texts in CLCs and a discussion of strengths and weaknesses of the different methods. A pioneer project to link a learner corpus of Norwegian (ASK) to the Common European Framework of Reference for Languages (CEFR) is presented to illustrate that a reliable assignment of corpus texts is viable by applying insights and practice from the professional field of language testing and assessment. Finally, some advantages of a learner corpus reliably linked to the CEFR are discussed in relation to SLA research, to language test development, and to a validation of the CEFR-level descriptions.

  
  (show less)
  
  
• Development and validation of Extract the Base: An English Derivational Morphology Test for third through fifth grade monolingual students and Spanish-speaking English language learners
  Posted on 23 Apr 2012 at 6:42 am
  by Goodwin, A. P., Huggins, A. C., Carlo, M., Malabonga, V., Kenyon, D., Louguit, M., August, D.
  

  This study describes the development and validation of the Extract the Base test (ETB), which assesses derivational morphological awareness. Scores on this test were validated for 580 monolingual s...

  (show all)

  This study describes the development and validation of the Extract the Base test (ETB), which assesses derivational morphological awareness. Scores on this test were validated for 580 monolingual students and 373 Spanish-speaking English language learners (ELLs) in third through fifth grade. As part of the validation of the internal structure, which involved using the Generalized Partial Credit Model for tests with polytomous items, items on this test were shown to provide information about students of different abilities and also discriminate amongst such heterogeneous students. As part of the validation of the test’s relationship to criterion, items were shown to correlate with measures of word identification, reading comprehension, and vocabulary measures. Differences in performances for fluent English students and ELLs, students of varied home language environments, and different grade levels were noted. Additionally, the task was validated using a dichotomous scoring system to provide reliability and validity information using this alternate scoring method.

  
  (show less)
  
  
• The contribution of test-takers' speech content to scores on an English oral proficiency test
  Posted on 23 Apr 2012 at 6:42 am
  by Sato, T.
  

  The content that test-takers attempt to convey is not always included in the construct definition of general English oral proficiency tests, although some English-for-academic-purposes (EAP) speaki...

  (show all)

  The content that test-takers attempt to convey is not always included in the construct definition of general English oral proficiency tests, although some English-for-academic-purposes (EAP) speaking tests and most writing tests tend to place great emphasis on the evaluation of the content or ideas in the performance. This study investigated the relative contribution of linguistic criteria and the elaboration of speech content to scores on a test of speaking proficiency. A speaking test was designed and administered to Japanese undergraduates to determine what criteria English teachers associate with general oral proficiency. Nine raters were recruited to rate 30 students’ monologues on three topics, using intuitive judgments of oral proficiency (referred to as Overall communicative effectiveness). Following this, they assigned scores to the monologues using five criteria: Grammatical accuracy, Fluency, Vocabulary range, Pronunciation, and Content elaboration/development. The raters were also asked to provide open-ended written comments on the factors contributing to their intuitive judgments. Statistical analyses of the scores – Rasch measurement, multiple regression, and multivariate generalizability (G) theory analysis – revealed that Content elaboration/development made a substantive contribution to the intuitive judgments and composite score. The present study enriches our understanding of general oral proficiency and the construct definition of proficiency tests.

  
  (show less)
  
  
• Investigating the possibility of using TOEFL scores for university ESL decision-making: Placement trends and effect of time lag
  Posted on 23 Apr 2012 at 6:42 am
  by Kokhan, K.
  

  The English Placement Test (EPT) at the University of Illinois at Urbana-Champaign (UIUC) is designed to provide an accurate placement (or exemption) of international students into the ESL writing ...

  (show all)

  The English Placement Test (EPT) at the University of Illinois at Urbana-Champaign (UIUC) is designed to provide an accurate placement (or exemption) of international students into the ESL writing and pronunciation classes. Over the last five years, UIUC has experienced an increase in the number of international students taking the EPT. Because of the costs of the EPT, various stakeholders have suggested using TOEFL as a substitute placement tool. The purpose of this research is to find out whether TOEFL iBT can serve as a suitable pre-screening tool for placement of international students into the ESL writing courses at UIUC.

  While there is a general statistically significant tendency that students with higher TOEFL iBT scores are placed into higher levels of ESL classes, there is no particular set of either total or section scores which can be used as a reliable criterion for dividing students into ESL classes without significant misplacement. Furthermore, the correlation between TOEFL iBT and the EPT shows a distinct pattern of time dependence: the correlation is stronger when the time gap between these tests is short but it dramatically decreases and even becomes slightly negative around Week 50; however, starting from the 50th week it increases. Some possible interpretations of the findings as well as recommendations for score users are discussed.

  
  (show less)
  
  
• Book review: English Language Assessment and the Chinese Learner
  Posted on 23 Apr 2012 at 6:42 am
  by Yang, R.
  
  
• Accent, listening assessment and the potential for a shared-L1 advantage: A DIF perspective
  Posted on 23 Apr 2012 at 6:42 am
  by Harding, L.
  

  This paper reports on an investigation of the potential for a shared-L1 advantage on an academic English listening test featuring speakers with L2 accents. Two hundred and twelve second-language li...

  (show all)

  This paper reports on an investigation of the potential for a shared-L1 advantage on an academic English listening test featuring speakers with L2 accents. Two hundred and twelve second-language listeners (including 70 Mandarin Chinese L1 listeners and 60 Japanese L1 listeners) completed three versions of the University Test of English as a Second Language (UTESL) listening sub-test which featured an Australian English-accented speaker, a Japanese-accented speaker and a Mandarin Chinese-accented speaker. Differential item functioning (DIF) analyses were conducted on data from the tests which featured L2-accented speakers using two methods of DIF detection – the standardization procedure and the Mantel-Haenszel procedure – with candidates matched for ability on the test featuring the Australian English-accented speaker. Findings showed that Japanese L1 listeners were advantaged on a small number of items on the test featuring the Japanese-accented speaker, but these were balanced by items which favoured non-Japanese L1 listeners. By contrast, Mandarin Chinese L1 listeners were clearly advantaged across several items on the test featuring a Mandarin Chinese L1 speaker. The implications of these findings for claims of bias are discussed with reference to the role of speaker accent in the listening construct.

  
  (show less)
  
  
• Methodological and theoretical issues in the adaptation of sign language tests: An example from the adaptation of a test to German Sign Language
  Posted on 23 Apr 2012 at 6:42 am
  by Haug, T.
  

  Despite the current need for reliable and valid test instruments in different countries in order to monitor the sign language acquisition of deaf children, very few tests are commercially available...

  (show all)

  Despite the current need for reliable and valid test instruments in different countries in order to monitor the sign language acquisition of deaf children, very few tests are commercially available that offer strong evidence for their psychometric properties. This mirrors the current state of affairs for many sign languages, where very little research is available. No previous empirical study has focused explicitly on the linguistic, methodological, and theoretical issues involved in the process of adapting a test from a source sign language to a target sign language. Problems during the adaptation process can arise from linguistic differences between the source and the target language and differences in the source and the target cultures. Both are important aspects that need to be considered in the adaptation of a sign language test from a source to a target language. This study proposes a model for sign language test adaptation, based on the adaptation of the British Sign Language Receptive Skills Test to German Sign Language. The model includes different methodological steps, with a particular focus on construct validation.

  
  (show less)
  
  
• The mediating role of textbooks in high-stakes assessment reform
  Posted on 18 Apr 2012 at 7:02 am
  by Leung, C. Y., Andrews, S.
  

  Whenever high-stakes assessment/curriculum reforms take place, new textbooks appear on the market. These textbooks inevitably play a significant mediating role in the implementation of any reform a...

  (show all)

  Whenever high-stakes assessment/curriculum reforms take place, new textbooks appear on the market. These textbooks inevitably play a significant mediating role in the implementation of any reform and on teaching and learning. This paper reports on a small-scale study which attempts to investigate the role of textbooks in the mediation of a high-stakes assessment reform: the introduction of school-based assessment (SBA) as part of the 2007 Hong Kong Certificate of Education Examination in English Language. The introduction of SBA represented a major challenge for teachers accustomed to a system of external examinations and ‘teaching to the test’. The paper examines the extent to which SBA textbooks align with the SBA principles and rationales. It then investigates the extent to which teachers rely on textbooks to prepare their students for SBA and the kind of materials that teachers feel they need most in order to prepare their students for SBA.

  
  (show less)
  
  
• Are More Stringent NCLB State Accountability Systems Associated With Better Student Outcomes? An Analysis of NAEP Results Across States
  Posted on 18 Apr 2012 at 6:20 am
  by Wei, X.
  

  This study developed a comprehensive measure of the stringency level of NCLB states’ accountability systems, including the strength of their annual measurable objectives, confidence intervals, perf...

  (show all)

  This study developed a comprehensive measure of the stringency level of NCLB states’ accountability systems, including the strength of their annual measurable objectives, confidence intervals, performance indexing, retesting, minimum subgroup size, and the difficulty levels of proficiency standards. This study related accountability stringency in 2003 to student achievement and achievement gaps on NAEP math and reading tests from 2003 and 2005. The results were inconsistent across grades, subjects, and ethnic groups. An increase in accountability stringency, such as requiring faster progress, not allowing for retesting, and allowing a smaller minimum subgroup size were related to improved math achievement for fourth-grade Hispanic students. Not using confidence intervals was related to higher math and reading achievement for White and Hispanic students. However, accountability stringency did not have any positive effects and even negative effects on eighth-grade Black students.

  
  (show less)
  
  
• An Empirical Evaluation of the Slip Correction in the Four Parameter Logistic Models With Computerized Adaptive Testing
  Posted on 27 Mar 2012 at 3:48 pm
  by Yen, Y.-C., Ho, R.-G., Laio, W.-W., Chen, L.-J., Kuo, C.-C.
  

  In a selected response test, aberrant responses such as careless errors and lucky guesses might cause error in ability estimation because these responses do not actually reflect the knowledge that ...

  (show all)

  In a selected response test, aberrant responses such as careless errors and lucky guesses might cause error in ability estimation because these responses do not actually reflect the knowledge that examinees possess. In a computerized adaptive test (CAT), these aberrant responses could further cause serious estimation error due to dynamic item administration. To enhance the robust performance of CAT against aberrant responses, Barton and Lord proposed the four-parameter logistic (4PL) item response theory (IRT) model. However, most studies relevant to the 4PL IRT model were conducted based on simulation experiments. This study attempts to investigate the performance of the 4PL IRT model as a slip-correction mechanism with an empirical experiment. The results showed that the 4PL IRT model could not only reduce the problematic underestimation of the examinees’ ability introduced by careless mistakes in practical situations but also improve measurement efficiency.

  
  (show less)
  
  
• Confirming Testlet Effects
  Posted on 27 Mar 2012 at 3:48 pm
  by DeMars, C. E.
  

  A testlet is a cluster of items that share a common passage, scenario, or other context. These items might measure something in common beyond the trait measured by the test as a whole; if so, the m...

  (show all)

  A testlet is a cluster of items that share a common passage, scenario, or other context. These items might measure something in common beyond the trait measured by the test as a whole; if so, the model for the item responses should allow for this testlet trait. But modeling testlet effects that are negligible makes the model unnecessarily complicated and risks capitalization on chance, increasing the error in parameter estimates. Checking each testlet to see if the items within the testlet share something beyond the primary trait could therefore be useful. This study included (a) comparison between a model with no testlets and a model with testlet g, (b) comparison between a model with all suspected testlets and a model with all suspected testlets except testlet g, and (c) a test of essential unidimensionality. Overall, Comparison b was most useful for detecting testlet effects. Model comparisons based on information criteria, specifically the sample-size adjusted Bayesian Information Criteria (SSA-BIC) and BIC, resulted in fewer false alarms than statistical significance tests. The test of essential unidimensionality had true hit rates and false alarm rates similar to the SSA-BIC when the testlet effect was zero for all testlets except the studied testlet. But the presence of additional testlet effects in the partitioning test led to higher false alarm rates for the test of essential unidimensionality.

  
  (show less)
  
  
• The Impact of State-Wide Exit Exams in Germany: A Descriptive Case Study
  Posted on 19 Mar 2012 at 7:00 pm
  by Isabell van Ackeren, Rainer Block, Esther Dominique Klein, Svenja Mareike Kühn
  
  This study analyzes the possible intended and unintended impact of statewide exit exams as a governance tool used by education authorities. In a descriptive case study based on quantitative empiric... (show all)
  This study analyzes the possible intended and unintended impact of statewide exit exams as a governance tool used by education authorities. In a descriptive case study based on quantitative empirical research, three German states (Baden-Württemberg, Rhineland-Palatinate, North Rhine-Westphalia) with differing exit exam traditions (statewide versus school-based) are compared. Consistent with other recent research findings, the potential influences of statewide exit exams on teaching and learning processes can be detected for mathematics, but not for German or biology. In particular, these influences show in the fact that teachers whose students have to take a statewide exit exam are less responsive to students? interests regarding the choice of topics in the classroom, perceive a tendency to narrow the delivered curriculum and increase cooperation with other teachers. In addition, both teachers and students feel a higher pressure to perform and perceive their roles to be different ?they see themselves as allies preparing for an externally set exam. Teachers also tend to feel ?de-professionalized?. In those areas where statewide exams do seem to affect schooling, most of the results are consistent with what was intended by the authorities when they introduced the exams, and there are only few unintended side-effects. From a governance perspective, however, a general capacity of statewide exit exams to move schooling in a desired direction remains rather doubtful, particularly considering the fact that the exam procedures that have recently been implemented in the German states are very heterogeneous and only partly standardized.
  (show less)
  
  
• Accuracy of Teacher Judgments of Preschoolers' Math Skills
  Posted on 19 Mar 2012 at 2:30 am
  by Kilday, C. R., Kinzie, M. B., Mashburn, A. J., Whittaker, J. V.
  

  As the interest in improving children’s early math skills has grown, due in part to their strong associations with later overall school achievement, questions have been raised regarding teachers’ k...

  (show all)

  As the interest in improving children’s early math skills has grown, due in part to their strong associations with later overall school achievement, questions have been raised regarding teachers’ knowledge about children’s math abilities. The current study uses hierarchical linear modeling to examine the associations between teachers’ judgments of children’s math skills using an indirect rating scale assessment and children’s performance on two direct assessments of their number sense and geometry and measurement skills. Approximately 40% of the variance in the rating scale completed by the teacher is attributable to systematic differences between classrooms, not specifically to the child. Overall, the association between teacher report and students’ skills is approximately r = .50, which suggests that teachers can accurately determine whether students are above or below the mean but do not appropriately rate students as high or low as a direct assessment of their skills would indicate. This finding has implications for teachers, particularly in terms of teacher preparation, and for researchers, because of the information regarding the accuracy of teacher rating scales of preschool students’ math skills.

  
  (show less)
  
  
• The language policy of state drivers? license testing: expediency, symbolism, or creeping incrementalism?
  Posted on 13 Mar 2012 at 11:53 am
  

  Until recently, educational language policy in the US has been the chief site of contention about language, as seen in recent initiatives, referenda, and state constitutional amendments. Provision...

  (show all)

  Until recently, educational language policy in the US has been the chief site of contention about language, as seen in recent initiatives, referenda, and state constitutional amendments. Provision for drivers? licensing testing in languages other than English (LotE), on the other hand, has often exemplified what we call expedient language policy, i.e. using a LotE for a higher end, that of ensuring highway safety and enhancing opportunities (freedom of travel, especially for economic benefits, i.e. work). In some states, however, notably an Alabama case Alexander v. Sandoval, language policy of vehicle licensing has become symbolic of other issues, and the ACLU is now pitted against the National Review, the English-Only and English-First organizations, as well as disability-rights organizations, many of whom have provided amicus curae briefs, all of which seems at first glance out of proportion to the importance of the issue at hand. Between the time this paper was proposed and the present, the Supreme Court has heard this case and found in favor of the state of Alabama and against the parties to the original suit. (The court heard Alexander v. Sandoval No. 99-1908 on January 16, 2001, and issued its opinion, for Alexander, on April 24, 2001.)

  Content Type Journal ArticleCategory CommentaryPages 189-196DOI 10.1007/s10993-012-9233-zAuthors Harold F. Schiffman, South Asia Studies, University of Pennsylvania, Philadelphia, PA, USARichard E. Weiner, International Law Department, Fredrikson & Byron, P.A., Minneapolis, MN, USA Journal Language PolicyOnline ISSN 1573-1863Print ISSN 1568-4555 Journal Volume Volume 11 Journal Issue Volume 11, Number 2
  (show less)
  
  
• Variation in teachers? conceptions of alternative assessment in Singapore primary schools
  Posted on 6 Mar 2012 at 11:20 am
  

  Abstract  Alternative assessment is an increasingly common and popular discourse in education. The potential value and benefit of alternative assessment practices are premised on significant chang...

  (show all)

  Abstract  Alternative assessment is an increasingly common and popular discourse in education. The potential value and benefit of alternative assessment practices are premised on significant changes in assessment practices. Many alternative assessment practices seek and promise alternatives to tests and examinations. However, labelling a practice as ?Alternative Assessment? in itself is not a guarantee of meaningful departure from existing practice. Such alternative assessment practices risk retaining and perpetuating the same limitations and adverse consequences of tests and examinations they claim to avoid. Recent developments in the assessment system in Singapore have given rise to increased attention to alternative assessment as a means of alleviating negative effects of testing and high stakes examinations. The success or limitations of these attempts are helpful in understanding how alternative assessment can be understood and used in different ways, and how each way of experiencing alternative assessment may be more suited to supporting learning than others. In this article, the results of a phenomenographic study on teachers? experiences of alternative assessment in Singapore are presented. Three conceptions of alternative assessment are identified, each depicting a particular way of understanding and using alternative assessment in schools. These findings are subsequently examined in terms of the potential for teachers to understand and use alternative assessment to enhance students learning in sustainable ways.

  Content Type Journal ArticlePages 1-21DOI 10.1007/s10671-012-9130-4Authors Kelvin H. K. Tan, Curriculum, Teaching and Learning, National Institute of Education, 1 Nanyang Walk, Singapore, 637616 Singapore Journal Educational Research for Policy and PracticeOnline ISSN 1573-1723Print ISSN 1570-2081
  (show less)
  
  
• Flexible Response: Executive Federalism and the No Child Left Behind Act of 2001
  Posted on 28 Feb 2012 at 5:54 am
  by Shelly, B.
  

  The federal government promised that it would limit waiver grants to states for the No Child Left Behind Act of 2001 (NCLB). It largely kept that promise, but states did gain significant flexibilit...

  (show all)

  The federal government promised that it would limit waiver grants to states for the No Child Left Behind Act of 2001 (NCLB). It largely kept that promise, but states did gain significant flexibility through amendments to accountability plans. OLS model estimates showed that larger, more affluent, and more Republican states submitted more amendment requests and the federal Department of Education approved the amendments of states that received more federal funding, had a more developed standardized testing system in place prior to NCLB, and voted Republican. These findings suggested new avenues of exploration for scholars of executive federalism.

  
  (show less)
  
  
• Firestar-D: Computerized Adaptive Testing Simulation Program for Dichotomous Item Response Theory Models
  Posted on 23 Feb 2012 at 3:20 pm
  by Choi, S. W., Podrabsky, T., McKinney, N.
  
  
• The MIMIC Model as a Tool for Differential Bundle Functioning Detection
  Posted on 23 Feb 2012 at 3:20 pm
  by Finch, W. H.
  

  Increasingly, researchers interested in identifying potentially biased test items are encouraged to use a confirmatory, rather than exploratory, approach. One such method for confirmatory testing i...

  (show all)

  Increasingly, researchers interested in identifying potentially biased test items are encouraged to use a confirmatory, rather than exploratory, approach. One such method for confirmatory testing is rooted in differential bundle functioning (DBF), where hypotheses regarding potential differential item functioning (DIF) for sets of items (bundles) are developed based on the substantive nature of the items and expected differences in group performance. Most often, analyses of these bundles for DBF have been conducted using simultaneous item bias test (SIBTEST). The goal of the current study was to introduce an alternative methodology for DBF detection based on the multiple indicators multiple cause (MIMIC) model and to compare this alternative with the traditional SIBTEST-based approach using a Monte Carlo simulation study. The results of this study showed that the MIMIC model performed as well as SIBTEST in ideal conditions, and better when the reference and focal groups had different means on the primary latent trait being measured. In addition, the MIMIC model was more accurate at detecting the presence of DBF for two bundles simultaneously than was SIBTEST. The discussion focuses on recommendations for practitioners.

  
  (show less)
  
  
• Assessment Practices and the Underrepresentation of Minority Students in Gifted and Talented Education
  Posted on 9 Feb 2012 at 5:31 am
  by Erwin, J. O., Worrell, F. C.
  

  Several scholars have asserted that the underrepresentation of minority students in gifted and talented education (GATE) programs is the result of biased assessment practices. However, an examinati...

  (show all)

  Several scholars have asserted that the underrepresentation of minority students in gifted and talented education (GATE) programs is the result of biased assessment practices. However, an examination of the psychometric properties of scores on cognitive ability, achievement tests, and rating scales do not support these claims. We contend that the underrepresentation of some racial/ethnic groups in GATE program is another manifestation of the longstanding and intractable achievement gap in the United States. Although we agree that the goal of having more equitable representation of the school population in GATE programs is laudable, we argue that the solution is one that needs to come from policy changes rather than changes in assessment instruments. We provide recommendations for identifying gifted students who may be at a disadvantage because of their group membership and illustrate this process using recent data from a summer program for academically talented students.

  
  (show less)
  
  
• Identifying and Assessing Creativity as a Component of Giftedness
  Posted on 9 Feb 2012 at 5:31 am
  by Kaufman, J. C., Plucker, J. A., Russell, C. M.
  

  Most theories of giftedness include creativity as a central component. Creativity assessment has a key role, therefore, in measuring giftedness. This article reviews the state of the creativity ass...

  (show all)

  Most theories of giftedness include creativity as a central component. Creativity assessment has a key role, therefore, in measuring giftedness. This article reviews the state of the creativity assessment, from divergent thinking tests (including the Torrance Tests of Creative Thinking) to the consensual assessment technique to rating scales and self assessments. Despite the many flaws present in every type of creativity measurement, there are compelling reasons for including creativity as part of a gifted assessment battery.

  
  (show less)
  
  
• Using Nonverbal Tests to Help Identify Academically Talented Children
  Posted on 9 Feb 2012 at 5:31 am
  by Lohman, D. F., Gambrell, J. L.
  

  Language-reduced (nonverbal) ability tests are the primary talent identification tools for ELL children. The appropriate use of such tests with low-SES and minority children is more nuanced. Whenev...

  (show all)

  Language-reduced (nonverbal) ability tests are the primary talent identification tools for ELL children. The appropriate use of such tests with low-SES and minority children is more nuanced. Whenever language-reduced tests are used for talent identification, nonverbal tests that measure more than figural reasoning abilities should be employed. For young children, picture-based reasoning tests can significantly broaden the scope of the assessment. We report new data showing that ELL, low SES, and minority children in Grades K to 2 typically perform as well or better on such tests than on figural reasoning tests. We also show that for children in Grades 3 to 6, language-reduced quantitative tests are an often overlooked alternative to figural tests. Finally, inferences about ability require comparison of a child’s performance with the performance of other children who have had similar opportunities to develop skills tested. This is often not the case for ELL, poor, and minority children, even on nonverbal tests. In such cases, multiple normative perspectives can greatly assist in the identification of academically talented children.

  
  (show less)
  
  
• The Arizona home language survey: The under-identification of students for English language services
  Posted on 3 Feb 2012 at 11:11 am
  

  Abstract  Assuring that English learners (ELs) receive the support services to which they are entitled requires accurately identifying students who are limited in their English proficiency. As a f...

  (show all)

  Abstract  Assuring that English learners (ELs) receive the support services to which they are entitled requires accurately identifying students who are limited in their English proficiency. As a first step in the identification process, students? parents fill out a home language survey. If the survey indicates a language other than English is spoken in the home, students are then assessed for English language proficiency. The home language survey thus plays an important gatekeeping role. In July, 2009, the Arizona Department of Education reduced the number of questions on the state?s home language survey from three to one. This article addresses whether this reduction in questions can lead to under-identification of students who, by the state?s own criterion (i.e., performance on the Arizona English Language Learner Assessment, or AZELLA), are entitled to EL services. Analyses of data from two Arizona school districts clearly show that use of a single home language survey question will under-identify substantial numbers of English learners. Further, it is highly unlikely that a fail-safe mechanism established by the state, whereby teachers can nominate potential ELs for language testing, will successfully identify more than a small number of students missed by the single question survey.

  Content Type Journal ArticleCategory Original PaperPages 21-30DOI 10.1007/s10993-011-9224-5Authors Claude Goldenberg, Stanford University, Stanford, CA, USASara Rutherford-Quach, Stanford University, Stanford, CA, USA Journal Language PolicyOnline ISSN 1573-1863Print ISSN 1568-4555 Journal Volume Volume 11 Journal Issue Volume 11, Number 1
  (show less)
  
  
• Examining the validity of the Arizona English Language Learners Assessment cut scores
  Posted on 2 Feb 2012 at 12:13 pm
  

  Abstract  The Arizona English Language Learners Assessment (AZELLA) is used by the Arizona Department of Education to determine which children should receive English support services. AZELLA resul...

  (show all)

  Abstract  The Arizona English Language Learners Assessment (AZELLA) is used by the Arizona Department of Education to determine which children should receive English support services. AZELLA results are used to determine if children are either proficient in English or have English language skills in one of four non-proficient categories (pre-emergent, emergent, basic, intermediate). Children who test at or above the proficient cut score in English are placed in mainstream classes without English language support. Children who obtain scores below the proficient cut scores receive English language support services in state-mandated Structured English Immersion classes. Whenever tests are used to make high-stakes decisions, especially about vulnerable populations (e.g., children), it is the test developers? responsibility to ensure the instrument yields fair and valid results. When cut scores are used as the primary interpretation of the test they are key to establishing the test?s validity. This validation study found that cut scores for the AZELLA are of questionable validity. The procedure used to set the cut scores is criticized by national measurement experts as ineffective and obsolete. Further, the test developers do not adequately establish the expertise of the judges used to set the cut scores. Evidence from the cut-score-setting process indicates judges did not come to consensus at the kindergarten level. Analysis of empirical evidence suggests cut scores over-identify kindergarten children and under-identify older children. Finally, the test developers rejected 85% of the cut scores recommended by the standard-setting judges, setting cut scores higher than recommended for kindergarten and lower than recommended for older children, without describing their process or rationale.

  Content Type Journal ArticleCategory Original PaperPages 33-45DOI 10.1007/s10993-011-9225-4Authors Ida Rose Florez, Arizona State University, Phoenix, AZ, USA Journal Language PolicyOnline ISSN 1573-1863Print ISSN 1568-4555 Journal Volume Volume 11 Journal Issue Volume 11, Number 1
  (show less)
  
  
• L'enseignement d'un vocabulaire disciplinaire dans deux contextes d'immersion universitaire : Quelle approche favoriser?
  Posted on 1 Feb 2012 at 9:56 am
  

  La présente étude vise à vérifier si un enseignement du vocabulaire dans deux contextes d'immersion différents permet une amélioration des connaissances lexicales. Cet article examinera premièreme...

  (show all)

  La présente étude vise à vérifier si un enseignement du vocabulaire dans deux contextes d'immersion différents permet une amélioration des connaissances lexicales. Cet article examinera premièrement le contexte de notre étude effectuée auprès de 11 étudiants en droit et 13 étudiants en histoire inscrits à l'université d'Ottawa dans un cours d'encadrement linguistique associé à un cours d'immersion. Puis, nous décrirons le protocole utilisé : chaque semaine, pendant une session universitaire, les étudiants ont complété une série d'activités reliées à un vocabulaire disciplinaire provenant de leurs lectures et de leurs cours. Nous présenterons également les résultats des analyses aux différents tests (pré-test, post-test et test différé) pour vérifier l'amélioration des connaissances chez les étudiants et la rétention de ce vocabulaire entre le début et la fin de la session, entre les mots enseignés et les mots non enseignés. Nos résultats permettront ainsi de proposer les meilleures approches pédagogiques dans les deux cours visés, approches qui tiendront compte des caractéristiques spécifiques au vocabulaire de chacune des disciplines.

  Content Type Journal ArticleCategory ArticlesPages 1-27DOI 10.3138/cmlr.68.1.001Authors Alysse WeinbergDalila BoukacemSandra Burger Journal Canadian Modern Language Review/ La Revue canadienne des langues vivantesOnline ISSN 1710-1131Print ISSN 0008-4506 Journal Volume Volume 68 Journal Issue Volume 68, Number 1 / February 2012
  (show less)
  
  
• Validity argument for language assessment: The framework is simple...
  Posted on 25 Jan 2012 at 6:17 pm
  by Chapelle, C. A.
  
  
• Re-fitting for a different purpose: A case study of item writer practices in adapting source texts for a test of academic reading
  Posted on 25 Jan 2012 at 6:17 pm
  by Green, A., Hawkey, R.
  

  The important yet under-researched role of item writers in the selection and adaptation of texts for high-stakes reading tests is investigated through a case study involving a group of trained item...

  (show all)

  The important yet under-researched role of item writers in the selection and adaptation of texts for high-stakes reading tests is investigated through a case study involving a group of trained item writers working on the International English Language Testing System (IELTS). In the first phase of the study, participants were invited to reflect in writing, and then audio-recorded in a semantic-differential-based joint discussion, on the processes they employed to generate test material. The group were next observed at a simulated item writers’ editing meeting to refine their texts and items for an IELTS reading test module. The participants’ written descriptions and recorded discussions provided rich data on how source texts were perceived, selected and adapted for the Test. The study reports findings from textual analyses using indices of readability and lexical density from the original material sourced by the item writers and their adapted versions for the test. Results from qualitative and quantitive analyses are discussed in terms of the implications for the IELTS reading module of editing actions such as: reducing redundancy and technical language, changing styles, deciding on potentially sensitive issues and relationships between texts and test items. The important issue of text authenticity in tests such as IELTS is also broached.

  
  (show less)
  
  
• Note-taking quality and performance on an L2 academic listening test
  Posted on 25 Jan 2012 at 6:17 pm
  by Song, M.-Y.
  

  This study investigated the relationships among the quality of L2 test takers’ notes evaluated in terms of different levels of information and test takers’ performance on open-ended listening tasks...

  (show all)

  This study investigated the relationships among the quality of L2 test takers’ notes evaluated in terms of different levels of information and test takers’ performance on open-ended listening tasks tapping into different comprehension subskills. In addition, this study examined the invariance of the structural relationships among the variables across two different note-taking formats, that is, a blank format and an outline format, by employing a multi-group structural equation modeling (SEM) approach. The results indicated that note quality measures, in particular the number of topical ideas found in the notes and the organization of these notes, may be good indicators of test takers’ second language academic listening proficiency. It was also found that despite the invariance of structural relationships among variables across the two note-taking formats, the associations between the open-ended listening measures and note quality measures were slightly stronger in the outline format than in the blank format. The implications of these results for L2 academic listening assessment are considered.

  
  (show less)
  
  
• Grounding the argument-based framework for validating score interpretations and uses
  Posted on 25 Jan 2012 at 6:17 pm
  by Oller, J. W.
  

  Kane’s argument-based framework is summarized and examined. He implicitly appeals to the backgrounded concepts of fairness and justice. From there it is a short distance to grounding the whole syst...

  (show all)

  Kane’s argument-based framework is summarized and examined. He implicitly appeals to the backgrounded concepts of fairness and justice. From there it is a short distance to grounding the whole system in the mundane notion of truth. In fact, valid argument systems must depend on representations that are ‘true’ by virtue of agreement with purported facts. As a friendly amendment, therefore, I argue that (provided the ceteris paribus, all else being equal, requirement is met) agreement with known facts in testing, experimental research, and scientific measurement counts for a great deal more than disagreement. It follows by Peircean ‘exact logic’ that higher test scores (if the tests have any validity at all) are invariably more informative (interpretable in general) and thus more useful than lower scores. Why? Because higher scores show more agreement between the test-makers and the higher scoring test-takers about whatever facts (or performances) may be at issue. Exceptions are cases where the ceteris paribus requirement is not met. Necessary (but testable) inferences follow for interpretations and uses of ‘cutscores.’

  
  (show less)
  
  
• Differential Item Functioning in While-Listening Performance Tests: The Case of the International English Language Testing System (IELTS) Listening Module
  Posted on 17 Jan 2012 at 9:57 am
  International Journal of Listening, Volume 26, Issue 1, Page 40-60, January-April 2012.
  
  
  
• Evaluating the effectiveness of explicit instruction on implicit and explicit L2 knowledge
  Posted on 12 Jan 2012 at 7:05 am
  by Akakura, M.
  

  This study examined the effectiveness of explicit instruction on second language (L2) learners’ implicit and explicit knowledge of English. Explicit instruction on the generic and non-generic use o...

  (show all)

  This study examined the effectiveness of explicit instruction on second language (L2) learners’ implicit and explicit knowledge of English. Explicit instruction on the generic and non-generic use of English articles was delivered by CALL activities. Four tasks assessed acquisition: elicited imitation, oral production, grammaticality judgement, and metalinguistic knowledge tasks. A pretest and two posttests were conducted immediately and six weeks after the treatment. Durable effects for explicit instruction were found on measures of implicit knowledge and on ungrammatically supplied items on measures of explicit knowledge. This study’s findings contribute towards our understanding of the efficacy of explicit instruction on implicit and explicit knowledge at relatively advanced stages of L2 acquisition.

  
  (show less)
  
  
• The 'promise' of three methods of word association analysis to L2 lexical research
  Posted on 3 Jan 2012 at 8:54 am
  by Zareva, A., Wolter, B.
  

  The present study is an attempt to empirically test and compare the results of three methods of word association (WA) analysis. Two of the methods – namely, associative commonality and nativelikene...

  (show all)

  The present study is an attempt to empirically test and compare the results of three methods of word association (WA) analysis. Two of the methods – namely, associative commonality and nativelikeness, and lexico-syntactic patterns of associative organization – have been traditionally used in both first language (L1) and second language (L2) associative research and the third one – collocational aspect of associative responses – is a more recent perspective on associative connections. The central assumption behind the study is that each method captures a different aspect of language users’ lexical organization and, in that, their findings may paint a different picture of how the L1 and L2 lexicons compare as proficiency increases. At the same time, the sensitivity of each method to reflect differences related to proficiency may also point to their potential as a research and assessment tool.

  Three equal size groups of participants (N = 180) at different proficiency levels – native speakers (NSs), second language (L2) advanced and intermediate learners of English – completed a familiarity and a WA test in writing. The tests contained 36 items equated for lexical class (nouns, verbs, and adjectives) and frequency of occurrence (high, mid, and low). The participants’ WAs generated to familiar vocabulary were analysed in three different ways, following the methods of analysis practiced by each of the three traditions under investigation. In the main, the results showed that examining the lexico-syntactic patterns of associative organization as well as the collocational aspect of associative links are two more ‘promising’ ways of looking at WAs than examining them with respect to their nativelike associative commonality.

  
  (show less)
  
  
• A Bifactor Multidimensional Item Response Theory Model for Differential Item Functioning Analysis on Testlet-Based Items
  Posted on 29 Dec 2011 at 5:22 pm
  by Fukuhara, H., Kamata, A.
  

  A differential item functioning (DIF) detection method for testlet-based data was proposed and evaluated in this study. The proposed DIF model is an extension of a bifactor multidimensional item re...

  (show all)

  A differential item functioning (DIF) detection method for testlet-based data was proposed and evaluated in this study. The proposed DIF model is an extension of a bifactor multidimensional item response theory (MIRT) model for testlets. Unlike traditional item response theory (IRT) DIF models, the proposed model takes testlet effects into account, thus estimating DIF magnitude appropriately when a test is composed of testlets. A fully Bayesian estimation method was adopted for parameter estimation. The recovery of parameters was evaluated for the proposed DIF model. Simulation results revealed that the proposed bifactor MIRT DIF model produced better estimates of DIF magnitude and higher DIF detection rates than the traditional IRT DIF model for all simulation conditions. A real data analysis was also conducted by applying the proposed DIF model to a statewide reading assessment data set.

  
  (show less)
  
  
• A Triangulated Study of Academic Language Needs of Iranian Students of Computer Engineering: Are the Courses on Track?
  Posted on 14 Dec 2011 at 10:11 am
  by Atai, M. R., Shoja, L.
  

  Even though English for Specific Academic Purposes (ESAP) courses constitute a significant part of the Iranian university curriculum, curriculum developers have generally developed the programs bas...

  (show all)

  Even though English for Specific Academic Purposes (ESAP) courses constitute a significant part of the Iranian university curriculum, curriculum developers have generally developed the programs based on intuition. This study assessed the present and target situation academic language needs of undergraduate students of computer engineering. To this end, data were elicited from 231 undergraduates, 30 graduate students, 20 computer engineering instructors and 15 ESP instructors at three major Iranian universities. Instrumentation included needs analysis and self-assessment questionnaires, a general English proficiency test, semi-structured interviews, and non-participant observations. Despite some inconsistencies in participants’ perceptions of target situation needs, the findings indicate that written skills and language components are important. The undergraduates perceived difficulties with some sub-skills of reading, writing, speaking, and listening. Moreover, General English Proficiency (GEP) level of the students, as probed by a GEP test and confirmed by self assessment results and teacher assessment data, was generally low. The undergraduates held that they need more general English than highly specific academic English. Finally, participants commented on several major problematic areas. The findings may promise implications for renewing the ESAP course under study and suggest a model framework for needs assessment to other EASP researchers worldwide.

  
  (show less)
  
  
• Identifying Problematic Segmental Features to Acquire Comprehensible Pronunciation in EFL Settings: The Case of Japanese Learners of English
  Posted on 14 Dec 2011 at 10:11 am
  by Saito, K.
  

  The present study examines how to identify problematic pronunciation features for particular EFL learners, namely native Japanese speakers (NJs) learning English, to acquire comprehensible pronunci...

  (show all)

  The present study examines how to identify problematic pronunciation features for particular EFL learners, namely native Japanese speakers (NJs) learning English, to acquire comprehensible pronunciation, and tests the appropriateness of the selection. The study comprises two phases. In the identification phase, eight English-specific segmentals, /æ, f, v, , ð, w, l, /, were selected as the most problematic for NJs by drawing on various cross-linguistic analyses (i.e. a remedial approach) as well as a survey in which the advice of 48 experienced NJ English teachers was examined (i.e. an expert judgment approach). In the experimental phase, the relative influence of these sounds on comprehensibility and accentedness was analyzed. Twenty NJ participants read two types of sentences: sentences containing eight English-specific segmentals and sentences without them. Four native English speakers (NEs) subsequently rated all speech stimuli on a rubric of accentedness and comprehensibility. Significant differences were found between NEs’ ratings of the two types of sentences both in the domain of comprehensibility and accentedness. The results indicate that the eight segmentals determine NEs’ speech perception to a great degree, which in turn provides some support for the validity of the identification procedure (i.e. the combination of the remedial and expert judgment approaches).

  
  (show less)
  
  
• Goal Setting and Student Achievement: A Longitudinal Study
  Posted on 5 Dec 2011 at 12:48 pm
  by ALEIDINE J. MOELLER, JANINE M. THEILER, CHAORONG WU
  
  The connection between goals and student motivation has been widely investigated in the research literature, but the relationship of goal setting and student achievement at the classroom level has ... (show all)
  The connection between goals and student motivation has been widely investigated in the research literature, but the relationship of goal setting and student achievement at the classroom level has remained largely unexplored. This article reports the findings of a 5-year quasi-experimental study examining goal setting and student achievement in the high school Spanish language classroom. The implementation of LinguaFolio, a portfolio that focuses on student self-assessment, goal setting, and collection of evidence of language achievement, was introduced into 23 high schools with a total of 1,273 students. By using a hierarchical linear model, researchers were able to analyze the relationship between goal setting and student achievement across time at both the individual student and teacher levels. A correlational analysis of the goal-setting process and language proficiency scores reveals a statistically significant relationship between the goal-setting process and language achievement (p < .01).
  (show less)
  
  
• Raising reading achievement in an ?at risk?, low socioeconomic, multicultural intermediate school
  Posted on 15 Nov 2011 at 9:08 am
  by Jo Fletcher, Michael Grimley, Janinka Greenwood, Faye Parkhill
  
  This article focuses on a multicultural, low socioeconomic, intermediate school that over the 4 years of this longitudinal, qualitative, case study made substantial positive shifts in developing a ... (show all)
  This article focuses on a multicultural, low socioeconomic, intermediate school that over the 4 years of this longitudinal, qualitative, case study made substantial positive shifts in developing a more effective learning environment and improving students' reading achievement. The study found that the factors appearing to have the most influence on this improvement were: effective and collaborative school leadership; ongoing school-wide professional development on teaching reading led by an externally appointed literacy expert; the appointment, within the school, of a literacy leader charged with supporting this development; assessment data being used to inform teaching and a school-wide action plan directed at literacy improvement; the implementation of reading programmes that were regular, focused and sustained; the school leadership proactively ensuring school-wide support for management of appropriate student behaviour; the fostering of home?school partnerships; and ongoing external reviews of school effectiveness.
  (show less)
  
  
• Very early language skills of fifth-grade poor comprehenders
  Posted on 15 Nov 2011 at 9:08 am
  by Laura Justice, Andrew Mashburn, Yaacov Petscher
  
  This study tested the theory that future poor comprehenders would show modest but pervasive deficits in both language comprehension and production during early childhood as compared with future poo... (show all)
  This study tested the theory that future poor comprehenders would show modest but pervasive deficits in both language comprehension and production during early childhood as compared with future poor decoders and typical readers. Using an existing database (NICHD ECCRN), fifth-grade students were identified as having poor comprehension skills (n=16), poor decoding skills (n=11) or typical reading skills (n=35) based on standardised assessments of word recognition and reading comprehension. Language comprehension and production during the toddler and preschool years were retrospectively compared across these subgroups. Compared with future typical readers and poor decoders, poor comprehenders had the lowest abilities on language assessments at 15, 24, 36 and 54 months. For nearly all contrasts, the difference between poor comprehenders and the other groups of readers exceeded .5 standard deviation in magnitude, indicating that the early language skills of poor comprehenders exhibit appreciable lags.
  (show less)
  
  
• Phonetically rich and balanced text and speech corpora for Arabic language
  Posted on 4 Nov 2011 at 11:54 am
  

  Abstract  This paper describes the preparation, recording, analyzing, and evaluation of a new speech corpus for Modern Standard Arabic (MSA). The speech corpus contains a total of 415 sentences re...

  (show all)

  Abstract  This paper describes the preparation, recording, analyzing, and evaluation of a new speech corpus for Modern Standard Arabic (MSA). The speech corpus contains a total of 415 sentences recorded by 40 (20 male and 20 female) Arabic native speakers from 11 different Arab countries representing three major regions (Levant, Gulf, and Africa). Three hundred and sixty seven sentences are considered as phonetically rich and balanced, which are used for training Arabic Automatic Speech Recognition (ASR) systems. The rich characteristic is in the sense that it must contain all phonemes of Arabic language, whereas the balanced characteristic is in the sense that it must preserve the phonetic distribution of Arabic language. The remaining 48 sentences are created for testing purposes, which are mostly foreign to the training sentences and there are hardly any similarities in words. In order to evaluate the speech corpus, Arabic ASR systems were developed using the Carnegie Mellon University (CMU) Sphinx 3 tools at both training and testing/decoding levels. The speech engine uses 3-emitting state Hidden Markov Models (HMM) for tri-phone based acoustic models. Based on experimental analysis of about 8 h of training speech data, the acoustic model is best using continuous observation?s probability model of 16 Gaussian mixture distributions and the state distributions were tied to 500 senones. The language model contains uni-grams, bi-grams, and tri-grams. For same speakers with different sentences, Arabic ASR systems obtained average Word Error Rate (WER) of 9.70%. For different speakers with same sentences, Arabic ASR systems obtained average WER of 4.58%, whereas for different speakers with different sentences, Arabic ASR systems obtained average WER of 12.39%.

  Content Type Journal ArticleCategory Original PaperPages 1-34DOI 10.1007/s10579-011-9166-8Authors Mohammad A. M. Abushariah, Faculty of Computer Science and Information Technology, University of Malaya, 50603 Kuala Lumpur, MalaysiaRaja N. Ainon, Faculty of Computer Science and Information Technology, University of Malaya, 50603 Kuala Lumpur, MalaysiaRoziati Zainuddin, Faculty of Computer Science and Information Technology, University of Malaya, 50603 Kuala Lumpur, MalaysiaMoustafa Elshafei, Department of Systems Engineering, King Fahd University of Petroleum and Minerals, KFUPM Box 405, Dhahran, 31261 Saudi ArabiaOthman O. Khalifa, Electrical and Computer Engineering Department, Faculty of Engineering, International Islamic University Malaysia, 53100 Gombak, Kuala Lumpur, Malaysia Journal Language Resources and EvaluationOnline ISSN 1574-0218Print ISSN 1574-020X
  (show less)
  
  
• Quantifying the "Degree of Linguistic Demand" in Spoken Intelligence Test Directions
  Posted on 2 Nov 2011 at 5:13 am
  by Cormier, D. C., McGrew, K. S., Evans, J. J.
  

  The linguistic demand of spoken instructions on individually administered norm-referenced psychological and educational tests is of concern when examining individuals who have varying levels of lan...

  (show all)

  The linguistic demand of spoken instructions on individually administered norm-referenced psychological and educational tests is of concern when examining individuals who have varying levels of language processing ability or varying cultural backgrounds. The authors present a new method for analyzing the level of verbosity, complexity, and total demand of spoken directions for individually administered test batteries. This preliminary methodological investigation suggests it is possible, and relatively easy, to gather useful empirical information regarding the complexity of spoken test directions using existing (readily available) text readability programs. It is suggested that best practice for individually administered psychological and educational tests may benefit from including this information when tests are published and/or compared.

  
  (show less)
  
  
• Is linguistic ability variation in paired oral language testing problematic?
  Posted on 20 Oct 2011 at 11:35 pm
  by Bennett, R.
  

  This study looks at the real and perceived effects of inter-candidate variation in linguistic ability during pair-tested oral examinations in a southern Italian context. Pair testing has not been t...

  (show all)

  This study looks at the real and perceived effects of inter-candidate variation in linguistic ability during pair-tested oral examinations in a southern Italian context. Pair testing has not been taken up in schools and universities in southern Italy despite its wide use in international testing systems. Colleagues and prospective candidates often report to teachers their belief that test results can be adversely or positively affected by candidate pairing. To investigate the existence and validity of these beliefs, a series of tests was devised consisting of paired oral tests and pre- and post-test questionnaires. It was found that while the great majority of candidates believed that their test scores could be, and in many cases had been, affected by their candidate pairing, no evidence came to light to support these beliefs. In fact the linguistic ability of a candidate’s partner appeared to have little or no effect on his or her performance.

  
  (show less)
  
  
• Book Review: Language assessment in practice: Developing language assessments and justifying their use in the real world
  Posted on 19 Oct 2011 at 5:21 am
  by Kane, M.
  
  
• The development and maintenance of rating quality in performance writing assessment: A longitudinal study of new and experienced raters
  Posted on 19 Oct 2011 at 5:21 am
  by Lim, G. S.
  

  Raters are central to writing performance assessment, and rater development – training, experience, and expertise – involves a temporal dimension. However, few studies have examined new and experie...

  (show all)

  Raters are central to writing performance assessment, and rater development – training, experience, and expertise – involves a temporal dimension. However, few studies have examined new and experienced raters’ rating performance longitudinally over multiple time points. This study uses operational data from the writing section of the MELAB (n = 20,662 ratings), an international exam of English proficiency, to investigate the rating quality of new and experienced raters over three time periods of 12 to 21 months. Rating quality was operationalized in terms of rater severity and consistency, and estimates of those modeled using multi-facet Rasch methodology. Results indicate that, within one particular rating context, (1) novice raters, where initially differing in performance, learn to rate appropriately relatively quickly, (2) raters are able to maintain rating quality over time, and (3) rating volume and rating quality may be related. Implications for rater preparation, rater certification, and the notion of expert rater are discussed.

  
  (show less)
  
  
• Building out a measurement model to incorporate complexities of testing in the language domain
  Posted on 19 Oct 2011 at 5:21 am
  by Wilson, M., Moore, S.
  

  This paper provides a summary of a novel and integrated way to think about the item response models (most often used in measurement applications in social science areas such as psychology, educatio...

  (show all)

  This paper provides a summary of a novel and integrated way to think about the item response models (most often used in measurement applications in social science areas such as psychology, education, and especially testing of various kinds) from the viewpoint of the statistical theory of generalized linear and nonlinear mixed models. In addition, this new approach emphasizes how item response models can be coordinated and broadened to emphasize their explanatory uses beyond their standard descriptive uses. The basic explanatory principle is that item responses can be modeled as a function of qualities and features of various measurement contexts. These qualities and features can be: (a) characteristics of (i) items, (ii) persons, and (iii) combinations of items and persons; (b) observed or latent (of either items or persons); and (c) continuous or categorical. These ideas are exemplified in the context of a reading comprehension test. The paper starts with an introduction to the framework and then provides: (a) a description of the data that will be used to illustrate the new framework; (b) a discussion of data structure; (c) a brief description of the statistical approach we used; (d) a discussion of how the framework helps one to conceptualize existing item response models, linking the formal features of the models to substantive issues in the assessment of reading comprehension, as well as incorporating an example that goes beyond the usual range of item response models; and (e) a brief summary of further expansion.

  
  (show less)
  
  
• Diagnosing EAP writing ability using the Reduced Reparameterized Unified Model
  Posted on 19 Oct 2011 at 5:21 am
  by Kim, Y.-H.
  

  Despite the increasing interest in and need for test information for use in instructional practice and student learning, there have been few attempts to systematically link a diagnostic approach to...

  (show all)

  Despite the increasing interest in and need for test information for use in instructional practice and student learning, there have been few attempts to systematically link a diagnostic approach to English for academic purposes (EAP) writing instruction and assessment. In response to this need for research, this study examined the extent to which the diagnostic information generated by the Reduced Reparameterized Unified Model (Reduced RUM; Hartz, Roussos, & Stout, 2002) was a discriminant, accurate, and reliable method of determining student performance in EAP writing. Ten English as a second language (ESL) teachers assessed 480 Test of English as a Foreign Language™ Internet-based test (TOEFL® iBT ) independent essays using the Empirically-derived Descriptor-based Diagnostic (EDD) checklist, which consisted of 35 concrete, fine-grained descriptors. The resultant ratings were then analyzed using Arpeggio, the estimation software of the Reduced RUM. The findings showed that the skills diagnosis approach not only classified skill masters and non-masters accurately and reliably, but that it also had high discriminant function, with only a small number of students classified into flat profiles. However, concerns were raised with regard to the diagnostic power of some descriptors and the stability of one writing skill parameter (i.e., mechanics). The theoretical and pedagogical implications of the use of diagnostic assessment in EAP writing are discussed in this paper, as are suggestions for future research in applying psychometric diagnostic assessment models to performance assessment.

  
  (show less)
  
  
• Effects of test-taker characteristics and the number of participants in group oral tests
  Posted on 19 Oct 2011 at 5:21 am
  by Nakatsuhara, F.
  

  This study explores the nature of co-constructed interaction in group oral tests by examining whether a test-taker’s own and his or her group members’ extraversion levels and oral proficiency level...

  (show all)

  This study explores the nature of co-constructed interaction in group oral tests by examining whether a test-taker’s own and his or her group members’ extraversion levels and oral proficiency levels have different influences on conversational styles between two group sizes: groups of three and groups of four. Data were collected from 269 Japanese upper-secondary school students, who took group oral tests either in groups of three or four. All sessions were video-taped and transcribed following Conversation Analysis (CA) conventions. The data were quantitatively analysed in terms of goal-orientation, interactional contingency and quantitative dominance. Then, CA methodology was used to interpret and elaborate the statistical results. The findings have implications for our understanding of the group oral test construct and for appropriate choices of group size in group oral testing.

  
  (show less)
  
  
• Testing of second language pragmatics: Past and future
  Posted on 19 Oct 2011 at 5:21 am
  by Roever, C.
  

  Testing of second language pragmatic competence is an underexplored but growing area of second language assessment. Tests have focused on assessing learners’ sociopragmatic and pragmalinguistic abi...

  (show all)

  Testing of second language pragmatic competence is an underexplored but growing area of second language assessment. Tests have focused on assessing learners’ sociopragmatic and pragmalinguistic abilities but the speech act framework informing most current productive testing instruments in interlanguage pragmatics has been criticized for under-representing the construct. In particular, the assessment of learners’ ability to produce extended monologic and dialogic discourse is a missing component in existing assessments. This paper reviews existing tests and argues for a discursive re-orientation of pragmatics tests. Suggestions for tasks and scoring approaches to assess discursive abilities while maintaining practicality are provided, and the problematicity of native speaker benchmarking is discussed.

  
  (show less)
  
  
• Second language idiom learning: The effects of lexical knowledge and pedagogical sequencing
  Posted on 14 Sep 2011 at 10:54 am
  by Zyzik, E.
  

  This article examines the acquisition of Spanish idioms in a classroom setting that was supplemented with explicit instruction over a 10-week period. The research design manipulated two variables: ...

  (show all)

  This article examines the acquisition of Spanish idioms in a classroom setting that was supplemented with explicit instruction over a 10-week period. The research design manipulated two variables: prior lexical knowledge and idiom organization. Sixty-five second language (L2) learners completed pre- and posttests that measured their ability to recognize and produce the target idioms, as well as a vocabulary test to control for lexical knowledge. Participants in the experimental groups received contextualized idiom presentation that encouraged noticing, retrieving, and generating (Nation, 2001). The results indicate significant treatment effects, although no significant advantage was found for the thematic grouping of idioms. The results also show a significant effect for prior lexical knowledge on one of the dependent variables. These findings are discussed in relation to prior studies of idiom learning from a cognitive linguistics perspective (Boers et al., 2007) as well as psycholinguistic studies that emphasize the salience of literal meanings (Cieslicka, 2006).

  
  (show less)
  
  
• Curriculum Differentiation and Comprehensive School Reform: Challenges in Providing Educational Opportunity
  Posted on 21 Aug 2011 at 10:57 pm
  by Harris, D. M.
  

  Given the efforts of comprehensive school reform to improve the quality of educational opportunities for students by providing a standards based curriculum, this analysis examines the issue of trac...

  (show all)

  Given the efforts of comprehensive school reform to improve the quality of educational opportunities for students by providing a standards based curriculum, this analysis examines the issue of tracking and its implications regarding curriculum differentiation. Using data from middle schools involved with the comprehensive school reform model, America’s Choice, this mixed method analysis shows a complex manifestation of curriculum differentiation where schools had multiple types of academic tracks that responded to students differences including comprehensive learning groups, subject-specific groups, and temporary learning groups used for test preparation. Although school-level reports indicated that most regular education and gifted students were exposed to the America’s Choice English Language Arts and mathematics units, special education students and English Language learners were least likely to gain access to this content. Individual teacher responses to tracking were idiosyncratic where some modified the curriculum and instructional pace by learning group ultimately impacting the amount of content exposure where those in the top learning groups were able to cover greater material than those in lower groups. Even though some teachers resisted tracking by addressing the classroom issues that caused differences in curriculum and instruction, tracking persisted as an entrenched practice in these middle schools through preexisting academic groups and were also replicated in new learning groups that emerged to address student academic needs related to state testing demands.

  
  (show less)
  
  
• English educational policies of the U.S. Army Military Government in Korea from 1945 to 1948 and their effects on the development of English language teaching in Korea
  Posted on 28 Jul 2011 at 10:54 am
  

  Abstract  This study examines the English language teaching (ELT) policies and measures taken under the United States Army Military Government in Korea (USAMGIK) from 1945 to 1948, in an attempt t...

  (show all)

  Abstract  This study examines the English language teaching (ELT) policies and measures taken under the United States Army Military Government in Korea (USAMGIK) from 1945 to 1948, in an attempt to illuminate their implications on the current ELT in Korea. The study analyzes data derived from documents of the Korean and the U.S. governments, literature on Korean history, and literature on Korean education and language education. During the previous decade of Japanese colonial rule, the development of ELT had been stalled due to Japan?s war against the U.S. and its allies and the colonial government?s consequent suppression of Anglophone elements in the colony. With the arrival of the U.S. forces in 1945, however, the climate unfavorable to ELT reversed. The U.S. military government took various educational measures to enhance ELT. It reintroduced English as a required subject in the secondary-school curriculum and as a key subject in university entrance exams. It also established a state-run English language school and provided teacher training nationwide through the school. Despite these reform measures, however, Korean ELT maintained core characteristics set earlier during Japanese colonial rule.

  Content Type Journal ArticleCategory Original PaperPages 193-220DOI 10.1007/s10993-011-9204-9Authors Eun Gyong Kim, Department of Humanities and Social Sciences, Korea Advanced Institute of Science and Technology (KAIST), 335 Gwahangno, Yuseong-gu, Daejeon, 305-701 Republic of Korea Journal Language PolicyOnline ISSN 1573-1863Print ISSN 1568-4555 Journal Volume Volume 10 Journal Issue Volume 10, Number 3
  (show less)
  
  
• Standards-based classroom assessments of English proficiency: A review of issues, current developments, and future directions for research
  Posted on 27 Jul 2011 at 5:41 am
  by Llosa, L.
  

  With the United States’ adoption of a standards-based approach to education, most attention has focused on the large-scale, high-stakes assessments intended to measure students’ mastery of standard...

  (show all)

  With the United States’ adoption of a standards-based approach to education, most attention has focused on the large-scale, high-stakes assessments intended to measure students’ mastery of standards for accountability purposes. Less attention has been paid to the role of standards-based assessments in the classroom. The purpose of this paper is to discuss key issues and challenges related to the use of standards-based classroom assessments to assess English language learners’ English proficiency. First, the paper describes a study of a standards-based classroom assessment of English proficiency in a large urban school district in California. Second, using this study as an example and drawing from the literature in language testing on classroom assessment, this paper highlights the major issues and challenges involved in using English proficiency standards as the basis for classroom assessment. Finally, the article outlines a research agenda for the field given current developments in the areas of English proficiency standards and classroom assessment.

  
  (show less)
  
  
• Book review: K. Menken (2008). English learners left behind: Standardized testing as Language Policy. Clevedon, UK: Multilingual Matters. 216 pp. ISBN 9781853599972, $39.95 (pbk)
  Posted on 27 Jul 2011 at 5:41 am
  by Burke, A. M.
  
  
• Do current English language development/proficiency standards reflect the English needed for success in school?
  Posted on 27 Jul 2011 at 5:41 am
  by Bailey, A. L., Huang, B. H.
  

  English language development or proficiency (ELD/P) standards promise to play an important role in the instruction and assessment of the language development of English language learner (ELL) pre-K...

  (show all)

  English language development or proficiency (ELD/P) standards promise to play an important role in the instruction and assessment of the language development of English language learner (ELL) pre-K-12 students, but to do so effectively they must convey the progression of student language learning in authentic school contexts for authentic academic purposes. The construct of academic English is defined as the vocabulary, sentence structures, and discourse associated with language used to teach academic content as well as the language used to navigate the school setting more generally. The construct definition is informed by a relatively modest number of empirical studies of textbooks, content assessments, and observations of classroom discourse. The standards of a state with a large ELL population and a large multi-state consortium are then reviewed to illustrate the role of the academic English construct in the standards’ coverage of language modalities or domains, levels of attainment or proficiency, grade spans, and the needs of the large number of young English learners. Recommendations and potential strategies for validating, creating, and augmenting standards that reflect authentic uses of academic language in school settings are also made.

  
  (show less)
  
  
• Oral translation as a test accommodation for ELLs
  Posted on 27 Jul 2011 at 5:41 am
  by Stansfield, C. W.
  

  The No Child Left Behind (NCLB) Act (US Government, 2001), the current iteration of the Elementary and Secondary Education Act of 1965, makes it clear that states, districts, schools and teachers a...

  (show all)

  The No Child Left Behind (NCLB) Act (US Government, 2001), the current iteration of the Elementary and Secondary Education Act of 1965, makes it clear that states, districts, schools and teachers are accountable for the education of English language learners (ELLs), as well as all other students. To implement an accountability system, NCLB requires states to create content standards (a set of curricular goals and objectives) for English language arts, mathematics, and science. They must also develop and administer assessments that measure student progress toward mastery of these content standards. Finally, states, districts, and schools must demonstrate that they are making adequate yearly progress toward bringing all students to a previously established level of mastery of these standards (called Proficient) with the goal that by the year 2014 all students will achieve this level.

  
  (show less)
  
  
• Issues in vertical scaling of a K-12 English language proficiency test
  Posted on 27 Jul 2011 at 5:41 am
  by Kenyon, D. M., MacGregor, D., Li, D., Cook, H. G.
  

  One of the mandates of the No Child Left Behind Act is that states show adequate yearly progress in their English language learners’ (ELLs) acquisition of English language proficiency. States are r...

  (show all)

  One of the mandates of the No Child Left Behind Act is that states show adequate yearly progress in their English language learners’ (ELLs) acquisition of English language proficiency. States are required to assess ELLs’ English language proficiency annually in four language domains (listening, reading, writing, and speaking) to measure their progress; they are also required to report on a composite comprehension measure. Often the clearest way to effectively monitor students’ progress is to measure assessment results across grades on the same scale. In measurement terms, scores from tests across all grade levels can be put on the same scale using vertical scaling. In addition, to help stakeholders understand and interpret the results, these scale scores are often interpreted in terms of proficiency levels. In this article, we use the vertical scaling of WIDA ACCESS for ELLs®, a large-scale K-12 Academic English Language Proficiency assessment to illustrate measurement and practical issues involved in this technique. We first give background on the need for vertical scaling. We then assess the literature on vertical scaling and describe the procedures used for WIDA ACCESS for ELLs® to vertically scale test scores and interpret the results in terms of the WIDA ACCESS for ELLs® Proficiency Scale. Next we review several studies that have been conducted to gauge the effectiveness of that scaling. We end the paper with a discussion of the broad issues that arise from vertical scaling.

  
  (show less)
  
  
• Accountability-assessment under No Child Left Behind: Agenda, practice, and future
  Posted on 27 Jul 2011 at 5:41 am
  by Deville, C., Chalhoub-Deville, M.
  
  
• Testing English language learners under No Child Left Behind
  Posted on 27 Jul 2011 at 5:41 am
  by Bunch, M. B.
  

  Title III of Public Law 107-110 (No Child Left Behind; NCLB) provided for creation of assessments of English language learners (ELLs) and established, through the Enhanced Assessment Grant program,...

  (show all)

  Title III of Public Law 107-110 (No Child Left Behind; NCLB) provided for creation of assessments of English language learners (ELLs) and established, through the Enhanced Assessment Grant program, a platform from which four consortia of states developed ELL tests aligned to rigorous statewide content standards. Those four tests (ACCESS for ELLs, CELLA, ELDA, and MWA) are now in use in one or more states, along with a host of other commercially available or locally developed tests. The tests (those developed by consortia as well as the others) are quite similar in many ways, principally in their contents: Listening, Speaking, Reading, and Writing. Most measure these domains with a combination of multiple-choice (MC) and open-ended (OE) test items. This article provides an overview to the four consortium-developed tests as well as an in-depth analysis of one representative example. It also provides a summary of the characteristics of four commercially available tests. Not surprisingly, the four commercially available tests are rather similar to one another and to the consortium-developed tests in terms of content, psychometric characteristics, and development. The primary difference between the two sets is that the commercially available tests tend to report percentile ranks as well as proficiency levels. Now that the Race to the Top program is in place, we face many of the same challenges we faced a decade ago when NCLB was passed. While the Enhanced Assessment Grant competition emphasized summative assessment, the latest competition emphasizes formative assessment, which gives rise to the hope that educators can not only discover students’ strengths and weaknesses with these new tests, but do so in a timely manner and have opportunities to use the information constructively. Current work by at least one organization is encouraging in this regard.

  
  (show less)
  
  
• Developing a Framework to Measure Process-oriented Writing Competence: A Case of Vietnamese EFL Students' Formal Portfolio Assessment
  Posted on 30 Jun 2011 at 11:38 am
  by Mai Thu Duong, , Nguyen Thi Kim Cuc, , Griffin, P.
  

  Portfolios have gained great popularity as a method of process-oriented writing assessment in the last 20 years. Despite the apparent advantages, as with other new methods of assessment, they still...

  (show all)

  Portfolios have gained great popularity as a method of process-oriented writing assessment in the last 20 years. Despite the apparent advantages, as with other new methods of assessment, they still raise many controversies, particularly those regarding the validity of their contents and assessment. This article describes the first stages in the process of developing the framework for assessing the performance in writing portfolios of students who speak English as a foreign language at Vietnam National University (HULIS-VNU). One of the aims is to argue for the validity of the portfolio writing competence as a construct for use in formal assessment. In terms of methodology, developmental assessment theories are integrated in a five-stage process of instrument development, which integrates qualitative and quantitative research techniques. The completion of the first two stages of the study reveals that 36 indicators, which are specifiable into 138 quality criteria, are applicable in measuring portfolio writing competence. Further validation stages are being implemented to examine these results, but the initial findings have established the assessment framework for the construct. The results indicate the feasibility of designing assessment instruments which encompass both product-oriented and process-oriented features of the writing job for formal assessment purposes and achieving a higher efficiency for performance assessment in general.

  
  (show less)
  
  
• Competencies Needed in Oral Communication in English among Thai Undergraduate Public Relations Students: A Substantial Gap between Expectations and Reality
  Posted on 30 Jun 2011 at 11:38 am
  by Pattanapichet, F., Chinokul, S.
  

  This article investigates the competencies needed for oral communication in English among Thai undergraduate public relations students for handling public relations job interviews and performing en...

  (show all)

  This article investigates the competencies needed for oral communication in English among Thai undergraduate public relations students for handling public relations job interviews and performing entry-level public relations work. To identify these competencies, the study identified and involved all of the stakeholders in the data reliability assessment process. The DACUM process, a research-based methodology of developing a curriculum, was modified through interviews with experts in the field to compile and validate the list of the competencies needed. As a result, the validated list consists of four competencies needed for public relations job interviews and eight competencies needed for entry-level public relations work. For triangulation purposes, a questionnaire survey on 222 public relations students and semi-structured interviews with a focus group were conducted to explore the students’ perspectives. The article discusses a huge gap between expectations and the real-world situation and provides suggestions to help bridge this gap.

  
  (show less)
  
  
• Structural Priming and Second Language Learning
  Posted on 16 Jun 2011 at 12:57 pm
  by Jeong-Ah Shin, Kiel Christianson
  
  Structural priming (or syntactic priming) is a speaker's tendency to reuse the same structural pattern as one that was previously encountered (Bock, 1986). This study investigated (a) whether the i... (show all)
  Structural priming (or syntactic priming) is a speaker's tendency to reuse the same structural pattern as one that was previously encountered (Bock, 1986). This study investigated (a) whether the implicit learning processes involved in long-lag structural priming lead to differential second language (L2) improvement in producing two structural types (complex, double-object dative and simple, separated phrasal-verb structures) compared to more explicit memory processes involved in no-lag structural priming and (b) whether additional explicit instruction leads to increased production of target structures than either implicit learning or explicit memory processes alone. Learners showed an overall increase in target structure production in a picture description task and marginal improvement in grammaticality judgment tests after the structural priming session. Results revealed that explicit instruction combined with structural priming speeded short-term improvement more than implicit instruction involving implicit learning alone in the form of long-lag structural priming. However, only implicit learning via long-lag structural priming resulted in increased production of the complex structure during a second testing session 1 day later. This study is the first to directly compare explicit instruction to implicit instruction in a structural priming paradigm, taking into account both the complexity of structures and the long-term effects of instruction on L2 production.
  (show less)
  
  
• Research on academic literacy development in sheltered instruction classrooms
  Posted on 14 Jun 2011 at 10:46 am
  by Short, D. J., Echevarria, J., Richards-Tutor, C.
  

  This article describes an extended program of research in sheltered instruction and the effects on the academic literacy development of English language learners. It also highlights the challenges ...

  (show all)

  This article describes an extended program of research in sheltered instruction and the effects on the academic literacy development of English language learners. It also highlights the challenges of scaling up an instructional intervention. The intervention was the Sheltered Instruction Observation Protocol (SIOP) Model, an approach that teaches subject area curriculum to students learning through a second language using techniques that make the content material accessible and also help develop the students’ second language skills. Three successive studies looked at teacher change over time and student achievement on standardized assessments and researcher-developed measures. Results of the three studies reveal that students with teachers who were trained in the SIOP Model of sheltered instruction and implemented it with fidelity performed significantly better on assessments of academic language and literacy than students with teachers who were not trained in the model. The article offers guidance for strengthening professional development for teachers so the quality of instruction they deliver to English language learners improves and the students strengthen their English language and academic outcomes.

  
  (show less)
  
  
• English Language Learners and Academic Achievement: Revisiting the Threshold Hypothesis
  Posted on 25 May 2011 at 11:26 am
  by Yuliya Ardasheva, Thomas R. Tretter, Marti Kinny
  
  This nonexperimental study explored the predictive strength of English proficiency levels on academic achievement of middle school students in a sample of 17,470 native English-speaking (NES) stude... (show all)
  This nonexperimental study explored the predictive strength of English proficiency levels on academic achievement of middle school students in a sample of 17,470 native English-speaking (NES) students, 558 English language learners (current ELLs), and 500 redesignated fluent English proficient students (former ELLs). Results of multilevel analyses indicated that after controlling for relevant student- and school-level characteristics, former ELLs significantly outperformed current ELL and NES students in reading (effect sizes: 1.07 and 0.52) and mathematics (effect sizes: 0.86 and 0.42). The results support Cummins's (1979, 2000) lower level threshold hypothesis predicting that upon reaching adequate proficiency in the language of schooling and testing, ELLs would no longer experience academic disadvantages. Refinements for the theory and directions for future research are discussed.
  (show less)
  
  
• Reviewing evidence-based practice for pupils with dyslexia and literacy difficulties
  Posted on 18 May 2011 at 8:40 am
  by Yvonne Griffiths, Morag Stuart
  
  There is now a strong evidence base from theory and research providing a ?template? to inform practice at Wave 2, guiding the design and implementation of time-limited effective early intervention ... (show all)
  There is now a strong evidence base from theory and research providing a ?template? to inform practice at Wave 2, guiding the design and implementation of time-limited effective early intervention programmes for pupils identified as ?at risk? of reading difficulties following initial literacy instruction (Rose, 2009). In contrast, there is currently an absence of evidence to guide the more fine-grained selection and design of specific interventions (Wave 3) for those nonresponders to otherwise effective reading intervention. In this paper, we first outline our current understanding of the characteristics of effective early intervention programmes, and of the pupils who do not respond. Three theoretically motivated single-case studies from developmental theory and processing models of skilled performance are then considered as a source of evidence providing useful insights into the type of assessment needed to inform the planning of highly individualised intervention programmes for pupils with severe and persisting literacy difficulties.
  (show less)
  
  
• Just-in-time pedagogy: teachers' perspectives on the response to intervention framework
  Posted on 17 May 2011 at 10:13 am
  by Kathleen A. Wilcox, Elizabeth Murakami-Ramalho, Angela Urick
  
  The purpose of this mixed methods research is to examine teachers' perspectives on the response to intervention (RTI) framework and its implementation in Michigan and Texas schools. Both states hav... (show all)
  The purpose of this mixed methods research is to examine teachers' perspectives on the response to intervention (RTI) framework and its implementation in Michigan and Texas schools. Both states have been leaders in literacy, increasing preservice and in-service teacher certification standards and developing similar batteries for assessing literacy skills. Using the International Reading Association's RTI principles, the following question directed this inquiry: what are the perspectives of teachers in various educational, geographic, economic and cultural settings of the RTI? The research was developed through questionnaires, focus groups and semi-structured interviews. Findings revealed that teacher professional development, assessments and collaboration for instruction were highly integrated themes when developing RTI strategies as reforming practice and increasing student literacy. Michigan and Texas teachers were more confident and comfortable in measuring and identifying students with reading difficulties over their ability to prevent learning disabilities through their instruction.
  (show less)
  
  
• The role of background knowledge in text comprehension for children learning English as an additional language
  Posted on 17 May 2011 at 10:12 am
  by Kelly Burgoyne, Helen E. Whiteley, Jane M. Hutchinson
  
  Children learning English as an additional language (EAL) often experience difficulties with reading comprehension relative to their monolingual peers. While low levels of vocabulary appear to be o... (show all)
  Children learning English as an additional language (EAL) often experience difficulties with reading comprehension relative to their monolingual peers. While low levels of vocabulary appear to be one factor underlying these difficulties, other factors such as a relative lack of appropriate background knowledge may also contribute. Sixteen children learning EAL and 16 of their monolingual peers, matched for word reading accuracy, were assessed using a standard measure of reading comprehension and an experimental measure of reading comprehension for which relevant background knowledge was taught before assessing understanding. Tests of receptive and expressive vocabulary were also completed. Results confirmed lower levels of reading comprehension for children learning EAL for both standard and ?background? controlled measures. Analysis of comprehension by question type on the experimental measure showed that while both groups made use of taught knowledge to answer inferential questions, children learning EAL had specific difficulties with both literal questions and questions requiring the interpretation of a simile. It is suggested that relevant background information should be used to facilitate children's text comprehension. Furthermore, several factors, especially vocabulary differences, but also text search strategies, context use and comprehension monitoring skills, may contribute to the comprehension difficulties experienced by children learning EAL.
  (show less)
  
  
• Comprehension of written sentences as a core component of children's reading comprehension
  Posted on 6 May 2011 at 4:18 am
  by Jean Ecalle, Houria Bouchafa, Anna Potocki, Annie Magnan
  
  Two experiments were conducted to test the hypothesis that sentence processing is an essential mediatory skill between word recognition and text comprehension in reading. In Experiment 1, a semanti... (show all)
  Two experiments were conducted to test the hypothesis that sentence processing is an essential mediatory skill between word recognition and text comprehension in reading. In Experiment 1, a semantic similarity judgement task was used with children from Grade 2 to Grade 9. They had to say whether two written sentences had the same (or very similar) meanings or whether the meanings of the two sentences were very different. As expected, performance improved with age both on the high-frequency words and with increasingly complex sentences. In Experiment 2 with children in Grade 3, scores in written sentence comprehension and vocabulary made the most important unique contribution to the reading comprehension of an expository text. The results are discussed first, in the light of a general framework in which sentence-level skill is at the core of reading comprehension and second, with reference to the issue of reading assessment.
  (show less)
  
  
• The Impact of Induction and Mentoring Programs for Beginning Teachers: A Critical Review of the Research
  Posted on 5 May 2011 at 12:50 pm
  by Ingersoll, R. M., Strong, M.
  

  This review critically examines 15 empirical studies, conducted since the mid-1980s, on the effects of support, guidance, and orientation programs—collectively known as induction—for beginning teac...

  (show all)

  This review critically examines 15 empirical studies, conducted since the mid-1980s, on the effects of support, guidance, and orientation programs—collectively known as induction—for beginning teachers. Most of the studies reviewed provide empirical support for the claim that support and assistance for beginning teachers have a positive impact on three sets of outcomes: teacher commitment and retention, teacher classroom instructional practices, and student achievement. Of the studies on commitment and retention, most showed that beginning teachers who participated in induction showed positive impacts. For classroom instructional practices, the majority of studies reviewed showed that beginning teachers who participated in some kind of induction performed better at various aspects of teaching, such as keeping students on task, using effective student questioning practices, adjusting classroom activities to meet students’ interests, maintaining a positive classroom atmosphere, and demonstrating successful classroom management. For student achievement, almost all of the studies showed that students of beginning teachers who participated in induction had higher scores, or gains, on academic achievement tests. There were, however, exceptions to this overall pattern—in particular a large randomized controlled trial of induction in a sample of large, urban, low-income schools—which found some significant positive effects on student achievement but no effects on either teacher retention or teachers’ classroom practices. The review closes by attempting to reconcile these contradictory findings and by identifying gaps in the research base and relevant questions that have not been addressed and warrant further research.

  
  (show less)
  
  
• The TORGO database of acoustic and articulatory speech from speakers with dysarthria
  Posted on 25 Mar 2011 at 1:55 pm
  

  Abstract  This paper describes the acquisition of a new database of dysarthric speech in terms of aligned acoustics and articulatory data. This database currently includes data from seven individu...

  (show all)

  Abstract  This paper describes the acquisition of a new database of dysarthric speech in terms of aligned acoustics and articulatory data. This database currently includes data from seven individuals with speech impediments caused by cerebral palsy or amyotrophic lateral sclerosis and age- and gender-matched control subjects. Each of the individuals with speech impediments are given standardized assessments of speech-motor function by a speech-language pathologist. Acoustic data is obtained by one head-mounted and one directional microphone. Articulatory data is obtained by electromagnetic articulography, which allows the measurement of the tongue and other articulators during speech, and by 3D reconstruction from binocular video sequences. The stimuli are obtained from a variety of sources including the TIMIT database, lists of identified phonetic contrasts, and assessments of speech intelligibility. This paper also includes some analysis as to how dysarthric speech differs from non-dysarthric speech according to features such as length of phonemes, and pronunciation errors.

  Content Type Journal ArticleCategory Original PaperPages 1-19DOI 10.1007/s10579-011-9145-0Authors Frank Rudzicz, Department of Computer Science, University of Toronto, Toronto, ON CanadaAravind Kumar Namasivayam, The Speech and Stuttering Institute, Toronto, ON CanadaTalya Wolff, Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON Canada Journal Language Resources and EvaluationOnline ISSN 1574-0218Print ISSN 1574-020X
  (show less)
  
  
• A comparative study of the effects of input-based and production-based instruction on vocabulary acquisition by young EFL learners
  Posted on 14 Mar 2011 at 12:15 pm
  by Shintani, N.
  

  The study reported in this article investigated the comparative effects of two types of treatment — one of which emphasized input and the other output — on the vocabulary acquisition of young EFL l...

  (show all)

  The study reported in this article investigated the comparative effects of two types of treatment — one of which emphasized input and the other output — on the vocabulary acquisition of young EFL learners. In the input-based instruction, the students were not required to produce output whereas in the production-based instruction the students were required to produce output. Thirty-six Japanese children aged 6—8 were divided into three groups (input-based, production-based and control group), received six weeks instruction and took four types of vocabulary tests as a pre-, post- and delayed post-test. The findings provide further evidence that both input-based and production-based instruction lead to both receptive and productive vocabulary knowledge. In general, the results show similar levels of effects for input-based and production-based instruction on vocabulary acquisition. However, an examination of process features indicates that the input-based tasks provided opportunities for richer interaction for the learners than the production-based activities. This may explain the better performance of the input-based group on the task-based comprehension test and the same levels of achievement in the production tests despite relatively fewer opportunities for second language (L2) production.

  
  (show less)
  
  
• Volume 31
  Posted on 28 Feb 2011 at 6:00 pm
  
  Annual Review of Applied Linguistics, Volume 31 Annual Review of Applied Linguistics reviews research in key areas in the broad field of applied linguistics. Each issue is thematic, covering... (show all)
  Annual Review of Applied Linguistics, Volume 31 Annual Review of Applied Linguistics reviews research in key areas in the broad field of applied linguistics. Each issue is thematic, covering the topic by means of critical summaries, overviews and bibliographic citations. Every fourth or fifth issue surveys applied linguistics broadly, offering timely essays on language learning and pedagogy, discourse analysis, teaching innovations, second language acquisition, computer-assisted instruction, language use in professional contexts, sociolinguistics, language policy, and language assessment, to name just a few of the areas reviewed. It provides over 500 new citations each year.
  (show less)
  
  
• A Bilingual Vocabulary Size Test of English for Vietnamese Learners
  Posted on 28 Feb 2011 at 12:23 pm
  by Le Thi Cam Nguyen, , Nation, P.
  

  This article describes the development and validation of a Vietnamese bilingual version of the Vocabulary Size Test — a test which measures written receptive vocabulary size. The test can be used t...

  (show all)

  This article describes the development and validation of a Vietnamese bilingual version of the Vocabulary Size Test — a test which measures written receptive vocabulary size. The test can be used to measure the English vocabulary size of Vietnamese learners of English. A learner’s total vocabulary size is calculated by multiplying their test result by 100. The research adds to our knowledge of vocabulary size testing in the following ways. First, it shows that a bilingual version of a monolingual test performs in much the same way as the monolingual test, distinguishing learners of different proficiency levels and returning lower scores at later levels of the test. Second, it shows that every level of the test should be sat, otherwise there will be a considerable underestimation of learners’ vocabulary sizes. This suggests limitations to the previously accepted assumption that learners’ vocabulary growth can be largely related to word frequency. Third, it shows that bilingual tests are feasible alternatives to more challenging and time-consuming monolingual tests.

  
  (show less)
  
  
• Understanding Discrepancies in Rater Judgement on National-Level Oral Examination Tasks
  Posted on 28 Feb 2011 at 12:23 pm
  by Ang-Aw, H. T., Chuen Meng Goh, C.
  

  The oral examination is an important component of the high-stakes ‘O’ Level examination in Singapore taken by 16-17 year olds whose first language may or may not be English. In spite of this, there...

  (show all)

  The oral examination is an important component of the high-stakes ‘O’ Level examination in Singapore taken by 16-17 year olds whose first language may or may not be English. In spite of this, there has been sparse research into the examination. This paper reports findings of an exploratory study which attempted to determine whether there were any discrepancies in rater judgements and thereafter, explore the nature and scope of the discrepancies identified. Four audio recordings were obtained from a simulated oral examination of four candidates conducted by a trained ‘O’ Level oral examiner. Seven other trained ‘O’ Level oral examiners were asked to rate the recordings individually and provide concurrent verbal reports. Questionnaires were also given to the raters for data triangulation after the verbalization. The data were analysed through Verbal Protocol Analysis and descriptive statistics. Rater discrepancies detected in the scores were qualitatively determined to be due to four differences: emphases on factors assessed, constructs of oral proficiency, rater interpretations and approaches in assessment. These findings provide valuable insights into raters’ perceptions of the construct of speaking and offer implications for rater training and the development of rating scales.

  
  (show less)
  
  
• Impact of early code-skill and oral-comprehension training on reading achievement in first grade
  Posted on 24 Jan 2011 at 8:35 am
  by Maryse Bianco, Catherine Pellenq, Eric Lambert, Pascal Bressoux, Laurent Lima, Anne-Lise Doyen
  
  In a 3-year longitudinal study, we examined the relationships between oral language development, early training and reading acquisition on word-identification and reading-comprehension tests admini... (show all)
  In a 3-year longitudinal study, we examined the relationships between oral language development, early training and reading acquisition on word-identification and reading-comprehension tests administered to a sample of 687 French children. Hierarchical linear models showed that both phonological awareness and oral comprehension at the age of 4 years were relevant to reading acquisition 2 years later. These two broad skills explained separate parts of the variance on both outcome measures, while revealing opposite effects: phonological skills explained more variance for alphabetic reading skills and oral comprehension explained more variance for reading comprehension. We also assessed the effects of two preschool training programmes focusing on either phonological awareness or comprehension skills. The results showed that phonological awareness training had a positive effect on alphabetic scores, and comprehension training had a positive effect on reading comprehension. These results provide insight into early oral instruction and contribute to the theoretical debate about the linguistic predictors of literacy acquisition.
  (show less)
  
  
• Models of power and the deletion of participation in a classroom literacy event
  Posted on 7 Dec 2010 at 3:42 am
  by Gloria E. Jacobs
  
  This article uses the analytic lens of power relations and models of power to examine how the participation of two students in an urban US eighth-grade English language arts class was symbolically ... (show all)
  This article uses the analytic lens of power relations and models of power to examine how the participation of two students in an urban US eighth-grade English language arts class was symbolically and literally deleted. The researcher asks why do some students fail despite being placed in a technologically rich educational environment that ostensibly draws on best practices? Drawing on data collected during an academic year, the author uses a telling case of a dyad engaged in on-line research and the creation of a PowerPoint presentation to illustrate how the students try to participate and complete the assignment but experience failure. The author raises questions about the context of the students' lives as well as the nature of the classroom instruction and assessment structure. It is argued that further research into the nature of participation is needed to address the lack of student success in urban schools.
  (show less)