bright
bright
bright
bright
bright
tl
 
tr
lunder
Recent Content of Assessing Writing
This site designed and maintained by
Prof. Glenn Fulcher

@languagetesting.info

runder
lcunder
rcunder
lnavl
Navigation
lnavr
navtop
lnavs
rnavs
lnavs
rnavs
lnavs
rnavs
lnavs
rnavs
lnavs
rnavs
bottomnav
 
 

 

 
Titles and Abstracts
     
 
ScienceDirect Publication: Assessing Writing

Analysis of syntactic complexity in secondary education ELF writers at different proficiency levels
Publication date: January 2018
Source:Assessing Writing, Volume 35
Author(s): Ana Cristina Lahuerta Martínez
The present study examines differences in syntactic complexity in English writing among lower intermediate and intermediate secondary education writers by means of quantitative measures of syntactic complexity, and compares the scores on the selected syntactic complexity measures with holistic ratings of learners? overall writing quality. We examined the writing of 188 students at years 3 (lower intermediate) and 4 (intermediate) of secondary education including gender in the analysis. Essays were evaluated by holistic ratings of writing quality and quantitative measures gauging complexification at the sentential, the clausal, and the phrasal level of syntactic organisation. Data revealed significant strong correlations between the holistic ratings and all but one of the complexity metrics. The scores on the general quality of the writings and on all syntactic complexity measures increased from grade 3 to grade 4 and for all but one sentential complexity measure (compound-complex sentence ratio) the increase was statistically significant. Girls obtained a higher score in the general quality of the compositions and in all the measures examined, and for four measures the difference in score was significant.



From independent ratings to communal ratings: A study of CWA raters? decision-making behaviors
Publication date: January 2018
Source:Assessing Writing, Volume 35
Author(s): Vivian Lindhardsen
The present exploratory study maps the decision-making behaviors of raters in a well-established communal writing assessment (CWA) context, tracing their behaviors all the way from independent rating sessions, where initial images and judgments are formed, to communal rating sessions, where final scores are assigned on the basis of collaboration between two raters. Results from think-aloud protocols, recorded discussions, and retrospective reports from 20 experienced raters rating 15 EFL essays showed that when moving from independent ratings to communal ratings, raters gradually refined their assessments and balanced their attention more evenly among the official assessment criteria to reach what they believed to be more accurate scores. These interpretations support a hermeneutic rather than a psychometric approach to establishing the validity of the CWA practices.



Effects of indirect coded corrective feedback with and without short affective teacher comments on L2 writing performance, learner uptake and motivation
Publication date: January 2018
Source:Assessing Writing, Volume 35
Author(s): Chiachieh Tang, Yeu-Ting Liu
Though studies have shown the benefits of oral corrective feedback (CF), there is a paucity of research exploring the potency of indirect written CF. Studies have indicated the need of further research on indirect written CF and teacher comments as a way to encourage L2 learners to be better writers. To this end, this study investigated if indirect coded correction feedback (ICCF) and short affective comments were more effective than ICCF alone in enhancing L2 learners? writing performance, uptake, and motivation. L2 learner participants (n?=?56) received the two aforementioned feedback modes and completed three writing tasks at successive times. Analyses of the writings showed a significant improvement in overall writing performance and learner uptake irrespective of the feedback mode they received. This seems to indicate that adding affective comments to ICCF did not significantly boost L2 learners? writing; however, further analysis of the participants? questionnaire data showed that the addition did foster a positive mindset motivating them to take further actions to improve their writing and that the pedagogical potency of ICCF and short affective comments seems to be complementary. Pedagogical implications and applications for how ICCF and short affective teacher comments can impact L2 writing are provided.
Graphical abstract


Miroslaw Pawlak (ed.) Error Correction in the Foreign Language Classroom: Reconsidering the Issues. Springer-Verlag, Berlin, Heidelberg (2015).
Publication date: Available online 12 December 2017
Source:Assessing Writing
Author(s): Shima Ghahari




Ed.Board/Aims and scope
Publication date: October 2017
Source:Assessing Writing, Volume 34





EDITORIAL FOR ASSESSING WRITING VOL 34
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Liz Hamp-Lyons




Exploring the relationship between textual characteristics and rating quality in rater-mediated writing assessments: An illustration with L1 and L2 writing assessments
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Stefanie A. Wind, Catanya Stager, Yogendra J. Patil
Numerous researchers have explored the degree to which specific textual characteristics of student compositions are associated with high and low ratings, as well as differences in these relationships across subgroups of students (e.g., English language learners). These studies provide insight into rater judgments and the development of writing proficiency. However, the degree to which textual characteristics are associated with the psychometric quality of ratings is relatively unexplored. This study illustrates a procedure for exploring the influence of textual characteristics of essays on rating quality in the context of rater-mediated writing performance assessments in order to gain a more-complete understanding of rating quality. Two illustrative datasets are used that reflect writing assessments for native English speakers and English language learners. The CohMetrix software program was used to obtain measures of textual characteristics, and the Partial Credit model was used to obtain indicators of rating quality. The relationship between essay features and rating quality was explored using correlation and profile analyses. Results suggested that rating quality varies across essays with different features, and the relationship between rating quality and essay features is unique to individual writing assessments. Implications are discussed as they relate to research and practice for rater-mediated writing assessments.



Automated formative writing assessment using a levels of language framework
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Joshua Wilson, Rod Roscoe, Yusra Ahmed
This study investigates a novel approach to conducting formative writing assessment that involves evaluating students' writing skills across three levels of language (word, sentence, and discourse) using automated measures of word choice, syntax, and cohesion. Writing from students in Grades 6 and 8 (n=240 each) was analyzed using Coh-Metrix. Multigroup confirmatory factor analysis evaluated a hypothesized three factor levels of language model, and multigroup structural equation modeling determined if these factors predicted performance on a state writing achievement test comprised of a Direct Assessment of Writing (DAW) and an Editing and Revising test (ER). Results indicated that a subset of 9 Coh-Metrix measures successfully modeled three latent levels of language factors at each grade level. Results also indicated that the DAW test was predicted by the latent Discourse factor and the ER test was predicted by the latent Discourse and Sentence factors. Findings provide a proof of concept for automated formative assessment using a levels of language framework. Furthermore, although not the primary goal of the study, results may lay the groundwork for new levels of language detection algorithms that could be incorporated within automated writing evaluation software programs to expand automated+teacher assessment and feedback approaches.



Assessing C2 writing ability on the Certificate of English Language Proficiency: Rater and examinee age effects
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Daniel R. Isbell
Differentiating between advanced L2 writers at the higher levels of the Common European Framework of Reference (CEFR) presents a challenge in assessment. The distinction between descriptors at the C1 and C2 levels are fine-grained, and even native speakers of a language may not consistently achieve them. At the same time, the CEFR has generally been conceived with the language abilities and contexts of use of adults in mind, thus making CEFR-based interpretations of young language learner?s abilities problematic. This study examines two issues in the assessment of C2-level writing in the context of the Certificate of English Language Proficiency (CELP) writing task: rater effects and examinee age. Interrater reliability and many-facet Rasch analysis showed that raters varied substantially in severity. CELP scoring procedures for rater disagreement partially mitigated severity differences. Contrary to expectations, age differentiated examinee abilities minimally and defied hypothesized ordering (i.e., that writing ability would increase with age). Additionally, some raters were found to demonstrate bias towards the youngest examinees. Specific implications for the CELP?s validity argument and broader implications for assessing young writers in CEFR terms are discussed.



Integrating assessment as, for, and of learning in a large-scale exam preparation course
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Karim Sadeghi, Teymour Rahmati
This empirical study examined the validity of arguments regarding assessment integration tensions, strategies, and the potential of an integrated assessment model in enhancing students? writing ability. To this end, an integrated assessment as, for, and of learning model was experimented with a group of learners preparing to take the Cambridge English: Preliminary English Test. Moreover, an assessment for and of (non-integrated) model was used with another group of candidates as the control group. Subsequently, the candidates? writing performances measured by Cambridge Assessment in terms of overall band descriptions were converted into numerical indices. The Mann-Whitney U Test comparison of the participants? converted scores revealed that the integrated assessment group performed better than the non-integrated assessment candidates. Furthermore, classroom observations and a focus-group interview with the integrated assessment group clarified a number of issues concerning assessment integration and AaL implementation tensions and strategies. The results indicated that an integrated assessment model tailored to contextual specifications can contribute both theoretically and practically to teaching and assessing writing.



Design and evaluation of automated writing evaluation models: Relationships with writing in naturalistic settings
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Brent Bridgeman, Chaitanya Ramineni
Automated Writing Evaluation (AWE)systems are built by extracting features from a 30min essay and using a statistical model that weights those features to optimally predict human scores on the 30min essays. But the goal of AWE should be to predict performance in real world naturalistic tasks, not just to predict human scores on 30min essays. Therefore, a more meaningful way of creating the feature weights in the AWE model is to select weights that are optimized to predict the real world criterion. This unique new approach was used in a sample of 194 graduate students who supplied two examples of their writing from required graduate school coursework. Contrary to results from a prior study predicting portfolio scores, the experimental model was no more effective than the traditional model in predicting scores on actual writing done in graduate school. Importantly, when the new weights were evaluated in large samples of international students, the population subgroups that were advantaged or disadvantaged by the new weights were different from the groups advantaged/disadvantaged by the traditional weights. It is critically important for any developer of AWE models to recognize that models that are equally effective in predicting an external criterion may advantage/disadvantage different groups.



College student perceptions of writing errors, text quality, and author characteristics
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Adam C. Johnson, Joshua Wilson, Rod D. Roscoe
Both conventional wisdom and empirical research suggest that errors in writing impact perceptions of both writing quality and characteristics of the author. Texts that exhibit poor spelling and grammar, or lack compelling arguments and clear structure, are perceived as lower quality. Moreover, the authors themselves may be perceived as less intelligent, creative, hardworking or trustworthy. Using a within-subjects design, the current study systematically examined the effects of lower-level errors and higher-level errors on college students? (n =70) perceptions of multiple aspects of writing quality and author characteristics. Results demonstrated that students noticed both kinds of errors but were much more sensitive to lower-level errors than higher-level errors. Nearly identical patterns were observed for judgments of text quality and authors, and the sensitivity to lower-level errors was stronger for more-skilled readers. Implications for challenges and biases in peer assessment are discussed.



Student and instructor perceptions of writing tasks and performance on TOEFL iBT versus university writing courses
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Lorena Llosa, Margaret E. Malone
This study examined student and instructor perceptions of writing tasks and performance on TOEFL iBT versus university writing courses. Participants included 103 international, nonnative-English-speaking undergraduate students enrolled in required university writing courses and their writing instructors (n=18). Students completed a background questionnaire, two TOEFL iBT writing tasks (one Integrated and one Independent), and a questionnaire about their perceptions of the writing tasks. The 18 instructors also completed a questionnaire and sx participated in an extended interview. Students and instructors reported that neither the Independent nor the Integrated task alone was representative of the types of writing they do in their writing course. However, the Independent and Integrated task together represented many of the characteristics of course assignments. Additionally, instructors perceived the criteria in the TOEFL iBT writing rubrics to be very similar to the criteria that they use in class to assess student writing suggesting that TOEFL iBT tasks and course assignments are based on a similar operationalization of the writing construct. Finally, students and instructors generally perceived the quality of the writing produced for the TOEFL iBT writing tasks to be comparable to the quality of writing produced in course assignments.



Reclaiming Accountability: Improving Writing Programs through Accreditation and Large-Scale Assessments, W. Sharer, T.A. Morse, M.F. Eble, W.P. Banks. Utah State University Press (2016), ISBN: 978-1-60732-434-8
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Ashley Velazquez




Assessing Writing, Teaching Writers: Putting the Analytic Writing Continuum to Work in Your Classroom, M.A. Smith, S.S. Swain. Teachers College Press, New York (2017)
Publication date: October 2017
Source:Assessing Writing, Volume 34
Author(s): Les Perelman