The oral fluency level of an L2 speaker is often used as a measure in assessing language proficiency. The present study reports on four experiments investigating the contributions of three fluency aspects (pauses, speed and repairs) to perceived fluency. In Experiment 1 untrained raters evaluated the oral fluency of L2 Dutch speakers. Using specific acoustic measures of pause, speed and repair phenomena, linear regression analyses revealed that pause and speed measures best predicted the subjective fluency ratings, and that repair measures contributed only very little. A second research question sought to account for these results by investigating perceptual sensitivity to acoustic pause, speed and repair phenomena, possibly accounting for the results from Experiment 1. In Experiments 2–4 three new groups of untrained raters rated the same L2 speech materials from Experiment 1 on the use of pauses, speed and repairs. A comparison of the results from perceptual sensitivity (Experiments 2–4) with fluency perception (Experiment 1) showed that perceptual sensitivity alone could not account for the contributions of the three aspects to perceived fluency. We conclude that listeners weigh the importance of the perceived aspects of fluency to come to an overall judgment.

Partial dictation is a measure of EFL listening proficiency that can be easily constructed, administered, and scored by EFL teachers. However, it is controversial whether this form of test measures lower-order abilities exclusively or involves both lower- and higher-order abilities. In order to answer this question, a study was designed to examine the difference between partial dictation and test forms believed to measure more higher-order abilities. In a series of confirmatory factor analyses, the simplex, second-order, and bi-factor models were fitted to the scores of 367 college-level EFL learners in China in a listening test composed of partial dictation, gap-filling and constructed response tasks. The bi-factor model was identified as the best-fitting and this supports the view that partial dictation measures the same construct as test forms believed to measure more higher-order abilities. Concomitant statistical analyses also showed that the partial dictation tasks were suited to the ability level of the test takers and had high internal consistency.

Although a key concept in various writing textbooks, learning standards, and writing rubrics, voice remains a construct that is only loosely defined in the literature and impressionistically assessed in practice. Few attempts have been made to formally investigate whether and how the strength of an author’s voice in written texts can be reliably measured. Using a mixed-method approach, this study develops and validates an analytic rubric that measures voice strength in second language (L2) argumentative writing. Factor analysis of ratings from six raters on voice strength in a total of 400 TOEFL® iBT writing samples, together with qualitative analysis of four raters’ in-depth think-aloud and interview data, points to an alternative conceptualization of voice that sees authorial voice in written discourse as being realized primarily through the following dimensions: (1) the presence and clarity of ideas in the content; (2) the manner of the presentation of ideas; and (3) the writer and reader presence. Implications of such results for L2 writing instruction and assessment are discussed.

Based on evidence that listeners may favor certain foreign accents over others (Gass & Varonis, 1984; Major, Fitzmaurice, Bunta, & Balasubramanian, 2002; Tauroza & Luk, 1997) and that language-test raters may better comprehend and/or rate the speech of test takers whose native languages (L1s) are more familiar on some level (Carey, Mannell, & Dunn, 2011; Fayer & Krasinski, 1987; Scales, Wennerstrom, Richard, & Wu, 2006), we investigated whether accent familiarity (defined as having learned the test takers’ L1) leads to rater bias. We examined 107 raters’ ratings on 432 TOEFL iBTTM speech samples from 72 test takers. The raters of interest were L2 speakers of Spanish, Chinese, or Korean, while the test takers comprised three native-speaker groups (24 each) of Spanish, Chinese, and Korean. We analyzed the ratings using a multifaceted Rasch measurement approach. Results indicated that L2 Spanish raters were significantly more lenient with L1 Spanish test takers, as were L2 Chinese raters with L1 Chinese test takers. We conclude by concurring with Xi and Mollaun (2009, 2011) and Carey et al. that rater training should address raters’ linguistic background as a potential rater effect. Furthermore, we discuss the importance of recognizing rater L2 as a possible source of bias.

This study examines the development and evaluation of a bilingual Vocabulary Size Test (VST, Nation, 2006). A bilingual (English–Russian) test was developed and administered to 121 intermediate proficiency EFL learners (native speakers of Russian), alongside the original monolingual (English-only) version of the test. A comparison of the bilingual and monolingual test scores showed that participants achieved significantly higher scores on the bilingual version of the test. Accuracy of responses to individual test items was reliably higher when the meanings of test items were presented in the L1 (Russian) and when these items were cognates. The findings also revealed that the bilingual version is likely to be a more sensitive measure of written receptive vocabulary knowledge. Finally, analyses showed that the effect of using L1 for multiple-choice options is likely to be larger for low-proficiency learners and that the difference in response accuracy to cognates and non-cognates decreases as item frequency increases. The paper concludes with recommendations on developing and using bilingual vocabulary size tests.

Differential skill functioning (DSF) exists when examinees from different groups have different probabilities of successful performance in a certain subskill underlying the measured construct, given that they have the same ability on the overall construct. Using a DSF approach, this study examined the differences between two native language groups – a group with an East Asian language background and one with a Romance language background – in regard to reading subskills as represented in the Michigan English Language Assessment Battery (MELAB) reading test. Based on a combination of literature review and think-aloud reports from a sample of ESL students, hypotheses on reading subskill differences between the two groups were generated. These hypotheses were tested by first identifying the subskill profile of each examinee in a large MELAB database via the application of a previously determined item-skill Q-matrix to a Fusion Model of cognitive diagnostic modeling. The subskill profiles of the East Asian examinees were then compared against those of examinees with a Romance language background through logistic regression techniques. Some important DSFs were found between the two groups. Based on results of this study, instructional strategies were suggested to address some specific weaknesses in ESL learners’ reading subskills.