var quotes = new Array; quotes.push(""Tests are standardized when the directions, conditions of administration, and scoring are clearly defined and fixed for all examinees, administrations and forms."

Cohen & Wollack, 2006."); quotes.push(""The nature of the individual test items should be such as to provide specific, recognisable evidence of the examinee's readiness to perform in a life-situation, where lack of ability to understand and speak extemporaneously might be a serious handicap to safety and comfort, or to the effective execution of military responsibilities."

Kaulfers, 1944."); quotes.push(""The task for the ethical language tester is to look into the future, to picture the effect the test is intended to have, and to structure the test development to achieve that effect. This is what we refer to as effect-driven testing."

Fulcher and Davidson, 2007."); quotes.push(""Validity might also be expressed more simply as the 'worthwhileness' of an examination. For an examination to possess validity it is necessary that the materials actually included be of prime importance, that the questions sample widely among the essentials over which complete mastery can reasonably be expected on the part of the pupils, and that proof can be brought forward that the test elements (questions) can be defended by arguments based on more than mere personal opinion."

Ruch, 1924."); quotes.push(""The efficacy of examinations as a means of calling out the interest of a pupil and directing it into the desired channels was soon recognized by teachers."

Latham, 1877."); quotes.push(""Educators seem to be agreed that pupils tend to accomplish more when confronted with the realization that a day of reckoning is surely at hand."

Ruch, 1924."); quotes.push(""Educators seem to be agreed that pupils tend to accomplish more when confronted with the realization that a day of reckoning is surely at hand."

Ruch, 1924."); quotes.push(""In [Dynamic Assessment], assessment and instruction are a single activity that seeks to simultaneously diagnose and promote learner development by offering learners mediation, a qualitatively different form of support from feedback. Mediation is provided during the assessment procedure and is intended to bring to light underlying problems and help learners overcome them."

Lantolf and Poehner, 2008."); quotes.push(""Achievement measurement can be defined as the assessment of terminal or criterion behaviour; this involves the determination of the characteristics of student performance with respect to specified standards."

Glaser, 1963."); quotes.push(""I believe we should explicitly address with our teacher education students how they might cope with the contesting forces of good and evil assessment as they compete in classrooms to control curriculum, time, and student attitudes about learning."

Shepard, 2000."); quotes.push(""We have learned that computer adaptive testing is hideously expensive in large-scale tests. Item pools became vats, then lakes, then oceans, just to maintain test security in environments like China. The era of adaptivity in mass international language testing is dead."

Fulcher, 2005."); quotes.push(""The essence of the AEI proficiency lies not in verbal descriptions of it, but in its thirty-year-long tradition of practice - making training in AEI proficiency testing a desideratum."

Lowe, 1986."); quotes.push(""The tests must not be guilty of the 'correlation fallacy', the common delusion that a certain level of ability on a pencil-and-paper test of vocabulary, grammar, or reading comprehension can automatically be interpreted to mean a corresponding level of ability to understand the spoken language, or to speak the language fluently."

Kaulfers, 1944."); quotes.push(""Final examinations in particular are likely to be corrected on the day that all the multitudinous tasks incident to the closing of a semester of year of school rush in on the tired teacher, who must finish the papers, make out the report cards, balance the register, pack her trunk, and catch the earliest train home. Where, then, does the opportunity for training in the correct use of English come in under the situation as it really is?"

Ruch, 1924."); quotes.push(""This is counting in the real world. It is not a science, it is not precise, in some ways it is almost, with respect to those who do it, absurd. Get down to the grit of the way data is gathered, and you often find something slightly disturbing: human mess and muddle, luck and judgment, and always a margin of error in gathering only a small slice of the true total, from which hopeful sample we simply extrapolate."

Blastland & Dilnot, 2008."); quotes.push(""The purpose of language testing is always to render information to aid in making intelligent decisions about possible courses of action."

Carroll, 1961."); quotes.push(""In some ways, a good test is like an experiment, in the sense that it must eliminate or at least keep constant all extraneous sources of variation. We want our tests to reflect only the particular kind of variation in knowledge or skill that we are interested in at the moment."

Carroll, 1961."); quotes.push(""It is reasonable to expect that administration conditions have a nonnegligible effect on examinee performance. If it is too hot or too cold in the testing room, performances of some examinees are likely to be negatively affected. Similarly, if it is too noisy, some examinees may be distracted and perform below their potential."

Cohen & Wollack, 2006."); quotes.push(""During a test neither teach nor criticize. These are the two lapses which, by sheer force of professional habit, the teacher most inclines. Criticism diminishes candour and destroys self-confidence. Instruction transforms the examinee's entire attitude toward the remainder of the tests."

Burt, 1922."); quotes.push(""Accommodations are designed to remove or mitigate as much as possible the effects of the disabling condition(s) from the measurement of the ability of interest...the exception is that it is not necessary to grant accommodations that will compromise the fundamental interpretation of the test."

Cohen & Wolack, 2006."); quotes.push(""Parents want something to shew for education; a place in an examination list seems to gauge the advantage which they have paid for, and besides it frequently has a positive market value as opening the door to some emolument or profession."

Latham, 1877."); quotes.push(""The total estimated cost for states using only multiple-choice tests was approximately $1.9 billion, whereas the cost if states also included a small number of hand-scored open-response items such as essays was estimated to be about $5.3 billion."

Koretz and Hambleton, 2006."); quotes.push(""If we damage the general standard of truthfulness by leading young men to glory in having outwitted Examiners, and seemed to be what they are not...then we lose far more morally than we gain in any other way."

Latham, 1877."); quotes.push(""Here we come to the truth on which we must rest. If we can frame an examination in which that which will enable the candidate to do the best is that which it is best for him to learn, and to learn in the best way, then we shall have constructed a perfect educational instrument."

Latham, 1877."); quotes.push(""Testing is primarily about establishing ways of making decisions that are (hopefully) not random, and seen as 'fair' by the population. Whenever we establish ways of making decisions we reveal what we believe about society and political organization. So the practice of testing and assessment can never be separated from social and political values."

Fulcher, 2010."); quotes.push(""A rationale should be presented for each recommended interpretation and use of test scores, together with a comprehensive summary of the evidence and theory bearing on the intended use or interpretation."

AERA, 1999."); quotes.push(""If a test is used in a way that has not been validated, it is incumbent on the user to justify the new use, collecting new evidence if necessary."

AERA, 1999."); quotes.push(""For each total score, subscore, or combination of scores that is to be interpreted, estimates of relevant reliabilities and standard errors of measurement or test information functions should be reported."

AERA, 1999."); quotes.push(""The single most important consideration in both the development of language tests and the interpretation of their results is the purpose or purposes the particular tests are intended to serve."

Bachman, 1990."); quotes.push(""...language tests...provide a reflection, a mirror, of the complexities and power struggles of society. They lie at the crossroads of many conflicts and therefore should be studied, protected and guarded as part of the process of preserving and perpetuating democratic cultures, values and ethics, as well as the quality of learning."

Shohamy, 2001."); quotes.push(""Construct validation takes place when an investigator believes that his instrument reflects a particular construct, to which are attached certain meanings. The proposed interpretation generates specific testable hypotheses, which are a means of confirming or disconfirming the claim."

Cronbach and Meehl, 1955."); quotes.push(""[Validity is] an integrated evaluative judgment of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores or other modes of assessment."

Messick, 1989."); quotes.push(""Whenever a test is administered, the test user would like some assurance that the results could be replicated if the same individuals were tested again under similar circumstances. This desired consistency (or reproducibility) of test scores is called reliability."

Crocker & Algina, 1986."); quotes.push(""...we cannot label a test as valid or not valid except for some purpose...The thing to ask for is a proof that the test does its job, but not to ask always for the same kind of evidence that it does."

P. J. Rulon, 1946."); quotes.push(""Having multiple sources/pieces of evidence to inform a consequential interpretation/decision is a fundamental feature of the epistemology and ethics in any of the social science perspectives that I have encountered."

Pamela Moss, 2003."); quotes.push(""...communicative testing must be devoted not only to what the learner knows about the second language and about how to use it (competence) but also the extent the learner is able to actually demonstrate this knowledge in a meaningful communicative situation (performance)."

Canale & Swain, 1980."); quotes.push(""Washback refers to the extent to which the introduction and the use of a test influences language teachers and learners to do things that they would not otherwise do that promote or inhibit language learning."

Messick, 1996."); quotes.push(""Validity is associated with the interpretations assigned to test scores rather than with the scores themselves or the test and involves an evaluation of the appropriateness of these interpretations."

Michael Kane, 1992."); quotes.push(""Often, the social and political character of language assessments is implicit; however, increasingly, governments and other agencies are using assessment in the furtherance of explicit policies of social engineering."

McNamara & Roever, 2006."); quotes.push(""In language testing there will always be a tension between the ethical requirements of fairness, precision, and accountability and the reality of our uncertainty about the nature of the object of our measurements."

Dan Douglas, 2009."); quotes.push(""A public examination is already a sort of lottery of the graduated species, one which the chances are not equal, but are better for the more deserving...It is a species of sortition infinitely preferable to the ancient method of casting lots."

F. Y. Edgeworth, 1888."); quotes.push(""A major strength of an argument-based approach to validation is the guidance it provides in allocating research effort and in deciding on the kinds of validity evidence that are needed. The kinds of validity evidence that are most relevant are those that evaluate the main inferences and assumptions in the interpretive argument, particularly those that are the most problematic."

Michael Kane, 2001."); quotes.push(""If examinees respond in a haphazard or non-reflective manner, their obtained scores may not represent their actual ability. Also, if instructions are unclear and the test format is unfamiliar to the students, their responses may not reflect their true ability, and in this way the test may be said to lack response validity."

Grant Henning, 1987."); quotes.push(""Tests allow us to present all our students with the same instructions and the same input under the same conditions. Tests also allow us to get a 'second opinion' about our students' progress - they can help confirm our own assessments and help make decisions about students' needs with more confidence."

Dan Douglas, 2010."); quotes.push(""Recall that the word 'criterion' in criterion-referenced testing, has two meanings. It can refer to the criterion level, or pass-fail cut-point used in decision making with criterion-referenced tests, or it can refer to the criteria (or course objectives in many cases) on the basis of which such tests are designed. When the latter sense of 'criterion' is applied, criterion-referenced tests take on an advantage that makes them particularly useful for language curriculum development."

J. D. Brown and Thom Hudson, 2002."); quotes.push(""Testing is a universal feature of social life. Throughout history people have been put to the test to prove their capabilities or to establish their credentials; this is the stuff of Homeric epic, of Arthurian Legend....What is true of testing in general is true also of language testing."

McNamara, 2000."); quotes.push(""Construct validation is involved whenever a test is to be interpreted as a measure of some attribute or quality which is not 'operationally defined.' The problem faced by the investigator is 'What constructs account for variance in test performance?'."

Cronbach & Meehl, 1955."); quotes.push(""...since predictive, concurrent, and content validities are all essentially ad hoc, construct validity is the whole of validity from a scientific point of view."

Loevinger, 1957."); quotes.push(""...it is not possible to verify the interpretive argument in any absolute sense. The best that can be done is to show that the interpretive argument is highly plausible, given all available evidence."

Kane, 1992."); quotes.push(""Validation is a matter of making the most reasonable case to guide both current use of the test and current research to advance understanding of what the test scores mean...To validate an interpretive inference is to ascertain the degree to which multiple lines of evidence are consonant with the inference, while establishing that the alternative inferences are less well supported."

Messick, 1989."); quotes.push(""It is now a widely accepted tenet of measurement theory that the work of standard-setting panels is not to search for a knowable boundary between categories that exist. Instead, standard-setting procedures enable participants to bring to bear their judgments in such a way as to translate policy decisions....into locations that create the effective performance categories. This translation and creation are seldom, if ever, purely statistical, impartial, apolitical, or ideologically neutral activities."

Cizek, Bunch and Koons (2004)."); quotes.push(""At the heart of CTT is the assertion that an observed score is determined by the actual state of the unobservable variable of interest plus error contributed by all other influences on the observable variable. The actual state of the unobserved variable is its hypothetical true score."

DeVellis, 2006."); quotes.push(""...we have arrived at the consensus opinion that (a) validity refers to the interpretation of test scores, not the test itself, and (b) what we strive to validate are inrerpretations (and actions) based on test scores, not a test per se."

Sireci, 2009."); quotes.push(""Validity is usually described as the extent to which a test measures what it is purported to measure. This is an unsatisfactory and not very useful concept of validity, because under it the validity of a test may be altered completely by arbitrarily changing its 'purport'."

P. J. Rulon, 1946."); quotes.push(""The essential question of test validity is how well a test does the job it is employed to do. The same test may be used for several different purposes, and its validity may be high for one, moderate for another, and low for a third. Hence, we cannot label the validity of a test as 'high', 'moderate' or 'low' except for some particular purpose."

Cureton, 1951."); quotes.push(""Narrowly considered, validation is the process of examining the accuracy of a specific prediction or inference made from a test score....More broadly, validation examines the soundness of all interpretations of a test - descriptive and explanatory interpretations as well as situation-bound predictions."

Cronbach, 1971."); quotes.push(""An observable attribute can be defined in terms of a target domain of possible observations, and the value of the observable attribute for a person, or the person's target score for the domain, can be defined as the person's expected score over the target domain."

Kane, 2009."); quotes.push(""The interpretive argument for observable variables includes three major inferences. The observed performances are scored and combined into an observed score (a raw score or scaled score of some kind), the observed score is generalized to the universe score, and the universe score is extrapolated to the target score, representing the value of the observable attribute."

Kane, 2009."); quotes.push(""Semi-direct tests may be proposed as second-order substitutes for direct techniques when general proficiency measurement is at issue but it is not operationally possible to administer a direct test. In these instances, it is considered highly important to determine - through appropriate experimental means - a high level of correlation between the two types of instruments when used with representative examinee groups."

Clark, 1979."); quotes.push(""As with all mental measures, language tests are indirect indicators of the underlying traits in which we are interested."

Bachman & Savignon, 1986."); quotes.push(""In order for a system of ratings or descriptions to be considered a scale, the ratings or descriptions must (1) denote the relative presence or absence of some substance or quality and (2) be capable of validly and reliably ordering persons or objects according to the extent to which they possess the substance or quality at issue."

Clark and Lett, 1988."); quotes.push(""A glance at the literature on fluency reveals it to be replete with vacuous definitions, overlapping terminology, and impractical assessment strategies."

Hieke, 1985."); quotes.push(""...there is more to Lado than analytical tests, since his culture, literature, comprehension tasks, while themselves offering points of contrast on critical points of difficulty, all subsume within themselves control over a whole range of forms which are, in miniature, integrative."

Davies, 1982."); quotes.push(""...What remains a convincing argument in favour of linguistic competence tests (both discrete and integrative) is that grammar is at the core of language learning....Grammar is far more powerful in terms of generalisability than any other language feature. Therefore grammar may still be the most salient feature to teach, and to test."

Davies, 1982."); quotes.push(""Examinations, sir, are pure humbug from beginning to end. If a man is a gentleman, he knows quite enough, and if he is not a gentleman, whatever he knows is bad for him."

Oscar Wilde, ND."); quotes.push(""In examinations, those who do not wish to know ask questions of those who cannot tell."

attributed to Sir Walter Raleigh, 1554 - 1618."); quotes.push(""As long as learning is connected with earning, as long as certain jobs can only be reached through exams, so long must we take this examination system seriously. If another ladder to employment was contrived, much so-called education would disappear, and no one would be a penny the stupider."

E. M. Forster, ND."); quotes.push(""Of College labours, of the Lecturer's room All studded round, as thick as chairs could stand, With loyal students, faithful to their books, Half-and-half idlers, hardy recusants, And honest dunces--of important days, Examinations, when the man was weighed As in a balance!"

Wordsworth, 1799."); quotes.push(""No test preparation practice should increase students' test scores without simultaneously increasing student mastery of the content domain tested."

Popham, 1991."); quotes.push(""The ability to speak a foreign language is without doubt the most highly prized language skill, and rightly so, because he who can speak a language well can also understand it and can learn to read it with relative ease....Yet, testing the ability to speak a foreign language is perhaps the least developed and the least practiced in the language testing field."

Lado, 1961."); quotes.push(""Reliability has to do with the stability of scores for the same individuals. If the scores of students are stable the test is reliable: if the scores tend to fluctuate for no apparent reason, the test is unreliable."

Lado, 1961."); quotes.push(""Language tests do not provide exact information, it is always 'more' or 'less' and 'within confidence limits'. It is important to recognize that uncertainty from the start, to accept (and later describe) it and then to welcome it."

Davies, 1990."); quotes.push(""Language testers can be criticized in many cases for perpetuating the testing of grammar with discrete-point tasks of grammatical form; for constructing scoring rubrics with descriptions of grammatical development that have little support from SLA findings...or from a coherent model of grammatical ability; and for downplaying the role of grammatical accuracy in favor of communicative effectiveness."

Purpura, 2004."); quotes.push(""...a test is a procedure designed to elicit certain behavior from which one can make inferences about certain characteristics of an individual."

Carroll, 1968."); quotes.push(""Measurement is the process of quantifying the characteristics of an object of interest according to explicit rules and procedures."

Bachman, 1990."); quotes.push(""Stability is one of the twin pillars of public examinations that are essential if exams are to fulfil the purposes for which they are intended. However, innovation linked to improvement is just as vital if the examination is to keep up with developments and insights available from research in the field."

Weir, 2003."); quotes.push(""In architecture, a retrofit may be initiated to meet new design standards, introduce safety features unknown when a building was originally constructed, make equipment work more efficiently, or to make a structure fit for a new use or a new user. Test design is no different in principle, and we can identify two distinct types of retrofit...an upgrade retrofit...[and]a change retrofit."

Fulcher and Davidson, 2009."); quotes.push(""The primary purpose of any language assessment is to collect information for making decisions. Furthermore, the use of an assessment and the decisions made will have consequences for stakeholders, the individuals and programs in the educational and social setting in which language assessment takes place...the intended uses of language assessment are to help us make decisions that will ideally lead to beneficial consequences for stakeholders."

Bachman and Palmer, 2010."); quotes.push(""A construct is a meaningful interpretation of observed behavior. When a researcher interprets a learner's score on a vocabulary test, for example, as an indicator of vocabulary knowledge, then 'vocabulary knowledge' is the construct that gives meaning to the score. The fundamental requirement for interpreting observed behavior as a construct is that the behavior reflects performance consistency."

Chapelle, 1998."); quotes.push(""...strategies are mental operations or processes that learners consciously select when accomplishing language tasks...test taking strategies will be viewed as those test-taking processes that [learners] have selected and of which they are conscious, at least to some degree. In other words, the notion of strategy implies an element of selection. Otherwise the processes would not be considered strategies."

Cohen, 1998."); quotes.push(""There is no doubt that context does play a role in influencing language choice and acquisition; the problem has been arriving at a common understanding of the nature of context and what constitutes it, and then determining specifically how various contextual features influence language use and development."

Douglas, 1998."); quotes.push(""I was thrown out of college for cheating on the metaphysics exam. I looked into the soul of the boy sitting next to me."

Woody Allen (in Annie Hall)."); document.write(quotes[(Math.floor(Math.random() * quotes.length))]);