OEPT Rating Glossary
revised August 4, 2016

Academic words

Academic words are words that are frequent within an academic context or, more specifically, words appearing on the Academic Word List (AWL). http://www.victoria.ac.nz/lals/resources/academicwordlist/default.aspx The AWL was developed by Averil Coxhead at the School of Linguistics and Applied Language Studies at Victoria University of Wellington, New Zealand. The list contains 570 word families which were selected because they appear with great frequency in a broad range of academic texts. The list does not include words that are in the most frequent 2000 words of English (The General Service List http://jbauman.com/aboutgsl.html) thus making it specific to academic contexts.

Accent

Variation that deviates from native-speaker norms due to differences such as sound substitutions and syllabic stress or word stress.

Accuracy

Correctness, of grammar, syntax, vocabulary use.

Analytic

Referring to the parts or subsets of a whole.

Analytic language domains, features, skills, or abilities

Analytic language domains, features, or subskills are those features that can be considered subsets of general ability or proficiency. For example, speaking proficiency can be described as consisting of the subproficiencies of pronunciation, delivery, grammar, vocabulary, content, and so on.

Analytic scale

An analytic scale requires raters to provide separate scores on subcomponents, such as pronunciation, fluency, and coherence.

Approximate sounds

To approximate a sound means to articulate a sound almost but not perfectly correctly - to approach a closely correct pronunciation.

Argument

A course of reasoning aimed at demonstrating or persuading. Verbal arguments can be supported by examples, anecdotes, analogies, and other information of various kinds, including that supplied by prosodic and intonational features.

Articles

The, a, an. Article errors are hard to classify so we usually just refer to them as "article errors", or "omits articles" "rarely uses articles" , "uses the wrong articles"

Articulation

Producing an individual sound in the correct place (e.g. placement of tongue in the oral cavity: alveolar, dental, etc.) and in the correct manner (voiced, unvoiced, stop, fricative, with proper aspiration, tension or laxity, etcetera). For example, some speakers may pronounce [f] as [p]; or may pronounce the diphthong [ei] as in 'plane' as [E] as in 'plen'. If articulation issues such as these are systematic, they can be referred to as 'sound substitution'.

Audience awareness/accommodation

In an OEPT, awareness of the audience is shown when the speaker explains the context of a response (framing) and provides organizational markers.

Automaticity

The absence of attentional control when conducting a cognitive activity. When something is automatic, there is no need to consciously think about the activity when involved in the activity. The classic example is riding a bicycle after years of bicycle riding. When responding to OEPT prompts, all examinees have to think about what they want to say, but a highly proficient speaker will have some runs of speech that sound automatic or delivered effortlessly. It is rare that any speaker will deliver an entire response seemingly effortlessly, however.

Basic vocabulary

Refers to the most commonly used words of English, as opposed to any specialized or sophisticated lexis.

Benchmark

An exemplar or standard by which something can be measured.

Bound morphology

A morpheme that cannot stand alone as an independent unit of meaning. For example, the past tense /-ed/ is a bound morpheme in English.

Coherence and organization (of argument)

Coherent

Cohesion

For our purposes, coherence refers to how well a response holds together as a logical and orderly message, argument or text, with apparent relationships between parts. A speaker could be comprehensible but not highly coherent: if we understand the meaning of the sentences and the general message, but the argument is weak, vague, out of order, with lack of specificity, evidence, and organization. One example of lack of coherence occurs when we're not sure what some pronouns refer to (see "referent").

The linguistic elements that hold a sentence or text together, give it meaning, and make grammatical and lexical inking are referred to as "cohesion". Some examples of these elements are pronouns, conjunctions, transitions, and discourse markers.

If a response contains several incomprehensible sentences, it is very likely to also be incoherent.

Collocations

A sequence of words or terms that co-occur more often than would be expected by chance. E.g.: critical juncture, inextricably linked, a real problem.

Complexity

Generally used to refer to syntactic complexity: is the speaker able to produce complex sentence structures with embedded clauses, for example.

Comprehensibility

(of sentence-level text)

For our purposes, a speaker is comprehensible if you can easily understand the meaning at the sentence level. Comprehensibility depends on being able to put lexis and grammar together to form sentences that make sense. Someone can be perfectly intelligible but not very comprehensible: e.g. "Several minds are not going right in that case." Or "Student asking difficult situations." Incomprehensibility is generally due to errors of grammar, syntax, and vocabulary, but unidiomatic intonation patterns can also contribute by giving confusing signals. Frequent long pauses and other disfluencies can cause a listener to lose comprehension.

Consequences

What happens as a result of the score assigned to the examinee. In the context of the OEPT, how the assigned score may affect the examinee's graduate career, i.e. the need to take ENGL 620 and the ability to receive a teaching assistantship or the type of assistantship a student may be qualified for.

Consistent, consistently

For Consistency of scoring, see reliability

On the OEPT scale, the term consistent or consistently refers to something that occurs all of the time, or almost all the time; maybe occasionally not. Consistent can be synonymous with systematic, when referring to bound morphology errors for example. "Student makes consistent (systematic) subject verb agreement errors." See "generally" and "occasionally". Level 45 requires consistency of intelligibility and coherence, but 45s may otherwise be inconsistent, such with fluency of delivery, vocab usage, etc.

Contextualization

see also Framing

A response is contextualized if the speaker gives the listener background information related to the response. See "framing".

Discourse markers or transitions

Words or phrases that indicate a relationship between what was just said and what is about to be said, or that signal a change of topic or orientation. E.g.: "On the other hand" indicates that what follows will be in contrast to what was just said. "The next important factor..." indicates that the speaker is moving on to the next item or concept in a series.

Disfluencies

Obvious efforts and difficulties of delivery when trying to compose a response, indicating a lack of automaticity: restarting sentences, phrases, or words; unneeded/unproductive repetition of the same words or parts of words; frequent or consistent hesitation, choppiness, or isolated word-by-word delivery, or even syllable-by-syllable delivery; frequent searching for words; frequent long pauses in unexpected places; unusually slow speed of delivery which affects the listener's ability to follow along.

Disfluent

A speaker could be considered disfluent if the type and frequency of disfluencies mentioned above cause the listener to lose the message or make it very hard to follow.

Elaboration

Examinees elaborate when they add support for their argument in the form of content that is not included in the prompt, such as examples, anecdotes, or unsolicited comments and opinions.

Enunciation

Clear, crisp, complete articulation of sounds and pronunciation of words.

Filled pauses

Pausing accompanied by sound: Er, um, uh, etc.

Fluency (of delivery)

For our purposes, fluency refers to the amount of effort it appears the speaker must make to compose their response, the amount of time it takes for them to deliver it, and the manner in which they deliver it. A very fluent speaker appears not to make any special effort to respond - the language flows freely and automatically. Most speakers, however, including native speakers, use a certain amount of pausing (filled and unfilled) and hesitation, but usually in expected ways, and not to the extent that the listener cannot follow them. See "disfluent" and "automaticity."

Framing (of response)

We refer to a frame in an OEPT response as a brief introduction that orients the listener to the topic, the information already known, and the speaker's plan. The frame sets up appropriate expectations about what will follow. The speaker may reiterate part of the question or state the title or labels of a table or graph. Frames provide a foundation for the rest of the response.

Frequent

Noticeable, multiple occurrences. More than occasional.

Functionally bilingual

Capable of performing in two languages with nearly equal proficiency.

Generally

Most of the time, but not as often as consistently. More often than occasionally. More than 50% of the time.

Global syntax error

See "syntax errors"

Grammar

see also: Syntax and

syntax errors

Rules for combining morphemes and words into sentences and longer texts. Commonly we use grammar to refer to morphology-related errors and syntax to refer to word order related errors.

Holistic

Relating to the whole rather than the analysis of parts. In assessment, assigning a single score based on an overall assessment of performance rather than by scoring or analyzing dimensions individually. The product is considered to be more than the sum of its parts and so the quality of a final product or performance is evaluated rather than individual dimensions of performance. A holistic scoring rubric such as the OEPT might combine a number of elements on a single scale (intelligibility, coherence, fluency.) The OEPT uses a Holistic scale: we are most interested in the overall effect of the performance.

Hypothetical

Statement that expresses something counter to reality, conditional, doubtful, suppositional: If I were rich, I would travel. If I had to choose, I would select the second option.

Idiomatic phrasing, use, or expressions/

idiomaticity

In the OEPP, we use idiomaticity to refer to the appropriate use of any sequence of words that can be described as a set or fixed expression. Idiomatic use may refer to the speaker's use of highly frequent phrases or collocations, some of which allow substitutions (I think that, to get along with, a relatively short time) or fixed phrases more commonly recognized as figures of speech (on the right track, off the beaten path.) Idiomatic expressions are multi-word sequences that are conceptually stored and accessed as one unit (How's it going?), often contrasted with non-idiomatic sequences that are generated through grammatical rules (Are things going well?).

Idiomatic intonation refers to intonation patterns that sound similar to those used by native speakers.

Incoherent (text, argument)

Doesn't hold together logically, doesn't make sense, or is so poorly organized or expressed that it is hard to follow. Overuse of pronouns or nonspecific language (thing, it, something, stuff) can make a response incoherent if we are not sure who or what the pronoun or nonspecific word refers to.

Inconsistent

No clear pattern. Not consistent. More than occasional (errors, for example). Level 45 requires consistency of intelligibility and coherence, but 45s may be inconsistent in fluency or accuracy, for example.

Inferences

Probable conclusions drawn from evidence or scores. When a student passes the OEPT, it may be inferred that the student is capable of communicating successfully in a Purdue undergraduate classroom.

Intelligibility (of speech)

For our purposes, a speaker is intelligible if you can recognize the words he or she is saying. There are levels of intelligibility. One speaker may be completely intelligible all the time with no effort on your part. Another speaker may require some effort on your part to adjust to her accent, but after the initial adjustment you have no trouble recognizing the words she says. Some speakers, because of articulation issues, may be difficult to understand occasionally, only when certain sounds are involved, or perhaps certain morphemes. A different speaker may require you to exert frequent or consistent efforts to identify words she is saying.

Intelligibility can depend on many different factors: articulation of sounds; pronunciation of bound morphemes; syllable or phrasal stress; phrasal or sentence level intonation; rhythm, volume, speed. Your inability to recognize a word because you are not familiar with that word (for example a technical term), is not a matter of the speaker's intelligibility. See also unintelligible

Intonation

Variation of pitch while speaking. One component of prosody. Intonation can be described as rising, falling, monotonous, singsong (regular rising and falling), idiomatic (patterns like a native speaker.)

L1

First language

L1 Characteristics

Features that carry over from the L1 into English. E.g. sound substitutions, intonation patterns. Most speakers on the OEPT display L1 characteristics.

L2

Second language

Lexico-syntactic

Refers to the combination of vocabulary and grammar

Lexis

Vocabulary

Limited (vocab or syntax range)

A speaker's vocab or syntax range may be limited if they are generally able to convey messages successfully but only perhaps in a relatively simplistic or unsophisticated manner, because they lack the broader range of vocab and grammar needed to express their ideas fully. See "restricted".

Local syntax error

See "syntax errors"

Marked

Something is marked if it stands out and is very noticeable. A marked deficiency restricts the speaker's ability to communicate successfully. One type of 35 level performance has marked deficiencies in 3 general areas of proficiency such as fluency, vocabulary, and pronunciation.

Modality

Referring to the appropriate use of English modal verbs: shall and should, will and would, may and might, can and could, and must.

Mood

In linguistics, many grammars have the concept of grammatical mood (or mode), which describes the relationship of a verb with reality and intent. Many languages express distinctions of mood through morphology, by changing (inflecting) the form of the verb. For example, If I were you indicates the subjunctive mood in English - perhaps the only grammatical expression of subjunctive in English. In other languages the grammatical indication of mood is much more common.

More than occasional(ly)

More than a few times here and there, but perhaps less than half the time. For example, if someone was incoherent about 30% of the time, you could call that more than occasionally (but less than generally) incoherent.

Morphemes/Morphology

Morphemes are building blocks of words, the smallest meaningful units of words. Free morphemes can stand alone as words (walk, house, fear), while bound morphemes cannot (-ed, -s, -ing).

Non-specific language

Language that is vague or not precise. Words such as it, some, something, stuff, thing, and some uses of pronouns can result in a vague or imprecise message.

Occasional, occasionally

These terms refer to instances that are not systematic, don't occur often, but rather a few times here and there during the test, perhaps in a couple of items. If something occurs occasionally there is no discernable pattern. For example, "occasional unintelligible words" could mean that there were 2 or 3 words or phrases during the entire test that you did not identify. This implies that the rest of the time there were no outstanding problems with intelligibility. You could then say "occasional unintelligible words, but overall, consistently intelligible."

Oral English proficiency

Ability to use spoken English to communicate accurately and successfully.

Personalized message

An OEPT response can be characterized as personalized if the speaker elaborates with new information related to their own experience and background knowledge.

Phrasal stress

Appropriate placement of stress or emphasis on certain words within a phrase or sentence.

Phrasal stress patterns

Appropriate patterns of assignment of phrasal stress.

Prompt

A test item prompt is the information given to examinees for each test item, the basis for their item response. The visual and aural texts of each item.

Pronunciation

Word-level production of the correct assemblage of sounds and syllables or morphemes. Examples of pronunciation errors: 'technoloj' for 'technology'; 'pass-ed' for 'passed'

Prosody/Prosodic features

(Suprasegmental features)

Prosody is the music of language: intonation, rhythm, and vocal stress.

Prosodic features are those features added on top of the consonants and vowels, such as volume, syllable length, stress, tone, pitch, nasality, tension, speed. Also called suprasegmental features because they are in addition to the sound segments or phones.

Range

of vocab

of syntactic structures

of intonational patterns

Vocab range refers to the depth and breadth of vocabulary a speaker has available for use. We say the range is limited if it appears there is some lack of depth of vocabulary that affects how sophisticatedly the speaker expresses ideas. We say the vocab range is restricted if the speaker is unable to successfully express his or her ideas because of a lack of available vocabulary. Sophistication of expression requires a broad range of vocabulary. The same is true for range of syntactic structures. We could also observe that a speaker shows little range of intonational patterns if he or she uses the same pattern in every utterance.

Referent

A person or thing to which a linguistic expression such as "it" or "that" refers.

Register

A variety of a language used for a particular purpose or in a particular type of social setting. For example, formal, academic, or casual register.

Reliability

Dependability, consistency. Yielding the same or compatible results in different situations. We are concerned with the reliability of OEPT test scores. Can different raters assign compatible scores to the same test (can they agree about how to use the scale)? Can individual raters be consistent in how they assign scores across different exams?

Repetitions

See also: Unproductive

Repeating a word several times in a row, or a phrase several times in a row. "I think, I think, I think that that that we should....."

Repetitive

When a speaker repeats the same phrases or ideas several times throughout a single response.

Restarts

see also: Unproductive

When a speaker starts a sentence then starts it again, sometimes with changes, sometimes with no change. Unproductive restart: "I went to the, I went to the store." Productive restart: "I go to the, I went to the store."

Restricted

Restricted range

A speaker's vocab or syntax range may be restricted if they are generally NOT able to convey messages very successfully because they lack the vocab and syntax to do so. Or they may convey a very basic message using only very simplistic language. See "limited." If a speaker is considered restricted in any area (vocab, syntax, fluency, listening comprehension, articulation/pronunciation, prosody), it means that lack of knowledge/ability in that area restricts the speaker's ability to communicate; they are not able to communicate successfully due to these restrictions.

Semi-direct test

A language test performance that is recorded and then rated later as opposed to a direct test in which the examinee is interviewed and rated simultaneously.

Sophistication

Accuracy, complexity, range, and variety in lexis, syntax, structure of argument.

Sound substitutions

Substituting one sound for another systematically. For example, r for l; w for v; AE for ei. If a substitution is not systematic, it may be mispronunciation.

Stress

Emphasis produced by pitch, volume, and vowel length. A vowel, a syllable, or a word can be stressed.

Syllable stress

In multisyllabic words, the placement of primary (and sometimes secondary) stress or emphasis on certain syllables. (tech NIQUE; TECH ni cal; AIR bag; pur DUE) Also called word stress.

Syntax

see also: Grammar

Grammatical rules pertaining to how words are arranged to form phrases and sentences. Commonly used by OEPT raters to refer to word order. We commonly use grammar to refer to morphological errors.

Syntax/grammar errors -

local and global

Local errors affect only a small portion of a phrase or sentence, - a missing article or a bound morpheme for person or number, for example. A global error has a greater effect on the meaning of the entire sentence, and therefore on listener understanding, and may involve incorrect word order.

Tense/aspect

Category of verb primarily used to designate the passage of time.

Unintelligible

At the word level, a speaker is unintelligible if you cannot recognize or decode the words he or she is saying based on the sounds you hear. There is a range of intelligibility depending on the amount of effort it takes the listener to recognize or decode the words. It may take no apparent effort on your part to recognize a word immediately, or you may not recognize the word at all, even after some efforts to decode what you heard. Someone who speaks too fast for you to identify the words being said could also be considered unintelligible.

Unproductive restarts/repetitions

An unproductive restart or repetition is one that is done for no apparent reason, with no noticeable improvement resulting. Productive restarts or repetitions might be done to correct an error or to add new information.

Validity

Uses of scores

Construct validity

Validity can refer to appropriate inferences made based on a test score. For example, an OEPT score of 50 is valid if the examinees who receive that score have sufficient language skills for teaching an undergraduate class. An inappropriate or invalid use of an OEPT score might be to kick someone out of a graduate program who failed the test. An appropriate use is to determine if a grad student can teach in a Purdue classroom. There are many subcomponents of validity. An important aspect of test validity is: Does the test measure what it is intended to measure? (In our case, oral English proficiency of Purdue grad students.)

Variety (of a language)

Variety of register or style

Variety of use of syntactic structures and vocabulary

A variety can refer to a specific form or subset of a greater language family; for example, American English is a variety of English. Variety can also refer to variations of usage, such as registers or styles, used for specific purposes or situations (at a conference you might use an academic variety or register of your language, while at home with family you might use an informal variety or style.)

We look for variety in use of syntactic structures and vocabulary, as opposed to use of the same types of sentence structures used over and over, or the same words and phrases occurring frequently. See 'repetitive'

Verb tense

The time of a verb's action or state of being, such as past, present, and future.

Vocab usage errors

Generally, errors of word choice or word class (for ex. adverb instead of adjective.) Examples: It was very quietly - rather than it was very quiet. I was deferred to a doctor - rather than referred. I got on the car - rather than in the car.

Word stress

Emphasis or stress falling on a particular syllable of a word. Also called "syllable stress". See "phrasal stress".