Communicative competence
The concept of communicative competence has sparked extensive debate, leading to numerous research initiatives aimed at defining what it truly means to know a language Over time, a refined model of communicative competence has been developed to meet various specific needs.
(eg assessment, pedagogy) by various scholars
Canale 1983; Canale & Swain 1980; Celcé-Murcia 2008;
Researchers have not reached a consensus on the definition of communicative competence; however, it is widely recognized as a multi-faceted phenomenon composed of various sub-competencies The interplay and integration of these components significantly determine a learner's proficiency in a language.
Thus, the notion of communicative competence is believed to minimally encompass dimensions relating to the following:
! knowledge of how to arrange formal units of language into unified units of discourse
! knowledge and understanding of the socio-cultural and communicative context in which communication takes place
! knowledge of how to interact successfully with an interlocutor in a communicative exchange in a L2
Bachman (1990) and Bachman and Palmer (1996) conceptualised communicative competence as
Language users possess a capacity that allows them to create and interpret discourse, which is fundamental to effective communication This model consists of two primary components: language competence, referring to the knowledge of the language itself, and strategic competence, which includes metacognitive strategies that guide how users engage with the specific context of language use.
The language knowledge model categorizes understanding into two main areas: organisational knowledge and pragmatic knowledge Organisational knowledge focuses on the structure of utterances, encompassing grammatical and textual aspects, while pragmatic knowledge relates to the communicative goals of users and the context of language use, including functional and sociolinguistic elements Despite being a comprehensive representation of language proficiency, as noted by researchers like Alderson & Banerjee (2002) and Chalhoub-Deville (1997), the specific contributions of each knowledge component to communicative language ability remain unclear, complicating its application in assessment practices (McNamara, 1996) Additionally, the concept of communicative competence has evolved to include spoken communication.
‘interactional competence’ (eg Hall & Doehler 2011; Young 2011).
Discourse competence
Discourse and text
Discourse and text are often used interchangeably, yet scholars have noted distinct differences between the two concepts Widdowson (1984) highlights this discourse-text dichotomy, defining discourse as a communicative process involving interaction that leads to changes in a state of affairs, while text represents the linguistic outcome of this process.
Communication relies on the negotiation of meaning through interaction, a process defined as discourse, which involves the exchange of perspectives for conveying information and intention (p 100) Christiansen (2011) noted that in non-specialized contexts, 'text' often refers to written language, while 'discourse' pertains to spoken language, highlighting the distinction between medium and channel (p 34).
(2011) also noted that this distinction is simplistic and proposed a differentiation based on “text for the form and discourse for the content” (p 34)
Functional–systemic linguistics defines text as encompassing all forms of oral and written communication According to Eggins (1994), a text is any spoken or written passage, regardless of length, that constitutes a unified whole.
(Halliday & Hasan 1976, p 1) To describe the way in which a text enacts itself as a unified whole, Halliday and
Hasan (1976) advanced the notion of ‘texture’, a property that holds “the clauses of a text together to give them unity [and] distinguishes text from non-text” (p 2)
Texture in language is created through cohesion and coherence, among other elements Halliday and Matthiessen (2013) revised the definition of text, describing it as “any instance of language, in any medium, that makes sense to someone who knows the language.”
Text can be defined as language operating within a specific context, highlighting the significance of discourse in shaping meaning Discourse encompasses various textual elements and the linguistic resources available for constructing coherent text, as noted by Eggins (1994).
Within the context of text linguistics, text has been defined as a “communicative occurrence which meets seven standards of textuality” (De Beaugrande &
Textuality is a fundamental characteristic that differentiates text from non-text, as highlighted by Dressler (1981) Scholars assert that textuality is established through adherence to specific standards, including cohesion and coherence When any of these standards are not met, the text loses its communicative function and is categorized as 'non-text.'
Discourse competence
Discourse competence refers to the ability to create and comprehend text effectively According to Canale, it involves mastering the combination and interpretation of meanings and forms to produce cohesive and coherent texts across various modes This includes utilizing cohesion devices to connect forms and applying coherence rules to structure meanings logically.
According to Celcé-Murcia (2008), discourse competence is fundamental to communicative competence, as it integrates linguistic, actional, and sociocultural skills to effectively produce discourse This competence involves the careful selection, sequencing, and arrangement of language elements to create a cohesive spoken message Four key sub-areas enhance discourse processing: cohesion, deixis, coherence, and generic structure.
Discourse competence is essential to communicative competence, as language serves to interpret, negotiate, and convey meaning Effective language use necessitates the integration of various types of knowledge inherent in language ability to produce coherent discourse The concept of textuality, which emphasizes the connectedness of sentences, is vital in distinguishing discourse from disjointed passages For a sequence of sentences to qualify as text, it must meet the criteria of textuality, as outlined by scholars such as De Beaugrande & Dressler (1981), Eggins (1994), and Halliday.
Of the seven standards that texts need to display in order to qualify as text, the features of cohesion and coherence are the core standards that provide texts with
‘connectivity’ (ie unity) (De Beaugrande & Dessler
1981) Cohesion refers to semantic relations between sentences within a text, which offer a text a degree of unity (Cameron, Lee, Webster, Munro, Hunt & Linton
Coherence in writing refers to textual unity and the elements that enhance a text's meaningfulness (De Beaugrande & Dessler, 1981) These features are recognized as essential for creating connectedness within a text, making them key contributors to unified discourse Consequently, cohesion and coherence have garnered significant attention in discourse studies (Halliday).
Analyzing discourse competence in both oral and written performances can be effectively achieved by examining the features of cohesion and coherence This study highlights the importance of these textual attributes in operationalizing discourse competence, as identified by Matthiessen (2013) and Kang (2005).
Investigation of discourse competence in learner performance
The importance of discourse competence has been well acknowledged in studies investigating the quality of learner performance in both writing and speaking
Coherence and cohesion are critical components of discourse competence, as highlighted in various studies Research has focused on the use of discourse markers by international teaching assistants (ITAs) in U.S universities, aiming to pinpoint the challenges in understanding non-native speakers’ speech (Tyler 1992; Williams 1992).
Research has compared the oral production quality of non-native speakers to that of native speakers, while also examining factors that influence speech quality, such as language proficiency, task types, and preparation time Findings indicate that the infrequent or inappropriate use of discourse markers can hinder comprehension (Fung & Carter, 2007; Tyler, 1992) Additionally, the frequency and types of discourse markers vary based on the learner’s proficiency and the nature of the tasks.
(Geva 1992), and planning time (Williams 1992)
Despite recognizing the significance of discourse competence, learners often lack awareness of discourse devices, while educators tend to focus more on grammar and vocabulary rather than these essential elements of L2 proficiency (Hellermann & Vergun, 2007).
Recent research in language testing has increasingly focused on analyzing the language produced by test-takers during oral assessments, both in monologues and interactions Van Lier (1989) highlighted the significance of speech analysis, emphasizing the need to examine oral tests based on actual test performances to ensure validity Douglas and Selinker (1992, 1993) noted that raters might give similar scores for different reasons, indicating that speakers can achieve comparable ratings despite delivering qualitatively distinct performances Previous studies have conducted in-depth analyses comparing rater scores with the actual outputs of test-takers (e.g., Douglas 1994; Fulcher 1996).
More recent studies (eg Brown, Iwashita & McNamara
2005) investigated features of performance identified in rating scales focusing on individual performance, while other studies concerned interactional features observed in oral interview or peer interaction assessment (eg Brooks
In a context of speaking scale development for the
A study by Brown et al (2005) investigated the correlation between specific characteristics of spoken language produced by TOEFL iBT test-takers and the holistic scores assigned by raters The analysis encompassed 200 spoken performances across five different tasks and proficiency levels, evaluating various aspects such as grammatical accuracy, complexity, vocabulary, pronunciation, and fluency.
The study highlighted that specific features from various categories significantly influenced overall performance levels, with vocabulary and fluency being the most impactful While Brown et al (2005) explored certain elements of discourse competence, such as conjunction usage and schematic structure within a limited dataset, many other aspects of discourse competence remain underexplored.
Despite the common belief in the importance of discourse competence within the communicative competence model, there is limited understanding of how this competence is demonstrated in speaking test performance This study aims to explore the relationship between discourse competence and learner performance on speaking tasks, addressing a significant gap in existing research.
What are the distinctive features of performance that characterise test-taker discourse in IELTS Speaking Task 3 at each of the Levels 5, 6 and 7?
Data
The present study analysed transcribed speech samples provided by IELTS The data comprised a total of
58 test-taker performances corresponding to the three proficiency levels (ie, Levels 5, 6 and 7)
The study involved 58 participants representing a wide array of first language (L1) backgrounds, encompassing 22 different languages The most prominent L1 group was Chinese, with 8 participants, followed by Tagalog, Urdu, Vietnamese, and Arabic, each contributing 5 to 6 participants Notably, no single L1 group was predominant at any level, as the diverse L1 backgrounds were evenly distributed across all levels.
Approximately 32% of the performances in the data correspond to female test-takers (Female", Male6) Detailed information about the test-takers’ L1 and levels is summarised in Appendix 1.
Method
This study's methodology is based on prior research that examines test-taker performance by analyzing writing skills, specifically focusing on IELTS Academic Writing Task outcomes across different proficiency levels.
This study, unlike prior research, aimed to identify unique characteristics of oral discourse in IELTS Speaking Part 2, where test-takers discuss a specific topic Participants are given two minutes to prepare and can take notes for use during the interview The examiner typically concludes this section by asking one or two related questions.
In the data analysis, discourse competence was defined through the textual features of cohesion and coherence, which are recognized as essential elements in constructing discourse (Canale, 1983; Halliday &).
Matthiessen 2013), but they have also been integrated as
The IELTS rating scale emphasizes the significance of cohesion and coherence as essential elements of discourse competence in test-taker performance (Banerjee et al., 2004, p 11) This study investigates discourse competence by analyzing the levels of cohesion and coherence present in the candidates' performances.
To achieve cohesion and coherence in discourse, we conducted qualitative and quantitative analyses of the textual resources utilized by IELTS test-takers We compiled a section of transcribed data from IELTS Speaking Part 2, where test-takers delivered monologue responses, into a database This data was then analyzed to assess the levels of cohesion and coherence present in their performances.
Analysis
Cohesion
In order to examine cohesion in the test-taker discourse, it was considered best to employ the method used by
Banerjee et al (2004) examined writing performance in the IELTS test by analyzing features such as cohesive devices, lexical richness, syntactic complexity, and grammatical accuracy While they aimed to create a "reliable learner language profile," the categories employed in their study did not fully capture test-taker performance in terms of discourse.
Furthermore, although the study acknowledged the importance of the features of cohesion and coherence in
The study focused on IELTS rating scales and examined cohesion devices, specifically demonstratives as anaphoric references, as well as ellipsis and substitutions Consequently, we selected the four aspects of cohesion defined by Halliday and Matthiessen (2013) to guide our analysis These four methods for identifying cohesion in English are detailed below.
1 Conjunction: This resource “creates cohesion by linking whole clauses or combinations of clauses”
(Halliday & Matthiessen 2013, p 604) It represents logico-semantic relationships between components of a text at the clause level
2 Reference: This generates cohesion by creating links between an element of the text and something else
(entities, facts, or phenomena) in reference to which it is interpreted
3 Lexical cohesion: This resource operates at the lexical level and “it is achieved through the choice/selection of lexical items [ ] these cohesive relations [may] hold between single lexical units [or] wordings having more than one lexical item in them”
4 Ellipsis and substitution: These cohesive resources function at the level of the clause or a smaller item Ellipsis “allows for the language user to leave parts of a structure when they can be presumed from what has gone before” (Halliday & Matthiessen 2013, p 606) while substitution enables the replacement of one item by another
This study aimed to assess the level of cohesion in test-taker discourse by analyzing three of the four cohesive resources identified earlier While previous research (Banerjee et al 2004) examined ellipsis and substitution in IELTS test-taker discourse, our initial analysis also sought to identify these features in test-taker performance Ultimately, the analysis was discontinued due to the infrequent occurrence of ellipsis and substitution in the performances observed.
Each transcribed file underwent a thorough scan to detect conjunctions that indicate textual relationships These conjunctions were then categorized into four distinct types based on their logico-semantic functions This classification aligns with Martin’s (1992) framework, which outlines the four primary categories of conjunctions.
! additive (eg and, or, moreover, in addition, alternatively)
! comparative (eg whereas, but, on the other hand, likewise, equally)
! temporal (eg while, when, after, then, meanwhile, finally)
! consequential (eg so that, because, thus, since, if, therefore)
A thorough analysis of the transcribed performances across various band levels was conducted, resulting in the calculation of the total occurrences of each type of conjunction, along with the overall count of conjunctions used An example of this analysis and its coding is provided below.
113 the job i’d like to do is sports management (.6) then what i consequential
114 need to bring the job is to manage the different kinds of
115 sports (.5) and share advice with different sports people and additive
116 note their (.7) feelings or attitude towards the sport they additive
117 do (.8) then (.7) why- wa- they skills i need to the job consequential
118 (.2) can be:: (.8) i have to be active in the sports i
119 manage (.5) andi have to be able to:: (.8) know how the additive
The analysis of conjunction usage revealed significant findings in lines 113, 115, 117, and 119, where each conjunctive element was highlighted and examined within its contextual framework These elements were then categorized, with sub-categories noted in an adjacent column Statistical analysis was conducted to determine the mean and median of conjunction types and their frequency across different band levels of test-taker performance It is important to highlight that the frequent use of additive conjunctions, like 'and,' and consequential conjunctions, such as 'because,' may be indicative of the characteristics of oral language, potentially serving as fillers in speech.
The current study focuses on the analysis of referential expressions, specifically anaphoric reference, due to previous research by Banerjee et al (2004), which indicated that most occurrences in their data were instances of anaphoric reference An example of this data coding can be found in Table 1.
Line Text Reference Referent Accuracy
67 okay (.) alright (.) thank you (1.0) the piece of equipment that I find
68 very useful (.7) is in the home (1.2) and is the rice cooker (.8) the 1 rice cooker
69 rice cooker is very useful to cook (.) because it is very easy to use (.) it 1 RI
70 erm (1.5) the instruction when I bought the equipment is very easy to
71 follow (1.5) and I got it in 1997 (1.3) I think that er::
72 equipment is very useful to for everyone (.7) and especially for me (.)
73 besides I used it for cooking rice (.4) and I also use to warm the rice it 1 RI
74 (.9) the night before I cook the rice (1.6) but I can use it when I want it 1 RI
75 to make some cookies (1.1) I can use it for that (.7) and also I can you it 1 RI
76 it to boil some water (.5) this piece of equipment (.) I find it very It 1 RI
Assessment No of referential expressions – 6,
Table 1: Example of analysis of referential expression (Level 6, ID606)
In the analysis, instances of anaphoric reference were identified in lines 69 and 76 with the pronoun 'it', which linked back to the noun 'rice cooker' mentioned in line 68 Once the referent 'it' was confirmed as 'rice cooker', it was documented in the referent column as '1 rice cooker', indicating it as the first referent in the text.
The referential expression 'it' in line 69 refers back to 'rice cooker,' indicating agreement and is marked as 'RI' in column five Similarly, in line 75, the test-taker uses 'it,' which, when analyzed within the surrounding context, seems to connect to referent 1.
In the analysis of referential expressions, the term 'rice cooker' was identified in line 68 and noted as 'it' in the reference column, with a count of 1 in the referent column The total number of referential expressions and the percentage of accurate usage were calculated and presented at the bottom of the table In this instance, the test-taker utilized six anaphoric references, all of which were correctly identified, resulting in an accuracy rate of 100%.
Table 2 illustrates that out of five referential expressions ('they') analyzed, only one could be accurately traced back to its referent Consequently, the overall accuracy of the referential expressions was determined to be 20%.
Line Role Text Reference Referent
Accuracy RI– right WR- wrong
200 schoo::ls: (0.4) i believe has a really (.) really great impact 1 school
201 on students:: (0.6) they for onc::e (0.7) are the big
202 factor (0.7) for? (0.2) for the:: growth and development (0.2)
206 of every children that comes along- that comes:: they are (used they No referent WR
209 E so do they effectively prepare people for jobs? they 1 school RI
211 er yes i believe [so::: ] (0.4) er::m:: (0.3) it is a stepping212 [m hm::]
213 =sto::ne? (0.7) erm (0.7) learning in- a- in an (.)
214 institutional school (0.2) would serve an es- (.)
215 (0.3) erm stepping stone for an individual (0.2) to be
216 effective (0.3) to be fruitful (0.3) erm::: (1.1) in their their No referent WR
217 er:m: (0.2) in the jobs that they er:: want there to be they No referent WR
E – examiner Assessment No of referential expressions – 5
Table 2: Example of analysis of referential expression (Level 6, ID263)
Strategies to achieve cohesion involve the use of linguistic resources (grammatical items), however, cohesion can also
Cohesion in discourse is achieved through the use of lexical items, which establish relationships among content words (Halliday & Matthiessen, 2013; Paltridge, 2000) The analysis of lexical cohesion in the data focuses on four key relations: repetition, synonymy (e.g., pupil vs student), hyponymy (e.g., animal for dog, kangaroo, koala), and meronymy (e.g., apple as a part of an apple tree) An example will demonstrate the analysis procedure and coding involved in this process.
123 (.3) i’m not sure (.) i’m not er:: (.5) i don’t know ab-
124 (.3) about foreign countries (1) like canada (a) america (b) or (.4) foreign countries (1) hyp 1a hyp 1b
125 er:::: eu- (.6) europe (.4) [but ] in iran (c) i am:: (.6) a c-126 [mhm]
127 accustomed to iran’s(c) house (.3) [er ] (.2) in (.5) in the= hyp 1c x 2
129 =past the houses were very big [for ] example er a- i mean(Level 7 ID206) (Note: hyp – hyponymy)
In line 124, the test-taker identified the lexical category of 'foreign countries,' marked as 'foreign countries (1),' which served as the foundation for the lexical chain Following this, the test-taker included three more lexical items: 'Canada' and 'America' in line 124, along with 'Iran' in line 125.
Coherence
Cohesion pertains to the internal elements of a text, whereas coherence relates to its contextual significance and how it connects with the surrounding situation (Paltridge, 2000) To ensure a text is meaningful, there must be effective interaction between the information presented and the existing knowledge that readers or listeners bring to the context.
Coherence in a text arises from the relationships among propositions, the connections between concepts, and the logical organization of thematic content Language users follow conventional principles of discourse organization to establish coherence This study operationalized the assessment of coherence in test-taker performance by examining text generic structure and the development of themes and rhemes.
Text generic structure pertains to the formal schemata of a text that enables users to recognize various types of oral discourse, such as conversations, narratives, interviews, service encounters, reports, lectures, and sermons This concept is based on the generic structure mappings adapted from the model established by Eggins in 2012.
To assess test-takers' adherence to a specific text structure, we analyzed the performances of 58 participants in a two-step process Initially, we reviewed the prompts given to the test-takers during the speaking task, as illustrated by the example of the examiner's discourse elicitation.
101 E: [he]re’s some paper? (.) and a pencil for making notes
102 (0.5) hhh and here’s your topic (0.6) i’d like you to
103 describe a JO::B (.) you think would be interesting
In Speaking Part 2 of the exam, the examiner asked the test-taker to describe an interesting job, which indicates that the prompt is designed to elicit a descriptive text response.
Secondly, we conducted analysis to establish the degree of test-takers’ compliance with the conventional generic structure associated with particular text types (Paltridge
2000) The following is an example taken from the data file in the study of an IELTS Speaking Part 2 question asked of a test-taker by the IELTS examiner
[ ] good, paper and pen there for making notes (.2) and here’s your topic (.3) so i’d like you to describe your favourite newspaper or magazine
In our initial analysis, we followed Paltridge's (2000) framework for categorizing text types and their structures, focusing on the specific goals and expected moves within various genres For instance, descriptive texts are defined as discourse aimed at portraying a particular person, place, or object.
(p 111) Its schematic structure was suggested to include two items or moves, namely, identification (id) and description (de) It has the conventional structure as illustrated
Purpose: To describe a particular person, place or thing
The analysis revealed that the test-taker was tasked with producing a descriptive text We assessed the extent to which the test-taker's performance aligned with the recommended generic structure for descriptive texts To evaluate this compliance, we developed a scale based on the prototypical text schematic structure outlined by Paltridge (2000) This scale categorized compliance into three levels: 1, 0.5, and 0.
0) indicating the degree of test-taker discourse adherence to the suggested structure for a given type of text:
1 for conforming to generic schematic structure to a full extent, 0.5 for some extent, and 0 for no obvious structure conforming to generic schematic structure
Test-takers were instructed to select an item and not only describe it but also articulate the rationale behind their choice, highlighting its usefulness.
Therefore, test-takers’ descriptions included some explanation The examples below illustrate the generic structure analysis
In the following example, the test-taker was asked to describe their favourite newspaper The test-taker identified it as ‘the times of india’ in line 208, noted as
‘id a’ in the adjoining column (Note that the letters ‘a’,
The test-taker effectively described the newspaper by detailing the types of information it contains, as noted in line 218 They also introduced the various sections of the paper, explaining the specific information found in each By producing a descriptive text that adhered to the schematic features of this discourse type, the test-taker met the task requirements and received a rating of 1.0.
208 yeah like er my (.3) favourite newspaper is the times of india (a) id a
212 and like er: (.4) only cos i have seen (.) it’s the (.2)
213 newspaper which i have been using since the time i was born
217 and it’s (.) really (.2) unique (.5) in aspect like it covers
218 all parts er (.2) life (a) (.) like (.6) starting the front page de a
219 is the headlines (b) (.6) where is all the political news and de b
220 whatever happening (.) around the world
224 and then comes into the bangalore city (.6) and you get all
225 the crime news and whatever happened in the previous day (c) (.3) de c
226 in bangalore (.7) and then you have the business (.7) where
227 you get all the business related materials (d) (.6) and the sports de d
228 section (e ) (.2) and you have an exclusive section for the de e
Sure, please provide the content of the article you'd like me to rewrite.
In a recent assessment, a test-taker was tasked with describing a watch While they listed potential uses of the watch, their description lacked crucial details such as the materials, brand, and color of the item Consequently, the performance was rated 0.5 due to its insufficient adherence to the conventional structure of descriptive writing.
90 The watch (a) (.6) we: use it for uh: (.6) every, kind of (.8) in any in ev- id a
91 every time (.2) in the time ((inaudible 2.7)) do anything (de a) (.5) eh we de a
92 (.5) we finishing eh fin- finishing and work (de b) uh: (.3) when uh we have de b
93 ha- uh (.5) ha- have an appointment (de c) (1.1) when uh (.9) we uh with de c
94 with with other (1.1) eh men (1.5) we uh: (1.0) ((inaudible)) (.9)
In this example, the test-taker selected their favorite structure, the Bayoke Building, as indicated in lines 146–147 (noted as ‘id a’) They proceeded to detail the building's unique properties and characteristics, highlighting its architectural features and significance.
The performances rated as 'de a', 'de b', 'de c', 'de d', 'de e', and 'de f' received a score of 1.0, indicating full compliance with the descriptive discourse schematic structure Notably, no performance was rated 0 on this scale The assessment of all 58 test-taker performances utilized the generic text structure compliance index, followed by the calculation of the mean and standard deviation for comparative analysis across different levels.
146 going to (.) to say is:: (.6) is the bayoke (.8) bayoke
147 building (a) (.7) it (.3) it is:::: the (1.3) tallest building in id a
148 (.2) thailand (a) (.8) erm (2.2) erm it is::::: situated (.7) de a
149 on: (.) pr- (.7) pat- (.) pat- (.6) pratunam (b) (.6) road (.6) de b
150 °(i should think)° (.3) hhhh yes (.) erm::: (1.4) it is::: u-
151 (1.2) it is the 1.1 building fo::r (.3) erm (.5) used for like
152 (.3) hotel (c) :: (.6) um like (.5) apartment (d) (1.3) a::nd (.4) de c / de d
153 some restaurants (e): (.6) in there (.8) yeah erm:: (.8) i think de e
154 it (.) it is::: (1) you can say that it is the land- (.2)
155 landmark of:: (.2) bangkok (f) (1.2) erm (3) although it- (.2) it de f
(Level 5 ID209) 4.3.2.2 Theme–rheme development
Theme-rheme development is a crucial organizational pattern in discourse that helps create a unified message (Halliday & Matthiessen, 2013) The theme serves as the starting point of the message, representing the first structure that functions as a participant, circumstance, or process In contrast, the rheme is where the theme is further developed Together, these elements form the structure of a message Thematic progression, defined as the relationship between themes and rhemes across clauses, indicates how a theme may reflect or build upon previous meanings (Paltridge, 2000) This study focuses on identifying theme-rheme development within main clauses, utilizing clause boundaries marked by tone-units, as suggested by Roche (2000), where pauses longer than 0.6 seconds signify the end of a clause Examples of this analysis will be provided.
113 the job (t1) i’d like to do is sports management (r1) (.6) then what i
114 need to bring the job (t1) is to manage the different kinds of
115 sports (.5) and share advice with different sports people and
116 note their (.7) feelings or attitude towards the sport they (r2)
(Level 5 ID202) (Notes: t – theme, r – rheme)
In the provided example, the term 'job' serves as the starting point of the message and is categorized as the 'theme.' The accompanying 'rheme' expands upon this concept throughout the remainder of the clause This illustrates how theme 1 is identified and coded for clarity in communication.
Lexical richness
The current study not only assessed the level of discourse competence in test-taker responses but also included an analysis of lexical richness in their performance According to Skehan, this dual focus enhances the understanding of language proficiency in test-takers.
(2009), lexis can be used as a measure of general performance as it “represents a form of complexity that
[…] has to be assessed in second language speech performance if any sort of complete picture is to be
To enhance the understanding of test-taker performance across various IELTS levels, we deemed it essential to incorporate measures of lexical richness into our analysis In this study, lexical richness was operationalized through specific categories of analysis, providing a comprehensive characterization of test-taker performance.
1 lexical output: analysis to look at the number of tokens produced in test-takers’ performance at different IELTS levels
2 lexical variation: analysis to explore the number of different word types in test-taker performance in relation to the total number of words produced
3 lexical density: analysis aiming at calculating the proportion of lexical words to grammatical words in test-taker performance
4 lexical sophistication: analysis to provide information about the number of unusual words (in relation to specified word lists) in test-taker performance at different IELTS levels of proficiency
To analyze four key lexical features, we utilized the web-based tool VocabProfile (Cobb 2013), which assesses the distribution of low and high-frequency vocabulary among both native and non-native speakers This program focuses on vocabulary measurement to provide insights into language use.
Profile (Laufer & Nation 1995) and performs lexical text analysis using the Academic Word List (Coxhead 2000)
VocabProfile not only computes metrics for word-token, word-type, and type-token ratios but also analyzes the percentage of words across four categories, including the most frequent 1,000 words in English.
(K1), the second most frequent 1000 words of English
(K2), words found in the Academic Word List (AWL), and any remaining words (Off List) It also calculates lexical density and word-token per word-type
To analyze the transcribed speech, we removed repair features and imported the data into VocabProfile We generated frequency counts for nine measures, focusing on word-token and word-token per type to evaluate lexical output The word-type measure assessed vocabulary range, with the hypothesis that more proficient speakers utilize a broader variety of word-types Additionally, we examined lexical sophistication by calculating the percentage of words across four categories: K1, K2, AWL, and Off List.
We examined the type-token ratio as part of our analysis, which assesses the semantic density of speech and can differ based on proficiency levels As proficiency increases, the number of tokens may rise without a corresponding increase in types, leading to variations in the density of speech across different levels.
In this section, we report the results of the data analysis in light of the research question, including the findings of both quantitative and qualitative data analyses
For cohesion, we report the results yielded from both quantitative and qualitative analyses In reporting the results of the analysis of the three categories of cohesion
(ie, conjunction, reference and lexical cohesion), we use frequency data (ie, per 100 words) as the length of speech varied
To effectively compare performance across three levels, the bootstrapping method was employed due to the non-normal distribution of data and significant individual variation, indicated by large standard deviations Bootstrapping is a robust statistical technique that allows for reliable analysis under these conditions.
In our study, we employed non-parametric statistics, specifically the Kruskal-Wallis test, due to the small frequency of each feature under investigation Additionally, we conducted qualitative analysis to support and validate the outcomes of our statistical findings This approach allows for a more stable and statistically accurate interpretation of the observed data set (Plonsky, Egbert & Laflair, in press).
This article presents descriptive statistics including means, standard deviations, and medians, with a primary focus on qualitative analysis due to the nature of the data Section 5.2.1 discusses the generic text structure compliance index, while section 5.2.2 compares the descriptive statistics of main clauses Additionally, we analyze lexical richness through quantitative methods, utilizing parametric statistics such as one-way ANOVA.
Cohesion analysis
Conjunction
Table 3 presents a summary of the descriptive statistics regarding conjunction usage, highlighting the total number of conjunctions employed in each data file It also reveals the preferred types of conjunctions utilized by the test-takers, following the taxonomy established by Martin (1992).
In the analysis of conjunction usage across various levels, additive conjunctions emerged as the most frequently utilized, followed by consequential conjunctions In contrast, comparative and temporal conjunctions were used sparingly, with temporal conjunctions showing the lowest frequency across all levels This predominance of additive and consequential conjunctions can be attributed to the characteristics of oral language.
Additive conjunctions like "and" and consequential conjunctions such as "because" may have been used as fillers in the analysis The independent non-parametric test results, summarized in Table 4, indicate a significant difference in conjunction usage among three levels, specifically for comparative conjunctions Post-hoc testing revealed significant differences between Level 5 and Level 6 (U = 10.5, p = 018), as well as between Level 5 and Level 7 (U = 58, p = 000).
Type Mean SD Median Mean SD Median Mean SD Median
Table 3: Descriptive statistics of use of conjunctions (frequency, per 100 words)
Table 4: Results of Kruskal–Wallis analysis (conjunction use)
Table 5 and Figure 6 illustrate the distribution of four conjunction types utilized by test-takers across three levels In Level 5, the combined use of additive and consequential conjunctions is more prominent compared to Levels 6 and 7 Conversely, as the levels increase, there is a noticeable rise in the use of comparative conjunctions Additionally, the proportion of temporal conjunctions varies significantly between Level 5 and Level 7.
Table 5: Distribution of conjunction use (%)
Figure 6: Distribution of conjunction use
The qualitative analysis concentrated on two key aspects: identifying the types of conjunctions present in performances and assessing the alignment between specific conjunctions and the relationships they establish between clauses For instance, in the additive relations category, we evaluated whether the conjunction ‘and’ accurately represented an additive relationship in the test-taker's performance The following extract illustrates the use of additive conjunctions (in bold) in a Level 5 performance.
83 help immediately (.8) I first used it er (1.0) three years ago I remember
84 well (.) when I was in high school my mother er (.8) bought me one for 85 temporal
85 pr- for my birth day (.6) and I have it since (.2) er with me and er (.8) additive
86 now er (.) now (2.0) because I am in another town from my parents they consequential
87 (.) find me any time (.2) and we (.) talk (.6) er I think it is useful (.) additive
In this analysis, the additive conjunction "and" appears twice, creating a relationship of addition between clauses In the first instance, it connects "my mother bought me one for my birthday" with "I have it since with me," indicating that the second clause provides additional information about the first Similarly, the second occurrence of "and" links "I am in another town from my parents they find me any time" with "we talk," further enhancing the context of the relationship described.
The analysis confirmed that the second clause provided additional insights in both cases Notably, it revealed that the test-taker retained the gift from his mother, as mentioned in clause 1 (lines 84-85), and maintained communication with his parents despite being in a different town, which was also highlighted in clause 1 (lines 86-87).
The analysis reveals that additive conjunctions effectively establish addition relations among clauses across various performance band levels Notably, there are no significant differences in the types of additive conjunctions preferred by test-takers at different levels, indicating a consistent usage pattern Specifically, the conjunctions 'and' and 'or' were used interchangeably among all band levels to convey additive relations, with no other additive conjunctions noted in the performances.
Test-takers across various proficiency levels utilized a range of conjunctions, with an increase in the frequency and diversity of comparative conjunctions as proficiency improved Higher-level test-takers demonstrated the ability to employ comparative conjunctions for both positive and negative comparisons (Paltridge 2000) For instance, in Level 5 performance data, a test-taker effectively compared different television show types, expressing a preference for news over others by using the conjunction 'but' in line 134.
132 (0.4) the kind of mo- er of (1) the 1.show that i like is (0.3)
133 ((inaudible)) or talk show quiz show (0.4) er (0.3) and news additive
134 (0.2) because it’s not just only for relax (0.5) but is give me comparative
135 an (0.9) er (0.2) about (1.8) er the thing that happen
Comparative conjunctions were less frequently used in Level 5 performances compared to Levels 6 and 7 For instance, in a Level 6 example, the test-taker effectively employed a variety of comparative conjunctions, including 'like' and 'as' (line 260) along with 'but' (line 262).
The test-taker reflects on their father's expectations for them to pursue the same career, using the conjunctions "like" and "as" to draw a positive comparison However, they express their disagreement with this expectation by employing the conjunction "but," indicating their dislike for the job in contrast to their father's perspective This creates a nuanced comparative relationship that highlights both admiration and personal preference.
251 m:: (0.5) the (0.2) the the: international situation (0.3)
252 an:::d the:: (0.3) the weapons (0.2) and the cars (0.3) and so additive
253 so many (0.4) so i like the (0.5) the- (0.6) the time magazine consequential
254 (0.9) and erm (0.7) m: (0.8) er the fame- the famous part (0.2) additive
255 er (0.2) i like is car (0.2) er (0.7) cos m:: (0.6) (clears
256 throat) (0.7) my father was er er was (producter) (0.4) er was
257 m:: (0.6) was a car fact- (.) er was a car factory (0.3) worker 258
259 came here my father hoped me (0.3) to: to do er:: (0.4) to
260 continue to work (car) (0.6) like the same as my father (0.5) 261 comparative
261 to do er car (0.2) to to product the car with him (0.5) but eh
262 (0.5) er: (2) but i don’t (0.3) i don’t like it (0.3) because consequential
263 i don’t like to (0.4) erm (.) to product i like to see it (0.6)
The following example demonstrates the effective use of comparative conjunctions in a Level 7 performance, highlighting four instances of comparison: 'as', 'but', 'but', and 'like' These conjunctions, found in a discussion about medical equipment, establish comparative relations that convey both positive meanings ('as' and 'like') and negative implications ('but') This analysis is supported by Paltridge (2012).
229 equipment which eh (0.6) which I found very useful, (.) as I told you (.) comparative
230 as a doctor (0.5) we prefer our equipments (0.7) and eh there are one eh additive
231 like in modified situation like CT scan and many things like that (0.4) additive
232 but eh the basic eh (0.4) equipment (.) which is long lasting time tested comparative
233 (0.6) is our stethoscope (1.1) yeah and eh (0.3) I think doctors are eh additive
234 identified by the stethoscopes (.) no ma- (0.4) no matter if it’s really consequential
235 ((inaudible)) (0.8) and when I wear the (.) stethoscope people feel that he additive
236 is a doctor (0.5) and (.) this is e- this is equipment which gives us eh additive
237 (0.3) identification which g- gives us (0.3) eh a help in our eh (0.4)
238 diagnosing a common patients (1.4) I eh (0.5) when I became doctor I temporal
239 was in final year (0.5) I first time used the stethoscope and was quite eh additive
240 (0.3) eh convinced to s- to feel that it it it it is going to have (0.3) eh in
243 particularly when I (0.3) listen the heart sounds (0.6) and eh th- i-it temporal, additive
244 gives you a specific feelings eh (.) eh you know (0.3) that’s the
245 difference (0.4) but then I change my field and went into just medicine comparative, additive
246 speciality (0.5) where I found that this (0.3) this (.) particular instrument 247
247 (0.3) is the key in my field (0.6) because (.) it it really helps in consequential
248 diagnosing disease- (.) diseases (0.6) which even (.) other invasive
249 investigations like j- eh like CT scan, (0.3) like eh (.) bone scan like eh comparative
As proficiency levels increase, a greater variety of comparative conjunctions in Level 7 suggests an enhanced ability to establish comparisons in text For instance, in example ID710, the test-taker's use of diverse conjunctions indicates that the articulation of comparative relations in advanced discourse develops in harmony with other logical relations, such as temporal, additive, and consequential connections.
Reference
Table 6 presents the results of the analysis in terms of the total number of referential expressions used and the percentage of referential expressions that were used accurately
Mean SD Median Mean SD Median Mean SD Median
Table 6: Descriptive statistics of use of referential expressions (frequency, per 100 words) and percentage of accurate use
In analyzing test-taker performance, the frequency of referential expressions (such as "it," "they," and "there") was highest in Level 5 and lowest in Level 6, though this difference was not statistically significant (H = 2.72, df = 2, ns) Conversely, the accuracy of these expressions was notably highest among Level 7 test-takers and lowest in Level 5, with significant statistical differences observed (H = 1.24, df = 2, p < 006) Post-hoc tests revealed significant differences in accuracy between Levels 5 and 7 (U = 80, p = 003) and between Levels 6 and 7 (U = 109, p = 013).
The qualitative analysis indicated that higher-level test-takers effectively established a clear relationship between reference items and their antecedents For instance, in Table 7, a Level 7 performance demonstrated the test-taker's use of multiple reference items, such as 'it' in line 120, 'they' in line 123, and 'there' in line 126 We assessed the alignment between these expressions and their referents, noting that 'it' in lines 119 and 120 was traced back to its referent, 'Maldives,' in line 119.
When test-takers expressed sentiments like "it's such a beautiful place" and "it makes me breathless," listeners could easily identify the Maldives as the subject Similarly, successful relationships were established through referential expressions such as "they" in lines 122 and 123, referring back to "white sand beaches," and "they" in line 125, which referred to "people."
Line Text (excerpt) Referent Reference Accuracy
119 but eh (.) Maldives is such a nice place (.3) and it ’s such a beautiful
120 place (.3) that eh (.6) mm I just (.6) it ga- it (.3) it gave me
121 make me (.) breathless (.5) and the scenery (.3) and eh
122 beaches (.) white sand beaches (.) they were (.4) I mean (.4) 2 white sand beaches they RI
123 outrageously (.6) and eh (.) outrageous (.) they were (.3) they were so 2 they RI
124 beautiful (.4) and em (.4) what I (.) found over there (.) that eh (.3) 2 there RI
125 the people (.) over there (.6) they are Islamic in a sense that when I
126 went over there it was the holy month of Ramadan (.) and eh (.3) I 1 1 there
127 found out that eh (.) the shops were closed (.6) eh around six o’clock
Table 7: Example of Level 7 performance (excerpt) (Level 7 ID709)
In lower-level performances, such as Level 5, the phenomenon of establishing connections between reference items and their antecedents was less evident For instance, as shown in Table 8, many test-takers struggled to effectively link reference items like "this" to their corresponding antecedents, highlighting a gap in comprehension and coherence in their responses.
‘they’ in line 140, ‘this’ in line 142), but these reference items cannot be satisfactorily traced back to an antecedent
The ambiguity in the use of 'this' in line 140 complicates the identification of whether it refers to the paper or magazine mentioned in line 139 This uncertainty makes it challenging to recover the reference item effectively Similarly, in Table 9, the antecedent for the reference term 'they' in line 65 remains unclear and cannot be traced back to the text.
Line Text (excerpt) Referent Reference
139 information in er (.4) in this (.4) paper (.4) this magazine
140 (.4) erm some tier (.2) in this/they are many (.3) er (.3) No ref this/they WR
141 (page) for entertainment hhherm an er: especially i like uh
142 (.3) er the (1.4) information about competition in this (.3) No ref this WR
143 because when i finished er university i went to hhher (.2)
144 join in er (.2) some competition with (.4) everybody (2.3)
Table 8: Example of Level 5 performance (excerpt) (Level 5 ID226)
Line Text (excerpt) Referent Reference Accuracy
64 (.4) choose France (.) err: Paris is the capital of France
(.) err:: and it is 1 Paris it RI
65 also (1.5) a fashion city (.4) errm (.) they also have destinies land there No Ref they WR
66 (.) so I went there (.8) err::m (2.1) I went there by Eur-
(.) Eurostar and 1 Paris there RI
67 I think this is a great city to visit (.) and (.) because errm
(2.5) the people 1 Paris this RI
68 are very nice (.) and also the place is also very nice (.7) and (.) I think
69 this is there is (.4) a good place to buy (.8) clothes (.7) so I spent a lot
Table 9: Example of Level 5 performance (excerpt) (Level 5 ID501)
Proficient test-takers establish a clear relationship between reference items and their referents, enhancing the listener's ability to identify previously mentioned elements This alignment leads to a more effective use of cohesive reference resources, resulting in comprehensible texts Quantitative analysis indicates that higher accuracy in referencing contributes to greater coherence in their performances.
Lexical cohesion
Table 10 highlights the use of lexical cohesion devices, revealing that their overall frequency is quite low, with the exception of repetition and hyponymy Notably, the usage of hyponymy increases with higher levels of analysis.
The analysis revealed a statistically significant difference in hyponymy and repetition across various levels, as detailed in Table 11 Specifically, post hoc analysis indicated a notable difference in hyponymy between Levels 5 and 7 (U = 32, p = 000) In contrast, a higher frequency of repetition was noted as the levels decreased, with significant differences observed between Levels 6 and 7 (U = 66, p = 000) and Levels 5 and 7 (U = 35, p = 000).
Table 12 and Figure 7 illustrate the distribution of lexical cohesion devices across different performance levels, with percentages calculated from the total frequency of each type Notably, the use of hyponymy and repetition in Level 5 data contrasts with that in Level 7 data In Level 6, the proportion of repetition and hyponymy closely resembles that of Level 7 Additionally, a qualitative analysis was conducted to annotate the use of lexical items as cohesive resources, with examples provided to demonstrate the identification and coding of these items in the data.
Type Mean SD Median Mean SD Median Mean SD Median
Table 10: Descriptive statistics of use of lexical cohesions (frequency, per 100 words)
Table 11: Results of Kruskal-Wallis analysis (lexical cohesion)
Table 12: Distribution of lexical cohesion (%)
Figure 7: Distribution of lexical cohesion
Meronymy Repetition Hyponymy Antonymy Synonymy
The term 'computer' is introduced in line 97 and subsequently repeated in lines 97, 100, 101, 102, and 108, demonstrating the test-taker's preference for repetition to create lexical cohesion However, as proficiency levels rise to 6 and 7, the reliance on repetition as a cohesive device diminishes, as evidenced by examples from these higher-level performances.
95 (3.5) today I am going to talk about (.2) a 1.piece of equipment (.4) equip 1
96 which I find is useful (.4) in my opinion the equipment is (.2)
97 2.computer (.6) as you know in modern society computer (a) is many (.6) 98 comp 2 hyp 1a rep 2a
98 bring many benefit to our live (.5) um firstly (.5) um: in study (.5)
99 secondly in work (.3) and finally (.2) in daily life (.5) firstly I will 100
100 talk about the advantage of using computer (b) (.) in study (1.0) um I think 101 rep 2b
101 computer (c) is most useful because (.4) er in er study (.3) you can use er 102 rep 2c
102 computer (d) (.) to finish your homework (.) your assessment (.3) even you 103 rep 2d
103 use the computer (e) um (.6) especially using internet (.2) to find (.2) er 104 rep 2e
104 huge information (.3) in the internet (.2) and (.3) it can save your
105 time and your money (.2) because you can do (.) you can finish your 106
106 (.5) you can finish your homework (.2) very quickly (.2) and
107 sometime you can find (.4) some er large information (.) using
108 computer (f) (.2) you don’t need to go to library or (.2) to book (.2) or to 109 rep 2f
(Level 5 ID512) (Notes: hyp – hyponymy; rep – repetition)
In Level 6 performances, the use of repetition as a means of establishing lexical cohesion is less frequent compared to Level 5 Notably, this example showcases how repetition is enhanced by hyponymy, highlighting the test-taker’s capability to utilize a broader range of resources for effective lexical cohesion in the text.
57 uh::m washing machine (1) (1.6) uh::m ten years before the washing w machine1
58 machine (a) is very simple made ((inaudible)) (.6) and now I saw the rep 1a
59 washing machine (b) got lots of (1.1) got lots of ((inaudible)) different (.4) rep 1b
60 ways for working (.2) they can wash the clothes and dry the clothes
61 (.5) this is that impossible when I was a child (.6) an:d uh:: (.4) TV (c)
62 (.2) digital TV (1.8) uh:m maybe I don’t know when (.7) the TV co-hyp 1c
63 before (.5) is black and white (.4) now it’s colour and changing to the
(Level 6 ID603) (Notes: rep – repetition; co-hyp – co-hyponymy)
In the analyzed extract (Level 7), a single instance of repetition is noted, indicated by the code ‘rep 1a’ The test-taker's text demonstrates a strong use of hyponymy, as evidenced by codes ‘hyp 1b’, ‘1c 2’, ‘hyp 1d’, and ‘1e’.
8 ewspaper is the nation (.) the daily nation .hhh (.) and er newspaper 1
9 (2) it has::: (.6) since it is a newspaper it has very many rep 1a
10 Articles (b)in it .hhhh an it has er news (c) updates of cours::e hyp 1b 1c2
11 an::d it has: educative areas as well sometimes? hhhhh (.5)
12 it also has fashion (d) it has also::: advertisements (e) all sorts of hyp 1d 1e
(Level 7 ID201) (Notes: rep – repetition; hyp – hyponymy)
The use of synonymy in performances exhibited an irregular pattern across different band levels, contrasting with the more consistent nature of repetition Notably, synonymy was more frequently employed at the lower band level (Level 5) compared to the higher band level (Level 7), as demonstrated in the examples provided.
99 hh (1.2) first they must be honest (e) and truthful to him (.8) and err and honest e syn e1
100 don’t err:: don’t lie with him and err say the same with real problem (.5)
101 problems and characters and help him out to err change his behaviour behaviour f
102 (.7) because they are concerned with their behaviour and err I think that 103 rep f1
103 err to help each other and to encourage each other (.5) like my friends I 104
(Level 5 ID506) (Notes: syn – synonymy; rep – repetition)
73 I’m going to describe a piece of equipment (a) that I use a lot (.7) and piece of equipment a
74 that is my car (b) (.5) first of all this thing is a vehicle (.7) which is car b hyp a syn b
75 motorised by a petrol engine and its black (1.1) it is a four seater (1.6) hyp a1/mer b1
76 and er (.7) most of all I use it to transport myself and my friends to
(Level 7 ID708) (Notes: hyp – hyponymy; mer – meronymy)
Despite the uneven distribution of synonymy, it can be argued that this lexical relationship, defined by identity, is present in both high and low levels of language proficiency.
Hyponymy and meronymy are lexical relations that enhance text coherence, often functioning together without clear distinctions (Halliday & Matthiessen, 2013) Analysis of lexical cohesion reveals that higher proficiency test-takers utilize hyponymy more frequently than their lower-level counterparts.
Higher proficiency test-takers may find the elaboration of lexical relations based on attribution less challenging, which leads to its increased use in their discourse This effective utilization of lexical resources at advanced levels enhances their overall performance and demonstrates a greater degree of competence.
Halliday and Matthiessen (2013) categorize lexical relations into two types: identity, where one lexical item restates another, and attribute, which involves classification from specific to general Our analysis indicates that lower-level test-takers tend to utilize lexical relations based on identity, such as repetition and synonymy, rather than attribute-based relations like hyponymy and meronymy This preference can be attributed to the significant cognitive load involved in language production for less proficient learners, as suggested by Levelt (1989) Consequently, simpler forms of lexical cohesion, such as repetition and synonymy, are favored due to their lower processing demands, facilitating easier communication.
Coherence analysis
Text generic structure
In the methodology section, we outlined our approach to analyzing test-taker performance by first identifying the expected text structure We then evaluated each test-taker's adherence to this structure, assigning scores between 0 and 1 based on their performance The findings are summarized in Table 13.
Mean SD Median Mean SD Median Mean SD Median
The analysis of the compliance index across three proficiency levels reveals that higher proficiency is associated with greater familiarity and adherence to the generic structure of the speaking task, although the differences between levels are not statistically significant (H = 3.46, ns) A qualitative examination of all 58 texts was conducted to assess their conformity to the identified generic structure outlined in the methodology.
73 I’m going to describe a piece of equipment that I use a lot (.7) and
74 that is my car (a) (.5) first of all this thing is a vehicle (.7) which is id a
75 motorised by a petrol engine (a) and its black (b) ( 1.1) it is a four seater (c) (1.6) de a/ de b/ de c
76 and er (.7) most of all I use it to transport myself and my friends to
77 leisure things (d) (.4) I (.) yes (1.3) I use it for everything (e) (.8) I first de d/ de e
(Level 7 ID708) (Notes: id – identification, de – description)
In the provided example, the test-taker effectively described a piece of equipment, adhering to the schematic features of descriptive discourse This type of discourse requires both an identification element and a description Initially, the test-taker identified the item, specifically their car, marked as ‘id a’ Following this identification, they proceeded to detail the car's properties and characteristics, as indicated in subsequent lines coded as ‘de a’, ‘de b’, ‘de c’, and ‘de d’.
Qualitative analysis indicated that Level 5 performances demonstrated less adherence to the descriptive discourse structure compared to Levels 6 and 7 Specifically, some Level 5 test-takers did not adequately include or develop the descriptions of the identified items, as illustrated in the following example.
147 iser my favourite news- (.) newspaper is er:::::m (.) the new
148 er thailand (a) (.6) and the parts of (.) i like to read in the:: id a
149 (.4) education hhhh (.) because (.2) in the future i would
150 like (.3) to be a 2.teacher::? (.3) and if i have a free time
151 (.) i would like to stud- (.4) dy (.4) i would like to know
152 ifer:: what the matter in the:: (.2) education (.5) yes
153 (.8) an::: (.9) there’s (opposite) people read it (.6) who
154 is a::: (.5) 3.educator or who is interest- (.6) interested in
156 prefer:: (.) in the:: (.2) newspaper maybe it’s because (1.4)
157 it’ser::: cheaper than magazine (.5) yes and it have a::
158 day- daily (.3) daily newspaper (.3) yeah (.5) and it’s er
159 have many kind of er::: (.8) newspaper (.5) it have an::::
160 ((inaudible)) (.6) to:: (.5) read (.3) in them (.)
161 newspaper (.5) yes(Level 5 ID216) (Note: id – identification)
In this example, the test-taker discussed their favorite newspaper, "New Thailand" (coded as 'id a') This response partially followed the descriptive discourse structure by including an identifying element, specifically the name of the newspaper However, the test-taker failed to provide a detailed description of the publication.
In their speech, the test-taker primarily focused on their enjoyment of the education section and their future aspirations, rather than addressing the topic of 'new Thailand.' They also briefly compared the costs of newspapers and magazines However, the lack of descriptive detail led to a partial compliance with the expected discourse structure.
In contrast to this example above, the extract below, taken from a Level 7 performance displayed a higher degree of compliance with the schematic structure of descriptive text
199 magazine called ((inaudible)) ‘Friday’ (a) hhh so (.3) (win:) this id a
200 is actually m::: (.2) it covers everything (.3) y’know it’s
201 where a according to the (emirates wife) the news has been
202 given in this magazine (.3) and (.) m: starting from the first de a
203 page all news about the emirates (a) (.4) the dubai because the
204 printing is done in dubai hhh then the the remaining pages one de b
205 by one it covers with the other (.2) emirates it is there
206 are seven emirates anyway so it is covers by (.2)
207 according to the emirates (b) hh and most of the time the
208 political issues are given in the first page (c) and er hh it de c
209 describes about any changes especially the immigrant rules (d) de d
210 whatever is changing y’know (.3) er::m according to the m:
211 fresh news what is called hot news (e) is being explained hhh and de e
212 erm:: (.4) er second thing is er one m er m: the- there are
213 some important categories which they’re they are explaining
214 About the health column is a hherm (.2) the thethe editor
215 writes about the health column (f) and er some of the hh special de f
(Level 7 ID227) (Notes: id – identification, de – description)
The test-taker was asked to describe their favourite magazine to which they responded by identifying the publication as
In the article, the test-taker effectively identifies and describes the magazine 'Friday' by outlining its key sections, which include political news, hot news, and health advice Each section is characterized by specific types of information, and the test-taker notes the corresponding page numbers where this information can be found, demonstrating a clear understanding of the magazine's structure and content.
The performance of test-takers, coded as ‘de a’, ‘de b’, ‘de c’, ‘de d’, ‘de e’, and ‘de f’, was evaluated, with a score of 1.0 indicating full compliance with the descriptive discourse generic structure The findings revealed that higher-level test-takers demonstrate a superior ability to adhere to the expected text structure compared to their lower-level counterparts It is important to note that the task card instructed test-takers to not only provide a description but also to explain various aspects of an item of personal significance, including its origin, duration of ownership, usage, and its importance Consequently, lower-level test-takers tended to emphasize the explanatory part of the task more than the descriptive element.
Theme–rheme development
The analysis of 58 test-takers' performances revealed distinct theme and rheme patterns, categorized into theme-reiteration, zigzag, and split/multiple theme patterns, as detailed in Tables 14 and 15 Results indicated that higher proficiency levels produced more mean clauses; however, the differences in mean main clauses across levels were not statistically significant (F (2, 55) = 2.391, ns) Notably, Level 7 test-takers exhibited more complex thematic patterns without theme-reiteration, while one-third of Level 5 test-takers demonstrated the theme-reiteration pattern.
Level Mean no of main clauses
Table 14: Descriptive statistics of quantity of text in each proficiency group
Combination of theme–reiteration/ constant and zigzag/ linear theme patterns
Table 15: Theme–rheme patterns in each proficiency group
The findings of the examination of theme–rheme progression reveal that test-takers across various proficiency levels effectively structured their texts by selecting a point of departure and developing it throughout their messages to ensure a coherent flow of information The thematic progression patterns identified included theme–reiteration/constant theme and zigzag/linear theme, with no significant differences in frequency across proficiency levels However, Level 5 test-takers exhibited a higher occurrence of simple theme–rheme development compared to Level 7 An example highlighted that among five main clauses, the new information introduced in the rhemes was not revisited in subsequent themes, indicating a lack of follow-up on new information Additionally, test-takers showed a greater focus on the latter part of the task instruction rather than the initial descriptive component.
Theme 1 Rheme 1 the thing I do to relax is to watching television because I will share the idea with my family and use the time together
I enjoy a variety of shows, including talk shows, quiz shows, and news programs These formats not only provide entertainment but also keep me informed about current events and important happenings.
I use about two hour everyday to watching television
I can talk about the thing that me see yesterday or every
((inaudible)) if I can I must see it I cannot talk
Figure 8: Theme–rheme development (Reiteration/constant theme pattern) (Level 5-1)
The theme-rheme development in Level 6, illustrated in Figure 9, showcases a complex combination of zigzag/linear theme patterns and theme-reiteration/constant theme patterns The rheme from the first main clause is reiterated as a theme in five subsequent clauses, enhancing cohesion and further developing the initial theme However, the test-taker redundantly used the phrase "very important" in rhemes 4, 6, and 8, opting to explain why the chosen equipment is useful rather than providing a detailed description.
Rheme 1 would like to talk about watches
‘s necessary to use this to see the time
’s a very important piece of equipment
Rheme 5 must visit and to stick to the time and to be on time and to know that they have time and to know when and where to do it
Rheme 6 is very important piece
(at the same time) it
Is important to know when to start and when to finish and every examine and some other topics
‘s a very important piece of equipment
Rheme 10 need it too much
Note: In the analysis of this extract, the element ‘I think’ in theme 4 is considered a conversation filler and thus not considered in the thematic progression analysis
Figure 9: Theme–rheme development (combination of theme-reiteration/constant theme and zigzag/linear theme patterns) (Level 6-1) !
A similar thematic progression to the Level 6 performance in Figure 9 is observed in the Level 7 performance shown in
In this example, a distinct thematic development is evident through a zigzag/linear theme pattern, where the selected item for discussion is elaborated upon in subsequent clauses to enhance cohesion, as demonstrated in themes 3, 4, and 5 Unlike the previous example, the theme 'it' in these themes builds upon the preceding rheme, showcasing a clear connection and continuity in the narrative.
(rheme 2) with additional information about the test-taker’s car shown in rhemes 3, 4 and 5
‘m going to describe a piece of equipment that I use a lot
(first of all) this thing
Rheme 3 is a vehicle which is motorised by a petrol engine
Rheme 6 use it to transport myself and my friends to leisure things
Rheme 7 first use it when I was eighteen
Rheme 8 started to use it on the first day the same day I got it
Rheme 10 have to use it
Rheme 11 said to get back and forth to do necessary things to do some unnecessary things as well
Figure 10: Theme–rheme development (combination of theme-reiteration/constant theme and zigzag/linear theme patterns) (Level 7-1)
The three examples illustrate texts with fewer main clauses, leading to simpler theme-rheme progression Table 2 indicates significant variation in the number of main clauses across different levels In the subsequent examples, thematic progression is evident in texts featuring a greater number of main clauses.
Figure 11 illustrates performance at Level 5, showcasing enhanced thematic development compared to the earlier example in Figure 8 This example features a theme-reiteration pattern with two sets of constant themes, including 'I' as themes 1, 2, 5, 6, 7, 10, 12, 13, 14, and 'it' as additional themes.
The test-taker's initial response to the examiner's question about relaxation involved mentioning a visit to the poultry house, which is highlighted in themes 3 and 4 Instead of elaborating on the visit itself, the test-taker focused on explaining the reasons why spending time in the poultry houses is a relaxing experience, as discussed in themes 3, 4, 9, and 11.
The thematic progression highlights the flow of information, yet it lacks additional details about relaxation Notably, the element in rheme 9 is reiterated in theme 11, providing further insights into the previously mentioned poultry house.
I usually do when I want to relax
I usually go to the to my poultry houses cos in there
Theme 3 Rheme 3 it diverts my attention in work cos in my work
Theme 4 Rheme 4 it is too tense for me if I go there
I ‘ll be relaxed at seeing them feeding them and seeing them grow
I just sit there an have some time for myself to think what I have done today
Theme 8 Rheme 8 that is it
Theme 9 Rheme 9 it makes me more relaxed because of the environment
The environment is too silence for
Theme 12 Rheme 12 er:: after that (0.8) when:: er:: (0.6)!
(when) I go to the basketball court and have fun with my peers yes play basketball my mind is relaxed an of course basically
I ‘m a little basically prepared conditioned and after that nothing more after my after
I have been playing basketball i do some refreshment
Figure 11: Theme–rheme development (combination of theme-reiteration/constant theme and zigzag/linear theme patterns) (Level 5-2)
In the analysis of the test-taker's narrative, their favorite trip is introduced in rheme 1 and revisited in theme 2, though the reference to 'this' lacks clarity The trip is elaborated upon through theme-reiteration and constant theme patterns in themes 3-6 and 8, ensuring a consistent flow of information centered around the theme of 'I' Additionally, 'London' is mentioned in rheme 9, further contributing to the narrative's development.
‘theme 11’ as ‘it’ and followed up again in theme 12
Theme 1 Rheme 1 one my favourite trip was to London in during Christmas vacation
This is the first time I came to United St- no sorry United
I think I should to go there should go there and have a look
I travel by train from Aberystwyth to London
(in London) I Bought a ticket for three days by which is by subway and then travelled with my friends
(I think) London is quite beautiful city the building there
Theme 7 Rheme 6 the architecture is very very old
I went to the Buckingham Palace went to the British museum and a lot of places which are quite interesting and attractive for me
I think you impressed me most is London is quite noisy I think maybe compare with
Theme 10 Rheme 10 it ‘s quite busy
I can feel the atmosphere of the industry
(although) it ‘s still very modern,
London is quite a nice place and for visiting
I think my visitings this is my favourite travel
Figure 12: Theme–rheme development (combination of theme-reiteration/constant theme and zigzag/linear theme patterns) (Level 6-2)
Figure 13 illustrates a performance at Level 7, showcasing a thematic progression structure that features two distinct theme–rheme patterns The first pattern is a theme–reiteration, also known as a constant theme, which includes two sets of constant themes, specifically using ‘it’ as themes 2.
The thematic progression in themes 7, 8, and 9 shows that the test-taker effectively adhered to the task card's instructions by providing a clear explanation for their preference for the newspaper This approach contrasts with examples from lower levels, which exhibit a zigzag or linear theme pattern.
The concept of 'the nation' introduced in rheme 1 is further explored in subsequent themes, specifically themes 3, 4, 5, 6, 7, 13, 14, and 15, creating a cohesive narrative that facilitates smooth information flow The selected topic is thoroughly developed throughout the text, contributing to its rich content Additionally, the term 'it' in theme 2 serves as a reference to 'the nation' from theme 1, while themes 3, 4, 5, and 6 are directly derived from 'it' in rheme 2, ensuring clarity and continuity in the discussion.
My favourite newspaper is the nation
(since) it is a newspaper it has very many articles in it
Theme 3 Rheme 3 it has news updates of course
Theme 4 Rheme 4 it has educative areas as well sometimes
Theme 5 Rheme 5 it also has fashion
Theme 6 Rheme 6 it has also advertisements all sorts of things
I like the news updates and the actually
I like to read that newspaper because it has the latest updates for all the topics
Theme 10 Rheme 10 the information is true because it comes from reliable sources
(if) you watch the news at night you get the news early in fact
Theme 12 Rheme 12 all sorts of people are reading it young middle age old because it has something for everyone
Theme 13 Rheme 13 it has various parts
(on Wednesdays) it would have midweek small magazine included
(in the weekend) it would have some another weekend magazine which would include you know what you should do in the weekends
Theme 16 Rheme 16 that is places to go out the latest updates on fashion and various stuff
Figure 13: Theme–rheme development (combination of theme–reiteration/constant theme and zigzag/linear theme patterns) (Level 7-1)
The analysis reveals that while identifying thematic progression patterns and counting main clauses do not directly characterize thematic development in test-taker performance across proficiency levels, a detailed examination shows significant differences linked to proficiency Notably, higher proficiency levels (6 and 7) exhibit a greater number of constant theme sets in their theme-reiteration patterns Furthermore, these advanced performances demonstrate more intricate zigzag and linear patterns The effective use of rheme elements as subsequent themes enhances discourse cohesion and enriches content, a trend evident in both shorter and longer speech segments among higher proficiency test-takers.
Lexical richness
Table 16 presents the descriptive statistics for lexical richness measures obtained through VocabProfile (Cobb 2013) As expected, both type and token measures increase with higher proficiency levels, indicating a greater variety and quantity of words used One-way ANOVA analysis reveals significant differences among the levels for these measures, as shown in Table 17 The effect size for token is marginal (!² = 143), while it is small for type (!² = 300) Post hoc analysis with Bonferroni correction identifies significant differences in token usage between Levels 5 and 7 (p = 011), and in type usage between Levels 5 and 6 (p = 005) and Levels 5 and 7 (p = 0001).
The analysis of the three ratio measures—type-token, token per type, and lexical density—reveals minimal differences among the groups, with the exception of lexical density, which shows significant variations between Levels 6 and 7 (p = 005) and Levels 5 and 7 (p = 034) Additionally, the percentages of the four word lists demonstrate significant differences only in K1 and Off List categories.
Post-hoc analysis shows that the significant differences lie between Levels 5 and 6 (p = 034) and Levels 6 and 7 (p = 008) for K1 and between Levels 6 and 7 (p = 012) for Off List
Mean SD Mean SD Mean SD
Table 16: Descriptive statistics of lexical richness measures
Squares df Mean Square F p Partial Eta
Table 17: Results of one-way ANOVA analysis (lexical richness)
Figure 14 illustrates the distribution of four lexical categories—K1, K2, AWL, and Off List—summarized in Table 16 Notably, the distribution across all three performance levels is quite similar, with the majority of words utilized in the test being classified as K1 (1000 level) In contrast, only a minimal percentage of the words fall into the K2 (above 2000) category and the Academic Word List.
Figure 14: Distribution of the four word categories
Summary of the results
This section summarizes the findings from the previous analysis, highlighting the correlation between these results and the two items of the IELTS Speaking Descriptors (Public Version) The results indicate that coherence, cohesive devices, and lexical richness differ according to assessed proficiency levels Notably, significant differences aligned with expectations were observed across these levels, although the effect sizes of these differences were generally marginal or small.
! Cohesive devices: o Comparative conjunction o Accuracy of use of referential expressions o Lexical cohesions – Hyponymy and repetitions
! Lexical richness: o Word-token o Word type o Lexical density o K1 o Off List
The standard deviations (SDs) of most features indicate significant individual variations in test-taker performances A summary of the statistical analysis findings is presented in Table 18 Additionally, qualitative analysis offers deeper insights into how test-takers utilize various cohesive and coherence devices Notably, certain features, like the compliance index of the text's generic structure, exhibit clear differences across band levels, despite the statistical analysis showing no significant differences among these levels.
On the whole a clear difference is observed between Levels 5 and 7, but the difference between the adjacent levels
(ie, Levels 5 and 6, and Levels 6 and 7) is not very clear These findings are further examined in the Discussion section
Category Sub-category Difference across the three levels Effect size Post-hoc analysis
Table 18: Summary of the results of quantitative analysis
Co-reference with the IELTS Speaking Band Descriptor
Level 7
! Speaks at length without noticeable effort or loss of coherence
! May demonstrate language-related hesitation at times, or some repetition and/or self-correction
! Uses a range of connectives and discourse markers with some flexibility
The Level 7 test-takers achieved the highest mean score in coherence, specifically in text generic structure and the number of main clauses in thematic progression, indicating their ability to produce text that aligns with examiner instructions Additionally, while additive and consequential conjunctions are more prevalent than temporal and comparative conjunctions across all levels, the distribution of these conjunctions is more balanced at Level 7 compared to Levels 5 and 6 Notably, the use of comparative conjunctions is significantly higher among Level 7 test-takers than in the lower levels.
Test-takers demonstrated an improved ability to accurately use a diverse range of cohesive devices, reflected in their lexical cohesion scores and the effective use of referential expressions The analysis of theme-rheme development indicates that their texts exhibited enhanced cohesiveness through complex thematic progression and a rich flow of information, leading to more comprehensible and content-rich writing.
Level 6
! Is willing to speak at length, though may lose coherence at times due to occasional repetition, self-correction or hesitation
! Uses a range of connectives and discourse markers but not always appropriately
Level 6 test-takers demonstrated a less accurate adherence to the generic structure compared to Level 7, yet their scores were significantly higher than those of Level 5 test-takers Additionally, their use of the four conjunctions was less balanced than that of Level 7 test-takers.
Level 6 test-takers demonstrate a notably higher frequency of comparative conjunctions compared to Level 5 test-takers, while the use of additive conjunctions is the most prevalent among all three levels.
Although the frequency of the referential expression of
Level 5
! Usually maintains flow of speech but uses repetition, self correction and/or slow speech to keep going
! May over-use certain connectives and discourse markers
! Produces simple speech fluently, but more complex communication causes fluency problems
The current study indicates that test-takers at this level can utilize connectives and discourse markers; however, about 85% of the conjunctions used were either consequential or additive, in contrast to higher-level test-takers While lexical expressions appeared frequently in their speech, the accuracy of these expressions was the lowest among the three levels Additionally, Level 5 test-takers predominantly relied on repetition devices to maintain lexical cohesion They also demonstrated the ability to adhere to the expected generic structure.
Test-takers at this level primarily concentrated on justifying their choices rather than providing descriptions of the items This focus is reflected in the thematic progression patterns noted in their performance.
This study analyzed discourse competence in 58 IELTS Speaking Part 2 test-takers, focusing on three cohesive devices and two coherence aspects, along with evaluating lexical richness The research aimed to identify distinctive features of test-taker performance across three levels by quantifying the results for statistical analysis A detailed qualitative analysis was subsequently conducted to validate the statistical findings and offer deeper insights into the characteristics of the performances.
The analysis of cohesive devices revealed that higher proficiency test-takers utilized a greater variety of these devices and demonstrated more accurate referential expressions compared to their lower proficiency counterparts Additionally, the text produced by advanced test-takers aligned closely with the expected descriptive structure and featured rich content Notably, test-takers at Levels 6 and 7 adhered more closely to the expected text structure than those at Level 5.
The findings, summarized in Table 17 of the Results section, revealed significant statistical differences among certain features However, there was considerable individual variation across these features and levels Notably, more pronounced differences were anticipated and observed.
The findings suggest that Level 7 test-takers exhibit superior control over cohesive and coherence devices, leading to more comprehensive and content-rich texts However, the results are not entirely straightforward, as significant individual differences were noted, with marked disparities observed in only a few aspects of discourse competence Analyzing the transcribed data further highlighted variations in the length of speech, measured by word-token, across all three levels.
On the whole, Level 7 test-takers spoke longer than Level
The descriptive statistics presented in Table 16 reveal that the word-token count ranges from 5 to 6, indicating a notable variation in lexical richness across all levels For detailed results, refer to Appendix 2.
The analysis revealed that the word-token count was relatively consistent across different proficiency levels, likely influenced by the 1-2 minute speech requirement and the varying strategies employed by test-takers While some individuals opted for concise descriptions, others chose to elaborate extensively on their selected topics.
The study did not assess the speech rate of test-takers, leaving it unclear whether differences in speech speed exist across various levels However, previous research, such as that by Brown et al (2005) and Iwashita and Brown, suggests potential variations in speech rate among different proficiency levels.
Research by McNamara & O’Hagan (2008) indicates that lower level test-takers exhibit frequent pauses in their speech In contrast, higher level test-takers tend to speak more quickly and fluently, leading to a greater volume of spoken language.
This shows Level 7 test-takers had more opportunities to use a variety of cohesive and coherence devices to make their speech structured and comprehensive
The minimal difference in cohesion and coherence features may be linked to the low frequency of devices utilized by test-takers This observation aligns with findings from earlier studies on discourse competence, such as those by Brown et al (2005) and Banerjee et al (2004) Descriptive statistics reveal the frequency of conjunctions, referential expressions, and lexical cohesions, indicating a need for improved usage among test-takers.
According to Table 11, the frequency of using cohesion devices varied, with conjunctions ranging from 3.75 to 4.15 across four types, lexical expressions between 2.14 and 2.60, and lexical cohesion devices falling between 1.70 and 1.94 across five types.
The current study found that ellipsis and substitution devices were infrequently used by test-takers, leading to their exclusion from the analysis This outcome aligns with expectations given the brevity of the speech involved.
Qualitative analysis reveals distinct differences across various levels in features such as conjunction types, lexical cohesion devices, text structure, and thematic progression.
The few instances of cohesion and coherence devices observed in the current study might also be explained in terms of test-takers’ awareness of these devices
In Speaking Part 2, test-takers are allotted two minutes to prepare for their talk following the examiner's instructions Research in pedagogic contexts indicates that participants primarily focus on strategies, vocabulary, and forms during their planning time (Ortega 1999; Sangarun 2005; Tajima 2003) However, no studies have addressed the consideration of coherence and cohesive devices in this context Therefore, it is likely that the test-takers in the current study also prioritized these elements, as suggested by previous planning studies.