Concerning research question 1, the results of pre- and post-tests were analyzed with the help of SPSS 23 (the statistical package for Social Science 23). The procedure and results of data analysis occurred after scoring the pretests and posttests of two groups in the following steps:
First, the researcher investigated whether the students' oral communication quality in the experimental group and control group was reliable or not. Thereby, Pearson correlation analysis using SPSS was executed for both groups to check the inter-rater reliability of pretests and posttests in Tables 4.1 and 4.2, respectively.
Table 4.1 Correlation Coefficient _Inter- Rater reliability: pretests
The correlation coefficient obtained for two raters in the pretest in EG and CG is.92 and.798, respectively, as shown in Table 4.1, indicating high inter-rater reliability.
Also, it was found that the correlation coefficient for the first and second raters was statistically significant with a p-value smaller than.05, which means that the test scores assigned to the students by the two raters are reliable.
In the same vein, the inter-rater reliability of the experimental and control groups' performance on the posttests was also calculated by means of Pearson correlation, as illustrated in Table 4.2 below:
68
Table 4.2 Correlation Coefficient _Inter- Rater reliability: posttests
The statistical analysis results are shown in Table 4.2, and they show that there is a strong and significant correlation between two raters' scores on the experimental group's posttest (r =.95, p.05).
Also, the inter-rater reliability of the control group's performance on the posttest revealed a significant relationship between the scores of the posttest obtained by two raters in the control group (r = 0.876, p.05).
The purpose of using Pearson-product moment correlation was to check the inter-rater reliability of pretest scores, which were independently reported by two raters. The p- value from the two tables is less than 0.05; hence, the correlation is statistically significant.
Second, as mentioned, in order to investigate whether the two groups were at the same level of ability prior to and subsequent to the treatment between the two groups of students, an oral test was administered to both the experimental group and the control group. The mean scores of 22 students in the experimental group and 38 students in the control group are compared using the Independent Samples Test. The descriptive and independent sample t-test statistics are presented in Table 4.3.
Table 4.3 The results of Independent Samples T-test of the Pre-tests
In table 4.3, the output indicates that the mean score of the experimental group is 6.49 (M = 6.49), and that of the control group is 6.55 (M = 6.42). From this data, the mean score of the experimental group is slightly higher than that of the control group. The amount of difference in mean scores (0.07) is enough to say that the scores come from the same population. This means that the students' English oral communication performance in the two groups is similar. Moreover, the result of the Independent Samples T-test shows that the significance of the test F in the Levene test is 0.733 (F
=.733), which is greater than 0.05, and the p-values (p =.699) are both greater than the significance level (.05). There are no significant differences between the two groups. In other words, students’ English oral communication performance in the two
70
groups is similar. Therefore, the samples from the two groups are proportional and meet the requirements for this study.
Third, after proving that the level before applying metacognitive strategy training to English oral communication of the two groups is the same, only EG was taught and reviewed the training through performing five components (content, fluency, vocabulary and grammar, pronunciation; and interaction) in oral communication.
Then, the students in both groups were asked to do post-tests. The researcher also used the independent samples t-test to compare two groups and detect whether there was any significant difference at these different points of time. The descriptive statistics of the two groups' posttests are presented in Table 4.4.
Table 4.4 The results of Independent Samples T-test of the Post-Tests
As presented in Table 4.4, the homogeneity of variances was to be accepted since the score variance between the two groups was equal based on Levene’s test results (p
=.590 >.05). Accordingly, the p values between the 2 groups are.012 and.019, which are smaller than p =.05. Hence, there was a significant difference in English oral communication scores for the CG (M = 6.7368, SD =.69103) and EG (M = 7.2636, SD =.85665). It was proven that the EG outperformed the CG in the post-test.
Fourth, a pair-sample t-test was used to compare (1) students’ oral communication scores of CG and EG after the treatment, students’ oral communication scores of the experimental group after the treatment, and (2) the scores of the five components of the assessment known as content, grammar and vocabulary, pronunciation; fluency;
and interaction of the experimental group after the treatment.
The summary of the SPSS output of the paired sample T-tests in oral communication performance tests of both the CG and EG is given in Table 4.5.
Table 4.5 The results of Paired Samples T-test of two groups EG-CG
The comparison of students’ oral communication performance on the pretest and the posttest in the control and experimental groups is revealed in the table below:
It can be seen from the data in the Paired Samples Statistics of Table 4.5 that the students in CG have shown a little improvement in the posttest. The mean score rises from 6.4211 to 6.7368, and the mean difference between the two tests is -.31579.
whereas the paired sample statistics output in EG indicates that the mean for the pretest is 6.49 (M = 6.49) and for the posttest it is 7.26 (M = 7.26). The average difference in mean between the paired pretest and posttest scores of EG (difference = -0.77) is statistically significant. If the p-value is less than your significance level, the difference does not equal zero. In fact, turning to the paired sample T-test of CG and
72
EG, the p-value for the paired sample T-test of 0.000 alpha (0.05) is less than the standard significance level of 0.05.
The next section of the comparison of the students’ oral communication performance on the pretest and the posttest in the experimental groups (2) is also concerned with the same running of the paired samples T-test. Here is the description.
The two steps above indicated that after this course, students in both the control group without the treatment and the experimental group with the treatment all improved their oral communication performance in English. That is the reason why the research needed to take the last step to answer the first research question—making a comparison between the post-test scores of both the control group and experimental group.
Admittedly, metacognitive strategy training has enhanced students’ oral communication performance, and the researcher would like to know which component criterion improves in the training. As listed in the oral communication assessment, there are five criteria for measuring the students’ English oral communication performance: content; vocabulary and grammar; pronunciation;
fluency; and interaction. For more details about how different students’ English oral communication performance is, a detailed statistical analysis still using the pair sample t-test will be run to measure those components in Table 4.6.
Table 4.6 Comparing Components Scores in Students’ English Oral Communication performance of Experimental Group after the treatment
Paired Samples Statistics
Mean N Std. Deviation
Std. Error Mean
Pair Content _Pretest 1.5182 22 .10970 .02339
1 Content _Posttest 1.5523 22 .14432 .03077
Pair Fluency_Pretest 1.4364 22 .23714 .05056
2 Fluency_Posttest 1.4773 22 .23540 .05019
Pair VocabGram_Pretest 1.4295 22 .18038 .03846
3 VocabGram_Posttest 1.4591 22 .14027 .02991
Pair Pronunciation_Pretest 1.4341 22 .22698 .04839
4 Pronunciation_Posttest 1.4864 22 .20247 .04317
Pair Interaction_Pretest .6545 22 .20407 .04351
5 Interaction _Posttest 1.2614 22 .34363 .07326
Paired Differences
Std.
95% Confidence Interval of the
Pair 1 Content_Pretest - Content _Posttest
Pair 2 Fluency_Pretest - Fluency_
Posttest
Pair 3 VocabGram_ Pretest VocabGram_ Posttest
-
Pair 4 Pronunciation_ Pretest Pronunciation_ Posttest
-
Pair 5 Interaction _Pretest Interaction _Posttest
-
The summary statistics table 4.6 contains five pairs and five rows for each of the two variables to be tested. After the pair column, which contains five variables in terms of content (fluency, vocabulary, and grammar), pronunciation, and interaction. It can be seen that the means of the content, fluency, vocabulary, grammar, pronunciation, and
Mean
Std.
Deviation
Error Mean
Difference
Lower Upper t df
Sig. (2- tailed) -.03409 .13308 .02837 -.09310 .02491 -1.202 21 .243
-.04091 .12016 .02562 -.09418 .01236 -1.597 21 .125
-.02955 .11818 .02520 -.08194 .02285 -1.173 21 .254
-.05227 .15077 .03215 -.11912 .01458 -1.626 21 .119
-.60682 .37458 .07986 -.77290 -.44074 -7.599 21 .000
74
interaction pretest variables are 1.5182, 1.4364, 1.4295, 1.4341, and 1.6545, respectively, while the means of the posttest variables are correspondingly 1.5523, 1.4773, 1.4591, 1.4864, and 1.2614. Hence, all of the posttest scores are a little bit higher than the pretest. That is, the differences between these variables are statistically significant. -0.04; -0.04; -0.03; -0.05; and -0.61. Among these components, the mean of the interaction variable gets the highest score. The results show that the p-values for the paired sample T-test of content, fluency, vocabulary, grammar, and pronunciation are.243;.125;.254; and.119, which are higher than 0.05 (p >.05), except for interaction, its p-value is.000 (p<.05).