THE FORUM The TESOL Quarterly invites commentary on current trends or practices in the TESOL profession It also welcomes responses to rebuttals to any articles or remarks published here in The Forum or elsewhere in the Quarterly A Reader Responds to Andreou, Andreou, & Vlachos’s ‘‘Learning Styles and Performance in Second Language Tasks’’ Learning Styles and Performance in Second Language Tasks: Instrumentation Matters PHILLIP DAVID JONES Hong Kong Institute of Education Hong Kong SAR, China & In the December 2008 issue of TESOL Quarterly Andreou, Andreou and Vlachos reported on a study that examined the learning styles of 452 undergraduate students and their performance in second language tasks First, I would like to say how delighted I was to read an article in TESOL Quarterly that deals with this underexplored yet important aspect of TESOL Further, I would like to publicly state my agreement with some of the issues raised in the report This agreement can be briefly summarized as a call for more research concerning the effect of learners’ learning styles on second language acquisition (SLA) and their performance in second language tasks; the stability of these styles over time; and the formulation of intervention strategies, curricula, materials, and activities based on these styles However, although, I am convinced that these areas of further research are indeed important for exploration, and that, if undertaken, such exploration will result in a better understanding of how individuals learn language and how this understanding can be exploited through course design and pedagogy, I believe we must tread carefully to ensure that the instrumentation used produces valid and reliable results My concern with the work of Andreou et al is that they have not fully considered which instrument should be used for establishing the learning styles of the subjects in their study This consideration is 722 TESOL QUARTERLY Vol 43, No 4, December 2009 Tesol Quarterly tesol208126.3d 31/12/09 19:00:45 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) important because if the instrument used to establish the learning style of the participants can be criticized for its reliability in predicting learning styles, then the results of the study overall are compromised CRITICISM OF KOLB’S LEARNING STYLE INVENTORY (LSI) There are issues surrounding the reliability and validity of Kolb’s LSI, although Andreou et al not consider them in their report Newstead (1992), DeCiantis and Kirton (1996), and Wierstra and DeJong (2002) have criticized its validity, but of even more concern in this instance is its test–retest reliability (see Cornwell, Manfredo, & Dunlap, 1991; Freedman & Stumpf, 1978; Lam, 1997; Newstead, 1992; Stumpf & Freedman, 1981; Veres, Sims, & Shake, 1987; Wilson, 1986) Although the subjects of Andreou et al.’s study may have exhibited a preference for one learning style over another at the time of testing, it is possible that if retested a little later, say, within a few weeks, they may have switched to another or even opposite style Unfortunately, this lack of consideration for the reliability and validity of Kolb’s LSI raises questions about the reliability of the Andreou et al.’s results and any inferences drawn from them Another worrying aspect of Andreou et al.’s work is which version of the LSI was used Three different versions of the LSI have been issued, in 1976, 1985, and 1999 Each version has seen improvement in regard to reliability; however, as yet, the LSI is still not at a stage of development where, empirically speaking, it has been proved to be suitable for research From reading Andreou et al.’s report, I am concerned that perhaps the 1985 version was used, which would be another reason to question the results and conclusions of the report For readers ease of use, I have included the reference for Kolb (1999) in the reference section of this article OTHER POSSIBLE INSTRUMENTS Coffield, Moseley, Hall, and Ecclestone (2004a, 2004b) provide valuable meta-analyses of 13 well-known learning style instruments according to four criteria: internal consistency, test–retest reliability, construct validity, and predictive validity The only instrument that Coffield et al showed to have acceptable reliability and validity for research is Allison and Hayes’s (1996) Cognitive Style Index (CSI) The CSI is a psychometric test originally designed to be used with professionals It has been used with people from many different countries, such as Nepal, Jordan, Russia, India, Singapore, Hong Kong, France, and Germany The test comprises 38 questions and can be completed relatively quickly by students at the start or end of a class Low scores indicate a strong preference for an intuitive cognitive style, THE FORUM 723 Tesol Quarterly tesol208126.3d 31/12/09 19:00:45 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) whereby attention is focused on the big picture and thinking is unstructured, whereas high scores indicate a strong preference for an analytical cognitive style whereby thinking occurs logically, step by step The CSI has very sound results for test–retest reliability as confirmed by Murphy, Kelleher, Doucette, and Young (1998) Additionally, there is evidence of high internal consistency as confirmed by Murphy et al and Sadler-Smith, Spicer, and Tsang (2000) The CSI, therefore, offers good reliability, although it should be noted that according to Hodgkinson and Sadler-Smith (2003), it seems that the CSI should be considered as measuring two moderately correlated factors, analysis and intuition, and on this basis makes a very sound instrument for research The CSI, therefore, offers an opportunity to TESOL researchers to explore the relationship between cognitive style and performance in second language tasks and adjust pedagogical approaches accordingly It is interesting to note that the CSI could have replaced the LSI in the study currently under discussion because a key objective was to explore the effect of learning styles on performance in second language tasks RESITUATING FURTHER RESEARCH At the start of this response, I agreed that further research in this area is both necessary and important In particular, I put forward domains in which I see valuable research opportunities However, given this response and the unreliable nature of most instruments in this area, I suggest that the CSI (valid and reliable translated versions, if necessary) be used for the purpose of this research and that further research is resituated in the following way More research is required concerning the effect of learners’ cognitive styles (as determined by the CSI) on second language acquisition and their performance in second language tasks; the stability of these styles over time; and the formulation of intervention strategies, curricula, materials, and activities based on these styles REFERENCES Allinson, C W., & Hayes, J (1996) The cognitive style index Journal of Management Studies, 33, 119–135 Coffield, F J., Moseley, D V., & Hall, E., Ecclestone, K (2004a) Learning styles and pedagogy in post-16 learning: A systematic and critical review London: Learning and Skills Research Centre, Learning and Skills Development, Department for Education and Skills Retrieved December 1, 2009, from http://www.pedagogy ir/images/pdf/learning-styles-pedagogy.pdf Coffield, F J., D.V Moseley, D V., Hall, E & Ecclestone, K (2004b) Should we be using learning styles? What research has to say to practice London: Learning and Skills Research Centre, Learning and Skills Development, Department for Education 724 TESOL QUARTERLY Tesol Quarterly tesol208126.3d 31/12/09 19:00:46 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) and Skills Retrieved December 1, 2009, from http://www.ttrb.ac.uk/attachments/c455e462-95c4-4b0d-8308-bbc5ed1053a7.pdf Cornwell, J M., Manfredo, P A., & Dunlap, W P (1991) Factor analysis of the 1985 revision of Kolb’s learning style inventory Educational and Psychological Measurement, 51, 455–462 DeCiantis, S M., & Kirton, M J (1996) A psychometric re-examination of Kolb’s experiential learning cycle construct: A separation of level, style and process Educational and Psychological Measurement, 56, 809–820 Freedman, R D., & Stumpf, S A (1978) What can one learn from the learning style inventory? Academy of Management Journal, 21, 275–282 Hodgkinson, G P., & Sadler-Smith, E (2003) Complex or unitary? A critique and empirical re-assessment of the Allinson-Hayes Cognitive Style Index Journal of Occupational and Organisational Psychology, 76, 243–268 Kolb, D A (1999) The Kolb learning style inventory, version Boston: Hay Group Lam, S S K (1997) Reliability and classification stability of learning style inventory in Hong Kong Perceptual and Motor Skills, 85, 141–142 Murphy, H J., Kelleher, W E., Doucette, P A., & Young, J D (1998) Test–retest reliability and construct validity of the cognitive style index for business undergraduates Psychological Reports, 82, 595–600 Newstead, S E (1992) A study of two ‘‘quick-and-easy’’ methods of assessing individual differences in student learning British Journal of Educational Psychology, 62, 299–312 Sadler-Smith, E., Spicer, D P., & Tsang, F (2000).Validity of the cognitive style index: Replication and extension British Journal of Management, 11, 175–181 Stumpf, S A., & Freedman, R D (1981) The learning style inventory: Still less than meets the eye Academy of Management Review, 6, 297–299 Veres, J G., & Sims, R R., & Shake, L G (1987) The reliability and classification stability of the learning style inventory in corporate settings Educational and Psychological Measurement, 47, 1127–1133 Wierstra, R F A., & DeJong, J A (2002, June) A scaling theoretical evaluation of Kolb’s learning style inventory–2 In M Valcke & D Gombeir (Eds.), Learning styles: Reliability and validity Proceedings of the 7th Annual European Learning Styles Information Network Conference (pp 431–440) Ghent, Belgium: University of Ghent Wilson, D K (1986) An investigation of the properties of Kolb’s learning style inventory Leadership and Organisation Development Journal, 7(3), 3–15 The Authors Reply ELENI ANDREOU, GEORGIA ANDREOU AND FILIPPOS VLACHOS University of Thessaly Volos, Greece & It is well known that Kolb’s work has been criticized for logical inconsistencies in theory construction but mainly for the psychometric properties of the LSI-1985 While relevant research has generally supported the internal reliability of the revised LSI-1985 (although with concerns about the stability of test–retest reliability scores), it has demonstrated inconclusive results in terms of its construct validity (e.g., THE FORUM 725 Tesol Quarterly tesol208126.3d 31/12/09 19:00:46 The Charlesworth Group, Wakefield +44(0)1924 369598 - Rev 7.51n/W (Jan 20 2003) ... offers an opportunity to TESOL researchers to explore the relationship between cognitive style and performance in second language tasks and adjust pedagogical approaches accordingly It is interesting... Moseley, D V., & Hall, E., Ecclestone, K (200 4a) Learning styles and pedagogy in post-16 learning: A systematic and critical review London: Learning and Skills Research Centre, Learning and Skills Development,... using learning styles? What research has to say to practice London: Learning and Skills Research Centre, Learning and Skills Development, Department for Education 724 TESOL QUARTERLY Tesol Quarterly