Duong Thi Hong An vd Dtg Tgp chi KHOA HQC & CONG NGHE 175(15) 103 10 A N E V A L U A T I O N O N T H E E F F E C T I V E N E S S O F T H E E N G L I S H S O F T W A R E S Y S T E M F O R N O N E N G L[.]
Duong Thi Hong An vd Dtg Tgp chi KHOA HQC & CONG NGHE 175(15): 103-10 AN EVALUATION ON THE EFFECTIVENESS OF THE ENGLISH SOFTWARE SYSTEM FOR NON-ENGLISH M A J O R STUDENTS AT THAI NGUYEN UNIVERSITY OF INFORMATION AND COMMUNICATION TECHNOLOGY Duong Thi Hong An", Dang Phuong Mai TNU University of Information and Communicalion Technology SUMMARY CALL (Computer Assisted Language Leaming) has proven its high effectiveness in the frend of information and communication technology (ICT) applications in education and fraining in general and teachmg English in particular The project "Building the software system for supporting the learning, teaching, reviewing and self-assessing English" funded by National Foreign Language Project 2020 and Thai Nguyen University (TNU) was piloted at Universfty of Information and Communication Technology (ICTU) fi-om August 2015 to May 2016 The participants were 400 first-year undergraduates and 15 EFL lecturers at ICTU Positive feedbacks via survey questionnaires fi-om the lecturers and the students were sent to the project group Better study results made by the fu-st group of 200 students usmg the courseware compared with the ones made by the second group of 200 students who did not use the courseware were also the proof of its effectiveness This bright perspective opened a broader implementation in the next stage ofthe project Keywords: CALL; courseware; multimedia; evaluation, effectiveness INTRODUCTION The use of TCT is one of the factors that changed the Vietnamese students' practice of learning English as a foreign language Such views are m synchrony with the official directive from the Mmistry of Education and Training, which urges EFL lecturers to integrate multimedia and digital technology into theu teaching At TNU, the project "Buildmg the software system for supporting the learning, teaching, reviewing and selfassessing English" has been cairied out by TCTU project group since 2015 The project aims at supporting English teachmg and learning at T N U and enhancing CALL abilities of both lecturers and students In order to examine the quality of this English language teaching (ELT) multimedia courseware, it is necessary to conduct quality evaluations on it Thus, we examined and evaluated this multimedia courseware piloted at ICTU, TNU THEORETICAL REVTEW Computer assisted language learning (CALL) CALL has been defined as "the search for and study of applications of the computer in teaching and learning' [5] Corresponding to the evolution of computer technology in the past decades, the development of CALL can be categorized into three stages: behaviouristic CALL, communicative CALL, and integrative CALL [10] The first stage of CALL has been termed behaviouristic or sfructural CALL, which was characterized by route learning in the forms of pattern and drill exercises and explicit instructions of grammar Communicative CALL emerged in late 1970s and sfressed that students should be encouraged to produce language in authentic communicative settings instead of manipulating prefabricated forms The mid 1990s witnessed the emergence of the integrative CALL which sought to integrate both basic language skills and computer technology more naturally and fully into language learning Nowadays, the digitalization of language materials through information technology provides multiple chatmels for students to obtain information, which unobtrusively breaks the fraditional view of input of language learning Not only can multimedia courseware redirect learner's attentional resources, but also it has the Tel: 0904267 273, Email: dthan@lctu.edu.vi 103 Duong Thj Hong An va Dig Tap chi KHOA HOC & CONG NGHE potential to enhance the salience of language features Highlighting, colour-coding, underlining, and font manipulation are typical examples of visual input enhancement [6], CALL was introduced into Vietnam in early 2000s and the research on English instructional courseware in Vietnam lagged behind that of their foreign peers who started lo investigate courseware' in 1990s, Furthermore, in the past decade, English multimedia courseware has been widely used in English teaching al tertiary level Therefore, there is a need to conduct an evaluation on the quality of courseware, which could not only guide courseware design but also facilitate the development of future courseware evaluation Courseware evaluation Systematic evaluation of CALL courseware generally fell behind the development efforts [2] Courseware evaluation refers to the process of measuring the appropriateness and effectiveness of a particular instructional system [1], [3], Hubbard sfressed that judgments made by instructors and learners should be at the core of CALL software evaluation so that teachers' and learners' considerations could be combined to yield assessments of the appropriateness of a piece of CALL courseware in a given context Several research has been done to evaluate multimedia learning materials from both teachers' and students' perspective [4], [9], METHODOLOGY Contents ofthe courseware Theframeofthe courseware includes five main parts: Lecture Notes - Language Reference — Practice - Mini-test - Trial Exam The course books integrated in the system are English Unlimited with levels from A2 to Bl [7], [8], Currently, three TNU university members are using them as the main course books These books aim at developing the language knowledge and skills as in Common 104 175(15): 103-108 European Framework Reference (CEFR) from Al to CI The four skills associated with Vocabulary, Grammar, Pronunciation, Cultures are integi'ated in each book The English curriculum at TNU has from 10 to 12 credits, which is equivalent lo 200 lecturing hours With the average input level of English is Al while the English output standard is A2 and Bl This means one level raise of students' language competence Based on previous studies, in order to reach this goal, 300 to 600 class hours and selfstudy should be met However, due to time constraints, it is hard to exploit the couise books Therefore, the integration some parts of these course books were integrated into the courseware so that students can study on their own, i.e., grammatical practice, reading, listening, writing, pronunciation and minitests Both TNU lecturers and students can exploit the learning and teaching recourses The Grammar Reference is designed with theme-based units in which learners can he supported online anytime when they want to check grammar theories They are in both English and Vietnamese so that student can understand easily The Lecture Notes are lesson-based are designed for teachers and students to use in class or at home since they are available online Teachers can actively share or upload their teaching materials with other teachers or students under the confrol of an professional administrator The Practices are collected and processed lo avoid copyright violation They follow the lecture contents supporting learners to review what thay have learnt in class for each unit Teachers also can assign further practice for students in the online system to assess their self-studying and participation Some audio and video are recorded by English native speakers who are our foreign students al ICTU The vividly-designed exercises can motivate learners to study online The Mini-tests are designed along with the contents of each unit so learners can review and assess their progress themselves The Trial Exam level from A2 to B l is collected, processed and mixed to created different trial test for learners to practice and self-assess before they 175(15): 103-108 Tgp chi KHOA HOC & CONG NGHE Duong Thi Hong An va Dtg take real English output exams at TCTU (5) The system assists users with activities and with adjusting or information to the system creative adding (6) The courseware is suitable for psychological of learners without overloaded knowledge and stress-free interface (colour, design, audio ) (7) The system helps users to solve problems emerging while using and give timely feedback Participants The first group of participants were 200 nonEnglish major undergraduate E F L students and 15 E F L teachers at ICTU, TNU The questionnaire survey was administered in a laboratory where the instructors taught the students English skills using the courseware The teacher questionnaire survey was carried out in a meeting room after the teachers delivered their lessons (8) The system can archive the studying results and duration for learners and also for the purpose of evaluation or research The evaluation based on eight criteria with a five-point Likert-type scale: (1) Totally (2) Agree, Totally (3) Not sure, agree, (4) Disagree, (5) disagree Survey results Table Questionnaire survey results The second group of participants were 200 non-English major undergraduate EFL students who did not use the courseware Both are at the same level of English language competence ( A l ) based on their placement test results After 10 months, both groups took the same final exams The results o f t h e final semester exam results were compared Instruments Criteria Totally ^ ^ ^ agree " 133 65 146 54 125 72 117 75 131 65 123 69 112 70 134 75 196 91,2 Not sure 12 14 IS 9.75 4,5 Dis- Totally e disagree S 7 9,25 4,3 The first group of participants were given the Grit i Cnt2 Grit Grit CritS Grit Cnt CritS Average Percentage survey From the table above, we can questionnaires to evaluate the courseware after 10-month-use majority Eight courseware evaluation criteria include: ag'ee (1) The contents provided in the courscMwe are appropriate with the standardized curriculum evaluation (2) Users can use the system al the stages: building knowledge, improve skills, knowledge application, (3)^ The courseware favourable creates to be more active and and testing the better teaching and learning titan other traditional different reinforce and environment ones, which assists them creative (4) The courseware ensures the learning objectives evaluated by the teachers for the targeted students see that of the participants totally agree/ (91,2%) criteria whh eight while the courseware number of participants who are not sure, disagree or totally disagree was minor (4,3-4,5%), Compared final exam results Table Comparedfinal exam results of two groups F G D A B Group ( Use tht• courseware) 22% 12% Semester 15% 48% 25% 11% Semester 17% 45% 3% 2% Group (Do not use the courseware) 49% 19% Semester 5% 20% 45% 22% Semester 7% 21% 7% 5% Grade Duong Thi Hong An vd Btg Tap chi KHOA HQC & CONG NGH$ 175(15): 103-lOl through their practice, review and revise for From the study results collected fi-om exam With the parts that need teacher's Department of Academic Affairs of TCTU, we evaluation and marking, students can send can easily see that the percentages of student their works directly to their teachers and get getting grade A, D, F from both group are feedback via SMS and email system Students almost the same The remarkable difference are managed in class according to their levels can be shown in the percentage of students of English profieney and are assigned getting grade B and C Tn the first group, homework individually During theu onlme nearly half can get grade B while in the study, students can send MSM, email or they second one, only 20% of them can reach B, can share their learning experiences via our Courseware analysis forum or Facebook, Students can make their Strengths own study plans and save their study results, More importantly, first group of participants which make teachers easier to evaluate their assessed the strengths and weaknesses of the students and viee versa This software system online system and gave some useful is connected to Intelligent University (IU) so suggestions for the project group The teachers can actively manage their own strength of the system can be noted as classes automatically Thus, students are more follows Firstly, the contents are evaluated to active in leaming English online because both be appropriate with the cirieulum and participation in class and online are assessed asociated sfrictly with the course book by teachers Thirdly, the intergrated system English Unlimited Every lesson has creates more favourable conditions for not contents: Lecture Notes Grammar only teachers but also students to teach and Reference — Practice — Minitest Practice part learn English online Teachers can design is really useful for learners with vivid design further practice or tests for their class under and connected with the grammar focus and the control of professional admin They can vocabulary It helps to develop students's also exploit the availble teaching resourses in listening, reading, writing and pronunciation the software system Moreover, e-Lessons skills, Minitests is applied after learners finish with vivid and harmonious design make each unit so they can have quick reflection students feel more comfortable and sfress-free from what they have learnt From this, they for students to study continuously in or can adjust their learning method suitably A2hours a day Bl trial exam bank in the software system is Weaknesses designed on Cambridge ESOL format, which Courseware users pointed out several assists students to get accustomed to test weaknesses of the courseware Firstly, it does sample with different skills and levels They not have fimctions for developing speaking can self-assess their own progress and skill or voice recognition for learners lo actively practice for the output exam by TNU improve their pronunciation Secondly, it when they are in their first year Secondly, does not have interactive activities or games learners' studying results are achieved during for widening learners' vocabulary Thirdly, the learning process so lecturers can easily the evaluation and marking by teachers not supervise their student learning progress implemented immediately In fact, with the through Course Management, Class large amount of students' writing works sent Management They can also know the selfby students, teachers have difficulties in studying hours by students via Online solving them at onee, normally within 24-48 learning duration This can count hours Last but not least, the number of automatically students self-learning duration 106 Duong Thj Hong An vd Dtg Tgp chi KHOA HOC & CONG NGHE questions for revising for A2-B1 exam is not adequate Within the scope of this project, we provide 1000 questions for A2 exam and 1000 questions for Bl exam CONCLUSION After 10 month-piloting, the courseware proved to be partially successfiil in supporting ICTU lecturers and students in their teaching, learning, reviewing and self-assessing English Learners could be more active and independent in their self-study while teachers could save much time in theu class management Moreover, the shortcomings of the courseware were adjusted to cater for the practical needs from both lecturers and students The next stage ofthe project is using the courseware for the whole undergraduates at ICTU then if any TNU university members find it useful, TCTU is willing to share ACKNOWLEDGEMENTS This paper Is the product ofthe project, code T2017-07-21, funded by TNU-University of Information Technology {Bdi bdo la sdn phdm cita di tdi cd md sd T2017-07-21, duac ldi trg b&i kinh phi cua tru&ng Dgi hgc Cdng nghi thong tin & Truyin thong, Dgi hgc Thdi Nguyen) 175(15): 103-108 REFERENCES Colpaert, J (2006), "Pedagogy-driven design for online language teaching and leaming", CALICO Journal 23(3), pp 477-497 Flagg, B (1990), Formative evaluation for educational technologies, Lawrence Eribaum Associates, New Jersey Hubbard, P.(2006), "Calling in CALL: From theory and research to new directions in foreign language teaching", CALICO 20(3), pp 313-338 Jamieson, L, & Chapelle, C (2010), "Evaluating CALL use across multiple contexts" System, 38(3), pp 357-369 Levy, M (1997), Computer-assisted language learning: Context and conceptualization, Oxford University Press Sharwood Smith, M, (1991), "Speaking to many minds: On the relevance of different types of language information for the L2 leamer" Second Language Research 7(2), pp 118-132, Tibury, A, Clementson, T, Hendra LA and Rea.D (201 \) English Unlimited Elementary A2 Course book with e-Portfolio, Cambridge University Press, London Tibury, A, Clementson, T Hendra, LA and Rea,D (2011), English Unlimited Pre-inlermediale Bl Course book with e-Portfolio: Cambridge University Press, London Van Doremalen,!., Boves, L., Colpaert, J,, Cucchiarini, C , & Strik, H, (2016), "Evaluating automatic speech recognition-based language leammg systems: A case study", Computer Assisted Language Learning, 29, pp, 833-851 10 Warschauer, M„ & Healey, D (1998), "Computers and language leaming: An overview" Language Teaching, 3(\), pp 57-71 Duong Thj Hong An vd Dig Tap chi KHOA HOC & CONG NGHE 175( 15): 103 - 108 TOM T A T D A N H GIA HIEU QUA SU" D U N G H £ T H O N G P H A N M E M T I E N G A N H CHO SINH VIEN K H O N G C H U Y E N TIENG ANH T A I TRlTOfNG D A I HOC CONG N G H E T H O N G TIN VA T R U Y E N T H O N G - D A i H O C THAI NGUYEN Duong Thj Hong An', Dang Phiro-ng Mai Trudng Dgi hoc Cong nghe thong tin vd Ti uyen thong - DH Thdi Nguyen Viec hoc ngoai ngii co su ho tro ciia may tinh da chung to hieu qua xu the ling dyng CNTT&TT giao due va dao tao noi chung va giang day tieng Anh noi rieng Du an '•'Xdy dung phdn mim hoc liiu ho tra hoc lap, gidng dgy, on luyen vd iu ddnh gid mdn tiing Anh " tai trgr boi De an Ngoai ngiJ quoc gia 2020 va Dai hoc Thai Nguyen, duoc ap dung thi diem tgi Dai hgc Cong nghe thong tin & Truyin thong tu thang nam 2015 den thang nam 2016 Doi tuong nghien cuu la 400 smh vien nam thii nhdt va 15 giang vien tieng Anh cua Trucmg Sau giai dogn thi diem, nhom du an nhgn duoc cac phan h6i tich cue tii phia giang vien va sinh vien thmg qua phieu khao sat Ket qua hgc tap cao hon o nhom 200 sinh vien co ket hpp sir dung phan mem so voi nhom 200 sinh vien khong sir dung ph4n mem chiing to hiSu qua ciia hg thong Dilu mb trien vong tot cho he thong hgc lieu duoc trien khai tren phgm vi rgng hon d giai doan tiep theo ciia duan Til' khoa: Hoc ngoai ngd cd ho trg cua mdy linh, phdn mem hgc lap, da phuang tiin; ddnh gid; hieu qud Ngdy nhgn bdi: 10/9/2017; Ngdy phdn biin: 09/10/2017; Ngdy duyit ddng: 29/12/2017 Tel: 0904 267 273 EmaiT dlhan@iclu.edu 108 ... N G H £ T H O N G P H A N M E M T I E N G A N H CHO SINH VIEN K H O N G C H U Y E N TIENG ANH T A I TRlTOfNG D A I HOC CONG N G H E T H O N G TIN VA T R U Y E N T H O N G - D A i H O C THAI NGUYEN... thong tin vd Ti uyen thong - DH Thdi Nguyen Viec hoc ngoai ngii co su ho tro ciia may tinh da chung to hieu qua xu the ling dyng CNTT&TT giao due va dao tao noi chung va giang day tieng Anh noi... vien tieng Anh cua Trucmg Sau giai dogn thi diem, nhom du an nhgn duoc cac phan h6i tich cue tii phia giang vien va sinh vien thmg qua phieu khao sat Ket qua hgc tap cao hon o nhom 200 sinh vien