Enhancing the mechanical engineering program of thai nguyen university through redesigned learning outcomes

THAI NGUYEN UNIVERSITY Socialist Republic of Vietnam SOUTHERN LUZON STATE UNIVERSITY Republic of the Philippines DEM CANDIDATE: PHAM VAN HUNG ENHANCING THE MECHANICAL ENGINEERING PROGRAM OF THAI NGUYEN UNIVERSITY THROUGH REDESIGNED LEARNING OUTCOMES Specialty: Educational Management DISSERTATION SUMMARY OF EDUCATIONAL MANAGEMENT THAI NGUYEN, 2014 The study was done at: INTERNATIONAL GRADUATE STUDY PROGRAMS, THAI NGUYEN UNIVERSITY Scientific supervisor: The peer review 1: The peer review 2: The peer review 3: The dissertation is defended to the Panel of Experts at: Dissertation can be found at: - National Library; - Learning Resource Center, Thai Nguyen University; - Library of International Training and Development Center; - Library of Southern Luzon State University, Philippines 3 Chapter I INTRODUCTION In recent years, in Vietnam the scale of development of training has increased significantly Providing accountability data and consumer information on the quality of teaching and learning have been increasing pressures on the HEIs Thai Nguyen University is fully aware of the importance of training to meet the social needs and international integration Since 2008, the University has directed its members in making plans to provide quality training The University has organized training courses on formulating learning outcomes for its member colleges and units However, in the implementation process, the consultation of the employers and alumni as well as other stakeholders has not been focused and the learning outcomes of some programs were at the following three types (1) too high to achieve and (2) too low that does not meet with the requirements of the present socio-economic development and (3) some are too general which not provide basis for the development of the program It is for this reason that this study is very imperative in Thai Nguyen University BACKGROUND OF THE STUDY FACULTY OF MECHANICAL ENGINEERING The Faculty of Mechanical Engineering (TFME) is one of the largest and oldest departments of the Thai Nguyen University of Technology It is responsible for the training of highly qualified technicians to meet the needs of industrialization and modernization of the country and the need to integrate with the world In addition, the faculty organizes and implements scientific research, applying the results in production for engineering as well as other fields THE MECHANICAL ENGINEERING PROGRAM (MEP) The MEP was designed and formulated based on regulation of the Vietnamese Ministry of Education and Training The contents of the program were initially designed by some of the experienced senior lecturers of the University The contents of the program have been modified recently due to the Vietnamese policy to import the programs of developed countries and the MEP has imported some of the course contents of the Mechanical Engineering program of University of Buffalo, USA However, after completed 5-year-course training, the program has shown some problems that need to be fixed First, many graduates have to attend further training of 4-6 months before finding a job according to the results of the survey by the university Most graduates lack the necessary skills such as team work and communication, besides they lack practice to work in a new environment, it takes a lot of time for them to be familiar with the new situations Second, with regard to the course contents of the program, the logistics among subjects is not properly arranged, some can be cut down or cut off without affecting other subjects while these subjects not directly provide expected output knowledge and skills Third, the schedule and program specifications of some modules are not suitable to the needs of the students especially those that require the students’ skills The learning outcomes are not specified to develop suitable program specification Using the modern theory in developing a program, the Faculty should start from the learning outcomes that are designed based on the demand of the society, employers, alumni and other stakeholders of the program It is for this reason why the learning outcomes of the MEP should be redesigned to start the enhancement of the MEP OBJECTIVES OF THE STUDY Determine the importance of draft learning outcomes items of the Mechanical Engineering Program in terms of: 1.1 Disciplinary Knowledge and Reasoning 1.2 Personal and Professional Skills and Attributes 1.3 Interpersonal Skills: Teamwork and Communication 1.4 Applying Knowledge to Benefit Society Evaluate the current status of learning outcome topics and the expected LO topics of MEP by: 2.1 Alumni 2.2 Faculty 2.3 Graduating students 2.4 Employers Redesign the learning outcomes for the MEP SIGNIFICANCE OF THE STUDY This study was conducted to develop the learning outcomes for Mechanical and Engineering Program in order to meet the requirements of the society and the international integration Hence, this study would be beneficial to the Administrators, Teachers/ Professors, Students, Employers and Future researchers SCOPE AND LIMITATION OF THE STUDY The study was implemented in Faculty of Mechanical Engineering - College of Engineering - Thai Nguyen University with the involvement of 250 students who are in the final year and going to graduate in June, 2013; 50 faculty who are teaching in the program; 100 enterprises representatives; 100 alumni of the MEP The development of learning outcomes was based on a literature review of relevant formulation of mechanical engineering learning outcomes, engineering program accreditation standards, standards of engineer, using questionnaires, expert discussions and consultation with stakeholders of the program The questionnaire was built to serve the research process The timeframe for the study was from February 2013 to July 2013 Chapter II REVIEW OF LITERATURE AND STUDIES In this chapter, the following were discussed: the literature and studies of the learning outcomes, the roles of the learning outcomes in developing the curriculum and the program, how to formulate the learning outcomes of a program, graduate criteria and the engineer criteria of some organizations Learning outcomes “Learning outcomes describe what a learner is expected to know, understand and be able to demonstrate after successful completion of a process of learning They are statements of concrete and verifiable signs that witness/ certify how the planned competences, including the required levels of knowledge are being developed or acquired”3 Types of learning outcomes With regards to learning outcomes, different writers have different classifications of learning outcomes The most common classification is the two types of LO – they are cognitive outcomes: the recall or recognition of knowledge and to the development of intellectual abilities and skills (Posner, 1992) and noncognitive outcomes: beliefs or certain values (Ewell, 2005) Learning outcomes and outcomes-based approaches Learning outcomes and outcomes-based approaches’ have a strong impact on curriculum design, teaching, learning and assessment, as well as quality assurance They constitute an important part of modern approaches to higher education and the reconsideration of such vital questions as to what, who, how, where and when we teach and assess The role of learning outcomes in program design and improvement The learning outcomes are used to set the schedule of training activities - mapping to develop learners’ thinking and other necessary skills The roadmap starts from the fundamental to advanced knowledge in science and technology Learning outcomes are the direction and destination of teaching and learning Learning outcome is the basis for the design of teaching content, teaching strategies, and selection of appropriate and effective methods and assessment tools Learning outcome helps institutions to determine requirements of human resources (faculty, staff), financial resources and other resources The different between learning outcomes and learning objectives Learning outcomes and objectives are more difficult to distinguish as objectives can be written in terms of learning outcomes 14 In general, in comparison to program objectives, LOs are detailed statements of what learners should know and be able to by the time they graduate and program objectives are description of the competences of the graduate some years after their graduation 17 How to write Learning outcomes The learning outcomes will be developed under consideration of many variables including: qualifications frameworks, external reference points, past experience, subject benchmark statements, employer requirements, student feedback, qualifications descriptors, etc And the Bloom’s taxonomy and corresponding verbs are often used to write learning outcomes Engineer standards/ criteria In Vietnam, Vietnamese Law of Education and Law of Higher Education state the general competence of a graduate of higher education All over the world, the learning outcomes of graduated engineer which are stated in the Criteria for Accrediting Engineering Programs by Accreditation Board for Engineering and Technology; EUR-ACE Framework Standards for the Accreditation of Engineering Programs (2005) and Boeing's desired attributes of an engineer (Boeing, 1996) include knowledge, skills and attributes the followings: A good understanding of engineering science fundamentals; design and manufacturing processes; A multidisciplinary, systems perspective; A basic understanding of the context in which engineering is practiced; Good communication skills; High ethical standards; An ability to think both critically and creatively - independently and cooperatively; Flexibility; Curiosity and a desire to learn for life; A profound understanding of the importance of teamwork CDIO Initiative The CDIO™ INITIATIVE 25 is an innovative educational framework for producing the next generation of engineers CDIO is the framework for curricular planning and outcome-based assessment CDIO Syllabus, a statement of undergraduate engineering education goals, and a set of 12 Standards designed to help achieve the goals The Standards address program philosophy, curriculum development, design-build experiences and workspaces, new methods of teaching and learning, faculty development, and assessment and evaluation The Syllabus provides the answer to the question of what skills, knowledge, and attitudes should engineering graduates should possess RESEARCH DIAGRAM Current learning outcomes CDIO syllabus Draft LOs level References: - University/ College mission; - College vision; - MEP’s objectives - National and International accreditation standards; - Engineer standards Draft LOs level Survey on draft LOs level Main stakeholders: - School administrators - Employers - Alumni - Faculty - Graduating students Complete the LOs - Processing data - Fulfilling the LOs Figure The schematic presentation of the study to develop the learning outcomes which meet the requirements of the society and international integration Chapter III METHODOLOGY This chapter dealt with the locale of the study, research design, population and sampling, data gathering procedures, and statistical treatment used in the study LOCALE OF THE STUDY This study was conducted in the Faculty of Mechanical Engineering – College of Engineering - Thai Nguyen University – Thai Nguyen Province – Vietnam to collect the ideas of teachers and students about the present level of competences of the students, the importance of each outcome and the desired learning outcomes 9 In order to collect information and suggestions as well as other ideas of the alumni and the employers about the learning outcomes, the study was carried out at the organizations, factories and institutions where the alumni work RESEARCH DESIGN The study employed the descriptive research design The researcher determined the draft LOs item of MEP then conducted the research to evaluate the current status of the outcomes that the graduates have and what different stakeholders expected of the graduates of the program On the basis of the result, faculty and education researcher met and discussed on the level of proficiency that each LOs topic deserved and finalized the intended learning outcomes POPULATION AND SAMPLING Population • Industry’s representatives: enterprise representatives (or enterprises) who hire graduating engineers from the Mechanical Engineering Program (MEP), head of the technical division, head of the marketing division and director and vice director of enterprise We focused on the enterprises in Thai Nguyen Province • Alumni: alumni who graduated from the MEP and had more than two years of working experience • Undergraduate students: the senior students of or alumni who just graduated from the MEP • Academic staff: senior lecturers that had a teaching experience of at least 10 years and been responsible for teaching the advanced engineering programs Sampling For the pilot survey, the number of alumni who are working in Thai Nguyen Province (20), enterprises (20), teachers (20) and students (30) in their final year For the real survey, since the number of graduates from the MEP who kept in contact with the College was limited, so that we incuded in the survey were the alumni who were working in Thai 10 Nguyen Province (100), enterprises (100), teachers (50) and students (250) in their final year INSTRUMENTATION The research employed a survey type questionnaire to determine the attitudes of school administrators, employers, alumni and graduating students The questionnaire was composed of three parts: Part I Letter to the respondents Part II Personal information Part III Guidance on answering the questionnaire which includes the following: Learning outcomes topics Level of importance Current proficiency level 1.1.1 …… a B c d Intended proficiency level       - Level of importance, level scales (a-b); from “not important to very important” - Current proficiency level, level scales (0-5); - Intended proficiency level, level scale ( - ); Scales for the proficiency level were as follows: 0/ Do not know or possess 1/ To have experienced or been exposed to 2/ To be able to participate in and contribute to 3/ To be able to understand and explain 4/ To be skilled in the practice or implementation of 5/ To be able to lead or innovate in The explanation for the researcher using these six scales is as follows: In order to define specific learning objectives at a level commensurate with the proficiency rating, a correspondence must be developed between Bloom’s taxonomy and the activity based proficiency scale By considering the levels of cognitive, affective, 11 and psychomotor skills that underlie a certain activity based proficiency, a correspondence can be drawn In fact, there is no cognitive skill associated with the first proficiency level, “to have experienced or been exposed to.” The second level, “participation,” implies at least “knowledge," the first Bloom level in the cognitive domain “Comprehension," as defined by Bloom, specifically includes “explaining.” Likewise, “skill in the practice” arguably implies the ability to “apply knowledge” and “analyze." Finally, the ability to “lead and innovate” requires an ability to “synthesize and evaluate.” Similar approximate correspondences can be drawn to the affective and psychomotor domains STATISTICAL TREATMENT The reliability of the assessment tools of different group of respondents was examined as follows: Student, Alumni, Employer and Teacher by using: Cronbach 's Alpha Levene's test was used to examine the different among groups of respondents Dependent T-test was employed to compare the means of current proficient level and the expected learning outcomes Weighted arithmetic mean was also used to calculate the rating of different groups of respondents 12 Chapter IV RESULTS AND DISCUSSION This chapter presents the results of the study Analysis and interpretation are also given in the discussion The researcher’s acceptance or rejection of hypothesis given is also provided in the light of findings generated 4.1 The importance of each learning outcome topic under the view of the four main stakeholders of the MEP 4.1.1 The importance of each learning outcome topic under the view of the students All the topics under the Disciplinary knowledge and reasoning are considered as important in the learning program, and it is necessary to have them in the learning outcomes of the MEP Among the topics surveyed, the topics belong to the group of Knowledge of underlying mathematics and sciences have the important level which is a bit lower than in the groups of Core engineering fundamental and advanced knowledge Topics related to Other Supplement Subjects get lower rate than those in other groups Regarding the Personal and Professional Skills and Attributes, most of the topics have the important level of above ranging from 2.11 to 2.37 which means they are important to very important in the learning program outcomes The only topic which has the important level less than two is 2.1.4 Analysis With Uncertainty (1.97) As interpersonal skills: Teamwork and communication is concerned, the students are also concerned about the skills of teamwork and communication which reflect the reality that the graduates should possess those skills to be successful in their jobs In relation to the CDIO skills which are conceiving, designing, implementing and operating systems in the enterprise, societal and environmental context, the students considered those topics to be included in the learning outcomes as important The important weighted means (WM) were from 1.94 to 2.20 It can be 13 seen from the results that all topics were rated as rather important which should be the learning outcomes of the MEP 4.1.2 The importance of each learning outcome topic under the view of the alumni The researcher put the same questions to the alumni about the importance of each learning outcome topic The questionnaires were delivered and there were 36 feedbacks The data were recorded and analyzed The results are as follows: Regarding to the topics belonging to Disciplinary knowledge and reasoning, the alumni evaluated them at high level of importance The topic which got the lowest rate is 1.1.6 Chemistry (1.44) and the topic which got the highest level of importance is 1.2.12 Tolerance (2.86) As to the Personal and Professional Skills and Attributes and Interpersonal skills: Teamwork and communication skills are concerned, the alumni reflect their concern that the graduates should possess those skills to be successful in their jobs The important levels range from 2.06 to 2.83 No topics have the important level below which show that they are necessary to be in the outcomes of the program With regard to the interpersonal skills: teamwork and communication, the alumni considered the interpersonal skills very important since all the topics were rated above 2.0 Related to the CDIO skills, the alumni rated them as rather important The important levels were rated from 2.08 to 2.75 In conclusion, according to the alumni, all the topics are important and should be included in the outcomes of the MEP 4.1.3 The importance of each learning outcome topic under the view of the teachers The researcher delivered 50 questionnaires to the teachers and received 48 feedbacks which were eligible for data processing Under the view of the teacher, most of the topics are important and very important The topics which are very important 14 are those in the groups of personal and professional skills and attributes, interpersonal skills and applying knowledge It can be seen from the results that some of the topics related to disciplinary knowledge and reasoning are not evaluated as important as other topics Knowledge related to the engineering skills and personal skills and communication skills are important according to the teachers Under the view of the teachers, the importance levels of the interpersonal skills: Teamwork and communication topics range from 2.00 to 2.48 No topics have the important level below which show that they are necessary to be in the outcomes of the program Related to the skills which are conceiving, designing, implementing and operating systems in the enterprise, societal and environmental context, and teachers considered those topics included in the learning outcomes as important The importance were rated from 1.77 to 2.48 in which topic 4.4.4 Disciplinary Design got the highest important value (2.48) and 4.1.6 Developing a Global Perspective got the lowest (1.77) It is interesting to note that the students, alumni and teachers share the same opinion of the importance of those learning outcome topics In conclusion, according to the teachers, all the topics are important and should be included in the outcomes of the MEP 4.1.4 The importance of each learning outcome topic under the view of the employers The researcher did study on 100 employers and received 28 feedback, all of them are qualified for data processing The employers are working in different companies and factories in Thai Nguyen Province They are leaders of the organizations and engineering divisions: In Chemical Engineering MTV limited company No13 (6), Pho Yen Engineering Joint stock Company (5); Luu Xa steel mill (3); Coking plant, Thai Nguyen Iron and Steel Corporation (6); Factory for iron making (4); Sông Công Diesel Company (2); Medical Instruments plant (2) In general, among the four groups of topics of learning outcome which are Disciplinary knowledge and reasoning, Personal and Professional Skills and Attributes, Interpersonal skills and 15 Conceiving, designing, implementing and operating systems in the enterprise, societal and environmental context, the employers placed more important on the engineering knowledge and skills, the topics under Knowledge of underlying mathematics and sciences are not evaluated as importance as other topics As Personal and Professional Skills and Attributes and Interpersonal skills: Teamwork and communication are concerned, the importance levels range from 1.04 to 2.68 Interpersonal skills are those which draw concerns of the employers Levels of importance range from 1.89 to 2.32 Communication in English seems to get lower important level than other topics (1.96) and from interview, it is clear that the employers think engineers could learn when they are at work Besides, 3.2.2 Communications Structure got the lowest level of importance (1.89) The employers highly rated the skill of leader which got the highest level of importance within the field (2.32) Related to the conceiving, designing, implementing and operating systems in the enterprise, societal and environmental context topics, the importance were rated from 1.61 to 2.64 in which topic 4.4.3 Utilization of Knowledge in Design got the highest important value (2.64) with 71.43% of respondents rated as very important and topic 4.1.6 Developing a Global Perspective got the lowest (1.61) which got 10.71% of the employers rated as not important, 7.14% rated as very important and 57.14% rated as rather important In conclusion, it can be seen from the survey results that under the point of view of the employers, all the topics are important and should be included in the outcomes of the MEP however the competent level should be considered 4.2 Evaluate the current status of learning outcomes and the expected learning outcomes of MEP In this part, the details on what each group of stakeholders expected from the graduate of the MEP and what is the gap between the current and expected learning outcomes were discussed Since this study goes in the direction of outcomes-based learning, all the results were considered but the results from the groups of alumni and 16 employers were given more attention in the formation of the new learning outcomes 4.2.1 Students According to the senior students, they have had already enough knowledge of basic mathematics and sciences They need more advanced engineering fundamental knowledge, other personal and professional skills as well as skills in communicating with others and in building teamwork They rate themselves weak in applying knowledge into real situation like designing, implementing and operating engineering system within the enterprise, social context This once confirms the lack of skills of students before graduating What students hope that the program would provide them would be skilled at the 3rd level that is able to understand and explain and it is very interesting to compare with students at MIT, USA that the hope to be at 4th and 5th level that engineer would be skilled in the practice or implementation or able to lead or innovate in the field In the topics of Personal and Professional Skills and Attributes, there are also gaps between current and expected proficient level The survey results show an interesting result that students wish to stay updated to the current world of engineering the gap is the biggest within the group (1.04) It is the evidence that students expected more from the program especially the updating of the engineering knowledge and skills In the topic of Interpersonal Skills: Teamwork and Communication, the gaps are much bigger as students think they are not very good at this matter and they hope to be more competent in the field Four (4) over eleven (11) topics have the gap of more than and the smallest is 0.86 The students evaluated their English communication at low level (2.14) and they expected to be more skillful in the field (3.36) Using t-test to test the difference between the current and expected proficient level of MEP graduates, the researcher found a difference between the current and expected level of proficiency (t= -8.75, p=0.000) In relation to the applying knowledge to benefit the society section: Conceiving, designing, implementing and operating systems 17 in the enterprise, societal and environmental context, students rate the current skills at rather low level ranging from 1.80 to 2.20 It means that some the students now have experienced or been exposed to the practice of and some are able to participate in and contribute to the designing, implementing and operating systems in the enterprise, societal and environmental context, few of them are capable of understanding and explaining Expected competent levels range from 2.62 to 3.15 and the gaps between current and expected competent level range from 0.7 to 1.12 According to the students, there is a gap between the current and expected competent levels of the graduate of the MEP, the difference was proved by t-test results 4.2.2 Faculty Regarding to the competence level of the graduates of MEP, the results of the survey have figured out that the current level of competency does not meet the requirements of the faculty of the MEP With regard to Disciplinary Knowledge and Reasoning, teachers rate students’ current level of proficiency at low level, ranging from 1.73 to 2.92 which are mainly “be able to participate in and contribute to”, and some teachers rate them as “able to understand and explain” Based on the current proficient levels, the teachers expect the proficient level to be from 2.6 to 3.69 – which means that the students should be able to understand and explain, and be able to implement and operate T-test results confirmed the difference between current and expected proficient level of the graduates (t value ranges from -10.25 to -5.07, p=0.000) Interpersonal skills including teamwork and communication are considered very important skills that an engineer should have in order to work effectively as a partner and a leader In reality, the matter takes much more consideration of the faculty and students when they are at school than before But the results of the survey show that what they have tried is not enough and the requirement of the faculty is higher in every topic The current competent level is below with 2.00 is the lowest The gaps between the current and 18 expected proficient level are all bigger than one (1) and the t-test results have once again confirmed the differences between them The competences to implement in CDIO approach include topics from designing to management The current proficient level is evaluated at low level ranging from 1.88 to 2.27 And there should be changes so that the graduate could possess higher level of competences ranging from 2.88 to 3.33 The gaps between the current and expected level of proficiency are from 0.69 to 1.35 Ttest confirmed the significant difference between the current and expected proficient levels of the graduates The research results of the current status of the learning outcomes and the expected ones regarding topics are as follows in pair: Disciplinary Knowledge and Reasoning (2.29 and 3.20), Personal and Professional Skills and Attributes (2.16 and 3.35), Interpersonal Skills: Teamwork and Communication (2.27 and 3.53) and Applying Knowledge to Benefit Society (2.01 and 3.21) 4.2.3 Alumni The alumni play an important role in evaluating the proficiency of the engineer since they have enough experience in their work They have the added perspective of the workplace or further education It is a perspective well worth tapping As far as Disciplinary Knowledge and Reasoning is concerned, the alumni evaluate the current proficiency of the graduates at higher level than the teachers did The results suggest that the alumni expect that all the topics need improvement with respect to student’s knowledge with the expected proficient levels range from 2.92 to 4.36 Interpersonal skills including teamwork and communication, the program did not meet the requirement of the alumni The competences at current level is below with 2.06 is the lowest which is for 3.3.1 Communications in English and all expected level is above with 4.00 is the highest level for 3.1.5 Technical and Multidisciplinary teamwork The gaps between the current and expected proficient level are all bigger than one (1) and the t-test results have once again confirmed the differences between them 19 The competences to implement in CDIO approach include topics from designing to management The current proficient level is evaluated at low level ranging from 2.19 to 2.39 And there should be changes so that the graduates could possess higher level of competences ranging from 3.44 to 3.61 The gaps between the current and expected level of proficiency are from 1.14 to 1.25 The topic which needs the greatest improvement is 4.5.4 Hardware Software Integration which is now 2.19 and the expected level is 3.44 T-test confirmed the significant difference between the current and expected proficient levels of the graduates The gaps between the expected and current proficient levels are rather high, ranging from 1.14 to 1.25 The results reflect the desire of the alumni that the graduate engineers should be more skillful in practice and even more, they should be able to implement or to lead in the fields The results of the survey have figured out that the current level of competency does not meet the requirements of the faculty of the MEP Especially, those competencies are acquired by practicing The research results of the current status of the learning outcomes regarding Disciplinary Knowledge and Reasoning (2.59), Personal and Professional Skills and Attributes (2.55), Interpersonal Skills: Teamwork and Communication (2.48) and Applying Knowledge to Benefit Society (2.42) while the expected level of proficiency of those topics are 3.50, 3.60, 3.75 and 3.48 respectively 4.2.4 Employers If the program is preparing students for a particular set of jobs, it might be worthwhile to survey employers regarding the students’ on-the-job performance However, it is important to survey those who would have first-hand knowledge of particular students rather than relying on general opinions or stereotypes In general, the employers evaluated the graduates’ current proficiency lower than what the students and the alumni did and they expected much more from the graduate of the MEP With regard to Disciplinary Knowledge and Reasoning, the current proficient level was 2.26 while the expected level of proficiency of the topic was 3.22 As to the Personal and Professional Skills and Attributes was concerned, the results were 1.93 for the current and 2.97 for the 20 expected) The employers also rated the current proficient level of Interpersonal Skills: Teamwork and Communication topics at 1.81 and what they expected was at 3.29 Learning outcomes relating to Applying Knowledge to Benefit Society got big gap (1.18 point) between the current (1.72) and expected level of proficiency (2.90) 4.3 The correlation among the important level of learning outcomes, current proficient level of the students, the expected proficient level that the students should achieve Chi-square test of the relationship between current competent level variable and the level of importance variable confirms a close relationship between these two variables Chisquared = 30861.056 and p-value = 0.000