Luận án quản lý kiểm tra, đánh giá kết quả học tập học phần theo chuẩn đầu ra trong các trường đại học thuộc bộ công thương TT

1 MINISTRY OF EDUCATION AND TRAINING THE VIETNAM NATIONAL INSTITUTE OF EDUCATIONAL SCIENCES  DUONG THE VIET TESTING, ASSESSMENT MANAGEMENT ON STUDY RESULTS COURSE ACCORDING TO LEARNING OUTCOMES FROM UNIVERSITIES IN THE MINISTRY OF INDUSTRY AND TRADE Major: Educational Management Code: 14 01 14 SUMMARY OF PHD.THESIS ON EDUCATION SCIENCE Hà nội, 2022 Finalized at: The Vietnam National Institute of Educational Sciences Scientific supervisors: Ass.Prof.Dr Phan Van Nhan Ass.Prof.Dr Nguyen Duc Son Reviewer 1: …………………………………………………… Reviewer 2: …………………………………………………… Reviewer 3: …………………………………………………… This thesis will be reviewed at Institute - level Thesis Review Council, The Vietnam National Institute of Educational Science, 101 Tran Hung Dao Street, Ha Noi On….Month….Date…Year This thesis can be found at: - The National Library - The library of The Vietnam National Institute of Educational Sciences INTRODUCTION Reason for choosing the topic In the trend of globalization and international economic integration, higher education plays an important role in training human resources, especially high-quality human resources, making decisions to the socio-economic development of each country; Simultaneously, the industrial revolution 4.0 will create profound changes in the structure of the labor industry, new requirements on the capacity and skills of learners as well as a fundamental change in the training model, curriculum, teaching and learning methods, assessment method For the assessment of the course curriculum to achieve the learning outcomes (LO), it is an activity in the training organization process and is considered a breakthrough to improve the quality of higher education However, there are still shortcomings in theory and practice Ministry of Industry and Trade currently manages 09 universities, so it is necessary to make changes in the content, method and manner of assessment to be able to respond the goal as "Developing and improving the quality of human resources, especially high-quality human resources "[4] Therefore, studying for the testing, assessment about LO of the courses according to the criteria to propose possible management solutions, in accordance with the context and development characteristics of each university belongs to the Ministry of Industry and Trade is an important and pressing issue Research purpose of the thesis On the basis of theoretical and practical research, thesis aims proposed some innovative solutions for management and testing, assessment about LO of the courses according to the criteria for the universities in the Ministry of Industry and Trade to meet the needs of human resources for society Objects, research subjects 3.1 Objects of research: Assessment for study results of the course according to LO 3.2 Subjects of research: Solutions managing the assessment for study results of the course according to LO in universities at the Ministry of Industry and Trade Scientific hypothesis Currently, there are many shortcomings in management for the assessment according to LO at universities in the Ministry of Industry and Trade If researching and proposing solutions managing the assessment for study results of the course according to LO that are practical and feasible, the training quality of the universities will be improved, meeting of needs of human resource development for society Research tasks and scope 5.1 Research tasks: - Research theoretical basis of the in the management assessment for results following the LO - Survey and evaluate the state situation in the management assessment courses according to the LO in some universities at the Ministry of Industry and Trade - Propose solutions in order to manage the assessment for the LO - Test necessary measures, practicability and trail the solutions assessment about the LO are proved 5.2 Research scopes: - Thesis is limited to research on real stuiation in management assessment of the LO according to some training programs at universities in the Ministry of Industry and Trade - Solutions are managed by Ministry of Industry and Trade such as the evaluation, test for LO at theoretical courses according to the LO of the training programs, which are researched and applied at some universities belong to Ministry of Industry and Trade - The objects of the survey are administrators, teachers, students, some experts in education and businesses have directly related to the training program Methodology and research methods 6.1 Research approaches: Approaching labor market; approaching education based on LO; approaching CDIO management process; approach according to management function 6.2 Research methods: Theoretical research methods; Practical research methods; Other complementary methods Theoretical poit defends a thesis - Managing assessment according to the LO is an inevitable trend by defining the goals, perfecting the policies, tools, and organizing assessment to achieve the results of LO - The actual situation of the assessment of the course curriculum according to the LO at universities in the Ministry of Industry and Trade shown that causes of the inadequacies belong to the management - The solutions manage the assessment for study result of the course according to the LO are proposed in order to remove difficulties and overcome inadequacies The results of testing and experimentation of the proposed solutions confirm the necessity and feasibility of the solutions New contributions of the thesis - Determine for requirements and elements in management assessment of results of learning according to LO at the universities in the Ministry of Industry and Trade - Assess the actual situation, then point out advantages, difficulties and causes - Proposing solutions contribute to improve the quality of training Place of research topic - Vietnam Institute of Educational Sciences - Surveying the uctual situation at the University of Economics - Industrial Technology and some universities in the Ministry of Industry and Trade 10 The layout of the thesis In addition to the introduction, conclusions and recommendations, the thesis includes chapters CHAPTER THEORETICAL BASIS FOR MANAGE THE ASSESSMENT FOR STUDY RESULT OF COURSE ACCORDING TO THE LEARNING OUTCOMES 1.1 Research problem overview Studies shown that there has been a change in opinion about assessment Testing Assessment becomes an integrated activity in the teaching process, shifting from attaching importance of a summary assessment to a processed assessment, the assessment activities are interwoven in the teaching and learning process The trend of testing, assessment in higher education today is approached with the terms: "Authentic Assessment", "competence Assessment", "evaluating output", with the goal of measuring and evaluating knowledge, skills and attitudes of learners after a course, a learning process There are not many research works on the management of assessment of the students, each study approaches one aspect such as model building or the management process In order to study clearly about the management of the results for learning of the students, modules according to the criteria at universities in the Ministry of Industry and Trade, this thesis needs to continue to study: - Theoretically: Clarifying the trend, the role of assessment activities followes the LO is a requirement for innovation in education management; point out factors affecting to the course’s assessment activities, elements impact the LO - In terms of practice: It is necessary to clearly assess the state situation, from that recommend solutions to manage the assessement in some universities in the Ministry of Industry and Trade at present 1.2 Basic concepts of assessment for study results according to learning outcomes 1.2.1 Course: Couuse is a learning unit in the curriculum, including a amount of knowledge in relatively complete, convenient for students to accumulate during the learning process The knowledge in each module is designed as a module model for each subject or structured in the form of a combination from many subjects 1.2.2 Study result of the course: Study result of the course is understood as: i) Systems of knowledge (knowledge, skills, attitudes) that learners accumulate through the learning process; ii) Reflect for response level of learners' knowledge, skills and attitudes compared to the LO of a module or the entire training program, which is assessed through measurement with evaluation criteria and expressed by score; iii) Through, it will reflect the quality of programs 1.2.3 Kiểm tra, đánh giá kết học tập 1.2.3 Testing, assessments a) Testing: Testing is an activity of collecting information in the teaching process by using pre-prepared tools, in order to measure actual status of the result laerning, as well as find out causes that exist to take measures to overcome to achieve the identified educational goals b) Assessment: Assesment is a extensive terminology that includes both testing and assessment, testing as a form of assessment In the thesis, the concept "testing, assessment" is used to refer to both "test and asessment" activities or "test" or "assessment" to refer activities in different contexts In the process of teaching - learning, the assessment has two functions: i) Assessment for the reusult learning; ii) Assessment helps to adjust the teaching process to achieve the defined goals 1.2.4 Learning outcomes: Statements of what a learner will know, understand and/or be able to demonstrate after completing a lecture, a course, or the entire curriculum Thus, the assessment of the study result of the course according to the LO is the process of collecting and processing information about the qualifications and abilities that the learners have in comparison with the expected LO / identified outcomes of the course, in order to create the basis for pedagogical decisions of teachers, faculty, schools and for learners themselves to help them learn more progress; simultaneously, confirm the study result to prepare for the next learning phase 1.3 Kiểm tra, đánh giá kết học tập học phần theo CĐR 1.3 Assessment for study results of the course according learning outcomes - Examining and asessment the course according to the LO have the following roles: i) teachers and learners use it to determine what knowledge and skills learners have acquired as defined in the LO; ii) Teachers use it to monitor the progress of the learners, thereby teaching - learning activities will adjust so that the learners achieve the LO of the course; iii) learners develop higher-order thinking abilities, self-concept skills to meet future career development requirements - The principles: i) Convert from mainly using summative assessment to attach importance to process evaluation; ii) Convert from cognitive assessment to skill and competence assessment; iii) Switching from one - or two-way assessment to multidimensional assessment, combining many assessment and helping learners develop selfassessment skills; iv) Change from assessment (testing, assessment) to become an integrated part of the teaching - learning process; v) Using IT and statistical models in analyzing and interpret assessment results - Type: i) formative assessment and summative assessment; ii) norm-referenced assessment and criterion-referenced assessment - Method: Direct measurement; Comments from colleagues; Analyze statistical indicators; Comments of experts (internal and external) and recently, educators have introduced a number of authentic assessment methods, including types: demonstration assessment, quick search, opened - ended questions, document learning, and selfassessment 1.4 Approaches management for assessment of study result the course according to learning outcomes 1.4.1 Management for assessment study result of the course according to learning outcomes We believe: Management for the testing, assessment study result of the course according to learning outcomes is the process of determining the factors to affect the planning, organization, direction and inspection in order to collect, process information about the qualifications and abilities that learners have compared to the expected learning outcomes, creating the basis for pedagogical decisions of teachers, professional faculties, universities, and for learners themselves to help them learn better; as well as learner’s confirm on learning outcomes to prepare for the next learning phase 1.4.2 Approaches management for the testing, assessment study result of the course according to CIPO’s model Applying the CIPO model to manage the assessment study result of the course includes components: - Inputs include: Policy management, staff; develop a database of questions bank; assessment plan for the whole course, conditions for implementing assessment - The process includes: Method and form of formative assessment, final test - Outputs include: Manage assessment results for course, evaluation of the implementation of course assessment according to the LO - Impact of context: Policy of state management agencies, learners' capacity requirements for the labor market When applying the CIPO’ model to the management of learning assessment according to the criteria, there are still limitations, such as: Not bring into play maximized flexibility, initiative and responsibility of members in the organization; The decentralization has not been shown: who manages, at what level 1.4.3 Approaching managing the assessment of resut learning by function According to modern management, from the management functions can generalize some basic management functions: 1) Planning, 2) Organizing, 3) Directing, 4) Checking and evaluating The management of the assessment of the study result according to the LO is approaching by work’s function, which is the sum total of the works of administrators, teachers and learners, including setting out mechanisms, policies, evaluation criteria, and implementation solutions, allocate resources, organize the implementation, inspect and test to best perform all stages in the asessment process in order to accurately assess learners' learning outcomes and help improve teaching and learning to achieve get the LO of each course The elements of asessment management include: The management subject is the leader (setting out the mechanisms, policies, statutes and regulations on the assessment) and the responsible management staffs (both managing and implementing directly); the staffs are managed by leaders (responsible management staffs, teachers, learners, ) 1.4.4 Theoretical framework of management assessment of study result in course according to the learning outcome From the factors of input, process, output, along with the impact of the context, the thesis proposes a theoretical framework (table 1.1) presented below, as a basis for practical research and recommendations for the solutions to manage the management assessment of study result of the course according to the LO Table 1.1: Theoretical framework of management assessment of study result according to the learning outcomes No Management according to CIPO Management functions Planning Organization I Management inputs I.1 Organizing for setup Planning to improve the policies in policies related to assessment: operations study results Tasking to check up for assign departments, supplement policies related to assessment for study result I.2 Development of responsible human resources in giving an take examination and teachers on assessment activities Assign specialized departments and faculties to coordinate in organizing training courses, refresher courses, seminars on asessment skills I.3 Establish assessment courses I.4 Give planning on training and retraining to improve the capacity of responsible human resources in giving an take testing and teachers on assessmnet activities plan Setting up assessment Assign the training department, for for whole course the examination department to make the whole course's assessment plan Organizing in Make a plan to check Assigning departments to development of the up and supplement the review and supplement the questions banks questions banks questions banks in accordance with the regulations on "building, managing and using Directing Monitoring Checking Input management Directing to check up and Check on supplement for policies implementation, related to activities in simultaneous adding assessment the policies related to assessment activities Motivating the full-time Monitor the capacity staffs and teachers to of the team in take of participate in training testing and teachers courses on testing work on assessment Encourage teachers and activities responsible human resources to research and exchange experiences in assessment activities according to the characteristics and nature of specific training disciplines and course Guide the full- time team in Monitor the process take examination, make the of the plan on full course's assessment assessment activities plan Directing the departments Supervising the to review and supplement process of reviewing the questions banks and supplementing the questions banks No Management according to CIPO Management functions Planning I.5 Prepare necessary - Planning to set up a and sufficient specialized department conditions to carry in testing out the assessment - Making a plan for infrastructure investment to serve the testing II Process Management II.1 Management of formative assessment activities according to LO (methods, tools and contents) Planning for the formative assessment according to LO (method; tools and contents) Organization exam questions at the end of the course" - Assign the functional department to select personnel to set up a department specialized in testing - Assign take examination, training and unit departments to determine the need for equipments to serve the course assessment - Organize for teachers and departments to develop criteria for assessing the learner’s results; select methods, forms, and tools for assessment in order to achieve satisfactory results - The teacher directly conducts the assessment and recognizes the formative assessment for students Directing Monitoring Checking Input management - Guide the process of recruiting personnel to set up a department specialized in testing Directing the examination, training and unit departments in the process of planning for equipment investment in service of the assessment of the leraning results Supervising establishment of a department in take of testing - Ckeck using the equiments service of the evaluation of the leraning results Flexibly applies the reward and discipline policy to assessmets activities in class; at the same time, encourages teachers to innovate the assessment method to achieve results follow LO - Supervising the teachers to perform the course assessment in class - Proposing corrected measures, reorganize and strengthen shortcoming formative assessment of whole course No II.2 Management according to CIPO Management functions Planning Organization Managing the Plan to organize the Assign tasks to the specialized organization of the end-of-course exam by department (examination, end-of-course exam semester training) to coordinate with the faculties to organize the course assessment, including: making exam questions, organizing exams, marking exams, announce scores and manage exam results III Output management III.1 Management of study Planning to announce results the course and organize the management of students' academic results according to regulations III.2 Organize the assessment of the implementation of the evaluation for study result according to LO IV Implementation context Make a plan to periodically evaluate the performance of course according to the criteria Planning to collect information on labor market requirements from stakeholders Assign a specialized department (examination, training) in check on testing to announce course results for students and organizations manage student's performance following regulations Assign the functional department (examination, training, faculty) to periodically evaluate the performance of course according to LO Assign functional departments (training, quality assurance) and Faculty to collect information on Labor market requirements from stakeholders Directing Monitoring Hướng dẫn phận Guide the specialized department (examination, training) to coordinate with the Faculty to organize the course assessment, including: making exam questions, organizing exams, marking exams, announce results Checking Input management Supervising and checking the process of organizing the endof-term exam by semester Monitor and manage exam Check/inspect the results management of student's academic performance in accordance with regulations Guide the full-time department (exams, training) coordinate with teachers to evaluate the course's according to LO Monitor periodically evaluate the performance in the assessment of the course results according to the LO Direct the collection of Monitor the information on labor implementation of the market requirements from information collection stakeholders on market requirements labor from stakeholders 11 administrative investment for teaching - learning and assessment activities is at a good level, with an average value is 𝑋𝑗 = 2.81 Most teachers and administrators think that the budget for the development of training programs is the most reasonable (ranked 1st, 𝑋𝑗 = 2.88); The level of funding investment for the information technology system ranks the lowest (ranked 6th, 𝑋𝑗 = 2.60), thereby partly reflecting the current situation of universities needs a large amount of funding and must perform following path in a long time 2.3.2 Current status of assessment for course according to factors in process 2.3.2.1 Current status of formative assessment The average score of the criteria for surveying the status of the formative assessment is at a good level (𝑋𝑗 = 2.82) The survey results also shown that there is a large dispersion among the survey criteria Through, the assessment activities process also shown that teachers have not really attached importance to the innovation of teaching methods to achieve the results (𝑋𝑗 = 2.91), which is one of the requirements for innovation of assessment activities For students, there are two notable criteria: "Students clearly understand the objectives and outcomes before teaching the course" and "Students adjust their learning methods (including classroom learning, self-study, self-study research) to achieve the LO” is also not evaluated as high (𝑋𝑗 = 2.39 and 𝑋𝑗 = 2.20) these are the “necessary” conditions of students when participating in the learning process to achieve the competencies required at the end of the course 2.3.2.2 Current status summative assessment (exam) of the course The average score of 16 survey criteria is at an average level of good (𝑋𝑗 = 2.73), the criteria for assessing the level of achievement not have a big difference and are revolved around the average value Regarding the exam questions, there is a division between the management staffs and the teachers, there are 91.3% of the management staffs who think that the exam questions fully reflect the objectives and outcomes of the course, while only 61.3% of the teachers agree with this idea Control and inspection work, according to the management staff’s assessment: "The current inspection team is still thin and has only performed well in regular control and inspection, mainly monitoring and reminding teachers in the process of examining and marking exams and students strictly complying with the regulations in the exams; For thematic inspection is still leave it open, because the thematic inspection needs to mobilize a team of human resources with deep expertise, to carry out for a long time and not to mention the funding that must be spent for this activity” 2.3.3 Actual status of assessment for course according to output factors According to the survey results, currently, the training programs have about 38.5% of the final exams in the form of multiple-choice exams and about 42.3% of the basic courses of the majors/specialties take multiple-choice exams The analysis of the entire score spectrum for the first test showed that: The score spectrum is deviated from the normal distribution or is much left skewed This distribution manifests three possibilities: i) the questions in the exam are not based on the LO or the question builder does not understand the LO in depth; ii) the teaching has not covered the knowledge students need to achieve; iii) there is a difference between students in terms of grasping core knowledge 12 At the same time, according to the survey results, there are about 19.6% of students on studying and think that the method of assessment of international relations is not suitable with the nature and characteristics of the subject This showed that teachers need to invest effort in developing criteria and implementing more effective application of information technology in the formative assessment 2.4 The current situation of management assessment of the course according to the learning outcomes 2.4.1 Current status of management according to input factors 2.4.1.1 The current status of awareness of the learning outcomes and testing assessment of the management staffs and lecturers In general, teachers' perception of the content of the assessment of course results in universities under the Ministry of Industry and Trade is evaluated at a good level, the average value of 𝑋𝑗 = 3.27 Through the discussion, the teachers said that training management depends on many factors and the assessment depends not only on the students' academic performance, but also on the policy mechanism, the students' learning consciousness, the seriousness of the students' academic performance of the organization of exams Therefore, in order to improve the quality of management assessment of course assessment activities, in addition to raising awareness of the position and role of assessment activities for managers, teachers also need to have measures to synchronize management with other factors 2.4.1.2 Actual situation of organization in formulation of assessment policy The system of documents on management of course assessment, including processes and regulations on functions and tasks, is built by the universities in accordance with the general requirements of the Ministry of Education and Training, in order to meet training objectives, ensure fairness, objectivity and proper assessment of capacity, aiming at the LO for students after the end of the course However, through the results of interviews and discussions with the examiners at some universities, there are still limitations: - The system of documents has overlapping contents leading to difficulties in teaching, students look up and search for instructions on building evaluation criteria - The bank system of essay questions has not been scientifically built according to the matrix of cognitive levels (understanding, knowing, applying, ) - Survey results in 2020 shown that there is still a large percentage (23.78%) of students who are not satisfied with the way in which the results of the formative assessment are announced and responded to policy management related to the assessment of the course performance assessment was assessed by the management staffs and teachers at the level of Good (𝑋𝑗 = 3.24) Through the survey data, it was found that the management staffs, teachers perceived the policy of reviewing, adjusting and supplementing the regulations related to assessment activities to be more appreciated than the implementation of policies from individuals In order for administrators, teachers and students to properly and seriously implement assessment activities, the University has diversified forms of disseminating rules and regulations However, there are still 18.2% of teachers and 12.6% of students are not really satisfied, so a management solution is needed to improve the efficiency of this work 13 2.4.1.3 The status of planning the assessment of the course In general, teachers and management staffs attach great importance to planning the testing assessment of the course's performance; weight value is 𝑋𝑗 = 2.79 and the average index of planning contents is estimated to be approximately the upper limit of good Through the above data, together with the discussion with the respondents, they all said that: - Currently, many modules of LO are unclear and difficult to measure Outcomes cannot be used as a standard for course assessment - Issue a process test questions that is assigned directly by the faculty (subject group) to the teachers who teach the subject The test only stops at the level of knowledge presented by the teacher in class, only 21.12% of students are satisfied with the question "The content of the test/exam can synthesize the knowledge and skills” and only 18.51% of students think that the feedback from the assessment helps students improve their performance Therefore, it does not reflect the true nature and requirements of the assessment, and the effectiveness of the formative assessment is not high 2.4.1.4 The situation of developing the assessment capacity for the staffs and lecturers in taking of testing The survey results shown that, the University builds and prescribes functions and duties for the testing department and manages the capacity of staffs and teachers on the assessment work, which is assessed by management staffs and teachers at the level of Good𝑋𝑗 (=2 ,96) Through discussions with the management staffs, at the beginning of the school year, the school often surveys the needs to develop training and retraining plans for the units, however, the number of staffs and teachers registered for the assessment profession is small and not yet able to control the effectiveness of application after training and retraining sessions on assigned tasks 2.4.1.5 Status of management and planning of the assessment In general, administrators and teachers think that the organization and management of planning the whole course is not really good, the weighted average value 𝑋𝑗 = 2.84 and the average indicators of the planning contents are evaluated approximately upper approximation of level - The management and planning has been standardized according to a process consisting of steps, however, in each step, there is still a lack of guiding contents on technical aspects of implementation as well as no mechanism to encourage stakeholders outside the educational institution to participate (Enterprises, experts) in the process, so it is subjective of the group of teachers assigned to build the LO - Regarding the method of assessment, the current process is still undertaken by the teachers The management of the formative assessment is only managed in terms of progress and the number of points according to the regulations, it is necessary to have measures to monitor the quality of the teacher's formative assessment organization 14 2.4.1.6 The status of management and development of questions banks The development of the questions banks is currently the responsibility of the subjects This work is assigned by the Department to the group of teachers to perform annually The current situation of management and development of the questions banks was assessed by administrators and teachers as quite good, with the average score in the upper bound of the good (𝑋𝑗 = 3.01) Among the stages, the process of managing the development of the exam data bank was not appreciated, partly reflecting the reason for the lack of good organization of the review stage from the subject, outside experts and the funding for the exam This activity has not been focused, and the capacity of the team in implement of this work is still limited 2.4.1.7 Actual situation of financial management and facilities The current situation of financial management and administrative facilities related to assessment activities was assessed by management staffs and teachers as quite good, with an average score (𝑋𝑗 = 2.89) In which, the average score in the field of equiments (𝑋𝑗 =2.93) is rated higher than the average score in the field of finance (𝑋= 2.85), which partly reflects the situation from the Universities, when implementing the plan The financial plan for training organization activities in general, including assessment activities, encounters obstacles bound by general regulations from state management agencies (for example, funding sources for training and retraining) annual maintenance) 2.4.2 The current situation of managing the formative assessment and the final exam according to the learning outcomes 2.4.2.1 Current status of formative assessment management The assessment of formative assessment is one of the requirements for implementing the innovation of assessment in higher education Formative assessment not only recognizes the results for learners but also has the function to help learners accumulate the knowledge and skills necessary to achieve the expected outcomes The results of the management survey on assessment activities were evaluated by management staffs and teachers as good, with an average score (𝑋𝑗 = 2.87) Through discussions with teachers, currently the school manages the assessment of the process, only stopping at the management of the points according to regulations and comparing the final score of the process with the final exam score of the course to see the difference How to make judgments about the quality of teaching There is still no monitoring mechanism to encourage teachers to innovate the checking assessment to help people achieve the standards 2.4.2.2 Status of management exam at the end of the course Management of the end-of-course exam was assessed by the management staffs and teachers as being in the upper range of good, with an average score (𝑋𝑗 = 2.93) As for the essay form, according to the regulations, two independent rounds of grading are performed, but it has not been strictly implemented, without close supervision and inspection Therefore, the test results are still inaccurate and not guaranteed to be fair and objective The survey results showed that 10.6% of students were not satisfied with the question "Students are assessed correctly and fairly in examination, course exam and grading and assessment of training scores" 15 2.4.3 Current status of management by output factors Management of LO for students is uniformly managed by the universities with training management software and is rated at a high level (𝑋𝑗 = 3.09) Through discussions with teachers, the university needs to have better measures to manage the final exam by analyzing the questions and structure of the exam, analyzing the score spectrum to see if it has achieved the goal, outcome standars yet? From there, information is available to: i) adjust the objectives and outcomes of the course; ii) remove or edit bad questions to increase reliability and validity In addition, the teacher can also compare the students' ability with the difficulty of the questions in the exam, in order to adjust and supplement the questions to assess the students' ability; iii) help teachers change teaching methods and duration for knowledge and skills to achieve goals and outcomes 2.4.4 The impact of context on the management of the course assessment according to the learning outcomes Difficulties, challenges and impacts on the development direction of each university Specifically: - The economic development model changed from economic growth in breadth to depth, resulting in changes in the structure and quality of human resources of the national economy - Rapid changes in production, business and service technology will affect the employment structure - High quality requirements on skills and ability to work in an international environment of technical and technological human resources - There are no mechanisms and policies to encourage businesses to closely coordinate with universities in the process of training and recruiting workers; lack of policies to encourage the role of the private sector and professional associations in standardizing, monitoring and accrediting the quality of human resource training - Information on the labor market is limited Students not have enough information about recruitment needs and quality career counseling services Therefore, improving the quality of training requires a lot of effort, the determination of the LO and the assessment of the outcomes according to the outcomes needs to be innovated to equip students with technical skills that can be solved in a relatively short time, and improving soft skills requires long-term preparation, especially industrial style 2.5 Overall assessment of survey results 2.5.1 Summary of survey results Looking at the overall situation related to the testing management of the universities under the Ministry of Industry and Trade, it was found that: - None of the management elements were assessed at a weak or moderate level - There is a component that is evaluated at a good level, which is the awareness of the management staffs and teachers about the management of quality standards and the examination and assessment of the course, with the average value 𝑋𝑗 = 3.27 16 - There are three components of policy management, development management of exam data bank and management of output factors, which are assessed at the upper end of the good level, the average value 𝑋𝑗 = 3.24, 𝑋𝑗 = 3.01 and 𝑋𝑗 = 3.00 - The management elements are assessed at a fairly average level, belonging to the capacity of the teaching staff and the department in implimenting of the assessment work, the management of the assessment planning, and the financial management and invest in equiements, manage the process of assessment and manage the organization of the end-ofcourse exam 2.5.2 Comment on survey results a) Advantages - The system of regulatory documents on training organization and course assessment has been fully promulgated Activities related to testing and training quality assurance have been gradually concretized in terms of organization and operation in schools in the Ministry of Industry and Trade, contribute increasingly the operation of assessment management activities more and more is better transformation - Assessment activities of students' course results are continuously being renewed and improved gradually in the direction of focusing on the process (regular testing, periodical testing, mid-term, ) and diversifying courses tools and forms of assessment b) Restrictions - A part of teachers and students have not really paid enough attention to the course assessment activities according to the LO compared to its inherent role and importance - Policies and regulations on assessment of students have not yet met the actual conditions in training management and organization - The survey results showed that the practical shortcomings of the course management assessment activities are not really oriented towards capacity development for students Sub-conclusion of chapter The survey results also shown that there are still basic limitations in the management of the testing assessment of the course assessment according to the learning outcome: - The perception of a part of the teachers on the testing assessment of the study result according to the LO is not yet met in comparison with the role of the students its inherent role and importance - Managing and developing the exam data bank have not achieved the desired goal - The monitoring and inspection work have only stopped at the level of regular inspection / inspection, so the effectiveness of this work is not high - There is no unified management on methods and techniques to build LO for the curriculum, so assessment activities are not really a tool to adjust teaching and learning activities, and to maximize capacity development for learners - The work of summarizing assessment activities has only stopped at the level of classifying students' economic performance, not deeply analyzed the causes (advantages, bad reasons) why there are such results The research results in this Chapter, together with the arguments in Chapter 1, will be the basis for proposing scientific solutions presented in Chapter below 17 CHAPTER SOLUTIONS FOR ASSESSMENT MANAGEMENT OF STUDY RESULT OF COURSE ACCORDING TO LEARNING OUTCOMES AT UNIVERSITIES IN THE MINISTRY OF INDUSTRY AND TRADE 3.1 Perspectives and principles for proposing solutions to management of checking assessment according to learning outcomes 3.1.1 Proposed solution point of view Policy on education and training reform of the Party and State and Resolution 29NQ/TW dated November 4, 2013 of the Central Committee of the Communist Party of Vietnam, term XI Our country's economy now has a great need for high-quality human resources In order to overcome the existing weaknesses, it is necessary to have innovation in the organization and management of teaching and learning activities In particular, the innovation of assessment according to the criteria will make an important contribution to creating a change in the teaching style of teachers and the learning style of students with the goal of meeting the increasingly high quality requirements of human resources of the society 3.1.2 Principles of proposed solutions: Ensure objectivity; Ensure fairness; Ensure comprehensiveness; Ensure regularity and systematicity; Ensure efficiency; Guaranteed development 3.2 Solutions for assessment management of study result according to the learning outcomes 3.2.1 Solution 1: Organize awareness raising and capacity testing, assessment of study result for students, lecturers, experts and managers according to the learning outcome 3.2.1.1 Objectives and meaning of the solution: Improve the capacity of teachers, staffs and managers to help meet the requirements of team’s development and innovate asessment activities to achieve the LO 3.2.1.2 Content of solutions and implementation measures: i) Contents first: Organize awareness raising about the role of assessment for students, teachers, staffs, management staffs ii) The second content: Organizing capacity building for assessment of the course curriculum for teachers, staffs and management staffs 3.2.1.3 Implementation conditions: University needs to have a mechanism to motivate, encourage and set aside appropriate funds for training and retraining activities in the field of assessment 3.2.2 Solution 2: Organize the development of learning outcomes 3.2.2.1 Objectives and meaning of the solution: Build a set of learning outcome of the modules in line with the training curriculum, meeting the training needs of the labor market 18 3.2.2.2 Contents of solutions and measures for implementation: i) The first content: Assess the level of students' achievement of the course's LO ii) The second content: Mobilizing scientists, managers inside and outside the educational institution, stakeholders to participate in the development of LO iii) The third content: Reviewing, adjusting, supplementing and announcing learning outcome in accordance with social needs 3.2.2.3 Implementation conditions: Teachers need to understand the requirements and assessment methods suitable for each LO; Establishing a steering committee; make connections with stakeholders; financial planning for this activity 3.2.3 Solution 3: Develop of questions banks 3.2.3.1 Objectives and meaning of the solution: Develop a database of questions banks in order to standardize knowledge and skills to achieve results according to LO 3.2.3.2 Content of solutions and implementation measures: Every year, the questions banks bank must be regularly supplemented by faculties and departments, adjusted to suit the changes of the curriculum, training programs and student’s course 3.2.3.3 Implementation conditions: The university needs to mobilize a team of qualified teachers and managers with deep experience in professionalism on LO 3.2.4 Solution 4: Organize formative assessment of the learning outcome 3.2.4.1 Objectives and meaning of the solution: Standardize the stages, the teaching organization processes, the formative assessment to achieve according to LOs 3.2.4.2 Contents of solutions and measures for implementation i) Content 1: Manage inspection and assessment content ii) Content 2: Management of methods and forms of assessment 3.2.4.3 Implementation conditions: Teachers must master the methods and forms of assessment according to the criteria 3.2.5 Solution 5: Organize the summative assessment 3.2.5.1 Objectives and meaning of the solution: Provide information about the accumulation of knowledge, skills and attitudes after the end of the course and recognize academic achievements for students 3.2.5.2 Solution content and implementation measures: i) Management of exam questions ii) Manage the examination stage iii) Management of exam marking and announcement of results for students 3.2.5.3 Implementation conditions: The universities should have a policy to develop a database of exam questions to ensure reliability and validity; Information technology systems and training management software must be regularly maintained and upgraded 3.2.6 Solution 6: Organize the inspection, checking associated with the reward mode 3.2.6.1 Objectives and meaning of the solution: Warning, preventing possible errors and negatives, and contributing to improving the quality of training 19 3.2.6.2 Contents of solutions and implementation measures: i) The first content: Forming a network for inspection and control ii) The second content: Strengthening the inspection and examination for the organization of the end-of-course exam 3.2.6.3 Implementation conditions: Selection of members to participate in inspection and superviour teams must have good expertise and especially, must have a high sense of responsibility and professional ethics 3.2.7 Solution 7: Organize the evaluation of the implementation of the test plan and assess the course 3.2.7.1 Objectives and meaning of the solution: Assess the level of response to assessment activities to achieve the results, from that give measures to improve teaching methods and assessment methods 3.2.7.2 Contents of solutions and measures for implementation: i) The first content: Organizing and summarizing process assessment activities ii) The second content: Organize a summary of the end-of-course exam activities iii) The third content: Get students' feedback on the teacher's activities at the end of the course 3.2.7.3 Implementation conditions: The individual/department assigned to carry out the summarization work needs to have a good grasp of the process of developing the LO, the assessment method, and the requirements for innovation in assessment activities 3.3 Experiment proposed solutions 3.3.1 Organize experiments on the necessity level and feasibility of solutions 3.3.1.1 Experimented purpose The author conducts a survey to consider the level of assessment of the need and feasibility for teachers, managers and enterprises 3.3.1.2 Contents and methods for experiment Contents and methods of experiment are conducted by questionnaires, in which one set of questionnaires to assess the level of necessity and one set of questionnaires to evaluate the feasibility of each solution to manage the testing assessment of the course's performance according to LO The test has 03 levels: Level (the highest) has a value of 2.33 < 𝑋 ≤ 3.0; level has a value of 1.67< 𝑋 ≤ 2.33; level has a value of 𝑋 ≤1.67 3.3.1.3 Experimented subjects: teachers, managers, enterprises 3.3.1.4 Data processing tools: Calculate the weighted average and calculate the Spearman correlation coefficient to draw conclusions 3.3.2 Experimented results 3.3.2.1 Experimented results on the need for solutions The results of data analysis of 98 questionnaires shown that the average value of the solutions (𝑋 = 2.67) is within the range of level 3, showing that the solutions are considered essential Among the solutions, the solution “The organization raises awareness about the role of assessment; capacity to assess international relations for students, teachers, staffs, and management staff according to the criteria" has the highest necessary level (𝑋 = 2.79), which reflects the level of 20 influence and awareness of teachers and management staffs about the role of LO for students in general and assessment in particular And organizing capacity building for the testing team was assessed by 82.6% of management staffs at a very necessary level Solutions related to policies, regulations and procedures such as inspection work, management of end-of-course exams or assessment activities (process assessment, exams) are assessed at the lowest necessary level It also reflects the regulatory documents on assessment that have been issued quite fully by the universities, meeting the requirements of training and recognizing the assessment for students 3.3.2.2 Experimented results on feasibility of solutions The average value of the solutions (𝑋= 2.67) is within the range of level 3, shown that the solutions are evaluated as very feasible Among the solutions, the respondents said that "Organization for the development of course LO" with the assessment of students' achievement of the subject's LO; mobilize scientists, managers inside and outside the educational institution, stakeholders to participate in the development of LO; reviewing, adjusting, supplementing and announcing the output standars in accordance with social needs has an influence on the entire assessment management activities and has the highest feasibility (𝑋 = 2.73) The solution "Development of exam data bank" has the average score (𝑋 = 2.69) ranked second, with 70% of management staffs thinking that it is very feasible, which shows that schools need to focus on applying management solutions For this reason, it is a necessary condition for renovating assessment activities, especially process assessment 3.3.2.3 Experimental results on the correlation between solutions To consider the degree of correlation of solutions, the thesis uses the Spearman correlation coefficient calculation formula: ∑ 𝑑2 𝑟 =1− 𝑛 (𝑛 − 1) With r = 0.85 reflecting the proportional correlation between the degree of necessity and feasibility, and concurrently, it is a close relationship (r is close to 1) of the solutions, that is the solution with a high necessary, is highly feasible 3.4 Test the proposed solution 3.4.1 Test the first solution 3.4.1.1 Purpose, contents of experiments: Experimental purpose: Solution "Organize raising awareness and capacity for assessment of international relations for students, teachers, staffs, management staffs according to the LO." for the purpose of assessing the impact of management work to convert recognition of teachers and management staffs about the position and meaning of LO Experimental content: Management impact of university through fostering knowledge and equipping about the quality standards, the conception and methods of assessment according to the criteria, which will lead to a change in positive – oriented in capacity for teachers, who perform the assessment tasks during the training process 21 3.4.1.2 Test organization The testing is carried out through a process consisting of steps: Step 1: Select 75 teachers and 25 management staffs from the sections and some functional departments to conduct a survey on the level of understanding of LO, thinking and methods testing management for LO Step 2: Develop a set of questions (Appendix 2) and distribute the questionnaires to the survey subjects (1st time) Step 3: Collect the survey form and summarize the results according to the answer options and calculate the percentage of correct answers compared to the number of respondents Step 4: Prepare documents and send them to the respondents Step 5: Monitor and encourage survey subjects to self-study the documents (above) within 30 days or be sent to attend training and retraining courses on curriculum development, construction training programme, development of the quality assurance system inside the University, etc Step 6: Conduct a survey and summarize the results according to the answer options and calculate the percentage of correct answers compared to the number of correct answers respondents (2nd time) 3.4.1.3 Test results After distributing 100 survey questionnaires, the author collected 92 votes (Teacher 69 votes, Management staff 23 votes) The results show that: - Most of the surveyed subjects are well aware of the contents of the documents of the State management agencies related to the training organization and the assessment in particular; understand the concept and requirements of today's assessment in higher education in order to improve the capacity of learners after graduation Regarding quality assurance, through the test, it can be divided into two groups, the group of questions about awareness for the role and meaning of the LO affecting the assessment activities, which has a higher rate of correct answers than the group of questions about technical requirements, setting up LO (at all levels: curriculum, modules, and lessons) However, when there is an impact of management by orienting the goals, preparing documents related to the assessment activities according to the LO; motivate and encourage the human resources involved in assessment activities to conduct self-study, or send them to participate in training courses related to knowledge and skills of training curriculum development, quality assurance, pedagogical skills, etc they will be a change in awareness in a positive direction, which will be a premise for the design of objectives for testing assessments 3.4.2 Test the fourth solution 3.4.2.1 Purpose, experimental content Experimental purpose: Solution "Organize the process of assessment to achieve the quality." aims to test the impact of management will change the quality of training 22 Experimental content: Based on the above test results, the thesis continues to test the design of tools to manage process assessment activities and assess the influence on students' assessment LO performance 3.4.2.2 Test organization The solution testing is carried out through a process that includes the following steps: Step 1: Build a logical structure diagram for the management of the LO of the module Step 2: Coordinate with the Faculty of Information Technology, University of Economics - Industrial Engineering to design the LO, assessment activities, learning activities, for the Java Technology module according to the structure diagram in step Step 3: Organize experimental teaching for students of course 11, majoring in Information Technology, in the second semester of the 2019-2020 school year Step 4: Synthesize, compare and evaluate the results of the Java Technology module of the 11th class with the 10th class 3.4.1.3 Classification of Student's study result Classification of Student's Classification of Student's study result Course 10 (%) study result Course 11 (%) Scores M4 M3 M2 M1 M4 M3 M2 M1 Formative assessment 36,4 18,6 41,3 3,7 23,0 48,0 26,8 2,2 Summative assessment 24,8 16,9 41,7 16,5 33,8 40,9 19,0 6,3 Study result 17,4 21,9 50,4 10,3 26,8 41,3 28,3 3,7 Mean 6,2 7,1 Median 6,5 7,3 Mode 7,7 7,4 Table 3.5: Classification of Student's study result Tổng kết học phần 50,4 Thi học phần 41,3 41,7 28,3 M2(K11) M1(K10) M4(K10) M4(K11) M3(K10) M3(K11) M2(K10) 6,3 3,7 2,2 3,7 10,3 16,5 26,8 19 18,6 16,9 21,9 17,4 23 24,8 26,8 33,8 36,4 40,9 41,3 48 Quá trình M1(K11) Chart 2.3: Compare student study results So Compare the study results of students Through the data in table 3.5 and chart 3.2, it can be seen that: 23 - The study results of the course from the good level or higher (M3 and M4) of the 11th grade students are higher than the 10th grade students (61, 8% vs 39.3%); the percentage of students with weak and poor academic performance is lower (3.7% compared to 10.3%) - The percentage of process scores and exam scores of students of course 11 is similar to that of students of course 10, reflecting the process assessment that has timely adjusted teaching and learning methods to achieve the LO - Mean value (average score) of class 11 students is higher than class 10 students (7.1 versus 6.2); At the same time, the Mean, Median, and Mode values tend to be more concentrated, reflecting the Key 11's normal distribution more than the Key 10 Thus, the impact from the management of the "first standard development organization" learning course" through the logical structure diagram is feasible and the quality of training will be improved Sub-conclusion of Chapter In order to well manage the ASSESS according to the criteria to improve the quality of training, the universities need to implement 07 management solutions The test results shown that the management solutions have the necessary level, the high feasibility and positive correlation; As well as, the test results showed that if the University manages the assessment activities as proposed, the quality of training in general and assessment in particular will be improved CONCLUSIONS AND RECOMMENDATIONS CONCLUSION Management of the curriculum results according to the LO is a fairly new and effective approach in higher education in general and universities in the Ministry of Industry and Trade in particular, in order to contribute to improving the quality of human resources to meet the requirements of industrialization and modernization of the country One of the solutions to contribute to improving the quality of training identified by the universities is to well manage the performance assessment of the courses curriculum according to the LO of the training programs However, at present, this work has also revealed inadequacies and limitations such as: The awareness of a part of teachers about the assessment for the result’s learning according to the LO has not yet been met in comparison with its inherent role and importance; There has not been a unified management on methods and techniques for building LO for the curriculum, so assessment activities are not really a tool to adjust teaching and learning activities, and to maximize capacity development for learners From the theoretical basis and the current field survey, the thesis proposes 07 solutions to manage the assessment for the module in universities in the Ministry of Industry and Trade These solutions have an organic relationship with each other, complement each other, forming a unified and synchronous whole to improve the quality of training The test 24 and trial results shown that the management solutions have a high degree of necessity and feasibility and have a positive correlation, ensuring reliability and validity for the universities to apply and deploy RECOMMENDATION a For the Ministry of Education and Training - Coordinate with relevant ministries and branches to issue documents on human resource training to meet the needs of socio-economic development as well as regulations on training and assessment to meet the criteria - Periodically, coordinate with relevant agencies to review and issue minimum standard documents on capacity for training qualifications - Annually publicizing the jobs of students after graduation, the demand for job positions in each industry group/industry b For the Ministry of Industry and Trade and business enterprises: - Formulate, promulgate and publicize the development strategy of each branch according to the field of management so that the universities have a basis to adjust training programs and LO - Develop a policy to connect enterprises to participate in the process of training organization and assessment with universities in the management of the Ministry - Enterprises raise awareness, actively participate in contributing ideas to the draft LO of the training programs - Enterprises directly participate in the formative assessment to assess the capacity of "training products" to meet the requirements of use of the enterprises themselves c For schools under the Ministry of Industry and Trade - Determining the philosophy of the training program, the assessment and the teacher's strengths, this is very important to determine the strengths for each training program, as a basis for taking measures to manage the assessment program, to have taking into account the specific factors of each specific course - Periodically review internal and external references to adjust and announce the LO of the program and the module - Implement synchronously the solutions to manage the assessemt of the course according to the objectives proposed by the thesis - Annually organize training courses, update knowledge about quality assurance, course assessment skills for teachers 25 LIST OF SCIENTIFIC WORKS OF THE AUTHOR RELATED TO THE THESIS Duong The Viet (2014), Training management of output standards at the Economics - Industrial Engineering University, Journal of Education Management, No 61, June 2014 (pp 26 - 29) Duong Thi Nhu Hien, Duong The Viet (2017), How to enhance instrumental motivation for in-service learners, Proceedings of the third national conference on interdisciplinary research on language and language teaching, Hue, November 2017 (pp 511-522) Duong The Viet (2020), Testing and assessment management for modules based on the learning outcome standard at universities National, academy of education management, No 7, July 2020 (pp 85 - 91) Duong The Viet (2020), Solutions for managing outcome - based assessment and testing of modules National, academy of education management, No 10, October 2020 (pp 47 - 54) Duong The Viet (2021), The situation of inspection management, learning outcomes assessment through Outcome standard of Universities of Ministry of Industry and Trade HNUE Journal of science, Issue (pp 147 - 158) ... simultaneously, confirm the study result to prepare for the next learning phase 1.3 Kiểm tra, đánh giá kết học tập học phần theo CĐR 1.3 Assessment for study results of the course according learning outcomes... training program, which is assessed through measurement with evaluation criteria and expressed by score; iii) Through, it will reflect the quality of programs 1.2.3 Kiểm tra, đánh giá kết học tập 1.2.3... and encourage survey subjects to self-study the documents (above) within 30 days or be sent to attend training and retraining courses on curriculum development, construction training programme,