Guidelines for AUN-QA Assessment and Assessors Guidelines for AUN Quality Assessment and Assessors is published by: AUN Secretariat Room 210, Jamjuree Bldg Chulalongkorn University Phayathai Road, Bangkok 10330 Thailand © ASEAN University Network (AUN) All rights reserved No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording and/or otherwise without the prior written permission of the Executive Director, ASEAN University Network Version No 2.0 Date of Publication: February 2013 Acknowledgement The content in this handbook is prepared by Mr Johnson Ong Chee Bin, AUNQA Expert with contribution from Assoc Prof Tan Kay Chuan, AUN-QA CQO from National University of Singapore Table of Contents Foreword 1.0 1.1 1.2 1.3 1.3.1 1.3.2 1.3.3 1.3.4 1.3.5 1.3.6 1.3.7 1.3.8 Introduction to AUN-QA Network About AUN-QA Objectives of AUN-QA Structure of AUN-QA Functions of the Board of Trustees Functions of the QA Council Functions of the Executive Director Roles of the Participating Universities Functions of the Secretariat Panel of AUN Assessors Structure and Roles of Assessors Code of Ethical Conduct for Assessors 2 3 4 13 2.0 2.1 2.2 2.2.1 2.2.2 2.2.3 2.3 2.3.1 2.3.2 2.4 2.5 Quality Assessment Process The PDCA Process Plan Phase Types of Assessment Assessment Team Schedule and Itinerary Do Phase Desktop Assessment Site Assessment Check Phase Act Phase 14 14 17 17 18 19 24 24 25 30 34 3.0 3.1 3.2 3.3 3.4 3.5 3.6 Quality Assessment Requirements Eligibility for Assessment Requirements for Assessment Requirements for Self-Assessment Report (SAR) Requirements for Stakeholders’ Interviews Requirements for Logistics and Accommodation Requirements for Permitting Observers 32 32 32 33 34 35 35 4.0 4.1 4.2 4.3 Assessment Scheme and Fees Assessment Scheme Use of AUN-QA Logo and Certificates Assessment Fees 41 41 41 42 Foreword The handbook documents the operating guidelines for AUN quality assessment It sets out the roles, responsibilities and requirements of the applying university (assessee), assessors, observers and staff of AUN Secretariat The guidelines aim to provide a common frame of reference and accountability to the AUN quality assessment process and stakeholders They are formulated based on Section 3.8 of the Standards and Guidelines for Quality Assurance in the European Higher Education Area, Third Edition, 2009 The European standards and guidelines for external quality agencies on accountability procedures are reproduced below A published policy for the assurance of the quality of the agency itself, made available on its website; Documentation which demonstrates that: the agency’s processes and results reflect its mission and goals of quality assurance; the agency has in place, and enforces, a no-conflict-of-interest mechanism in the work of its external experts; the agency has reliable mechanisms that ensure the quality of any activities and material produced by subcontractors, if some or all of the elements in its quality assurance procedure are subcontracted to other parties; the agency has in place internal quality assurance procedures which include an internal feedback mechanism (i.e means to collect feedback from its own staff and council/board); an internal reflection mechanism (i.e means to react to internal and external recommendations for improvement); and an external feedback mechanism (i.e means to collect feedback from experts and reviewed institutions for future development) in order to inform and underpin its own development and improvement A mandatory cyclical external review of the agency’s activities at least once every five years The publication of this handbook is the first step towards fulfilling the above guidelines and satisfying all of them is expected to be evolved over time Feedback on the content of the handbook is welcome and it should be sent to the Executive Director, AUN Secretariat Assoc Prof Nantana Gajaseni, Ph.D Executive Director ASEAN University Network 1.0 Introduction to AUN-QA Network 1.1 About AUN-QA ASEAN University Network-Quality Assurance (AUN-QA) network is a group of Chief Quality Officers (CQOs) appointed by the AUN member universities as the focal point for coordinating activities to realise the mission of harmonising educational standards and seeking continuous improvement of academic quality of universities in ASEAN The AUN-QA activities are carried out by the CQOs in accordance to the Bangkok Accord adopted in 2000, which provides a series of guidelines to promote the development of a quality assurance system as instruments for maintaining, improving and enhancing teaching, research and the overall academic standards of AUN member universities Since the establishment of the Bangkok Accord in 2000, AUN-QA has been actively promoting, developing, and implementing quality assurance practices based on an empirical approach where quality assurance practices are shared, tested, evaluated, and improved The AUN-QA timeline is depicted below 1.2 Objectives of AUN-QA AUN-QA is established under the auspices of the ASEAN University Network (AUN), and is the de facto regional quality assurance agency for higher education in ASEAN The main objectives of AUN-QA are: Develop, promote, implement and enhance the AUN-QA Guidelines and criteria Facilitate and conduct AUN quality assessment Serve as the authority for issuing AUN quality labels Develop and train quality assurance professionals and practitioners Provide advisory and consulting services on best quality assurance practices Collaborate with other quality agencies inside and outside ASEAN in relation to harmonisation of quality assurance frameworks and development of professionals in quality assurance 1.3 Structure of AUN-QA The AUN-QA network consists of all Participating Universities, a Quality Assurance (QA) Council, a Secretariat and a pool of assessors, all of whom are governed by the Board of Trustees of the AUN The organisation of AUN-QA network is documented below 1.3.1 Functions of the Board of Trustees (BOT) The AUN-BOT, as the highest policy making body of the network, shall have the additional responsibilities necessary to carry out the objectives of the AUN-QA Formulate policies for the operation of the AUN-QA network; Approve proposals for projects, including budget allocation, work programmes and implementation activities; Approve the functions of the QA Council and the appointment of its members; Conduct periodic review and evaluation of on-going projects and activities based on the recommendations and assessment of the Executive Director; and, Perform such other functions that may be necessary to carry out the objectives of the AUN 1.3.2 Functions of the QA Council The QA Council shall have the following functions: Formulate, govern and review directions and policies for the operation of the AUN-QA network; Formulate and review guidelines, criteria, and documentation for AUN-QA models, assessment process, assessor’s appointment, and certification scheme and fees; Appoint AUN assessors and develop the system for training and certifying them; Endorse assessment reports and certification status; and Revoke certification status of university if it fails to honour and fulfill its public and social duties, undertakings and obligations to its stakeholders including AUN-QA network The QA Council will be headed by the Executive Director with current Lead Assessors and AUN-QA Expert (invited by the AUN Secretariat) The term of the council members will be years so as to allow the phase-in of new members The council shall meet on a need-to basis via formal and informal channels such as tele-conferencing, skype, emails, etc 1.3.3 Functions of the Executive Director The Executive Director shall have the following functions relating to AUN-QA: Manage the activities of the AUN Secretariat and QA Council; Coordinate and monitor the AUN-QA programmes of the participating universities; Establish and maintain links with other quality assurance bodies within and outside ASEAN; Perform the role of a Registrar of Assessors who is responsible for the development of a pool of qualified AUN assessors, assignment and performance of assessors, and handling of feedback and complaints of assessors; Engage a third party to carry out an external review of the agency’s QA activities at least once every five years; Propose and develop ideas, innovations or mechanisms for sourcing and generating funds for AUN-QA activities; and Perform such other functions which may be assigned by the Board of Trustees from time to time The Executive Director must have an outstanding record in managing or directing international programmes and cooperation projects which are recognised in the region He/she shall be appointed by the AUN BOT for a fixed term of (four) years, renewable for another term 1.3.4 Roles of the Participating Universities The Participating Universities shall support and participate in AUN-QA programmes and activities They shall appoint CQOs for their universities and nominate suitable candidates to be trained and performed as AUN assessors 1.3.5 Functions of the Secretariat The Secretariat shall be headed by an Executive Director and have the following functions relating to AUN-QA: Plan and organise AUN-QA quality assessment, workshop, programmes, projects and activities; Coordinate and monitor AUN-QA quality assessment process, programmes, projects and activities; Support the AUN assessors before, during and after the quality assessment; Maintain and update AUN-QA documentations; Maintain and update the register of AUN assessors and CQOs; and Ensure the implementation of the policies and directives of the AUN BOT and QA Council 1.3.6 Panel of AUN Assessors AUN-QA has established a pool of CQOs and professionals who serve as panel of AUN assessors in actual quality assessment From time to time, the AUN-QA Secretariat will invite or request AUN member universities to nominate CQOs and staff to be considered for appointment to the panel of AUN assessors Suitably qualified candidates are requested to submit curriculum vitae and the names of two referees In selecting the assessor, consideration s h o u l d be given to the assessor’s competence and personal attributes in addition to education and work experience The minimum requirements of an assessor to be appointed to the panel of AUN Assessors are listed below Education and Experience A recognised university education years of experience in quality assurance or quality assessment in educational, professional or industrial setting Knowledge and understanding of the higher education Personal Attributes The assessor should posses a number of personal attributes that contribute to the successful performance of quality assessment An assessor should be ethical, open minded, diplomatic, observant, perceptive, versatile, tenacious, decisive and self-reliant He/she should fulfilled all obligations assigned as an assessor and has not received any adverse report from the university being assessed, peers or AUN Secretariat AUN Assessor’sTraining AUN-QA requires assessors to complete a 2-day AUN-QA Assessor Training Workshop and observe at least one actual quality assessment at their own expense as a condition of their appointment The topics in the 2-day training workshop include: Introduction to quality in education History and development of AUN-QA The AUN-QA models and their criteria 36 They must submit a request stating the purpose of the observation and a short biographical statement to the AUN Executive Director at least months before the site assessment; They are required to formally declare that they have no conflicts of interest with the university being assessed; They are not allowed to record the observed session using any electronic devices; They will not be allowed to observe any private discussion held by the assessment team; They may be excluded from any meeting or interview by the chief/lead assessor and/or AUN Secretariat at their discretion; They must remain silent throughout the site assessment and interviews; They are responsible for their own accommodation and travel; and They may be invited to provide comments about the assessment process by AUN Secretariat 37 4.1 Assessment Scheme and Fees Assessment Scheme The certification which will be provided after the assessment period by AUN Secretariat is valid for years only 4.2 Use of AUN-QA Logo and Certificates University qualified with any of the above certification status can use the relevant AUN-QA logo and certificates within the validity period The AUN-QA Logo and Certificates can only be used on corporate materials, brochures, publicity materials, website and premises of the awarded university Otherwise, written permission must be sought from AUN Secretariat The logo and certificate must be used as one entity and no distortions or modifications are allowed Below is an example of the AUN-QA Certificate 38 4.3 Assessment Fees To maintain the sustainability of the AUN-QA system, it is imperative that the adequate funding and financial support be secured with consistency Assessment fee schemes The revised assessment fee schemes for AUN Member Institutions undergoing AUNQA Assessment are as follows: Item Member Associate Member Honorarium for Lead Assessors USD 600 per programme (borne by host) USD 600 per programme (borne by host) Honorarium for Assessors USD 300 per programme (borne by host) USD 300 per programme (borne by host) Airfare for Assessors and two AUN Secretariats Borne by host(economy class) Borne by host(economy class) Accommodation for Assessors and two AUN Secretariats Borne by host Borne by host Administrative Fee toAUN Secretariats USD 1,000 per assessment (borne by host) USD 1,000 per assessment (borne by host) Local Hospitality (meals and transportation) Borne by host Borne by host Development fund None USD 1,000 per programme (borne by host) In case of the non-Member, the service fee shall be applied AUN-QA Development Fund The AUN-QA Development Fund should be formally established with TOR and regulations The development fund can be collected from the following sources: - The AUN-QA Assessment for Associate Member 39 - 10% of the revenue generated under the AUN-QA activities will be deposited - into the development fund Potential funding agencies such as private sector, international organization, international funding agencies, etc 40 Framework of AUN-QA Strategic Action Plan 2012-2015 41 Table of Contents Objective/components of the framework page Components of the framework a The core component i The QA Assessment ii Capacity Building b The support component i Expansion of AUN-QA implementation ii Funding and finances Framework of AUN-QA Strategic Action Plan (2012-2015) The AUN-QA Workshop and CQOs’ Meeting 2012 was organized by the AUN Secretariat on 5-6 January 2012 in Chiang Mai, Thailand The workshop was attended by 24 Chief Quality Officers and representatives from AUN Member Universities in ASEAN Member States except Brunei Darussalam and Myanmar Objective of the framework The objective of the framework of AUN-QA Strategic Action Plan is to set the outline of strategic plan including thekey activities, in order to further develop the AUN-QA system through the enhancement of QA assessment and capacity building among the AUN Member Universities and also extend to non-Member in ASEAN region; which could lead to the acquisition of regional and international recognitions Components of the framework The framework consists of two components, the Core and the Support component The core component has two strands; 1.) The QA assessment, and 2.) The capacity building On the other hand, the support component is composed of 1.) Peopleplan, and 2.) The funding and finance The support component will serve as assisting tools for the core component to mature, and ultimately fulfill the framework’s objective The Core component 1.) The QA Assessment The AUN-QA sees the need for and the value of infusing new ideas and providing feedback towards improving AUN-QA instruments and processes As such, the idea of inviting guests or peer assessors from outside the AUN-QA will be pursued as part of the AUN-QA’s future plan 1|Page Peer Assessment The peer assessment will be in a form of collaboration with external assessors The external assessors will take on observer status during the AUN-QA programme assessment and provide feedback on the assessment process Likewise, the AUN-QA Assessors are encouraged to participate as the observer during the assessment of the partner institutions/agencies The following conditions will be applied during the AUN-QA peer assessment: - External assessors must understand the spirit and philosophy of AUN-QA assessment and abide by it The external assessors shall abide by the guideline used by the AUN-QA Assessors (code of conduct) Lead assessor must be AUN-QA member The AUN-QA Assessment at the institutional level The AUN-QA Assessment at programme level will continue to be pursued and that the experiences of the AUN-QA Assessors be further developed and cultivated so as to efficiently initiate the AUN-QA assessment at the institutional level within this framework’s timeline (2012-2015) 2.) The capacity building Regionalization and internationalization of the AUN-QA To uplift the regional and international recognition on the AUN-QA system, a bottom-up approach should be put into practice The rationale for this approach is identified as follows: How to enhance capacity building? Increase the number and capacity of lead assessors and assessors Improving the assessment's efficiency Acquire regional and international recognition Increase the number and capacity of lead assessors and assessors External assessors herein are referred to assessors from non AUN-QA Member 2|Page Increasing the potential to cooperate with other QA agencies around the globe The capacity building of the AUN-QA Assessors focuses on the increase in the number of both lead assessors and the assessors The following key activities should be taken into consideration: - The series of training for trainers - Trainees shall be mobilized to conduct assessment - The existing assessors should also be encouraged to attend the training periodically - The CQOs are still eligible to conduct assessment after the retirement However, priority should be given to personnel in service - The compilation of the list of QA experts recommended by the Member Universities, the QA experts should be identified to create the pool of QA experts that helps improve Member Universities’ QA system and practices The QA experts can be independent from the university - Formal certification for assessors and the assessor development plan should be practically implemented Improving the assessment's efficiency The following has been recommended: - Clarifying questions pertaining to the SAR, sent to the university prior to the visit - The AUN-QA certificate is valid for years and the universities shall provide and update of their programme for renewal of the certificate - AUN-QA Council shall be established and part of its function is to approve the report of the assessments - Compilation of ‘best practices’ database - AUN-QA Web-based information management system shall be established in order to improve AUN-QA capacity building The system should be composed of: o Call for invitation o Appointment of assessors Cultural and national practices such as religious activities should be taken into consideration o SAR submission (applicant) o Evidence submission (applicant) o Listing of assessors/ universities o Schedules of assessments o Benchmarking, best practices More details operation for assessments o Tracking updates of AUN-QA o Feedback to assessors/ AUN 3|Page o The posting of QA analysis on AUN website Funding for capacity building The ultimate goal of the AUN-QA capacity building is to strengthen the AUN-QA system and the AUN-QA Assessment This will in turn improve the quality education of the assessed university to produce graduates that will be part of skilled/knowledgeable human resources in the ASEAN region In order to strengthen the human resource development for AUN-QA, it is necessary to seek the funding support from Dialogue partners, funding agencies, national QA agencies, Ministry of Education, etc Acquisition of regional and international recognition With a more efficient AUN-QA Assessment system, it could pave the way for the AUNQA to gain recognition region-wide, and will also extends to the international level at the next stage The following implementations should be established and practiced: - Conduct research that concerns “the impacts of the AUN-QA towards the quality improvement” The evidences and indicators that reflect the improvement of the assessed programme should be addressed and consolidated in order to reflect the efficiency of the AUN-QA Assessment This in turns will foster the recognition of the AUN-QA worldwide - Conduct thematic workshops/conferences to share strengths and improving weaknesses - Building the cooperation with international QA associations/agencies to improve the education in the region - Engaging and sharing resources with other QA networks/associations for recognition to establish reputable AUN-QA standard Increasing the potential to cooperate with other QA agencies around the globe AUN-QA is a regional educational body that helpshigher education institutions develop, improve, promote, and benchmark their internal QA in order for them toattain accreditation The AUN-QA also endeavors to extend its assistance to other universities in the region.To increase the potentiality of the AUN-QA to cooperate with other global QA agencies, there are two levels of action plans that need to be set 4|Page Extension of the cooperation - Extend the cooperation to QA agencies in Japan, Republic of Korea, PR China, and Australia, New Zealand - Explore cooperation with other international bodies like OIC, South American Organisation, etc - Cooperate with international QA agencies such as ENQA, INQAAHE, APQN, AQAN, ACCSP (accreditation body) AUN-QA code of conduct - Extending the invitation to observers for each other’s meeting - The Guide to AUN-QA, including activities and the role of AUN-QA should be recognized by SEAMEO RIHED, UNESCO Global QA Forum, national accreditation body and other professional agencies such as FEIAP dialogues - Involving the higher education administrators to have support More regional MOU The IQA should be initially accomplished before seeking for EQA AUN-QA shallnot compete with other accreditation bodies The Support component 1.) Expansion of AUN-QA Implementation The AUN-QA system is value-added in nature and should be implemented in a wider perspective As such, the expansion of membership should be introduced, although the growth in membership should be carefully managed so not to undermine the efficiency and the quality of the AUN-QA projects and activities The CQOs have agreed to extend the AUNQA Network to Associate Member; provided below are the membership modality and fee scheme: - Modality o The applicant institution will need to be recognized by the national QA agencies or the relevant Ministry o The applicant institution’s final approval should be addressed by AUN-QA council - Fee scheme o Member - will be exempted from annual fee Member is referred to AUN Member Universities 5|Page o Associate Member , the annual fee will be USD 500 per year provided in accordance with calendar year 2.) Funding and Finance To maintain the sustainability of the AUN-QA system, it is imperative that the adequate funding and financial support be secured with consistency Assessment fee schemes The Meeting agreed to the revised assessment fee schemes for AUN Member Institutions undergoing AUN-QA Assessment In addition, the Meeting agreed on fees for Associate Member as follows: Item Member Associate Member Honorarium for Lead Assessors USD 600 per programme (borne by host) USD 600 per programme (borne by host) Honorarium for Assessors USD 300 per programme (borne by host) USD 300 per programme (borne by host) Airfare for Assessors and two AUN Secretariats Borne by host(economy class) Borne by host(economy class) Accommodation for Assessors and two AUN Secretariats Borne by host Borne by host Administrative Fee toAUN Secretariats USD 1,000 per assessment (borne by host) USD 1,000 per assessment (borne by host) Associate Member is referred to non AUN-Member Universities 6|Page Local Hospitality Development fund (meals and transportation) None Borne by host Borne by host USD 1,000 per programme (borne by host) In case of the non-Member, the service fee shall be applied AUN-QA Development Fund The Meeting also agreed that the AUN-QA Development Fund should be formally established with TOR and regulations The development fund can be collected from the following sources: - The AUN-QA Assessment for Associate Member - 10% of the revenue generated under the AUN-QA activities will be deposited into the development fund - Potential funding agencies such as private sector, international organization, international funding agencies, etc The Meeting agreed to the Framework of AUN-QA Strategic Action Plan 2012-2015, adopted inChiang Mai, Thailand on January 2012 7|Page 8|Page