Institutional audit: a guide for student representatives © The Quality Assurance Agency for Higher Education 2009 Southgate House, Southgate Street, Gloucester GL1 1UB ISBN 978 1 84979 052 9 All QAA's publications are available from our website www.qaa.ac.uk Printed copies of current publications are available from: Linney Direct, Adamsway, Mansfield NG18 4FN Tel 01623 450788 Fax 01623 450481 Email qaa@linney.com Registered charity numbers 1062746 and SC037786 What is Institutional audit? Institutional audit is a process used by the Quality Assurance Agency for Higher Education (QAA) to check how well higher education institutions in England and Northern Ireland manage and maintain their academic standards and quality. Students and their learning are a central focus of this process. What are 'academic standards'? Academic standards describe the level of achievement that a student has to reach to gain an academic award (for example, a degree). It is expected that the threshold for achievement is the same across the UK for similar awards. So to be awarded a bachelor's degree in physics from X University should demand the same standard of achievement as a bachelor's degree in physics from Y University. Standards are agreed nationally and set out in frameworks for higher education qualifications, published by QAA. 1 What is 'academic quality'? Academic quality describes how effective the learning opportunities available to students are in helping them to achieve their award. This includes the quality of teaching, academic support, and resources such as the library and IT. Expectations for what constitutes a quality student experience are set out in the Code of practice for the assurance of academic quality and standards in higher education, published by QAA. 2 Institutional audit aims to report to the public about whether universities and colleges of higher education have effective means of: ensuring that the qualifications they award are of the standards referred to in The framework for higher education qualifications in England, Wales and Northern Ireland providing and managing learning opportunities for students to help them achieve their award through appropriate and effective teaching, support and assessment enhancing the quality of educational provision, making appropriate use of information gained through monitoring, internal and external reviews, and feedback from stakeholders. 1 'Students have a crucial role to play in the Institutional audit process. Who better to tell you about the quality of the student experience at an institution than the students themselves?' Gary Hughes, Student Engagement Officer, Manchester Metropolitan University Students' Union 2008-09 1 www.qaa.ac.uk/academicinfrastructure/FHEQ 2 www.qaa.ac.uk/academicinfrastructure/codeofpractice Who are the members of the Institutional audit team? Audit team members are senior people from other higher education institutions. The team will normally include five or six auditors (including one student) and an audit secretary. The audit team Four or five senior academics One student One senior higher education administrator (as audit secretary) They will be supported by a member of QAA staff, usually an Assistant Director, who is available to give advice to the institution and student representatives on their preparation for audit. What does an Institutional audit look at? Institutional audit usually focuses on seven areas: In all of these areas, the audit is interested in how the institution manages its learning and teaching activities. Audit wants to find out how the institution knows that it is doing a good job, how it knows when things go wrong, and how it knows what to do to put them right. Auditors understand that not everything in an institution might be perfect, and that providing imaginative and innovative teaching can be challenging. But they will expect an institution to be aware of this and to be taking effective steps to manage risk. 2 Introduction and background (to the institution) Management of academic standards Approach to quality enhancement Collaborative arrangements Published information Institutional arrangements for postgraduate research students Management of academic quality How can you get involved? There are several ways in which student representatives can get involved in the audit process: Student written submission (SWS) QAA invites student representatives to submit a report from the student perspective, which focuses on the institution's academic quality and standards. QAA advises student representatives to structure the SWS around four questions: How accurate is the information that the institution publishes? Do students know what is expected of them? What is the student experience as a learner like? Do students have a voice in the institution and is it listened to? Meeting with the audit team During at least one of its visits to the institution the audit team will want to meet with key students' union representatives (usually the authors of the SWS). During the briefing visit they will want to ask questions about the SWS and any further questions they have about the institution. This is an opportunity for students to highlight any academic issues that are important to the student body. The audit team will also want to meet a wider range of students during the audit visit. The students they meet will depend on the areas they have decided to explore in detail (for example, postgraduate issues, distance learning, widening participation and student support.) Working with the institution Student representatives are encouraged to work with the institution throughout the audit process. The institution may well have a large amount of evidence that student representatives can use when writing the SWS, such as survey results. One of the advantages of being honest and open with the university from the beginning is that you can highlight areas for development that need to be addressed. This way, the university can work with the students' union afterwards to change things for the better and help spread good practice. Action planning The auditors write a report summarising their conclusions about the institution. Once a draft report has been agreed, this will be sent to the institution for comment. Student representatives can ask your institution for a copy of this to help them check through for factual accuracy. Once the report is finalised, the institution is invited to submit a statement of no more than 500 words which will appear alongside the published report. This is a good opportunity to work with your institution to communicate with the student body, and also to start planning how they can tackle any challenges highlighted and spread good practice. 3 Who is your main contact? The main point of contact for student representatives at QAA is the Assistant Director (AD) who is co-ordinating the audit. The AD is a senior member of QAA who has experience in co-ordinating audits and other review activities. He or she will have worked with a variety of institutions and their students. The AD will arrange a day with the institution when they will meet key people to explain the audit process and how different people will be involved. The AD will want to meet students' union officers at this meeting, so make sure that you know when it is happening. You can also arrange to have a separate meeting with the AD on that day. If you don't get a chance to meet the AD at the preliminary meeting (it may be that this meeting took place before you started as an officer), your institution will be able to give you their contact details and they will be happy to talk to you by telephone or email. Another important contact will be your institutional contact - the person at your institution who is responsible for co-ordinating the preparations for audit (maybe the Director of Quality, Pro Vice-Chancellor (Learning and Teaching) or Head of Quality and Standards). They are a vital contact for student representatives because they will be able to tell you what is going on at any particular point in the process and share the initial and draft conclusions with you. 4 Preparing the student written submission What is the student written submission? The student written submission (SWS) is one of a number of key documents that help to give auditors a better understanding of your institution. It will really help the audit team if it uses evidence of student opinion and experience to draw auditors' attention to how your institution's efforts to assure academic quality and standards are impacting on students. Your institution will also produce a document - the institutional briefing paper (IBP) - which will describe the mechanisms they use to assure quality and standards. The IBP will try to show auditors what policies and procedures are in place, why these are the most appropriate systems, and how their success is measured. The SWS should help auditors to understand students' experiences of these policies and procedures. It is an opportunity to provide critical reflection, highlighting strengths and weaknesses that can guide auditors to reach well-informed conclusions about your institution. The SWS is independently produced by students at your institution, normally through the students' union. The only 'rule' about the SWS is that it must only report on academic quality and standards. It should focus on the policies and procedures related to learning because this is the only area of your institution's work that QAA is able to investigate. Example If you have evidence which shows that students are generally dissatisfied with sports facilities at your institution, this is probably more useful for internal campaigning than the SWS. But, if your institution runs courses which rely on these services, such as sports science or coaching degrees, feedback from these students could help auditors see how your institution manages facilities that help students to succeed; this could be useful information but you shouldn't let it distract you from matters that will affect the learning experience of the majority of students. The SWS should be as long as you need it to be but as concise as you can make it. Auditors will find a short report that focuses really well on a few aspects of the learning experience more useful than a large report that touches all topics, but with little evidence or analysis. As a guide, the SWS (including references) should be no longer than 12-15 pages of A4. Each SWS is unique; you decide what to tell the auditors. QAA does not prescribe exactly what you should include but you might find it helpful to frame your research around suggested key areas. Examples of approaches to developing the SWS can be found on QAA's website. 3 5 'The student written submission provided an ideal opportunity to alert my university to key issues that students had raised.' Mohammed Surve, Education Officer, University of Warwick Students' Union 2008-09 3 www.qaa.ac.uk/reviews/institutionalaudit/casestudies What should be in the student written submission? Auditors will read anything related to students' experiences of academic quality and standards that you include in the SWS, however many students' unions have based their SWS around four questions: 1 Is the information that the institution publishes about itself accurate? Auditors will want to know whether your institution is well managed and meeting its commitments to students, or whether it is 'over-promising' and needs to focus on improving provision. It will be very useful for them to have examples of key documents, accompanied by student opinion of their usefulness and accuracy. The large number of key documents produced by your institution means that you will not be able to research student opinions on all of them! Select the documents you believe are most important to the learning experience of students and think about any evidence you have that shows what students think about their accuracy and value, and whether you need or want any more. Examples The National Student Survey (NSS) asks questions about whether students are satisfied with their timetable: timetables are vital documents and if students have particular comments about disruptions to their teaching or appreciate allowances for evening or part-time study, this will be useful information for auditors. You may have a university/student charter that describes a commitment to library and computer resources: some institutions participate in the Postgraduate Research Experience Surveys, which will show whether a certain group of students agree that this commitment is met. You might also consider looking at your institution's prospectus, website or course booklets. Auditors will welcome evidence of satisfaction with any publication that influences students ' expectations about what their education will offer or consist of. 2 Do students know what is expected of them in order to do well? Auditors will be interested to know what induction, training and information your institution gives students at each stage of their programme. What you say will help the auditors judge whether the institution is providing adequate guidance and support so that students have a fair chance to succeed, or whether students are unable to meet standards of performance because of lack of knowledge about academic expectations and the resources available. Auditors will be interested in whatever perspective you choose to give them. Whether you focus on induction, assessment criteria, the prompt return of assessed work with feedback or the quality of feedback, it will all be useful as long as you can support your comments with evidence. Examples Your institution is likely to set students multiple assessment tasks during their course. QAA expects institutions to tell students the criteria against which their work will be judged in advance, and there is a question in the NSS that directly addresses this issue. If you can find out why certain groups of students agree or disagree with this question, perhaps by holding focus groups, auditors will be interested in this. Course reps could be a valuable source of information here: you could ask them, as an interested group of students, for examples of good and bad feedback. If there is a pattern, for example of students wanting feedback that includes an indication of other sources that could help them learn, then auditors will be interested to know if this is happening at your institution and whether it is useful. 6 3 What is the student experience like at the institution, including teaching and learning opportunities, support received and access to learning facilities? The SWS will be particularly useful for auditors if it gives them an honest opinion about what it is actually like to study at your institution: Are lectures a useful format for learning? Are lecturers good at explaining their subject? Are there enough reading list books in the library? Are the art studios good enough for all of the students that want to use them? Does the personal tutor system do everything it promises? These sorts of questions will help aid the auditors' understanding of whether the way the institution manages teaching and learning is appropriate and helps students to learn effectively. It is important to note that this could be a huge topic to cover. The questions above are only examples of what you could look at, but they could easily be the subject of an SWS on their own. It is a good idea to select issues that are important to students at your institution and explore them in some depth. These could come from your students' union officer manifestos or internal committee minutes, or your institution's NSS or internal satisfaction survey results. Examples Almost all students will attend some teaching sessions, whether a lecture or smaller group. Course reps will be uniquely placed to give an all-round perspective on the condition of the teaching venues, whether the sessions are linked to the course guidelines, how good the academic staff are at explaining things, and all other aspects of academic quality. Analysis of their opinions will be useful to auditors investigating the student learning experience at your institution. 4 Do students have a voice, and how do you know that anyone listens to it? Auditors will want to know not just about the design of your formal representation systems, but also your judgement of how effective it is: What impact do students have on the management of academic standards and quality? Your SWS shouldn't just describe what committees students sit on, it should also reflect on the changes that have happened as a result. It will also be useful to discuss the informal relationships that exist between students and the institution. Sometimes, change for students comes from conversations with teaching staff, supervisors or senior staff that aren't formally written down. Auditors will want to know how these sorts of contributions are handled, recorded and good practice shared. They will use this information to form an opinion on how the institution uses the views of learners to assess the effectiveness of their management of academic quality and standards. Examples Students' union volunteers and officers will almost certainly sit on university committees, ranging from programme committees or boards of study to boards of governing (or equivalent). If you are able to use reference to the minutes of these meetings to show that students are active participants in quality management and their contributions are responded to or acted upon (or not), auditors will be interested in your views. If you are able to show that your institution took the evidence of one of your campaigns to improve the quality of learning, and you can obtain quotes from colleagues or demonstrate the response (at whatever level), this will also show how the student voice features in your institution's management processes. 7 Where to find the evidence? Auditors will expect all statements, conclusions and recommendations in your SWS to be accurate interpretations from solid evidence. Some evidence will be external - such as the NSS, the Postgraduate Experience Survey, the newly qualified teacher survey or the International Student Barometer. These are fine and credible sources of evidence that auditors will be familiar with. Some evidence will be internal - such as student satisfaction surveys, course rep information and the minutes of committees. These sources are also fine, and are likely to cover even more of the areas of the learning experience at your institution in greater depth. With the wide range of information already available to you, you might decide that it is not necessary to do any new research. If there is already evidence of the opinions of students at your institution, repeating research is unlikely to be the best use of your time. You will most likely find that exploring a few areas in more depth with the students affected will be more useful. Whatever sources of evidence you use, it is essential that you're clear and honest about how reliable they are. In your SWS, probably in the section where you explain your methodology, you should describe the range, sample and scope of your evidence: What sorts of students were asked to participate? How many students were asked to participate? Who and how many did? What were they asked? This will help auditors decide whether they need to ask for more information to understand your point or whether your evidence is enough. Does the SWS have to be shared with the institution? The SWS can be kept confidential from your institution if you want it to be. However, if it has not been shared with the institution this will mean that it is harder for the audit team to use it during the audit - they will not be able to quote it in the report and will need to use the evidence you provide in a more roundabout way. But more importantly, you will probably find that you will be able to make most impact with your report if you make sure that people with the power to make changes in the institution are aware of your work. If you do share your SWS with the institution, you will find that it can form an invaluable base for campaigning and working with your institution to improve the learning experience for years to come. The Head of Quality (or equivalent) in your institution may well be a useful source of support. While the SWS is entirely independent of the institution, the Head of Quality will be able to advise you about where evidence can be found and how best to present it. He or she will also be able to show you the institutional briefing paper (IBP). It is important to think about including examples of good practice as well as where you think there are weaknesses and challenges. By identifying good practice, the QAA audit can help to spread it throughout the institution. 8 [...]... Head of Quality at your institution For more information, contact: Derfel Owen Development Officer (Student Enhancement), QAA Email: sws@qaa.ac.uk Maureen McLaughlin Assistant Director (Reviews), QAA Email: sws@qaa.ac.uk QAA has a dedicated portal for students on its website For more information on QAA and the Institutional audit process, visit: www.qaa.ac.uk/students, or email: students@qaa.ac.uk QAA... knowledge of the formal issues raised within this document as a reference point You should be able to find out the names of these students from the institutional contact for audit at your institution 11 Institutional audit outcomes After each Institutional audit, QAA publishes a final report that includes a summary of the findings of the audit and a main report The audit team makes judgements based on the... team will also comment on: Quality and standards of postgraduate research programmes Accuracy and completeness of an institution's published information How the institution ensures a strategic approach to enhancing the quality of education The audit team may also comment on areas of good practice 12 Beyond the audit visit It is important to note that no formal oral feedback is given to the institution... Vice-Chancellor or Principal, one with the senior staff with responsibility for the management of academic standards and quality (for example, Pro Vice-Chancellors, Deans, Heads of Quality) and one with students' union officers Each meeting lasts 90 minutes As well as asking questions about the content of the SWS, they will ask about its methodology, analysis and conclusions, so if you have any students'... auditors read and analyse all of the documentation they have been sent Based on this information they will start to form some early hypotheses about how academic quality and standards are managed at your institution Student participation in meetings The first event of the Institutional audit is the briefing visit, which lasts for three days Auditors will visit your institution and have three key meetings:... develop a plan of action and to help implement changes and spread good practice The Institutional audit report is also invaluable in informing the work of your students' union You may wish to campaign and lobby your institution on areas for development identified in the report If you have any questions in this five month period, you should contact the QAA Assistant Director who co-ordinated the audit... it can place in the institution's management of: Quality of learning opportunities available to students Standards of its awards The institution will receive one of three judgements relating to these areas: Limited Confidence Confidence No Confidence The report will highlight recommendations around areas for development that might be 'essential', 'advisable' or 'desirable' The audit team will also... the audit team will read through a great deal more evidence and have a series of meetings - the topics for these will depend on the areas of enquiry At the end of this week, the audit team is joined once more by the AD and will reach its judgements; identifying good practice and recommending areas for improvement and development During the audit week, it is likely that the auditors will see at least...9 After submitting the SWS, what next? QAA will tell you the deadline for submitting your SWS - this will be the same day as your institution's deadline for the IBP From that date, you can expect QAA to follow a clearly set-out timetable If you don't know what this is, you should get in touch with your institution or QAA After your deadline, there will be five-week period when auditors read and analyse... union staff support it might be useful for them to attend the meeting too The audit team will also ask questions about what the institution has said about itself in the IBP, so it's a good idea to make sure you're familiar with it As a general rule the institution is asked not to field more than eight people in each meeting this includes the team's meeting with student representatives The team will . What are 'academic standards'? Academic standards describe the level of achievement that a student has to reach to gain an academic award (for. QAA Email: sws@qaa.ac.uk Maureen McLaughlin Assistant Director (Reviews), QAA Email: sws@qaa.ac.uk QAA has a dedicated portal for students on its website. For