Customer satisfaction report

EUROCONTROL Customer Satisfaction Report IANS A Centre of Excellence in ATM Training Report Copyright Notice © 2006 European Organisation for the Safety of Air Navigation (EUROCONTROL) All rights reserved No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise Any other use is subject to prior written consent by EUROCONTROL Request shall be addressed to: Director of the Institute of Air Navigation Services, 12, rue Antoine de Saint-Exupéry, L-1432 Kirchberg/Luxembourg The EUROCONTROL Institute of Air Navigation Services aims to provide the services that you want and to make your stay in the Institute as enjoyable as possible All Institute personnel are there to ensure that your stay at the Institute is successful However, if you have a complaint (or a compliment) please tell us If you are not satisfied with the service we provide or you would like to propose an improvement then please fill out the form at http://www.eurocontrol.int/ians/complaint.html, or contact IANS.complaints@eurocontrol.int directly ii © Copyright 2006 EUROCONTROL Table of Contents INTRODUCTION ATM TRAINING ATC TRAINING 12 E-LEARNING 13 STAKEHOLDER FEEDBACK 14 COMPLAINTS 15 CONCLUSION 16 Printed on 30/11/2004 Produced by the Business and Administration Unit Customer Satisfaction Report Introduction This is the 7th report that the Institute devotes to the assessment of the level of satisfaction of its customers The study on which this report is based has been conducted in the framework of the quality improvement programme implemented at the Institute since end of 1996 • Measuring customer satisfaction on a regular basis is one of the key elements of the quality strategy implemented by the Institute since 1997 • Our objective is to understand exactly how our customers perceive the Institute and, based on their level of satisfaction, to improve our services and products This report proposes a summary and an analysis of the data collected in 2005 The customer segments assessed in 2005 IANS applies a policy to measure the satisfaction of its customers with all its products and services These include: • • • • • • The ATM course participants The ATC course participants The E-Learning Students Training Sponsors Consultancy Bodies User Groups • A Student Evaluation Form filled in by the students at the end of a course This evaluation form is used for both the ATM course participants and ATC course participants An E-Learning Evaluation Form filled in by a student upon completion of an E-Learning Module A generic Stakeholder Feedback Form filled in by various stakeholders The form is available at http://www.eurocontrol.int/ians/feedback.html Customer Visits IANS visits a sample of its customers on a yearly basis The customer visits serve as a main source of feedback on existing products and a source of information for improvements and new products The customer visits include visits to Air Navigation Service Providers and civil and military State Authorities Complaint Procedure and Complaint Form IANS uses an anonymous form, available on its web-site for the collection of complaints related to its products, services and staff A reference to the complaint procedure and associated form is made in all products and services The complaint form is available at http://www.eurocontrol.int/ians/complaint.html Assessment Tools The assessment is based on a set of assessment tools and measurement methods During 2005 further work has been done to increase the integrity of the data collection Evaluation Forms To measure the satisfaction of its customers with the products and services IANS uses the following evaluation forms: © Copyright 2006 EUROCONTROL ATM Training The Satisfaction of Students with the ATM Training Products is measured through the use of a Course Evaluation Form Measurement Method At the end of each (classroom) course, students are presented with an evaluation form containing a number of criteria to be assessed Each criterion can be given a result between and meaning unacceptable (1), unsatisfactory (2), good (3) an excellent (4) The complete set of criteria are grouped in five main categories: • • • • • Training Content Instructional Team Documentation Equipment Administration The first three categories are directly related to the course The last two categories are directly related to the supporting environment and organisation Figure Student Evaluation Form Printed on 30/11/2004 Produced by the Business and Administration Unit Customer Satisfaction Report From Forms to Results We map the marks of the students on a scale from (all students mark Unacceptable) to +100 (all students mark Excellent) When all students mark Good then the result is 75 and when students mark, on average, between Good and Excellent then the result will be around 85 Likewise, if students fill in 'all-Uncacceptable' then the result will be If students fill in 'all-UnSatisfactory' then the result will be 25 Measuring Satisfaction Marks from Students Unsatisfactory Satisfactory Good Excellent 25 75 100 All All Result All All ©Copyright 2003 EUROCONTROL Figure How we measure satisfaction IANS aims to meet our objective of being a Centre of Excellence in ATM Training For us this means we aim to achieve results on any criterion between 75 and 100 As an example: the overall average (all courses, all criteria, 3090 forms) for IANS in 2005 is 87.5 – compared to 85 in 2004 The highest result on a single criterion is 95 (Subject Knowledge of the Instructor) The lowest result remains pre-course information (76.5 – compared to 70 in 2004) Results The results presented below are extracted from the automated student satisfaction evaluation system Results can be presented per category, per training domain and per criterion We have selected the most significant results to be included in this report © Copyright 2006 EUROCONTROL Size of the Population 3500 This graph shows the number of evaluation forms received and processed by IANS In 2005 the Institute continued to optimise the processing of the forms 3000 2500 2000 This led to another significant increase of the number of processed forms in 2005 1500 1000 500 Number of Forms 2002 2003 2004 2005 1937 1915 2624 3090 Figure Number of Evaluation Forms Composition of the Population Students indicate on the evaluation form which type of staff they are 1400 1200 1000 This chart shows that a majority of Students at IANS in 2005 are Operational and Technical Staff These results are consistent with the results of the previous years 1298 977 2004 824 2005 800 693 600 600 345 400 192 223 200 Administrative Managerial Operational Technical Figure Composition of the population (number of students) Printed on 30/11/2004 Produced by the Business and Administration Unit Customer Satisfaction Report Satisfaction 100 This chart shows the satisfaction of the students with the ATM courses The result is obtained by averaging the results obtained on every criterion in the categories Training Content, Instructional Team and Documentation 95 90 85 80 75 70 A result of 87 indicates that, on average, students mark criteria between Good and Excellent 65 60 55 50 2002 2003 2004 2005 83 84,5 86 87,5 Course Results Figure ATM Course Results The increase in satisfaction has been achieved through detailed analysis of the results of each course and introducing corrective action wherever required Segmented View 100 This chart presents the results per measurement Category These are Training Content, Instructional Team and Documentation 95 90 85 80 75 IANS has improved the results on each of these three categories during the last four years 70 65 60 55 50 2002 2003 2004 2005 81,5 83 84,5 86,5 Instructor 88 89 91 92 Documentation 80 80,5 83,5 85 Content Figure ATM Course Results: per Category © Copyright 2006 EUROCONTROL IANS will continue to analyse the results and make improvements where possible and practicable ‘Best’ Results 100 Customers often praise IANS for its staff and excellent infrastructure 90 This informal feedback is confirmed by the course evaluation results The criteria ‘Instructor Subject Knowledge’ and ‘Classroom Equipment’ have consistently received the highest marks from the students during the last three years 80 70 60 50 IANS will continue to invest in the competence of its instructors by continuation training and competency assessment and it will maintain the building at its high standard 2002 2003 2004 2005 Instructor Subject Knowledge 92,5 93 94,5 95,5 Classroom Equipment 88,5 89 90,5 91 Figure Criteria with ‘Best’ Results Areas for Improvement 100 IANS continuously looks for opportunities to improve Criteria with the less positive results are the Pre-Course Information and the Course Duration 90 80 70 60 50 Pre Course Information Course Duration Canteen 2002 2003 2004 2005 61,5 64 70,5 78 71 74 76 76 73,5 72,5 75 77,5 Figure Criteria with less positive Results In 2004 IANS has put in place an action plan to improve the Pre-Course information and significant progress has been made in this domain in 2004 and 2005 The optimum duration for each course will be reviewed on a courseby-course basis In 2005 IANS changed the layout of the coffee area and installed new equipment in the kitchen to improve the flow of students Printed on 30/11/2004 Produced by the Business and Administration Unit Customer Satisfaction Report Focus on Safety IANS is heavily involved in the development and delivery of Safety Courses to support Safety Regulation and Safety Management In 2005 the number of courses delivered in the Safety domain increased from 36 to 50 and the number of participants increased from 614 to 844 We are pleased to note that the results in this relatively new area for IANS equal the average obtained in other, more mature, training domains 100 95 90 85 80 75 70 65 60 55 50 2003 2004 2005 Safety 86 88,5 90 Overall 84,5 86 87,5 Figure Safety Domain Course Results It is a clear indication that the rigorous development approach at IANS leads to products that meet customer expectation Measuring Loyalty 100% 90% In 2004 we introduced a new question on the evaluation form: ‘Would you recommend the course to a colleague’ 80% 70% 60% 50% 40% 30% 20% 10% 0% Yes No Blank 2004 91% 1% 8% 2005 92% 1% 7% Figure 10 Would recommend course to a colleague 10 © Copyright 2006 EUROCONTROL In 2005, like in 2004, more than 90% of the students would recommend the course that they attended to a colleague  ... Unit Customer Satisfaction Report Introduction This is the 7th report that the Institute devotes to the assessment of the level of satisfaction of its customers The study on which this report. .. on 30/11/2004 Produced by the Business and Administration Unit Customer Satisfaction Report Satisfaction 100 This chart shows the satisfaction of the students with the ATM courses The result is... and, based on their level of satisfaction, to improve our services and products This report proposes a summary and an analysis of the data collected in 2005 The customer segments assessed in