Indicators for inclusive education

INDICATORS FOR INCLUSIVE EDUCATION Indicators for monitoring the implementation of Inclusive Education in accordance with Article 24 of the Convention on the Rights of Persons with Disabilities DRUŠTVO UJEDINJENIH GRAĐANSKIH AKCIJA PUBLISHER MyRight - Empowers people with disabilities | Azize Šaćirbegović 80a, Sarajevo | www.myright.ba “DUGA” (Društvo ujedinjenih građanskih akcija) Association | Petra Tiješića 10, Sarajevo | www.dugabih.ba Light for the World | Niederhofstrasse 26, 1120 Vienna, Austria | www.light-for-the-world.org TRANSLATOR Karla Drpić EDITORS Nadja Popovic Christopher Wizda GRAPHIC DESIGN AND DTP: DER ROTE FADEN grafikdesign TITLE PICTURE ©MyRight, from the #ProudOfOurselves Campaign in Bosnia and Herzegovina, 2016 YEAR OF PUBLICATION 2020 / published online This publication was created in partnership with the organisation Light for the World with the support of the Austrian Development Cooperation and the Kahane Foundation The content of the publication does not necessarily reflect the opinion and views of the mentioned donors but is the sole responsibility of the authors Indicators for monitoring the implementation of Inclusive Education in accordance with Article 24 of the Convention on the Rights of Persons with Disabilities INDICATORS FOR INCLUSIVE EDUCATION Authors ON BEHALF OF THE EDUCATION AUTHORITIES Adela Dupanović, Assistant Minister for General and Normative Legal Affairs – Una-Sana Canton Amila Herceg-Hodžić, Expert Advisor for Education at the Pedagogical Institute of Zenica – Ministry of Education, Science, Culture, and Sports of the Zenica-Doboj Canton Sanel Bašić, Representative of the Department of Education of the Government of the Brčko District of Bosnia and Herzegovina Amira Borovac, Expert Advisor for Preschool and Primary Education – Ministry of Education, Youth, Science, Culture, and Sports, Government of the Bosnian-Podrinje Canton Goražde Vanja Čolić, Senior Expert Associate for Inclusion of Students with Special Educational Needs – Ministry of Education and Culture of the Republic of Srpska Ana Ćavar, History and geography teacher, Senior Independent Officer for Primary Education – Ministry of Education, Science, Culture, and Sports of the West Herzegovina Canton Zineta Bogunić, Minister – Ministry of Education, Science and Youth of the Sarajevo Canton Ante Bokanović, Representative of the Ministry of Education – Canton 10 Zorica Raguz-Osmanagić, Education Inspector – Ministry of Education, Science, Culture, and Sports Posavina Canton Bojana Marić, Senior Expert Associate for Statistics, Analytics and Reporting of the Education Sector – Ministry of Civil Affairs of BiH Dr Marija Nalatelić, Head of the Common Core Curriculum Department – Agency for Preschool, Primary, and Secondary Education of Bosnia and Herzegovina Fikret Vrtagić, Assistant Minister of Education and Science – Ministry of Education and Science of the Tuzla Canton Nada Grahovac, Deputy Ombudsman – Office of the Ombudsman for Children of the Republic of Srpska Lejla Divović, Psychologist, Expert Advisor for European Integration – Education Sector, Ministry of Civil Affairs of BiH Nefiza Dautović, MA Inclusive Education, Expert Advisor for Preschool and Inclusive Education – Herzegovina-Neretva Canton AUTHORS THIS SET OF INDICATORS WAS ALSO SENT TO OTHER RELEVANT BODIES AND ORGANISATIONS FOR CONSIDERATION Federal Ministry of Education and Science Council for Persons with Disabilities of Bosnia and Herzegovina Office of the Ombudsman of Bosnia and Herzegovina Institute for the Education of Children with Disabilities of Tuzla Ministry of Education, Science, Youth, Culture, and Sports – Central Bosnia Canton Coalition of organisations of persons with disabilities of the City of Bijeljina KOLOSI Coalition of organisations of persons with disabilities in the region of Doboj Coalition of organisations of persons with disabilities of the Herzegovina-Neretva Canton Together We Are Stronger (orig “Zajedno smo jači”) Coordinating Committee of the Association of Persons with Disabilities of Sarajevo County Coalition of organisations of persons with disabilities of Tuzla County Inclusive Education committees as advocacy bodies formed in each of mentioned coalitions of organisations of persons with disabilities ON BEHALF OF THE ORGANISERS AND PARTNERS Anka Izetbegović, Executive Director, “DUGA” Association Marina Nezirović, Project Coordinator, “DUGA” Association Binasa Goralija, Coordinator for Europe, MyRight - Empowers people with disabilities Nataša Maros, Project Coordinator, MyRight Empowers people with disabilities INDICATORS FOR INCLUSIVE EDUCATION Table of Contents Foreword .7 Introduction The child as a rights-holder The right of a child with disability to education Inclusive Education Indicators for monitoring the implementation of Inclusive Education 11 Elements of quality Inclusive Education 11 Glossary 14 Documents and materials used 20 Indicators for monitoring the implementation of Inclusive Education in Legislation 21 Indicators for monitoring the implementation of Inclusive Education in Participation 27 Indicators for monitoring the implementation of Inclusive Education in Financing 32 TABLE OF CONTENTS | FOREWORD Foreword The material in front of you is the result of the project “Quality Inclusive Education Now!”, supported by Light for the World with funds from the Austrian Development Cooperation and Kahane Foundation During this project, the “DUGA” association and MyRight-Empowers people with disabilities supported and enabled the education authorities of Bosnia and Herzegovina (BiH) to collectively create a set of indicators that will enable the monitoring of the quality and implementation of Inclusive Education via their participation in a Working Group, which consisted of appointed representatives of all levels of ministries of education from both entities – the Federation of BiH and Republika Srpska – the Education Sector of the Ministry of Civil Affairs of BiH, the Office of the Ombudsman for Children’s Rights of Republika Srpska, as well as the representative of the Agency for Preschool, Primary and Secondary Education of BiH By developing a set of indicators for monitoring the implementation of quality Inclusive Education, the Working Group addressed the challenges of establishing a functional system of Inclusive Education that entails the full and active inclusion of children with disabilities in regular education, taking into account the fulfilment of standards set in the General Commentary on Article 24 of the Convention on the Rights of Persons with Disabilities Binasa Goralija MyRight Regional Coordinator Europe Creating the conditions for quality Inclusive Education enables respect for the broader concept of inclusion and the creation of an inclusive society, in which all citizens are equally valuable and are given the opportunity to participate, regardless of their differences Quality Inclusive Education represents the foundation that guarantees a better and safer future for every person, including people with disabilities, both in the economic and social sense through employment opportunities and active participation in society, as well as in terms of protection from violence and exploitation Bosnia and Herzegovina is obliged to fulfil the obligations assumed by ratifying the Convention on the Rights of the Child and the Convention on the Rights of Persons with Disabilities, as well as signing the 2030 Agenda for Sustainable Development, which is why it is necessary to accelerate the work on quality and fair Inclusive Education for children with disabilities It is important to implement concrete activities that show that we are moving towards a clear goal of achieving the human rights of persons with disabilities, and the creation of indicators, their monitoring, reporting on the achieved results and improvement plans, all of which represent an important step on that journey Anka Izetbegović DUGA Association INDICATORS FOR INCLUSIVE EDUCATION Introduction THE CHILD AS A RIGHTS-HOLDER Among numerous documents human rights, the UN Convention on the Rights of the Child1 is of particular importance The significance of the Convention is reflected in the fact that it introduces the child as a subject and a rights-holder for the first time What makes it special is the fact that it recognises all categories of human rights civil, political, economic, social, and cultural, and it obliges member states to respect its provisions in relation to all children2 - including children with disabilities, at all stages of their development, and in all places where children find themselves while growing up The UN Convention on the Rights of Persons with Disabilities3 does not establish new rights but elaborates what existing human rights mean for persons with disabilities and clarifies the state’s obligations to protect, respect, and promote these rights A state that has ratified the Convention is obliged to not only remove the obstacles that may hinder the exercise of the rights of persons/ children with disabilities but provide support to equalise their opportunities for equal participation in society In order to establish the approach to children with disabilities that is based on human rights, it is particularly important to enable constant communication and coordination of different entities responsible for ensuring all forms of respect and protection of human rights, thus ensuring the maximum benefit for each child under the same conditions In this way, these two complementing conventions form the necessary framework which ensures the realisation of all rights for all children, and in particular equal access in the realisation of the right to Inclusive Education of children with disabilities Both conventions recognise the rights of the child and oblige member states to respect their rights, ensure the necessary protection in their implementation and take all necessary measures – legislative, administrative, educational, and social – to meet the obligations of accessing and ratifying both conventions THE RIGHT OF A CHILD WITH DISABILITY TO EDUCATION Inclusive Education has not always been the primary goal of education at the international level The right to education is being emphasised since the drafting of the first human rights instrument in the Article 26 of the Universal Declaration of Human Rights, and later it was also explained in the International Covenant on Economic, Social and Cultural Rights (ICESCR, 1966) ICESCR emphasises that everyone has the right to education aimed at the full development of in- BiH acceded to the Convention on the Rights of the Child in 1993, and in 2002 and 2003 it ratified the two Optional Protocols to the Convention: on the sale of children, child prostitution and child pornography, and on the involvement of children in armed conflict In 2018, BiH ratified the third Optional Protocol relating to petitions According to Article of the Convention, every human being under the age of 18 is a child BiH ratified the Convention on the Rights of Persons with Disabilities and the Optional Protocol in 2010 INTRODUCTION dividuality, a sense of dignity, and respect for human rights and fundamental freedoms The right to education should enable all people to effectively participate in a free society, to promote understanding, tolerance and friendship, and to help maintain peace Education is considered a cultural right, but it also relates to many other human rights because the enjoyment and realisation of other rights depends on education Due to this relationship, the right to education demonstrates the inseparability and interrelatedness of all human rights “As a right to empowerment, education is an essential means by which economically and socially marginalised adults and children can… achieve a way to fully participate in their communities.” Education creates opportunities and enables people to enjoy their birth-given rights The Committee on Economic, Social and Cultural Rights (CESCR) further explains the right to education in General Comment No on persons with disabilities and No 13 on education, where it defines that education at all levels should be available, accessible, acceptable, and adaptable Availability refers to the creation and maintenance of functional educational content, which includes all materials and resources needed for education This implies non-discriminatory education that is physically and economically accessible to all Acceptability of education refers to the form, content, curriculum, and essence of education All these segments need to be acceptable in order for all students to receive a quality education Finally, education needs to maintain adaptability to adjust to changes and different needs of students Education is not a concept in which “one size fits all”, it must be tailored to the needs of students and not the other way around The Convention on the Rights of the Child protects the rights of all children and includes an article that specifically refers to children with disabilities However, the Convention on the Rights of Persons with Disabilities provides an even more detailed protection of their rights and provides a foundation for the definition of Inclusive Education with an explanation of all its key viewpoints Both conventions advocate for the principles of equality and non-discrimination of children with disabilities in education, which are the basis for the active participation of children in society The signatory states to these conventions, as is Bosnia and Herzegovina, are obliged to enable the right to education without discrimination because this is crucial for the realisation of other economic, social, and cultural rights This duty requires states to take positive action to enable all their citizens, including persons with disabilities, to enjoy the right to education The concept of progressive realisation is based on the assumption that it is not possible to realise all rights immediately and at once, but that it is necessary to work continuously and make progress towards their realisation INCLUSIVE EDUCATION Inclusive Education is the result of many years of experience and transformation of educating children and youth with disabilities As societies develop and progress, so democracy, rule of law, and human rights Special education was the first step in building a society that, through a charitable approach, gives an opportunity for people with disabilities – who until then had been completely deprived of any opportunity to be educated – to be placed in special institutions As the societal attitude towards human rights – and thus the rights of persons with disabilities as their equal members – evolved, there was a transformation of the approach to the education of children with disabilities From the exclusion phase, when children are denied the right to education; through segregation, when children are com- INDICATORS FOR INCLUSIVE EDUCATION pletely separated from peers in special schools, or in the same school but in special classes4; integration, when children with disabilities are physically present in the classroom with their peers and asked to adapt/assimilate to the (existing) environment, without the educational system offering any support or responsiveness/adoption according to the individual needs of the children, thus heavily relying on the persistence and resources of parents/caregivers or projects delivered by NGOs; we come to Inclusive Education Inclusive Education implies the inclusion of children and youth with disabilities in regular schools closest to their place of residence, where all students receive quality education tailored to their individual abilities together What makes Inclusive Education the best option for all is the development of academic and social skills in all students “The Article 24 of the General Comment applies to all persons (including those) with disabilities, i.e those who have long-term physical, mental, intellectual or sensory impairments which, in interaction with various barriers, may hinder their full and effective participation in society on an equal basis with others The shift in thinking is in the second part of the sentence: the Comment refers not only to a school for all, but also to the full participation of everyone involved in the education system Although the focus is on students with disabilities, it is important to remember that the Comment applies to all persons, which is much broader than just persons with disabilities.”5 all its forms All binding laws and regulations, such as strategies, action plans, rulebooks, and standards, should be aligned with the fundamental commitment to the implementation of Inclusive Education Quality Inclusive Education is the most important part of a much broader concept of inclusion and an inclusive society in which all citizens are equally valuable and have the opportunity to participate in it, regardless of their characteristics According to UNESCO’s (1994) Salamanca Statement, “regular schools with this inclusive orientation are the most effective means of combating discriminatory attitudes, creating welcoming communities, building an inclusive society and achieving education for all; moreover, they provide an effective education to the majority of children and improve the efficiency and ultimately the cost-effectiveness of the entire education system.” In order for an education system to be inclusive, beliefs, commitment, and resources must change and adapt to enable quality education for all students in a non-discriminatory environment that guarantees equality and equity from pre-school to higher education, including adult education in The terms “special classes” and “special schools” are used for all different names of institutions at different levels of education in different areas of BiH which segregate children with disabilities in accordance with Article 24 of the Convention on the Rights of Persons with Disabilities Paula Hunt, Creating a more Inclusive Education system 10 Process Indicators 389 Context - achievement s t i m u l a n t s from h i g h e r a d m i n i s t r a t i o n levels - development of educational c o n s u m e r i s m - "co-variables" like school size, s t u d e n t - b o d y composition, school category, u r b a n / r u r a l V In~)uts Process Out~ut~ - t e a c h e r experience [ school level - per pupil expenditure I - degree of achievement oriented policy - parent support I - educational leadership I a d j u s t e d for: I - c o n s e n s u s , cooperative planning of [ • previous I teacher s I achievement [ t" intelligence l _ student I I " quality of school curricula in t e r m s of I content covered, and formal structure ! [ [ Classroom level ] [ - t i m e - o n - t a s k (including homework) I [ - s t r u c t u r e d teaching [ ] - opportunity to learn [ I high expectations of pupils' progress i ] - degree of evaluation and monitoring of J] pupils' progress Figure 2: Empirical Problems have varies been I[ Integral Model of School Effectiveness in Interpreting the Results Research of School Effectiveness Basis The research Figure I j [ - reinforcement Some l lI [ - orderly a t m o s p h e r e I achievement, base that considerably shown to have suppc~rts the list of factors among the individual a positive association that variables with is presented Some achievement in variables in many 390 J Scheerens studies, others, like for instance per pupil expenditure, have been investigated many times but the results show no consistent pattern Some o t h e r v a r i a b l e s o n l y h a v e a relatively w e a k r e s e a r c h b a s e a n d t h e i r i n c l u s i o n in t h e m o d e l is m o r e a m a t t e r of c o n j e c t u r e t h a n solid e m p i r i c a l e v i d e n c e T a b l e g i v e s a n o v e r v i e w of t h e d e g r e e to w h i c h e a c h of t h e f a c t o r s in Figure has been empirically supported Table 3: Degree of Empirical Support for Associated Educational Process Variables with Achievement Characteristic Strong Moderate An, as yet, empirical empirical weak empibasis basis rical basis Mostly conjecture Environmental incentives Consumerism/parent involvement X Teacher experience X Per pupil expenditure X Achievement oriented policy, high expectations X Educational leadership Consensus, cooperative planning X Quality of curricula Evaluative potential Orderly climate Structured teaching Time on task Opportunity to learn Reinforcement X Process Indicators 391 As a p p e a r s from Table we are still far from an empirically s u p p o r t e d c o m p r e h e n s i v e s c h o o l e f f e c t i ve ne s s model In m y o p i n i o n t hi s s h o u l d n o t p r e c l u d e t h e u s e of t h e a v a i l a b l e r e s e a r c h e v i d e n c e as a r e a s o n a b l e d e s i d e r a t u m to i d e n t i f y p r o s p e c t i v e p r o c e s s i n d i c a t o r s At t h e s a m e time we s h o u l d r e a l i z e t h a t t h e a v a i l a b l e r e s e a r c h b a s e still l e a v e s m u c h u n c e r t a i n t y as to t h e s t a t u s of c e r t a i n factors, w h i c h also implies t h a t o t h e r r e v i e w e r s of t h e l i t e r a t u r e will p r o b a b l y com e u p w i t h s o m e w h a t v a r y i n g lists of m o s t i m p o r t a n t f a c t o r s (see for e x a m p l e P u r k e y & S m i t h , 1983; B o s s e r t , 1988) N ar r o w F o c u s ? A l t h o u g h the u s e of e d u c a t i o n a l a t t a i n m e n t d a t a (e.g t h e p r o p o r t i o n of a n age c o h o r t t h a t p a s s e s t he final e x a m i n a t i o n of s e c o n d a r y school) is not at all u n c o m m o n in s c h o o l e f f e c t i v e n e s s r e s e a r c h , m o s t e f f e c t i v e n e s s s t u d i e s h a v e u s e d a c h i e v e m e n t t e s t d a t a on a l i m i t ed s e t of s u b j e c t s (language a n d m a t h e m a t i c s ) S i n c e n a t i o n a l e d u c a t i o n a l s y s t e m s , s c h o o l d i s t r i c t s a n d i ndi vi dual s c h o o l s m a y differ in t h e d e g r e e to w h i c h t h e y v a l u e t h e s e p a r t i c u l a r e d u c a t i o n a l o u t p u t s , t h e y will also look differently u p o n t he u s e f u l n e s s of s c h o o l ef f ecti ve ne s s m o d e l s t h a t r e s t u p o n this n a r r o w definition of school o u t p u t A c o u n t e r - a r g u m e n t to this possible s o u r c e of n o r m a t i v e relativity of s ch o o l effectiveness r e s e a r c h findings is t h a t we are dealing h e r e with basic e d u c a t i o n a l objectives t h a t a p p e a r in t est s a n d e x a m i n a t i o n s exactly b e c a u s e t h e y are t h o u g h t of as ver y i m p o r t a n t Yet, it s e e m s wise to keep a n open m i n d to d i f f e r e n c e s in n o r m a t i v e c o n t e x t s in w h i c h i n d i c a t o r s y s t e m s , i n s p i r e d b y school effectiveness models, are to be applied "Small" Effects A p r o b l e m in i n t e r p r e t i n g t h e r e s u l t s of school effectiveness r e s e a r c h is in h o w far t he sizes of malleable e d u c a t i o n a l c h a r a c t e r i s t i c s are to be seen as e d u c a t i o n a l l y significant It is c o m m o n k n o w l e d g e t h a t t h e s e p r o c e s s c h a r a c t e r i s t i c s g e n e r a l l y e x p l a i n r e l a t i v e l y little v a r i a n c e in p u p i l a c h i e v e m e n t , Before going into this, it s h o u l d be n o t e d first a n d foremost, \ t h a t we are always dealing with reZative school effects, t h a t is, the degree to w h i c h s c h o o l s differ a m o n g t h e m s e l v e s in average a c h i e v e m e n t D e f e n s i v e r e s e a r c h titles like: "Schools c a n m a k e a difference" a n d "School m a t t e r s " s e e m to have b e e n u s e d as w e a p o n s a g a i n s t the s t r a w - m a n a r g u m e n t t h a t it does n o t m a t t e r w h e t h e r or not chi l dren go to school In a 392 J Scheerens s a m p l e of a b o u t 30 s c h o o l e f f e c t i v e n e s s s t u d i e s ( S c h e e r e n s , 1989) t h e a v e r a g e v a r i a n c e b e t w e e n s c h o o l s w as a b o u t 12% of t h e t ot al (i.e pupillevel) v a r i a n c e in a c h i e v e m e n t (range: 5% - 40%) (All fi gures on s c h o o l effect sizes t h a t are given in this s e c t i o n are b a s e d on a c h i e v e m e n t s c o r e s t h a t h a v e b e e n a d j u s t e d for pupi l s ' b a c k g r o u n d c h a r a c t e r i s t i c s ) T hi s imp lies t h a t specific s c h o o l - l e v e l c h a r a c t e r i s t i c s c a n o n l y a c c o u n t for a c e r t a i n p r o p o r t i o n of t he v a r i a n c e b e t w e e n schools, w h i c h m e a n s t h a t t h e s e specific school-level c h a r a c t e r i s t i c s m a y n o t expl n m o r e t h a n , say, 4% of pupil-level v a r i a n c e This t i ny effect w oul d - in a s i t u a t i o n w h e r e v a r i a n c e b e t w e e n s c h o o l s is 12%- still m e a n t h a t t he v a r i a b l e in q u e s t i o n expl ned o n e - t h i r d of all t h a t specific school f a c t o r s coul d e x p l a i n in t hi s p a r t i c u l a r situation In i n t e r p r e t i n g s c hool effects we s h o u l d b e a r in m i n d t h a t u p to 75% of th e v a r i a n c e in pupi l a c h i e v e m e n t c a n be a c c o u n t e d for by f a c t o r s like intelligence a n d social e c o n o m i c factors w h i c h are to a large e x t e n d b e y o n d t h e r e a c h of e d u c a t i v e m a n i p u l a t i o n T h i s m e a n s t h a t t h e t ot al d i r e c t i n f l u e n c e of s c h o o l i n g is r e l a t i v e l y s m a l l as far as a p u p i l ' s overall a c h i e v e m e n t level is c o n c e r n e d I n t e r p r e t i n g s c h o o l effects in t h e i r p r o p e r p e r s p e c t i v e c o u l d b e n e f i t f r o m m o r e i n s i g h t f u l w a y s of e x p r e s s i n g effect sizes (see R u t t e r , 1983; B o s k e r & S c h e e r e n s , 1989) A g e n e r a l a p p r o a c h to a c c o m p l i s h i n g t h i s would be to a t t a c h s o m e kind of societal val ue to score levels on t he o u t p u t variable P u r k e y & S m i t h (1983) give a n e x a m p l e of t hi s b y t r a n s l a t i n g a d i f f e r e n c e of t w o - t h i r d s of a s t a n d a r d d e v i a t i o n in a c h i e v e m e n t b e t w e e n h i g h e s t s c o r i n g a n d lowest s c o r i n g s c h o o l s into a c e r t a i n t i m e i nt erval (in t h e i r c a s e o ne year) t h a t e x p r e s s e s t h e d e g r e e t h e a v e r a g e p u p i l of t h e h i g h e s t s c o r i n g s chool is a h e a d of t he average pupi l of t h e l ow est s c o r i n g school Th e c o n c l u s i o n on t he "small effects" in school effectiveness r e s e a r c h is t h a t we s h o u l d u s e m o r e specific e x p r e s s i o n s of effects t h a n m e r e l y " p e r c e n t a g e of total v a r i a n c e explained", a n d b y doing so we m a y discover t h a t so-called "small effects" m a y still be of e d u c a t i o n a l i m p o r t a n c e C o n c e p t u a l a n d Methodological Pr obl e m s in School Effectiveness R e s e a r c h C o n c e p t u a l p r o b l e m s in i n t e r p r e t i n g t h e r e s u l t s of s c h o o l effectiveness r e s e a r c h are closely r el at ed to t he c o r r e l a t i o n a l n a t u r e of m o s t studies and t h e " i n d u c t i v e " a p p r o a c h in b u i l d i n g s c h o o l e f f e c t i v e n e s s m o d e l s So, for i n s t a n c e , it is h a r d to s a y w h e t h e r a v a r i a b l e like "high Process Indicators 393 e x p e c t a t i o n s of pupils' progress" is to be s e e n as c a u s e or effect of high a c h i e v e m e n t , w h e n we h a v e n o t e x p e r i m e n t a l l y m a n i p u l a t e d t h e expectations variable More generally, we m a y have some ideas now on w h a t variables work in education, yet, we still have little knowledge on the causal m e c h a n i s m s t h a t explain t h e correlations The l i n k s w i t h more general e x p l a n a t o r y principles or theories are still relatively w e a k l y developed (ef S c h e e r e n s & Stoel, 1988) Methodological w e a k n e s s of school effectiveness r e s e a r c h h a s been t h o r o u g h l y described in v a r i o u s review articles (e.g., R a l p h & F e n n e s s e y , 1983) "Small samples", "insufficient a d j u s t m e n t for i m p o r t a n t b a c k g r o u n d c h a r a c t e r i s t i c s of s c h o o l i n g " , "reactive r e s e a r c h a r r a n g e m e n t s " a n d "improper u s e of a n a l y s i s techniques" are the m a i n points of criticism A l t h o u g h t h e s e c o n c e p t u a l a n d methodological problems are certainly one m o r e r e a s o n to t r e a t t h e r e s u l t s of s c h o o l e f f e c t i v e n e s s r e s e a r c h c a u t i o u s l y , s o m e c o u n t e r - a r g u m e n t s s h o u l d be c o n s i d e r e d too As to the methodological criticism, it s h o u l d be r e m a r k e d t h a t r e s u l t s have s h o w n a c o n s i d e r a b l e r o b u s t n e s s in t h e face of v a r y i n g r e s e a r c h s e t t i n g s a n d r e s e a r c h a p p r o a c h e s An a r g u m e n t for the e m e r g e n t school effectiveness m o d e l t h a t is h a r d to neglect - t h o u g h it is n o t very scientific - is its intuitive appeal Some of the basic principles, like more s u c c e s s in specific k i n d s of school o u t p u t w h e n these o u t p u t s are actively strived for and are an explicit policy of the school, a n d better r e s u l t s w h e n more time is invested, s i m p l y m a k e s e n s e Finally, some a t t e m p t s h a v e b e e n m a d e to link the f i n d i n g s of s c h o o l e f f e c t i v e n e s s r e s e a r c h to m o r e g e n e r a l e x p l a n a t o r y principles like l e a r n i n g theory, X-efficiency, a n d c e r t a i n c o n c e p t u a l i z a t i o n s from organization t h e o r y (Scheerens & Stoel, 1988) Despite all c a u t i o n a r y notes t h a t have been m a d e in this section it still s e e m s permissible to u s e the r e s e a r c h literature on school effectiveness as a b e s t g u e s s for proposing likely c a n d i d a t e s for process indicators Perhaps the m o s t i m p o r t a n t suggestion t h a t these points of criticism have yielded is to c o n s i d e r c a r e f u l l y t h e u s e f u l n e s s of specific v a r i a b l e s w i t h i n the p a r t i c u l a r (national or local) e d u c a t i o n a l context Multiple Context, Multiple Level a n d Multiple M e a s u r e m e n t Applications of Pro c~ess Indicators As s t a t e d in the i n t r o d u c t o r y section, indicator s y s t e m s can be u s e d in several a d m i n i s t r a t i v e contexts: by n a t i o n a l e d u c a t i o n a l policymakers; by 394 J Scheerens officers at local level, b y m a n a g e r s at school level a n d even b y t e a c h e r s at th e c l a s s r o o m level A second distinction that has been used in t h i s paper is t h e a g g r e g a t i o n level a t w h i c h b a s i c v a r i a b l e s , f r o m w h i c h i n d i c a t o r s c a n be computed, are measured In our comprehensive model of school ef f ectiv en es s we d i s c e r n e d m e a s u r e s at s t u d e n t , t e a c h e r / c l a s s , school a n d s c h o o l - c o n t e x t level A l t h o u g h we u s e t h e s a m e levels as r e f e r e n c e p o i n t s for b o t h d i s t i n c t i o n s t h e r e is no c o m p e l l i n g o n e - t o - o n e c o r r e s p o n d e n c e b e t w e e n t h e two, s i nce t h e h i g h e s t a d m i n i s t r a t i v e level m a y w i s h to u s e m e a s u r e s at a n y level of aggregation A t h i r d d i m e n s i o n t h a t c a n be u s e d for classifying i n d i c a t o r s refers to t h e n a t u r e of m e a s u r e m e n t s W h e t h e r we are d e a l i n g w i t h h i g h or low i n f e r e n c e m e a s u r e m e n t s (e.g., t h e n u m b e r of b o o k s in a s c h o o l l i b r a r y vs e d u c a t i o n a l leadership), w h e t h e r or n o t m e a s u r e s are readi l y available from ex is tin g d a t a s o u r c e s a n d w h e t h e r we c a n u s e s t a n d a r d i z e d q u a n t i t a t i v e scales or m u s t rely on relatively u n s t r u c t u r e d m e t h o d s (such as p a r t i c i p a n t o b s e r v a t i o n or "open" i n t e r v i e w s ) T h e n a t u r e of m e a s u r e s on w h i c h e d u c a t i o n a l i n d i c a t o r s r e l y d e p e n d s on w h e t h e r i n d i c a t o r s y s t e m s are exclusively t h o u g h t of as "closed" i n f o r m a t i o n s y s t e m s of l ongi t udi nal d a t a or as "looser" s t r u c t u r e s t h a t also allow for t he i n c l u s i o n of specific evaluative s t u d i e s ( s u ch as t h e i n t e r n a t i o n a l c o m p a r i s o n s of e d u c a t i o n a l a c h i e v e m e n t c o n d u c t e d b y t he I n t e r n a t i o n a l Association for the E v a l u a t i o n of E d u c a t i o n a l Achievement, [IEA], or i n - d e p t h s t u d i e s t h a t coul d be c o n d u c t e d by a u d i t c o m m i s s i o n s or t h e i n s p e c t o r a t e W h e n we a r e l o o k i n g at e d u c a t i o n a l i n d i c a t o r s y s t e m s f r o m t h e p e r s p e c t i v e of m u l t i p l e a d m i n i s t r a t i v e levels, m u l t i p l e lev el s of a g g r e g a t i o n of b a s i c d a t a a n d m u l t i p l e t y p e s of m e a s u r e m e n t , we a r e c l e a r l y u s i n g a b r o a d i n t e r p r e t a t i o n of t h e t e r m " e d u c a t i o n a l i n d i c a t o r " T h e c o m m o n core of all t h e s e a p p l i c a t i o n s still c o n f o r m s to t h e definition that was stated in t h e ideology section: m e a s u r e m e n t of ke y a s p e c t s of e d u c a t i o n a l s y s t e m s t h a t are of evaluative relevance According to the t h r e e d i s t i n c t i o n s t h a t were d i s c u s s e d , we could u s e a t h r e e - d i m e n s i o n a l f r a m e w o r k to classify t y p e s of i n d i c a t o r a p p l i c a t i o n s (see Figure 3) Process Indicators 395 I A d m i n i s t r a t i v e level II Aggregation Ill Complexity of level of measures measurement Figure 3: A Three-dimensional Framework to Classify Types of Indicator Applications In order framework to d e t e r m i n e of a p p l i c a t i o n s the r o l e of p r o c e s s indicators within s o m e of t h e m o s t l i k e l y c o m b i n a t i o n s this of s c a l e p o i n t s o n t h e t h r e e d i m e n s i o n s of F i g u r e will b e e x a m i n e d D e t e r m i n i n g t h e C o n d i t i o n of E d u c a t i o n a t t h e N a t i o n a l Level F i r s t , w e c o n s i d e r d e c i s i o n - m a k i n g a t t h e n a t i o n a l level s u p p o r t e d b y summary statistics measures and at high levels existing data bases of a g g r e g a t i o n using low inference T h i s is t h e m o r e t r a d i t i o n a l t y p e of a p p l i c a t i o n of e d u c a t i o n a l i n d i c a t o r s D e s c r i p t i v e s t a t i s t i c s on, for i n s t a n c e , e n r o l m e n t s and financial data a r e l i k e l y to b e i n c l u d e d in t h i s t y p e of a p p l i c a t i o n of i n d i c a t o r s , as are s t a t i s t i c s o n e d u c a t i o n a l p r o d u c t i v i t y in t e r m s of, for i n s t a n c e , p r o p o r t i o n of a g e c o h o r t s t h a t p a s s final e x a m i n a t i o n s C o u l d o n e c o n c e i v e of a p l a c e for p r o c e s s i n d i c a t o r s in t h i s k i n d of m a c r o - l e v e l i n d i c a t o r s y s t e m ? T h e a i m of i n c l u d i n g p r o c e s s i n d i c a t o r s w o u l d b e to g a i n i n s i g h t i n t o p o s s i b l e c a u s e s for h i g h or l o w e d u c a t i o n a l p r o d u c t i v i t y in a p a r t i c u l a r p e r i o d T h e difficulty with process variables (see t h e list t h a t is c o n t a i n e d in F i g u r e 2) is, of c o u r s e , t h a t m o s t of t h e m a r e n e i t h e r e a s i l y m e a s u r a b l e n o r r e a d i l y a v a i l a b l e from existing data sources Some proxy variables that might be considered a s a b a s i s for m a c r o - l e v e l p r o c e s s i n d i c a t o r s are: - number of schooldays p e r y e a r for a p a r t i c u l a r s c h o o l t y p e in a particular year; - depending on formal regulations which require s c h o o l s to r e g i s t e r l e s s o n " d r o p - o u t " , t h e p e r c e n t a g e of l e s s o n s t h a t - for o n e r e a s o n or a n o t h e r - w a s n o t g i v e n for a p a r t i c u l a r s c h o o l t y p e , d u r i n g a specific period 396 J Scheerens T h e s e two v a r i a b l e s could be u s e d for c a l c u l a t i n g a n i n d i c a t o r of nettime for tuition T he policy r e l e v a n c e of s u c h a n i n d i c a t o r w oul d d e p e n d on t h e v a r i a b i l i t y of t h i s i n d i c a t o r over time It c o u l d also p l a y a role in international comparisons of t h e f u n c t i o n i n g of n a t i o n a l educational s y s tems On e co ul d also t h i n k of c o n s t r u c t i n g a m a c r o - l e v e l i n d i c a t o r for t he e v a l u a t i v e p o t e n t i a l of e d u c a t i o n a l s y s t e m s In t h i s r e s p e c t it w o u l d be r e l e v a n t w h e t h e r or n o t a c o u n t r y h a s a n a t i o n a l a s s e s s m e n t p r o j e c t a n d w h e t h e r or n o t t h e r e s u l t s of a s s e s s m e n t are fed b a c k to individual schools At t h e n a t i o n a l level this i n d i c a t o r could be of r e l e v a n c e b e c a u s e of possible c h a n g e s over time, for i n s t a n c e , w h e n a n a t i o n a l a s s e s s m e n t p r o j e c t is still in a d e v e l o p m e n t a l p h a s e (as is t he case in t he Netherlands) A final possibility for a p r o x y - m a c r o - p r o c e s s i n d i c a t o r coul d be in t he a r e a of e d u c a t i o n a l l e a d e r s h i p , for i n s t a n c e , b y u s i n g a m e a s u r e of t he total effort (time, m o n e y , e n r o l m e n t s ) of m a n a g e m e n t t r a i n i n g c o u r s e s d i r e c t e d at s t i m u l a t i n g this It s h o u l d be n o t e d , how ever, t h a t s u c h a n i n d i c a t o r w o u l d r e q u i r e r a t h e r d e t a i l e d c o n t e n t - a n a l y s i s of e x i s t i n g m a n a g e m e n t t r a i n i n g c o u r s e s a n d t h e r e f o r e w o u l d p r o b a b l y n o t fit t h e r e q u i r e m e n t of easy measurement T h e c o n c l u s i o n is t h a t p r o c e s s i n d i c a t o r s onl y h a v e a limited pl ace w i t h i n p u r e m a c r o level i n d i c a t o r s y s t e m s If, however, o n e w o u l d c o n s i d e r p a t c h i n g u p t h e s e i n d i c a t o r s y s t e m s b y m e a n s of i n - d e p t h s t u d i e s of p r o c e s s v a r i a b l e s t h a t coul d be l i nked to r e g u l a r d a t a s t r e a m s , t h e r e is a t r e m e n d o u s i n c r e a s e in possibilities I n - d e p t h s t u d i e s c o u l d t a k e t he form of c o m p a r a t i v e i n t e r n a t i o n a l s u r v e y s , specific r e s e a r c h p r o j e c t s at t h e n a t i o n a l level, or a u d i t s b y e x p e r t c o m m i t t i e s or t h e i n s p e c t o r a t e In t h e s e s t u d i e s all p r o c e s s va r i abl es m e n t i o n e d in Figure could be i ncl uded, be it b y m e a n s of s c a l e s a n d q u e s t i o n n a i r e i t e m s or b y m e a n s of m o r e o p e n c h e c k - l i s t s for o b s e r v a t i o n or c o n t e n t - a n a l y s i s Sch o o l Monitoring at t h e District Level District-level m o n i t o r i n g of s chool s could benefi t from s y s t e m a t i c d a t a collection as a b a s i s for a limited set of i n d i c a t o r s For s u c h m a n a g e m e n t i n f o r m a t i o n s y s t e m s , a s for t h e p r e v i o u s l y d i s c u s s e d a p p l i c a t i o n of i n d i c a t o r s , low i n f e r e n c e m e a s u r e s w o u l d be p r e f e r a b l e T h e m o s t likely level of a g g r e g a t i o n of t he d a t a w oul d be school level, t h o u g h t e a c h e r - a n d s t u d e n t level d a t a m i g h t also be u s e d F i n a n c i a l a n d o u t p u t d a t a (both in t e r m s of o v e r al l p r o d u c t i v i t y a n d s t u d e n t a c h i e v e m e n t ) a r e t h e m o s t Process Indicators 397 important data categories for such monitoring systems Examples of process variables t h a t could be included are: W h e t h e r or n o t s c h o o l s u s e explicit a c h i e v e m e n t s t a n d a r d s (to be d e d u c e d from school c u r r i c u l a , d e v e l o p m e n t p l a n s or o t h e r official documents); W h e t h e r or n o t school leaders refer to a c h i e v e m e n t records in official d o c u m e n t s a n d b r o c h u r e s i n t e n d e d for the school c o m m u n i t y ; The a m o u n t of n o n - t e a c h i n g t i m e t h a t is s p e n t on collaborative p l a n n i n g of curricula a n d lessons by the staff; N u m b e r of s t u d e n t s expelled from school, in a p a r t i c u l a r school year; W h e t h e r or not h e a d t e a c h e r s formally evaluate staff on a regular basis; The f r e q u e n c y of the u s e of a c h i e v e m e n t t e s t s at all grade levels; Figures on pupil a b s e n t e e i s m a n d lesson drop-out The above variables are a s s u m e d to be available from a d m i n i s t r a t i v e s c h o o l r e c o r d s or a s s e s s e d b y m e a n s of relatively brief e n c o u n t e r s w i t h h e a d t e a c h e r s The process variables could be enlarged if research-like d a t a collection w a s a d d e d at regular intervals School Self-Evaluation One m i g h t s a y t h a t t h e two a p p l i c a t i o n s of i n d i c a t o r s y s t e m s t h a t were d i s c u s s e d in t h e above are a c c o u n t a b i l i t y o r i e n t e d r a t h e r t h a n i m p r o v e m e n t oriented In m y opinion, t h e d i s t i n c t i o n b e t w e e n e v a l u a t i n g for a c c o u n t a b i l i t y vs e v a l u a t i n g for i m p r o v e m e n t s h o u l d n o t be d r a w n too s h a r p , since t h e r e is a n i m p o r t a n t c o m m o n e l e m e n t in both: the element of l e a r n i n g b y m e a n s of empirical test a n d feedback of i n f o r m a t i o n to relevant actors However, the third kind of application of e d u c a t i o n a l indicators t h a t will be referred to here, is m o s t s t r o n g l y a s s o c i a t e d with i m p r o v e m e n t (cf Hopkins & Leask, 1989) Although o u t p u t indicators deserve a central place in s e l f - e v a l u a t i o n p r o c e d u r e s like in all o t h e r a p p l i c a t i o n s of e d u c a t i o n a l indicators, in p r o c e d u r e s for self-evaluation or school-based review, process variables are of particular relevance Since organizational learning is the key motive for u s i n g self-evaluation s c h e d u l e s for school improvement, one will n e e d r a t h e r d e t a i l e d information, on t h o s e p r o c e s s e s t h a t m i g h t explain d i s a p p o i n t i n g r e s u l t s a n d at the s a m e time offer h a n d l e s for improvement The list of p r o c e s s variables, b a s e d on the r e s u l t s of school effectiveness r e s u l t s , m i g h t be u s e d to review existing s c h e m e s a n d c h e c k - l i s t s for 398 J Scheerens s c h o o l - b a s e d review (for a n overview of t h e s e c h e c k - l i s t s , see Hopkins, 1987) One m i g h t conceive of i n t e g r a t i n g t h e s e macro-, m e s o - a n d microlevel a p p l i c a t i o n s of indicators In fact, the US e d u c a t i o n a l d a t a redesign project (Teaubert, 1987) does exactly this In this way, detailed information is collected at the lowest u n i t (the classroom), a n d p a r t of this information is aggregated to be u s e d at the next level up, a n d so on A l t h o u g h s u c h a n i n t e g r a t e d m u l t i p l e c o n t e x t , m u l t i p l e level a n d m u l t i p l e m e a s u r e m e n t i n d i c a t o r s y s t e m s h a v e i m p o r t a n t a d v a n t a g e s , it is a v e r y a m b i t i o u s e n d e a v o u r One could also conceive of looser coupling of m a c r o - m e s o a n d micro a p p l i c a t i o n s , for i n s t a n c e b y e x c h a n g i n g i n s t r u m e n t s b e t w e e n the various contexts of application Conclusion Process i n d i c a t o r s are a s o m e w h a t difficult c a t e g o r y a m o n g other, more e s t a b l i s h e d t y p e s of indicators The m a i n difficulty is t h a t process indicators u s u a l l y require r a t h e r complicated p r o c e d u r e s of d a t a collection a n d m e a s u r e m e n t a n d t h u s n o t fit the r e q u i r e m e n t of readily available d a t a for c o m p u t i n g indicators Yet, there is an i m p o r t a n t motive to try and i n c l u d e p r o c e s s i n f o r m a t i o n in i n d i c a t o r s y s t e m s Like d a t a on r e s o u r c e s a n d i n p u t s , p r o c e s s d a t a provide b a c k g r o u n d m a t e r i a l t h a t is helpful in m a k i n g s e n s e o u t of m e r e p e r f o r m a n c e d a t a on t h e f u n c t i o n i n g of educational systems In this p a p e r the literature on school a n d i n s t r u c t i o n a l effectiveness h a s b e e n u s e d to s u g g e s t likely c a n d i d a t e s for process indicators At the s a m e time the scope for d i s c u s s i n g the application of p r o c e s s i n d i c a t o r s was enlarged by also c o n s i d e r i n g i n d i c a t o r a p p l i c a t i o n s at a d m i n i s t r a t i v e levels below t h a t of n a t i o n a l g o v e r n m e n t s In Table the list of variables t h a t is proposed here as a basis for the development of process indicators is c o m p a r e d to other lists of process indicators t h a t have been proposed As a p p e a r s from Table 4, t h e r e is c o n s i d e r a b l e a g r e e m e n t b e t w e e n the proposal t h a t is m a d e here a n d those by Oakes, Benveniste a n d Teauber The proposals by Unesco a n d W i n d h a m are s t a t e d in more general terms The one factor t h a t is included in all of the proposals s u m m a r i z e d in Table is i n s t r u c t i o n a l time Process Indicators Table 4: 399 Comparison of Sets of Process Indicators Scheerens 1989 Unesco 1976 Windham 1988 Achievement s t i m u l a n t s Achievement oriented policy Educational leadership Teachers' cooperative planning Quality of curriculum Evaluative potential Orderly climate Time on task Structured teaching Opportunity to learn High expectations Monitoring progress Reinforcement Allocation of resources Retention & progression rates Teacher/hours per pupil per year Cost and management Instructional organization Alternative technologies Use of teacher and student time Teauber 1987 InStructional leadership curriculum type of instruction (whole class, small group, etc.) Time on task School climate Influence of peer group BcnvCntste, 191~7 Oakes 1987 Teacher time (teaching/non-teaching) Access to knowledge (e.g instructional time) Student learning time: Press for achievement (e.g graduation requirements) - course enrolment turnover rates Professional conditions for teaching (e.g time spent on collaborative planning) pupil/teacher ratios schoolday activities - length of schoolyear - out of school learning time Order and consistency: - truancy, absenteeism, vandalism, disruptions student turnover - student cooperative behaviour - - - - T h e c a t e g o r i e s of v a r i a b l e s t h a t d i s t i n g u i s h from the others are: achievement structured stimulants t h e p r o p o s a l t h a t is m a d e h e r e from the school environment, t e a c h i n g a n d t h e e v a l u a t i v e p o t e n t i a l of t h e s c h o o l 400 J Scheerens References Achilles, C.M & Lintz, M.N (1986) Evaluation of an "effective schools" intervention AERA-paper San Francisco Averch, H.A et al (1974) How effective is schooling?: A critical review of research A rand educational policy study Englewood Cliffs: Educational Technology Publications Benveniste, G (1987) The design of school accountability systems Paper presented at the Conference on Educational Indicators, Washington Bosker, R.J & Scheerens, J (1989) Criterion choice, effect size and stability Three fundamental problems in school effectiveness research In: B.P.M Creemers and B Reynolds (Eds.), School effectiveness and school improvement, Lisse: Swets & Zeitlinger Bossert, S.T (1988) School effects In: N.J Boyan (Ed.), Handbook research on educational administration New York: Longman Inc of Breton, A & Wintrobe, R (1982) The logic o f bureaucratic conduct Cambridge: Cambridge University Press Brookover, W.B., Beady, C., Flood, P., Schweitzer, J.H., Schneider, J.M & W i s e n b a k e r , J.M, (1979) School social s y s t e m s and s t u d e n t achievement - schools can make a difference New York: Praeger Publishers Brophy, J & Good, Th L (1986) T e a c h e r b e h a v i o r a n d s t u d e n t achievement In: M.C Wittrock (Ed.), Handbook of research on teaching (pp 328-375) New York: McMillan Inc Cameron, K.S & Whetten, D.A (Eds,) (1983) Organizational effectiveness A comparison of multiple models New York: Academic Press Coleman, J.S., Campbell, E.Q., Hobson, C.Y., McPartland, Y., Mood, F.D., Weinfeld, F.D & York, R.L (1966) Equality of educational opportunity Washington D.C.: U.S Government Printing Office Collins, A & Stevens, A (1982) Goals and strategies of inquiry teachers Advances in Instructional Psychology, Vol II Ed.: R Glaser Hillsdale, N.J.: Lawrence Erlbaum Associates Doyle, W (1985) Effective s e c o n d a r y classroom practices In: M.J Kyle (ed.), Reaching f o r excellence An effective schools sourcebook Washington: U.S Government Printing Office Faerman, S.R & R.E Quinn (1985) Effectiveness: the perspective from organization theory The Review of Higher Education, 9, 83-100 F i r e s t o n e , W.A & Herriott, R.E (1982) P r e s c r i p t i o n for effective elementary schools don't fit secondary schools Educational Leadership, 40, 51-53 Process Indicators 401 Fraser, B.J., Walberg, H.J., Welch, W.W & Hattie, J.A (1987) S y n t h e s e s of e d u c a t i o n a l p r o d u c t i v i t y r e s e a r c h Special I s s u e of t h e International Journal of Educational Research, 11, no Friedkin, N.E & Necochea, J (1988) School s y s t e m size a n d performance: a c o n t i n g e n c y perspective Educational Evaluation and Policy Analysis, 10 - G l a s m a n , N.S & Biniaminov, J (1981) I n p u t - o u t p u t a n a l y s i s of schools Review of Educational Research, 51, 509-539 H a n u s h e k , E.A (1979) C o n c e p t u a l a n d empirical i s s u e s in the estimation of e d u c a t i o n a l p r o d u c t i o n functions Journal of Human Resources, 14, 351-388 H a n u s h e k , E.A (1986) The e c o n o m i c s of schooling: p r o d u c t i o n a n d efficiency in p u b l i c schools Journal of Economic Literature, 24, 11411177 H a u s e r , R.M., Sewell, W.H & Alwin, D.F (1976) High School effects on achievement In: W.H Sewell, R.M H a u s e r and D.L F e a t h e r m a n (Eds.), Schooling and achievement in American Society New York: Academic Press Herpen, M v a n (1989) Conceptual models in use for educational indicators Paper for the Conference on E d u c a t i o n a l Indicators in S a n Francisco Hopkins, D (1987) Doing School Based Review Leuven, Belgium: ACCO H o p k i n s , D & M L e a s k (1989) P e r f o r m a n c e I n d i c a t o r s Development School Organization, 9, n o l , 3-20 and School J e n c k s , C.S., Smith, M., Ackland, H., Bane, M.Y., Cohen, D., Gintis, H., Heyns, B & Michelson, S (1972) Inequality: r e a s s e s s m e n t of the effect of family and schooling in America New York: Basic Books J e n c k s , C., Bartlett, S., Corcoran, M., Crouse, J., Eaglesfield, F., J a c k s o n , G., McClelland, K., Ward, S & Williams, I (1979) Who gets ahead? The determinants of economic success in America New York: Basic Books Kulik, C.L.C & Kulik, J.A (1982) Effects of ability grouping on s e c o n d a r y s c h o o l s t u d e n t s : a m e t a - a n a l y s i s of r e s e a r c h findings A m e r i c a n Educational Research Journal, 19, 415-428 Kyle, M.J (ed.) (1985) Reaching for excellence An effective schools sourcebook W a s h i n g t o n D.C.: U.S G o v e r n m e n t Printing Office Levin, H.H (1988) Cost-effectiveness a n d e d u c a t i o n a l policy Educational Evaluation and Policy Analg~.is, 10, 51-69 Lezotte, L.W., E d m o n d s , R & R a t n e r , G (1974) A final report: remedy for school failure to equitably deliver school basic skills Michigan S t a t e University 402 J Scheerens M c C o r m a c k - L a r k i n , M (1985) I n g r e d i e n t s of a s u c c e s s f u l s c h o o l effectiveness project Educational Leadership March 1985, 31-37 Miller, S.K., Cohen, S.R & Sayre, K.A (1985) Significant a c h i e v e m e n t g a m e s u s i n g the effective schools model Educational Leadership, March 1985, 28-43 Mintzberg, H (1979) The structuring o f organizations E n g l e w o o d Cliffs: Prentice Hall Mortimore, P., S a m m o n s , P., Stoll, L., Lewis, D & Ecob, R (1988) School matters: the junior years Somerset: Open Books Mosteller, F & Moynihan, D.D (Eds.) (1972) On equality o f educational opportunity New York: R a n d o m House N i s k a n e n , W.A (1971) B u r e a u c r a c y Chicago: Aldine-Atherton and representative government Nuttal, D.L (1989) International E d u c a t i o n a l Indicators The conceptual paper Paper for the meeting of the OECD E d u c a t i o n a l Indicator project San Francisco Oakes, J (1987) Conceptual a n d m e a s u r e m e n t i s s u e s in the construction o f school quality indicators Paper p r e s e n t e d at the A E R A - a n n u a l meeting Washington D.C P u r k e y , S.C & Smith, M.S (1983) Effective s c h o o l s : Elementary School Journal, 83, - a review T h e Ralph, J H & F e n n e s s e y , J (1983) Science or reform: s o m e q u e s t i o n s a b o u t the effective schools model Phi Delta Kappan, - Rutter, M (1983) School Effects on Pupil Progress: R e s e a r c h Findings and Policy Implications Child Development, 54, 1-29 S c h e e r e n s , J & Stoel, W.R.G (1988) T h e o r y f o r m a t i o n on school effectiveness Paper p r e s e n t e d at the A n n u a l Meeting of the AERA, New Orleans Scheerens, J., Stoel, W.G.R., Vermeulen, C.J.A.J and Pelgrum, W.J (1988) De haalbaarheid van een indicatorenstelsel voor het basis- en voortgezet onderwijs E n s c h e d e : Universiteit Twente Scheerens, J (1989) W a t m a a k t scholen effectief? S a m e n v a t t i n g en a n a l y s e van o n d e r z o e k s r e s u l t a t e n Den Haag: Stichting voor O n d e r z o e k van het Onderwijs Scheerens, J., Nanninga, H.C.R & Pelgrum, W.J (1989) Generaliability of instructional a n d school effectiveness indicators a c r o s s nations B.P.M C r e e m e r s a n d J S c h e e r e n s (eds.) To be p u b l i s h e d in D e v e l o p m e n t s in school effectiveness research Special issue of the International Journal of Educational Research, 13 Process Indicators 403 Stern, J.D (1986) The educational indicators project at the U.S Department o f Education Center for Statistics, U.S Department of Education Teauber, R.C (ed.) (1987) E d u c a t i o n d a t a s y s t e m redesign Special issue of the International Journal of Educational Research, 11, no 4, 391-511 Teddlie, C., Stringfield, S & Wimpelberg, R (1987) Contextual differences in effective schooling in Louisiana AERA-paper W a s h i n g t o n T h o m p s o n , J.D (1967) Organizations in action New York: McGraw Hill T h o r n d i k e , R.L (1973) Reading comprehension countries Stockholm: Alonqvist & Wiksell education in f i f t e e n U n e s c o (1976) Towards a s y s t e m of educational indicators Division of E d u c a t i o n a l Policy and Planning, Paris Walberg, H.J (1984) I m p r o v i n g the p r o d u c t i v i t y of A m e r i c a n schools Educational Leadership, 41, 19-27 Weber, G (1971) Inner-city children can be t a u g h t to read: F o u r successful schools Washington D.C.: Council for Basic Education W i n d h a m , D.M (1988) Effectiveness indicators in the economic a n a l y s i s of e d u c a t i o n a l activities Special i s s u e of t h e International Journal of Educational Research 12, no The A u t h o r JAAP SCHEERENS is a professor of e d u c a t i o n at the University of Twente He is p r e s e n t l y c h a i r m a n of the e d u c a t i o n a l a d m i n i s t r a t i o n division a n d d i r e c t o r of t h e C e n t r e for Applied R e s e a r c h in E d u c a t i o n His r e s e a r c h i n t e r e s t s are in t h e field of o r g a n i z a t i o n a l effectiveness a n d e d u c a t i o n a l evaluation