CHAPTER 2 EXAMINING PUPIL ATTAINMENT AND PUPIL PROGRESS WITHIN THE
2.3 An Overview of School Effectiveness Research
The first school effectiveness studies were of the input-output type. These studies were driven by a rejection of the assertions made by Coleman et al. (1966) and by Jencks et al. (1972) that pupil achievement is more strongly associated with social determinants rather than the more malleable school factors. The study by Coleman employed regression analysis that could not discriminate between the individual level of the pupil and the group level of the school. Besides mixing levels of data, Coleman also included school factors that were not very strongly related to achievement. Factors such as pupil expenditure, school facilities and number of library books. In spite of these limitations and the conclusion that schools do not influence pupil achievement, Coleman found that 5% to 9% of the variance between schools was accounted for by
school factors. Ironically, this constituted a first benchmark as to the effects of schooling for pupil achievement (Daly, 1995). Other studies such as those by Hauser (1971) and Hauser et al. (1976) concluded that the variance in pupil achievement between schools was in the 15% to 30% range. However, after controlling for the contribution of socio-economic factors, only 1% to 2% of the variance was accounted for by schools.
Input-output studies, also known as education-production in function studies (Brown &
Saks, 1986; Coates, 2003), such as those conducted by Mayeske et al. (1972), had serious methodological limitations due to issues of multicollinearity. These statistical issues not only plagued these early school effectiveness studies but also studies by Coleman (1966) and Hauser et al. (1976). In spite of these limitations, Mayeske et al.
(1972) found that 37% of the variance was between schools and that this was accounted for by pupil and school variables. This ―original input-output paradigm‖ (Teddlie &
Reynolds, 2000:4) also proved limited because it did not include measures, that were better related to pupil outcome, such as school climate and school processes (Averch et al., 1971).
The inclusion of variables that measured school processes and the inclusion of additional pupil outcome variables led to the second stage of school effectiveness research characterized by input-process/product-output studies. Variables such as teacher characteristics (Hanushek, 1986), human resource characteristics (Summers &
Wolfe, 1977), teacher behaviours (Murnane, 1975) and school climate (Brookover et al., 1979) were now included. Initially, such studies focused in dispelling the mistaken belief that schools made little difference for pupil achievement. Such studies therefore focused in researching conditions in primary schools associated with children from disadvantaged socio-economic backgrounds. Weber (1971) elaborated four case studies of four inner-city schools. This highlighted the importance of school processes such as leadership, high expectations, a good school climate and evaluation of pupil learning.
The inclusion of pupil level data that was now associated with specific teachers was an important development of later input-process/product-output studies. Teddlie and Reynolds (2000:7) explain how this ―emphasized input from the classroom (teacher)
level, as well as the school level; and it associated student-level output variable with student-level input variables, rather than school-level input variables.‖ Research by Summers and Wolfe (1977) utilized datasets in which teacher input variables were associated with pupils taught by teachers. School level inputs, including the characteristics of the specific teachers were also included. Together the school and the teacher inputs explained 25% of the variance in gain scores achieved by pupils.
Findings from such studies also indicated that variables related to school expenditure, such as teacher experience and teacher salary, did not demonstrate a consistent effect for pupil achievement (Hanushek, 1986). However, qualities associated with pupil, teacher and head teacher resources such as pupils‘ sense of control of their environment, head teachers‘ evaluations of teachers, quality of teacher education and teachers‘ high expectations for pupils were significantly associated with pupil achievement (Murnane, 1975; Summers & Wolfe, 1977).
Two important advances of input-process/product-output studies concerned the inclusion of psychosocial and school climate measures (Brookover et al., 1979) and the realization as to the importance of tests used to assess pupil achievement. In the Brookover et al. (1979) study, additional measures included pupils‘ sense of academic futility and self-concept, teacher expectations and academic/school climate. Brookover et al. (1979) examined the relationship between school climate variables, school level variables that referred to pupils‘ socio-economic status, racial composition of the school and the mean outcomes achieved by pupils at school. At this stage, Brookover et al. (1979) still had to grapple with serious issues of multicollinearity. For example, when socio-economic status and percent white were included first in the regression model, school climate only accounted for 4.1% of the school level variance in pupil achievement. When school climate was entered first the same two variables now accounted for 10.2% of the school level variance. When school climate, pupils‘ sense of academic futility and pupils‘ sense of control were entered first this explained approximately half of the school level variance. Research conducted during this stage also highlighted the importance regarding the choice of test to assess pupil achievement (Madaus et al., 1979). On tests that were curriculum specific, the variance between classrooms stood at around 40% (average of various tests). Madaus et al. (1979) indicated that classroom factors explained a larger proportion of the variance unique to classrooms on curriculum specific tests (17%) than standardised tests (5%). Issues of
multicollinearity (Teddlie & Reynolds, 2000) and the lack of standardised measures of pupil achievement (Brimer et al., 1978) led researchers to focus in examining differences in schools serving disadvantaged areas.
The focus on equity and schooling led to the development of the input-process/product- output with school improvement model. At this third stage, proponents such as Edmonds (1979) were not merely content in describing the effects of effective schools.
They also wanted to create effective schools, particularly for children from poorer urban areas. Research about effective schools (Edmonds, 1979; Lezotte & Bancroft, 1985; Weber, 1971), led to the development of the five factor model that identified leadership, vision, school climate, high expectations and the ongoing assessment of pupils as correlates of effective schools. These studies focused in examining the achievement outcomes of pupils from low socio-economic backgrounds. This led to much criticism about the sampling methods employed in these studies (Good &
Brophy, 1986; Ralph & Fennessy, 1983). Wimpelberg, Teddlie and Stringfield (1989) argued that this highlighted the importance of the school context as an issue for further examination.
The inclusion of variables associated with context factors led towards the normalization of the science of school effectiveness research (Teddlie & Reynolds, 2000) and its importance highlighted by Scheerens (2004:1):
The major task of school effectiveness research is to reveal the impact of relevant input characteristics on output and to ―break open‖ the black box in order to show which process or throughput factors ―work‖, next to the impact of contextual conditions. Within the school it is helpful to distinguish a school and a classroom level and, accordingly, school organizational and instructional processes.
Studies now could explore effects across different schools with different contexts instead of sampling schools with similar contexts (Teddlie et al., 1985, 1990). The input-context/process-output model was established by advances in statistical techniques that were able to measure more accurately the multilevel effects of schooling in respect of the hierarchical structure of the data. More sophisticated forms of multivariate analyses also facilitated the examination of factors associated with the differential effectiveness of schools. More recent developments in structural equation modelling have strengthened statistical approaches to ascertain the structural, as
opposed to the face validity, of constructs that undergird educational processes. The input-context/process-output model is still an important tool for school and educational effectiveness researchers. Increased recognition regarding the utility of mixing, combining and integrating research perspectives and approaches has meant that the input-context/process-product model has been developed and consolidated through studies that utilise both quantitative and qualitative approaches. Studies such as the Effective Provision of Preschool Education Project (Sylva et al., 1999, 2004) and the International School Effectiveness Research Project (Reynolds et al., 2002).