INTRODUCTION
The Centre for Program Evaluation (CPE) at the University of Melbourne, in collaboration with Shelby Consulting and Murdoch University, was commissioned by the Western Australian Department of Education to evaluate the Independent Public Schools (IPS) initiative Now in its fourth year, the IPS initiative aims to enhance school autonomy and decrease bureaucracy within the Western Australian public school system Additionally, it seeks to foster stronger community engagement through varied accountability processes while continuing to support schools within the public framework.
This executive summary provides a high-level summary of the key findings of the evaluation, which was commissioned in September 2011 There were four key evaluation questions:
1 What are the effects of the Independent Public Schools initiative on participating schools?
2 What effect has the initiative had on the public school system overall?
3 Are there any issues that are hampering the efficiency or effectiveness of the Independent Public Schools initiative?
4 What advice can be provided to guide future implementation of the IPS initiative?
CONTENTS
EVALUATION DESIGN AND METHODS
The evaluation employed a comprehensive framework that integrated various quantitative and qualitative data sources, ensuring a robust stakeholder engagement process This approach emphasized the importance of readiness and adoption, which are influenced by several key factors.
• capabilities of the leadership team,
• levels of support from the system and the community
The evaluation of the IPS initiative revealed several significant impacts, highlighting the principal's role as a key change agent The transition period proved crucial, as it was during this time that principals acquired the essential resources, information, and skills needed to initiate and implement the initiative effectively The findings are summarized in the following sections, based on data collected from the Department of Education, surveys of principals, interviews with key stakeholders, and comprehensive school site visits.
EVALUTION FINDINGS
The implementation of the IPS initiative in Western Australia has been largely positive, with most principals supporting its concept Many IPS principals report that, even in the early stages, the initiative has significantly improved their schools' operations and provided access to additional benefits, ultimately promising better outcomes for the entire school community However, there are some challenges and dissenting opinions, particularly regarding specific issues.
• increased workload, mainly in the transition to becoming IPS; and
The establishment of a network of schools with distinct advantages over traditional institutions marks a significant shift in education The choice to pursue Independent Public School (IPS) status represents a substantial commitment for many schools, influenced by various factors, including the desire for greater autonomy and enhanced educational outcomes.
Principals' willingness to apply for the IPS program is driven by their recognition of its potential benefits, while those who opted not to apply cited various reasons, such as a lack of perceived advantages for their school, contentment with existing conditions, limited capacity for change, or philosophical disagreements with the initiative.
The IPS initiative is progressing well, though schools vary in their self-belief, autonomy, support, and ability to adopt necessary flexibilities Principals demonstrate differing levels of these attributes, with stronger attributes correlating to a deeper implementation of the initiative.
The principals identified key factors essential for the continued success of the IPS, highlighting the importance of effective leadership They emphasized that educational leaders must not only excel in their roles but also foster strong relationships with teachers and the community.
Support for principals should be customized to align with their individual capabilities and the specific context of their schools This is especially important for less experienced and aspiring principals, as future cohorts in the IPS program are anticipated to include a greater number of these individuals.
The principal's mindset plays a pivotal role in fostering the adoption of autonomy, especially during the transition period This phase is crucial, as it allows principals to collect essential resources, information, and skills needed to initiate and effectively implement the initiative.
In the early stages of the initiative, IPS schools report high satisfaction levels, with principals feeling empowered to enhance their leadership and better address students' needs The combination of autonomy in staff selection and budget management has motivated them to take on greater responsibility As a result, principals experience significant changes in their roles, becoming more accountable and invested in their schools' success This newfound autonomy fosters a mindset of entrepreneurship and engagement, prompting principals to actively monitor classroom dynamics and leverage their increased authority to maximize positive outcomes within IPS classrooms.
The role of the teacher is a crucial aspect of the IPS initiative, with principals noting its positive influence on many educators Many teachers have shown increased motivation and energy to implement changes and embrace innovative practices that enhance teaching and learning While some teachers enthusiastically report benefits such as improved collaboration, additional resources, and tailored professional development, others express concerns regarding the impact of IPS on their workload and career trajectories.
One significant advantage of Independent Public Schools (IPS) is the enhanced control they have over staffing, allowing them to recruit personnel that align with their specific school context and requirements Additionally, IPS have the discretion to accept or decline redeployed staff It's important to note that while redeployees may not be of inferior quality, they might not meet the unique needs of IPS schools.
Most teachers in the IPS initiative reported feeling more professional, accountable, and in control of their careers, leading to increased self-worth However, these findings primarily reflect perceptions and attitudes at this stage of implementation In contrast, other public schools did not observe significant changes within their communities and expressed relative satisfaction with their existing status.
The IPS initiative has fostered a significant cultural transformation within schools, where strategic thinking has become more prevalent This shift is particularly noticeable in school communities that fully embrace the vision promoted by their principals, demonstrating a collective commitment to the IPS goals.
Schools in the IPS initiative have experienced heightened community engagement and accountability through their school boards, leading to enhanced school profiles and increased pride among both students and parents Many parents perceive the schools as more welcoming, while collaboration and resource sharing among schools have also improved The IPS initiative shifts decision-making from the central office to individual schools, requiring a supportive role from the central office and prompting a realignment of departmental roles, culture, systems, and resources This transition has occurred at varying paces, with changing priorities driving comprehensive reviews of processes and policies to better support all schools, including staff deployment.
Equity emerged as a significant theme, especially among principals outside the IPS initiative, highlighting the need for targeted placements in regional and indigenous schools However, the evaluation revealed no evidence indicating that the IPS initiative differentially affected very remote, remote, regional, and metropolitan schools Additionally, some principals who either chose not to apply or were unsuccessful in their IPS applications expressed concerns about potential inequities arising from the initiative.
The emergence of a 'two-tiered' system in Western Australia raises questions about the potential negative implications of diverse school operation methods Notably, schools that applied for Independent Public School (IPS) status but were not accepted reported feelings of failure, which adversely affected staff morale and community confidence Despite these concerns, evidence indicates no significant differences in outcomes between schools that were granted IPS status and those that were not.
EVALUATING THE IPS INITIATIVE
The evaluation of the IPS initiative utilized a staged approach, beginning with stakeholder engagement, program clarification, and evaluation design focus Key activities included meetings, workshops, the development of program logic validated by a literature review, and analysis of existing Department of Education data on IPS characteristics The second stage comprised two data collection phases, featuring in-depth site visits, stakeholder interviews, surveys across all Western Australian public schools, and secondary data collection from the Department of Education The final stage revisited the program logic based on current school dynamics and central services, integrating and analyzing evaluation data to formulate overall improvement recommendations This report summarizes the information gathered throughout all evaluation stages.
The following section outlines the evaluation methodology utilised, the objectives and questions used, and the methods of data collection and analysis
The implementation of the evaluation is underpinned by the framework developed by the Centre for Diseases Control and Prevention (CDC&P),
The "Framework for Program Evaluation of Public Health Initiatives" developed by CDC&P in 1999 adapts a widely recognized evaluation model used in educational and social contexts This framework outlines a comprehensive approach consisting of six essential steps: engaging stakeholders, describing the program, and focusing the evaluation design Figure 1 illustrates this high-level framework, emphasizing its structured methodology for effectively assessing public health initiatives.
The evaluation process consists of six key steps that are implemented at different stages of a three-part framework, ensuring data collection, justification of conclusions, and effective sharing of lessons learned This framework is supported by established standards that uphold the accuracy, reliability, and validity of the evaluation findings For a detailed explanation of these standards, please refer to Appendix B.
The evaluation of the IPS initiative employed a mixed-methods approach, integrating both qualitative and quantitative data collection and analysis This methodology leverages the strengths of each method to enhance the study design and effectively address evaluation questions As noted by Green (2007), a mixed methods approach is "generative and open," aiming for a deeper understanding of complex social dynamics This strategy allows for the tailored use of various data collection techniques, capturing diverse perspectives from key stakeholders in the IPS initiative and facilitating triangulation to comprehensively address the main evaluation questions.
The WA Department of Education initiated an evaluation of the IPS initiative to assess its effectiveness before further expansion The study was designed to achieve specific objectives, focusing on examining and reporting the initiative's impact.
Stage 3 Testing Integration Analyses Recommendations
• The implementation of the Independent Public Schools initiative and whether there are opportunities for its improvement
• The impacts of the initiative on the effectiveness and efficiency of Independent Public Schools and public confidence in the broader public school system
• The extent to which the experience of schools demonstrates that the policy objectives of the initiative are being met
To guide the collection, analysis and reporting of information on each of these objectives, four key evaluation questions were developed and agreed upon:
1 What are the effects of the Independent Public Schools initiative on participating schools?
2 Are there any issues that are hampering the efficiency or effectiveness of the Independent Public Schools initiative?
3 What effect has the initiative had on the public school system overall?
4 What advice can be provided to guide future implementation of the IPS initiative?
Using a mixed-methods strategy, the evaluation study takes a multi-pronged approach to data collection by collecting information appropriate to each evaluation question
In Stage 1, the information required was sought from:
• In depth discussions with DoE senior staff and school principals to develop a program logic for the implementation of the initiative
• Program documents sourced from the Department and the schools, such as: guidelines, reports, plans, IPS agreement templates, school applications/submissions, principal business plans, school delivery performance reports etc
• Interviews with persons who had expertise in the practice and theory of school autonomy
In Stage 2, information was sought from:
The analysis utilized secondary data from the Department and schools, including student academic outcomes, enrolment trends, attendance and behaviour records, staff mobility, and financial data This information was gathered over a five-year period, starting before the IPS initiative and concluding at the end of 2012, allowing for meaningful comparisons between IPS and other public schools.
• Surveys with principals of IPS and other public schools
• Interviews with key stakeholders with a particular focus on staff in central and Regional offices, and representatives of professional and staff associations
• In-depth visits to a number of IPS to interview staff and parents.
Using a mixed-methods strategy, the evaluation study takes a multi-pronged approach to data collection by collecting information appropriate to each evaluation question
In Stage 1, the information required was sought from:
• In depth discussions with DoE senior staff and school principals to develop a program logic for the implementation of the initiative
• Program documents sourced from the Department and the schools, such as: guidelines, reports, plans, IPS agreement templates, school applications/submissions, principal business plans, school delivery performance reports etc
• Interviews with persons who had expertise in the practice and theory of school autonomy
In Stage 2, information was sought from:
Secondary data from the Department and schools includes student academic outcomes, enrolment trends, attendance and behaviour records, teacher and staff mobility, as well as financial and budgeting information This data was collected over a five-year period, starting before the IPS initiative and concluding at the end of 2012, allowing for meaningful comparisons between IPS and other public schools.
• Surveys with principals of IPS and other public schools
• Interviews with key stakeholders with a particular focus on staff in central and Regional offices, and representatives of professional and staff associations
In-depth visits to a number of IPS to interview staff and parents.
Between 2009 and 2012, the Department of Education (DoE) provided comprehensive data from all 770 public schools in Western Australia, which was compiled into a single SPSS database This database includes crucial demographic information such as school names, regions, socioeconomic indices (SEI), percentages of Aboriginal and Torres Strait Islander students, and various geographic codes It also encompasses student attributes, including total enrollment numbers, ATAR statistics, and attendance risk categories Additionally, the performance metrics feature median ATAR scores and National Assessment Program for Literacy and Numeracy (NAPLAN) results for Years 3, 5, 7, and 9 in key areas like Numeracy, Reading, Writing, Spelling, and Grammar, alongside semester enrollment figures and exam scores for the WA Society and Environment and Science subjects, as well as the classification of schools based on their independent status.
An online survey was conducted among principals of public schools in Washington State to investigate their reasons for applying or not applying to become Independent Public Schools (IPS) The survey aimed to gather feedback on transition processes, principal satisfaction, teaching staff, school operations, and departmental support Developed through a thorough literature review and stakeholder discussions, the survey included four versions tailored for different principal groups To ensure validity and appropriateness, feedback was collected from stakeholders and an expert panel, and a pilot test was conducted to enhance readability and quality The survey's design and analysis aimed to capture a comprehensive range of perspectives on the IPS initiative from all public schools.
On October 30, 2012, a survey was launched through the Murdoch University survey system, targeting principals of all public schools in Western Australia Individual email invitations were sent to 772 principals, with reminders issued on November 5 and November 19, 2012 To further enhance participation from non-Independent Public Schools, an additional reminder was sent on November 29, 2012 This effort resulted in a strong response rate, with 590 principals participating, equating to 76% of the total.
Consultations were held with representatives from the Department's central and regional offices, as well as union and association representatives, to assess the impact of the IPS initiative on the public school system and its future implementation A total of 24 interviews were conducted, including 10 face-to-face interviews in metropolitan areas and 9 phone interviews in regional areas, with detailed notes taken throughout the process Participation included 22 central office staff, 10 regional staff, and 13 representatives from staff and parent unions and associations.
To understand how schools are adapting to the Independent Public Schools (IPS) initiative, the evaluation team selected a diverse group of 13 schools from three IPS cohorts for site visits These visits included interviews with principals, staff, parents, and school board members to gather insights on innovative approaches to increased autonomy The selected schools, comprising six primary, four secondary, one district high school, and two education support settings, were chosen to represent a variety of geographical locations and contexts Additionally, the selection process considered the Socio Economic Index to ensure a broad range of schools was included Notably, schools that began participating in the IPS initiative in 2013 were excluded from the selection, as they had not yet had adequate time to implement the initiative's key elements.
Site visits were carried out in 13 schools over one or two days, with the three 2012 cohort schools being visited in both Terms 1 and 4, while the ten 2010 and 2011 cohort schools were visited once during Term 2 During these visits, interviews were conducted with various stakeholder groups, with smaller schools allowing for feedback from most teachers, whereas larger schools had a limited number of teachers participating in the interviews.
During visits to selected schools, individual and group interviews were conducted with various stakeholders, including principals, deputy/associate principals, registrars, teaching staff, board members, and parents Four tailored interview schedules were created for each stakeholder group, focusing on the core issues related to the four key evaluation questions Notes from these interviews were recorded and subsequently analyzed using Excel.
A total of 121 interviews were conducted with 289 participants, comprising 13 principals, around 25 deputy principals, 10 registrars, 130 teachers, 45 board members, and 55 parents The precise count for each group is not clearly defined due to the presence of various stakeholders within some groups.
1 some stakeholders could be placed in more than one group, for example parent and board member
FINDINGS
This section outlines the key findings of the evaluation, organized according to the four evaluation questions The results have been cross-verified using the data sources mentioned in the Methodology section Unless specified, references to principals are based on findings from the online survey of public school principals, with results categorized by relevant principal and school type.
The article highlights that illustrations and patterns derived from case studies are integrated throughout the text It also clarifies that when a school is referenced, it reflects a consensus among various stakeholders within that institution; if not, the specific stakeholder group is identified.
Feedback from essential stakeholders, including staff from regional offices, central office personnel, and representatives from school staff associations, unions, and parent organizations, has been documented in this text.
Feedback from IPS site visits encompasses insights from principals, deputy/associate principals, business managers, teachers, parents, and board members When there is consensus among these stakeholders, it is recognized as collective feedback from the site visit schools.
Findings from the analysis of departmental and public school data about students, teachers, staff and finances are cited as “secondary data.”
2.4.1.WHAT ARE THE EFFECTS OF THE IPS INITIATIVE ON PARTICIPATING SCHOOLS?
Survey respondents across various schools indicated that their main motivation for applying to become Independent Public Schools (IPS) was the ability to choose staff who align with student needs and the school's ethos Additionally, they valued the flexibility and autonomy to make decisions that enhance school effectiveness and reputation, as well as the capability to manage budgets While the support available for IPS and related benefits played a moderate role in their decision, particularly for later intakes, the prospect of additional work was not seen as a significant obstacle.
Figure 2: PrincipalSurvey- Reasonsforapplying to become IPS
Public school principals recognize that Independent Public Schools (IPS) have advantages in hiring suitable staff and utilizing resources flexibly, despite facing a heavier administrative burden Site visits revealed that schools tailored their choices to meet specific goals, demonstrating that flexibility extends beyond resource allocation to include the ability to adapt strategies based on outcomes For instance, one school opted to reduce class sizes and enhance classroom assistance, while another focused on providing support to struggling students outside their regular classrooms Additionally, some schools prioritized increasing teacher professional development, whereas others scaled it back to address more urgent needs, showcasing the diverse approaches schools take to achieve their objectives.
We have enhanced our staffing and purchasing flexibility by hiring additional office staff and extending the hours of education assistants A part-time position has transitioned to full-time to support project areas, allowing us to allocate more funding for additional resources for certain students.
To optimize staffing efficiency, it's essential to assign roles appropriately, such as hiring clerical workers for administrative tasks instead of burdening teachers with clerical duties at their salary level By replacing some teaching positions with clerical staff, we can streamline operations, reduce the number of teachers needed, and enhance overall productivity within the educational environment.
Our approach emphasizes the flexibility to quickly adapt resources and strategically select specialized staff for specific roles We successfully established a team of non-timetabled teachers to assist in the classroom, comprising 1.4 teachers, a 1.0 team leader, and 2 FTE education assistants dedicated to enhancing literacy and numeracy This additional support, funded by savings from relief and a low socioeconomic grant, was preferred over reducing class sizes and hiring more support staff.
Our initiative aimed not only to enhance NAPLAN results but also to ensure the consistency of our educational programs Often, we received funding for a program, but once we achieved improved results, that funding was withdrawn.
We have successfully implemented programs that allow us to hire staff dedicated to enhancing student support For instance, our new upper school initiative allocates time for an educational assistant to work with struggling students through guided reading, providing essential support for special needs and weaker learners.
IPS faced significant challenges due to administrative changes, particularly in adapting to new flexibilities like the one-line budget and staffing profile Additionally, these changes resulted in an increased workload for both administrative and leadership personnel.
Positive changes in our school have emerged from the flexibility of the one-line budget and the recruitment of dedicated staff members However, these improvements have also led to an increased workload for our team.
Factor analysis of the principal survey revealed notable differences in perceptions of change among IPS intake groups Principals from the first intake reported a stronger agreement on improvements in teaching, resources, leadership roles, and community engagement within their schools This suggests that later intake principals may still be navigating transitional challenges, indicating that satisfaction and perceived change may grow as schools adapt to operating as an IPS Alternatively, it could reflect varying attitudes among different principal cohorts.
Earlier intake IPS have benefited from increased time to leverage their autonomy in realizing their school vision This aspect is crucial for evaluating outcomes, as a well-defined school vision is essential for establishing goals that align with the needs of the school community and serve as a foundation for accountability.
APPENDIX F: COMPONENTS OF MICHAEL BARBER DELIVEREOLOGY MODEL
Component 1: Develop a foundation for delivery This requires three steps:
1 Defining and clarify aspirations which in this case is knowing and valuing the impact the school has on the learning of the students This involves being clear on the success criteria from any intervention (such as levels of progress, retention in the school system, convincing parents to enrol in public schools, achievement outcomes)
2 Building the delivery unit This is not about accountability methods or external imperatives but about a commitment to action to achieve the aspiration The unit is not necessarily the teachers or school leaders but a small group within each school (but can be shared across schools) responsible to ensuring delivery Barber recommends the unit be small, reside outside the school hierarchy (as they must influence them as well), and have time and sufficient resources to ensure delivery
3 Establishing a guiding coalition so as to remove barriers to change, have influence to support the unit’s work at crucial moments, and can provide counsel and advice - aiming to helping ensuring maximum probability of success The coalition is essential for developing the trust that is so important in school change
Component 2: Understand the delivery challenge
1 Evaluate past and present performance What is the evidence most indicative of performance (or whatever other outcomes); how dependable and credible is this evidence to the teachers, school leaders, students, and parents (and whomever else); what are the target indicators; what are the correlates of these target indicators, and the indicators of unintended consequences Does the school share a program logic of how learning occurs in this school?
2 Understand drivers of performance and relevant systems activities Do all in the school understand the drivers of student learning, are they drivers that have some control over, are there mindsets that inhibit the impact we need to have on learning (e.g., “Give me bright students and I can achieve”, “But it is all about poverty and the home”, “If they do not come to class prepared that is not my fault”, “We know ‘group x’ are underachievers and do not value education”) Or do the teachers in the school see themselves as change agents, which all students can learn, that they can have marked positive impacts on all students, that they are tasked primarily with knowing their impact on students
1 Determine the reform strategy Strategy is primarily the role of the school leaders, and the role of the delivery leader is to inform this strategy There is no magic formula, programs, or quick ways to have systematic, genuine, and identifiable impacts on student learning It requires all in the school wanting to have this impact, adopting theories of change that allow the best ways of getting there, building capacity, capability, and culture, and evaluating strategies
2 Set targets and trajectories Setting challenging and defensible targets is critical for all levels in the school – from the front office, school leaders, teachers, and students This involves targets at each student level and work forward, and avoiding the notion of ‘averaging” The flaw of the average is that change can affects only some students and many others are left behind. Decide on the trajectories to attain these targets, and then devise systems to evaluate the success in this trajectory
3 Produce delivery plans The planning is everything, it is a work in progress, and it requires revision, rework, and realistic support This is where school leadership comes to the fore
Establishing routines is essential for driving and monitoring performance, ensuring that everyone understands their roles in achieving targets By planning stock takes and maintaining transparency in progress reporting, teams can effectively address challenges while fostering a culture of trust This proactive approach not only exceeds expectations but also aligns efforts with the mission at hand.
Addressing problems promptly and thoroughly is crucial Acknowledging the reality of the issue for the individual affected is essential, followed by a reassessment of its priority and severity It is important to evaluate the urgency of resolving the problem in relation to achieving the target delivery.
To sustain and continually build momentum, it is essential to establish high-quality routines, embrace problem-solving, and acknowledge evidence of success This requires persistence in the face of distractions, effective management of resistance to change, and a commitment to challenging the status quo Celebrating successes along the way is crucial for maintaining motivation and driving progress.
Component 5: Develop, identify and esteem success
To foster a culture of continuous improvement in schools, it is essential to implement systems that regularly assess the progress of students, teachers, and school leaders towards their goals This reflective practice encourages problem-solving and appreciation rather than blame, ultimately uniting educators, students, and families in their commitment to valuing and enhancing the learning experience.