Independent Statewide Evaluation of High School After School Programs May 1, 2008-December 31, 2011

Independent Statewide Evaluation of High School After School Programs May 1, 2008-December 31, 2011 CDE4/CN077738/2011/Deliverable - January 2012 Denise Huang, Jia Wang and the CRESST Team CRESST/University of California, Los Angeles National Center for Research on Evaluation, Standards, and Student Testing (CRESST) Center for the Study of Evaluation (CSE) Graduate School of Education & Information Studies University of California, Los Angeles 300 Charles E Young Drive North GSE&IS Bldg., Box 951522 Los Angeles, CA 90095-1522 (310) 206-1532 Copyright © 2012 The Regents of the University of California The work reported herein was supported by grant number CN077738 from California Department of Education with funding to the National Center for Research on Evaluation, Standards, and Student Testing (CRESST) The findings and opinions expressed in this report are those of the authors and not necessarily reflect the positions or policies of California Department of Education EXECUTIVE SUMMARY For nearly a decade, after school programs in elementary, middle, and high schools have been Federally funded by the 21st Century Community Learning Centers (21st CCLC) The 21st CCLC has afforded youth living in high poverty communities across the nation with opportunities to participate in after school programs The California Department of Education (CDE) receives funding for the 21st CCLC and also oversees the state funded After School Education and Safety (ASES) program The high school component of the 21 st CCLC program is called the After School Safety and Enrichment for Teens (ASSETs) program Similar to the ASES program, the ASSETs program creates incentives for establishing locally driven after school enrichment programs that partner with schools and communities to provide academic support and safe, constructive alternatives for high school students outside of the regular school day, and assists students in passing the California High School Exit Examination (CAHSEE) This report on the ASSETs program, as well as the companion report on the 21 st CCLC and ASES programs, is submitted as part of the independent statewide evaluation called for in California Education Code (EC) Sections 8428 and 8483.55(c) The following evaluation questions were designed by the Advisory Committee on Before and After School Programs and approved by the State Board of Education (per EC Sections 8421.5, 8428, 8482.4, 8483.55(c), and 8484): • What are the similarities and differences in program structure and implementation? How and why has implementation varied across programs and schools, and what impact have these variations had on program participation, student achievement, and behavior change? • What is the nature and impact of organizations involved in local partnerships? • What is the impact of after school programs on the academic performance of participating students? Does participation in after school programs appear to contribute to improved academic achievement? • Does participation in after school programs affect other behaviors such as: school day attendance, homework completion, positive behavior, skill development, and healthy youth development? • What is the level of student, parent, staff, and administration satisfaction concerning the implementation and impact of after school programs? • What unintended consequences have resulted from the implementation of the after school programs? Methodology and Procedures To address the evaluation questions, a multi-method approach combining qualitative and quantitative research methodologies was used This included longitudinal administrative data collected by the CDE and school districts (secondary data), as well as new data collected by the evaluation team (primary data sources) The secondary data sources were intended to provide student-level information pertaining to after school program participation, demographics, grade progression, mobility, and test score performance The primary data sources – surveys, focus groups, interviews, and observations – were intended to provide detailed information about the after school program characteristics and operations Four study samples were used to address the evaluation questions Sample I included all schools in the STAR database with an after school program funded through the ASSETs program The purpose of this sample was to examine statewide after school attendance patterns and estimate effects of participation on academic achievement Sample II included a sub-sample of 30 districts to examine behavioral outcomes from the district-collected data Sample III included all agencies and program sites that completed a yearly profile questionnaire Finally, Sample IV consisted of 20 randomly selected program sites The purpose of these final two samples was to collect site-level information about program structures and implementations Due to the longitudinal nature of the evaluation, Samples I and III changed every year depending on the actual after school program participation for the given year Key Findings Currently over 90 grantees and more than 300 schools receive funding through the ASSETs program Because of this, it was important to examine similarities and differences in program structures and styles of implementation The following provides the key findings concerning these critical components: Goal Setting, Activities, and Evaluation • Most grantees set goals that closely aligned with the ASSETs guidelines concerning academic support, as well as program attendance Somewhat less emphasized were behavioral goals • Site coordinators often aligned activities more closely with the program features they personally emphasized than with the goals set for them by the grantees • In alignment with the ASSETs guidelines, sites reported offering both academic and non-academic forms of enrichment Overall, the most commonly offered activities were academic enrichment, arts/music, homework assistance, physical fitness/sports, recreation, and tutoring iv • While specific knowledge of the principles of youth development (PYD) was limited, staff at many of the Sample IV sites practiced the philosophies of PYD in their interactions with students • Grantees utilized a variety of data sources and stakeholders when conducting evaluations for goal setting and the assessing of outcomes Stakeholders whose feedback was sought normally included program staff, site coordinators, and/or day school administrators The most common data sources were state achievement data, after school attendance records, site observations, and surveys • Stakeholders at most of the Sample IV sites agreed that student and after school staff satisfaction were monitored In addition, the majority of site coordinators reported that parent and day school staff opinions were sought Resources, Support, and Professional Development • Overall, the Sample IV sites had adequate access to materials and physical space at their host schools Despite this, the type of physical space provided was not always optimal for implementation of the activities For example, some of the staff members reported that they had to use small spaces, move locations on some days, or conduct activities off campus • Staff turnover was an ongoing and predominant problem These changes primarily involved site staff, but also involved changes in leadership at about one-quarter of the sites • Site coordinators tried to create collaborative work environments and reported using different techniques to recruit and retain their staffs The most common technique reported for recruitment was salary, while the most common technique for retention was recognition of staff • Site coordinators and non-credentialed site staff were given opportunities for professional development These opportunities normally took the form of trainings, workshops, and/or staff meetings • Organizations that commonly serve as grantees, such as districts and county offices of education, were the primary providers of professional development • The most common professional development topics – classroom management, behavior management, and student motivation – focused on making sure that staff were prepared to work directly with students • The most commonly voiced barriers involved the direct implementation of the activities For example, participants expressed concern about funding, access to activity specific materials, and the appropriateness of physical space Difficulty in recruiting well-qualified and efficacious staff members was also of great concern to some stakeholders v Student Participation • Each year less than 5% of all site coordinators reported that they could not enroll all interested students Despite this, about one-fifth of the site coordinators used waiting lists to manage mid-year enrollment • Site coordinators utilized teacher referrals and other techniques to actively recruit students who were academically at-risk, English learners, and/or at-risk because of emotional/behavioral issues • Site coordinators used flyers and had after school staff public relations to recruit the general population of students Because of this it was not surprising that one of the top reasons Sample IV parents enrolled their children was because their children wanted to attend Having interesting things to and spending time with friends were the most common reasons offered by the Sample IV students • With a population of students who were old enough to make their own decisions and care for themselves after school, it was not surprising to find that studentfocused barriers, such as student disinterest or the need to work after school, were more predominant than structural barriers involving lack of resources • Correlations revealed that sites with more student-focused or total barriers to recruitment might be less able to fill their programs to capacity • While most parents reported that their children attended their after school program at least three days per week, the average parent also indicated that they picked their child up early at least twice per week Local Partnerships • Level of participation at the after school sites varied by the type of partner Over half of the sites had local education agencies (LEAs) help with higher-level tasks such as program management, data collection for evaluation, and the providing of professional development In contrast, during most years less than one-third of the parents or other community members filled any specific role Furthermore, during the final year of data collection, providing goods/supplies was the most common role for parents and other community members • Stakeholders at program sites with strong day school partnerships perceived positive impacts on program implementation, academic performance, and academic goals In contrast, partnerships with other local organizations were perceived as enhancing positive youth development Sample IV sites seemed to emphasize parent communication More specifically, both parents and site coordinators reported that parents were kept informed and were able to give feedback about programming In contrast, only one-fifth of the Sample IV parents reported that they actively participated at their child’s site When parents did participate, they tended to attend special program events or parent meetings Longitudinal analyses revealed that the ASSETs programs had some minor positive and neutral effects More specifically, when comparing participants to non-participants, small vi positive effects were found concerning English-language arts assessment scores, while small to neutral effect was found on math assessment scores Furthermore, small positive to neutral effects were found for English language reclassifications, CAHSEE pass rates in Englishlanguage arts and math, and suspension In addition, students with any after school exposure were less likely to transfer schools or drop out of school They were also predicted to graduate at a higher rate than non-participants and showed small positive effects for school attendance When cross-sectional analyses were conducted for participation within a given year, further positive effects were found Key findings concerning general satisfaction and unintended outcomes are also presented: Academic Outcomes • Overall, students who attended ASSETs programs (grades 9-11) performed slightly better than non-participants did on their English-language arts and math assessment scores • Regular participants as well as frequent participants performed slightly better than non-participants did on the English-language arts and math parts of the CAHSEE Furthermore, frequent participants were slightly more likely than were the regular participants to pass the math part of the CAHSEE • English learners who were after school participants performed slightly better than non-participants on the CELDT This was true for both regular and frequent participants Behavioral Outcomes • Program sites that were observed as high in quality features of youth development impacted students’ positive perceptions of academic competence, socio-emotional competence, future aspirations, and life skills • When examining physical fitness outcomes, after school participants performed slightly better than non-participants In regards to most of the measures, the passing rate was largest for frequent participants Furthermore, significant subgroup results were found for all of the measures except body composition • Participation in an ASSETs program had a small positive effect on day school attendance • Frequent participants at the after school programs were found to be less likely to be suspended than students who did not participate at all Stakeholder Satisfaction • Sample IV stakeholders generally had high levels of satisfaction concerning their programs impact on student outcomes More specifically, all stakeholders felt that the programs helped students’ academic attitudes, cognitive competence, socioemotional competence, and future aspirations While staff and parents were also vii satisfied that their programs impacted academic skills, students who completed their survey generally had a neutral opinion about this outcome The exceptions involved students’ beliefs that their program was helping them to get better grades and better with homework • While stakeholders at all levels expressed general satisfaction with the programs, positive feelings were often highest among after school staff and parents In both instances, the quality of the relationships students developed with staff and their peers as well as the belief that students’ academic and emotional needs were being met were important factors Parents also expressed high levels of satisfaction concerning the locations and safety of the programs Unintended Consequences • Some of the program directors and principals felt that after school program enrollment and student accomplishments exceeded their expectations This suggests that when the after school programs cater to the needs and interests of the students, families, and communities, programs will be more appreciated, well attended, and achieve positive outcomes • The building of relationships was repeatedly mentioned by stakeholders as a positive, albeit unintended consequence of their after school programs Despite this, some stakeholders reported that funding cuts were impacting their ability to maintain staff, therefore creating potential negative effects to relationship building • Efficiency in the management of the after school program can either leverage up or down the level of communication and collaboration with the day school Effective management may result in unintended consequences such as motivation of day and after school staff to jointly promote the positive relationships with students and their families Recommendations In order to improve the operation and effectiveness of after school programs, federal and state policymakers, as well as after school practitioners should consider the following recommendations: Goals and Evaluation • When conducting evaluations, programs need to be intentional in the goals they set, the plans they make to meet their goals, and the outcomes they measure Furthermore, they should make efforts to build understanding and consensus across stakeholders • Evaluations of after school effectiveness should take into consideration variations in program quality and contextual differences within the neighborhoods • Government agencies and policymakers should encourage the use of research to inform policy and practice When conducting evaluations, programs need to be viii intentional in the goals they set, the plans they make to meet their goals, and the outcomes they measure • While academic assessments are commonly available, tested and validated instruments for measuring behavioral and socio-emotional outcomes are less common Since these two areas are commonly set as goals by grantees, the development of standardized measures would greatly benefit ASSETs programs with these focuses Policymakers should develop common outcome measures in order to measure the quality of functioning across different types of programs and different settings • During the independent statewide evaluation, the greatest response rates were obtained through online rather than on-site data collection Furthermore, the data provided valuable insight into the structures and implementations used across the state Therefore, the CDE should consider incorporating an online system as part of their annual accountability reporting requirements for the grantees Local Partnerships • Programs should consider inviting school administrators to participate in after school activities in order to improve communication and collaboration Conducting joint professional development can also provide an opportunity for after school and day school staffs to develop joint strategies on how to enhance student engagement and discipline, align curricula, and share school data • Sample IV site coordinators and site staff viewed parents as both assets and obstacles to their programs The negative consequences most mentioned by staff were a lack of support in working to improve students’ academic and behavioral performance Perhaps programs can gain buy-in by working with parents to develop consensus regarding expectations, discipline issues, and behavior management Through building psychological support for their program, staff members may indirectly be able to build active participation (e.g., volunteering, attending events) as well • Partnerships with local organizations such as government offices, private corporations, and small businesses generally have positive impacts on youth development By working together, after school programs and these organizations can work to provide space, activities, supportive relationships, and a sense of belonging for students In this way, students can be provided with positive norms for behavior including the ability to resist gangs, drugs, and bullying Therefore, government agencies should consider setting policies to facilitate the creation of these local partnerships Program Implementation • Sample IV students revealed that having interesting things to and getting to spend time with friends motivated them to participate in their ASSETs program In order to recruit and retain more students, programs can provide more learning activities that are meaningful to the students and in settings where they can communicate with their peers, and be engaged ix • During the Sample IV site visits, programs were consistently rated low concerning opportunities for cognitive growth In order to confront this issue, ASSETs programs should provide more stimulating lesson plans where students can have choices and participate in activities that develop their higher order thinking skills Staffing and Resources • Retaining staff is an essential component of quality programs Loss of staff not only effects relationships, but also creates gaps in the knowledge at the site In order to confront these issues, policymakers should further explore strategies for recruiting qualified staff and retaining them once they are trained • Most of the Sample IV sites had at least one stakeholder who was knowledgeable about the developmental settings described by the positive youth development approach Despite this, many more of the staff members were using these approaches Considering the impact of these settings on students’ perceived outcomes, programs can create more intentionality and further the benefits of these approaches by providing professional development opportunities to the frontline staff to get familiar with to the underlying principles and how these interrelationships affects youth development x All ASP Outcome N Estimate Frequent ASP (SE) N Estimate (SE) (0.243) 3,191 0.472 (0.238) (0.092) Advanced 5,467 0.357 Proficient 16,509 -0.134 (0.072) * 9,344 -0.046 Basic 19,234 0.276 (0.102) * 10,724 0.548 (0.117) ** Below Basic 20,542 0.279 (0.082) ** 11,388 0.481 (0.092) ** 8,526 0.278 (0.084) ** 4,773 0.386 (0.11) ** No 18,643 0.126 (0.076) 10,453 0.351 (0.083) ** Yes 51,635 0.214 (0.057) ** 28,967 0.375 (0.068) ** Far Below Basic National School Lunch Program Parent Education Level College 8,710 0.069 (0.107) 4,868 0.189 (0.12) Some College 11,011 -0.020 (0.103) 6,313 0.124 (0.127) High School 15,550 0.475 (0.107) ** 8,663 0.702 (0.121) ** Less Than High School 19,171 0.183 (0.102) 10,764 0.331 (0.117) ** Non Response 15,836 0.163 (0.071) * 8,812 0.379 (0.087) ** No 65,832 0.182 (0.051) ** 36,929 0.365 (0.061) ** Yes 4,446 0.348 (0.122) ** 2,491 0.437 (0.143) ** No 19,771 0.276 (0.098) ** 11,107 0.592 (0.11) ** Yes 50,507 0.161 (0.053) ** 28,313 0.292 (0.064) ** Special Education Status Title Status Note Participation effect estimates reflect percent change in likelihood of attaining healthy fitness zone, controlling for 2008-09 ELA CST scores Standard errors adjusted for school-level clustering ASP = After school participant * p < 0.05 ** p < 0.01 334 Table F11 Estimated Effect of After School Participation on Trunk Strength for 2009-10 (Matched Sample): High Schools All ASP Outcome ALL N Estimate Frequent ASP (SE) N Estimate (SE) 70,278 0.146 (0.044) ** 39,420 0.191 (0.066) ** Urban 44,449 0.115 (0.056) * 24,746 0.155 (0.071) * Rural 2,408 -0.038 (0.52) 1,694 -0.137 22,384 0.227 (0.069) ** 12,244 0.270 (0.128) * Advanced 9,108 0.097 (0.137) 5,213 0.098 (0.174) Proficient 15,868 0.079 (0.115) 8,822 0.114 (0.132) Basic 24,021 0.176 (0.079) * 13,541 0.188 (0.089) * Below Basic 12,925 0.164 (0.09) 7,168 0.288 (0.111) * 8,356 0.171 (0.094) 4,676 0.242 (0.126) English Learner 17,826 0.113 (0.08) 9,872 0.179 (0.115) English Only 25,651 0.090 (0.079) 14,561 0.153 (0.101) I-FEP 6,066 0.310 (0.134) * 3,334 0.391 (0.186) R-FEP 20,735 0.218 (0.095) * 11,653 0.232 (0.098) * Asian/Pacific Islander 7,050 0.084 (0.149) 4,019 0.021 (0.176) African American/Black 6,635 -0.010 (0.166) 4,040 0.056 (0.181) 47,528 0.191 (0.051) ** 26,301 0.275 Other 1,614 0.029 (0.271) 963 0.013 (0.328) White 7,451 0.070 (0.149) 4,097 0.185 (0.18) No 61,278 0.147 (0.046) ** 34,466 0.182 (0.069) * Yes 9,000 0.138 (0.142) 4,954 0.282 (0.171) Female 34,900 0.117 (0.065) 19,038 0.175 (0.08) * Male 35,378 0.172 (0.063) ** 20,382 0.219 (0.091) * 5,467 0.204 (0.267) 3,191 0.090 (0.285) School Location Suburban (0.683) ELA Performance Levels Far Below Basic English Proficiency Race/Ethnicity Hispanic/Latino (0.08) ** Gate Gender Math Performance Levels Advanced 335 All ASP Outcome N Estimate Frequent ASP (SE) N Estimate (SE) Proficient 16,509 0.017 (0.101) 9,344 0.044 (0.13) Basic 19,234 0.097 (0.079) 10,724 0.135 (0.106) Below Basic 20,542 0.253 (0.081) ** 11,388 0.298 (0.099) ** 8,526 0.157 (0.095) 4,773 0.296 (0.134) * No 18,643 0.098 (0.097) 10,453 0.196 (0.148) Yes 51,635 0.161 (0.051) ** 28,967 0.192 (0.069) ** Far Below Basic National School Lunch Program Parent Education Level College 8,710 -0.044 (0.146) 4,868 -0.034 Some College (0.178) 11,011 0.150 (0.103) 6,313 0.300 (0.127) * High School 15,550 0.258 (0.083) ** 8,663 0.298 (0.11) * Less Than High School 19,171 0.164 (0.101) 10,764 0.095 (0.112) Non Response 15,836 0.112 (0.082) 8,812 0.282 (0.102) * No 65,832 0.139 (0.045) ** 36,929 0.185 (0.069) * Yes 4,446 0.243 (0.157) 2,491 0.295 (0.184) No 19,771 0.366 (0.101) ** 11,107 0.527 (0.122) ** Yes 50,507 0.073 (0.051) 28,313 0.088 (0.074) Special Education Status Title Status Note Participation effect estimates reflect percent change in likelihood of attaining healthy fitness zone, controlling for 2008-09 ELA CST scores Standard errors adjusted for school-level clustering ASP = After school participant 336 Table F12 Estimated Effect of After School Participation on Upper Body Strength for 2009-10 (Matched Sample): High Schools All ASP Outcome ALL N Estimate Frequent ASP (SE) N Estimate (SE) 70,278 0.168 (0.038) ** 39,420 0.313 (0.048) ** Urban 44,449 0.105 (0.05) * 24,746 0.211 (0.066) ** Rural 2,408 0.158 (0.192) 1,694 0.182 (0.206) 22,384 0.279 (0.061) ** 12,244 0.529 (0.071) ** Advanced 9,108 -0.007 (0.083) 5,213 0.106 (0.107) Proficient 15,868 0.301 (0.093) ** 8,822 0.459 (0.102) ** Basic 24,021 0.143 (0.061) * 13,541 0.280 (0.07) ** Below Basic 12,925 0.171 (0.063) ** 7,168 0.336 (0.077) ** 8,356 0.157 (0.069) * 4,676 0.302 (0.087) ** English Learner 17,826 0.122 (0.052) * 9,872 0.272 (0.066) ** English Only 25,651 0.272 (0.069) ** 14,561 0.418 (0.082) ** I-FEP 6,066 0.380 (0.082) ** 3,334 0.752 (0.107) ** R-FEP 20,735 0.036 (0.068) 11,653 0.128 (0.08) Asian/Pacific Islander 7,050 0.231 (0.131) 4,019 0.313 (0.158) African American/Black 6,635 0.105 (0.141) 4,040 0.246 (0.152) 47,528 0.167 (0.046) ** 26,301 0.327 (0.058) ** Other 1,614 0.616 (0.211) * 963 0.726 (0.238) * White 7,451 0.075 (0.086) 4,097 0.169 (0.121) No 61,278 0.159 34,466 0.293 (0.05) ** Yes 9,000 0.231 (0.088) * 4,954 0.506 (0.11) ** Female 34,900 0.082 (0.049) 19,038 0.196 (0.063) ** Male 35,378 0.263 (0.056) ** 20,382 0.413 (0.064) ** School Location Suburban ELA Performance Levels Far Below Basic English Proficiency Race/Ethnicity Hispanic/Latino Gate (0.04) ** Gender Math Performance Levels 337 All ASP Outcome N Estimate Frequent ASP (SE) N Estimate (SE) Advanced 5,467 0.121 (0.13) 3,191 0.237 Proficient 16,509 0.092 (0.064) 9,344 0.223 (0.08) ** Basic 19,234 0.147 (0.077) 10,724 0.287 (0.082) ** Below Basic 20,542 0.253 (0.069) ** 11,388 0.424 (0.078) ** 8,526 0.154 (0.07) * 4,773 0.248 (0.088) ** No 18,643 0.207 (0.055) ** 10,453 0.435 (0.074) ** Yes 51,635 0.156 (0.046) ** 28,967 0.279 (0.056) ** 8,710 0.099 (0.09) 4,868 0.159 (0.108) Some College 11,011 0.175 (0.085) * 6,313 0.333 (0.103) ** High School 15,550 0.218 (0.073) ** 8,663 0.416 (0.084) ** Less Than High School 19,171 0.113 10,764 0.210 (0.085) * Non Response 15,836 0.226 (0.067) ** 8,812 0.428 (0.086) ** No 65,832 0.164 (0.039) ** 36,929 0.311 (0.049) ** Yes 4,446 0.243 (0.092) * 2,491 0.369 (0.114) ** No 19,771 0.294 (0.073) ** 11,107 0.498 (0.084) ** Yes 50,507 0.122 (0.045) ** 28,313 0.247 (0.056) ** Far Below Basic (0.149) National School Lunch Program Parent Education Level College (0.08) Special Education Status Title Status Note Participation effect estimates reflect percent change in likelihood of attaining healthy fitness zone, controlling for 2008-09 ELA CST scores Standard errors adjusted for school-level clustering ASP = After school participant 338 Table F13 Estimated Effect of After School Participation on Flexibility for 2009-10(Matched Sample): High Schools All ASP Outcome ALL N Estimate Frequent ASP (SE) N Estimate (SE) 70,278 0.154 (0.048) ** 39,420 0.111 (0.057) Urban 44,449 0.168 (0.056) ** 24,746 0.142 (0.066) * Rural 2,408 0.066 (0.156) 1,694 -0.010 (0.148) 22,384 0.085 (0.095) 12,244 0.057 (0.128) Advanced 9,108 0.143 (0.138) 5,213 0.053 (0.142) Proficient 15,868 0.294 (0.118) * 8,822 0.239 (0.127) Basic 24,021 0.099 (0.077) 13,541 0.017 (0.082) Below Basic 12,925 0.112 (0.084) 7,168 0.142 (0.099) 8,356 0.172 (0.071) * 4,676 0.197 (0.096) English Learner 17,826 0.106 (0.064) 9,872 0.111 (0.085) English Only 25,651 0.115 (0.091) 14,561 0.095 (0.095) I-FEP 6,066 0.165 (0.116) 3,334 0.094 (0.153) R-FEP 20,735 0.262 (0.091) ** 11,653 0.134 (0.1) Asian/Pacific Islander 7,050 0.138 (0.149) 4,019 0.199 (0.157) African American/Black 6,635 0.176 (0.191) 4,040 0.145 (0.194) 47,528 0.181 (0.052) ** 26,301 0.128 (0.066) Other 1,614 -0.021 (0.185) 963 -0.034 (0.236) White 7,451 -0.040 (0.11) 4,097 -0.079 (0.118) No 61,278 0.166 (0.052) ** 34,466 0.117 (0.061) Yes 9,000 0.012 (0.106) 4,954 0.036 (0.126) Female 34,900 0.180 (0.063) ** 19,038 0.161 (0.078) * Male 35,378 0.125 (0.065) 20,382 0.064 (0.073) 5,467 0.385 (0.224) 3,191 0.300 (0.23) School Location Suburban ELA Performance Levels Far Below Basic English Proficiency Race/Ethnicity Hispanic/Latino Gate Gender Math Performance Levels Advanced 339 All ASP Outcome N Estimate Frequent ASP (SE) N Estimate (SE) Proficient 16,509 0.098 (0.112) 9,344 0.021 (0.106) Basic 19,234 0.204 (0.103) 10,724 0.146 (0.106) Below Basic 20,542 0.113 (0.068) 11,388 0.078 (0.088) 8,526 0.155 (0.072) * 4,773 0.161 (0.099) No 18,643 0.092 (0.071) 10,453 0.032 (0.08) Yes 51,635 0.176 (0.055) ** 28,967 0.139 (0.067) * Far Below Basic National School Lunch Program Parent Education Level College 8,710 0.094 (0.109) 4,868 0.072 (0.12) Some College 11,011 0.043 (0.141) 6,313 -0.005 (0.133) High School 15,550 0.086 (0.08) 8,663 -0.006 (0.089) Less Than High School 19,171 0.309 (0.094) ** 10,764 0.269 (0.114) * Non Response 15,836 0.130 (0.064) * 8,812 0.138 (0.082) No 65,832 0.156 (0.051) ** 36,929 0.107 (0.059) Yes 4,446 0.150 (0.113) 2,491 0.176 (0.136) No 19,771 0.183 (0.112) 11,107 0.203 (0.105) Yes 50,507 0.143 (0.047) ** 28,313 0.078 (0.058) Special Education Status Title Status Note Participation effect estimates reflect percent change in likelihood of attaining healthy fitness zone, controlling for 2008-09 ELA CST scores Standard errors adjusted for school-level clustering ASP = After school participant * p < 0.05 ** p < 0.01 340 Table F14 Estimated Effect of After School Participation on School Attendance Rate for 2009-2010 (Matched Sample): High Schools All ASP Outcome N Estimate 65,617 0.023 Advanced 6,302 Proficient Frequent ASP N Estimate (SE) (0.008) ** 35,552 0.012 (0.015) -0.003 (0.025) 3,506 0.015 (0.035) 13,861 0.030 (0.016) 7,408 0.016 (0.022) Basic 22,239 0.048 (0.014) ** 12,031 0.015 (0.02) Below Basic 15,142 -0.004 (0.012) 8,277 -0.015 (0.016) Far Below Basic 8,073 0.005 (0.015) 4,330 -0.008 (0.021) English Learner 18,344 0.007 (0.011) 9,764 0.007 (0.019) English Only 21,047 0.027 (0.015) 11,260 0.019 (0.02) I-FEP 5,300 0.019 (0.021) 2,881 0.001 (0.03) R-FEP 20,926 0.033 (0.014) * 11,647 0.010 (0.023) Asian/Pacific Islander 5,942 0.020 (0.025) 3,398 0.055 (0.038) African American/Black 7,134 0.023 (0.024) 4,019 0.005 (0.024) Hispanic/Latino 46,350 0.026 (0.01) ** 24,898 0.009 (0.018) Other 572 -0.091 (0.07) 309 -0.172 (0.091) White 5,619 0.009 (0.023) 2,928 0.027 (0.033) No 57,635 0.023 (0.008) ** 31,170 0.007 (0.014) Yes 7,982 -0.005 (0.024) 4,382 -0.006 (0.035) Female 33,173 0.031 (0.011) ** 17,674 0.019 (0.018) Male 32,444 0.014 (0.012) 17,878 0.005 (0.018) Advanced 1,382 -0.058 (0.067) 790 0.026 (0.1) Proficient 8,248 0.030 (0.022) 4,417 0.042 (0.032) Basic 17,381 0.041 (0.015) ** 9,332 0.046 (0.02) Below Basic 26,226 0.017 (0.01) 14,207 -0.012 (0.018) ALL (SE) ELA Performance Levels English Proficiency Race/Ethnicity GATE Gender Math Performance Levels 341 * All ASP Outcome Frequent ASP N Estimate (SE) N Estimate (SE) 12,380 0.001 (0.012) 6,806 -0.039 (0.015) No 17,888 0.020 (0.016) 9,770 0.017 (0.022) Yes 47,729 0.023 (0.009) * 25,782 0.010 (0.016) College 6,486 0.000 (0.022) 3,625 0.013 (0.035) Some College 8,019 0.035 (0.024) 4,265 -0.005 (0.031) High School 12,881 0.027 (0.015) 6,762 -0.011 (0.02) Less Than High School 18,305 0.017 (0.012) 9,719 0.008 (0.025) Non Response 19,926 0.029 (0.011) * 11,181 0.041 (0.017) No 61,326 0.021 (0.009) * 33,249 0.013 (0.016) Yes 4,198 0.048 (0.017) ** 2,261 0.010 (0.024) No 13,327 0.028 (0.017) 7,265 0.031 (0.026) Yes 52,290 0.021 (0.01) * 28,287 0.007 (0.018) Far Below Basic ** National School Lunch Program Parent Education Level * Special Education Status Title Status Note Participation effect estimates control of 2008-09 School Attendance Rates and ELA CST scores Standard errors adjusted for school-level clustering ASP = After school participant * p < 0.05 ** p < 0.01 342 Table F15 Estimated Effect of After School Participation on Suspension Record for 2009-2010 (Matched Sample): High Schools All ASP Outcome ALL N Estimate Frequent ASP (SE) N Estimate (SE) 67,611 -0.069 (0.05) 36,944 -0.181 (0.061) ** Urban 43,664 -0.064 (0.064) 23,450 -0.197 (0.082) * Rural 2,799 -0.107 (0.17) 2,063 -0.154 (0.125) 21,063 -0.067 (0.086) 11,397 -0.106 (0.103) Advanced 8,910 0.034 (0.194) 4,856 -0.084 (0.238) Proficient 16,456 -0.128 (0.122) 8,947 -0.300 (0.136) * Basic 23,092 -0.054 (0.055) 12,657 -0.115 (0.071) Below Basic 11,918 -0.041 (0.075) 6,520 -0.128 (0.077) 7,235 -0.090 (0.102) 3,964 -0.285 (0.116) * English Learner 16,455 0.086 (0.072) 8,677 -0.096 (0.08) English Only 27,574 -0.184 15,321 -0.252 I-FEP 5,923 0.175 (0.12) 3,188 0.065 R-FEP 17,659 -0.107 (0.11) 9,758 -0.333 (0.119) ** 10,121 -0.165 (0.112) 5,876 -0.207 (0.131) 7,582 -0.262 (0.098) ** 4,586 -0.394 (0.116) ** 41,149 0.002 (0.067) 21,814 -0.154 (0.082) Other 1,801 -0.029 (0.246) 1,013 -0.288 (0.323) White 6,958 -0.019 (0.124) 3,655 -0.026 (0.148) No 57,471 -0.075 (0.051) 31,441 -0.189 (0.062) ** Yes 10,140 0.115 (0.144) 5,503 0.016 (0.17) Female 34,178 -0.033 (0.069) 18,103 -0.190 (0.097) Male 33,433 -0.091 (0.053) 18,841 -0.190 (0.057) ** School Location Suburban ELA Performance Levels Far Below Basic English Proficiency (0.06) ** (0.077) ** (0.143) Race/Ethnicity Asian/Pacific Islander African American/Black Hispanic/Latino GATE Gender Math Performance Levels 343 All ASP Outcome N Estimate Frequent ASP (SE) N Estimate (SE) Advanced 3,440 -0.005 (0.338) 1,904 -0.291 (0.365) Proficient 12,536 -0.135 (0.141) 6,723 -0.131 (0.155) Basic 17,738 -0.097 (0.092) 9,629 -0.210 (0.107) Below Basic 23,674 -0.031 (0.064) 12,988 -0.142 (0.079) Far Below Basic 10,223 -0.057 (0.08) 5,700 -0.224 (0.088) * No 19,638 -0.133 (0.074) 10,734 -0.168 (0.084) * Yes 47,973 -0.048 (0.054) 26,210 -0.190 (0.068) ** 8,785 -0.352 (0.106) ** 4,831 -0.408 (0.111) ** Some College 10,803 -0.080 (0.09) 5,983 -0.153 (0.122) High School 16,968 -0.025 (0.06) 9,207 -0.177 (0.075) * Less Than High School 18,217 0.003 (0.112) 9,876 -0.166 (0.108) Non Response 12,838 -0.069 (0.092) 7,047 -0.129 (0.099) No 63,249 -0.077 (0.048) 34,458 -0.203 (0.061) ** Yes 4,362 -0.001 (0.129) 2,486 0.025 (0.137) No 26,448 -0.114 (0.082) 14,709 -0.164 (0.084) Yes 41,163 -0.044 (0.059) 22,235 -0.199 (0.078) * National School Lunch Program Parent Education Level College Special Education Status Title Status Note Participation effect estimates control for 2008-09 Suspension Record and ELA CST scores Standard errors adjusted for school-level clustering ASP = After school participant * p < 0.05 ** p < 0.01 344 ... complete the ? ?After School Profiling Questionnaire” designed by the evaluation team Designing the After School Profiling Questionnaire It is essential that an evaluation of after school programming... and impact of after school programs? What unintended consequences have resulted from the implementation of the after school programs? This report focuses on the findings of the ASSETs programs. .. into high school, they face a different set of developmental challenges and need a different set of supports to engage them successfully in after school programs By high school, students are independent