STEM-Year-Three-Evaluation-Final-Report

Service Learning Transforming Educational Models in Science, Technology, Engineering, and Mathematics (STEM)2 Year Three Evaluation Final Report California State University Submitted by Cobblestone Applied Research & Evaluation, Inc Rebecca M Eddy, Ph.D., Erika Randall, M.Ed., & Mariana Schmalstig, M.A September 29, 2014 Page Table of Contents Executive Summary Introduction to Service Learning Transforming Educational Models (STEM)2 Background and Context of Service Learning (STEM)2 Program Purpose Program Evaluation Methods 11 Key Evaluation Questions 11 Section 1: Academic Program Development 12 STEM Service-Learning Self-Assessment Rubric 12 STEM Service-Learning Self-Assessment Results 14 Summary of Service-Learning Capacity in STEM Departments at the CSU 19 Innovative Practices in STEM Departments 19 Campus 1: Chico 20 Campus 2: Fresno 20 Campus 3: Monterey Bay 21 Campus 4: San Marcos 21 Campus 5: Sonoma 21 Summary of Innovative Practices Updates 22 Laying the Foundations Subgrants Summary 22 Engaged Department Initiative 23 Engaged Department Institute 23 Additional Funding Opportunities that Support Service Learning in STEM at the CSU 32 Energizing Career Opportunities through Local Education and Development (ECO LED) 32 CSU’s Service Learning and Energy Efficiency and Education Grant Program 35 Section 2: Student Development 40 Replicating Successful Service-Learning Initiatives: STEM C3 40 Creation of the STEM C3 Manual 41 STEM C3 Presentations 41 Follow-up to STEM C3 Presentations: One Year Later 42 Follow-up to STEM C3 Presentations: Two Years Later 43 Summary of STEM C3 Follow Up 45 STEM C3 Awards and Impact 45 Section 3: Statewide Partnership Development 46 Strengthen Current Partnerships 46 California Campus Compact 46 Society for Science and the Public 47 Project Kaleidoscope (PKAL) 47 Southern California Edison (SCE) 47 Develop New Partnerships 48 Southern California Gas 48 SESYNC 48 Helmsley Charitable Trust 49 AmeriCorps VISTA 49 Summary of Partnership Development 50 Program Dissemination 50 Summary of Program Dissemination 51 Page Section 4: (STEM)2 Program Results and Sustainability 52 Results of the (STEM)2 Program 52 Lessons Learned and Sustainability of the (STEM)2 Program at the CSU 54 Conclusions 56 References 57 Tables Table Components of Service-Learning Capacity Dimensions 13 Table Laying the Foundation Subgrants Participants and Activities 23 Table Engaged Department Institute Follow-Up Survey Respondents 27 Table Mean Participant Ratings for Engaged Department Institute: Comparison of Pretest, Posttest, and Follow-up Surveys 28 Table Engaged Department Institute Participant Mean Ratings, Service Learning Benefits 29 Table Engaged Department Institute Efficacy Mean Ratings 30 Table Summary of ECO LED Program Activities 33 Table Challenges and Possible Solutions for the ECO LED Program 35 Table Summary of Additional Grant Activities 37 Table 10 Offices Involved in Planning STEM Student Success Programs 43 Table 11 Most Viewed Pages on Chancellor’s Office Center for Community Engagement Website 51 Table 12 Program Indicators, Goals, and Status After Years and 52 Table 13 Overall and STEM Service Learning Courses 2008-09 through 2012-13 54 Figures Figure Pretest Self-Assessment Ratings for All CSU Campuses 14 Figure Posttest Self-Assessment Ratings for All CSU Campuses 15 Figure Average Pretest and Posttest Scores by Component 15 Figure Pretest and Posttest Self-Assessment Ratings for All CSU Campuses 18 Figure Continuums of Service Conference Presentation Ratings 42 Page Executive Summary In 2010, the CSU consortium was awarded $1.5 million in funding (over years) from Learn and Serve America: Corporation for National and Community Service to support efforts to promote service learning and community engagement in Science, Technology, Engineering, and Mathematics (STEM) fields throughout the California State University (CSU) The ServiceLearning Transforming Educational Models (STEM)2 program was delivered to the CSU campuses and supported by the CSU Chancellor’s Office from September 2010 through August 2011 Although funding did not continue beyond the initial $500,000 grant in the first year, the CSU campuses continued STEM service-learning efforts with support from the Chancellor’s Office This report serves as a follow-up study from the first year of funding intended to summarize the impact of (STEM)2 in which we attempt to identify how the elements of the original (STEM)2 program produced sustainable change on CSU campuses Development Area Activity Innovation Subgrants Academic Program Development Engaged Department Initiative Engaged Department Institute (EDI) EDI Grants Laying the Foundation Subgrants Student Development STEM C3 (Careers, Community and Connections) Partnership Development Statewide Partnership Development Description Five CSU campuses – Chico, Fresno, Monterey Bay, San Marcos and Sonoma – received $25,000 subgrants to implement creative and innovative approaches to institutionalize service learning within STEM departments The EDI was offered in June 2011 and was specific to science departments It included five teams from four campuses The three-day long conference brought together teams of faculty, staff, and community partners to develop strategies to support service-learning and community engagement activities in their departments Participating teams received $5,000 subgrants to carry out the action plans they developed at the Institute during the 2011-2012 academic year Fifteen of the twenty-three CSU campuses received funding for service-learning offices to assess, showcase and support current service-learning activities in their STEM departments Program Directors from CoyoteCareers in San Bernardino created a manual and presented at multiple events that provided information on how to develop and sustain a successful campus program for STEM students which includes tutoring, career preparation, servicelearning internships, and alumni networking Partnerships were established or strengthened with key organizations including: California Campus Compact, Cal STEM, California Volunteers, Society for Science and the Public, Project Kaleidoscope, the investorowned utilities, foundations and the AmeriCorps VISTA program Page The overarching goals of (STEM)2 were to institutionalize service learning in STEM departments and promote college student retention and achievement in STEM disciplines Three areas were targeted to support the grant goals: academic program development, student development, and partnership development throughout the CSU campuses and communities The grant supported these three areas with several activities throughout the CSU system Cobblestone Applied Research & Evaluation, Inc was again hired to evaluate the effectiveness of the (STEM)2 initiatives and to conduct a follow-up study of program activities that have occurred since the first year of the program We wanted to determine if any changes made from the original LSA funding were sustained over time and investigate the current state of service learning in STEM at the 23 CSU campuses The evaluation questions and corresponding results are summarized below Evaluation Question How has service-learning capacity in STEM departments on CSU campuses changed over time? What is the current state of this capacity? To what extent was participation in (STEM)2 initiatives related to the capacity for service learning in STEM on CSU campuses? What was the impact of the (STEM)2 activities such as Innovations subgrants, Laying the Foundation subgrants, and Engaged Department Initiative in promoting Academic Program Development? Answer According to the pretest self-assessment rubric, STEM departments on of the 23 campuses were at the “sustained institutionalization” stage of service learning During the posttest, this doubled to eight campuses at the “sustained institutionalization” stage Most campuses improved from pretest to posttest with thirteen campuses moving onto the next stage On average, improvement was made for each component of the rubric, although differences between the dimensions were found among the campuses In general, campuses that were engaged in more program components tended to have larger increases from their pretest to posttests scores On average, the six campuses who participated in three to five activities had an increase in FURCO score by 10.6 points Of the campuses (n = 9) who were involved in one or fewer activities, only points of improvement were made Innovations subgrants: The subgrants clearly provided CSU campuses with the ability to include service learning in a way that was tailored to the specific needs of the institution and surrounding community Some of the work that was started from these projects has continued over time, while other work has been expanded or suspended Sustainability of such projects requires ongoing effort and resources and many projects have been supported by faculty members at each campus Laying the Foundation subgrants: These subgrants allowed all fifteen campuses to gain a better understanding of the state of service learning in their STEM departments, deliver planned activities and create plans for future growth Despite lower than expected faculty support, the subgrants provided the necessary focus on STEM service-learning courses to facilitate the long-term growth of STEM service-learning courses system wide Engaged Department Institute: Results indicated that respondents benefitted from their experiences at the EDI Comparisons between the original EDI survey and the follow-up survey revealed that attitudes towards service learning and the EDI have remained favorable, despite the challenges that some groups have faced in implementing the vision that they created at the EDI Page What additional funding opportunities have facilitated service learning in STEM in the CSU? The Energizing Career Opportunities through Local Education and Development and Service-Learning and Energy Efficiency and Education grant programs allowed for some campuses to continue improving service-learning efforts Six individual campuses implemented a wide array of activities These accomplishments include the assessment and development of service-learning courses, the implementation of new curriculum, and the facilitation of learning communities Additionally, (STEM)2 created the foundation upon which a newly launching CSU STEM VISTA was built In AY 2014-2015, fifteen AmeriCorps VISTA members will be placed in STEM departments and colleges to build capacity for STEM student success through the development of high-impact practices like service learning All of the seven participating campuses felt that the printed manual they What was the impact of the received at the symposium was helpful A year after attending the symposium, five of the campuses confirmed that they plan to establish (STEM)2 activities in promoting Student a new STEM student success program using the tools learned from the Development? Have CSU symposium While the momentum generated from STEM C3 inspired several campuses to create plans for STEM student success programs, campuses implemented two years later this enthusiasm has largely waned While many long-term plans after individual faculty members have devoted time in planning and grantattending the STEM C3 writing for these initiatives, it is clear that grant funding is required to symposia? support intensive efforts on campus For the past three years, (STEM)2 was focused on both strengthening existing partnerships and developing new partnerships in STEM service How has Statewide learning Despite funding cuts, they have been successful at developing Partnership Development and fostering relationships throughout California It is likely that the from (STEM)2 continued or efforts to integrate more STEM service learning at the CSU will expanded? continue given the partnerships that have been developed and additional extramural funding acquired to support these efforts How have results of (STEM)2 initiatives been disseminated in the CSU? Did the activities conducted in the (STEM)2 program produce intended program results? What were the lessons learned from the (STEM)2 program? To increase visibility of program activities and results, the “Center for Community Engagement” (CCE) and “STEM Engaged Learning” websites have been updated with recent and relevant information This has promoted dissemination of activities and service-learning resources Additionally, the 2011-2012 annual publication distributed by CCE (in print and electronically) highlighted STEM service-learning courses and programs at CSU campuses A total of 529 STEM SL courses are offered at the CSU—an increase of 245 new courses since 2008-09 The percent of service-learning students in the STEM fields has increased from 9.6% to 14.9% from 2008-09 to 2012-13 and 17.7% of all service-learning courses are offered in the STEM fields (from 10.1% in 2008-09) While initial enthusiasm to integrate service learning into STEM curriculum may be strong, external funding is required to sustain such efforts Similarly, dedicated faculty and a commitment to inquiry-based learning are also required to maintain momentum for such initiatives Innovations subgrants were a useful process for creating new service-learning courses at multiple campuses The longer-term Page What were the most sustainable aspects of the (STEM)2 program? impacts have had mixed results with some efforts expanding while others have been suspended The STEM C3 presentations garnered a lot of interest from attendees; however, longer-term implementation of similar programs has not been fully integrated at other CSU campuses Service-learning course experiences have positive impacts on students, faculty and the community but adequate time and preparation must be allowed to maximize the student learning experience and community benefit STEM service-learning courses that were developed as part of (STEM)2 continue to be offered at CSU campuses and others have also been developed recently The continued increase in the number of STEM service-learning courses is evidence of this sustainability New initiatives such as VISTA program and the partnerships with Business and Finance around Campus as a Living Lab effort will continue to build capacity for long-term sustainability Service learning in STEM has expanded throughout the CSU over the past three years, in part from the LSA funding and subsequently other funding partnerships The increase in servicelearning courses and numbers of students served, particularly in STEM, can be attributed to this effort Despite the lack of funding beyond the first year, the CSU Chancellor’s Office, Center for Community Engagement (COCCE) has successfully leveraged new and existing partnerships to continue integration of service learning in STEM throughout the state Other evidence that service learning in STEM has broadened throughout the CSU was captured by the selfassessment instruments conducted as pretests and posttests Most CSU campuses increased selfassessment ratings in the integration of service learning in STEM, and a good number have reached the sustained institutionalization stage These data are encouraging both in terms of evidence that the COCCE efforts have been successful over the past few years, as well as the idea that more students, faculty and communities are expected to benefit from this increased integration Page Introduction to Service Learning Transforming Educational Models (STEM)2 The California State University (CSU) system is the largest system of higher education in the United States and recognizes the responsibility to provide its students with knowledge and opportunity to fully participate in civil society For more than a decade, the CSU system has been committed to increasing opportunities for student community engagement, and has been expanding partnerships to so Learn and Serve America (LSA) has been a partner to the CSU at various points since 2000 In 2010, the CSU consortium was awarded $1.5 million in funding (over years) from Learn and Serve America: Corporation for National and Community Service to support efforts to promote service learning and community engagement in the Science Technology, Engineering, and Mathematics (STEM) fields throughout the 23 campuses of the CSU The Service Learning Transforming Educational Models (STEM)2 program was delivered to the CSU campuses and supported by the CSU Chancellor’s Office from September 2010 to August 2011 The focus of this initiative was threefold: academic program development, student development, and statewide partnership development Unfortunately, funding for the Learn and Serve program overall was eliminated by Congress after the first year of the (STEM)2 program and therefore the ability to implement the three-year program was thwarted Despite this challenge, the CSU Chancellor’s Office Center for Community Engagement (COCCE) has been able to continue with its STEM-focused support from a combination of other funding sources and the Chancellor’s Office directly This report serves as a follow-up study from the first year of funding intended to summarize the impact of (STEM)2 This study attempts to capture how the elements of the original (STEM)2 produced sustainable change on CSU campuses and provides some evidence of what might be possible if similar initiatives are supported in the CSU in the future Background and Context of Service Learning Service learning is an educational methodology that combines community service with classroom learning to engage students in the educational process, teach civic responsibility, and strengthen communities (Bringle & Hatcher, 1995) Service learning helps students gain socially responsive knowledge, that is, education obtained through direct academic-based problem solving of social issues (Altman, 1996) Students participating in service learning apply the skills and knowledge they learn in their academic coursework to identify and solve real-world community problems and also access the expertise of community partners in addressing these Page problems Additionally, students become contributing citizens and active community members through the service they perform Research indicates that service learning has an overall positive impact on students’ social, personal, and cognitive outcomes (Giles & Eyler, 1994) Students who participated in servicelearning education had greater gains in perspective-taking, complex problem solving and critical thinking than students who did not participate in service-learning education (Batchelder & Root, 1994; Markus, Howard, & King, 1993) Additionally, service learning increases students’ awareness of contemporary social issues (Driscoll, Holland, Gelmon & Kerrigan, 1996), enhances students’ self-efficacy relating to community service (Reeb, Katsuyama, Sammon, & Yoder, 1998), and increases positive attitudes about civic engagement and social responsibility (Markus et al., 1993) Although this form of community-based education has been linked to positive student outcomes, it also creates new challenges for the institution Specifically, this type of undertaking changes the function of the institution, creates new demands for faculty roles, and affects not only what students learn but also what is taught Through the (STEM)2 program, students and faculty at a number of CSU campuses participated in one of several service-learning courses focused specifically on STEM These courses covered a wide variety of disciplines and strategies Given the suspension of funding after the first program year, (STEM)2 was not able to track more specific student indicators for those students participating in service-learning courses such as retention, graduation, selfefficacy relating to community service or other key indicators proposed However, overall rates of STEM service-learning courses and other institution-level data have been tracked In addition, the COCEE has captured some of the key activities related to (STEM)2 initiatives and related programs that have occurred at CSU campuses during the past three years From this information, we can make some inferences about the relationship between these activities facilitated by (STEM)2 and subsequent impacts on students and faculty Although these conclusions are far from definitive, they suggest how LSA funding has provided the CSU with both direct and indirect impacts even after suspension of initial funding The following report details these activities and findings (STEM)2 Program Purpose (STEM)2 was designed to integrate service learning as part of the undergraduate educational experience for students in the STEM fields Specifically, program goals were to 1) Page institutionalize service learning in STEM disciplines, and 2) promote college student retention and achievement in STEM disciplines These goals were addressed through academic program development, student development, and statewide partnership development A website was established to provide information about the initiative, and can be found at: http://www.calstate.edu/cce/stem/ (STEM)2 included a variety of program activities throughout the CSU campuses in the first program year These activities were intended to simultaneously expand service-learning experiences for CSU students while advancing their retention, graduation and career placement in the STEM disciplines Activities included: Laying the Foundation subgrants were awarded to fifteen service-learning offices on CSU campuses to assess current STEM service-learning activities on their campuses and promote service learning in STEM departments Innovation subgrants were awarded to five CSU campuses (Chico, Fresno, Monterey Bay, San Marcos, and Sonoma) to provide funding to carry out innovative projects and serve as demonstration sites from which to grow STEM service-learning initiatives throughout the CSU An Engaged Department Institute (EDI) was held with five teams from multiple CSU campuses (Bakersfield, Fresno, Los Angeles, and Monterey Bay) to provide training, support, and resources to strategically embed service learning and community engagement into academic programs These five teams then received follow-up subgrants to implement the plans they developed during the Institute A how-to manual for STEM student success was written by a team at CSU San Bernardino that provided specific information on developing a program modeled after their award-winning CoyoteCareers program, which includes facilitation of career soft skills, service-learning internships, tutoring support, and integration of alumni Efforts to promote partnerships and participate in service-learning activities with key organizations such as Project Kaleidoscope and the Society for Science and the Public were also developed Page 10 Fullerton: Fullerton was awarded a grant in 2011 from the U S Department of Education (prior to attending the symposium) that supports a Strengthening Transfer Education & Matriculation in STEM program, a partnership with three local community colleges Fullerton reported that they have successfully incorporated many elements of STEM C3 into their own program Although STEM C3 was not the initial impetus for Fullerton’s program, they likely gained knowledge about how to integrate program elements after attending the STEM C3 symposium Long Beach: Although initial plans were made to develop a program similar to STEM C3 on the Long Beach campus, plans were cancelled in 2012 to avoid duplication with other efforts on campus One faculty describes the idea that a “seed has been planted” and expects to devote time and energy once again to develop this model at Long Beach in the future San Francisco: Two faculty members at San Francisco State have worked to submit proposals for external funding to support a program like STEM C3 on their campus While they have received positive reviews, proposals to both NSF and National Institutes of Health have not been funded The faculty members continue to meet four times per year and seek funding to support service learning in STEM Another group of faculty has worked to designate San Francisco State as a Hispanic-Serving Institution and the Latino Educational Achievement Partnership has a STEM-specific working group that will continue to investigate ways to develop a similar program on campus San Jose: A lack of dedicated resources and other pressing needs have prevented San Jose State from developing a similar program on their campus Despite the lack of development of a formal program, support for other STEM service-learning efforts has occurred, such as gaining industry partners for STEM outreach efforts and delivering other STEM-based efforts in the community (i.e., STEM-focused outreach programs were delivered to 220 youth in fall 2014 and more were planned for spring 2014) In addition, a joint intern position has been funded from the College of Science and Career Center to develop resources that support STEM education related to careers, as well as directly supporting STEM students on career development issues Page 44 Summary of STEM C3 Follow Up While the momentum generated from the STEM C3 symposium inspired several campuses to form working groups and create plans for STEM service-learning based programs within a year of the presentation, two years later this enthusiasm has largely waned While many individual faculty members have devoted time in planning and grant-writing for these initiatives, it is clear that grant funding is required to support intensive efforts on campus STEM C3 Awards and Impact (STEM)2 was considered a catalyst for publicizing the STEM C3 program and has brought national recognition to the team For example, Ms Pamela Langford received the Council for Advancement and Support of Education (CASE) award Ms Carol Dixon received the National Association of Colleges and Employers (NACE) Innovation Excellence Award for Diversity Both programs recognized CoyoteCareers accomplishments, specifically for the sustainability achieved through the STEM C3 replication efforts In addition, the program directors were nominated for a James Irvine Foundation Leadership Award by Chancellor Reed, and the team received the President’s Team Achievement Award at the campus level The most important marker of success for the STEM C3 program was unanticipated, as the San Bernardino campus administration and study body approved a student success fee to sustain program efforts beyond external funding support The funding continued CoyoteCareers program activities at San Bernardino with half of the fee designated for career success initiatives Although the STEM C3 initiative was not able to fund the implementation of the model on other campuses as originally proposed, the first year was successful as evidenced by the completion of the manual, presentations, awards, and sustainability efforts in the form of student fees Page 45 Section 3: Statewide Partnership Development The last program component was statewide partnership development, in which the (STEM)2 program would strengthen current relationships or build new relationships with other organizations focused on service learning, particularly in the STEM fields We wanted to understand how these partnerships have been maintained or expanded since the inception of (STEM)2 How has Statewide Partnership Development from (STEM)2 continued or expanded? The COCCE continued to work with statewide and national partners to build a network of support, information, and resource sharing to ensure that California coordinates efforts to increase the number of professionals in STEM-related careers and the CSU increases the number of service-learning offerings in STEM disciplines The CSU continued to approach partnership development in two ways: (1) strengthen/expand current service-learning partnerships to include specific efforts focused on STEM; and (2) develop new partnerships with STEM organizations that include specific collaborations focused on service learning and community engagement Strengthen Current Partnerships During the 2012-2013 year, COCCE continued to strengthen existing partnerships with California Campus Compact (CACC), www.cacampuscompact.org; the Society for Science and the Public (SSP), www.societyforscience.org; Project Kaleidoscope (PKAL) within the Association of American Colleges and Universities (AAC&U), http://www.aacu.org/pkal/index.cfm; and Southern California Edison (SCE) They maintained regular communication with all organizations COCCE also built new partnerships with SESYNC, Southern California Gas (SoCal Gas), the Helmsley Trust, and AmeriCorps VISTA California Campus Compact With CACC, COCCE collaborated on the development of a research agenda for service learning across the state of California As a result, they have submitted a proposal to a private foundation to fund a multi-year, statewide research initiative on the impact of service learning on STEM students, particularly under-represented students They also worked actively with CACC to support CSU campuses and ensure their continued membership in CACC despite challenging Page 46 budget times Additionally, they hosted one of CACC’s regional roundtable meetings in February 2014 as a part of CACC’s 25th anniversary Society for Science and the Public With SSP, the External Affairs Office at CSU participated on the planning committee for the Intel International Science and Engineering Fair held in Los Angeles in May 2014 Once again, CSU faculty and students participated in the Fair as volunteers and judges A press release and video about the experience of CSU participants at the 2011 Intel International Science and Engineering Fair is available at: http://www.calstate.edu/pa/News/2011/Story/IntelISEF.shtml Project Kaleidoscope (PKAL) PKAL was involved at two levels The Cal Poly Pomona campus hosted the most recent PKAL regional meeting in February 2014 focused on STEM Educational Effectiveness This was a significant opportunity to highlight service learning to an audience of STEM educators who have not had experience with service learning in the past Additionally, Judy Botelho, the Director of the COCCE, is participating in another PKAL initiative, funded by the W.M Keck Foundation, to develop a STEM Education Effectiveness Framework (More information can be found at http://www.aacu.org/pkal/educationframework/index.cfm.) Judy is co-leading the CSU delegation from the Chancellor’s Office in this effort This year, Judy co-facilitated a process with the CSU delegation to write a proposal in response to the National Science Foundation’s WIDER solicitation Much of the proposal was based on the lessons learned from the Learn and Serve grants (past and present) about facilitating systematic change (utilizing the Engaged Department model; developing supportive regional networks; hosting system wide colloquia to ensure support at the highest levels of leadership and across a diversity of constituents) Southern California Edison (SCE) In addition to the grant the COCCE first received in 2011 from Edison International, COCCE received a second $25,000 grant in December 2013 to support the development and implementation of additional curricula focused on “sustaining a quality environment” during the Page 47 AY 2014-2015 This ongoing partnership will continue to sustain the foundational efforts of (STEM)2 Develop New Partnerships The second goal in partnership development was to develop new partnerships with STEM organizations focused on service learning and community engagement In October 2011, the Los Angeles Times published an outstanding article on the CSU’s (STEM)2 program entitled “STEM2: The Future of the Cal State University.” A link to the article can be found on the website, www.calstate.edu/cce/stem As a result of the visibility provided by the Los Angeles Times article, we were approached by Southern California Gas (SoCal Gas) Additional partnerships were also formed with the Helmsley Charitable Trust, AmeriCorps VISTA and through the SESYNC initiative Southern California Gas After a half dozen partnership meetings with SoCal Gas, in 2012-2013, the COCCE implemented a new initiative focused on service learning and energy efficiency/energy education across southern California CSU campuses Detailed information about the outcomes of the initiative are posted on COCCE website (www.calstate.edu/cce/stem), and shared earlier (pages 33-36) SESYNC Through the SESYNC initiative, the COCCE forged a new internal partnership with the division of Business and Finance COCCE partnered with CSU Capital Planning, Design and Construction and the CSU Academic Senate to launch the “Campus as a Living Lab for Sustainability” initiative The ‘Campus as a Living Lab’ initiative is a unique opportunity to partner faculty and facilities management staff in using the campus as a forum for the exploration of sustainability concepts and theories The program aligns the CSU’s long-standing commitment to sustainability with the fundamental goal of preparing students for the workforce Two symposia were held in March 2013, modeled after the type of courses funded under the Learn and Serve initiative (Dominguez Hills’ Natural Resources and Northridge’s Best Practices in Sustainability) One of the symposia was hosted by SoCal Gas’ Energy Resource Center, exposing dozens of CSU faculty to this rich learning environment Over 80 faculty, staff, Page 48 students and community partners participated in the two symposia The CSU put out a call for proposals to fund $250,000 in projects that address sustainability and energy efficiency across CSU campuses The response was tremendous with more than 70 proposals received in the first round Ultimately, funding awards were made to 22 projects across 14 campuses These projects commenced in September 2013 Helmsley Charitable Trust Program officers at the Helmsley Charitable Trust received copies of the COCCE annual report highlighting STEM service-learning activities across the CSU and contacted the Chancellor’s Office to learn more As a result of many conversations and planning, the CSU Chancellor’s Office is now launching a new project called STEM Collaboratives, funded by the Helmsley Charitable Trust at $4.6 million, that aims to improve student success and close achievement gaps in STEM This Collaborative will integrate high-impact practices on three concurrent fronts including summer bridge experiences, first-year experiences, and the redesign of introductory, gateway courses AmeriCorps VISTA As a result of COCCE’s performance with the (STEM)2 grant, they were encouraged to apply for an AmeriCorps VISTA grant focused on STEM As a result, in October 2013, CNCS awarded the CSU fifteen VISTA members for the 2014-2015 program year COCCE will place 15 VISTA (Volunteers in Service to America) members in nine CSU STEM colleges, institutes and departments for the 2014-2015 inaugural CSU STEM VISTA program year VISTA members will serve as liaisons between STEM students, faculty and staff, and community partners to develop long-term solutions for bolstering the number of underrepresented students (particularly low-income and minority students) who pursue and obtain STEM degrees Through community partnerships with industry leaders, nonprofits, and K-12 schools, the 15 CSU STEM VISTA members will build the capacity of the CSU STEM host sites and community partner organizations to more deeply engage students in STEM and provide more opportunities to participate in mentoring and academic support programs These engaged learning experiences, such as service learning, internships, and undergraduate research, will not Page 49 only make lasting, positive changes in local communities, but also will expand the CSU’s ability to give students the skills and knowledge needed to be successful in their majors and obtain STEM degrees and job placement, thus in turn, building the STEM workforce that our nation so desperately needs Summary of Partnership Development For the past three years, (STEM)2 was focused on both strengthening existing partnerships and developing new partnerships in STEM service learning Despite CNCS LSA funding cuts, they have been successful at developing and fostering relationships throughout California It is likely that the efforts to integrate more STEM service learning in the CSU will continue given the partnerships that have been developed and additional extramural funding to support these efforts that have been secured How have results of (STEM)2 initiatives been disseminated in the CSU? Program Dissemination To increase visibility of program activities and results, an account of the various servicelearning initiatives has been communicated through a website under the main CSU site The “Center for Community Engagement” (CCE) and “STEM Engaged Learning” websites have been updated with recent and relevant information This has allowed for dissemination of activities and for visitors to find service-learning resources As part of the initiative, the http://www.calstate.edu/cce and http://www.calstate.edu/cce/stem websites were analyzed from September 2012 to August 2013 to assess the number of visits and types of engagement within the past year The following indicates the interactions observed on the CCE and the STEM Engaged Learning websites: 28,850 total visits for the CCE website (89.7% new visitors; 10.3% returning) 18,515 visitors to the CCE website 3,673 visitors to the STEM Engaged Learning website 55.1% of referring domains came from within CSU; the remainder were a combination of other search engines (e.g., google, bing, yahoo) 651 visitors downloaded the LSA STEM C3 Manual, which was the most viewed document Page 50 Table 11 displays the ten pages with the most page views on the STEM Engaged Learning website The home page had the most views, followed by the “Products to Use | Learn and Serve America,” “ECO LED Initiative,” and “Learn and Service America Initiative” pages These top pages had an average of two and half minutes of view time The “Water and Conflict in the West” page had the highest average time spent on the page at almost four and a half minutes, followed by the “Products to Use | Learn and Serve America” page at nearly four minutes Table 11 Most Viewed Pages on Chancellor’s Office Center for Community Engagement Website Page (STEM)2 Home Products to Use | Learn and Serve America ECO LED Initiative Learn and Serve America Initiative Biogeography of Southern California Helpful Resources Mine Reclamation Making an Impact Aquatic Ecology at Fresno State Water and Conflict in the West Initiative Learn and Serve America Edison International Learn and Serve America Edison International Page Views 953 252 Average Time on Page 01:00 03:56 186 127 02:25 01:38 120 03:32 Edison International Edison International Edison International 106 84 75 53 50 02:14 02:14 01:15 03:09 04:29 Summary of Program Dissemination The CCE websites have been successful with almost 30,000 visits and 651 downloads of the LSA STEM C3 Manual This indicates that the dissemination of program activities has effectively reached many visitors interested in service learning The website will continue to be a valuable resource for not only the CSU, but also for other similar institutions concerned about integrating service-learning components within STEM departments Page 51 Section 4: (STEM)2 Program Results and Sustainability The effectiveness of the (STEM)2 program was determined by the extent to which the program produced intended outcomes as well as other successes and challenges identified by the evaluation team The evaluation was designed to be formative during year one and summative during year three Despite the lack of funding resources, we assessed whether program outcomes were achieved over the course of three years Did the activities conducted in the (STEM)2 program produce intended program results? Results of the (STEM)2 Program Table 12 provides a summary of each program indicator, the identified final performance goal, the reported status after year one, and the final achievement after year three In the 200809 school year, 10.1% of service-learning courses (284 of 2,809) were offered in STEM departments This increased to 11.8% (324 of 2,737) during the 2010-11 school year and then 529 of 2,985 or 17.7% by 2012-13 Other relevant findings can be found below Table 12 Program Indicators, Goals, and Status After Years and Strengthening Communities K-12 Students STEM Education Indicator Goal After Years CSU STEM SL students will reach K-12 students through STEM-content activities CSU STEM SL students will create activities and materials for quality K12 STEM education Participating K-12 students will report increased understanding of and increased interest in STEM 1,000 K-12 students CSU students will participate in new STEM SL activities 3,000 CSU students 50 types of activities and materials 75% reported increase Status After Year Status After Year Over 6,000 K-12 students participated in projects funded by the Innovation subgrant activities and materials created for projects funded by the Innovation subgrant 2,400 additional K-12 students participated in energy efficiency education through STEM servicelearning courses 53 additional syllabi or curriculum units were created Anecdotal evidence suggests an increased interest of K-12 students that participated in SL activities These K-12 students were not surveyed In comparison with the baseline year in 2008-09 where 5,750 students participated in STEM SL courses, in 2010-11 6,733 students participated in STEM SL courses—an increase of 983 students Anecdotal evidence suggests an increased understanding and interest of K-12 students that participated These K12 students were not surveyed Page 52 In 2012-13 a total of 9,399 students participated in STEM SL courses an increase of 3,649 students or a 63.5% increase College Students Faculty STEM C3 STEM courses will integrate new SL activities CSU campuses will have the internal capacity and external relationships sufficient to sustain SL in STEM disciplines Students, new to STEM SL courses, will participate in STEM SL projects 150 STEM courses STEM SL students will have increased academic engagement STEM SL students will graduate from CSU campus in STEM disciplines CSU STEM faculty will participate in EDI 75% reported increase 75% of SL participants 18 CSU campuses 3,000 CSU students 40 new STEM SL courses were added from 2008-09 to 2010-11 At pretest, campuses reported “sustained institutionalization” In comparison with the baseline year in 2008-09 where 5,750 students participated in STEM SL courses, in 2010-11 6,733 students participated in STEM SL courses—an increase of 983 students Not able to assess after year 245 new STEM SL courses were added from 2008-09 to 2012-13 At posttest, campuses reported “sustained institutionalization” and 13 were at the “quality building” stage; 20 campuses overall improved from pre to post In 2012-13 a total of 9,399 students participated in STEM SL courses an increase of 3,649 students or a 63.5% increase Did not assess after suspension of funding Not able to assess after year Did not assess after suspension of funding 120 CSU faculty 18 CSU faculty members attended EDI in Year No EDI held after Year EDI participating CSU faculty will report increased competence in integrating SL into their STEM courses CSU will increase in the number of SL STEM courses offered annually STEM C3 program will be operating on additional CSU campuses 75% of CSU faculty 100% reported increase competence No additional measures of competence assessed after Year 150 SL STEM courses CSU campuses 49 new STEM SL courses were added from 2008-09 to 2010-11 Not able to assess after year STEM C3 subgrantees will report an increase in retention and persistence among STEM students Students served by the STEM C3 programs will graduate in their STEM disciplines and be placed in STEM-related fields 25% reported increase 25% of students 245 new STEM SL courses were added from 2008-09 to 2012-13 for a total of 529 courses CSU campuses had plans to integrate programs modeled after STEMC3 a year after follow-up; two years after follow-up three have continued plans for the future or currently implementing some version of the model No grants provided, no data collected on students in STEM C3 programs No grants provided, no data collected on students in STEM C3 programs Data presented in Table 13 was generated by accessing the CSU Academic Planning Database It is noteworthy that despite the fact that overall student enrollment in the CSU has Page 53 decreased in the past five years (from 450,000 to 436,560), the number of service-learning courses and overall number of students enrolled in these courses has increased In addition, the proportion of CSU students enrolled in service-learning courses has increased The percent of service-learning students in the STEM fields has increased from 9.6% to 14.9% over this timeframe and 17.7% of all service-learning courses are offered in the STEM fields (from 10.1% in 2008-09) While all of these changes cannot be attributed to the (STEM)2 program, it has undoubtedly influenced the presence of STEM service-learning courses on CSU campuses Table 13 Overall and STEM Service Learning Courses 2008-09 through 2012-13 Baseline Service Learning Information Total Number of CSU Students Enrolled Number of Service-Learning Courses Number of Service-Learning Students Percent of all CSU Students Enrolled in Service-Learning Courses Number of STEM ServiceLearning Courses Percent of all Service-Learning courses in STEM Fields Number of STEM ServiceLearning Students Percent of all Service-Learning students in STEM Fields LSA Funding 2008-09 2009-10 2010-11 No LSA Funding but (STEM)2 activities continue 2011-12 2012-13 450,000 433,000 412,000 426,534 436,560 2,809 2,738 2,737 3,161 2,985 60,077 61,010 58,350 62,475 62,782 13.4% 14.1% 14.2% 14.6% 14.4% 284 323 324 463 529 10.1% 11.8% 11.8% 14.6% 17.7% 5,750 6,275 6,733 7,848 9,399 9.6% 10.3% 11.5% 12.6% 14.9% What were the lessons learned from the (STEM)2 program? Lessons Learned and Sustainability of the (STEM)2 Program at the CSU It is clear that the (STEM)2 program has had an impact on faculty, students and surrounding communities across the CSU There have been numerous examples provided in the current report where the courses and projects within courses would have not been possible to implement without the (STEM)2 program It is also evident that the initial year of funds has been leveraged greatly to maximize continuation of program activities In this regard, the funding has been both efficient and effective The initial seed of (STEM)2 support has facilitated other, Page 54 similar initiatives at the CSU to serve faculty and student development Unfortunately, the reciprocal benefits to students, faculty, and communities have not been fully measured due to a lack of available resources Overall, major lessons learned from this initiative include the following: While initial enthusiasm to integrate service learning into STEM curriculum may be strong, external funding is required to sustain such efforts Similarly, dedicated faculty and a commitment to inquiry-based learning are also required to maintain momentum for such initiatives Innovations subgrants were a useful process for creating new service-learning courses at multiple campuses The longer-term impacts have had mixed results with some efforts expanding while others have been suspended The STEM C3 presentations garnered a lot of interest from attendees, however, longerterm implementation of similar programs is yet to be seen at other CSU campuses particularly when external funding is lacking What were the most sustainable aspects of the (STEM)2 program? Across each of the elements of the (STEM)2 program, several themes have emerged While some elements of the program have been more sustainable than others, it is noteworthy to mention key findings from the compilation of (STEM)2 and related activities: Service-learning course experiences have positive impacts on students, faculty and the community However, these benefits are not entirely equal and often are contextdependent It takes more coordination and effort on the part of faculty to implement a servicelearning course Given the lag time of such preparation, adequate time must be allowed to maximize the student learning experience and community benefit New initiatives such as VISTA program and the partnerships with Business and Finance around Campus as a Living Lab effort will continue to build capacity for long-term sustainability Page 55 Conclusions Service learning in STEM has expanded throughout the CSU over the past three years, in part from the LSA funding and subsequently other funding partnerships The increase in servicelearning courses and numbers of students served, particularly in STEM, can be attributed to this effort Despite the lack of funding beyond the first year, the COCCE has successfully leveraged new and existing partnerships to continue integration of service learning in STEM throughout the state Other evidence that service learning in STEM has broadened throughout the CSU was captured by the self-assessment instruments conducted as pretests and posttests Most CSU campuses increased self-assessment ratings in the integration of service learning in STEM, and a good number have reached the sustained institutionalization stage These data are encouraging both in terms of evidence that the COCCE efforts have been successful over the past few years as well as the idea that more students, faculty and communities are expected to benefit from this increased integration Page 56 References Altman, I (1996) Higher education and psychology in the millennium American Psychologist, 51(4), 371–378 Batchelder, T H., & Root, S (1994) Effects of an undergraduate program to integrate academic learning and service: Cognitive, prosocial cognitive, and identity outcomes Journal of Adolescence, 17, 341–355 Bringle, R G., & Hatcher, J A (1995) A service-learning curriculum for faculty Michigan Journal of Community Service Learning, 2, 112–122 Driscoll, A., Holland, B., Gelmon, S., & Kerrigan, S (1996) An assessment model for servicelearning: Comprehensive case studies of impact on faculty, students, community, and institutions Michigan Journal of Community Service Learning, 3, 66–71 Giles, D., & Eyler, J (1994) The impact of a college community service laboratory on students' personal, social and cognitive outcomes Journal of Adolescence 17, 327-339 Markus, G B., Howard, J P F., & King, D C (1993) Integrating community service and classroom instruction enhances learning: Results from an experiment Educational Evaluation and Policy Analysis, 15, 410–419 Podolske, D., Dixon, C., Langford, P & Eddy, R M (2011) New partnerships for a new economy: A guide to effectively prepare STEM students for careers of choice The California State University, Office of the Chancellor: Center for Community Engagement Long Beach, CA Reeb, R N., Katsuyama, R M., Sammon, J A., & Yoder, D S (1998) The Community Service Self-Efficacy Scale: Evidence of reliability, construct validity, and pragmatic utility Michigan Journal of Community Service Learning, 5, 48–57 Page 57 Appendix A CSU Campus Participation Activities in STEM Service Learning 2011-13 Campus Innovation Subgrant San Diego Maritime Humboldt State Long Beach Sacramento Northridge San Francisco Channel Islands San Bernardino Pomona Chico East Bay Fullerton San Jose San Marcos Stanislaus Sonoma State San Luis Obispo Bakersfield Dominguez Hills Los Angeles Fresno Monterey Bay X X X X X X Laying the Foundation Subgrants X X X X X X X X X X X X X X X EDI participant X X X X Page 58 EDI Implementation Grant X X X X STEM C3 Symposium Intel ISEF 2011 SCE grantee SoCal Gas Grantee X X X X X X X X X X X X X X X X X X X X X - X X X - X X X X X X -