FSUEdLead Advisory Board Presentationpptx

Florida State University Educational Leadership Program Advisory Board Meeting Florida Educational Leadership Examination (FELE) Update Lauren White, Florida Department of Education Süleyman Olgar, Florida Department of Education February 1, 2019 www.FLDOE.org Office of Assessment Vince Verges Assistant Deputy Commissioner Bureau of Postsecondary Assessment Phil Canto Bureau Chief Süleyman Olgar, Ph.D Psychometrics, Administration, Scoring & Reporting Director Lauren White Test Development Supervisor www.FLDOE.org Today’s Postsecondary Assessment Presentation Current status of FELE program 2015-2018 FELE Testing Outcomes • • • • FSU FELE Pass Rates and Competency Performance FELE Pass Rates and Competency Performance State Approved Program Examinees Performance Top Performer Programs Available FELE Resources • • • Sample Passing/Non-Passing Responses Interactive Test Information Guide Examinee and faculty resources Q & A www.FLDOE.org Postsecondary Assessment Vision To maximize the number of skilled, knowledgeable, and highly effective educators across Florida schools Postsecondary Assessment Mission To identify beginning effective educators and leaders by developing fair, valid, and reliable assessments; administering these assessments under equitable and secure conditions; and scoring and reporting the results accurately and in a timely manner www.FLDOE.org FTCE/FELE Program at a Glance • • • • • Over 50 different tests and subtests available Over 180K tests/subtests administered in 2018 38 CBT centers in Florida and 249 around the country 90 military test sites including overseas bases General Knowledge test, Professional Education test, and a Subject Area examination is required for certification • Anyone can take an FTCE or FELE test (no program or administrative requirements) • Program flyer: LINK www.FLDOE.org FELE Program Updates • No planned substantive changes to current FELE content, structure, passing scores, or examination fees • Ongoing enhancement and expansion of M/C item pools for all three subtests - We constantly monitor to ensure quality control • New WPA sample written responses on contractor site www.FLDOE.org FELE Outcomes www.FLDOE.org FLORIDA EDUCATIONAL LEADERSHIP EXAMINATION (FELE) Pass Rate Comparison All Examinees First Attempt Florida State University Test Name FELE Subtest 1: Leadership for Student Learning FELE Subtest 2: Organizational Development FELE Subtest 3: Systems Leadership (Multiple Choice Section) FELE Subtest 3: Systems Leadership (Written Performance Section) N Pass Mean Rate Score 53 85% 52 Best Attempt Florida State University Statewide Statewide Pass Rate Mean Score N Pass Mean Rate Score 210.4 1,956 69% 205.0 53 98% 212.8 1,946 88% 209.0 90% 212.0 1,943 66% 203.5 54 98% 213.1 1,910 88% 208.1 57 84% 211.3 1,967 60% 201.2 57 98% 214.3 1,943 86% 207.8 57 63% 8.1 1,961 59% 7.5 58 95% 90% 8.3 N 9.2 N 1,927 Pass Mean Rate Score Note: Data matching for Florida State University is based on Title II records for examinees who had an attempt between August 2015- July 2017 Statewide is based on all Title II examinee records from July 2015- August 2017 www.FLDOE.org FLORIDA EDUCATIONAL LEADERSHIP EXAMINATION (FELE) Competency Performance Comparison Best Attempt (All Examinees) Test Name Competency Name Florida State University N % Correct Statewide N % Correct Florida Educational Leadership Examination (FELE) 01 (17-18) Facilitate student learning goals 77.9 02 (18) Prioritize student learning via leadership FELE Subtest 1: Leadership for Student Learning FELE Subtest 2: Organizational Development 03 (15-17) Develop/implement instructional framework 73.7 79.0 53 74.6 76.6 1,946 73.1 04 (17) School environment for student learning 74.8 74.1 01 (11-12) Recruitment and induction practices 77.3 76.6 02 (22-23) Faculty/staff development and retention 03 (16-17) Develop leadership w/in the organization 54 76.9 77.9 1,910 74.2 74.2 04 (17-18) Personal/professional behavior practices 78.7 74.5 FELE Subtest 3: Systems Leadership 01 (13-14) Decision-making processes 79.5 72.4 (Multiple Choice Section) 02 (13-14) School operation management practices 57 76.9 1,943 74.9 03 (14) Financial and resource management 73.4 70.1 04 (13) School legal practices and applications 75.6 71.9 FELE Subtest 3: Systems Leadership 05 (1) Effective communication practices 58 76.4 1,927 69.0 (Written Performance Section) Note: Note: Data matching for Florida State University is based on Title II records for examinees who had an attempt between August 2015- July 2017 Statewide is based on all Title II examinee records from July 2015- August 2017 The numbers in the parenthesis indicate range of item counts per competency Percentages correct are based on average examinee performance on each competency www.FLDOE.org FLORIDA EDUCATIONAL LEADERSHIP EXAMINATION (FELE) Pass Rate Comparison By Gender Status (Female) First Attempt Florida State University Test Name FELE Subtest 1: Leadership for Student Learning FELE Subtest 2: Organizational Development FELE Subtest 3: Systems Leadership (Multiple Choice Section) FELE Subtest 3: Systems Leadership (Written Performance Section) Best Attempt Florida State University Statewide Statewide N Pass Rate Mean Score N Pass Rate Mean Score N Pass Rate Mean Score N Pass Rate Mean Score 36 89% 212.0 1,463 89% 206.0 36 100% 213.5 1,446 89% 209.6 36 86% 210.4 1,454 86% 203.6 35 97% 212.0 1,418 88% 208.2 40 80% 209.4 1,472 80% 200.4 40 98% 213.0 1,439 86% 207.5 40 63% 8.2 1,470 63% 7.5 40 95% 9.3 1,440 90% 8.3 Note: Data matching for Florida State University is based on Title II records for examinees who had an attempt between August 2015- July 2017 Statewide is based on all Title II examinee records from July 2015- August 2017 www.FLDOE.org 10 FELE Pass Rates by Subtest (First vs Best Attempt) (August 2015 to July 2018) First-Attempt Best-Attempt # % # Pass Takers Pass Mean # % # Pass Total Takers Pass Score Mean Total Score 4,942 3,198 65% 203.4 5,039 4,058 81% 206.6 4,922 3,125 63% 202.6 5,029 4,085 81% 206.1 FELE Subtest 3: System Leadership (MC) 4,932 2,839 58% 200.1 5,069 4,005 79% 205.4 FELE Subtest 3: System Leadership (WPA) 4,926 2,728 55% 82% 8.0 Subtest FELE Subtest 1: Leadership for Student Learning FELE Subtest 2: Organizational Development 7.2 5,061 4,149 Note: Reports are generated from Results Analyzer, November 2018 The results are based on all examinee test records from August 2015 until July 2018 www.FLDOE.org 16 FELE Program Pass Rates (First vs Best Attempts) (August 2015 to July 2018) First Attempt Best Attempt # Takers # Pass % Pass Mean Total Score 1,746 1,171 67% 204.4 1,795 1,555 87% 208.1 1,739 1,125 65% 203.3 1,777 1,537 86% 207.5 FELE Subtest 3: System Leadership (MC) 1,749 1,031 59% 201.0 1,825 1,547 85% 207.5 FELE Subtest 3: System Leadership (WPA) 1,748 1,050 60% 7.5 1,823 1,637 90% 8.3 FELE Subtest FELE Subtest 1: Leadership for Student Learning FELE Subtest 2: Organizational Development # Takers # Pass % Pass Mean Total Score Note: Reports are generated from Results Analyzer, December 2018 Data matching is done based on Title II records up to 2016-17 School Year The results are based on examinee testing from August 2015 until July 2018 www.FLDOE.org 17 Examinee Performance on FELE Competencies (Programs vs Total) (August 2015 to July 2018) Examination Information Subtest Competency Name* 01 (17-18) Facilitate student learning goals 02 (16-18) Prioritize student learning via leadership FELE Subtest 1: Leadership for Student Learning 03 (15-17) Develop/implement instructional framework FELE Subtest 2: Organizational Development FELE Subtest 3: System Leadership Multiple-Choice (MC) Programs # of % # of % Takers Correct Takers Correct 73% 72% 1,795 76% 72% 5,039 75% 72% 04 (15-17) School environment for student learning 74% 73% 01 (11-12) Recruitment and induction practices 77% 76% 02 (20-23) Faculty/staff development and retention 03 (16-18) Develop leadership w/in the organization 1,777 74% 74% 5,029 73% 73% 04 (17-18) Personal/professional behavior practices 74% 73% 01 (13-14) Decision-making processes 72% 72% 02 (12-14) School operation management practices 03 (13-15) Financial and resource management 1,825 04 (13-14) School legal practices and applications FELE Subtest 3: System Leadership Written Performance (WPA) Total 05 (1) Effective communication practices 75% 70% 5,069 72% 1,823 69% 74% 69% 71% 5,061 66% * Numbers in the paranthesis indicate range of number of operational items on test forms Based on best attempt Note: Percent correct indicates average performance of all examinees (# of Takers) on the range of items for each competency www.FLDOE.org 18 FLORIDA EDUCATIONAL LEADERSHIP EXAMINATION (FELE) Competency Performance Comparison 2015-2018 First-time Examinees Test Name Florida State University Competency Name N % Correct Florida Educational Leadership Examination (FELE) 01 (17-18) Facilitate student learning goals 78.0 FELE Subtest 1: Leadership for Student Learning FELE Subtest 2: Organizational Development 02 (18) Prioritize student learning via leadership 52 03 (15-17) Develop/implement instructional framework 04 (17) School environment for student learning 01 (11-12) Recruitment and induction practices 02 (22-23) Faculty/staff development and retention 04 (13) School legal practices and applications N % Correct 51 77.3 2,135 N 76.4 2,111 74.0 5,014 4,950 72.9 72.9 72.2 70.8 74.3 2,136 74.9 71.8 72.5 75.6 73.9 73.9 71.5 77.1 % Correct 71.7 72.9 73.5 76.7 78.1 79.6 79.1 55 Statewide (All) 73.4 74.8 75.0 78.2 03 (16-17) Develop leadership w/in the organization 04 (17-18) Personal/professional behavior practices 01 (13-14) Decision-making processes FELE Subtest 3: Systems 02 (13-14) School operation management practices Leadership (Multiple Choice Section) 03 (14) Financial and resource management 78.5 Statewide (Program) 72.8 4,989 74.2 69.2 68.3 75.5 71.5 69.9 FELE Subtest 3: Systems Leadership 05 (1) Effective communication practices 56 75.7 2,121 68.3 4,938 65.5 (Written Performance Section) Note: Data matching for Florida State University and Statewide (Program) is based on Title II records for examinees who had a first-time testing attempt between January 2015-January 2018 The numbers in the parenthesis indicate range of item counts per competency Percentages correct are based on average examinee performance on each competency Statewide (All) is based on all first-time examinee records from January 2015-January 2018 www.FLDOE.org 19 FELE Program Pass Rates (Jan 15 to July 18) Top Performers FELE Subtest FELE Subtest Institution Institution N N % Passed Mean Score % Passed Mean Score Florida State University 55 91% 212.1 Florida State University 56 86% 211.1 University of Florida 49 80% 208.4 University of Florida 49 78% 208.3 Stetson University 32 78% 210.9 University of South Florida 100 77% 208.7 Florida Gulf Coast University 54 74% 207.8 Stetson University 34 76% 208.2 University of South Florida 100 74% 207.3 University of North Florida 105 76% 208.0 Florida International University 46 74% 204.1 Florida Gulf Coast University 54 76% 207.5 USF - St Petersburg 69 74% 206.1 American College of Education 123 75% 207.2 University of North Florida 104 70% 205.3 Florida Atlantic University 77 70% 204.6 Source: Results Analyzer, December 2018 Data matching is done based on Title II records up to 2016-17 School Year The results are based on examinee testing from January 2015 until July 2018 Ranking is based on at least 20 examinees www.FLDOE.org 20 FELE Program Pass Rates (Jan 15 to July 18) Top Performers FELE Subtest – WPA FELE Subtest – MC Institution N Institution N % Passed Mean Score % Passed Mean Score USF - St Petersburg 69 75% 8.1 Florida State University 61 84% 210.8 National Louis University 20 70% 7.8 USF - Sarasota-Manatee 26 81% 205.8 Florida International University 47 68% 7.9 University of Florida 50 74% 206.6 Florida State University 61 64% 8.2 Stetson University 33 73% 209.7 Stetson University 33 64% 7.8 National Louis University 20 70% 205.1 Florida Atlantic University 81 63% 7.5 University of North Florida 104 63% 7.6 Source: Results Analyzer, December 2018 Data matching is done based on Title II records up to 2016-17 School Year The results are based on examinee testing from January 2015 until July 2018 Ranking is based on at least 20 examinees www.FLDOE.org 21 Program Updates & Initiatives www.FLDOE.org 22 Initiatives - Scoring & Reporting • Weekly Scoring and Reporting • More often holistic scoring (bi-weekly) • Score reports to be released within weeks regardless of score report delay or examination type • most multiple-choice exams to be released within weeks* • performance exams to be released within weeks *Scores available within weeks when additional quality-control required • Interactive Test Information Guides • Score Verification — increased from sites across Florida to sites across Florida • Effective August 1, 2018, score verification fee is being refunded for any examinee who receives an updated score report www.FLDOE.org 23 Score Report Dates at a Glance Multiple Choice Exams Performance Exams Test Date Score Report Date Test Date Score Report Date 12/31/18–01/06/19 01/14/19 12/31/18–01/13/19 01/28/19 01/07/19–01/13/19 01/21/19 01/14/19–01/27/19 02/11/19 01/14/19–01/20/19 01/28/19 01/28/19–02/10/19 02/25/19 01/21/19–01/27/19 02/04/19 02/11/19–02/24/19 03/11/19 01/28/19–02/03/19 02/11/19 02/25/19–03/10/19 03/25/19 www.FLDOE.org 24 Sample Passing and Not-Passing WPA Responses • Posted January 2nd, 2019 • Include sample passing and not-passing responses • Carefully selected by SMEs • Available publically at no cost • Use of exact words and phrases will result in a Not Passing Score due to the lack of original work • Sample performance characteristics are provided (next slide) • Link: Sample Responses www.FLDOE.org 25 WPA Performance Characteristics Sample Passing Response Criteria Rating Category Description Data Findings The writing sample clearly presents a topic in its first paragraph relevant to the needs for improvement in High School LMN The data interpretation includes most of the relevant data points (specific numbers) for all students and subgroups with analysis Most of the data includes at least first and last years’ data comparisons Communication The response shows understanding that reading is the most significant area of need Plans included targeted professional development, mentoring, coaching and Professional Learning Communities to improve instruction in reading The examinee included an improvement plan for math; however, it was clearly a lower priority than reading In addition, the plan includes methods of monitoring instructional improvement, such as benchmark and formative assessments, teacher observations, and informal evaluations Writing Mechanics The level of writing is sufficient to maintain the point of view and present data and plans coherently, with few errors in mechanics www.FLDOE.org 26 Available FELE Resources www.FLDOE.org 27 Faculty Resources • http://www.fl.nesinc.com/FL_FacultyResources.asp • Includes faculty-relevant information regarding • • • • understanding FTCE/FELE test content IHEs assistance to candidates in preparing for the exams Interpreting candidate score reports Purchasing vouchers • Curriculum Mapping Tool, including tutorial and blank map documents Potential uses: • Faculty self-evaluation of curriculum • Program-level evaluation of curriculum • University-level interdisciplinary cooperation www.FLDOE.org 28 FELE Resources (Links to documents) 2017 Annual Administration and Technical Report Curriculum Mapping Tool FELE Interactive Test Information Guide WPA Resources: • Scoring Rubric • Sample Prompt #1 • Sample Prompt #1 Supplemental Rating Criteria • Sample Prompt #2 • Sample Prompt #2 Supplemental Rating Criteria • Sample Prompt #3 • Sample Prompt #3 Supplemental Rating Criteria • WPA Sample Responses www.FLDOE.org 29 www.FLDOE.org www.FLDOE.org 30