All Faculty Meeting Presentation Rev

GENERAL EDUCATION ASSESSMENT RESULTS ALL FACULTY MEETING OCTOBER, 24, 2017 Presentation of General Education Assessment Goals, Procedures, Data, and Conclusions – Thad Meeks (Chair of General Education Assessment Committee) Objectives for General Education and the Baccalaureate Degree • Analytic, Problem Solving, and Decision-Making Skills • Oral and Written Communication • Foundations in Liberal Arts • Value of Diversity • Scientific Literacy • Ethics First Cycle of Gen Ed Assessment First Cycle Analytic Problem Solving Communication (Written) Scientific Literacy Foundations Communication in Liberal (Oral) Arts Value of Diversity Ethics Identify Measurable Outcomes '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 Develop rubrics '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '14-'15 '14-'15 Map outcomes to the gen ed curriculum '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 '13-'14 Collect evidence '13-'14 '13-'14 '14-'15 '15-'16 '15-'16 '15-'16 '15-'16 Score with rubrics (team of faculty) '14-'15 '14-'15 '14-'15 '15-'16 '15-'16 '15-'16 '15-'16 Analyze results '14-'15 '14-'15 '14-'15 '15-'16 '15-'16 '16-'17 '16-'17 Act on findings to improve curricular and cocurricular '15-'16 '15-'16 '15-'16 '16-'17 '16-'17 '16-'17 '16-'17 Revise assessments '15-'16 '15-'16 '15-'16 '16-'17 '16-'17 '16-'17 '16-'17 Second Cycle of Gen Ed Assessment Second Cycle Analytic Problem Solving Communication (Written) Scientific Literacy Foundations Communication in Liberal (Oral) Arts Value of Diversity Ethics Collect evidence '17-'18 '17-'18 '17-'18 '18-'19 '18-'19 '19-‘20 '19-‘20 Score with rubrics (team of faculty) '17-'18 '17-'18 '17-'18 '18-'19 '18-'19 '19-‘20 '19-’20 Analyze results '18-'19 '18-'19 '18-'19 '19-‘20 '19-‘20 ‘20-’21 ‘20-’21 Act on findings to improve curricular and cocurricular '18-'19 '18-'19 '18-'19 '19-‘20 '19-‘20 ‘20-’21 ‘20-’21 Revise assessments '18-'19 '18-'19 '18-'19 '19-‘20 '19-‘20 ‘20-’21 ‘20-’21 Written Communication • Early attempts were made to collect existing artifacts from a variety of classes and score them on different written communication and critical thinking criteria • The mean scores for the upperclassmen were statistically higher than for the lowerclassmen • The primary limitations were the inconsistencies in the types of and requirements for the written assignments • Led the committee to seek a more standardized way to assess written communication (and perhaps other objectives) Critical Thinking Assessment Test (CAT) • 15-item test designed to assess and promote the improvement of critical thinking and real-world problem solving skills • Developed by the Center for Assessment and Improvement of Learning at Tennessee Tech University with funding from NSF • The test is scored by the faculty at the participating institution • Assesses four broad skills that align with our goals of Analytic Problem Solving, Written Communication, and Scientific Literacy • Evaluation and Interpretation of Information (Scientific Literacy) • Problem Solving (Analytic Problem Solving) • Creative Thinking (Scientific Literacy and Analytic Problem Solving) • Effective Communication (Written Communication) CAT Procedure • Thad Meeks (Chair of General Education Assessment Committee) and Zenia Agustin (Director of General Education) trained on the CAT process • Collected data from freshmen in NFS course (n = 141) and seniors in IS courses (n = 105) in the spring of 2015 • Gen Ed Assessment Committee scored the freshman data over the course of the spring 2015 semester and the senior data at the beginning of summer 2015 • At least two raters per item (a third rater settled disagreements) • Ratings were sent to the Center for Assessment and Improvement of Learning for scoring CAT Results; * = Significant Group Difference * * * * Conclusions from CAT • The positives were that overall (and with three subscales), there were statistical differences between freshmen and seniors • Despite this, the effects were not necessarily large and no effect in Evaluating/Interpreting Information • The limitations of the process included class selection, the timing of testing, the motivation of the students, and the scoring process • We are currently collecting and will soon be assessing new CAT data We are attempting to address some of these limitations Oral Communication Procedure • In the spring of 2016, we collected data from recorded student presentations in ACS 101 (n = 109) and various other sources for upperclassmen (e.g., IS courses, senior assignment presentations), (n = 69) • Students’ oral presentations were scored by the General Education Assessment Committee using a rubric on a 1-4 scale (higher numbers represented better performance) • Three subscales: • Organization • Language • Delivery Oral Communication Results; * = Significant Group Difference * * * * * * * * Conclusions from Oral Communication Data • Overall, and on all three subscales, there were significant differences favoring the upperclassmen • The primary limitation of this process was that the variety of presentations that were assessed made scoring consistency difficult Diversity and Ethics • These data were gathered as a part of NSSE Focus Groups in May of 2016 (60 total students) • One question related to students’ exposure to different types of perspectives (i.e., ethnic, political, economic, and social) • 43 responders claimed a combination of classes as their means of exposure • One question related to students’ exposure that helped them develop their own code of values and ethics • 32 responders claimed classroom experiences (e.g., discussion) as the source of this exposure • What can we conclude from these data? How can we better assess this? Overall Conclusions • In a positive sense, we showed statistically significant differences between our lower and upperclassmen, except in one subscale of the CAT • Challenges • Better measures • More student motivation • More consistent sources of data • More consistent scoring of data • But all of this requires more buy-in for assessment • Need for faculty assistance • Need to “close the loop” ... recorded student presentations in ACS 101 (n = 109) and various other sources for upperclassmen (e.g., IS courses, senior assignment presentations), (n = 69) • Students’ oral presentations were... Communication Data • Overall, and on all three subscales, there were significant differences favoring the upperclassmen • The primary limitation of this process was that the variety of presentations that... assess this? Overall Conclusions • In a positive sense, we showed statistically significant differences between our lower and upperclassmen, except in one subscale of the CAT • Challenges • Better