Baseline Data from Murray State University Student Data: Experiential Learning: Spring 2010-Summer 2012

Running head: EXPERIENTIAL LEARNING DATA Baseline Data from Murray State University Student Data: Experiential Learning: Spring 2010-Summer 2012 EXPERIENTIAL LEARNING DATA Abstract Student demographic and academic achievement data for two calendar years (Spring 2010 to Summer 2012) were used to create baselines for experiential learning outcome data Analyses were conducted by examining performance in experiential learning related-courses with respect to student demographics and academic achievement to determine baseline measures, to identify any specific demographic or academic indicators of experiential learning performance, and to compare performance and participation in required and optional experiential learning-related courses In a sample of experiential learning-related general education courses, high school GPA was the strongest predictor of success, even after accounting for ability Gender and ACT scores were also predictive, but not for all courses Performance in experiential learning-related courses was a significant predictor of retention: higher course grades were associated with increased probabilities of retention Student participation in optional experiential learning-related courses is low, and grades in all experiential learning-related courses were positively skewed This data will be used to develop the Murray State University’s Quality Enhancement Plan that promotes applying skills and knowledge learned in the classroom to realworld settings EXPERIENTIAL LEARNING DATA Introduction The Murray State University (MSU) community, including students, faculty, staff, alumni, and employers, identified the topic of applying knowledge and skills in a real world setting as a key area that the university should focus on improving Accordingly, MSU identified baseline data skills associated with the ability to apply knowledge and skills in a real world setting, and then conducted analyses of student data to determine whether these skills were a relative weakness for Murray State students Transdisciplinary skills that would enable students to apply knowledge and skills in a real world setting successfully were identified Because we wished to identify student learning outcomes that would be appropriate across all the disciplines pursued at Murray State, we sought construct definitions that were general and flexible enough to be applied to multiple approaches, assignments, and disciplines Consequently, the American Association of Colleges and University’s VALUE rubrics for critical thinking, creative thinking, inquiry and analysis, problem solving, and integrative learning were used for construct definitions and criteria The VALUE rubrics were developed at a national level by faculty experts who used existing rubrics and research to create and validate rubrics for institutional assessment of these common learning outcomes The following definitions from the VALUE rubrics were used:  Critical thinking is a habit of mind characterized by the comprehensive exploration of issues, ideas, artifacts, and events before accepting or formulating an opinion or conclusion  Creative thinking is both the capacity to combine or synthesize existing ideas, images, or expertise in original ways and the experience of thinking, reacting, and working in an EXPERIENTIAL LEARNING DATA imaginative way characterized by a high degree of innovation, divergent thinking, and risk taking  Inquiry is a systematic process of exploring issues, objects or works through the collection and analysis of evidence that results in informed conclusions or judgments Analysis is the process of breaking complex topics or issues into parts to gain a better understanding of them  Problem solving is the process of designing, evaluating and implementing a strategy to answer an open-ended question or achieve a desired goal  Integrative learning is an understanding and a disposition that a student builds across the curriculum and cocurriculum, from making simple connections among ideas and experiences to synthesizing and transferring learning to new, complex situations within and beyond the campus Method Data were compiled from two sources provided by the Registrar’s office One set of files consisted of semester-by-semester class rolls and grade data for every Murray State undergraduate enrolled from Spring 2010 to Summer 2012 A separate set of files provided individualized academic records for each student enrolled in each of these semesters This set of files included demographic data such as state of residence, high school attended, age, ethnicity, and gender Academic achievement data included ACT English, Math, Reading, and Composite test scores, high school rank, and high school GPA For students enrolled in multiple semesters, these data were redundant across the files However, each file also included dynamic data such as term GPA, enrollment status, and class standing EXPERIENTIAL LEARNING DATA Both sets of files were coded with unique and anonymous student identification numbers, making it possible to match the student data to grade data Different combinations of these data files were used in the analyses as noted in the Results section All analyses were completed using SPSS version 20 Analyses were conducted to determine baseline measure by examining performance in courses that included significant experiential learning components with respect to student demographics and academic achievement; to identify any specific demographic or academic indicators of experiential learning performance; and to compare performance and participation in required and optional experiential learning-related courses Results The first objective was to establish a baseline measure by examining performance in courses that included a significant experiential learning component Experiential learning components were defined as critical thinking, creative thinking, analysis, problem-solving, and integrative learning, and courses were identified as experiential learning-related courses though an analysis of course objectives Courses with more than 95% of the course objectives related to one or more of these components were considered experiential learning-related for the purposes of this study Appendix A provides the full list of identified courses and their course objectives Grade distributions were produced by aggregating enrollment files across the available semesters (Spring 2010, Summer 2010, Fall 2010, Spring 2011, Summer 2011, Fall 2011, Spring 2012, Summer 2012) as well as across course sections within semesters Grades were coded as A, B, C, D, E, AU (audit), and W (drop) Baseline distributions for selected critical thinking courses are presented in Appendix B In order to examine the relationship between performance in experiential learning-related courses and individual student demographics and academic achievement, it was first necessary to EXPERIENTIAL LEARNING DATA merge the enrollment and class rolls files This merge was done by first restructuring the aggregate class rolls file that contained a separate entry for each grade in each course so that each student was placed in their own row with each of their grades in columns, coded by course subject, number, and the semester in which it was taken This structure made it possible to match students’ anonymous identification numbers with the enrollment files containing their demographic and academic achievement data for each semester The restructuring was done using the merge function in SPSS Because a separate enrollment file was provided for each semester, individual student data was imported for the first semester in which the student was enrolled and redundant information was discarded However, because these files contained an enrollment status variable, it was possible to use these data to determine how many semesters in which the student was enrolled Thus, the merged file contained a dichotomous variable for each semester that indicated whether or not the student was enrolled that semester For the dynamic data, new variables were created for each additional semester for which a student had data These included term GPA, cumulative GPA, number of hours enrolled, and class standing The class standing variable in particular made it possible to select out specific cohorts for analysis The next objective was to determine whether any specific demographic or academic indicators of experiential learning performance could be identified To accomplish this goal, a hierarchical regression model was built that controlled for demographic factors in early steps and academic performance factors in later steps A sampling of courses from the core University Studies curriculum was selected in order to represent a broad range of areas of study (Accounting, Biology, World Civilizations, Geography, History, Humanities, Political Science, Psychology, and Spanish) Because grade distributions were relatively normal for these courses, EXPERIENTIAL LEARNING DATA grades were treated as a continuous dependent variable Drops and audits were not included in these analyses as there was no way to treat these variables as continuous with the grade variable A separate regression equation was computed for performance in each course (ACC 201, BIO 221, CIV 201, COM 161, ENG 105, GSC 110, HIS 221, HUM 211, POL 140, PSY 180, and SPA 102) Each step of the regression equation predicted performance in the course from the variable(s) entered at that step, controlling for any variance accounted for by variables in previous steps The specific model was as follows: Step 1: Gender Step 2: Ethnicity (White, Asian, Black, Hispanic, Multicultural) Step 3: State of origin (KY, TN, IL, IN, MO) Step 4: ACT (English, Math, Reading) Step 5: High school GPA Gender was treated dichotomously Ethnicity and State of origin were dummy-coded ACT scores and High school GPA were treated as continuous variables Patterns of results were similar across courses In general, females outperform males, although this difference was not always significant The gender gap was most pronounced in English and Psychology Generally, blacks underperformed relative to other ethnic groups However, this finding should be interpreted with a great deal of caution given their small representation in the sample Not surprisingly, ACT scores were highly predictive of success A couple of surprising findings are worth noting, however For BIO 221, only ACT English scores predicted success EXPERIENTIAL LEARNING DATA (controlling for ACT Math and ACT Reading) For most classes, however, ACT Math scores seemed to be the stronger predictor For ENG 105 and SPA 102, ACT Math was the only significant predictor (controlling for ACT English and ACT Reading) For two courses, POL 140 and PSY 180, all three ACT scores independently predicted success, suggesting that these courses in particular require a highly integrated set of academic skills In the final step of the model, high school GPA scores were also generally predictive of success This result was true for all of the courses except ACC 200, HIS 221, and SPA 102 This suggests that even after accounting for ability, past performance is a strong predictor of future performance Having established some of the antecedents of performance in a sampling of experiential learning-related courses, a third objective was to determine whether performance in these courses was predictive of academic success in general Retention in particular was a variable of interest; that is, are students who are successful in experiential learning-related courses more likely to remain at Murray State? In order to examine retention, a longitudinal approach has to be taken—to look at the impact of performance in one semester on a relevant criterion variable in a future semester To this analysis, the Fall 2010 first-time freshmen cohort was selected Retention was defined simply as a dichotomous variable indicating whether or not the student was enrolled Spring 2011 To determine if performance in an experiential learning-related course could predict whether first-time freshmen returned for a second semester, the previous model was used with the following modifications: first, the addition of performance in the course in question as a 6th step in the regression equation, and second, controlling for term GPA to uncouple performance in the course from overall semester performance Because the outcome variable was dichotomous, the logistic regression model was used Logistic regression provides a EXPERIENTIAL LEARNING DATA likelihood statistic known as an odds ratio that can be translated into a probability that the predictor contributes to inclusion in the variable of interest (in this case, retained vs not retained) Three courses from the list of experiential learning-related courses with high firstsemester enrollment rates were examined: PSY 180, ENG 105, and COM 161 For all three courses, performance was a significant predictor of retention Each course grade increase was associated with increased probabilities of retention ranging from 53% (PSY 180) to 57% (ENG 105) That said, these probabilities, while significant, were lower than those based on overall term GPAs Nevertheless, they were unique indicators of retention A final objective was to examine participation rates in courses that have a strong experiential learning component to see if there are differences in participation when the course is required or optional Courses were identified as experiential learning-related courses based on both the course description provided in the Murray State University Course Bulletin and the course objectives on syllabi Courses were split into two groups: required and optional experiential learning-related classes Participation in optional experiential learning-related courses is extremely low, averaging 21% of those students enrolled in experiential learning courses across the university, and ranging from 7% to 32% of enrollment in all colleges or schools except for the College of Business The College of Business is an anomaly, with 76% of participation in experiential learning-related courses in optional courses and only 24% in required courses Required courses are either pass/fail or, if graded, predominantly As were earned Graded courses were divided into two groups: those in which 80% of more of the students earned As, and those in which fewer than 80% of students earned As In 41% of all required EXPERIENTIAL LEARNING DATA 10 courses, 80% or more of the students in that class earned As When the target was lowered to 70% of students in the course earning As, the sample of required courses was evenly split, with 50% of all required courses awarding As to 70% or more of enrolled students A similar analysis conducted on pass/fail required courses revealed that 86% of required courses using a pass/fail grading scheme awarded passes to 80% or more of enrolled students Optional courses were also skewed in student performance In 66% of optional experiential learning-related courses, 80% or more of the students in that class earned As In 76% of optional experiential learning-related courses using a pass/fail grading scheme, 80% or more of the students in that course earned passes While predictive analyses were planned prior to data analysis, interpretation of the descriptive statistics led us to abandon these predictive analyses For optional courses, enrollment was too low to provide meaningful results Even collapsing across courses would not provide a useful prediction of student success because the few courses that have higher enrollments would carry the bulk of the variance and, thus, skew the results Similarly, the lack of variance in grade distributions in required experiential learning-related courses did not provide enough variance to predict any meaningful outcomes Discussion Across the broad range of areas of study (Accounting, Biology, World Civilizations, Geography, History, Humanities, Political Science, Psychology, and Spanish), gender, ethnicity, ACT, and high school GPA were found to be specific demographic/academic indicators of experiential learning-related performance State of origin was not found to have an impact on experiential learning-related performance, which suggests that variance in state and school