Medical Students at Risk: Application of Bootstrap Resampling Abolfazl Ghasemi Arkansas College of Osteopathic Medicine Introduction According to Stegers-Jager et al., (2012), “medical schools wish to better understand why some students excel academically and other have difficulty in passing medical courses” (p.679) Although undergraduate and graduate applicants are considered as the most talented and highly motivated students, not everyone can come to grips with medical school courses, trainings, and residency to become a competent physician Hence, it is not unusual that “approximately to 28 percent of medical trainees, regardless of their level of training or specialty, will require remediation in the form of an individualized learning plan to achieve competence” (Gurreasio et al., 2014, p.352) Winston et al (2014) suggested, “prediction and prevention of failure, or remediation after failure” as two proper strategies for dealing with this problem (p.26) To predict and prevent the failure a head of time, proper statistical analysis is needed The present study will extend existing knowledge about the timing of failure and results robustness by applying the bootstrap method Methods Design According to Banjanovic and Osborne (2016), “bootstrap resampling is a systemic method of computing CI for nearly any estimate” (p.2) Generally, this technique is useful where sample size is not enough, additional data cannot be obtained, and/or the data is not normally distributed For instance, as small sample size may not be a typical of the underlying population, we can use bootstrap to realize how well the statistical theory holds (Curran-Everett, 2009) Data Data included 1670 student’s records of to 17 exams during first and second years of pre-clinical education at Ohio University, Heritage College of Osteopathic Medicine throughout academic year of 2000 to 2014 Based on Cox Regression, we have defined right and left censored and have assigned them and respectively A total of 23512 observations i.e student/exam have been used for this simulation Results Conclusion Hazard Ratio Confidence Interval with 1000 resampling Hazard Ratio Confidence Interval with 1000 resampling Gender Coef HR LCI UCI Race Coef HR LCI UCI Main 0.09 1.10 0.95 1.27 Main 0.11 1.13 1.05 1.20 S1 -0.16 0.88 0.73 0.98 S1 0.11 1.12 1.04 1.19 S2 0.11 1.12 0.97 1.30 S2 0.09 1.10 1.03 1.18 S3 0.16 1.18 1.03 1.35 S3 0.16 1.18 1.10 1.26 S4 0.01 1.02 0.88 1.18 S4 0.14 1.16 1.08 1.23 S5 -0.13 0.90 0.76 1.02 S5 0.15 1.16 1.09 1.23 … S1000 … -0.14 0.90 0.74 1.02 S1000 0.09 1.10 1.02 1.18 The results of 1000 Bootstrap resampling of the Cox Regression hazard ratio obtained from R are reported in tables and graphs Interestingly, the results are not consistent with the main Cox Regression model After bootstrapping, the hazard ratio of (null hypothesis of two groups are equal) is included in the 95% confidence interval of hazard ratio range and we cannot reject the null hypothesis This include, Gender, Age, First generation (FG), in-state, and MCAT Bio However, other factors are not including the hazard ratio of 1, meaning that either group is above or below hazard ratios 1and they are consistent with previous results Discussion To avoid the large sample size fallacy, it is highly recommended to not use Null Hypothesis Significance Testing (NHST) as the sole determination of the relevance of the predictors Since variables of interest in the study have different hazard ratios, it would be much reasonable to calculate the confidence interval of effect sizes (i.e hazard ratios) as they are offering more information and context as opposed to NHST of a statistic The bootstrapping method were found to be useful over resampling the Cox Regression model With more certainty about the results, we can generate our forecasting model based on these factors and have more accuracy in terms of finding students at risk References: Banjanovic, E S., & Osborne, J W (2016) Confidence Intervals for Effect Sizes: Applying Bootstrap Resampling Practical Assessment, Research & Evaluation, 21(5) Curran-Everett, D (2009) Explorations in statistics: the bootstrap Advances in Physiology Education, 33(4), 286–292 doi.org/10.1152/advan.00062.2009 Guerrasio, J., Garrity, M J., & Aagaard, E M (2014) Learner deficits and academic outcomes of medical students, residents, fellows, and attending physicians referred to a remediation program, 2006-2012 Academic Medicine: Journal of the Association of American Medical Colleges, 89(2), 352–358 doi.org/10.1097/ACM.0000000000000122 Stegers-Jager, K M., Cohen-Schotanus, J., & Themmen, A P N (2012) Motivation, learning strategies, participation and medical school performance Medical Education, 46(7), 678–688 doi.org/10.1111/j.1365-2923.2012.04284.x Winston, K A., Vleuten, C P M van der, & Scherpbier, A J J A (2014) Prediction and prevention of failure: An early intervention to assist at-risk medical students Medical Teacher, 36(1), 25–31 doi.org/10.3109/0142159X.2013.836270