Revitalizing the Signaling Power of Class Rank at Colby College

Colby College Digital Commons @ Colby Honors Theses Student Research 2010 Revitalizing the Signaling Power of Class Rank at Colby College Nicholas Van Niel Colby College Follow this and additional works at: https://digitalcommons.colby.edu/honorstheses Part of the Educational Assessment, Evaluation, and Research Commons, and the Other Economics Commons Colby College theses are protected by copyright They may be viewed or downloaded from this site for the purposes of research and scholarship Reproduction or distribution for commercial purposes is prohibited without written permission of the author Recommended Citation Van Niel, Nicholas, "Revitalizing the Signaling Power of Class Rank at Colby College" (2010) Honors Theses Paper 573 https://digitalcommons.colby.edu/honorstheses/573 This Honors Thesis (Open Access) is brought to you for free and open access by the Student Research at Digital Commons @ Colby It has been accepted for inclusion in Honors Theses by an authorized administrator of Digital Commons @ Colby Revitalizing the Signaling Power of Class Rank at Colby College1 Nicholas Van Niel Colby College May 2010 Abstract Consistent with trends at American colleges and universities nationwide, grades have been monotonically increasing at Colby College over the past decade while controlling for student aptitude A rich data set that describes every Colby student over three cohorts is used to predict student performance By comparing the mean predicted grade to the mean actual grade in a department, it is seen that some departments award mean grades that are significantly higher than predicted grades warranted by explainable factors With some departments giving significantly higher grades than other departments, the current grading system is susceptible to awarding unwarranted higher grades to students who major in those departments Therefore, a normalized system of class rank is proposed in which performance is standardized against the mean grade at that particular class level in the respective department The alternative system of class rank appears to mitigate the detrimental effects that large discrepancies in mean grades between departments engender I would like to thank Professor Philip Brown and Professor Michael Donihue for their support and recommendations I would also like to thank Bill Wilson for providing data for such commentary I Introduction Average grades have increased at such a drastic pace over the past half century that nine in ten grades awarded at Colby College are in the A or B range Such findings are not unique as evidenced by the literature, which demonstrates that grades have increased significantly at American universities and colleges since the 1960s.2 For example, Juola (1976, 1980) find that the average GPA rose by 0.432 points, on average, between 1960 and 1974 in their sample of over 180 colleges (Table 1) Sabot and Wakeman-Linn (1991) find that the mean grade in introductory courses of eight large departments at Williams College rose from 2.49 in 1962-1963 to 2.93 in 1985-1986 In the other eight institutions in their sample, the mean grade in introductory courses of the same eight departments rose from 2.38 in 1962-1963 to 2.91 in 19851986 In a survey of 4,900 undergraduates from all types of institutions between 1969 and 1993, Levine and Cureton (1998) find that the percentage of A‘s awarded increased from percent to 26 percent while the share of C‘s decreased from 23 percent to percent Kuh and Hu (1999) compare the GPAs of 52,256 students from 112 institutions to find that student grades rose from 3.07 in the mid-1980s to 3.34 in the mid-1990s Gose (1997) finds that the mean GPA at Duke University rose from 2.7 in 1969 to 3.3 in 1996; at Lehigh University, it rose from 2.60 in 1972 to 2.90 in 1996; at Pacific Lutheran University it rose from 2.99 in 1974 to 3.20 in 1996; at University of California, Berkeley it rose from 2.95 in 1986 to 3.10 in 1996; and at the University of Washington, it rose from 2.31 in 1964 to 3.12 in 1996 34 McSpirit and Jones (1999) also find a consistent climb—more than 0.10 grade points every five years—in the There is a large debate in the literature concerning whether the increases in grades can be labeled as ‗grade inflation.‘ In order to remain agnostic, I avoid the term Eckert (1988), Alexander (1993), Cole (1993), Shea (1994), Johnson (1997), and Healy (2001) also report conclusive evidence that support steady increases in the average GPA during the latter half of the 20 th century More recent evidence of increases in grades can be found at http://cs.furman.edu/~chealy/stats/ A summary of this data was published on April 18, 2010 in the New York Times: Data; A as the New B average graduating GPA between 1983 and 1996 while controlling for aptitude, institutional, and other demographic factors While the evidence presented above indicates the prevalence of rising grades across a wide range of educational institutions, Rosovsky and Hartley (2002) report that increases in grades are especially prevalent at elite, selective colleges and universities; 91 percent of Harvard University‘s Class of 2001 graduated with Latin honors Indeed Lambert (1993) reports that the proportion of undergraduate grades that were an A- or higher Harvard University increased from 22 percent to 43 percent between 1966 and 1991 Valle (1993) finds that 88 percent of grades awarded at Stanford University were either in the A or B range between 1991 and 1993 Archibold (1998) reports that 83 percent of the grades awarded at Princeton University between 1992 and 1997 were between the A+ and B- range, compared to only 69 percent between 1973 and 1977 In addition, the median GPA at Princeton University rose from 3.08 for the Class of 1973 to 3.42 for the Class of 1997 These increases in average grades awarded over the past half century may be warranted if student ability has increased proportionally over the same period; however, Bowen and Bok (1998) and McSpirit and Jones (1999) find that when considered alongside multiple indices of student achievement, the increases in GPA not appear to be justified Birnbaum (1977) and Winzer (2002) note that there are institutional pressures that help explain some of the increases in grades over the past half century Rosovsky and Hartley (2002) point to the Vietnam War and the turmoil of the 1960s as the primary driver of increases in grades, for professors knew that awarding poor grades caused students to drop out of school and made the students subject to military service in Vietnam In addition, there have been curriculum adjustments that have engendered increases in grades During the turmoil of the 1960s, schools provided their students with the option to drop classes late in the semester or to ex post change a letter grade to a pass/fail mark Institutions also believed that awarding their students higher grades would improve retention and attract applicants Increases in grades have also led to significant differences in mean grades between departments Sabot and Wakeman-Linn (1991) find that Art, Music, English, Philosophy, Political Science, and Psychology were high-grading departments at Williams College, with a mean grade in introductory courses of 3.03, while Chemistry, Economics, and Math were lowgrading departments with a mean grade in introductory courses of 2.67 in 1985-1986 Consistent with the evidence provided by Sabot and Wakeman-Linn (1991), Quinones (2008) reports that grades are higher in the humanities at Princeton University: between 2001 and 2004, A‘s accounted for 55.6 percent of student grades in the humanities, 43.3 percent in the social sciences, and 37.2 percent in the natural sciences Johnson (2003) also finds that grades at Duke University tend to be higher in the humanities than in the natural and social sciences These differences would be warranted if the quality of students was different across disciplines, but Sabot and Wakeman-Linn (1991) find that there is no significant difference in either SAT scores of students or the grades for the same student in other courses across different departments Consistent with this analysis, Zirkel (1999) finds that there is no evidence that classes in the humanities generally attract superior students, which would explain their higher mean grades With student ability being similar across departments, Dickson (1984) argues that departments award higher grades in order to maintain high student enrollments and to avoid reductions in their size and in the resources available for their faculty members Freeman (1999) believes that high grades in the humanities are a compensating differential for lower post- graduation salaries that humanities majors may receive Bar and Zussman (2009) and Achen and Courant (2009) argue that two conditions are necessary in order to award relatively low average grades: students‘ enrollment demand must be high and inelastic; and the cost for professors of assigning low grades must be relatively low, which is the case when there are objective methods of assessment Bar and Zussman (2009) hypothesize that both of these requirements are more likely to hold in the natural sciences than in the humanities These departmental differences may be exacerbated by the pressures on faculty to award high grades Dickson (1984), Wallace and Wallace (1998), and Eiszler (2002) find that professors who award higher grades receive higher student evaluations Therefore since student evaluations heavily influence a professor‘s promotion, tenure decision, and merit-based pay increases, professors have an incentive to award higher grades Any of these factors may have triggered higher grades, but the upward trend in grades naturally perpetuates itself For example, younger faculty members who received higher grades while undergraduates may award higher mean grades because their frame of reference is based on a higher mean GPA as compared to older faculty In addition, low-grading professors will often be forced to conform to the policies of their high-grading peers in order to maintain enrollments The recent trends in grades have marginalized the effectiveness of grades as defined by Rosovsky and Hartley (2002), who argue that the purpose of grades is to inform students about how well/poorly they understand the content of their courses; to inform students of their strengths, weaknesses, and areas of talent; and to provide information to internal audiences, such as faculty and administration, and external audiences, such as graduate schools and employers With the upper constraint of grades at an A+, McSpirit and Jones (1999) and Winzer (2002) find that lower-ability students have experienced the highest rate of grade increase Therefore, as grades continue to rise at a faster rate for lower-quality students, the difference between good students and excellent students becomes blurred This creates problems for students in determining how well they understand the content of their courses Felten (2004) argues that a grade conveys information by telling us how a student‘s performance compares to that of other students Felten (2004) extends this analysis to find that a Princeton grade conveys about 11% less information in 1997-2002 than it did in 1973-1977 The discrepancy in mean grades across departments also has detrimental effects on students‘ ability to comprehend their academic strengths and weaknesses Both Sabot and Wakeman-Linn (1991) and Johnson (2003) show that grading disparities between departments bias students‘ course and major choices: Johnson (2003) estimates that if the disparities in grading practices between departments were eliminated, the average undergraduate student at Duke University would probably take fifty percent more natural science and mathematics electives than he or she did Romer (2002), Johnson (2003), and Bar and Zussman (2009) go as far as to purport that the dearth of students majoring in the natural sciences is one factor adversely affecting economic growth in the United States Sabot and Wakeman-Linn (1991), Shea (1994), Becker (1997), Edwards (2000) and Johnson (2003) find that large discrepancies in the grading practices between departments leads to internal, institutional allocation problems Since Edwards (2000) reports that students flock to high-grading departments and since higher-enrolled departments receive more funding, on average, low-grading departments are faced with inadequate enrollments and funding For example, Bruno (2007) reported that once Cornell class averages were posted online, students searched for the classes where the median grade was an A, and those courses experienced large increases in enrollments In addition to internal problems, unwarranted increases in grades also lead to employers putting less weight on their job candidates‘ academic performance: the percentage of human resource officers who agreed that transcripts of college grades ought to be included with an applicant‘s resume fell from 37.5 percent to 20 percent between 1978 and 1995 (Rosovsky and Hartley 2002) With the detrimental effects that increases in grades have engendered, it is important to determine whether or not these increases have been due to explainable factors In this paper, we see that grades have increased at Colby College consistently over the past decade Similar to the increases in GPAs observed nationwide, the median GPA at Colby College rose from 2.38 in 1968 to 3.16 in 1996, at a rate of 0.14 points every five years on average More recently, the rate of increase has slowed to 0.12 points every five years, with the median GPA increasing from 3.16 in 1996 to 3.37 in 2005 (Figure 1).5 These increases have led to a compression of grades in the A or B range Currently, nine in ten grades awarded at Colby College are either in the A or B range; moreover, nearly half (44%) of all grades awarded at Colby College are in the A range (Table 2) With such a narrow distribution of grades, employers and graduate schools may find it difficult to distinguish between excellence and average performance Recent increases in grades have also resulted in certain departments awarding grades significantly higher and lower than values warranted by explainable factors The increases in grades and discrepancies in grades across departments have caused grades at Colby College to lose much of their signaling power In order to combat this problem, an alternative measure of class rank that measures students against their peers is proposed By maintaining the current GPA-based grading system but altering class rank to be a relative measure of performance, grades are able to provide a clearer and more accurate signal to internal and external audiences, as well as to students The recent decrease in the rate of grade increases may be due to the upper compression of an A+ In Section II, data and variables are defined, and the theory of the regression models underlying the analysis is explained In Section III, general results are addressed In Section IV, grades are predicted for each department, and an analysis of the discrepancies between department mean grades is provided In Section V, an alternative, standardized system of class rank is proposed Section VI concludes the paper II Data and Variables Data come from student transcripts and admission forms.6 The data set tracks 1,224 students through three graduating classes over their four-year matriculation: the Class of 2002 includes 390 students; the Class of 2003 includes 426 students; and the Class of 2004 includes 408 students.789 Departments in which fewer than 100 grades were recorded over the six year period are excluded The data include the courses that each student at Colby College completed during his or her enrollment; classes taken off-campus are not included All told, 34,558 grade records are included in the data set Cumulatively over the three class years, the mean grade awarded is 3.28; and the median grade awarded is 3.30 The mean and median cumulative GPA upon graduation are 3.28 and 3.30, respectively In the final data set, the Class of 2002 has a median GPA of 3.26; the Class of 2003 has a median GPA of 3.30; and the Class of 2004 has a median GPA of 3.35 (Table 3) Even over the course of three class years, monotonic increases in cumulative median GPA can be seen Data are provided by Dr William P Wilson, Director of Institutional Research, Colby College, Waterville, ME Students for whom SAT scores and/or Dean‘s ratings were not available were omitted from the analysis, as were classes taken in 2005-2006 and those taken on a credit/non-credit or pass/fail basis and those taken for or credits In addition, observations from departments which had fewer than 100 observations were omitted from the analysis Lastly, non-U.S citizens who did not report high school type are omitted Observations that did not report school type and are international citizens are labeled as private-school observations I fail to reject the hypothesis that the mean grade after such observations are omitted is different than the original mean grade at the 99% confidence level In order to obtain a more nuanced analysis of the increases of grades at Colby College, it is important to determine if the increases are due to explainable factors of student aptitude A parsimonious OLS regression model is used that includes only exogenous factors of student performance: GRADE = + REGION + International + Female + White + Private + SAT + Early Decision + Dean Rating + 9Fall + YEAR 10 + u where the dependent variable GRADE is the grade awarded to a student in a particular course Grades are converted to their respective GPA equivalent.10 Region dummies are included because Colby considers geographic residence when considering an applicant‘s candidacy 11 8% of students are from the south, 8% are from the midwest, 9% are from the west, and 71% are from the northeast (Table 4).12 All priors are seen in Table Bar and Zussman (2009) not find that foreign students to perform any differently when compared to U.S citizens, but a measure of citizenship is included to confirm these findings: 7% of the sample is comprised of non-U.S citizens Adelman (1995) finds that gender is an important determinant of grades, for females generally earn higher grades than males; 54% of the sample is female Vars et al (1998) find that white students earn higher grades than their minority peers 88% of students in the sample are white Sabot and Wakeman-Linn (1991) find that high school type is a good proxy for high school quality, with private high schools producing higher-achieving students than public high schools 35% of the sample attended private high schools Sabot and Wakeman-Linn (1991) report that SAT is a good proxy for cognitive ability and find that SAT scores are positively correlated with grades The mean SAT score in the sample is 1300 In order to 10 A+: 4.3; A: 4.0; A-: 3.7; B+: 3.3; B: 3.0; B-: 2.7; C+: 2.3; C: 2.0; C-: 1.7; D+: 1.7; D: 1.0; D-: 0.7; F: http://www.colby.edu/administration_cs/ir/upload/cds2006.pdf 12 Colby strives to represent a large proportion of states and regions in order to increase the diversity of students Colby College is most likely to accept students from non-northeastern regions if the students are of equal ability, ceteris paribus 11 Table (continued): Prior Studies17 Author(s) Archibold (1998) Archen and Courant (2009) Years Studied 1973-1977; 1992-1997 1992-2008 Sample Princeton Dean‘s Report: A comprehensive review of undergraduates' grades over 24 years University of Michigan‘s College of LSA, which has 16,000 students of its own covering 25 departments from Fall 1992 through Winter 2008 Findings 83 percent of the grades awarded at Princeton between 1992 and 1997 were between A+ and B-, compared with 69% between 1973 and 1977 The amount of C‘s fell from 15% to 9% over the same period The median GPA at Princeton for the Class of 1997 was 3.42 compared with 3.08 for the Class of 1973 The median grade in lower-division courses at University of Michigan‘s College of LSA in 1992-1994 was 3.13 and in 2005-2007 were 3.30 The median grade in upper-division courses at University of Michigan‘s College of LSA was 3.24 in 19921994 and 3.40 in 2005-2007.18 17 Stuart Rojstaczer has collected a fairly large database that reports trends in grades for over 70 American colleges and universities Institutions included in the database are chosen strictly because they have either published their data or have sent their data to Stuart Rojstaczer This data can be accessed at www.gradeinflation.com 18 Lower-division courses are mostly introductory Higher-division courses are mostly taken by majors and others who have shown interest and/or aptitude for a field of study 26 Table 2: Grades Awarded at Colby College Grade A+ A AA's Frequency 646 7,230 7,115 14,991 Percent 1.87% 20.92% 20.59% 43.38% Cumulative Percent 1.87% 22.79% 43.38% 43.38% B+ B BB's 6,529 6,256 3,018 15,803 18.89% 18.10% 8.73% 45.73% 62.27% 80.37% 89.10% 89.10% C+ C CC's 1,595 1,192 536 3,323 4.62% 3.45% 1.55% 9.62% 93.72% 97.17% 98.72% 98.72% D+ D DD's 145 168 58 371 0.42% 0.49% 0.17% 1.07% 99.14% 99.63% 99.80% 99.80% F’s 70 0.20% 100.00% 34,558 100% 100% Total Grades shown were awarded to students of the Class of 2002, 2003, and 2004 Grades are from all departments on campus and all classes taken for a letter grade at Colby College No transfer credits or credits taken abroad are included Source: Head of Institutional Research, Colby College 27 Table 3: Median Graduating GPAs Median Final GPA Class of 2002 3.26 Class of 2003 3.30 28 Class of 2004 3.35 Table 4: Summary Statistics Grade Received International Mean 3.29 0.07 Std Dev 0.65 0.26 Minimum 0 Maximum 4.3 Observations 34,558 34,558 0.54 0.50 34,558 0.88 0.32 34,558 0.35 0.48 34,558 1300.6 100.9 820 1590 34,558 0.38 0.49 34,558 3.08 0.90 34,558 0.51 0.24 0.50 0.43 0 1 34,558 34,558 0.95 0.22 34,558 0.41 0.49 34,558 0.05 0.23 34,558 0.37 0.48 34,558 0.23 0.42 34,558 0.34 0.47 34,558 0.46 0.50 34,558 0.29 0.17 0.08 0.45 0.38 0.27 0 1 34,558 34,558 34,558 if non-USA citizen Female if female, if male White if white, if minority Private if from Private High School if from Public, Charter, or Other HS SAT Includes ACT equivalents true score Early Decision if accepted Early Decision Dean Rating is most likely to succeed academically is least likely Fall Semester Double Major if double majored, if one major Independent Study if Ind Study, if Class Major Match if majored in dept that class was taken Interdisciplinary if class was taken in Interdisciplinary area Humanities if class was taken in Humanities area Natural Sciences if class was taken in Natural Sciences Social Sciences if class was taken in Social Sciences Course Level 100 if course taken at 100 level Course Level 200 Course Level 300 Course Level 400 Note: Grade is measured as a GPA equivalent to letter grade A+: 4.3; A: 4.0; A-: 3.7; B+: 3.3…; D-: 0.7; F: Other controls include the year the class was taken (1998-2004), the course level (100-400), and region (Northeast, South, Midwest, and West) 29 Table 5: Priors Variable Midwest South West International Female White Private SAT Early Decision Dean Rating Fall More Recent Years Double Major Independent Study Major Match Humanities Natural Sciences Social Sciences Higher Course Levels Prior ? + + + + + + + + + ? + Source Bar and Zussman (2009) Adelman (1995) Vars et al (1998) Sabot and Wakeman-Linn (1991) Sabot and Wakeman-Linn (1991) Jensen (2009) Bar and Zussman (2009) Bar and Zussman (2009) Bar and Zussman (2009) Sabot and Wakeman-Linn (1991) Johnson (1997) Johnson (1997) Bar and Zussman (2009) 30 Table 6: Regression Results OLS Regressions (1) Midwest South West International Female White Private SAT Early Decision Dean Rating Fall Year 1999 Year 2000 Year 2001 Year 2002 Year 2003 Year 2004 Double Major Independent Study Major Match Tobit (2) (3) 0.0280 0.0319* (0.0209) (0.0169) -0.0423** -0.0486*** (0.0177) (0.0167) -0.0311* -0.0236* (0.0162) (0.0133) 0.0913*** 0.107*** (0.0272) (0.0222) 0.118*** 0.103*** (0.0187) (0.0178) 0.133*** 0.135*** (0.0248) (0.0226) -0.0212* -0.0288*** (0.0117) (0.00771) 0.0457*** 0.0398*** (0.00903) (0.00669) -0.00855 -0.00603 (0.0122) (0.0107) -0.155*** -0.159*** (0.0122) (0.00936) 0.0148 0.0116 (0.0133) (0.0140) 0.0448 0.0180 (0.0287) (0.0269) 0.0958** 0.0421 (0.0414) (0.0394) 0.201*** 0.110** (0.0467) (0.0415) 0.282*** 0.150*** (0.0582) (0.0508) 0.353*** 0.196*** (0.0531) (0.0483) 0.405*** 0.223*** (0.0518) (0.0452) 0.0895*** (0.0183) 0.410*** (0.0857) 0.0662** (0.0262) 31 0.0308* (0.0172) -0.0494*** (0.0170) -0.0254* (0.0137) 0.111*** (0.0233) 0.103*** (0.0181) 0.135*** (0.0228) -0.0304*** (0.00809) 0.0400*** (0.00678) -0.00668 (0.0111) -0.163*** (0.00970) 0.0111 (0.0145) 0.0178 (0.0273) 0.0429 (0.0398) 0.112*** (0.0416) 0.153*** (0.0509) 0.199*** (0.0488) 0.231*** (0.0462) 0.0914*** (0.0189) 0.423*** (0.0889) 0.0666** (0.0265) Fixed Effects Fixed Effects Student Department (4) (5) 0.0323*** (0.00655) 0.0502*** (0.0185) 0.112*** (0.0205) 0.214*** (0.0218) 0.271*** (0.0237) 0.353*** (0.0257) 0.400*** (0.0296) 0.333*** (0.0252) 0.0665*** (0.00916) 0.0317* (0.0164) -0.0589*** (0.0155) -0.0201 (0.0132) 0.128*** (0.0191) 0.0903*** (0.0179) 0.133*** (0.0224) -0.0235*** (0.00761) 0.0420*** (0.00611) -0.00628 (0.0108) -0.168*** (0.0101) 0.0103 (0.0123) 0.0301 (0.0267) 0.0567 (0.0382) 0.128*** (0.0392) 0.164*** (0.0494) 0.207*** (0.0497) 0.230*** (0.0448) 0.0810*** (0.0178) 0.358*** (0.0554) 0.117*** (0.0255) Table (continued): Regression Results Humanities Natural Sciences Social Sciences Course Level 200 Course Level 300 Course Level 400 Constant Observations R-squared Number of students Number of departments 2.787*** (0.124) 34,558 0.122 -0.0940 (0.0630) -0.290*** (0.0771) -0.240*** (0.0795) 0.0673* (0.0360) 0.0943*** (0.0301) 0.144*** (0.0509) 3.443*** (0.132) 34,558 0.178 -0.0914 (0.0639) -0.285*** (0.0792) -0.239*** (0.0802) 0.0664* (0.0367) 0.0921*** (0.0308) 0.143*** (0.0525) 3.464*** (0.136) 34,558 -0.121*** (0.0158) -0.349*** (0.0191) -0.263*** (0.0156) 0.0311*** (0.00977) 0.0345*** (0.0121) 0.0646*** (0.0178) 3.569*** (0.0339) 34,558 0.105 1,224 0.0344 (0.0377) 0.0624* (0.0324) 0.127*** (0.0418) 2.848*** (0.102) 34,558 0.167 30 For models 1, 2, and 3, robust standard errors clustered by department are given in parentheses For models and 5, robust standard errors are given in parentheses SAT is measured in 100 point increments *** p