[Andrew Morriss (guest-blogging), August 8, 2005 at 1:51pm] Trackbacks
Median LSAT scores and law school rankings:

Thanks to Eugene for inviting me to participate and talk about one of my favorite topics: legal education. Bill Henderson (Indiana) and I are dissecting the components of the U.S. News rankings in a series of papers, the first of which is now available on SSRN, and which we presented at a symposium at Indiana on rankings last spring. (There were a number of excellent papers presented there and anyone interested in the topic should read them all.) In this post I'll give a quick summary of our data and methodology. Later today I'll discuss the first of our findings, the strong evidence of market segmentation in legal education.

U.S. News began ranking law schools in 1987 with a "top 20" list based on a survey of law school deans, a method it repeated in 1988 and 1989. In 1990 and 1991 the magazine used a variety of data to generate a top 25 list. Starting in 1992, U.S. News began giving data on all the ABA-accredited law schools, although only the top 25 were ranked numerically.

It is hard to compare ranks across time because of a variety of data issues. One of the consistent inputs, and a particularly important one, is the median LSAT of entering law students. Bill and I decided to tackle this part of the rankings first, largely because it is something to which faculty members seem to pay a great deal of attention. (This past year U.S. News switched to using the 25th and 75th percentiles of entering class LSATs, in part because the magazine suspected schools were fudging the median numbers they reported. Since the 25/75th percentile numbers are reported to the American Bar Association, U.S. News thought that schools might be more honest in their reporting of those numbers.)

(A good discussion of some of these issues is in Alex Wellen's New York Times magazine story, which discusses our research, from this past Sunday.)

We used data on the law schools themselves (public/private, religious/secular, the number of large law firms (those on the American Lawyer 200 list) interviewing on campus, the percentage of the entering class in the full time program, average student loan debt, and so forth), data from the U.S. News rankings (academic reputation, lawyer-judge reputation) and the metropolitan statistical area (MSA) in which the school was located (how many law schools were in the same MSA, large firm jobs in the MSA, demographic trends in the MSA).

There are huge problems with the U.S. News rankings but there is no question that they are important. For example, a friend recently told me that she had been called by a law review about one of her manuscripts. The articles editor apologized for rejecting the manuscript and explained that the rejection had been made without reading the paper because the editors had mistakenly misclassified my friend's school as being in a lower tier law school. Now that they realized their error, the editor told her, they wanted to consider the article on the merits. I don't know how widespread this type of screening is, but that it occurred at a well-ranked, but not top journal is at least moderately disturbing.

The U.S. News rankings also have consequences for students -- several admissions directors have told both Bill and me that they are noticing a compression in the LSAT scores of applicants, with a strong clustering around the median scores reported in U.S. News. Whether applicants are relying on the overall rankings, or as Brian Leiter suggests, on individual components of the rankings, U.S. News is clearly having an impact on legal education.

The question I would like to throw out for Volokh Conspiracy readers is not whether U.S. News rankings are good or bad in aggregate, but how the competition for students with LSAT scores above each school's median (and, now, 25th and 75th percentiles) is affecting legal education.

Later today I will post a summary of our results on the segmentation of the market for legal education between the "top tier" and everyone else.

Richard Bellamy (mail):
When I took the LSATs, I received my score, but I really had no concept what it "meant" in terms of where I could go to Law School. I knew that GPA and LSATs were the #1 consideration for Law Schools, just as they were for undergraduate.

Just like I would have advised a "normal" student (e.g., no special skills or characteristics) with a 1000 SAT not to apply to Penn as an undergraduate, because it would be a waste of time and money, I needed "someone" to tell me whether it was a waste of time for someone "like me" to apply to Penn Law. So, I looked at the median rankings and saw, lo and behold, that I was exactly the Penn Law median. I applied, got in, and attended.

I also applied to Temple Law as a "safety school." I was well above the median there, and assume I would get in (I was right).

Had I seen that I was 10 points below the median, I likely would not have applied to Penn. Why should I? My undergrad GPA was very good, but not 4.0, and likely not any better than the other median applicants. I am not under the illusion that I am "special" in a way that a Law School would care about. I think I'm "smart", but the more elite the pool is, the less smart I am in comparison.

What you describe as a "compression" is more likely just the expression of information. Had I not known Temple's median, I may have also applied to Villanova and Widener, because my safety school would have felt less safe. Had I had a much lower LSAT, and not known Penn's median, I would have likely applied there too, considering it a "reach", but not knowing that it was next-to-impossible.

It simply makes no sense to apply to lots of schools you are either over or underqualified to attend. In my view, information helps the right people find the right schools, with little wastage in application fees.
8.8.2005 4:05pm
Chad (mail):
My limited knowledge of the selection process would lead me to believe that the effect would be school-specific. I attended a U.S. News top 25 public university, and it seemed that there were so many other factors that the members of the selection committee considered (residence, as well as "diversity" factors) that the LSAT effect was watered down some. I'm sure that schools without an in-state consideration would be different, but some may still spend as much time on race/gender/SO/"life experience" factors that it may be hard to ascertain the LSAT effect. My guess is that it would be similar to the legal hiring process in that if all else is equal go for the applicant with the big numbers, but all else is rarely equal.
8.8.2005 4:24pm
Compression results at better schools by discouraging less qualified applicants, which is a good thing according to Rick Sander.

However, the answer to the question of "good" or "bad" depends on the identity of the customer in the market.

If students are the schools' customers, students would pay a premium to schools that do the best job of turning qualifications into job opportunities. The extent a school depends on the quality of students to create marketable graduates would be seen as a weakness of the school. A "great" school would turn out Manhattan Biglaw associates from 2.5 GPA, 63 LSAT kids.

If large law employers are the schools' customers, then you'd expect the highly valued school to turn out consistently high level employees, which can be done most efficiently (cheaply) in the current system by starting with the best GPA and LSAT kids. Customers would pay a premium to get students of a guaranteed high caliber delivered efficienlty via exclusive admissions.
8.8.2005 4:50pm

I don't know how widespread this type of screening is, but that it occurred at a well-ranked, but not top journal is at least moderately disturbing.

It's more than moderately disturbing, it's a threat to academic inquiry. The journal editors are arrogantly asserting that only papers from their approved list of scholars will be considered for publication and that everyone else - regardless of the quality of their ideas - is considered second-class and unworthy of consideration. Is it common for legal journals to fail basic academic standards like the one here? More fundamentally, what's wrong with blind review and why are law journals doing it?
8.8.2005 4:57pm
Justin (mail):
For one, it is making it more difficult for lower middle class and lower class students (many of who I was friends with at Columbia) from going straight into the public sector, as University scholarship funds are drying up from people who either want to go into public sector or suffer financial hardship and towards "merit awards" based on LSAT and GPA.
8.8.2005 4:59pm
Jacob Lister (mail):
When I was looking at law schools, LSAT was the only measure I had that could be compared to other applicants. I had been out of undergrad for several years, and my UGPA was pretty bad. I had some graduate units in education, and that GGPA was good, but education doesn't compare the same way with normal academic graduate programs. And I had been working for a few years too.

I found the US News ranking very unhelpful because the statistics apply only to students who are just graduating from undergrad and going directly to law school. My LSAT score would compare favorably to other applicants. My UGPA would only hurt a law school thinking about accepting me. And my GGPA was fine though the law school would get no benefit from it.

I just applied at schools that looked interesting, didn't seem too focused on new graduates and were "top tier." I think it turned out fine, but I'll let you know in two more years.
8.8.2005 5:23pm
BKriplur (mail) (www):
If it were the case that there is a direct correlation between the quality of a lawyer and his or her score on the LSAT I would have no problem with the present system. I don't know for sure, but I doubt there is one. It seems, along with the GRE, completely arbitrary, especially with all of the "prepping" one can do.

Its interesting to consider if any well known lawyers from history would have been held back by the LSAT.
8.8.2005 6:04pm
"It simply makes no sense to apply to lots of schools you are either over or underqualified to attend. In my view, information helps the right people find the right schools, with little wastage in application fees"

I completely disagree with this often-voiced sentiment. Put simply, $60 bucks a pop is not that much money. What's more valuable to your average high-school or college student, a copy of the latest video game or a chance to go to a great "reach" college or law school? I went to a mediocre public school in central Texas and, with no expectations of getting in anywhere all that prestigious, applied to 13 schools. East coast, west coast, mid-west etc… My friends, most of whom applied to a single school (UT or A&M), thought it was a complete waste of money, but I ended up getting into a couple of Ivy's. When I applied to law schools I repeated the process, applied to 9 and ended up at a top 3 school (thereby greasing the wheels for the rest of my professional life). Strangely enough, I was rejected from a couple of schools that according to the US News numbers were more in my range of GPA and LSAT scores. I would recommend applying to as many "reach" schools as you can. You might just get lucky and all you have to lose is a few hundred bucks (and really, you probably spend more than that at Starbucks each year…)
8.8.2005 6:07pm
I take it the issue is not whether using LSAT scores as a factor in law school admissions is reasonable. Rather, I take it the premise is that the USNWR rankings and their use of reported median or 25/75 LSAT scores is distorting how the law schools would otherwise choose to use LSAT scores in admissions, and the issue is how that distortion might affect legal education.

It may seem obvious to some that such distortion is bound to be bad. I'm less sure, however. It seems likely that some students are being reallocated among the schools, in the sense that some students might have ended up at different schools if there was not this incentive to view LSAT scores in this way. But is this reallocation really going to affect legal education in general?

I am enough of skeptic about the LSAT to think it will not really matter much to the educational experience of the students. For example, I suspect that students ordinarily could not tell the difference between classes comprised of fellow students with more or less compressed LSAT distributions respectively.

So, although it all seems silly and wasteful to me, I'm not sure I see much overall harm being done--holding aside the general point that whatever resources are being wasted on this competition could be used in more productive ways.
8.8.2005 6:08pm
Stephen Aslett (mail):
I graduated from Rice University with history and philosophy degrees in 2003. My GPA was a 3.9, putting me solidly in the top 10% of the class. I even won a history department award and an award in a campus-wide writing contest. Unfortunately for me, I got a 161 on the LSAT despite extensive preparation and higher practice scores. (I had similar SAT scores--1300s. Good, but not within the median for elite schools.)

After graduation, I did temp work for a couple of law firms and taught high school. I never gave up going to law school, but I thought it would do me some good if I actually worked full time before applying. I still don't regret the choice that I made. Perhaps foolishly, I thought that work experience would help out my application. Good grades, solid recs, solid work experience: what else could I do? Oh yeah, score better on the LSAT. I studied even more, did well on the practice tests, took the test again and scored a 160. Damn.

I applied to 10 schools. I was accepted by only 2. One was Tulane (which had briefly dropped below the top 50 in U.S. News) and the other was a U.S. News third tier school. All the schools that rejected me were first-tier U.S. News schools (not necessarily elite like Harvard or Stanford, but first-tier nonetheless). I know that my essay wasn't atrocious, so what else can I chalk up these rejections to other than subpar LSAT scores for first tier schools?

Fortunately, I ended up grading onto law review at Tulane. I guess that shows ultimately how important silly tests like the SAT and LSAT are.

How do LSAT scores ultimately affect legal education? My educated guess is that they cause alot of folks who shouldn't really be going to law school to go to very good ones. More than a few undergrads who really have little or no interest in law study for and take the LSAT for lack of anything else to do after graduation. If they score well, and did well in undergrad, a first-tier law school will take them over someone who has demonstrated legal skills and interest (e.g., by working as a paralegal) but whose LSAT scores are subpar for the first tier. Considering that some elite law firms won't interview even law review members from below first-tier schools, people at lower-tier schools are effectively blackballed from working at such places. And such people can forget about snagging plum professorships, since prestigious clerkships and elite firm jobs are practically prerequisites.

(Yeah, I suppose you could distinguish yourself as an attorney and laterally move into a firm, but that can be tough to do.)

I just think it's sad that many of the high-paying and prestigious jobs in the law are effectively reserved for first tier law school grads, and getting into a first tier law school is mostly determined by one's LSAT score.

To any ambitious person about going to law school: better study hard for that LSAT!
8.8.2005 6:33pm
jallgor (mail):
Based on the variety of responses in these comments I gather I am not the only one who doesn't understand the question Mr. Morriss is posing here.

"The question I would like to throw out for Volokh Conspiracy readers is not whether U.S. News rankings are good or bad in aggregate, but how the competition for students with LSAT scores above each school's median (and, now, 25th and 75th percentiles) is affecting legal education."

What are schools doing in the competition for students with LSAT above each schools' median? I assume they must be competing but I don't know specifically what they are doing. Assuming they are competing in that way, how could such competition effect my legal education? Are they increasing class size or not hiring good professors? What? Or is it the possible distortion that Medis references in his post? I am totally confused by the question.
8.8.2005 6:36pm
Jeff B, (mail):
I must admit that, the many problems of the US News rankings notwithstanding, the median LSAT statistic not merely helpful, but instrumental in my application process.

I was one of those quixotic 'borderline' candidates for the schools I really wanted to go to; my GPA was quite low, but my LSAT was good. Moreover I'd only been out of school for 1.5 yrs before applying, so my GPA was still considered a highly relevant piece of recent data. Had I not known that my LSAT was comparable to the median LSAT of some very good schools, I would have just assumed that applying to Top 20 schools was an exercise in futility and decided not to waste my money. As it is I DID apply to many of those schools, and ultimately was accepted at the University of Chicago. Therefore, in my case at least, the US News Median LSAT stat was absolutely critical in helping me reevaluate my chances. Without it, I feel I would be significantly worse off.

N.B. The behavior of the law journal editor is, as others have said, a good deal more than "moderately" disturbing. It's appallingly unprofessional conduct; more appalling, perhaps, is the possibility that such frankly anti-intellectual arrogance would be official policy for a journal.
8.8.2005 6:41pm
Guest (mail):
With apologies for being more responsive to the thread than to the initial post, I thank god every day for the LSAT. I applied to law schools as an older, plain-vanilla guy with good grades from a mediocre state U. If it weren't for the LSAT, I don't think I would have been considered by the elite school where I ended up going. (And seeing the median LSAT scores in US News helped me decide it was worth applying there. Notwithstanding what Cabbage says above, at the time I needed every dollar I could hang on to.)
8.8.2005 7:24pm
Dave Bundrick:
I agree, thank god for the LSAT. After getting a fairly unimpressive GPA at a state school, I was able to get a scholarship to my first choice school. I can only assume that my high LSAT score did it, because there were no other factors strlongly in my favor.

I've done well (top 25%) at law school, but the skills that the LSAT tests are not the reason.
8.9.2005 9:15am