pageok
pageok
pageok
[Andrew Morriss (guest-blogging), August 11, 2005 at 7:31pm] Trackbacks
Law school games:

The importance law schools have placed on U.S. News rankings mean that schools are engaging in some gaming behavior. Indeed, law schools are - more or less - in the position of taxpayers figuring out how to manipulate the tax code to minimize their taxes. There is a fair amount of play in a number of the reporting rules and schools have taken advantage of this. Some, of course, have crossed over the line and engaged in "tax fraud".

Like the IRS, U.S. News has been modifying the rules to try to stop the gaming. Most recently, it switched to using the 25th/75th percentile LSAT and GPA numbers rather than the median numbers, reasoning that since the former are reported to the ABA, law schools would be less likely to fudge them.

We found that two strategies seemed to have worked in raising median LSAT scores. (Note, we simply examined the data - we didn't interview all 190 or so schools to see if they had explicitly adopted these strategies for U.S. News purposes.)

Schools can decrease the first year class by cutting from the bottom of the admitted pool (and, if they want to, make up the revenue by increasing the number of transfer applicants they take, since transfer applicants don't count for U.S. News purposes.) We found that schools in the first quartile whose first year class size had shrunk increased their median LSAT scores relative to those who didn't, with a 10% significance level (higher than we'd like, but there were only 44 schools in this group.)

Schools in the other quartiles played a different game - here we saw some schools shifting students from full time to part time (part timers don't count for U.S. News), which we measured by comparing the proportion of total 1L class in the full time program across time. This worked too - a 10% shift from FT to PT gave a 0.54 point median LSAT gain.

There are other games schools can play too - hiring unemployed grads to do filing for a few weeks, for example, fits the "employed at graduation" definition (any job will do).

Some of these games are harmless. Some are not. All are a diversion from competing on educational quality, innovative programs, etc. We recommended that U.S. News consider revising its part time rules (which is likely to hurt a number of schools with large part time programs) and the NYT Magazine story by Alex Wellen reported that U.S. News is considering doing so.

Sidenote: I highly recommend Alex's book, Barman, an account of law school, the bar exam, and practice with the interesting feature of noting the U.S. News ranks of everyone in it. Alex had a much more interesting law school social life than I did; the book offers a perceptive account of legal education. He has a nice web site too.

Sam (mail):
Disclosure: I am a transfer student.

I am personally in favor of the practice of accepting transfer students in larger numbers. Since it is more difficult to get into a school as a transfer than as a regular student the transfers tend to have really good credentials and are a proven quality that improves a law school's student population. Generally transfer students do very well (anecdotally there hasn't been much difference in the law schools besides name, ease of finding a job, and class availability; of which the last is the only real benefit in terms of intellectual development) It hurts the "lower ranked" school to lose some of its top students as well; so it's simply a balancing test in terms of overall utility to the law school field. It's also kind of a chance from the student's perspective (will I do well, how will I fit in, what about finding a job?), but I can see little to lose from the School's perspective, although I admit I'm a tad biased.
8.11.2005 10:21pm
Sam (mail):
Disclosure: I am a transfer student.

I am personally in favor of the practice of accepting transfer students in larger numbers. Since it is more difficult to get into a school as a transfer than as a regular student the transfers tend to have really good credentials and are a proven quality that improves a law school's student population. Generally transfer students do very well (anecdotally there hasn't been much difference in the law schools besides name, ease of finding a job, and class availability; of which the last is the only real benefit in terms of intellectual development) It hurts the "lower ranked" school to lose some of its top students as well; so it's simply a balancing test in terms of overall utility to the law school field. It's also kind of a chance from the student's perspective (will I do well, how will I fit in, what about finding a job?), but I can see little to lose from the School's perspective, although I admit I'm a tad biased.

Taking the transfer students into account for LSAT purposes makes no sense for USNWR rankings simply because the variable that LSAT is supposed to predict, 1st year grades, has already been set. 1st year grades are highly correlated with academic achievement over the course of a law school career so the LSAT is irrelevant for those as well.
8.11.2005 10:22pm
Mary Campbell Gallagher (mail) (www):
I'm glad to see you mention Alex Wellen's very funny book BarMan. The bar exam is my business, I think it has an important function protecting the public, and I am pleased when people take it seriously. So far as I know, Wellen is the only person who ever took it seriously enough to write most of a book about it.

Since gaming U.S. News and World Report is the topic of your post, let me just note that the measures U.S. News uses are not independent of one another. Take, again, the bar-pass rate. It affects the number of grads employed at graduation, the peer assessment score, and the assessment score by lawyers/judges.

One thing Wellen says that I like is that it is not the bar exam itself that is the rite of passage, it is the weeks of preparing for the bar exam. "Surely," Wellen say, "my future clients were entitled to an attorney capable of withstanding extreme pressure and surviving . . . intensive bar-review boot camp." The bar exam is a morale-boosting experience for a lot of people, and Wellen shows why that is so.
8.11.2005 10:43pm
Craig Oren (mail):
"The bar exam is a morale-boosting experience for a lot of people, and Wellen shows why that is so."

As a law professor I've met a lot of people studying for and taking the bar. *I've* even taken the bar! I never have met anyone whose morale was boosted by the bar. Studying for the bar may be a rite of passage, but only in the sense that it's one of the last puberty rites.
8.11.2005 11:39pm
Andy Morriss (mail):
I'm not sure Sam's comment represents the current state of transfers at all schools. My sense - and this is anecdotal and based on conversations with admissions people, faculty, and students - is that some schools (i.e. those who are cutting back the 1L class size) are taking as transfers people who would not have gotten in as 1Ls but who outperformed their predictors (LSAT and UGPA) as 1Ls. This suggests that is not harder to get into a school as a transfer student but easier, conditional on having done well as a 1L.

There certainly seems to be some evidence that transfers are up across a fairly wide range of schools (and perhaps some folks with knowledge of their schools' experiences could post some comments here) compared to the pre-US News era. Is this bad or good? I don't know. It is different.
8.11.2005 11:52pm
John_Doe (mail):
If the point of moving up in the rankings is based primarily (and almost exclusively) on LSAT, then the admittance of transfer students cannot help top schools. I have a couple transfer friends at a top ten school that generally (and anecdotally) do about average at the trasferee school, and they all readily admit that they had no chance at getting in without the first year grades at a lesser school. But doesnt that just corroborate the claim that LSAT scores corroborate well to grades in law school where all else is equal (i.e., incoming students have the same or similar LSATs)? That is, if the students who do so well at a lesser institution transfer and become average students at the greater institution then LSAT scores actually matter for the purposes of predicting that independent and relative factor of grades.
8.12.2005 12:37am
Andy Morriss (mail):
John Doe (is the former X member posting? I hope so!) Admitting transfer students can help - here's how: Suppose the top school usually admits 150 students in its 1L class. This year it admits 140 1Ls and the 10 people not admitted are from below the median of the 150 it would otherwise admit. The median LSAT and UGPA shift up as a result - that is, the median scores for the 140 1L class are larger than the median scores for the 150 1L class that would have otherwise been admitted.

Now, the school has given up 10 students worth of revenue for 3 years, or 30 student-years of tuition (we're ignoring attrition). To make this up, the school admits 15 2L transfer students (who pay 2 years tuition, total of 30 student-years). This is important because those last 10 1Ls were almost pure "profit" - there are very low marginal costs to law school education (unless you have to add another section or build a new building to get more classrooms).

Again anecdotally, the number of transfers in at some of the top schools seems to have gone up, suggesting that some schools are doing this.
8.12.2005 9:12am
Sam (mail):
If LSAT scores actually predicted 2nd and 3rd year grades at the transfer insitution then it would be relevant. The problem with that logic is that LSATs generally are far less accurate predictor of a transferee's grades at the new instituion than 1st year grades. E.G. generally transfer students don't become average students at the new institution; they continue to be good students at the new school. For my institution I know that generally transfers don't end up outside of the top 20% of the class.

As for difficulty of getting into an institution as a transfer note that transfers generally have to outperform most of the students actually at that school to transfer in. E.G. if you're transferring from Columbia to Chicago, generally you aren't accepted if you're a median student. Same with Washington and Lee to William and Mary or any other fairly comparable school. IF you're going up in the "rankings" you have to considerably outperform the median.

Sorry about the pervious double post.
8.12.2005 4:35pm
Mary Campbell Gallagher (mail) (www):
The topic of the thread is how law schools game U.S. News and World Report, and I don't want to drag it off-topic. I said that preparing for the bar exam is a morale-building experience for a lot of people. Craig Oren demurred. My observation was based on running a bar exam preparation business for more than 18 years, and teaching thousands of bar candidates. Next time, I will happily ask some of my high-spirited bar candidates to write to Craig Oren.
8.12.2005 8:34pm