pageok
pageok
pageok
WSJ Article on US News Law School Rankings:

The Journal has a story on the consideration U.S. News is giving to counting entering part-time students along with full-time students. For what it's worth, I think the problem is with law schools that have started bogus part-time day programs to admit students who don't otherwise have the proper credentials. Long established part-time evening programs, like George Mason's, tend to have students with a bit weaker academic credentials than their full-time counterparts, but with vastly more real-world experience. I've had, for example, evening students in their 40s and 50s who have successfully built multi-million dollar businesses. Does it make sense to judge them based on their undergraduate GPA from twenty years earlier?

I was also struck but this line: "Mr. Morse of U.S. News says the magazine will run tests of how the change would play out in rankings, and then decide in January." If the change is methodologically valid (or not), it shouldn't matter "how the change would play out in rankings." It's bizarre to first decide what you think the rankings should look like, and then create (or retain) criteria to meet that preconceived notion.

What I'd like to see is a separate ranking of part-time programs, especially given that most part-timers are working professionals who are choosing among part-time programs, and do not even consider full-time admissions. Not only would this provide very useful information to such students, but schools that, say, are revealed to have nine part-time day students out of a class of 200, with an LSAT 10 points below their full-time median, will be called out for abusing the system.

UPDATE: TAXPROF has links to much more commentary.

SATA_Interface:
David, for someone considering a part-time evening program to get his MBA, what other rankings do you consider useful to potential students? I've looked at U.S. News for some time, and wanted to get a second (and third and fourth) opinion if possible.
8.26.2008 1:18pm
Jon Roland (mail) (www):
There is a general problem with rankings as being a kind of elitist cronyism. As the article indicates, there may be little correlation between the eventual professional performance of a person and where he went to school, the grades he made, or his LSAT score.

I remember when I took the LSAT I found in the "logical reasoning" sections several questions that had either no correct answers, or more than one, and I wrote the LSAT authors about that, explaining the problem in terms of well-formed statements in the predicate calculus. They replied that the questions were only intended to test "informal reasoning", not rigorous logic. I replied that such "informal reasoning" often contains logical contradictions, and that if a single contradiction is admitted into a system of propositions, all other statements in it become both true and false at the same time, and no statement can be refuted as inconsistent with the premises. They had no answer for that.

This points to a problem in legal practice, that it tolerates logical inconsistency. As Federal Judge Walter P. Smith commented after reversing his own formerly correct ruling in the Davidian case, "The law doesn't have to be logical!"
8.26.2008 1:32pm
Jerome Cole (mail) (www):
These rankings are bogus. They rely far too much on reputation, admissions rates, yields, LSAT scores, and GPAs. Rankings have to be outcome based in order to have any real meaning. Rank them according to graduating students' starting salaries, placement rates, bar exam pass rates, etc. It might even be worth considering creating a standardized test of legal knowledge (not as parochial as state bar exams) in order to see exactly how well law schools are educating their students. As far as I am concerned results talk and reputation walks.

In any event given how important these rankings are to law schools I don't blame them a bit for gaming the system. US News is to blame here. They have created perverse incentives for law schools to do all manner of weird things in order to improve or maintain their rankings that don't necessarily improve educational quality.

Down with the Ivy League and down with US News. Come the revolution they should be the first up against the wall.
8.26.2008 1:45pm
AnneS:
ALthough I understand the concerns about "bogus" part time programs, I'm afraid that this change will push law schools to make damaging changes in order to "game" the system - like killing their bona fide part time and evening programs. As it is, the (otherwise lovely) dean of my Tier 1 school tried to kill our evening program twice during my time there, and our school used the same admissions standards for day and evening, freely permitted transfer between divisions, and ranked all graduating students together. It was strongly suspected that she was trying to enhance the school's reputation.

I think a separate ranking system is an excellent idea. We evening students* bring unique and valuable experiences and perspectives to our schools and to the practice of law. It would be a shame for worship of the tin god of the U.S. News rankings to kill our programs.

*Full Disclosure: I don't really include myself in the "unique" perspectives category, since I was a baby compared to my fellow evening students. I'm thinking of the doctors, engineers, business people, and nurses who, unlike me, could not have attended law school at all if evening programs had not been available. THe evening program was merely convenient for me; it was necessary for them.
8.26.2008 1:47pm
Cornellian (mail):
I'd be fine with simply exempting some small percentage of the lowest LSAT scores from the ranking calculations, to give each school an equal opportunity to admit a few people based on other factors (life experience etc.) without having to worry about how it will affect their rankings. It's sort of like the way you will sometime exclude a few outliers so that the mean doesn't give one a misleading impression.
8.26.2008 1:55pm
Bruce Hayden (mail) (www):
I agree with David. I went to a long established night school because I was in my late 30s and had a real job paying a lot of money as a software engineer. I did it in 3 years by going summers. I couldn't have afforded quitting a good paying job for those three years. I also couldn't move because my job paid too well. Instead, I graduated, on time, debt free. And the night students made up for typically weaker SATs (though mine weren't) and grades by being more serious students.

I think his suggestion that night programs be rated separately has a lot of merit. That would likely solve the problem of schools pushing their weaker students into night programs the first year to game the ratings, without putting too much pressure on the schools with night divisions to dump them to keep up their ratings.
8.26.2008 1:59pm
theobromophile (www):
I also like the idea of ranking the night programmes. As most information out there is geared towards the traditional, straight-out-of-college law student, US News could provide a very valuable service.

Some of the schools ask applicants with low GPAs or LSAT scores, whom they nonetheless want as students, to attend at night for a year and transfer into the day programme, or to apply as a transfer. (I, with a none-too-stellar GPA, albeit one that was respectable for my major, got a lot of this. It was kind of weird.)
8.26.2008 2:14pm
Shertaugh:

"Long established part-time evening programs, like George Mason's, tend to have students . . . with vastly more real-world experience."


DB:

Exactly what world do you live in?
8.26.2008 2:19pm
Robert West (mail) (www):
The other problem with ranking part time programs is that part time night students are <i>in general<i> unwilling to relocate for school, which means the student pool is more directly correlated to the pool of available local candidates.

Bruce: that is precisely what I am doing now. Borrowing money doesn't make sense when I can work and not take out loans.
8.26.2008 2:19pm
Wallace:
The funny thing about the part time programs is that some students are gaming schools which game the system. Savvy students join the part-time program thinking that they can do better on first-year exams then students who tend to have either lower LSATs, or full-time jobs, or child-care duties. Getting a higher GPA after the first year, these students switch into the full time program with a leg up when applying for law review or jobs determined primarily by GPA's.
8.26.2008 2:34pm
Dissenter:
"It's bizarre to first decide what you think the rankings should look like, and then create (or retain) criteria to meet that preconceived notion."

Yet there you have it. If the top three schools were ever anything but Harvard, Yale, or Stanford, there would be a revolt and the list would be dismissed as fraudulent. In reality, these rankings are no more rigorous or statustically sound than the AP rankings of college sports teams. It's just a bunch of people guessing at who they think is best.
8.26.2008 2:34pm
zippypinhead:
[the LSAT authors] replied that the questions were only intended to test "informal reasoning", not rigorous logic....

This points to a problem in legal practice, that it tolerates logical inconsistency. As Federal Judge Walter P. Smith commented after reversing his own formerly correct ruling in the Davidian case, "The law doesn't have to be logical!"

Or as Oliver Wendall Holmes said "the life of the law has not been logic; it has been experience." Which raises an interesting question about U.S. News' ranking of part-time programs -- how does one measure intangible factors like students' pre-law school "experience" in ratings?

An "experience" methodology that works might help tease out the difference between part-time programs primarily used to dump less-qualified students, versus programs like top-15 ranked Georgetown Law's well-respected 4-year night program.
8.26.2008 2:51pm
Modus Ponens:
David,

Does it make sense to judge anyone based almost exclusively on his/her undergraduate GPA from n years earlier, and to punish schools which admit otherwise attractive candidates whose raw admission scores fall below the median of students admitted to that school's peer institutions?

Tunnel-vision.
8.26.2008 3:59pm
Shamrock:
You make a great point that a separate ranking of part-time programs would be extremely beneficial and would make a great resource for those students seeking part-time programs. But I don't know if I necessarily agree with another point you make:

It's bizarre to first decide what you think the rankings should look like, and then create (or retain) criteria to meet that preconceived notion.

Yes, it seems bizarre generally to come up with a preexisting conclusion before generating a list of rankings, but because the USNWR rankings are so heavily relied upon, it might be worthwhile to do an experimental trial of the results to see where schools fall. Extremely large magnitude changes in the rankings may be pretty chaotic for law school admissions and may convince USNWR that, even if the new rankings are more accurate, it isn't worth it.
8.26.2008 4:00pm
loki13 (mail):
I'd like to add my personal anecdote (which is not the singular form of data):

When I was applying to law schools not too long ago, after a fairly decent interval from college, I had a problem. I had a ridiculously low UGPA, and a really, really high LSAT. The UGPA was caused by two factors:

1. My undergrad, at the time, had not experienced rapid grade inflation (damn you academic standards!), and I also took a number of courses in a "hard sciencey" major.

2. Beer. Uh... no, make that outside interests.

Anyway, I was what was known as a "splitter". Anyway, I was accepted at all the decent schools I applied to (T50), and at the one T14 I applied to, I was rejected, but was told (unofficially) that they would very much like me transfer in after a year.

I went to the decent school, did amazingly well (I had, uh, fewer outisde interests given my new maturty) and have the swanky job, high salary, and no life that comes with it.

Wait... what? Damn.

Anyway, the point, I think, is that some schools guard their UGPA numbers jealously. Or that the reward of this pie-eating contest is more pie. Or something.
8.26.2008 4:05pm
wolfefan (mail):
Hi -

I'm at work so I can't do the research to check, but I remember an article some years ago in either The Washington Monthly or The New Republic saying that the original rankings were designed to get a specific result. According to the article, when US News designed their criteria and ran the numbers, the first results had a few schools at the top that no one expected, and some of the "usual suspect" top-tier schools further down than expected. The response of the editors was "Everyone knows that (Harvard, Yale, MIT, or whatever) are the best schools!" and so the criteria was tweaked to get those specific results. If that's what's happening with the Law School rankings, it would not surprise me.
8.26.2008 4:12pm
Mikeyes (mail):
As someone who has considered going to a part time Law or MBA program, I have to ask, "What difference will the rankings make?"

I have an excellent paying job and would have to travel 60 miles to go to an evening program. I don't have very many choices (2) and even if one program is "better" by reputation I would probably choose the program that fits my needs best. Once I had a second professional degree my job prospects would be enhanced whether I went to Harvard or the YMCA night school. My first degree (and I suspect other's experience and training) probably makes the future employers perk up when my resume' shows up.

Since night school, like politics, is local, I see no value in ranking them for the mature student. For someone who wants to transfer or is looking to move and study part time, it may make a difference. But how many students like that are applying to night schools as a first choice?
8.26.2008 5:26pm
Robert West (mail) (www):
Mikeyes, you'd be surprised. Fewer than half of the people in my night 2L class are working full time, and on the order of a third weren't local to begin with.
8.26.2008 6:45pm
DavidBernstein (mail):
Mikeyes, there are many such students, but in the DC area, depending on how broadly you define it, we have around a half dozen "part-time" law programs. So, for example, if a particular school's night program is "weaker" in some way than its day program (putting credentials aside, at some schools, for example, many of the "regular" faculty never, ever teach at night, and at some schools part-time students are not eligible for law review), that would be of great interest.
8.26.2008 6:52pm
Eli Rabett (www):
I think there are a lot of assumptions here and would like to know the factual basis upon which they rest (I suspect the data exists in about every law school admissions office)

1. What is the correlation between success in passing the bar and UGPA.

2. Same for LSATs

3. Is there a difference between 1 &2 for part time students?
8.26.2008 7:22pm
Tony Tutins (mail):
DB is right. US News believes that "the more impressive the students, the better the school." (Shouldn't the school that makes lawyers out of the least promising material be ranked higher because it has accomplished more? Just a thought.) LSAT and UGPA are quick and dirty measures of how impressive the students are. In particular, UGPA records the student's academic performance. But, real world performance should be an acceptable substitute for academic performance and thus UGPA.

For US News scoring purposes, I would think it would be fair to discard UGPAs that are more than say, ten years old. This eliminates the grade inflation penalty, the stayed-too-long-in-the-wrong-major penalty, the late bloomer penalty, the reformed slacker penalty, etc. LSAT numbers are all reasonably fresh, so all of them could continue to be used.
8.26.2008 7:40pm
wm13:
It doesn't seem wrong to me that US News tweaks its formula to produce the "right" results. Formulas like this are primarily useful to evaluate candidates in the middle range. That is, design a logical formula, check that it produces credible results by ranking Harvard, Yale and Stamford at the top, and (say) Hofstra and Touro way, way down, and then ask: using that formula, which ranks higher, BU or Emory? St. John's or Cardozo? Etc.

It's like the formulas that people create to evaluate baseball players. If your formula tells you that Roger Maris is the greatest ballplayer of all time, then there's something wrong with your formula. Once you have a formula that gives credible results (e.g., Babe Ruth as the best), then you can use that formula to evaluate interesting questions such as: who was greater, Mantle or Mays? Ty Cobb or Rickey Henderson? Etc.
8.26.2008 7:47pm
justanotherguy (mail):
I like any solution that keeps the evening programs apart from the dreaded U.S. News rankings. If evening programs were ever to be ranked, I am not sure how one would perform that analysis and ranking. I attended my first year in the evening program at Georgetown while an active duty Navy Captain stationed at the Pentagon. My class certainly had a host of others with extensive life experience including a fellow US Naval Academy graduate who had built up a multi-million dollar business, a TV producer, several working medical doctors, several active research PhDs, a host of working federal police officers, etc. Yes about 1/3 of the class were younger students who seemed out of place in the evening program - they were without the same amount life experience of the rest of us older students.

Lumping the evening programs of colleges in with the "22 to 26 year old college kids" based on GPA and LSAT seems to miss the boat on what the evening program actually provides. (I know that the rankings use reputation among other measures - but GPA and LSAT are what everyone is fixated on)

Of course I don't place much stock in the U.S. New ranking either or the gaming that goes occurs. There were informal rankings that affected where firms looked for new hires long before the U.S. New ranking were developed. The informal rankings were based on reputation and I think that the new hires are still based on those reputations. I think that outside of the top law schools which have always been the same 15-20 schools, a firm's view of reputation matters more than some mysterious increase of 1 point in an LSAT score of incoming students and a magazine's rankings. A school gains in reputation because of things the school achieves or provides outside of the rankings (LSAT and GPA), and the law firms that hire will know of those improvements and the enhanced reputation.

Just because we can measure LSAT and undergrad GPA- a pulp magazine uses it for some mumbo jumbo and we now get hysterical over it. Go figure.
8.26.2008 8:51pm
Brian G (mail) (www):

I've had, for example, evening students in their 40s and 50s who have successfully built multi-million dollar businesses. Does it make sense to judge them based on their undergraduate GPA from twenty years earlier?


Sounds like a bunch of people that would be too smart to fall for the leftist "logic" of how you need the government to regulate every aspect of your lives. I doubt the administrators of most law schools would want these type of students. At the law school I went to, I know they would not be too welcome.
8.27.2008 3:42am
Mikeyes (mail):
David Berstein sez:

...if a particular school's night program is "weaker" in some way than its day program (putting credentials aside, at some schools, for example, many of the "regular" faculty never, ever teach at night, and at some schools part-time students are not eligible for law review), that would be of great interest.

I was speaking to the "mature" student (meaning older, possibly wiser, with a very specific reason to get a law degree, did I mention older?) when I asked my question about the utility of rankings in night schools. I suspect that if the night school was not on the par with the day school (the way Harvard summer school is to Harvard) that it woul be a disadvantage to the student who got in for future employers to the rankings in hand. Also, if you are a graduate of a night program, do you get a different diploma? I suspect that some schools have a different transcript at least. If I went to the Northwestern night program and it was taught by clinical staff only, I would I still be a graduate of that program in the eyes of those who might hire me?

I suppose it comes down to the reason you are going to school. I know that if I were to pass all the courses and get the degree I would most likely pass the bar since I have spent most of my adult life taking critical tests of my knowledge and passing them. I would not want a job as an associate working my ass off 100 hours a week. My dual degrees would probably get me a better, less tenuous job in a big firm. I would not make partner, but then I would not need to be a partner. Or I could teach in a night school.

On the other hand, if someone goes to a good school's night program and they are interested in the quality of the teaching (or at least the reputation) then the rankings would help some. But since none of the schools would be ranked as high as their daytime equivalents, the penumbral effect of being close but not quite the same would be lost. I suspect that even if an employer knew you went to the night program (and especially if they were a graduate of the regular program) that the reputation of the university would still have an effect. If Joe's Law School and Stanford night School were rated in the same general vicinity on the rankings because the LSAT scores and GPAs were the same, then the employer might have a different view.
8.27.2008 3:39pm
Hoosier:

In any event given how important these rankings are to law schools I don't blame them a bit for gaming the system. US News is to blame here. They have created perverse incentives for law schools to do all manner of weird things in order to improve or maintain their rankings that don't necessarily improve educational quality.

I agree with your overall point about institutional rankings. But I can't agree that US News is to blame, and university administrators have to game the system, given the significance of the rankings. The lack of courage on the part of university presidents, provosts, and boards is depressing to me. (I've spent my adult life in acadmia, and my late dad was an admissions director at a small college. So this stuff obsesses my little brain.)

The fear is expressed by the prisoner's dilemma. Example: Indiana University is hurt by the rankings in a very clear way. IU has the lowest incoming SAT scores of any Big 10 school. Is it a bad university? No. It is actually quite impressive.

BUT--Students at Indiana high schools who plan to attend IU or Purdue will generally take the ACT. The students who come in with "lower" SAT scores are mostly out-of-state kids from Ohio, Michigan, Illinois, etc. IU uses them to subsidize in-state-tuition payers. If the in-state kids were to take the SAT, the numbers would go up substantially, since IU is like most flagship state Us in that it takes many of the best kids from high schools around the state.

Why won't IU just tell USNWR to take a hike? Because other schools hurt by similar criteria might not join them in doing so. U of Iowa might use IU's decision to refuse to participate as a way of moving up in the rankings.

And we Hoosiers cannot allow that!

Pathetic.
8.27.2008 6:04pm