"Makes you feel so, sort of, insignificant, doesn't it?"

To quote Monty Python. Tom Smith has the sad truth here. An excerpt:

A voice, crying in the wilderness, and then just crying

By Tom Smith

I just got some new data back from Lexis, with whom I am engaged in a massive citation study, but that's another story. This data concerns law review articles that are in their Shepard's database and how much they get cited. This data covers about 385,000 law review articles, notes, comments, etc. etc. that appear in 726 law reviews and journals, and looks at how often they are cited. Cited by other law reviews, or cases.

First of all, 43 percent of the articles are not cited . . . at all. Zero, nada, zilch. Almost 80 percent (i.e. 79 percent) of law review articles get ten or fewer citations. ...

Yoi and Double Yoi!


Now they're really
piling on. Does anyone else feel a midlife crisis coming on?

I respect Prof. Zywicki, who often posts on controversial topics, for opening comments on his posts and sometimes taking quite a bit of heat as the result.

I can't say the same for Jim Lindgren, who just posted one of the most overtly political diatribes I have seen in a while, but without comments enabled. And unfortunately, the very first claim he makes is completely false.

"As the Washington Post reported: "According to the former Niger mining minister, Wilson told his CIA contacts, Iraq tried to buy 400 tons of uranium in 1998." So Wilson had found evidence that tended to confirm the substance of the sentence in Bush's 2003 State of the Union address: "The British government has learned that Saddam Hussein recently sought significant quantities of uranium from Africa.""

If Lindgren had followed his own link to the Washington Post article, he would have seen that his claim was false and that the Post had retracted it. Again, it would have been nice to post this in his comments section, or to e-mail him, but the link with his name at the right side of the page goes nowhere.

The remainder of the post, including the WSJ's provocative (yet unsourced and unsupported) claim, could be the subject of many fruitful hours of discussion, I'm sure, but I won't take up any more of Prof. Zywicki's space to do so, and I apologize to him for this intrusion.
7.14.2005 11:43am
Anonymous jim (www):
Hmmmm Haaaa.
7.14.2005 11:43am
jallgor (mail):
On the other hand, law review articles can be very useful for practicing lawyers for researching a topic. The lawyer doesn't cite the law review article in that case they just use it to find relevant case law.
7.14.2005 11:45am
J (mail) (www):
Honestly, I tend to think of LR articles as the invented currency for the professor. There's very little vertical movement (tenure track, and not tenure track), but as professors are competitive generally (as are most successful individuals), they need some sort of measuring stick.

I'm not sure who's idea it was to make publications the billable hour of academia, but it's too settled to change now I think. Much like a substantial portion of the lawyer's time is all for naught (hundreds of hours spent on a case lost, smaller bits of hours preparing motions denied, etc.), so can a professor's. In that perspective, this isn't that big of a deal.
7.14.2005 11:58am
Jerry L. Liberman (mail):
IS JIM LINDGREN A COWARD???? If not, why did he not enable comments on his ridiculous, completely hack-worthy post on Joe Wilson.

This marks the day Volokh's blog has jumped the shark. Lindgren's post is a complete joke. Anyone who would defer to PowerLine cannot be taken seriously. PowerLine openly admits that it does not write things critical of conservatives (see their FAQs). John Hinderaker has accused Jimmy Carter of committing treason, he has stated on video which is available widely on the internet that all people to "the left of Lieberman and Zell Miller" are against America and want it to fail. Yet, Lindgren relies on them. Give me a break --- of course what they are writing is completely favorable to the administration. They NEVER criticize the administration.

Moreover, as anyone who has worked on criminal law knows, we don't know what Fitzgerald knows, we are not privy to the grand jury, and Fitzgerald has been very good about leaks (unlike Lindgren's apparent hero Ken Starr), yet Lindgren states that we can know that it is unlikely that a crime can be proven beyond a reasonable doubt --- give me a f---ing break!

Finally, Jim's "analysis" that the poor administration was forced to possibly out a CIA agent because they had no other way to counter WIlson's charges is completely outrageous. I am sure they could have argued their case on the merits without it. I seem to remember the previous administration dealing with new unfounded accusations every week (Clinton sold Coke, Clinton fathered bastard children, etc.), yet they didn't see it necessary to out CIA agents and commit other crimes to counter these charges.

Professor, your blog has jumped the shark with the right-wing hacks you let post here. Lindgren and Zywicki are jokes. This place used to be a place to read thoughtful, reasoned, nonpartisan analysis from generally right-wing/libertarian intellectuals. Now, it's a sorry excuse for PowerLine and Instapundit.
7.14.2005 12:21pm
John J. P. (mail):
Don't be discouraged. I think Smith's finding simply means that 43% of law review articles are, well, crap. There will always be an audience for good articles. On the other hand, articles that are superficial, wholly derivative, or badly written should be forgotten ASAP.
7.14.2005 12:31pm
Jake (mail):
And, even worse, the data (as Tom Smith says in the full posting) doesn't exclude self-citations. Yikes.

On a side note, I used to think that those constant admonitions to keep comments polite, respectful, and on-topic were unnecessary.
7.14.2005 12:58pm
Daniel Chapman (mail):
It's a hazard of being linked to by some of the darker corners of the political web.
7.14.2005 1:05pm
Adam (mail) (www):
I was going to launch a similar complaint about Lindgren's post -- my understanding was always that this website wasn't going to be the place for that sort of thing -- that it was about law, not brute force politics.

If he (or anyone) wants to get into whether any law was violated, based on the facts as alleged, fine, but I thought this wasn't The Corner, and I like it that way.

(Also, fwiw, should this website routinely engage in the promotion, attack, support or opposition of federal candidates for election, where Lindgren is leading, and if y'all have spent >$1000 in server and other costs, then, arguably, depending on how the FEC lays down its pending regs, the Conspiracy may become a political committee with legally enforceable reporting and disclosure requirements. Just saying, is all.)

Back on topic: my Comment, published at 1996 U Chi Legal F 495, is still in the <10 zone.
7.14.2005 1:15pm
M (mail):
Maybe we should start a sort of log-rolling network for law review articles and all agree to cite each others papers, if this is needed or not, so that we can all feel good about ourselves. (I wish I could say that I was totally joking.)
7.14.2005 1:24pm
Hans Bader (mail):
I'm glad to hear that most law journal articles are never cited, since I've been too lazy to write anything other than a short case comment in law journals. My laziness has been rewarded! By contrast, one of my letters to the editor in the New York Times was cited favorably in a law journal case comment, and a short Op/Ed I published in the National Law Journal was cited with approval in a law journal article in the Howard Law Journal. Maybe I've been rewarded for my short attention span, which permits me to write Op/Eds and letters to the editor, rather than in-depth law journal pieces.
7.14.2005 1:28pm
JonBuck (mail):
I'm a librarian. One of the things we do to see if our collection is any good is a use study. Most of these studies find that 20% of the collection gets 80% of the use. It's just the way things are for some reason.
7.14.2005 1:34pm
Split Lip Rayfield (mail):
J said it. But I would add that the other purpose of law reviews is to give the top 10% or whatever of the class something to do and something to put on their resumes.

Rarely do I use a law review article. I mean I can't even remember the last time I did.

I agree about the comments concerning Zirwicki; he posts too much and they're not as good as Volok's.
7.14.2005 2:01pm
The data seems to include notes, which make up a significant portion of total law review articles. I'd imagine notes are cited significantly less than other articles, so that may skew the numbers somewhat.
7.14.2005 2:11pm
I think Jerry L. Liberman should take a chill pill.
7.14.2005 2:39pm
Dick King:
Assuming all or most citations of law review articles are by other law review articles, it pretty much had to be this way. The average number of times law review articles are cited has to equal the average number of citations law review articles make.

Now it's possible for articles to be uniformly cited, but most human artifacts involving popularity have what's known as a Zipfian distribution. If you crank the numbers, and the number of law review articles is substantially greater than the square of the average number of citations in law review articles, you will get many uncited articles just by a counting argument.

The fact that law review articles can be cited by cases makes things a bit more complicated, but this may be uncommon. Cases can be cited by law review articles as well, but it could be a fact that the majority of uncited entities are cases.

385K entities were checked. If the number of citations in the average entity is substantially less than several hundred, we will have over half of the entities going completely uncited — as a consequence of graph theory and the Zipfian distribution.

7.14.2005 6:37pm
Zywicki (mail):
Dick King:

Excellent point--now that you mention it, I recall reading that for "great" scholars in general (such as Nobel Prize economists), the distribution of citations for each author was similar to this. So, for instance, Coase actually wrote many, many articles--but only a handful of them have been cited more than a dozen times. Most of his, or anyone else's articles, largely disappear.

I remember the author making the point that those who write famous articles (that are cited a lot) often also write a lot more articles than the average scholar. And only rarely do they accurately predict in advance which of their articles will turn out to be significant, and which will be largely ignored. Often they are remembered for something completely different from what they thought would be their legacy.
7.14.2005 7:16pm
Robert Schwartz (mail):
If only 80% of law review articles sink without a trace they are out performing ,Sturgeon's Law.
7.14.2005 8:08pm
Robert Schwartz (mail):
I hope the URL works better this time

If only 80% of law review articles sink without a trace they are out performing, Sturgeon's Law.
7.14.2005 8:11pm
DC Lawyer:
Might we have your liver, then?
7.14.2005 10:29pm
Zywicki (mail):
All right, you talked me into it.
7.15.2005 5:48pm