pageok
pageok
pageok
Does Scholarly Productivity Improve Teaching Quality in Legal Academia?

There is a longstanding argument over the question of whether scholarship improves teaching in academia or detracts from it. Some claim that productive scholars are better teachers because they have a greater command of the subject and more original insights to convey to their students. On the other hand, it's also possible that scholarly productivity detracts from teaching. The time academics spend writing articles and books could instead have been devoted to improving their teaching skills.

Ben Barton of the University of Tennessee Law School has an excellent new article testing the impact of scholarship on teaching. He shows that there is little or no correlation between scholarly productivity and teaching ability (as measured by student evaluations) in a sample of over 600 law professors from 19 schools. Thus, scholarship neither improves teaching ability nor detracts from it.

This study is an important advance over the existing literature. But I have two significant reservations about it. First, Barton was not able to find data controlling for other variables that might affect teaching quality. For example, the quality of teaching might be influenced by a professor's speaking style, organization, personality, and so on. Some of these omitted variables might well be inversely correlated with scholarship. If so, it's possible that if we controlled for them, we might find that better scholars are better teachers after all.

Second, student evaluations are a highly imperfect measure of teaching quality; some of their shortcomings could end up biasing Barton's results against the hypothesis that scholarship improves teaching. I'm not one of those professors who thinks that student evaluations are useless. To the contrary, they often contain important and valid criticisms of the professor. My own teaching, I believe, has improved as a result of attending to such critiques. Nonetheless, teaching quality is far from the only variable that influences student evaluation scores. For example, we know from previous research that scores go up if the professor makes the class easier. Evaluation results are also influenced by such irrelevant factors as the professor's physical attractiveness. Although I don't know of a study testing this, I'm willing to bet that student evaluation scores are heavily influenced by the quality of the professor's sense of humor.

Some of these factors may well be inversely correlated with scholarship. For example, it's possible that the most productive scholars make their courses harder than those of the less productive ones, thus taking a hit on their evaluations. It's even possible that more productive scholars are, on average, not as well-dressed as their less productive counterparts (perhaps because they spend more time doing research and less time paying attention to fashion trends). If so, this too would lead them to get lower evaluation scores.

In Barton's defense, it's very difficult to get data on many of the relevant control variables. And student evaluations are perhaps the only available quantitative data on teaching effectiveness at most schools. Overall, I think he did the best he could with the available evidence. However, I don't think his evidence is good enough to decisively reject the theory that scholarly productivity improves teaching.

Eric Muller (www):
"Some claim that productive scholars better teachers because they have a greater command of the subject and more original insights to convey to their students."

This claim could only be very minimally true. I am a productive scholar, and my work in recent years has mostly been on the legal history of the Japanese American internment in World War II. But how could that make me a better teacher in my Criminal Adjudication class, or my Supreme Court seminar, or even the 98% of my Con Law class when I'm not teaching Korematsu?
9.10.2008 12:58pm
Ilya Somin:
This claim could only be very minimally true. I am a productive scholar, and my work in recent years has mostly been on the legal history of the Japanese American internment in World War II. But how could that make me a better teacher in my Criminal Adjudication class, or my Supreme Court seminar, or even the 98% of my Con Law class when I'm not teaching Korematsu?

Eric, I don't know whether your scholarship makes you a better teacher or not. However, it's possible that it does. For example, maybe your understanding of the development of con law during WWII also gives you useful insights on its development more generally. And even if your excellent scholarship didn't make you a better teacher, it's possible that such an effect did occur with other professors (e.g. - perhaps because their scholarship covers a higher percentage of the subjects they teach).
9.10.2008 1:02pm
TCO:
I think there is a lot of evidence in science, that premier researchers (and grant army winners) are not as good as people with more of a "high school teacher" didacticism and care for the students.

Law School may be different as it's really less of an academic subject...and the students more empowered to demand quality (like business school). Still, it would not surprise me if there was a tension, a trade-off.

All that said, both areas are fascinating and ones that can bring enjoyment. Teaching fresh young minds, established knowledge. Creating new knowledge.
9.10.2008 1:09pm
Duffy Pratt (mail):
Professors usually work in a culture that puts a premium on publication (which I assume is what the study uses to measure scholarship). Let's assume for the sake of argument that the professors who publish alot are industrious and ambitious. And that the professors who don't publish much are lazy and able to get away with it. Now, if you switch the premium to being a "good teacher" by some standard, then perhaps the publishing teachers would focus their ambition and industry on being better teachers. The lazy profs will probably do just what they are doing.

On a different note, I find it curious that you are so skeptical about student evaluations as a proxy for teaching ability, but you don't for a second question whatever proxy they are using for scholarship. It seems to me that the idea of a good scholar is at least as slippery as the idea of a good teacher.
9.10.2008 1:17pm
Ilya Somin:
On a different note, I find it curious that you are so skeptical about student evaluations as a proxy for teaching ability, but you don't for a second question whatever proxy they are using for scholarship. It seems to me that the idea of a good scholar is at least as slippery as the idea of a good teacher.

I don't question it because Barton uses multiple measures of scholarly productivity which, in my judgment, are quite compelling - especially in combination.
9.10.2008 1:29pm
Hoosier:
Pardon my ignorance of even the basic "rules of the profession" among law faculties. But . . .

I'm in the humanities, but with administrative responsibility over undergrads in all colleges and schools at this university. My strong sense is that, at the undergrad level:

A) There is no obvious correlation between scholarly output and undergrad teaching quality, positive or negative;

B) The real impact upon undergrads comes from issues outside the classroom, and in this case, research emphasis does detract from undergrad education: Faculty are either on leave or reduction more frequently if the administration increases expectations of output. Courses are often not available; are cancelled at the last minute, forcing students to scramble to get into whatever happens to be open; adjuncts are brought in to fill courses, but will not be around the following year to write recommendations, mentor students, etc.

C) Faculty significantly limit office hours.


Question: How does this work in law schools, as one goes up the ranks? Are research leave policies as "faculty friendly"? Do students have access to the full range of upper-level elective courses? Are mentoring opportunities readily available?

When we discuss these issues around here, I always try to raise tis matter, since I think we are asking the wrong questions when we focus only on the quality of twice a week lectures. How does it stand with you folks?
9.10.2008 1:31pm
Spitzer:
Wait, physical attractiveness is irrelevant when evaluating law school profs?

Seriously, though, I have always doubted whether research productivity enhances teaching or makes learning more efficient - in law school or not. I think it cannot be doubted that it is better for students to have a teacher who is intellectually engaged in the subject matter, and this factor may reinforce the research productivity notion somewhat. However, it is notoriously difficult to measure teaching productivity - if results are the tool of measurement, then it follows that teachers at higher-ranked schools (which attract a generally higher caliber of student) are almost certainly going to be measured as the best "teachers"; of course, the fact that researchers predominate in the elite schools would threaten to make any correlation of teaching/research to results tautological.

Unfortunately, research continues to be the measurement by which academics compare themselves (and are compared). I have always suspected that this became the case because it is much easier to put research creds on a CV than teaching evals, and research creds are an easier way for schools to compare candidates. Thus, research is a career-developing activity for many professors.

As I see it, there are two unfortunate consequences to this: (1) teaching skills are minimized, and so students may not be getting the best for their buck, and (2) the career-competitiveness of academics (whose cutthroat nature matches that of just about any profession in the world) generates a lot of useless schlock - for every useful piece of research, I daresay that there are many more that are just about useless (and, I may suggest, many of the authors of those useless products would themselves call them useless, at least in their more honest moments).

That said, I would love to see a host of studies attempting to correlate research with learning - I would love to have my instincts on this issue proved wrong.
9.10.2008 1:35pm
trad and anon:
What does this have to do with Sarah Palin?
9.10.2008 1:39pm
Jim at FSU (mail):
I have a theory that lack of tenure improves teaching quality. Lack of unionization in the teaching profession would probably have a significant positive effect as well.

I think most of the problems are due to incompetent, lazy or substance abusing professors who cannot be cast off despite their horrid performance year after year. Low performing untenured professors are a 1-2 year problem at most. A lazy, indifferent alcoholic with tenure is a 10-15 year drain on the department they inhabit. It drives students away from that practice area, encourages professors to defect elsewhere and wastes money that could have been spent hiring worthy faculty.

In my opinion, the best professors I had were the ones that had lots of real world experience and could hone in on the theoretical issues that had genuine relevance. They're non-tenured and they also tend to have the work ethic and knowledge necessary to succeed in the real world, so you're pretty much guaranteed to get your money's worth, and then some. One thing I love about FSU is how they have gotten judges (from the 1st DCA typically) to teach courses in their spare time. Who better to prepare us than the class of people who will one day judge the quality of our work?
9.10.2008 2:07pm
Aeon J. Skoble (mail):
IANAL, but the same argument goes on in undergrad academia. This : "Some claim that productive scholars better teachers because they have a greater command of the subject and more original insights to convey to their students" seems right to me and correlates with my own experience. Having an active scholarly life keeps one current and fresh. This, on the other hand: "The time academics spend writing articles and books could instead have been devoted to improving their teaching" seems wrong. Um, how exactly? "Getting better" at teaching isn't like getting better at playing the clarinet or getting better at your backhand. I've never heard any account of exactly what activity one could do during the time otherwise devoted to research that would make one a better teacher the way more time on your insrtrument makes you a better player. As far as I can tell, you only get better at teaching by doing a lot of it. That still leaves a lot of non-teaching time. Do you spend that doing research, or do you do something else, like reading and commenting on blogs... oh wait.
9.10.2008 2:10pm
Jim at FSU (mail):
Depends if they are teaching the same thing they research, which they almost certainly are not. The point of research is discovering novelty, the pointing of teaching is to convey what is known already so that students might someday "stand on the shoulders of giants." I don't think the two are necessarily connected at all.
9.10.2008 2:16pm
hattio1:
I have to challenge one aspect of your interpretation of the study. You claim that because there is no correlation between high scholarly output professors and teaching evaluations (eiher positive or negative) that there is no correlation between scholarly output and teaching quality. This is with the caveat that student evaluations are an imperfect measure of teaching quality. But, even with that caveat I'm not sure your interpretation holds true.

If a study could determine the change in quality of teachers as their scholarly output changes, I'm think you would have more of a correlation. All we can say from this study is that those who have high scholarly output are not intrinsically worse teachers. But, they may be worse than if they did not have high scholarly output.

As for my anecdotal experience, we had two professors my first year who were very well respected as far as their scholarship (I couldn't speak to how prolific they were). One was without a doubt my best professor in all of law school and the other was the worst. What I noticed is that the one who was good didn't seem to see teaching and scholarship as the same. The one who was the worst tried to explain his tort theories that he was publishing to a first year law class who still didn't really get the basics. So, in my opinion I don't think they're really related...unless the prof thinks they are.
9.10.2008 2:24pm
Aeon J. Skoble (mail):
Jim at FSU: Maybe this is more different than I thought for Law and for humanities. But if I'm doing scholarship and publications on, say, Aristotle's Ethics, that's going to have a direct impact on how I teach courses both in Ancient Phil and in Ethical Theory. If I'm writing a book on rights theory, it'll have an impact on my courses in Political Phil and Phil of Law and probably Ethical Theory also. Some might object that these examples are a little ad hoc, but in general, people teach courses in areas that interest them, so it's actually very likely that their research will be in the same areas they teach.
9.10.2008 2:25pm
M O'Brien (mail):
Of course, teaching skills and research skills don't have much in common! That doesn't mean that a scholar/professor shouldn't possess them both.

Or do you think that children should study either math or reading, but not both?
9.10.2008 2:36pm
PLR:
My oldest just began her freshman year at a major university, and enrolled in a ridiculously popular class on Greek mythology (not irrelevant to her curriculum as a whole).

I said to her, "As I remember these things, a ridiculously popular class like that usually means that it's an easy A or the professor is a hottie." She answered that both are true.

When can we expect the Volokh Conspiracy swimsuit calendar? Not for me, but I know some Log Cabin Republicans.
9.10.2008 2:39pm
Avatar (mail):
Part of the issue is that a professor without heavy research commitments is more likely to have time to dedicate to support-type activities, office hours, that sort of thing. Only so many hours in the day, so there's no getting around that.

In undergraduate courses, I've had several professors who were essentially phoning it in (and at least one charmer who came out and said "this is a useless course and a waste of my research time", and taught that way... though I don't think an effort made by him would have mattered much anyway.) As other posters have noted, there were at least a few who weren't regular professors, but lawyers, judges, and consultants, who generally did an excellent job in the classroom.

It all depends on the instructor, I suppose. When teaching courses becomes something that gets in the way of your vocation, instead of the content of your vocation, then your teaching is going to suffer...
9.10.2008 3:28pm
kidblue:
Why is physical attractiveness or sense of humor irrelevant? If they increase a student's enjoyment (and maybe even learning) in a class, then shouldn't they be positives?

Why be critical of the factors leading students to like some professors rather than simply accepting the students' preferences?
9.10.2008 3:43pm
Order of the Coif:
Each year that I've written more than one article, my student evaluations have complained that the professor is "distant" and "less interested in students than his reputation indicated." I guess that there are only so many hours in a day and so much psychic energy in a person. This is some anecdotal support for the idea that there are REAL trade-offs, not all positive.
9.10.2008 3:47pm
Order of the Coif:
Unfortunately, research continues to be the measurement by which academics compare themselves (and are compared). I have always suspected that this became the case because it is much easier to put research creds on a CV than teaching evals, and research creds are an easier way for schools to compare candidates. Thus, research is a career-developing activity for many professors.


Yes. It lends itself to "objective" bean counting and allows judgements about "scholarliness," salary, etc. without the need of actually interacting with the person.
Is has been my experience that colleagues and deans do more bean counting than literary criticism.
9.10.2008 3:58pm
Ilya Somin:
Why is physical attractiveness or sense of humor irrelevant? If they increase a student's enjoyment (and maybe even learning) in a class, then shouldn't they be positives?

They are relevant in the sense that they increase the student's enjoyment of the class. But they probably aren't relevant to teaching quality in the sense that they increase the amount that the student learns.
9.10.2008 4:31pm
pauldom:
@Skoble

"The time academics spend writing articles and books could instead have been devoted to improving their teaching" seems wrong. Um, how exactly?

I think the claim should be "could instead have been devoted to teaching." Teachers with more time could comment more extensively on papers, or comment on drafts plus final papers. Teachers could assign more essay exams rather than just multiple choice exams. Teachers could create additional practice tests or other supplemental learning materials. Teachers could spend more time updating lecture materials, perhaps using examples that are very current (e.g., "ripped from the headlines" vs "evergreen" examples). Teachers could hold more office hours, or they could respond to student email/phone messages more quickly. Then there are related activities that most depts consider "service" (program assessment, curricular reform, grad student mentoring, new faculty hiring &review, and so on). These activities arguably have just as much affect on student learning as publishing yet another article that <100 will actually read.

I believe that most faculty should engage in some mix of teaching and research. But there are only so many hours in a day. Time spent on one activity is unavailable to the other (which reminds me that I should quit reading VC and get back to real work).
9.10.2008 5:20pm
Cornellian (mail):
Evaluation results are also influenced by such irrelevant factors as the professor's physical attractiveness.

I admit, I would find it very difficult to give a bad evaluation to a prof who looked like Rachel McAdams. I'd probably just say nothing, or something politely non-committal or be very subtle in my criticism. Fortunately there was no risk of that situation ever arising at my law school.
9.10.2008 5:37pm
Aeon J. Skoble (mail):
"could spend more time updating lecture materials"
Yes - e.g., by actually learning something new.
"These activities arguably have just as much affect on student learning as publishing yet another article that <100 will actually read."
It need not be the case that few people read the publication that results from the scholarship; it's the doing the scholarship that makes _you_ better in the classroom.
But, like you, I need to get back to work! ;-)
9.10.2008 5:39pm
Aeon J. Skoble (mail):
"It need not be the case that few people read the publication that results from the scholarship; it's the doing the scholarship that makes _you_ better in the classroom"
Hmm, I kind of garbled that. If only there were a way to prevew comments before posting them...
I meant something like:
It need not be the case that many people read the publication that results from the scholarship; it's that having done the scholarship makes you better in the classroom.
9.10.2008 5:42pm
Aeon J. Skoble (mail):
Preview! D'OH!!
9.10.2008 5:43pm
pauldom:
OK, I couldn't stay away completely (busted!)

Skoble:

It need not be the case that many people read the publication that results from the scholarship; it's that having done the scholarship makes you better in the classroom.


Agreed. It's just not the only thing that does so.

And re your earlier message, "learning something new" to update lectures != producing scholarship (it *could* mean producing scholarship or it could mean reading the scholarship produced by others, which doesn't "count" in any productivity metric that I'm aware of).
9.10.2008 6:09pm
Adam J:
M O'Brien - I don't think they are equally important at all- teaching should be much more valued. The students are the school's clients, who are far more interested in having good teachers then good scholars. A school with great scholarship is like a firm with an amazing office, sure the client will be probably be impressed, but it really doesn't mean they're getting great service. I'd be fascinated to see how well an established school did if it decided to prioritize teaching over scholarship. Of course, those schools are risk adverse to any change that might hurt their prestige, and have too many with vested interests in the status quo.
9.10.2008 6:27pm
Adam J:
Ilya - I think a sense of humor or attractiveness are a definite asset to teachers- they can help students stay focused thru lectures that might otherwise be mundane.
9.10.2008 6:30pm
Bama 1L:
Student evals seem worse than useless for gauging pedagogical capability.

Pauldom presents several things professors could be doing other than scholarship that would improve teaching. Some responses:

1. More comments on written assignments: Good point, although you it's hard to refrain from rewriting the paper if you spend too long on this.

2. Essay exams rather than multiple choice: The time pressure here usually comes from the registrar. Grades have to be turned in by a certain date. If you only have a few days to grade a hundred papers, then you will right an exam that can be graded accurately in a few days.

3. Create and respond to more practice tests: Well, sure. I'm not sure this actually helps the students learn anything besides how to do well on your exams. It also burns up their time.

4. Update lectures: Oddly enough, when an instructor wants to talk about current events rather than the material, I perceive underpreparation. I listened to the radio on my drive in, too, and I got the same analysis you're presenting, bub. Now some instructors do add value and that's to be applauded.

5. More availability: Students who expect faculty to be at their beck and call 24/7 must not be humored.

6. Service: I have rarely seen a service activity directed toward helping students learn. What are you doing? Drafting revisions to the university's parental leave policy, sending tutors to the underperforming middle school in town, and finding more ways to trap students in the department's intro classes.

Furthermore, most research universities simply don't value teaching enough to rationally incentivize spending extra time on it. The way to get promoted is to research. Thus, if you want to feed your kids, you produce.
9.10.2008 10:06pm
Dan Simon (mail) (www):
He shows that there is little or no correlation between scholarly productivity and teaching ability (as measured by student evaluations) in a sample of over 600 law professors from 19 schools. Thus, scholarship neither improves teaching ability nor detracts from it.

On the contrary--even assuming that scholarly productivity and teaching ability are entirely uncorrelated, it follows that to the extent that a faculty factors scholarship into its ratings of faculty candidates, it will effectively be making its rating more random with respect to teaching ability. It will thus most likely end up with substantially worse teachers than had it evaluated teaching ability exclusively, and chosen the very best available teachers.
9.11.2008 12:13am
Angus:
To add an additional (albeit anectdotal) perspective: I have worked in humanities departments at 4 different colleges and universities. My observation was that the top few scholars in the department at each institution were the worst teachers. To them, teaching was a waste of time, so why put much effort into it?

In the late 1990s, for example, I stepped into a class for one scholar who was traveling during the semester to do research. He gave me his lecture notes, which were typed and yellowing. I asked him about them later, and he said he had made his lectures up in 1971 and hadn't touched them since. It was time better spent on publishing.
9.11.2008 10:29am
Aeon J. Skoble (mail):
Angus: I have no doubt that there are plenty of lame teachers. But actually, it doesn't even follow from your anecdote that the colleague was one of them. It's possible that even though he hadn't updated his lecture notes in ages, he's actually a dynamic and effective lecturer, while the fellow in the next room who makes a new set of powerpoints every semester is dull, hard-to-follow, and makes unhelpful comments on student papers (when he's not using multiple-choice tests). In any case, I wasn't suggesting that people with an active scholarly life are the only ones who can be good teachers, I was arguing that it's false to say that those people _aren't_ and can't be good teachers. It's a false dichotomy between "teacher" and "scholar."
9.11.2008 12:07pm