Saturday, March 26, 2005

Boondoggling....again

I really should be working on my paper for the Midwest (yes, I haven't finished it yet), but instead I've been distracted by a current discussion on a email list for political methodologists. It all began with an email from an untenured professor who had an article rejected without review by the AJPS. The AJPS is usually referred to as one of the 'big 3' journals in political science, and many top-tier political science departments have explicit or implicit policies that all untenured profs must publish in the big 3 to get tenure. This means that every untenured person dreams of the day their work will appear in one of the big 3. In any event, this email post created quite a discussion on the PolMeth listserv.

The rejection letter, according to the post, said:

"Over the last year or so our reviewer panels in all fields have consistently recommended that papers which only advance formal theoretical formulations without systematic theory tests should not be published in the AJPS. We interpret this change in peer reviewer sentiment as a result of the EITM movement. Whatever its cause, this change is quite consistent. Thus we are not accepting for review papers that fail to meet these reviewer expectations."

[Note: The EITM 'movement' is a program funded by the NSF. Read more about it here.]

Several of the PolMeth members proceeded to lament the fact that such an important and prestigious journal (at least for promotion and tenure getting purposes) would reject out of hand an article with a formal model, but no empirical test of the model.

This led me to wonder two things (or at least two things that I'm willing to voice openly):
1. In the AJPS editor's response to the email, he explains that the policy has the unamimous support of the AJPS editorial board and that it was instituted in order to relieve pressures on reviewers.

I wonder how much of an iron-clad rule this really is.

I suspect that if a theory only (i.e., formal model with no empirical test) paper were submitted by Dr. SuperSmart with a Well-Endowed Chair of Nonsense Science of IvyLeague University that made a substantial advance to theory (instead of just a minor tweak to existing models), the AJPS editors would send it out for review for two reasons. First, it's Dr. SuperSmart, and she has a proven track record (and might even be eligible for a Nobel Prize, if there were one for Political Science). Second, it's a paper with a substantial advance to theory. Now, maybe this is unfair, since I haven't read the paper in question, but it's the big elephant in the room in this email discussion. Maybe that explanation in the form letter was a nice way of letting the untenured prof down lightly. I suspect the AJPS policy is not as much of a hard and fast rule and the PolMethers assume that it is. Me thinks they doth protest too much.

2. I find it mildly amusing that a group of formal modelers are complaining about an editorial policy that might relegate their "pure theory" contributions to 'specialized' or [gasp] second-tier journals. Now, I should clarify that I am NOT a member of the Prestroika movement in political science; I do not wear a badge at political science meetings. I am, however, a comparativist whose research employs both qualitative and quantitative methods.

As a comparativist, I understand that there are some papers that I might write (or avoid writing by blogging or think about writing but then never write becasue they wouldn't get published) that would never get published in AJPS because it is a general interest journal of political science. What that means is that the AJPS has a responsibility to publish articles of interest to a WIDE political science audience. My (imaginary) article on the political preferences of Mexico City sex workers would probably not be of general interest to political scientists, so AJPS should probably not bother reviewing it (and more importantly, I wouldn't waste their and my time sending it to them).

The amusing part of the PolMeth discussion is that, apparently, many members of the list believe that a theory only (again, I feel the need to reiterate that this would a be a highly technical formal model only) paper would be of general interest. Why shouldn't it be? They are interested in it? Well, I may be interested in the political preferences of Mexico City sex workers, but I realize that not all political scientists may be interested in that topic. Further, I recognize that the potential research spin-offs of my sex worker article are fairly narrow. I suspect that if the potential research spin-offs (or contribution) of a pure formal theory article with no empirical test were obvious and compelling, then I bet the good editors at AJPS would send it out for review.

[Clearly, these guys have never had an NSF reviewer suggest rejecting their proposal mainly because it lacks a formal model...in a research area where 95% of the papers published are not formal. The formal modellers do hold positions of gatekeeping and power, and they are so dismayed to find that there might be one journal where they do not set the agenda.]

3. [I guess I have a third idea to add...] I think many faculty abuse the review process and highly efficient journals like AJPS suffer. I have had at least three different colleagues suggest that I send a paper to AJPS just because of the short turn-around time, even though I am skeptical that it would be appropriate for that journal. Now, however, that I know they are rejecting all those formal theory papers, maybe I should send my empirical (but theory-light...i.e., no formal model) paper to AJPS.

A completely unrelated comment

While looking for discussions about EITM, etc., I cam across a blog that links to a blog that links to ....you get the idea. I came across this blog by Mike Munger of Duke University. I was in a methods class that he teaches at Duke. Mike is a funny guy, which is ok if you have the hudspah (spelling?) to stand up to him or at least take his kidding. In our class, that wasn't always the case. Oh, I could dish it back but I distinctly recall a meek-mannered Duke grad student who looked like he was going to cry once during class. But, I digress. I particularly like this post, and its link to this other blog post.

I really must get back to that paper for the Midwest. BTW, it will be qualitative and not of general interest, but hey, maybe I should send it to the AJPS anyway.

posted by Michelle @ 1:22 PM,

0 Comments:

Post a Comment

<< Home