Notice & Comment

D.C. Circuit Review – Reviewed: Publishing Negative Results

(This was a very quiet week at the D.C. Circuit. No opinions. So feel free to skip this post. If you want something else to spend five minutes on, consider this–just for the experience. Warning: that link may haunt your dreams.)

This is not a novel observation, but even so, here goes: Researchers should publish their “negative” results too. Alas, there is a temptation to do just the opposite:

“Papers are less likely to be published and to be cited if they report ‘negative’ results” (Fanelli, 2010). Because scientists are involuntarily finding themselves engaged in competition for positions and funding, many are choosing not to proceed with their non-significant findings (those that support the null hypothesis) that yield less scientific interest and fewer citations. Consequently, the amount of non-significant data reported is progressively declining (Fanelli, 2012).

This is a serious problem—“if just (or even predominantly) the statistically significant studies are published, the published record misrepresents the true situation.”

So today, I’m going to do my part to fight the “file-drawer effect.” As far as I can tell, there is no correlation between the number of times a D.C. Circuit judge’s name is mentioned in Supreme Court briefs and that judge’s status as a feeder judge.

To begin, I expect that essentially everyone reading this blog knows perfectly well what a feeder judge is, but if not, I’ll explain. A feeder judge is a judge who “feeds” one of his or her clerks to the U.S. Supreme Court for a subsequent clerkship. Some judges do this much more often than other judges. Indeed, most judges never send a clerk to the Supreme Court. (If you need more background still, I’ve written a bit about the clerk-hiring process.)

The D.C. Circuit does extraordinarily well when it comes to feeding. Per Wikipedia:


And this chart undersells the D.C. Circuit. For instance, although not on the chart, Judges Srinivasan, Millett, and Pillard–relatively new judges–each fed a clerk this year, as did Judges Henderson and Williams. Last year Judges Sentelle and Silberman did too. Indeed, every judge on the D.C. Circuit is a potential feeder.

That got me thinking. Do feeder judges have their names mentioned more often than other judges in Supreme Court briefs? There are a number of reasons why that might be so. Perhaps litigants are most likely to mention feeder judges by name because such judges are more famous. Perhaps feeder judges are more likely to dissent, so they are cited more often (e.g., in the statement of the case). Maybe the fact that a judge is mentioned a lot causes that judge to be a feeder judge, because the Justices see his or her name more often (this strikes me as quite unlikely). Who knows? But it does not seem implausible that there may be a correlation.

So I decided to do a quick experiment to at least get a rough cut of the situation. In the Supreme Court Briefs section of Westlaw, my research assistant performed this search for each of the D.C. Circuit judges: “([insert name], J.)” & “D.C. Circuit.”* We also limited the search in two additional ways. First, my RA ran the numbers over the last ten years. And second, she ran them again after January 15, 2014—the date that Judge Wilkins, the newest judge, joined the Court. (I know; this is very imperfect because, for instance, some briefs within the last 10 years may cite cases that are older. There are many other problems.) We also excluded Judge Ginsburg for obvious reasons.

Here are the results:


What is the result? Well, Judge Kavanaugh is mentioned a lot in briefs and he feeds a lot of clerks. This does not surprise me. But Chief Judge Garland is (essentially) never mentioned, and he feeds even more clerks. That does surprise me. (Then again, Garland does not dissent very often.) And Judge Griffith is well respected and feeds a lot of clerks too, but he has not been mentioned in years. (Indeed, the “3” next to his name is misleading; one of the mentions—in this brief—references another Griffith (Griffith J. McRee) who edited The Life and Correspondence of James Iredell in 1858.) And Judge Edwards is mentioned a lot, but he no longer feeds especially often (although, to be sure, over his career he has sent dozens of clerks to the Supreme Court). Looking at all of this, I think it is safe to say that my hunch is wrong.

So what is the takeaway this week? Two things: Thank goodness blog posts are not peer reviewed and I’m sorry if you clicked on that first link. Definitely don’t click on this one.

* We also accounted for “(Garland, C.J.)” and “(Sentelle, C.J.)”


D.C. Circuit Review – Reviewed is designed to help you keep track of the nation’s “second most important court” in just five minutes a week.