The new England Journal of Medicine weighed in today with an editorial on a topic that should be important for readers of MassTortDefense – peer review. Curfman, et al., Peer Review in the Balance, 358 NEJM 2276 (May 22, 2208).

The editorial weighs in on two recent decisions in the Celebrex litigation involving product liability defendant’s attempts to go beyond the veil of the published article and obtain discovery on information created in the peer review process as manuscripts were assessed. Those manuscripts related to studies of science relevant to the litigation, and the argument was advanced that commentary, discussion, and analysis of those studies by expert peer reviewers likely also contained relevant information about that pertinent science.

Legal Setting

Expert testimony is essential in virtually every mass tort or complex product liability litigation. Under the gate-keeping function of the trial judge, designed to keep junk science out of the courtroom, the Daubert opinion lists important factors to consider in admission of expert testimony, including whether the theory or technique has been subjected to peer review and publication. This factor is relevant, but not dispositive, because sometimes well-grounded but innovative theories will not yet have been published, and sometimes a topic may be of too limited interest to be published. But the Court believed that peer review increases likelihood that substantive flaws will be detected in an article, and publication places the work within the test of the marketplace of ideas. The theory is available to be scrutinized, and has been scrutinized by experts in the relevant field. In essence, submission of the method, theory, reasoning, to scrutiny of scientific community is considered component of good science.

The Daubert Court was aware from amicus briefing of another view of peer review, emphasizing that the peer review system is not a litmus test for truth, not a guarantor of scientific certainty, and does not guarantee that the particular data, research and analytical methodologies, or conclusions of the accepted papers represent the consensus opinion of the relevant community. Brief of Amici Curiae Chubin et al., Daubert, 1992 U.S. S.Ct. Briefs LEXIS 938. But the Court generally rejected this view.

There is some lower court variation in application of Daubert and Frye on peer review; some courts emphasize the mere fact of peer review, Doe v. Ortho-Clinical Diagnostics Inc., 440 F.Supp. 2d 465 (M.D.N.C. 2006)(thimerosal), with others offering a more nuanced view of peer reviewed publication serving as a validity enhancer. The presence of the limited view encourages plaintiffs to seek to get theory published somewhere to “check the box” of peer reviewed publication. The more rigorous view reads the Supreme Court as clearly intending that judges use their own evaluation of peer review and publication as a tool for exploring whether substantive flaws in an expert’s methodology have been or could have been exposed. Black v. Rhone-Poulenc, Inc., 19 F.Supp.2d 592 (S.D.W.Va. 1998) (depends upon how the peer review was performed; who did it; plaintiffs failed to provide sufficient information to assess).

Now, before getting into the two recent decisions and the editorial, a brief background on peer review.

Brief History of Peer Review

Modern medical journals came into existence during the 19th century. They were originally a vehicle for personal journalism, with the models for medical editors including the crusading newspaper writer. Publication was a personal vehicle for the editor. Over time, newspapers came to resemble newspapers, with correspondents to cover medicine in different geographic and specialty areas. But in many medical journals, the editors wrote much or all of content. A second type of journal also arose, in addition to those journals that followed the newspaper model, and that was the official publications of research institutes, typically specializing in one kind of research.

In the 19th century editors often could not fill their columns. The problem was finding publishable manuscripts, not turning down original contributions. Editorial board members would help round up material for the editor to publish, rather than review and reject manuscripts. In the 20th century, the supply/demand ratio gradually reversed. The number of passable manuscripts began to increase greatly, so that choosing among good articles became an option. Between 1913 and 1925, the Journal of the American Medical Association received 1500-2000 manuscripts for consideration. And the end of World War II marked the beginning of a dramatic increase in the volume of research and the subsequent reporting of research results. So by the 1990’s, JAMA was receiving 4500 papers/year, plus 5500 letters to editor. Yet JAMA publishes about 500 articles a year. Or the NEJM, which was getting more than 4000 article submissions/year, and publishing less than 10%.

Today there are more than 10,000 medical and scientific journals, and more than 2 million scientific and medical articles published each year. Most are never cited again.

There are examples of historical scientific journals in France and England with well thought-out peer review process to select reports to be published in journals and memoirs. In 1752, the Royal Society of London, a medical society, began publishing Philosophical Transactions and established a “Committee on Papers” to review articles submitted for publication. But in the U.S., most journals were personal fiefdoms of editors. (E.g., James McKeen Cattell, who for 50 years edited (and owned) Science and other major journals at the same time. ) As noted, editors wrote much of the content; they considered themselves as expert as anyone to assess content. The only mechanism that resembled modern peer review, perhaps, was book publishers seeking outside advice regarding textbook manuscripts in technical fields.

Peer review developed in part to deal with situations in which an editor lacked the specialized knowledge to make decisions about highly technical articles. Particularly after WWII, more specialists in subject matter, in terms of technique and, especially, in medicine, and laboratory materials were needed. And when physicians began to specialize in particular fields, it also spawned a demand for specialized journals catering to that specialty.  The supply/demand ratio gradually reversed and there was a growing need to discriminate high-quality articles and turn down others in the growing volume of at least plausibly good submissions that they had to consider.

Fast Forward to Today
An example of rigorous peer review is the New England Journal of Medicine, which publishes less than 10% of the unsolicited manuscripts it receives. There is a multi-step process including an initial screen by and editor; and internal review by an associate editor; submission to at least 2 peer reviewers/referees; presentation of the article to an editorial meeting; a statistical/math review, etc. There is a step for an additional review and revision by the author.

Today, scientific and medical journal have a significant role in shaping the views and behavior of scientists, clinicians, and policy makers. Contrary to the theory that journals act as passive conduits for scientific content, studies show the impact of an article is strongly influenced by which journal published it, and authors know that. There is a potential for a few high-impact journals to shape heavily the use of scientific information. At the same time, the increasing number of biomedical journals, the advent of online publication, and the increasingly interdisciplinary nature and impact of research, are forcing journals to compete for the best research.

With that background, the current controversy in the next posting.