Articles on Law, Science, and Engineering

Phantom Risk

Review by Edward P. Richards, 13 IEEE Engineering in Medicine and Biology Magazine #3, pg. 427 (Jun/Jul 1994)

MIT Press, Cambridge, MA, 1993, Kenneth R. Foster, David E. Berstein, and Peter W. Huber, Eds. ISBN 0-262-06156-s, xi+457 pages, $39.95.

Phantom Risk, subtitled Scientific Inference and the Law, is a must read for anyone interested in the public policy or law of toxic exposures, and, more generally, the intersection of law and science. This is not a philosophical or political book. It is a collection of technically grounded essays and short reviews by leading scientists, interspersed by discussions of the relevant law and the legal controversies surrounding each topic.

As one who does research and teaching in this area, I found the book well reasoned and effectively presented. It ends, however as a cliff-hanger: at the time the book was published, a major junk science case was before the United States Supreme Court. As I discuss at the end of this review, the court pushed science off the cliff. Knowing this while reading Phantom Risk makes the book somewhat poignant because you know that the carefully presented facts and analysis are ultimately for not.

Phantom Risk has 15 chapters on specific controversies, including, among others; weak magnetic fields, spermicides and birth defects, Bendectin, miscarriages and video display terminals, asbestos, polychlorinated biphenyls, trauma and cancer, and dioxin. There are also six chapters on legal and policy issues that effectively put the technical chapters into a broader perspective.

The book is divided into two parts. Part One, titled "Phantom (Or Not So Phantom) Risks", deals with cases such as Bendectin where there is no credible scientific evidence establishing a link between the substance and human health risks. Part Two, entitled "Just a Little Bit of Poison", deals with substances, such as asbestos, environmental pollution, and dioxin, which have known health effects, but whose risks are greatly exaggerated by the lay press and the legal system.

The individual chapters are short and to the point. They present data to support their arguments and are well referenced to the literature. Each author provides a brief explanation of the methodological issues involved in measuring the risk under discussion. While these explanations are adequate for the scientifically literate reader, they are probably insufficient for laymen unversed in science. This is a reflection of the general problem discussed in the book: policy makers, lawyer, judges, jurors, and the general public is often so poorly educated in science that they are unable to independently evaluate the issues before them. (A previous book by Peter Huber, Galileo's Revenge, makes the same case for a lay audience, but it is correspondingly less effective because of the limitations on the depth of the presentation.)

The Expert Witness Game

As one reads Phantom Risk, one finds several common themes. The first is a classic signal noise problem: how do we separate the injuries caused by a specific agent from the background noise of injuries caused by known and unknown factors. If we cannot do this, we will blame innocuous substances for injuries they do not cause, and we may ignore substances that cause significant injuries. (Perhaps this should be called phantom comfort.) The most legally troublesome example is birth defects.

Approximately 3% of babies are born with noticeable birth defects, about 1.5% of which are significant threats to life or health. This is a much higher rate than is assumed by most persons who are unfamiliar with birth injury epidemiology. The second complicating factor is the "health is your own responsibility" rhetoric. Pregnant women are told not to drink, not to take drugs, to eat boring healthy diets, in general given the impression that if their baby is not perfect, it is their fault. They are also bombarded with advertising from women's hospitals that reinforces that notion that everyone can have a perfect baby. The result is easy to predict: when a baby with birth defects is born, the parents are looking for someone to blame to assuage their guilty feeling that it was their fault.

The numbers tell the rest of the story. Approximately 30,000,000 women took Bendectin to treat nausea of pregnancy. Using the 1.5% rate for severe birth defects, this group of women had approximately 450,000 children with severe birth defects. Imagine the headline: WOMEN WHO TOOK BENDECTIN HAVE 450,000 DEFORMED CHILDREN! To defend Bendectin, the manufacturer must convince the jury or judge that there is nothing out of the ordinary in 450,000 birth defects. This problem is complicated by the second theme: credentialed people giving fanciful testimony that is based on demonstrably false premises.

Plaintiffs must put on expert witnesses who will testify that the substance in question caused the plaintiffs' injuries. The expert witness system is intended to improve the accuracy of the court's decisionmaking by assuring that someone who understands that underlying science is willing to vouch for the validity of the plaintiff's or defendant's case. This system is predicated on the availability of impartial, knowledgeable experts.

Many expert witnesses still fit this ideal. They work within well accepted medical and scientific boundaries and present testimony that, while favoring one or another party, is wholly defensible. Their disagreements with the opposition witnesses stem from ambiguous facts or well accepted differences of opinion. The problem addressed in this book are experts who are willing to testify to theories that are well outside the bounds of acceptable scientific inference, or are even against the laws of nature.

In their chapters, the editors discuss the natural evolution of scientific thought and how it has been distorted by the expert witness system. In particular, as new theories enter the marketplace of ideas, they are taken up by many supporters. As evidence develops, either the theory is proven and gains more support, or it is discredited and loses supporters. This process was investigated by Thomas Kuhn, in his book, The Structure of Scientific Revolutions. Kuhn pointed out that all major scientific theories (he called them paradigm shifts) encounter great resistance from established scientists and only prevail after years of struggle.

This is taken by plaintiffs as proof that science is all political, that the general rejection of a scientific theory tells us nothing about it accuracy. In the extreme cases, experts who testify to especially farfetched theories are portrayed as heroes, fighting the scientific establishment to help the downtrodden. (Hence the title of the predecessor book, Galileo's Revenge: our punishment for suppressing Galileo is a plague of psudeoscientists.)

As the editors explain, this is a profound distortion of Kuhn. Kuhn recognized that, in the vast majority of cases, new theories are bunk and are rightly discredited. If a scientist is out of the mainstream, it is not proof she is wrong, but it is the way to bet. More fundamentally, Kuhn wrote about big ideas; paradigm shifts. These are resisted because of the innate and necessary skepticism of science. Most junk science is not about big ideas, it is about narrow exceptions to general theories, intended to reach a particular result in a specific case. In the general run of science, these are probably tested and accepted or rejected with little of institutional inertia that accompanies a paradigm shift.

The effect of the expert witness system (and television talk shows, and popular press reporting) is to give scientists who might otherwise discard an idea an incentive to suspend their skepticism: for money or fame or both. There is a lot of money to be made as an expert. It is easier than running a lab, and there are no federal strings attached. It can be combined with an academic career to allow a professor to make what her or she fells they are really worth. There is also limitless reassurance from the attorneys that hire experts that they are defending truth, justice, and the American way. (As a point of balance, Phantom Risk is concerned with plaintiffs' junk science. Defense lawyers will use junk science just as readily, as the Tobacco Institute, run by and for lawyers, has so aptly demonstrated.)

Unfortunately, experts are not evaluated on the scientific merit of their testimony. Jurors evaluate experts on their ability to tell a convincing story and their credibility, i.e., do they look smart and honest. In the final analysis, there is nothing else for a layperson to do, and it is a problem for well-educated judges as much as for jurors. Phantom Risk uses case involving liability for spermicidal jellies as an example. There was no scientific evidence to back up plaintiff's claim that her child was deformed by exposure to the contraceptive, only testimony by a physician who said that it must be due to the jelly and other physicians who presented testimony based on strained interpretations of animal studies.

The case was tried to a judge, sitting without a jury. The judge awarded the plaintiff $5,100,000. His explanation of his verdict is very illuminating:

"The Court's decision, therefore, turned on the oral testimony of a variety of expert witnesses whose opinions often were diametrically opposed on the major issues presented in the case. In assessing the credibility of these witnesses, the Court considered each expert's background, training, experience, and familiarity with the circumstances of this particular case; and the Court evaluated the rationality and internal consistency of each expert's testimony in light of all the evidence presented. The Court paid close attention to each expert's demeanor and tone. Perhaps most important, the Court did its best to ascertain the motives, biases, and interests that might have influences each expert's opinion." Wells v. Ortho, 615 F. Supp. 262, 266-267 (1985)

It is particularly telling that a major criticism of the defendant's experts was that they were unduly critical of the plaintiff's experts! In reaching this decision, the judge relied on the legal technique of setting up the false dichotomy: there must be two sides to every argument, therefore you must assume that either side may be correct, then you fault the defendant's because they do not take the plaintiff's experts seriously and because they keep talking about statistical generalities, while the plaintiff's experts address the real issue in the case, the plaintiff's injuries.

Societal Policy Issues

Phantom Risk also deals with public policy issues outside the courtroom. The editors and chapter authors attack the core problem of proving a negative, i.e., you cannot prove something is safe. It is well illustrated in the chapters on weak magnetic fields and on radiation fallout risks, among others. In each case there have been many studies that have either found little or no risk to human health. In each case, however, the popular press and politicians demand proof that there is no risk and continue to treat the question of human health effects as open.

What this ignores, of course, is that when you have large, well controlled studies that are negative or equivocal, you do know that whatever the health risks, they must be very small. If weak magnetic fields causes any substantial health effects, these would be obvious from the studies that have been done. Policy makers and the public seem unwilling to accept that if there is a substance that is making a lot of people sick, and we look at thousands of people exposed to the substance, we would notice the effects.

The chapter on dioxin is especially interesting because it deals with the political use of the fear of toxic exposures. Briefly put, based on animal studies, dioxin was thought to be one of the most toxic chemicals released into the environment. Much environmental policy and resulting laws were passed with the aid of horror stories about the risks of dioxin and other toxic wastes. Long term studies of people exposed to dioxin, including some with very high exposures, have failed to demonstrate any significant risk to human health.

This finding is politically unacceptable to many environmentalists, both inside and outside the government. By discrediting their claims about the risks of dioxin, it also threatens to undermine the public consensus that chemicals in the environment are major hazards to human health. Without this consensus, it will be difficult find political support for massively expensive cleanup programs such as superfund. Thus there is tremendous pressure to ignore the scientific findings because of their political consequences.

The Rest of the Story

At the time that Phantom Risk went to press, the United States Supreme Court was considering the case of Daubert v. Merrell Dow., 113 S.Ct. 2786 (1993). In this case, the plaintiff alleged that Bendectin caused her child to have birth defects. The trial judge refused to let the plaintiff's experts testify because they could not produce any generally accepted (or even published) evidence that Bendectin caused birth injuries. This would have resulted in a verdict for the defendants.

In its review of the trial judge's decision in Daubert, the United States Supreme Court determined that it was improper for the trial judge to hold the plaintiffs to the standard of generally acceptable scientific evidence. This was based on the court interpretation of the federal rules of evidence, which govern the necessary qualifications for expert witnesses and testimony.

The effect of the Daubert decision is two fold: 1) it sends a message to judges who have set high standards for the admission of scientific evidence that they should review their actions; and 2) it gives a green light to less strict judges to admit any testimony they want.

Daubert does not detract from the value of Phantom Risk. If anything, it makes the questions raised by the book more critical to resolve. Irrespective of whether one agrees with all the points of view in the book, it is an important book that should be read and recommended to anyone who wants to better understand the nexus of law and science.