Articles on Law, Science, and Engineering

The Supreme Court Sets Standards for Engineering Expert Testimony

By Edward P. Richards and Charles Walter, 18 IEEE Engineering In Medicine And Biology Magazine #6, 83-4, 88.

The United States Supreme Court set the standards for admitting testimony about scientific evidence in the Daubert case [1]. Daubert dealt with the methods that an expert witness used to interpret epidemiologic studies of birth defects in the babies born to women who took an antinausea drug, Bendectin, when pregnant. Daubert sets forth four nonexclusive factors that the Supreme Court felt might bear on a judge's decision in determining the admissibility of expert testimony about scientific knowledge.

These four nonexclusive Daubert factors are:

(1) Whether a theory or technique can be (and has been) tested.

(2) Whether it has been published and subjected to peer review.

(3) Whether it has a high potential rate of error, and whether there are standards controlling its operation, and

(4) Whether it enjoys general acceptance within the relevant scientific community.

Since Daubert clearly states that these factors might apply to testimony about scientific knowledge, most courts understood that, in a given case, they might apply to such testimony, or, for that matter, to testimony about technical or other specialized knowledge as well. Nothing in Daubert suggests that these factors can not be applied to testimony about technical or other specialized knowledge. Nevertheless, the issue upon which the Supreme Court granted certiorari in Kumho Tire [2] was whether a trial court judge could ever consider any of the four Daubert factors in determining the admissibility of an engineering expert's testimony. Naturally, the answer in the unanimous opinion decided by the Court on March 23, 1999, was "yes."

However, the most important message from Kumho Tire is not about the four Daubert factors. In the clearest language possible, the Supreme Court is saying that trial court judges have the discretion to use the Daubert factors, or any other factor that is helpful in evaluating the reliability of any expert testimony, regardless of the field of specialized knowledge upon which it is based, and to exclude whatever testimony they find in their discretion to be unreliable. Using untrained lay judges in this manner to resolve complex scientific and technical issues creates problems for plaintiffs and defendants alike.

The Facts of the Kumho Case

On July 6, 1993, the right rear tire of a minivan driven by Patrick Carmichael blew out. In the accident that followed, one of the passengers died, and others were severely injured. In October 1993, the Carmichaels brought this diversity suit against the tire's maker and its distributor, whom we refer to collectively as Kumho Tire, claiming that the tire was defective. The plaintiffs rested their case in significant part upon deposition testimony provided by an expert in tire-failure analysis, Dennis Carlson, Jr., who intended to testify in support of their conclusion.

Before an expert witness such as Carlson is allowed to testify before the jury, the judge will have a private hearing with the attorneys to decide if the expert's testimony should be allowed. The judge must first determine if the evidence will assist the jury in deciding the case. If it will, then the judge must determine if the evidence is so prejudicial that it improperly biases the jury. For example, a prosecutor in a murder case may want to show the jury in a criminal trial gruesome pictures of the victim, ostensibly to help them understand the way the crime was committed. The judge first decides if the pictures provide useful information. If they do, the judge may still refuse to admit them because the added information they provide will be outweighed by their tendency to make the jury hate the defendant.

While most scientific evidence is not so shocking as gruesome crime scene photos, its very appearance of neutrality and cool detachment makes it potentially much more prejudicial. Juries are, at best, a cross-section of the general population in a country with endemic scientific illiteracy. Individual jurors with no back-ground in science and mathematics cannot independently evaluate the credibility of scientific testimony.

They can only determine which expert tells the best story, or whose testimony best supports their wish to deny an unsympathetic plaintiff or punish a corporate villain. The judge bears a heavy burden when vetting expert testimony for such an audience: if the plaintiff's expert is not permitted to testify, the plaintiff may not meet the legal standard of proof and his/her case may be dismissed before trial. If the plaintiff's expert is allowed to testify, the jury will give weight to his/her testimony. In some cases, the judge effectively decides the outcome when he/she determines which experts can testify.

Carlson was well qualified by traditional standards to be an expert in this case. He had a master's degree in mechanical engineering, 10 years' work at Michelin America, Inc., and had given testimony as a tire-failure consultant in other tort cases. He used his own visual methods to examine tires and explain why they failed. Based on his examination of the tire that failed, he determined that it blew out because of a tread separation caused by defective construction. The defendants contested this conclusion because they found evidence of misuse and general wear and tear that could account for the condition of the tire.

In reviewing Carlson's proposed testimony, the court accepted that visual inspections could be a valid method of evaluating tire failures. It then used the Daubert test to determine if Carlson's methods were reliable. The court found that Carlson's methods had not been published or subjected to peer review, since they were not generally known it was impossible to say if they were generally accepted, and that Carlson did not have any studies or statistics to show how often they were correct. More fundamentally, the trial court judge was disturbed by the fact that Carlson's analysis of the data he obtained from his visual inspections was unreliable. For example, Carlson testified precisely that in the absence of at least two of four specific signs of abuse, he is able to conclude that a defect caused the tire to fail. Despite evidence of all four signs of abuse, Carlson testified that he could disregard the very data he observed, assume the evidence of abuse did not indicate abuse, and conclude that the tire failed due to a defect. In addition, Carlson could not tell if the tire had been driven 5000 or 50,000 miles, he ignored two improperly patched punctures in his analysis, and he disagreed with the observations of other experts on objective findings such as the degree and pattern of tread wear.

Ignoring Carlson's traditional qualifications as an expert, the trial judge ruled that he could not testify. Since the plaintiffs could not prove their claims without this testimony, the court dismissed their case. The plaintiffs appealed, claiming that the trial court erred in applying the Daubert test to nonscientific testimony. The court of appeals reversed, finding: "the Supreme Court in Daubert explicitly limited its holding to cover only the 'scientific context,'" adding that 'a Daubert analysis' applies only where an expert relies 'on the application of scientific principles, ' rather than 'on skill- or experience-based observation.'" It concluded that Carlson's testimony, which it viewed as relying on experience, "falls outside the scope of Daubert," that "the district court erred as a matter of law by applying Daubert in this case," and that the case "must be remanded for further (non-Daubert-type) consideration under Rule 702."

The Supreme Court's Opinion

The United States Supreme Court accepted the case to resolve whether the Daubert test applied to testimony based on technical or other specialized knowledge. The court first looked at Federal Rule of Evidence 702, which is the standard for the admission of all expert testimony:

"If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise."

The rule makes no distinction between scientific, technical, and other specialized knowledge. More fundamentally, the Supreme Court found that such a distinction would be impossible to enforce:

"Finally, it would prove difficult, if not impossible, for judges to administer evidentiary rules under which a gatekeeping obligation depended upon a distinction between 'scientific' knowledge and 'technical' or 'other specialized' knowledge. There is no clear line that divides the one from the others. Disciplines such as engineering rest upon scientific knowledge. Pure scientific theory itself may depend for its development upon observation and properly engineered machinery. And conceptual efforts to distinguish the two are unlikely to produce clear legal lines capable of application in particular cases."

The Court explained how it intended the Daubert standard to be applied. Its own words (citations omitted) are the best exposition of its position:

"The petitioners ask more specifically whether a trial judge determining the 'admissibility of an engineering expert's testimony' may consider several more specific factors that Daubert said might 'bear on' a judge's gate-keeping determination. Emphasizing the word 'may' in the question, we answer that question yes.

"Engineering testimony rests upon scientific foundations, the reliability of which will be at issue in some cases. In other cases, the relevant reliability concerns may focus upon personal knowledge or experience. As the Solicitor General points out, there are many different kinds of experts, and many different kinds of expertise. See Brief for United States as Amicus Curiae 18-19, and n. 5 (citing cases involving experts in drug terms, handwriting analysis, criminal modus operandi, land valuation, agricultural practices, railroad procedures, attorney's fee valuation, and others). Our emphasis on the word 'may' thus reflects Daubert's description of the Rule 702 inquiry as 'a flexible one.' Daubert makes clear that the factors it mentions do not constitute a 'definitive checklist or test.' And Daubert adds that the gatekeeping inquiry must be 'tied to the facts' of a particular 'case.' We agree with the Solicitor General that '[t]he factors identified in Daubert may or may not be pertinent in assessing reliability, depending on the nature of the issue, the expert's particular expertise, and the subject of his testimony.' The conclusion, in our view, is that we can neither rule out, nor rule in, for all cases and for all time the applicability of the factors mentioned in Daubert, nor can we now do so for subsets of cases categorized by category of expert or by kind of evidence. Too much depends upon the particular circumstances of the particular case at issue.

"Daubert itself is not to the contrary. It made clear that its list of factors was meant to be helpful, not definitive. Indeed, those factors do not all necessarily apply even in every instance in which the reliability of scientific testimony is challenged. It might not be surprising in a particular case, for example, that a claim made by a scientific witness has never been the subject of peer review, for the particular application at issue may never previously have interested any scientist. Nor, on the other hand, does the presence of Daubert's general acceptance factor help show that an expert's testimony is reliable where the discipline itself lacks reliability, as, for example, do theories grounded in any so-called generally accepted principles of astrology or necromancy.

"At the same time, and contrary to the Court of Appeals' view, some of Daubert's questions can help to evaluate the reliability even of experience-based testimony. In certain cases, it will be appropriate for the trial judge to ask, for example, how often an engineering expert's experience-based methodology has produced erroneous results, or whether such a method is generally accepted in the relevant engineering community. Likewise, it will at times be useful to ask even of a witness whose expertise is based purely on experience, say, a perfume tester able to distinguish among 140 odors at a sniff, whether his preparation is of a kind that others in the field would recognize as acceptable.

"We must therefore disagree with the Eleventh Circuit's holding that a trial judge may ask questions of the sort Daubert mentioned only where an expert 'relies on the application of scientific principles,' but not where an expert relies 'on skill- or experience-based observation.' We do not believe that Rule 702 creates a schematism that segregates expertise by type while mapping certain kinds of questions to certain kinds of experts. Life and the legal cases that it generates are too complex to warrant so definitive a match.

"To say this is not to deny the importance of Daubert's gatekeeping requirement. The objective of that requirement is to ensure the reliability and relevancy of expert testimony. It is to make certain that an expert, whether basing testimony upon professional studies or personal experience, employs in the courtroom the same level of intellectual rigor that characterizes the practice of an expert in the relevant field. Nor do we deny that, as stated in Daubert, the particular questions that it mentioned will often be appropriate for use in determining the reliability of challenged expert testimony. Rather, we conclude that the trial judge must have considerable leeway in deciding in a particular case how to go about determining whether particular expert testimony is reliable. That is to say, a trial court should consider the specific factors identified in Daubert where they are reasonable measures of the reliability of expert testimony."

Conclusion

In Daubert, the United States Supreme Court made an important change in the evaluation of expert testimony. The Daubert standard now clearly extends to all expert witnesses. Nevertheless, breast-implant manufacturers have agreed to a $3 billion-plus settlement for injuries that scientific studies and most mainstream competent scientific testimony say are not caused by silicone breast implants.

This can happen because Daubert depends entirely on the knowledge of the trial judge and that judge's willingness to enforce the Daubert standard. As we pointed out in "Science in the Supreme Court: Round Two," [3] the Supreme Court has also made it nearly impossible to overrule evidentiary decisions by trial court judges. Thus, whether the trial court judge goes though the Daubert analysis correctly or incorrectly, it is nearly impossible to overturn the decision. If the judge incorrectly allows the plaintiff's expert to testify, the defendant must rely on the jury to believe its expert, or appeal the unfavorable outcome at the end of the trial with little hope of success. On the other hand, if the judge incorrectly refuses to allow the plaintiff's expert to testify, the plaintiff is likely to lose without having the jury even hear his/her case. While such plaintiffs can appeal immediately, they also have little hope of success in obtaining a ruling that the judge overstepped his or her discretionary boundaries. Even if most judges follow the letter and spirit of Daubert, there are enough who do not to make the plaintiffs' portfolio of breast-implant cases worth more than $3 billion dollars and defendants' savings from not paying injured plaintiffs a number of dollars probably in the same order of magnitude. Daubert and Kumho are good decisions, but they do not solve the basic problem of trying to resolve complex scientific and technical issues with lay judges and juries.

References

1. Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579, 113 S.Ct. 2786, 125 L.Ed.2d 469 (1993).

2. Kumho Tire Co., Ltd. v. Carmichael, 119 S.Ct. 1167 (U.S. 1999).

3. IEEE Engineering In Medicine and Biology Magazine, vol. 17, no. 2 pp. 124-125, March-April 1998.