Statistical and subjective evidence

Are courts of law more receptive to subjective evidence (e.g. witness testimony) than to naked statistical evidence? This is the topic of a recent article [1], selected as one of last year's best philosophical papers. This "Blue Bus" problem seems to have a rather long history in the legal and psychological literature [2,3,4] and is loosely based on a real case ([2], n. 37). Wells [3] gives a particularly clear exposition.

The problem statement: A bus causes some harm, and we know for sure that it necessarily belongs either to the Blue Bus Company or to the Red Bus Company. Should the Blue Bus Company be held liable?

Two scenarios are put forward:

1.  A witness testifies that the bus does indeed belong to the Blue Bus Company, but we have good reason to believe the witness is only 80% accurate.
2. The Blue Bus Company accounts for 80% of the traffic in the relevant area.

In both cases, the probability we can assign to the offending bus belonging to the Blue Company is 80%. Nevertheless, the courts are unlikely to accept the second type of evidence.


To explain the difference between 1. and 2., Enoch et al. introduce a property they call sensitivity and which, as far as I can tell, is simply the lack of false positives:

"[H]ad the relevant proposition been false, you would have not believed it." ([1], p. 204)

Of course, such a property would be highly desirable, especially in court. The authors claim to find sensitivity in the first situation (that of the fallible witness) but not in the second one (naked statistical evidence). The probabilities being exactly the same in the two cases, I would say that the false positive rate is also identical.

I believe the more relevant difference is that of causality: the statistical evidence would exist even in the absence of an accident, while the testimony is a consequence of that specific event (even though the information it provides might be incorrect.) I emphasize that causality strictly concerns here the particular accident, and not the overall rate of accidents, as considered by Wells [3] and Tversky & Kahneman [5].

The same distinction applies in the case of the lottery paradox (also cited by Enoch et al.) Here, the two scenarios are:

1. Before the lottery draw, you own a ticket with a one-in-a-million chance of winning.
2. You own a ticket with a one-in-a-thousand chance of winning, and after the draw the local newspaper reports a number combination different from yours. However, the newspaper is unreliable and (without going into the detail of the probability calculations) your resulting chance of winning is still on in a million.

In the above cases, would you say that you lost? Most people would agree in case 2. but not in case 1., although the probabilities are exactly the same. However, in the second case the newspaper report (however inaccurate) is causally related to the event of interest (the draw), while the statistical information in 1. has no causal relevance.

---

^{[1] D. Enoch, L. Spectre, and T. Fisher, Statistical Evidence, Sensitivity, and the Legal Value of Knowledge, Philosophy & Public Affairs 40(3), 197-224, (2012).↩}

^{[2] L. H. Tribe, Trial by mathematics: precision and ritual in the legal process, Harvard Law Review 84(6), 1329-1393, (1971). ↩}

^{[3] G. L. Wells, Naked Statistical Evidence of Liability: Is Subjective Probability Enough? Journal of Personality and Social Psychology 62(5), 739-752, (1992).↩
[4] R. A Posner, An economic approach to the law of evidence, Stanford Law Review 51, 1477, (1999).↩
[5] A. Tversky and D. Kahneman, Evidential impact of base rates, chapter 10 in: D. Kahneman, P. Slovic, and A. Tversky, Judgment Under Uncertainty: Heuristics and Biases. Cambridge University Press, (1982).↩}