Should we throw out common sense?

During the course of the year I have heard that common sense can be deceiving with respect to probabilities and statistics. In fact, the theological advisor of the film assured many that “statistics are counterintuitive”.

The best known epigraphers and archaeologists who have worked on tomb inscriptions were confidently unanimous. They all said that the names on the Talpiot ossuaries were simply too common to link these ossuaries with any known historical figure. The response of one such scholar was “Lies, Damned Lies and Statistics”.

A year ago, Professor Andrey Feuerverger, a mathematician from the University of Toronto, told the world at a press conference that, based upon the assumptions that were given to him, the probability that the ossuaries of the Talpiot tomb were those of Jesus and his family was a 600 to 1 likelihood.

Now, a year later, the official publication of Prof. Feuerverger’s continued research on the topic has been published and is available to the world to see. It is called “Statistical Analysis of an Archaeological Find–A Bayesian View”. He now says that, based upon a priori assumptions , the probability should be raised to a “1600 to 1 probability” that the ossuaries of the Talpiot tomb were those of Jesus and his family.

Several statisticians were provided with copies of Feuerverger’s paper for review before its publication. The paper and the accompanying reviews are now available together on the web site of the Annals of Applied Statistics. Most of the reviewers, while offering constructive criticisms of the work, applauded the paper’s insightful advancements in the application of statistics to archaeological finds. Others were less generous in their final assessment, including Camil Fuchs and Don Bentley. (A decent summary has been provided here by Christopher Heard at the site Higgaion).

One thing keeps passing the non-statisticians by almost without notice. Feuerverger and his reviewers stress that the conclusions of their work are only valid if the a priori assumptions that were provided to them were true. In other words, if the premises of the filmmakers were faulty, if the assumptions were untrue, then the statistical probabilities are also false. Don Bentley puts it quite simply: “we can only accept the conclusion if we are willing to accept the assumptions.”

Let’s try some other a priori assumptions:”If the Moon were made of cheese . . . . ”

“If the cow can jump over the moon . . . ”

“Assuming that money can grow on trees . . .”

Assuming these things are true then the conclusion based upon these premises must be . . . . A) True or B) False.

******************

Now how about these premises (which are actually true):

What if we found a first century tomb with names on ossuaries “Jesus(?) son of Joseph”, “Joseh”, and two Marys . . .

And . . .

1) add to that: the unrelated names Mara, Mattiah and Judas son of Jesus also appeared in the tomb . . .

2) Also, these names represent less than 1/4 of all of the skeletons that were once in the tomb.

3) Also, add the fact that one of every five women among all ossuaries were named “Mary” (or some form of that name, this is actually true) . . . .

4) And at the same time, almost one in every twenty people (male or female) were named “Jesus”

5) Add to that: nearly one in every ten people (male or female) were named ”Joseph”

6) Also: one in every ten tombs should statistically have another ‘Jesus son of Joseph’ . . . .

Assuming these things are true, this really means that the tomb Jesus of Nazareth and his family has been discovered. True or False

Unfortunately, most people listening to the statisticians believe that they are giving statistics that have a bearing on real history. However, that is not true at all. On the contrary, if we are listening to them carefully we will find that they are only speaking hypothetically. Their premises, which form the basis for their statistics, are only a priori assumptions. The statistician must reserve his judgment as to the veracity of the assumptions which others have given him. He leaves this for others who are more qualified to judge. These same statisticians admit that if they would be given other assumptions, then they would have deduced totally different statistical probabilities.

Prof. Camil Fuchs of Tel Aviv University, after challenging the apparent positive and not the expected dispassionate view toward the a priori assumptions, goes on to conclude:

“Feuerverger was quoted as saying that ‘I did permit the number one in 600 to be used in the film. I’m prepared to stand behind that but on the understanding that these numbers were calculated based on assumptions that I was asked to use’, a statement far removed from the rigorous demand of a-priori assumptions. (In his webpage, Feuerverger (2007) mentions that the quotations in the interview are ‘sufficiently accurate to be considered fair’).

“In spite of the fact that in my opinion, the analysis of the “surprisingness” based on the configuration of names failed to yield the stated conclusions, I refrain in this article from passing judgment on the subject matter issue of whether or not this is the tombsite of the NT family.

“Furthermore, notwithstanding the reservations from the analyses applied to the discussed data, I applaud the bold initiative taken in the discussed paper to develop a new approach to tackle a problem characterized by a degree of complexity that precludes the straightforward application of the classical hypothesis framework. The general problem of rendering judgment on whether a multiple characteristics observation represents the pursued specific entity or is just the result from random draws is interesting and intriguing. Cases of disputed paternity and DNA matching come to mind in this context. Unlike the Talpiot case, in those cases a standard for comparison is available. The new approach and concepts of ‘surprisingness’, ‘relevance’ and ‘rareness’ may evolve and prove beneficial in cases in which no such standard exists.

“Classical methods, usually based on Bayesian analysis, are available for those cases, but their application may be difficult in complex situations. If the new approach is to be applied, its performance needs to be compared to existing methods in situations in which it is known whether the null hypothesis (or the analogous null hypothesis) is correct. I think that the features of the approach still need to be investigated theoretically or by simulations under various conditions of complexity. In any case, the assumptions have to be pre-specified to ensure valid results and a valid comparison.”

In all fairness, Prof. Feuerverger himself pronounced at the symposium:

“Without Mary Magdalene, the tomb is like any other tomb with an unremarkable common set of names.”

Andrey Feuerverger, Mathematician, University of Toronto