14.8. Example: Information Criteria and Bayes factors

An example of applying Bayesian methods to perform model comparisons is “Model comparison tests of modified gravity from the Eöt-Wash experiment” by Krishak and Desai. They re-examine the claim in “Hints of Modified Gravity in Cosmos and in the Lab?”” by Perivolaropoulos and Kazantzidis, made using frequentist methods, that there is evidence in the data of the Eöt-Wash experiment that looks for modifications of Newton’s Law of Gravity on sub-millimeter scales for a residual spatially oscillating signal in the data. This could point to a modification of general relativity (in particular, of some type of nonlocal gravity theory) or it could be due to statistical or systematic uncertainties in the data.

The experiment under consideration is a modern version of the classic torsion balance experiments to measure the force due to gravity. In particular, it is sensitive to departures from Newtonian gravity at sub-millimeter scales. The data analysis from the experimenters do not indicate signs of new physics, but the re-analysis by Perivolaropoulos and Kazantzidis claims that the residual data has signatures of an oscillating signal.

From https://www.npl.washington.edu/eotwash/inverse-square-law: “Below is a cartoon of one of our initial experimental devices which illustrates the technique we employ to measure gravity at short length scales. The pendulum ring, with 10 small holes bored into it, hangs by a thin tungsten fiber (typically 20 microns thick) and acts as the detector mass. The rotating plate just below it, with 10 similar holes bored into it, acts as the drive mass providing gravitational pull on the pendulum that twists it back and forth 10 times for every revolution of the plate. The pendulum twist is measured by shining a laser beam off of a mirror mounted above the ring. The induced twist (torque) on the pendulum at varying separation distances is then compared to a detailed Newtonian calculation of the system. For many of our measurements, the rotating attractor situated just below the detector ring actually consists of 2 disks. The upper disk has holes identical to those in the detector ring. The lower, thicker attractor disk also has a similar hole pattern bored into it, but these larger holes are rotated compared to those in the upper disk, so that they lie halfway between the holes in the upper disk. If the inverse-square law is correct, the lower holes are designed such that they produce a twist on the ring that just cancels the twist induced by the upper disk. However, if gravity changes strength at short distances as the theories suggest, the twist induced by the lower disk, which is farther from the ring, will no longer cancel the twist from the upper disk and we will see a clear twist signal. The actual situation is a bit more complicated; for ordinary gravity the cancellation is exact only for one particular separation between the ring and the attractor, but the variation of the magnitude of the twist with changing separation between the ring and the disks provides a clear ‘signature’ for any new gravitational or other short-range phenomena.”

There are 87 residual torque data points (\(\delta \tau\)) between measured torques and expected Newtonian values. Perivolaropoulos and Kazantzidis made fits to three functions (models):

\[\begin{split}\begin{align} \delta\tau_1(\alpha',m',r) &= \alpha' \qquad \text{offset Newtonian potential} \\ \delta\tau_2(\alpha',m',r) &= \alpha' e^{-m'r} \\ \delta\tau_3(\alpha',m',r) &= \alpha' \cos(m'r + \phi) \end{align}\end{split}\]

Figures of the data and some fits from figures in the two papers are shown here.

Next we have tables from Krishak and Desai showing the best-fit results and the adopted priors.

Krishak and Desai applied AIC, BIC, WAIC, and calculated the Bayes factor (ratio of Bayesian evidences).

Conclusions were based on interpretive scales found in the literature. For example:

\(\Delta\)BIC	Evidence against Model \(i\)
0 − 2	Not Worth More Than A Bare Mention
2 − 6	Positive
6 − 10	Strong
\(> 10\)	Very Strong

For interpreting the Bayes’ factor the Jeffrey’s scale is used: a value above 10 represents strong evidence in favor of the model in the numerator while a value above 100 represents decisive evidence. Krishak and Desai conclude that there is decisive support that the oscillating model with fixed phase (from the Perivolaropoulos and Kazantzidis fit) and strong evidence for varying phase. However, this statistical analysis does not distinguish between a physics origin and statistical effects in the data. Note: Krishak and Desai use nested sampling software to evaluate the evidence.