Prove It

December 6, 2016

Proof is an elusive thing. As we witness exponential increases in information exposure, our ability to decipher fact from fiction has become exponentially more difficult as well.

The sources of error and our vulnerabilities can be numerous. To err can take many forms, from a lapse in judgment, or a miscalculation to even blatant fabrication. And our ability to decipher each of these suffers from many challenges. Perhaps the most acute or chronic is our hesitancy to question or challenge. And as we become increasingly media saturated, our willingness to expend the energy and resources required to question or challenge the source plummets.

Yet attributing a cause to a witnessed correlation is ubiquitous. A mistake made even more frequently and thoughtlessly as a consequence of “availability bias,” a mental mistake (or heuristic) that derives from the rapidity and ease with which we access recently encountered information, as compared to the labored effort required to consider facts that may have been learned or encountered some time ago.

Enter the tech giants. Both Google and IBM have developed technology aimed at filling the “prove it” void through computation and analytics. In October – during the highly-charged presidential debate – Google launched a new fact check feature to provide additional credentials to items displayed in Google News searches. Google News determines whether an article may contain fact checks by looking for sites that follow the commonly accepted criteria for fact checks. IBM Watson also launched a team developing a product called Watson Angles, which seeks to build attribution histories from which the origin and evolution of information can be visualized, processed and interpreted.

Proof in mathematics or philosophy principally relies on deductive reasoning. If I have a variable Y in the context of a function F(x) then one can demonstrate the repetitive and predicative correlation of both variables in numerous setting, thus inferring the accuracy of the “proof.” In infectious diseases, we lean on the Koch Postulates – an established criteria to identify the causative agent of disease. To be deemed causative, a microbe has to be harvested from an infected host, cultured in the lab, transferred into a naïve recipient, who then comes down with the infection. In medicine, we largely depend on randomized placebo-controlled clinical trials. But what about all the other information surrounding us – how might we hope to validate its accuracy?

And can even the original source data be trusted? Science relies on corroboration, namely can someone else confirm the results or, said another way, are the experiments “reproducible.” But reproducibility is challenging even in the hallowed halls of science. In a survey conducted by Nature Magazine earlier this year, 52% of those asked, believed that we have a “crisis” in reproducibility of data published even in the most distinguished (and selective) journals.

So where does this all take us? Not as far as we’d like, but perhaps most importantly it should remind us that so much depends on our interest and ability to remain skeptical, to ask “just how do we know?” And to seek to explore the evidence presented in carefully considered ways. Perhaps to even ask questions such as these:

Site – is the item that we believe causes a particular outcome definitely present or near the location of the outcome?

Time – is the item that we believe causes a particular outcome definitely present at the location at the appropriate and sufficient time?

Deletion – if the item that we believe causes a particular outcome is removed or prevented from arriving at the location does it negate the outcome?

Transfer – can we take the item from its original location and place it in a new location and see evidence of the particular outcome?

Quantification – is the amount of the item sufficient or related to the scale of the particular outcome?

Reproducibility – can independent researchers who study the item also observe the same or similar particular outcome?

If most of these criteria can be met we should begin to feel more confident. Yet, we must always remain conscious that a mind once made (even if wrong) is extremely difficult to change. Our only antidote is to avoid “believing” too soon.