Framework for Assessing Weight of Evidence

Issue: Reports of scientific studies and expert opinion in the lay press are difficult to interpret. What criteria can be used to evaluate the veracity of scientific conclusions and expert opinion?

Background: Evaluating causal criteria that link a stressor with a specified outcome is surprisingly complex. This often involves integrating data from many studies that differ in terms of experimental conditions and in the endpoints that are examined. Many scientific issues are also fraught with conflicting findings making it difficult for even the informed reader to determine what the truth may be. Here we propose a set of criteria that can be used to evaluate the body of knowledge that has been published on a given topic.

The Framework

Trends: In considering claims that factors such as environmental contaminants are involved in an adverse health outcome it is suggested that changes in the prevalence of the health outcome of concern over time should be addressed. Specifically, if it is proposed that environmental contaminants are causing an particular health effect such as breast cancer then it needs to be determined if the number of cases of breast cancer have increased since the chemical was introduced.

Temporality: Since many diseases develop over a period of time it is necessary to consider the relationship between when exposure to the suspect chemical may have occurred and disease detection. Occurrence of the suspected chemical in the environment prior to changes in the disease of interest can be viewed as supporting the causal hypothesis. However, changes in disease frequency that pre-date the introduction of a suspected causative agent offer less credibility to the hypothesis that this chemical causes or contributes to cause of the disease.

Consistency of the data: If environmental contaminants are indeed playing a causal role in certain disease processes then it is expected that scientists working independently of each other would find similar results. Animal experiments examining the effects of a given test compound and following similar methodologies would also be expected to yield similar results. Disparate findings in the literature are an indication that there may be other factors at play than the test compound under study and thus the evidence either in favor of or against a particular hypothesis has to be considered weak and requiring further study.

Biological plausibility: The aspect of biological plausibility examines multiple areas of research that help determine the mechanism of action for the compounds of concern. Consideration of a substance's mechanism of action is critical because this criterion is central to the overall assessment of whether or not a substance is deemed to bean endocrine disruptor.

Moreover, it is essential that the concentration or dose at which the suspect agent is thought to induce adverse health effects should be placed into context of human exposure.

Reversibility: It is proposed that if an environmental contaminant is playing a causal role in a given disease process that elimination of the suspect compound from the environment such that human exposure is decreased then the frequency of the adverse health effect should decline.

Overall strength of evidence: The criteria listed above provide the framework that enables the determination of the overall strength of evidence that a there is a relationship between an outcome of concern and exposure to a substance.