Inside the experimental causal black box: A heuristic for generating (alternative) explanations in experiments

The pure experiment is considered the gold standard to those pursuing a good test of a causal hypothesis. Because of this status, one might forget that experimental results tend to allow multiple explanations like other designs. But how to identify plausible, alternative causal explanations for experimental findings? In psychology, the seminal work by Campbell et al. extended construct validity theory beyond measures to handle the construct validity of manipulations. The current paper posits that construct validity theory does not sufficiently facillitate the identification of alternative causal explanations and is therefore in need of elaboration. By deconstructing construct validity theory into a set of causally specified assumptions, this paper hopes to show that the development and evaluation of alternative causal hypotheses can be made more rigorous. In particular, this paper posits that changing perspective from validity as a question about “the correctness of a theoretical label” to one about the “causal relationship between instruments (measures and manipulations) and psychological attributes” benefits the interpretation of experimental findings. After describing several considerations about measures and manipulations in a causal language, the paper proposes a simple heuristic for experimental causal inference (HECI). The HECI can serve as a conceptual tool for thought, intended for the basic researcher to interpret experimental evidence and identify alternative explanations.