Then a miracle occurs: explicating measurement theory to achieve justified validity and facilitate epistemic iteration

At the heart of psychology’s replication crisis and underlying measurement and theory crises, the need for conceptual clarification and more rigorous measurement practices have been identified. These observations align with the persistent plea to bridge the chasm between substantive psychological theory and psychometrics and methodology. We introduce Narrative Response Models (NRMs) as a tool to study the validity of measurement instruments of aspects of the human psyche. In contrast to the commonly employed psychometric approaches that are used from the construct validity theory perspective, studying validity with NRMs allows claims of validity that are based on understanding of how a measurement instrument works. They were designed to be simple to use but epistemologically potent, contributing to conceptual clarification in an epistemic iteration framework. NRMs can be developed using expert knowledge, synthesis of theory and evidence, or inductive approaches. Once developed, NRMs can guide validation efforts using cognitive interviews, response process evaluation, experiments, or computational modeling, in an iterative process comprising adaptation of an item, construct definition, and the NRM connecting both. NRMs answer the call to again align “validity” with the common-sense definition: whether a measurement instrument measures what it was designed to measure.