Sales vs. Science: Impact Evaluation at a crossroads

The nonprofit sector has recently undergone an impact revolution. Unfortunately, as Fiennes and Berger (2016) note, the impact revolution has “gone wrong.” A perverse set of incentives and disincentives has created an industry that performs evaluations, but doesn’t generate impact. We believe that the ongoing “replication crisis” centered in academic social psychology may shed light on the causes, consequences, and remedies of this failure. We identify parallels between the histories of social psychology and impact evaluation, including the use of research as sales, the pressurized environments in which the research takes place, and the use of research methods as rhetorical devices. We argue that these parallels reflect the perverse incentive structures operating in both fields that undermine evidence quality and limit learning. This session explores what we can learn from the replication crisis, what solutions can be effective, and what role can evaluators play to create a credibility renaissance.
Click to fill survey.