What a Chastened Discipline Can Teach All of Social Science About Open Science
A new article in PS: Political Science & Politics, “Improving Social Science: Lessons from the Open Science Movement,” analyzes psychological science in the aftermath of a “replication crisis” and “credibility revolution” and explicitly examines “what social scientists can learn from this story.” The authors — Per Engzell, a research fellow in sociology at Nuffield College, University of Oxford, and Julia M. Rohrer, personality psychologist and lecturer at the University of Leipzig – note that while their careers have made them “outsiders” to political science and many other disciplines, “What unites us is an interest in meta-scientific questions that has made us wonder how disciplines beyond psychology can benefit from increased transparency.”
Psychology’s own increased transparency followed a series of “high-profile replication failures” that reflected structural problems. “Low power, misuse of significance testing, researcher degrees of freedom, and post hoc hypothesizing had created a cycle in which flashy but spurious results spread with little attempt of falsification.” Problems identified in psychology have been noted in political science, too, including “low computational reproducibility and sanitized research narratives that do not capture the actual complexity of the process.”
Psychology as a discipline responded to the issues with a broad range of remedies, “not always linked to openness,” such as better measurement, greater rigor and stricter significance thresholds.
The article makes several recommendations for improving social science, based on psychology’s experience. Still, the article cautions, attempts to improve the empirical status of a discipline should be localized to that discipline. One size, the authors state, does not fit all. Nonetheless, harnessing tacit knowledge, rewarding contributions to the common good and being inclusive are generalizable as beneficial.
Indeed, the benefits of transparency are large. “Sharing of data and other materials reduces duplicate work and increases the yield from a given dataset, enables pooling of evidence, imposes greater self-scrutiny, and allows others to adapt and build on existing efforts.” This can benefit researchers early on in their careers, as the entry barrier will be lowered as they become less dependent on access to prominent mentors.
Open science’s unifying core, the authors write, is its shared understanding that increased transparency and accessibility can enhance the quality of research and keep scientists’ biases in check. The recent push towards openness is “neither a fad nor an innovation,” but a recognition of “shared interest.”
The report comes to the “uplifting” conclusion that the aims of open science are largely those of the scientific method itself _ “open science is really just science.”
To see the full article, click here.
