On Oct. 4, Brian Nosek came to Indiana University to discuss the big challenges around improving transparency and reproducibility in research, as well as steps the scientific community can take to shift research culture in that direction.
Nosek, a psychology professor at the University of Virginia and the co-founder and executive director of the Center for Open Science, said his core research interest centers on the gap between values and practices in research.
His work is dedicated to closing that gap by changing the cultural incentives and norms around research to encourage reproducibility and rigor in science. For Nosek, this open, transparent culture will be critical in maximizing the efficiency and pace of science.
Nosek began by introducing sociologist Robert Merton’s norms of scientific ethos which are:
- Communality: ability to review a researcher’s path to end results vs. secrecy in methodology
- Universalism: evaluating research on its own merit vs. evaluating based on researcher’s reputation
- Disinterestedness: motivated by knowledge and discovery vs. science as a competition
- Organized skepticism: considers all new evidence, open to scrutiny vs. considering only evidence that benefits researcher
- Plus one Nosek added: quality vs. quantity of research
In a 2007 study Nosek cited, the vast majority of surveyed researchers endorsed these norms, even as a smaller majority admitted they were practitioners of them. However, the majority of the researchers believed other researchers don’t embrace those norms.
Building a framework
This ingrained belief within the scientific community that people do not participate in the scientific code of conduct is why we need to focus on a broader culture change that incentivizes researchers to align themselves with these ethos, Nosek says.
Open, transparent policies can help–but they must also become deep-rooted within the culture to be embraced.
At the foundation of the journey to a policy change is the infrastructure that makes the practice of creating open, transparent research possible. Working upward, the next steps include creating a user interface to make implementing the practice easy, changing communities to make it normative, creating incentives to make it rewarding, and then, finally, establishing the policy that makes it required.
Nosek and his team are encouraging behavior changes by advocating for journals to stamp articles with “behavior badges” for Open Data, Open Materials, and Preregistration of analysis plans. He’s also changing incentives for research by creating open calls for results that are typically not rewarded, such as negative and null results and replications of previous studies.
To view the Center for Open Science’s top guidelines for better aligning current practices with the ideal scientific norms, visit the Center for Open Science site.
Other resources Nosek shared include:
- OSF: https://osf.io
- TOP Guidelines: https://cos.io/top/
- Preregistration: https://cos.io/prereg/
- Registered reports: https://cos.io/rr/
- Badges for open practice: https://cos.io/our-services/open-science-badges/
- OSF Registration: https://cos.io/our-products/osf-registries/
- Brian’s slides: https://osf.io/95ge7/
View the full talk here: https://youtu.be/RLLoxxtBu50