By Lily Natter

On December 6th, Dr. Bei Yu, Professor at Syracuse University’s School of Information Studies, joined the Luddy School to present her talk, “An NLP Analysis of Causal Language Use in Scientific Communication.” This presentation discussed Project Precheck, which focuses on understanding press release exaggeration in scientific research. The project hopes to address the prevalence of drawing causal conclusions from observational studies, which has raised concerns among researchers and audiences that both scientific papers and news are making exaggerated claims.
Project Precheck works on developing natural language processing (NLP) models that can automatically detect and measure causal language, including correlational, causal, and conditional causal claims. These models are then applied to press releases, so the team can examine the patterns and trends of causal language used by both scientists and journalists. They found that 22% of press releases used causal language, when the original papers the releases were based on only used correlational language. However, since 2010 the overall exaggeration rate has been decreasing.
A question for the future is whether the adoption of large language models (LLMs) like ChatGPT for monitoring causal language in scientific writing will be useful. Dr. Yu has found that fine-tuned NLP models are better suited to recognizing the different types of causal or correlational claims, but LLMs could be used more widely and possibly be helpful to students and non-scientists. However, a wide adoption of LLMs could also standardize the use of causal language among scientists, raising the question: who or what is setting those standards?
Bei Yu is a Professor at the School of Information Studies, Syracuse University. Her research focuses on using machine learning and natural language processing techniques to assist scholarly communication and science communication, especially in monitoring and improving the quality of scientific information. At the 2025 iConference, she will be leading a workshop on AI research, titled “Rethinking the Relationship between Academic and Industry Research on AI: an Interdisciplinary Perspective from iSchools.”