HARRISBURG, PA — A large-scale international study published in Nature has found that scientific conclusions can shift dramatically depending on who analyzes the data. Harrisburg University of Science and Technology (HU) Professor Erin Buchanan, Ph.D., was among the researchers who helped make that finding possible.
The study, “Estimating the Analytic Robustness of Social and Behavioural Sciences,” brought together 457 independent analysts from institutions around the world to reanalyze data from 100 previously published studies in the social and behavioral sciences. Each analyst received the same dataset and the same research question but was free to make their own decisions about how to clean, model, and interpret the data.
The results were striking: analysts frequently arrived at different conclusions from the same underlying data. All analysts agreed with the original study’s findings in only about one third of cases – not because of errors or lack of expertise, but because of the many legitimate choices researchers make along the way, from how variables are defined to which statistical models are applied.
“This project reinforced something I think we need to talk about more openly in the sciences: that the path from data to conclusion is rarely a straight line,” said Dr. Buchanan, Professor of Cognitive Analytics at Harrisburg University. “Every analyst brings informed judgment to bear on decisions that can meaningfully shape results. That’s not a flaw in the process; it’s a feature we need to account for transparently.”
The study was led by Balázs Aczél and Barnabás Szászi of Eötvös Loránd University and Corvinus University as part of the Systematizing Confidence in Open Research and Evidence (SCORE) program. The researchers found that observational studies were less robust than experimental ones, suggesting that more complex data structures create more room for analytical variability. Experienced researchers with strong statistical backgrounds were just as likely to reach divergent results as others.
Rather than undermining confidence in existing research, the study’s authors say the findings point toward a constructive path forward: broader adoption of multi-analyst and “multiverse” approaches that make the full range of plausible results visible, especially for questions of high scientific or societal importance.
Dr. Buchanan’s participation in the collaboration reflects Harrisburg University’s continued engagement in research that advances scientific rigor and transparency. Her contribution adds to HU’s growing presence in national and international research efforts. The full study is available in Nature.
# # #
ABOUT HARRISBURG UNIVERSITY
Harrisburg University of Science and Technology (HU) is an independent, nonprofit university offering degrees in advanced manufacturing, engineering, robotics, nursing, cybersecurity, and other critical fields. Accredited by the Middle States Commission on Higher Education, HU serves a diverse student body through bachelor’s, master’s, and doctoral programs that link learning and research with practical applications. For information about HU’s affordable STEM degrees and professional development programs, call 717.901.5146 or email Connect@HarrisburgU.edu. Stay in the know by following Harrisburg University on LinkedIn, Instagram, and Facebook.
MEDIA CONTACT
Do you have questions about this story? Interested in lining up an interview? Please contact Dan Wilhelm, Communications Manager for Harrisburg University, at DWilhelm@HarrisburgU.edu or 717.901.5100×1724.
# # #
