We are a group of interdisciplinary metascience researchers based at the University of Melbourne.
The “meta” in MetaMelb stands for either metascience or meta-research. The terms are commonly used interchangeably, and as an interdisciplinary research group we happily embrace this plurality. What we do doesn’t change.
Dr Rose O'Dea is a Mary Lugton Postdoctoral Research Fellow at the University of Melbourne. Her research includes behavioural ecology, meta-analysis, and meta-science, and she co-founded the Society for Open, Reliable, and Transparent Ecology and Evolutionary Biology (SORTEE).
Dr Martin Bush is a Senior Research Fellow in the School of Historical and Philosophical Studies at the University of Melbourne. He has expertise in popular science and public reasoning practices and professional experience in science communication and the museum sector. Martin led the Reasoning Analysis team on the repliCATS project.
Dr David P. Wilkinson is a Research Fellow School of Agriculture, Food and Ecosystem Sciences. He was the team leader for the Data Management and Analysis team as part of the repliCATS project for the collaborative assessment of trustworthy science.
Dr Fallon Mody is a Research Fellow in the School of Historical and Philosophical Studies. Fallon’s research interests include science communications, qualitative analysis in mixed methods research, public engagement in science, and history of medicine. Fallon led recruitment and community management on the repliCATS project.
Beth Clarke a PhD student at the University of Melbourne doing metaresearch primarily in social and personality psychology. She is particularly interested in how science generates and disseminates knowledge. Her PhD research focuses on the claims psychologists make in their published articles, and the statistical evidence they report to support their claims. Beth also studies how research practices have changed in psychology over the course of the replication crisis.
Elliot Gould is a PhD student in BioSciences. Elliot is studying reproducibility and transparency of decisions in ecology and conservation. Elliot’s research interests include data science, decision analysis, ecological modelling. Elliot has been actively involved in developing new ‘adaptive preregistration’ templates for field work and ecological modelling.
Steve Kambouris is a PhD student studying reproducibility, open science and meta-analysis. He has a background in physics, psychometrics, education, and philosophy of science, amongst other things.
Daniel Hamilton is a PhD student exploring data sharing policies, practices and perceptions in oncology. Daniel originally trained as a radiation therapist at Epworth hospital in Melbourne, working both clinically and in a research support role between 2012 and 2017. Following his position at Epworth hospital Daniel worked as a research coordinator at Peter MacCallum Cancer Centre managing a large portfolio of national and international radiation oncology clinical trials. He is the lead author on multiple papers investigating novel radiotherapy treatment techniques for prostate and breast cancer, as well as papers examining ethical issues in scientific publishing.
Hannah Fraser is an ecologist studying how current ecological research practices relate to the rigor and reproducibility of results. Her research interests include QRPs, replication, preregistration, registered reports and improving scientific practice.
Dr Alexandra de Gendre is the 2022 McKenzie Postdoctoral Research Fellow hosted at the Department of Economics at the Faculty of Business and Economics and an economist with expertise in labor economics and applied microeconomics.
Dr Nicolas Salamanca Acosta is the Senior Research Fellow at the Melbourne Institute: Applied Economics & Social Research of The University of Melbourne with expertise in applied microeconomics.
Prof Peter Vesk is a quantitative ecologist who integrates fundamental and applied research, with expertise in plants trait analysis and conservation, management, and landscape restoration.
A/Prof Anca Hanea is an applied probabilist based at the Centre of Excellence for Biosecurity Risk Analysis (CEBRA) at the University of Melbourne with expertise in mathematics and risk and environmental modelling.
Prof Mike McGuckin is Deputy Dean and Director Research Strategy in the Faculty of Medicine, Dentistry and Health Sciences at the University of Melbourne. He is the author of over 170 scientific publications focused on mucosal infection, chronic inflammation and cancer in the gastrointestinal tract, and has held four patents.
Prof Michael (Mick) McCarthy is a Professor of quantitative ecology, and Deputy Dean of the Faculty of Science at the University of Melbourne. He builds and evaluates models that assist environmental management. He likes combining mathematics with ecology, and has interests in statistical ecology, including the way tests of statistical inferences are represented (or misrepresented) in textbooks.
Prof Mark Burgman is the Editor-in-Chief of the journal Conservation Biology, and was previously Adrienne Clarke Chair of Botany. (He is now a principal honorary fellow.) He works on expert judgement, ecological modelling, conservation biology and risk assessment, and has a long-standing interest in the reporting of inference and uncertainty in published research.
Dr Bonnie Wintle is a Senior Research Fellow in ecology. Bonnie develops structured methods for eliciting and aggregating quantitative and qualitative judgements from groups of experts, to support better decision and policy making. Bonnie’s also interested in expertise, uncertainty and prediction.
|Aurélien Allard is a postdoctoral research fellow at UC Davis. He completed a PhD in political philosophy at Paris 8 University. His research interests include metascience, philosophy of science, moral philosophy and moral psychology.||Julia Bottesini is a PhD student at UC Davis studying metascience, research practices, and quantitative psychology. Her current projects examine how research stakeholders, including study participants and science journalists, view scientists’ research practices.||Andy Head is a research assistant. He recently completed a graduate diploma of psychology at Deakin University and plans to start a PhD in 2020. Andy’s research interests include methodology, improving science practices, and public engagement with science.|
Aaron Wilcox recently completed a psychological science degree. Aaron also works as a research assistant at Deakin University under Emma Sciberras. Aaron is an R acolyte and an open science advocate and will be continuing studies into fourth year and beyond.
John Maaga is an undergrad psychology major, with the ultimate goal of becoming a psychiatrist. In his first stats class in community college, he became fascinated by the complexity of research methods. John hopes to influence the use of better research practices, especially as it relates to medical and psychological treatments.
Felix Singleton Thorn recently completed a PhD in psychology. Felix’s research examines how people plan, report and interpret the results of psychological experiments, with the goal of improving the accuracy of inferences and providing practical heuristics to guide the interpretation of results.
Our group is working on a large number of projects organised around the following topic areas.
1. Research quality
What are the dimensions we should consider when evaluating research quality? How do researchers evaluate the quality of each other's work, and how do they come to those evaluations? Can we improve the evaluation of research quality by introducing rubrics, group discussions, or other structured processes?
2. Journal policies and practices
What are common journal policies and informal practices around peer review and publication? How does peer review work and how can we improve it? How is the published literature changing over time?
3. Statistical inference
What are common errors in statistical inference, and how common are they? How can we improve statistical training and practice? What are the best tools for detecting and correcting errors in statistical inference?
What are the self-correcting mechanisms in science? How common and effective is post-publication critique? Do replications help to correct the scientific record? Who should do the work of verifying and correcting published studies, and how should it be incentivised?
5. Intellectual humility
Do research claims match the evidence presented? How common are hype and spin? How can we incentivise researchers to make calibrated claims?
6. Transparency and Reproducibility
Does increased transparency lead to better quality research, or better detection of errors and inaccuracies in research? What kinds of transparency are most important, and how common are they? How can important forms of transparency be incentivised? To what extent and under what conditions does transparency improve reproducibility.
The repliCATS project aims to crowdsource predictions about the reliability of 3000 published research claims in the social and behavioural sciences. The “CATS” in repliCATS stands for Collaborative Assessment for Trustworthy Science.
The best way to keep up to date with our publications is to look at our individual google scholar pages. Here are a couple of recent papers:
- Where are the self-correcting mechanisms in science? S Vazire, AO Holcombe; Review of General Psychology 26 (2), 212-223
- Credibility beyond replicability: Improving the four validities in psychological science S Vazire, SR Schiavone, JG Bottesini; Current Directions in Psychological Science 31 (2), 162-168
Publications associated with the repliCATS project:
- SCORE collaborative“Systematizing Confidence in Open Research and Evidence (SCORE)”
- Fraser et al.,“Predicting reliability through structured expert elicitation with repliCATS (Collaborative Assessments for Trustworthy Science)”
- Gould et al.,“aggreCAT: An R Package for Mathematically Aggregating Expert Judgments”
- Hanea et al.,“Mathematically aggregating experts’ predictions of possible futures”
- Pearson et al (2021),“Eliciting group judgements about replicability: a technical implementation of the IDEA Protocol”
- Wintle et al.,“Predicting and reasoning about replicability using structured groups”