Article: Is my study useless? Why researchers need methodological review boards

"Should researchers have the freedom to perform research that is a waste of time? Currently, the answer is a resounding ‘yes’. Or at least, no one stops to ask whether there are obvious methodological and statistical flaws in a proposed study that will make it useless from the get-go: a sample size that’s simply too small to test a hypothesis, for example.

In my role as chair of the central ethical review board at Eindhoven University of Technology in the Netherlands, I’ve lost count of the number of times that a board member has remarked that, although we’re not supposed to comment on non-ethical issues, the way a study has been designed means it won’t yield any informative data. And yet we routinely wait until peer review — after the study has been done — to identify flaws that can’t then be corrected.

In my own department at Eindhoven, we’ve been trialling a different approach. Five years ago, we instituted a local review board that also evaluates proposed methods. Although some colleagues found this extra hurdle frustrating at first, the improvements in study quality have led them to accept it. It’s time to make dedicated methodological review boards a standard feature at universities and other research institutions, as institutional review boards are."

https://www.nature.com/articles/d41586-022-04504-8

 

This sounds a bit behind the times. When I was on the UCL ethics review panel in the 1980s we took seriously whether the study would produce useful results if it involved any risk to patients. That would be the case for any drug testing, invasive intervention or even anything that might cause distress.

The whole point of ethics boards is to look at risk benefit. That means you have to assess the likely benefits - which are nil if the design is poor.

 

I would argue that research involving human subjects that is never going to produce meaningful results is inherently unethical:

• Firstly it is wasting subjects time, which is particularly significant when subjects are also patients whose time and energy is a precious limited resource. A central purpose of an ethics committee is surely to safeguard potential human subjects.
• Secondly academic research by respected institutes is assumed by the public to have a useful purpose, so if it is the case that a research project is never going to produce meaningful results, unless subjects know this, they can not be said to have given informed consent.

[Cross posted with @Jonathan Edwards above]

 

Going by the 'quality' of a great many of the studies we analyse here you must have been so far ahead of your time back then that few have managed to catch up yet.

Yes.

In addition it wastes resources that could have been spent on better research.

 

I think it is the other way around. Standards of ethics committees have probably fallen with commercialisation of the research ethic.