COMPare Trials, Ben Goldacre et al

But they reported that they changed it - so that's "OK".

 

Great band name...

 

Hello there. I came across this excellent comment (ignored of course) on Goldacre's COMpare blog yesterday. I wondered if G.Y. was on this forum...does anyone know?

https://www.compare-trials.org/blog/jama-reject-all-correction-letters/#comment-475

 

Exactly!! They would probably not have picked up SMILE either, as the swap was "explained" by Crawley as to do with the feasibility and the burden on participants. Ben and co. would have given that the all clear according to the FAQs https://www.compare-trials.org/faq. I don;t think this is made explicit in the protocol, rather ironically https://www.compare-trials.org/wp-content/uploads/2016/01/COMPare-Protocol-18.5.2016.pdf. I might be wrong about that, but if it's in the FAQs, it's probably not clear enough in the protocol.

I am thinking of proposing a study called "Beyond COMPare" (!!) to look at the same studies for other characteristics of the trials which will indicate whether the trials were done correctly in terms of the factors @Lucibee mentions, rather than if they were reported correctly. As we all know, and as @Lucibee says, you cannot assume reporting a trial correctly (according to CONSORT) means it is a well designed and well conducted trial. But people do that. In the same way, people assume that a Cochrane review is well-designed and well-conducted because Cochrane reviewers have to follow the Cochrane reporting process.

 

That sounds like an excellent idea!

 

Agree. That one of course seems remediable. I asked a UK academic friend of mine regarding research methods, reliability etc at her place (given there isn't a quality assessment, certainly outside REF times of year) and she replied that it is a worry but her place did have a process of internal peer review with 2 people reading the work etc.

I think that we can all see the potential for flaws. Given fixed term contracts and way academic careers work simply making this external wouldn't fix it. Competition is something that works incredibly effectively in academia though (despite many saying they hate it that is what drives the place)

Switch to having them reviewed externally, blindly (as students' papers are) with a league table for methodological requirements. Have other depts cross-review different subjects to avoid individual areas getting 'caught in a circle'.

Prizes that are based on work that has good strong design (internal prizes as well as external ones as you have book prizes and other types). Given the 'voices from the shadows' videos would Wellcome be a good place to begin discussions? Maybe Microsoft/the Gates foundation (whichever arm is relevant)? Make good design and proper methodology ie research quality the 'next big thing'

A badge worth having (or prize), particularly if it targets mid-career and early-career academics could nudge quite a lot if the format and types of prize (ie include school-level where portfolios representative of overall quality be submitted etc)

e.g. Require a summary section next to the abstract succinctly diagramming and defining the design chosen and methodology based on the objectives. Require results tables of primary. ie layout that makes it obvious if something is 'of quality' or pre-that time'. Other bug-bears made disqualifying etc.