Article - A solution to psychology’s reproducibility problem just failed its first test May 2019 ScienceMag

[Sly Saint]

Behavior change is difficult—just ask any psychologist. A new study shows behavior change among psychologists is no different. Efforts to improve the robustness of research by asking psychologists to state their methods and goals ahead of time, a process called preregistration, have stumbled at the first hurdle.

“Preregistration is not as easy as it may seem,” says Aline Claesen, a psychologist at the Catholic University of Leuven (KU Leuven) in Belgium. She and her colleagues examined 27 preregistration plans filed by psychologists from February 2015, when the journal Psychological Science started to offer badges for preregistered studies, to November 2017. In every case, her team reports this month in a preprint on the PsyArXiv server, the researchers deviated from their plan—and in every paper but one, they did not fully disclose these deviations.

https://www.sciencemag.org/news/201...ucibility-problem-just-failed-its-first-test?

 

[MEMarge]

And the reviewers did not notice this!

 

[DokaGirl]

So, what are these sought after badges?

Sounds to me like gold stars kids get in school. We never get past this liking for points it seems. Kind of like "Likes" on this forum!

I wonder if researchers who hide or omit findings, think of the ramifications of their work. How it may skew public health policy for instance.

 

[Adrian]

I think the reviewers are unlikely to check against the registered plan unless specifically asked. Reviewers are unpaid and it is part of the overall academic workload so will often be a light weight process unless something feels wrong (but that can just be a rejection).

 

[rvallee]

Preregistration is not as easy as it may seem

Uh, of course it is. But if it's just a suggestion without consequences then of course it's not going to happen.

Like do no harm, if nothing enforces it, then it's just a slogan.

Force it to happen, with consequences, and it will be done. People will be pissed because now they'll have to actually try to respect the scientific method but the consequences so far have been disastrous, with most of them, like what is happening to us, mostly out of sight.