University College London launches open access megajournal to help solve the world’s biggest challenges

[Alvin]

I think there is a difference with sci-hub - which is simply providing transmission. UCL is providing something much more than that - a guarantee of some form of quality control.

I agree, its not a journal, it just makes subscription locked material available without paying

The recent journal model was a con trick. UCL can reclaim the old model overnight and unhook it from production costs - which are now zero.

This is basically what i was getting at.

That remains costly but strangely the tradition has been that it was never charged for by the people who provided it (peer reviewers). Journals were originally organs of learned societies that were groups of people who agreed to provide each other with quality control free of charge.

This makes it even less likely for remuneration for peer reviewers to happen as it doesn't typically already.
Peer review has many deficiencies and not a guaranteed way to catch flaws, ideally we should have studies replicated by independent groups (but who is going to pay for that) so we stick with what we have available and hope for the best. The problem is when people act in bad faith this can be gamed, as much as we want to believe scientists as a group are dedicated to finding the truth they have beliefs and confirmation bias just the same as the general public. We hope they can transcend them and we hope they operate in good faith but some clearly don't. A few bad apples and we can't easily determine who they are or how many.

My guess would be that this is very likely to roll out within the year. It certainly could do. Then commercial journals will collapse just like the tape recorder industry did when CDs came out.

Very possible, creative destruction takes no prisoners, though one thing i have found is that when the marginal cost of something goes to zero quality can go with it, there may be a lot more data being created at record rates since the internet was created but how much of it is actually valuable. How much of facebook for example is going to improve the human condition? If anyone can publish papers it invites more low quality quick publishing (especially for those who have publishing quotas) and it makes peer review even less valuable because of the expanding volume and it gives even more opening for malicious or biased data to be published and given a veneer of scientific integrity. This is not insurmountable but it is a hazard to guard against.

But the stakes seem to be getting higher, once an idea is out there stopping it can be nigh impossible, vaccines causing autism is an example of scientific and peer review failure, now playing whack a mole...

 

[Jonathan Edwards]

If anyone can publish papers it invites more low quality quick publishing (especially for those who have publishing quotas) and it makes peer review even less valuable because of the expanding volume and it gives even more opening for malicious or biased data to be published and given a veneer of scientific integrity.

This phase has already played out - the UCL initiative is essentially a reaction to that. Anyone should be able to publish. In fact anyone who does any clinically relevant research should be obliged to publish. At the moment nobody notices if badly done work is published. If universities publish their own people's work then they are going to make sure it is done properly, other wis they will lose their reputation.

 

[Alvin]

If universities publish their own people's work then they are going to make sure it is done properly, other wis they will lose their reputation.

That is the conventional wisdom, I don't see Bristol or the Lancet failing by their reputation based on their actions regarding ME/CFS. In fact much of the media is still giving them the false balance and bothsiderism treatment.

 

[Jonathan Edwards]

I don't see Bristol or the Lancet failing by their reputation based on their actions regarding ME/CFS. In fact much of the media is still giving them the false balance and bothsiderism treatment.

That is a reflection of the fact that opinion has not yet shifted. But when it does the Lancet will suffer as much from this as it did from Wakefield. The Wakefield scandal reduced the Lancet from the most reliable journal to one just like any other. The impact was huge. It was one of the reasons I published in NEJM.

 

[Trish]

If universities publish their own people's work then they are going to make sure it is done properly, other wis they will lose their reputation.

I like this idea. If Bristol University had to publish all Crawley's work, or QMUL had to publish PACE, and host the criticism that results in the form of open peer review, they might think again about it.

But that assumes that the peer reviewers do their job impartially. What is to stop them using a few of the BPS crowd as the reviewers, as presumably happens now? Who would select and vet the reviewers? Or would any academic be able to publish post-publication reviews on the same site to counter the effect of biased or incompetent reviewers?

 

[Alvin]

That is a reflection of the fact that opinion has not yet shifted. But when it does the Lancet will suffer as much from this as it did from Wakefield. The Wakefield scandal reduced the Lancet from the most reliable journal to one just like any other. The impact was huge. It was one of the reasons I published in NEJM.

So you need to convince people not with numbers and science (which is already done) but with bludgeons to accept reality. For marginalized populations thats not always possible and when some have more power then others it just doesn't easily happen. For example politics is decided by convincing voters of easy answers and lies, they get elected, they predictably fail, voters choose the opposite, some repair is done then they choose the easy answers and lies once again leading to more failure. Little progress is durably accomplished when people fall for nonsense over and over and over again. I would love to see the day when good science conquers all, but we are not even close to that level of progress.
Unfortunately a percentage of the population consider Wakefield gospel despite the fact it was retracted. The "replication crisis" doesn't seem to have taken out vast numbers of reputations based on its observed percentages
The resources to fight just those unreplicable today would be enormous, if we start having several times as many studies published of lower quality it would take more resources then we have for research to begin with.

 

[Trish]

The resources to fight just those unreplicable today would be enormous, if we start having several times as many studies published of low quality it would take more resources then we have for research to begin with.

I think the point is, these junk studies are being done now. Not all of them get published but the research juggernaut continues, and a whole generation of students are being taught shoddy research methods. So lets bring it all out in the open and expose it for what it is.

Then maybe Universities will start taking a closer look at the junk some of their departments are churning out. It could even mean, for example, that what I call proper scientists start examining the research methods of the psychologists, sociologists etc in their university, and the examining the claims they are making and pointing out the flaws.

 

[Alvin]

I think the point is, these junk studies are being done now.

That is also my point, junk studies are done now and when you lower the bar your going to get more junk studies then we already have. Also if we have to summon far more resources then the study took to get published to get it retracted/dustbinned then thats going to be even harder with more volume of even lower quality and reputation did not prevent what we have now, it won't work better with increased volume and even lower quality

I'm not arguing in favour of closed journals with hefty fees (i agree with the axiom information yearns to be free), i am saying if we are going to make it easier to game the system we need to work much harder (and smarter) to prevent it from happening then we are today. Personally I'd rather try to prevent a bigger problem then react to it after it blows

 

[Trish]

That is also my point, junk studies are done now and when you lower the bar your going to get more junk studies then we already have.

I do understand your point, but I think the junk studies will come under greater scrutiny if they are on the University's own website because the University's reputation will be influenced by the amount of junk they are prepared to host.

Compare this with now, when it seems anyone can set up a journal and publish their friends' junk, and the authors can list it as a publication on their CV without anyone outside their obscure journal ever reading it. I don't think it will generate more junk, I think it may eventually generate less.

A lot of the small underpowered studies are probably generated as the result of Masters and PhD theses. Maybe Universities will start separating these as 'summaries of student research projects' instead of pretending they are on a par with major studies with clinical significance.

But this is all speculation on my part. I refused to to a PhD!

 

[Alvin]

I do understand your point, but I think the junk studies will come under greater scrutiny if they are on the University's own website because the University's reputation will be influenced by the amount of junk they are prepared to host.

Compare this with now, when it seems anyone can set up a journal and publish their friends' junk, and the authors can list it as a publication on their CV without anyone outside their obscure journal ever reading it. I don't think it will generate more junk, I think it may eventually generate less.

Thats a lot of faith.

 

[Jonathan Edwards]

But that assumes that the peer reviewers do their job impartially. What is to stop them using a few of the BPS crowd as the reviewers, as presumably happens now? Who would select and vet the reviewers? Or would any academic be able to publish post-publication reviews on the same site to counter the effect of biased or incompetent reviewers?

There is no need to select and vet reviewers in fact. I would suggest that authors are obliged to upload their study reports to the university webpages and take responsibility for making them look good. At present authors have no particular motive to make sure their manuscripts are free of bad practice because the MS is either accepted (hurray!) or privately send back to be re-done. If authors were obliged to publish all clinical studies in the knowledge that anyone could comment (as happens on PLOS One at present), and without a quality control check before going public I can guarantee that they will get their own peer review done in advance of uploading.

With open peer review, to which anyone can contribute, readers can judge for themselves whether the reviewers comments are fair. Very often at present a paper is rejected because a reviewer called Professor Bonhoff-Karnovsky complains that the paper did not use the Bomhoff-Karnovsky index for measuring fatigue. I had a paper on a B cell mechanism for RA turned down on the grounds that there was a perfectly good T cell theory - from the chap who published the T cell theory.

I don't think any bludgeoning is implied. We get to a situation where everyone has to make up their own mind - which has always been the position of the Royal Society - take nobody's word for it. There is no justification for saying that bad studies should not be published. They should all be published and be seen to be bad. Patients may have taken risks to be involved. So the world should know what happened, especially if the results did not confirm the author's preferred theory. There is not going to be a problem with too many junk data papers. Only a finite number of clinical studies can actually get done and they should all be reported. What I think will happen is that the number of junk review articles will actually reduce because there will be no kudos for authors and no profit for journals.

 

[Alvin]

I don't think any bludgeoning is implied. We get to a situation where everyone has to make up their own mind - which has always been the position of the Royal Society - take nobody's word for it. There is no justification for saying that bad studies should not be published. They should all be published and be seen to be bad.

This is great in theory, but in practice PACE is still not withdrawn and medical organizations and governments have used it to craft policy and treatments and against patients who apply for benefits and even as justification to take children away from their families. The courts had to weigh in to get the data released to get even as far as we have. Bludgeoning has been required
Self publishing makes it easier to release bad data, we cannot assume people who act in bad faith will decide to act in good faith because they can publish more easily. They are convinced they are right, they manipulate data to get the results they want, since they believe lying for the "greater good" is justified making things easier for them will not make them reconsider. I wish it would.
We are obviously not going to agree here, perhaps we should just agree to disagree?

 

[Trish]

They are convinced they are right, they manipulate data to get the results they want, since they believe lying for the "greater good" is justified making things easier for them will not make them reconsider. I wish it would.

If they had to publish the raw data alongside the study, others could reanalyse the data and publish critical reviews. If that had happened with PACE in 2011 that would have been so much better. As Plos is supposed to do, but hasn't with the PACE paper published by Plos.

@Jonathan Edwards, does the UCL plan include anonymised raw data release?

 

[Alvin]

If they had to publish the raw data alongside the study, others could reanalyse the data and publish critical reviews. If that had happened with PACE in 2011 that would have been so much better. As Plos is supposed to do, but hasn't with the PACE paper published by Plos.

Indeed, but if the university who supports the study is in charge of its publishing standards will they make that a requirement? I hope so.

 

[Jonathan Edwards]

If they had to publish the raw data alongside the study, others could reanalyse the data and publish critical reviews. If that had happened with PACE in 2011 that would have been so much better. As Plos is supposed to do, but hasn't with the PACE paper published by Plos.

@Jonathan Edwards, does the UCL plan include anonymised raw data release?

Don't know. They may not have considered that. I suspect it will become the norm.

 

[TiredSam]

Merely further proof of your visionary genius!

Indeed. Also further proof of how influential and widely read S4ME already is

 

[Valentijn]

Self publishing makes it easier to release bad data, we cannot assume people who act in bad faith will decide to act in good faith because they can publish more easily.

Yes, and hopefully that would result in more awareness that the quality of a paper cannot be assumed merely because it is published. Plenty of misleading crap is published currently, but gets a free pass because everyone assumes that only quality work is published, or that peer-review catches and removes all problems.

If those assumptions are dropped, perhaps we would get higher quality reviews by Cochrane and NICE, where the responsible parties really dig into and assess the research instead of parroting the results from the abstracts or even just the titles. Passing that responsibility to the publishing journals has allowed academics to become lazy and skate by without honing the necessary skills or putting in the effort which they should be.

 

[Alvin]

Yes, and hopefully that would result in more awareness that the quality of a paper cannot be assumed merely because it is published. Plenty of misleading crap is published currently, but gets a free pass because everyone assumes that only quality work is published, or that peer-review catches and removes all problems.

If those assumptions are dropped, perhaps we would get higher quality reviews by Cochrane and NICE, where the responsible parties really dig into and assess the research instead of parroting the results from the abstracts or even just the titles. Passing that responsibility to the publishing journals has allowed academics to become lazy and skate by without honing the necessary skills or putting in the effort which they should be.

Am i in the twilight zone?
97% of studies agree that climate change is occurring and the ones that don't were analyzed are typically written by people with connections to fossil fuel companies, have demonstrably obvious errors or rely on illogical interpretations. Your saying that increasing this will make a positive difference, if say conservative think tanks paid tons more money so say 60% of newly published papers claim climate change doesn't exist using lies this will sway public opinion towards supporting climate change mitigation.
I may be sleep deprived but this is preposterous
All this will accomplish is to convince more people that science is lies so they should believe the better grifter with easy answers and alternative facts.

 

[Invisible Woman]

In my own case, I have found myself discussing PACE with docs in the past. You know you go to see someone about something totally different and, wow, they've got an opinion about ME and would much rather discuss that.

At the moment you'll get: but this has been published in reputable, peer-reviewed journals. It's a thing of great beauty. You try to explain the cronyism and they think you're either bonkers or a raving conspiracy theorist.

If it was published as was and all the reviews were open....for a start peer reviewers would be more wary as their own reputations would be on the line. If critiques were open, I could just simply ask why they didn't scroll down the screen and read the comments by reviewers and other scientists? The argument would already be made for me.

Also, based on my time here in the forums, I've noticed that certain people command respect for their insight and quality of posts. Which isn't to say I'll always agree. If people could review or comment on papers publicly, it could certainly affect their own professional careers and reputations. This could not only weed out the cronies, but significantly improve the quality of the peer review.

 

[Trish]

if say conservative think tanks paid tons more money so say 60% of newly published papers claim climate change doesn't exist using lies this will sway public opinion towards supporting climate change mitigation.

But they could do this now - and even set up their own journals to publish it in. The point surely is, would any University be prepared to publish such non scientific stuff? If instead of a free for all of unregulated journals, it became the norm that all research was published by Universities, it might actually improve the situation.

By the way, I think you are absolutely right to raise issues that cause disquiet about this possible change in who controls scientific publication. I'm not disagreeing with you just to be disagreeable.