Bristol - New Network - Prof Munafo

I think in these sort of situations a valid goal will have been achieved, possibly reinforced by the follow-up reply @Graham suggests. Even though Professor Munafò is not going to take the action we (rather optimistically) hoped for, he may just be more likely to listen with some interest to any discussions on the subject, and maybe decide to get more involved in the future. It's not always about the big headline wins.
 
Draft 1: there's no point in spending a lot of time or effort into it!

Dear Prof. Munafò

Thank you for your prompt reply, and thank you for reading our briefing document.

Of course I am disappointed in your response, but not surprised. It has been a common response from many, varied professionals here in the UK. Sadly, with only a handful of exceptions, the only researchers who feel qualified to comment are the authors themselves. This puts us at a real disadvantage. How do we shine the spotlight on such work? The first paper was published in 2011: the NICE re-examination of the guidelines for ME/CFS is not due to finish until 2020. That is ten years of needless and great suffering from patients, despite our continued efforts to pursue the scientific line.

I cannot think of any example in society or history where setting a good example has had a real effect in negating inappropriate or plainly wrong behaviour: only tackling those issues head on has ever changed the situation. Can you suggest any way in which we might bring these problems out into the open in such a way that they might be discussed by researchers and medical practitioners? How can we mobilize academia against this bad trial, which is damaging the reputation of UK research? I would be very grateful for some professional advice here. The cynic in me thinks that only by making ME sexually transmissible would make those with power and influence sit up and take notice!

I wish you well with your network, but would still like you, as you organize its structure, to consider your approach to poor quality work, particularly when it has real influence. The media are very poor at sorting the wheat from the chaff, and the Science Media Centre clearly is more concerned with reflecting the views of its sponsors than in applying appropriate standards.

Thank you, and good luck

Graham
 
You know me, @Graham - I'd shorten it and focus it tightly on your 'ask' (and get your ask in early as well as at the punchline so he doesn't just think you're writing to moan at him, which might mean he doesn't read the rest):

Dear Prof. Munafò

Thank you for your prompt reply, and thank you for reading our briefing document.

Of course I am disappointed in your response, but not surprised. But perhaps you can advise us? Sadly, with very few exceptions, the only researchers who feel qualified to comment are the authors themselves. This puts us at a real disadvantage. How do we shine the spotlight on such work? The first paper was published in 2011: the NICE re-examination of the guidelines for ME/CFS is not due to finish until 2020. That is ten years of needless and great suffering from patients, despite our continued efforts to pursue the scientific line.

Can you suggest how we might mobilize academia - either as individuals or collectively - against this bad trial, which is damaging the health of patients and the reputation of UK research? What would you do in our position?

I wish you well with your network, but would still like you, as you organize its structure, to consider your approach to poor quality work, particularly when it has real influence. Fixing the future is only half the job while bad past studies inflict such damage.

Thank you, and good luck

Graham​
 
Last edited:
Graham said:
I think the root causes of problems of reproducibility lie in incentive structures and research cultures.
Click to expand...

Right. Because the conflicts of interest & paychecks that came from DWP and insurers makes no difference?

The research culture where it's okay to mark your own homework, yet still claim to be independent makes no difference?

The culture where people are afraid to speak out, or even simply question, isn't part of the issue.

A high level view is all very well, but there's no point having a stylish, statement roof when your foundations are rotten and crumbling.
 
Sasha said:
Can you suggest how we might mobilize academia - either as individuals or collectively - against this bad trial, which is damaging the health of patients and the reputation of UK research? What would you do in our position?
Click to expand...

Is it worth reiterating that some lecturers or universities in the US are now using a famous piece of British research to demonstrate how such research should not be done?
 
Sasha said:
Sadly, with very few exceptions, the only researchers who feel qualified to comment are the authors themselves.
Click to expand...

As you ( @Graham ) pointed out in your first e-mail (or letter?) there were and are comments - or don't you think the mentioned critics are researchers?

My suggestion would be rather something like this (apologies for the bad wording):

"Sadly, despite the scientifically sound criticisms of the PACE trial and related studies there continue to be studies replicating the same and worse errors. As I pointed out in my previous e-mail this is also happening at the University of Bristol. When the University is not interested in the bad science they are producing, how do we shine the spotlight on such work?"

[edited to replace "authors" by "critics"]​
 
Last edited:
MSEsperanza said:
As you ( @Graham ) pointed out in your first e-mail (or letter?) there were and are comments - or don't you think the mentioned authors are researchers?
Click to expand...
That's why I said "With only a handful of exceptions". I am always grateful to those who stood up and made comments, but as a proportion of related researchers etc. it is an absolutely tiny number, given how bad and yet how influential the studies are.

I like Sasha's rewording, and also like MSesp's suggestion of mentioning the fact that Bristol needs to put its own house in order. I don't think though that I will widen it out as Invis. suggests, despite me agreeing fully with her, simply because it would make the email too long and tangled.
 
Graham said:
My initial thought is just to reply briefly, thanking him for his response, and just to point out that a reluctance to get involved in specific debates lies at the heart of the problem in the UK, and allows third-rate work to not only prosper, but to have influence.
Click to expand...

Yeah - a vital problem with the incentive structures and systems within academia is that they allow researchers to produce junk science and then their colleagues try to be polite about it. That's partly a result of the decisions made as individuals by their colleagues. The way some people try to emphasise the way that privileged researchers who build their career on junk science that harms other people are victims of a flawed system is a bit irritating.

Sasha's approach is more tactful, and probably more likely to have a short-term benefit, but I do feel that there's some value in speaking truth to power.
 
Esther12 said:
Sasha's approach is more tactful, and probably more likely to have a short-term benefit, but I do feel that there's some value in speaking truth to power.
Click to expand...

I wasn't aiming for tact, just brevity! I'm all for speaking truth to power. No point speaking anything else. But I think it's important to focus on what Graham wants him to do (given that Prof M is unlikely to change his mind and come out against PACE himself). I think the best we can hope for now is that he's more conscious of the ongoing harms from past bad trials and that he'll do some thinking about how to advise us. That thinking in itself should ram home the nature of the problem.
 
Graham said:
That's why I said "With only a handful of exceptions". I am always grateful to those who stood up and made comments, but as a proportion of related researchers etc. it is an absolutely tiny number, given how bad and yet how influential the studies are.
Click to expand...

Sorry, I spotted this when I edited the comment for an other correction but siesta was overdue so I left it like it was.

Actually, I was meaning to say that even though the number of the researchers criticizing PACE and related trials might be relatively small, there is substantial, scientifically sound criticism that is not restricted to Twitter but published in scientific journals and books, and also part of academic education. I think it would be a wrong signal to understate this criticism.

That the PACE criticism could not stop the trial's bad influence until now might be not so much due to the small number of critics in the academic world but due to specific persons in responsible positions at specific institutions including universities who don't have scientifically sound standards and/or don't behave professionally. It is not needed to spell that out in that way in your reply, but perhaps it helps to understand my point.

Have to stop now but I hope I could make my point a bit clearer?
 
Last edited:
I agree with @MSEsperanza that it's worth pointing out the trials with the same problems happening right now in his own University.

Since his organisation is set up to increase collaboration and sharing of good practice between research universities in the UK, it is inevitable that his own university will play a large part. Is he aware that the Bristol Randomised Trials Collaboration lists MAGENTA and FITNET as part of their current trials and that these share the same methodological flaws as PACE?

Involving the people running these trials as part of his collaboration will bring his organisation into disrepute. PACE may seem like past history and therefore not relevant to his organisation but it's happening again right now on his doorstep and in danger of scuppering his organisation's good name.
 
Graham said:
My initial thought is just to reply briefly, thanking him for his response, and just to point out that a reluctance to get involved in specific debates lies at the heart of the problem in the UK, and allows third-rate work to not only prosper, but to have influence.
Click to expand...

Very nicely made point. To be frank his answer is wimpish and disingenuous. If he wants to change research culture he needs to get out there with a placard and get very mired in those debates.

Edit: I see things have moved on. I am sure Graham and Sasha will find the perfect compromise without rubbing each other up the wrong way.
 
Last edited:
Graham said:
In general, I prefer to focus on broad issues rather than specific studies. This is partly to ensure that I don’t get mired in any particular debate, and partly because I think the root causes of problems of reproducibility lie in incentive structures and research cultures. Ultimately I think we need to change these if we’re going to improve scientific quality, and the scientific environment more generally.
Click to expand...


One of the issues in incentive structures is that academics can get away with bad work and even in the case of PACE get a lot of credit for it. Also the universities involved were very keep to put PACE up as an example of good work for the research assessment exercise (which has a big impact on university funding levels). So the silence of other academics and the universities allowing bad governance and over hyping to the research assessment exercise is right at the heart of the incentive structures. The lack of willingness by other academics to challenge bad work plays right into making the incentive structures non-functional. As would say allowing bad research cultures like allowing Crawley to get away with claiming pre-registration of trials with her trick of switching from a feasibilty study to a full trial.

Incentive structures and research cultures only work if they are well governed and PACE and generally the UK approach to ME shows how these can breakdown.
 
Jonathan Edwards said:
I am sure Graham and Sasha will find the perfect compromise without rubbing each other up the wrong way.
Click to expand...
I've been good friends with Sasha for a long time now, and always value her help: that's a good thing, because I've needed it lots of times.

To be honest, no-one here can rub me up the wrong way. I know we are all on the same side, and when we have different points of view that just helps me to focus on the range of perceptions and approaches. I value clarity and honesty in these discussions, and have a lot of respect for those of you that engage with these debates.

Mind you, isn't that the wrong personality to have for someone with ME? Shouldn't I be an uptight, anxious, and stressed-out perfectionist? I'll have to try harder.
 
I'm thinking of sending this tonight:


Dear Prof. Munafò

Thank you for your prompt reply, and thank you for reading our briefing document.

Of course I am disappointed in your response, but not surprised. But perhaps you can advise us? Sadly, with very few exceptions, the only researchers who feel qualified to comment are the authors themselves. This puts us at a real disadvantage. How do we shine the spotlight on such work? The first paper was published in 2011: the NICE re-examination of the guidelines for ME/CFS is not due to finish until 2020. That is ten years of needless and great suffering from patients, despite our continued efforts to pursue the scientific line.

Can you suggest how we might mobilize academia - either as individuals or collectively - against this bad trial, which is damaging the health of patients and the reputation of UK research? What would you do in our position? The general reluctance by most UK researchers to get involved in criticizing poor work leads to the proliferation of similar poor studies (as is happening with ME/CFS at Bristol): then the sheer number of them becomes influential, while Cockroft weighs their consistency but not their quality.

I wish you well with your network, but would still like you, as you organize its structure, to consider your approach to poor quality work, particularly when it has real influence. Fixing the future is only half the job while bad past studies inflict such damage.

Thank you, and good luck

Graham
 
Graham said:
I'm thinking of sending this tonight:


Dear Prof. Munafò

Thank you for your prompt reply, and thank you for reading our briefing document.

Of course I am disappointed in your response, but not surprised. But perhaps you can advise us? Sadly, with very few exceptions, the only researchers who feel qualified to comment are the authors themselves. This puts us at a real disadvantage. How do we shine the spotlight on such work? The first paper was published in 2011: the NICE re-examination of the guidelines for ME/CFS is not due to finish until 2020. That is ten years of needless and great suffering from patients, despite our continued efforts to pursue the scientific line.

Can you suggest how we might mobilize academia - either as individuals or collectively - against this bad trial, which is damaging the health of patients and the reputation of UK research? What would you do in our position? The general reluctance by most UK researchers to get involved in criticizing poor work leads to the proliferation of similar poor studies (as is happening with ME/CFS at Bristol): then the sheer number of them becomes influential, while Cockroft weighs their consistency but not their quality.

I wish you well with your network, but would still like you, as you organize its structure, to consider your approach to poor quality work, particularly when it has real influence. Fixing the future is only half the job while bad past studies inflict such damage.

Thank you, and good luck

Graham
Click to expand...
Cochrane not Cockcroft?
 
I'd just modify the bit about the exceptions to make it clear that you're talking about the UK (because, as you've already told him, quite a few researchers outside the UK have signed the Lancet open letter.

Dear Prof. Munafò

Thank you for your prompt reply, and thank you for reading our briefing document.

Of course I am disappointed in your response, but not surprised. But perhaps you can advise us? Sadly, with very few exceptions, the only UK researchers who feel qualified to comment are the authors themselves. This puts us at a real disadvantage. How do we shine the spotlight on such work? The first paper was published in 2011: the NICE re-examination of the guidelines for ME/CFS is not due to finish until 2020. That is ten years of needless and great suffering from patients, despite our continued efforts to pursue the scientific line.

Can you suggest how we might mobilize academia - either as individuals or collectively - against this bad trial, which is damaging the health of patients and the reputation of UK research? What would you do in our position? The general reluctance by most UK researchers to get involved in criticizing poor work leads to the proliferation of similar poor studies (as is happening with ME/CFS at Bristol): then the sheer number of them becomes influential, while Cochrane weighs their consistency but not their quality.

I wish you well with your network, but would still like you, as you organize its structure, to consider your approach to poor quality work, particularly when it has real influence. Fixing the future is only half the job while bad past studies inflict such damage.

Thank you, and good luck

Graham​
 
You must log in or register to reply here.
Back
Top Bottom