How to create good quality research proposals?

[Jonathan Edwards]

Split from this thread

I don't think its enough for the MRC to stand back and say that they have no good proposals. They need to have a strategy to make sure such research areas are filled and that good quality proposals are created.

There is a lot to discuss in relation to this and at the moment I am only capable of throwing out one or two immediate thoughts.

I am fairly sure that there have indeed been few if any competitive proposals to the MRC. The projects funded in the ring fenced grant round recently were not exciting. There are lots of reasons why competitive proposals were not submitted but I think it is naive to discount this as the main problem.

And the reason why there are hardly any good proposals is that there are so few leads. And if you think about it 'filling the area' is exactly what went wrong last time. The MRC decided it wanted to put money into ME. It looked around for promising biological research and did not find anything. The work in the 1990s had come up blank. But some psychiatrists came along saying they had a treatment to test - so they 'filled the area', despite the proposal being rubbish. Next time it might be metabolomic rubbish or microbiome rubbish or autoimmune rubbish or whatever but the lesson is the same.

You cannot create good quality proposals in science to order. It is a bit like saying that the visual arts are in the doldrums so why hasn't the government commissioned a Rembrandt self-portrait or two. The answer is because Rembrandt is dead. The sort of science needed for ME is by definition going to arise from ideas that only one in ten thousand scientists can come up with. Otherwise somebody would already have come up with it. You might say that surely if you had a huge project that measured everything (as has been proposed here and Ron Davis is sort of doing) you are bound to find the answer. But it isn't like that. Unless you have the right ideas you will miss the answer in front of your nose because you measured things the wrong way.

Having said all that, there is another side to the coin. And that is that there must at least be some sort of optimum way of cultivating an environment in which the right people are able to come up with ideas. That sort of environment was I think not uncommon in the 1980s but I am afraid has largely gone. But there are still better and less good ways to do things. IiME made a major contribution with their meetings but my impression is that they have lost focus. As for the MRC itself my worry is that there is confusion about the layers of organisations. The CMRC in particular seems to be a strange Qango-style body that is sort of funder, sort of facilitator and sort of applicant. And the same people appear on the list of names on the actual MRC committees. And it is not clear that in five years they have got anywhere. That may shortly be proved wrong, in which case we can stop moaning, but I am not sure things are really sorted.

The stark difference between the 2013 Lords debate and today highlight the need for advocacy style to move on and become more sophisticated. I think that has to penetrate into the research culture in a way that is still not flowing well, even if certain log jams have been removed.

To put things differently, I agree that more funding is needed and that we should hope government would make a major contribution. But I don't think just asking for £20M to be put on the table is the answer. It will be swallowed up by the next set of -babblers before you can say Jack Robinson. The funding bodies are right to put the bar high. The scientist need to meet it.

 

[Sasha]

There is a lot to discuss in relation to this and at the moment I am only capable of throwing out one or two immediate thoughts.

I am fairly sure that there have indeed been few if any competitive proposals to the MRC. The projects funded in the ring fenced grant round recently were not exciting. There are lots of reasons why competitive proposals were not submitted but I think it is naive to discount this as the main problem.

And the reason why there are hardly any good proposals is that there are so few leads. And if you think about it 'filling the area' is exactly what went wrong last time. The MRC decided it wanted to put money into ME. It looked around for promising biological research and did not find anything. The work in the 1990s had come up blank. But some psychiatrists came along saying they had a treatment to test - so they 'filled the area', despite the proposal being rubbish. Next time it might be metabolomic rubbish or microbiome rubbish or autoimmune rubbish or whatever but the lesson is the same.

You cannot create good quality proposals in science to order. It is a bit like saying that the visual arts are in the doldrums so why hasn't the government commissioned a Rembrandt self-portrait or two. The answer is because Rembrandt is dead. The sort of science needed for ME is by definition going to arise from ideas that only one in ten thousand scientists can come up with. Otherwise somebody would already have come up with it. You might say that surely if you had a huge project that measured everything (as has been proposed here and Ron Davis is sort of doing) you are bound to find the answer. But it isn't like that. Unless you have the right ideas you will miss the answer in front of your nose because you measured things the wrong way.

Having said all that, there is another side to the coin. And that is that there must at least be some sort of optimum way of cultivating an environment in which the right people are able to come up with ideas. That sort of environment was I think not uncommon in the 1980s but I am afraid has largely gone. But there are still better and less good ways to do things. IiME made a major contribution with their meetings but my impression is that they have lost focus. As for the MRC itself my worry is that there is confusion about the layers of organisations. The CMRC in particular seems to be a strange Qango-style body that is sort of funder, sort of facilitator and sort of applicant. And the same people appear on the list of names on the actual MRC committees. And it is not clear that in five years they have got anywhere. That may shortly be proved wrong, in which case we can stop moaning, but I am not sure things are really sorted.

The stark difference between the 2013 Lords debate and today highlight the need for advocacy style to move on and become more sophisticated. I think that has to penetrate into the research culture in a way that is still not flowing well, even if certain log jams have been removed.

To put things differently, I agree that more funding is needed and that we should hope government would make a major contribution. But I don't think just asking for £20M to be put on the table is the answer. It will be swallowed up by the next set of -babblers before you can say Jack Robinson. The funding bodies are right to put the bar high. The scientist need to meet it.

I think we need to debate it. I've been thinking about it a lot recently: if any of us won the lottery tomorrow and had $20m to spend, what would be the most effective way to spend it? Because I don't believe that letting random scientific discovery take its course is the optimum way, any more than randomly swamping the field with money is.

This isn't/hasn't just been an issue for ME/CFS research - it's surely an issue for every disease that's still poorly understood, or that lacks effective treatments.

Which is most of them, it seems to me.

 

[Adrian]

You cannot create good quality proposals in science to order. It is a bit like saying that the visual arts are in the doldrums so why hasn't the government commissioned a Rembrandt self-portrait or two. The answer is because Rembrandt is dead. The sort of science needed for ME is by definition going to arise from ideas that only one in ten thousand scientists can come up with. Otherwise somebody would already have come up with it. You might say that surely if you had a huge project that measured everything (as has been proposed here and Ron Davis is sort of doing) you are bound to find the answer. But it isn't like that. Unless you have the right ideas you will miss the answer in front of your nose because you measured things the wrong way.

You can do thing like pull in experts get them talking, discussing, throwing around ideas and then fund some exploration of some of those ideas. Things rarely happen by chance. In other areas research councils do more to try to start or encourage research in important areas or try to shake it up where academics are heading in the wrong direction.

I think a call for proposals may well not get good funding but bringing together experts with experience of the different possible mechanisms to brain storm some possible experiments with the promise that some will get funded is possible.

I think some creativity is required in a process to pull people in and to think through the problem.

Having said all that, there is another side to the coin. And that is that there must at least be some sort of optimum way of cultivating an environment in which the right people are able to come up with ideas. That sort of environment was I think not uncommon in the 1980s but I am afraid has largely gone.

That's really what I am talking about and such things are (or were a few years ago) by EPSRC.

But there are still better and less good ways to do things. IiME made a major contribution with their meetings but my impression is that they have lost focus. As for the MRC itself my worry is that there is confusion about the layers of organisations. The CMRC in particular seems to be a strange Qango-style body that is sort of funder, sort of facilitator and sort of applicant. And the same people appear on the list of names on the actual MRC committees. And it is not clear that in five years they have got anywhere. That may shortly be proved wrong, in which case we can stop moaning, but I am not sure things are really sorted.

Its hard for someone like IiME to pull in experts who haven't had an interest in ME but may have a lot to contribute. Someone like the MRC can because there are potential funds at the end of it. I worry about proposals that pull in people who sit of committees my assumption is always to be suspicious of researchers who do too much committee work - is that because they have no research to follow?

And the reason why there are hardly any good proposals is that there are so few leads.

I agree with this and if I was an academic looking at this I think the first thing I would do is a large replication effort to look at which leads there are and see what is reliable when tested on larger samples and what fails. That would at least say which leads are worth looking at.

To put things differently, I agree that more funding is needed and that we should hope government would make a major contribution. But I don't think just asking for £20M to be put on the table is the answer. It will be swallowed up by the next set of -babblers before you can say Jack Robinson. The funding bodies are right to put the bar high. The scientist need to meet it.

I don't think it is about simply putting money up. The MRC needs to take a leadership position in pulling people together to come up with ideas, work together, then write proposals. The promise of a pot of money at the end of the process will help get academics to engage (with the current RAE system) but we need someone to lead a process to pull top researchers in from the relevant disciplines and get them to understand the problems. Then probably stage money with small feasibility studies with the promise of more funding for those that show promise.

I the end if we just wait for good proposals we will get nothing but if we just have a pot of money we will get academics jumping on the latest fad and trying to apply it to ME. So we need leadership from the MRC to do better.

 

[Jonathan Edwards]

This isn't/hasn't just been an issue for ME/CFS research - it's surely an issue for every disease that's still poorly understood, or that lacks effective treatments.

Which is most of them, it seems to me.

At this stage it is nearly always a matter of 'random discovery', which is not actually random because one particular individual has worked out what it is they need to look for rather than just looking for everything.

Edit: Are spellcheckers getting even worse at serving up one's posts? Had to tidy again, sorry.

 

[Robert 1973]

As past strategies have failed, I am in favour of ring-fencing. If there are insufficient high-quality proposals then funds can be carried over and added to the next year’s funds. But I believe that high-quality proposals will follow if the funding is there. As I told my MP about 15 years ago, we didn’t put a man on the moon because of the volume of high-quality research proposals that were being submitted.

I know there was criticism of the studies funded in 2012 when the MRC ring-fenced £1.5m, but such a small amount of money was never going to be enough to attract what was needed – particularly with the all the prejudices and misconceptions.

As a start, would it not be sensible for the MRC to invite bids to develop collaborative research centres as the NIH did? My understanding was that the quality of 10 bids received by the NIH was high.

I also think that new ME/CFS clinics, with knowledgeable physicians and specialist nurses (not psychologists and OTs) need to be integrated with research centres.

 

[Sean]

One good place to start is a proper assessment of the PEM phenomena, and the true nature and extent of the alleged deconditioning. Even if that does not provide any immediate clues it will help frame future progress, and firmly lay to rest any misuse or denial of PEM and deconditioning by the psychs.

 

[Hoopoe]

Michael van Elzakker said that one problem in the field is lack of serious research into root causes. We have many studies documenting the effects of the illness (usually small and often poorly replicated).

 

[arewenearlythereyet]

I think a central research hub that focuses on working with the Biobank, filtering out the crap 15 people type studies and replicating those with promise seems sensible to me. I worry that the scale of the money we are talking about will easily be gobbled up by a load of small poorly designed research studies. We need a lean and focussed approach not just throw a load of cash at the problem. Poor experimental design is not the preserve of the BPS crew..there’s plenty of crap out there in terms of “biomedical” science.

I worry that what we have at the moment needs a rethink ...the Biobank is a key part of this, but we need a better steering committee and maybe a few more endocrinologist, cell biologists and immunologists to work in the field. They need to be encouraged to move away from what they are currently doing somehow? I’m not sure what the Norwich team are doing or where it’s going ...they don’t seem to be engaging much with the patient community?

The patient community could help by lobbying for specific research avenues but at the moment it seems like slim pickings...I wouldn’t say that I have a load of papers I wish that we be followed up on...just a handful really.

I would be interested to know whether anybody has any more info on the Norwich team

 

[Adrian]

As past strategies have failed, I am in favour of ring-fencing. If there are insufficient high-quality proposals then funds can be carried over and added to the next year’s funds. But I believe that high-quality proposals will follow if the funding is there.

I'm not convinced they will come just by having a fund available. Two things happen; the first is that academics with a particular fashionable technique pile in (or even try to use such funding to get in with the latest fashion); the other is as time goes on the MRC will want to show it is spending the money so lower the barrier and take what ever is there. We also have issues with reviewers.

This is why I think something more than putting up a pot of money needs to be done.

 

[Sasha]

At this stage it is nearly always a matter of 'random discovery', which is not actually random because one particular individual has worked out what it is they need to look for rather than just looking for everything.

Sorry, I don't think my thought was well expressed. What I meant by 'random discovery' was funders and research-leaders just sitting back and waiting for scientists who happen to have an interest in a particular disease coming up with key discoveries.

That's what we've had for decades in ME. Not many scientists, maybe not the ones in the right subdisciplines or with the right ideas - and also with not enough money to thoroughly test the ideas that they had, leaving a wake of false positives and false negatives behind them.

If you have a serious problem, you organise people to look at it and you have resources available to enable them to do it.

I don't see anybody organising people to look at ME. I do see people from within the field trying to pull others in - Ron Davis does a lot of this very publicly, I think you've done it, I expect a lot of other ME researchers are doing it behind the scenes - but I don't see this happening at government level, treating ME as a long-standing medical emergency and trying to mobilise top talent to do something.

If the UK was at war, the government wouldn't just sit there and hope that scientists would come up with some good weapons. They'd organise the effort, and pour in support structures and resources.

ME is a public health emergency of long-standing. We should be at war against this disease. The government should be doing something.

What should they be doing?

What should we be doing?

What's the mechanism to move the science forward?

 

[Hoopoe]

What's the mechanism to move the science forward?

If society doesn't take action, it must be because there is so much doubt and uncertainty, or doesn't believe ME is a serious problem, or is hardly even aware of ME.

I feel like part of the problem is that the BPSers were very effective at influencing public opinion. Their narrative was engineered precisely to undermine the patient's narrative that this was an organic illness. We need to work to undo that.

A position paper similar to the IOM report would be helpful. This would have to be published by some official body. The MRC perhaps? Looking at their research projects in this area, a major shift appears to have occurred.

 

[Jonathan Edwards]

The Catch22 here is that if there was a way forward that involved following a recipe it would have been done long ago. Solving questions of disease mechanism is quite different from putting people on the moon. 95%+ of people in the field do not know how to solve problems of this sort but they are very happy to get funded to turn the handle.

IiME have actually done fantastically well to bring in relevant people: Angela Vincent from neuroimmunology, David Brooks from neuroimaging, Mike Murphy the mitochondrial man, Ann Cooke from basic autoimmunity, and so on. I am not sure that the MRC have brought anyone in. Maybe Chris Ponting but Chris had other motives for getting involved.

And maybe the central problem is 'someone like the MRC'. There is no such person. There are bureaucrats who have worked their way up the hierarchy to be in charge of funding but these are very often not the people who know how to solve problems. The raw truth is that the MRC has not been responsible for many innovative steps in disease mechanisms over the years. It funds research once the key problem has been cracked.

So my feeling is that rather than call for £20M ring fenced funding and the looking around to see who might make good use of it we have to find the people who can make use of it first. We also need to be sure that proposals will be judged intelligently. And that is the rub. So far it seems they have not at MRC. This is where I have sympathy with IimE in thinking that things have not really changed yet in the way they need to.

On the other hand there is no point in having a free for all with charities lobbying MPs to market their preferred scientists, as we had yesterday. The Norwich group was mentioned twice but from what we are told they have already been funded for eight years with five PhD students and they have not as far as I know produced a single data paper. Nobody can consider that the sort of track record needed for the task.

So we need a different way of identifying what needs doing and who might do it. As I have said, I see the CMRC as being too much of a hybrid of mixed interests. Maybe there needs to be a S4ME search committee to thrash out what is wanted. I would have more faith in people here than the MRC hierarchy. It might seem that the MRC mandarins should be the right people to show leadership but they were never any use to me in my career. I don't think people in general appreciate just how uninspired high profile people can be. As the saying goes, if you really want something done it is best to do it yourself.

 

[Jonathan Edwards]

I don't see this happening at government level, treating ME as a long-standing medical emergency and trying to mobilise top talent to do something.

Stephen Holgate has been tasked with this, or has tasked himself with this, at the MRC for about a decade. In fact the recognition of ME as a priority goes back to 2002. The 'government' have very much been doing what you say - but getting nowhere. We need to reflect on why that is.

 

[Sasha]

IiME have actually done fantastically well to bring in relevant people: Angela Vincent from neuroimmunology, David Brooks from neuroimaging, Mike Murphy the mitochondrial man, Ann Cooke from basic autoimmunity, and so on. I am not sure that the MRC have brought anyone in. Maybe Chris Ponting but Chris had other motives for getting involved.

Do we need some sort of 'ME roadshow' in the UK?

The IiME colloquium seems to have helped draw people in in past years but it's a once-a-year thing that people have to commit to spending a couple of days in London for, and I think it's invitation-only. Invitation-only can exclude as well as invite.

Maybe there needs to be a S4ME search committee to thrash out what is wanted. I would have more faith in people here than the MRC hierarchy. It might seem that the MRC mandarins should be the right people to show leadership but they were never any use to me in my career. I don't think people in general appreciate just how uninspired high profile people can be. As the saying goes, if you really want something done it is best to do it yourself.

We've been throwing ourselves mainly at the funding and awareness and PACE problems from the patient side, I think. If there's a way that we can throw ourselves directly at the research recruitment and support problem, we should do it.

When you say 'search committee', do you mean 'committee to search for scientists to recruit'?

If so, do we patients have that knowledge/expertise?

 

[Sasha]

Stephen Holgate has been tasked with this, or has tasked himself with this, at the MRC for about a decade. In fact the recognition of ME as a priority goes back to 2002. The 'government' have very much been doing what you say - but getting nowhere. We need to reflect on why that is.

I'm confused. I said, "you organise people to look at it and you have resources available to enable them to do it. " Has the government really done this? In what way?

 

[Jonathan Edwards]

Do we need some sort of 'ME roadshow' in the UK?

Maybe, what exactly would that be?

The IiME colloquium seems to have helped draw people in in past years but it's a once-a-year thing that people have to commit to spending a couple of days in London for, and I think it's invitation-only. Invitation-only can exclude as well as invite.

Exactly, and almost all UK researchers seemed to be excluded this year.

If there's a way that we can throw ourselves directly at the research recruitment and support problem, we should do it.

When you say 'search committee', do you mean 'committee to search for scientists to recruit'?

If so, do we patients have that knowledge/expertise?

I was thinking of searching for both target projects and scientists with the expertise to address them. The patient and carer community has all the necessary knowledge/expertise. The critique of research paper abstracts here is way above the level set by journal editors.

 

[Jonathan Edwards]

I'm confused. I said, "you organise people to look at it and you have resources available to enable them to do it. " Has the government really done this? In what way?

A working group was set up to look at ME research. And added to that the CMRC was set up. For the last five years that has been supposed to be finding the best projects and people. The resources have always been there. There is £20M available for ME research for the asking - there always is at MRC. Either the proposals have not been forthcoming or people at the MRC supposedly dedicated to finding good proposals cannot see what is in front of their noses. A have good reason to think it is a bit of both.

 

[Hoopoe]

Either the proposals have not been forthcoming or people at the MRC supposedly dedicated to finding good proposals cannot see what is in front of their noses.

FOI requests to see what kind of proposals they are getting and rejecting could be informative.

 

[Robert 1973]

I agree that just ring-fencing will not be enough. But I maintain that it is important, and may be a prerequisite. We certainly need the right sort of people at the MRC or NIHR making the decisions, which has not been the case to date.

The MRC website currently states:

Research strategy
CFS/ME is currently a highlighted area and a high priority for the MRC. In 2008 we set up the CFS/ME Expert Group to consider how new high-quality research into CFS/ME, and partnerships between researchers already working on CFS/ME and those in associated areas, might be encouraged. This work follows on from the Research Advisory Group set up in 2003 and the joint workshop held with Action for ME in 2006.

(https://mrc.ukri.org/funding/scienc...medicine/our-science-and-contacts-psmb/cfsme/)

The Expert Group is listed as:

Professor Stephen Holgate - University of Southampton (Chairman)
Professor Jill Belch - University of Dundee
Dr Esther Crawley - University of Bristol
Professor Philip Cowen - University of Oxford
Professor Malcolm Jackson - University of Liverpool
Dr Jonathan Kerr - St George’s University of London
Professor Ian Kimber - University of Manchester
Professor Hugh Perry - University of Southampton
Dr Derek Pheby - National CFS/ME Observatory
Professor Anthony Pinching - Peninsula Medical School
Dr Charles Shepherd - ME Association
Sir Peter Spencer - Action for ME
Professor Peter White - Bart’s and the London School of Medicine and Dentistry

That the list appears to be out of date perhaps exemplifies the importance that the MRC attributes to ME research – while some of names may explain why it has achieved so little.

Getting the right sort of people interested and talking to each other (both inside and outside the MRC and NIHR) is clearly necessary, as well as removing those who have inhibited progress. In order to awaken more of the right sort of people’s interest, it may require those already interested and involved to try to reach out to others – to try to persuade them of the need and the opportunities. But I’m guessing that would be easier if there was a pot of money waiting to be spent.

I’ve tried writing to various people over the years – most recently Sheena Cruikshank – but as an unqualified patient my emails have very limited impact. I suspect an email or even an invitation to lunch from an emeritus professor of medicine would be more effective.

Similarly, in order for a recruiting committee to be effective I suspect it would need to include a number of people with the right sort of formal qualifications. Knowledge alone may not be enough.

[edit: typo]

 

[Trish]

I thought the Expert group morphed into the CMRC, which has achieved bugger all.

There is hope now that Chris Ponting has replaced Esther Crawley as deputy chair, but as has already been said, the CMRC seems not to quite know what it is - funder, adviser, creator of research proposals, talking shop, conference organiser...

Chris Ponting's proposal for a huge GWAS (genome wide association study) could easily eat up £20 million, I think, and take many years. Is that the best next step?

How about replicating the NIH deep dive with a small sample of well defined patients brought into a centre for a couple of weeks and looking at everything they can think of?

Or a big expansion of the UK ME Biobank and associated studies?