Ensuring severely affected can be heard

I have started this thread as a lot of advocacy campaigns , and some research utilise online questionnaires to gather info.
The James Lind Alliance use this format for their PSPs , and will do so for the ME one.

As discussions elsewhere have highlighted , getting input from those severely affected can be difficult due to formats used: it may be that noone involved has had direct experience, or those with severe ME have not been asked re how things could be designed to ensure their info can be captured .

We saw this with the Oxford Brooke's paediatric survey format ( thus was primarily telephone based) and whilst the research staff tweaked the format to be more individual, many lost an opportunity to contribute.

To avoid the situation where those with least representation have difficulty/ cannot input into such research can we have a discussion re what would help ?

For example

Always having save and return function

Being able to print off a copy to think through and pull thoughts together before online completion ( could be done with family / carer )

Ensuring speech to text option for filling in is available

Would graded responses ( either ranking of simple options , or 5 point scales commonly used in marketing etc) be easier than descriptive sections , but perhaps maintaining option to have comments too ?

Tagging @lunarainbows , @Simbindi , @DigitalDrifter , @Louie41 , @andypants , @Trish
as these are names off top of my head . Apologies for missing others

 

As an autistic person I find questionnaires really difficult and stressful, I end up in knots as none of the response options provided ever fit my thoughts on the issue. Also I can't just pick an answer and then forget about it, the fact I've not given a precise answer will take over my thoughts for days, weeks or months. So there should always be an option to just use free form answers without being forced to tick a choice or rank items. I do recognise however, that this won't be the same for many severe ME sufferers and having simple tick box options can help people with very limited cognition to be included.

Definitely the option to save and return, and having a generous response period allowed (at least 4-6 weeks, longer if possible).

 

I also don't answer any survey if I can't read through the whole thing before starting it. Most of my autistic peers are the same in this regard. It's not just the cognitive fatigue, it's the stress of not knowing what to expect as the questions proceed.

 

@Simbindi agree with this. I struggle with questions where you have to pick one answer or answers from multiple choice, or where you have to rate from 1-10. I never come away feeling like it actually represents what I feel, or that I’ve chosen the right answers.

I need free form as well and being able to see all the questions.

I found the Severe ME survey for the NICE guidelines where you did one page before going onto the next page, very stressful. I ended up repeating answers and forgetting what I had written before. Wasn’t sure which box to put what in as some seemed so similar. Wasn’t sure whether to write it now or to wait as another question might come up later. Having to go back and forth and I got more cognitively impaired and unwell by having to do that.

Trying to figure out where your pain is on a scale of 1-10 or to try to rank things, can also be difficult. It actually takes a lot of effort for me. It’s like that part of my brain doesn’t work well even though I may be able to type other things. I can’t say if this is a “me” thing or a severe ME thing. I don’t know what the right answer is. But I would be surprised if this also wasn’t the same for some people with ME though, because I’ve noticed these issues have got worse with ME.

Like for example instead of being asked to think about where pain was or what their feelings were on a 1 to 10 or on a scale, being able to just write “awful” should be enough. So a free form IMO should always be provided.

(I think things like “yes/no” can be easier but even then I struggle when things are ambiguous).

Being able to save and come back to it, at least 6 weeks, I think even 8-12 weeks would be good.

Also, not a lot of questions at once and especially not many topic changes. I find when I have to jump from a question to another is when I really struggle. I may be able to write a lot on one topic or wait for the thoughts to come on that topic. Then I suddenly get a new question. Requires cognitive energy to think about the next one. Then the next one. Even if the topic change is small still require much more cognitive input and after a while I start to get very stressed and unwell. I find it difficult to keep jumping. Especially if the topic change is large then that’s really hard. It’s why I don’t like questionnaires at all. Especially multiple choice and ranking style ones as there’s so many questions and they each expect an answer. The fact they’re short doesn’t make it any better.

For example I struggled on the Action for ME survey on the NICE guidelines. I’m pretty sure I repeated answers and missed out things and their multiple choice was very difficult for me. I was confused on quite a few of the questions. Didn’t know what I had written or when. I was stressed and more unwell by the end. I can’t say I filled that in properly at all.

IMO if it was me, I would say stick to just a few questions per questionnaire and just one or a few important topics. Allow people to write as much or as little as needed. Have multiple choice for people who find it easier but have free form too. Have a save function.

I’ve used speech to text apps before but they take so much cognitive effort for me, now my arms are better with pain relief, I turned it off. It takes me much more energy to talk 2-3 words, than it does to actually type 2-3 paragraphs. I think a lot of people with severe ME struggle more with talking. And some can’t speak. So having it done via only telephone call is not helpful at all. Even for carers, if they have to talk/type on their behalf, still think it’ll be more accessible via form that can be filled in over many weeks.

 

I gave up on the Action for ME NICE guideline survey for the reasons given by @lunarainbows . It also felt like they were 'putting words into my mouth', I didn't like that at all.

 

I echo the need to be able to read the survey through before starting.

My reasoning is that given the historical problems that much ME research has been designed with a specific agenda I do not want to invest time and energy in a questionnaire that half way through leaves me feeling I can not trust the objectivity of the researchers.

 

This happens to me too.

It's quite frustrating. If I'm asked a question about rating a symptom like pain on a "good" or not as bad as it could be day I will probably answer more accurately than on a bad day. The problem is I'm so used to being clobbered by so.many symptoms at once I'm actually quite good at turning the volume of individual ones down to a certain extent. Like many of us I suspect. So in the past I've given a rating if maybe a 4 or a 5 on a 0-10 scale then afterwards realized I should have put a 3.

Being able to go back, review answers and edit answers, as well as being able to browse through the whole thing in advance is important for me.

 

Completely agree.

I would like to see questionnaires assess their own level of acceptability.

Currently, if I attempt to fill a questionnaire and wade through only to realize that it simply cannot capture ME accurately because the questions are inappropriate, then I just don't save it and "walk" away.

This means that the people who designed or commissioned the questionnaire never find out what the people who filled in the questionnaire or who ditched it in disgust actually thought of the questionnaire and their degree of confidence in the ability of the questions to elicit meaningful data.

So, I would like such a questionnaire to have the facility for someone who doesn't want to fill it for whatever reason to be able to say so and leave all the other questions unanswered if they want to.

Even if they do fill it in, I think they should be able to rate the questionnaire itself & how appropriate the questions are for ME patients.

 

I would rather see a questionnaire with more straightforward questions than one with fewer compound or complex questions.

Initially it might look like extra effort to fill in but, for those with bad cognitive problems, compound questions can be easily misinterpreted.

Another pet hate with questionnaires are questions that conflate two different things phrased that no matter how you answer the question it indicates you agree with the conflation when you do not. An example might be a question that would conflate "fatigue" with "PEM".

 

Just thought I would mention that I often want to hide the next questions when I’m designing questionnaires. This is to avoid pre-empting/biasing answers particularly when they relate to each other. Most of the time you want some questions to be able to stand alone without context to the others you ask later and this dictates the order of the questions you pose (depending upon the subject). If you can go backwards and forwards this rather ruins that.

I think a progress bar and save option (without the backwards and forwards), together with a generous estimate of the time it should take (5-10 minutes for a normal person) up front would give a cleaner result.

Bias is a real problem with subjective questionnaires and some of the suggestions to make it easier to fill in may make the questions asked or results more difficult. It’s better to ask fewer questions and get quality answers than overwhelm the responder and build questionnaire fatigue into the design.

Of course most of the things raised here and elsewhere are because the questionnaires experienced have been designed by people who don’t seem to have a clue what they are doing and they are bad for everybody (and only good for biased results). A 2hr questionnaire is ridiculous ...I agonised over one recently when the completion time came back as 11 minutes!

my rule of thumb is 12 questions max and for things like pain you would use a top and tail sliding scale since it’s a concept very difficult to describe a scale in words.

 

I think those are probably fair comments when writing questionnaires for normal, healthy individuals @arewenearlythereyet.

For example

Often with ME questionnaires that I look at and ditch, either the earlier question is unclear or compound and or I misinterpret it. Then I see the next question and realize I answered that with my first answer.

I think the problem may very well be that ME is simply not well enough understood to formulate a questionnaire in the same way you would for say people's preferences to.....different types of cheeses.

If you know all about the different types of cheese, ingredients, price range etc., the unknown is the participants perception.

In our case, not only do they want to know our perception but they are unclear about the subject of our perception as well. Worse still, they don't seem to be aware of that and so ask misleading questions assuming they know more than they do.

 

With respect I’m not really talking about cheese?...this is basic stuff that can be applied anywhere ...for services, feelings, whatever ... not just preference of a foodstuff. I know this because I’ve used it in things other than those you mention.

I don’t think ME is that special that it needs anything other than quality design by people who know what they are doing....part of that is tailoring your design to the audience ....so length of questions may need to be different for normal people yes ...you can chose different scales etc etc. You don’t need to have special types of question ?

The methodology basics are the same regardless of the question...you may need to choose different scales etc but elimination of bias via questionnaire design, order, size, appropriate scales etc etc is universal ....seems medical questionnaire designers may suffer from arrogant researchers thinking they don’t need training ...but I would say from experience that is a condition quite rife in research wherever I seem to look.

 

The CMRC PAG has forwarded this thread to the ME/CFS Priority Setting Partnership who are currently formulating the main survey/questionnaire that will be distributed to patients (and their carers, etc). (https://www.psp-me.co.uk/)

 

Thanks @InitialConditions .
That was one of the reasons why I started the thread

 

It would be something where you wouldn’t want to lead the respondent so you might start with a more general question such as

“how do you rate your quality of life” and then ask more specific questions about aspects of quality of life subsequently (e.g. mobility, self care, pain etc etc). Sometimes you might also ask a slightly different question along the same lines where you were concerned that you might get a biased answer so in that instance you would space the questions far apart.

 

By going backwards and forwards you create a bias where you measure people’s tendency to be influenced and go back and forth not the “clean” answer that you originally wanted without leading the respondent....you would have multiple variables of some that didn’t go back and change, some that did and some that went back and forth multiple times ...so you are introducing a questionnaire bias.

If you were more concerned about the bias of people not completing the questionnaire then you would have to do a series of questions that were unrelated and then probe the same sample later with another questionnaire...problem is the likelihood of getting the same sample large enough then becomes a problem.

 

I'm afraid I disagree.

I think the problem with questionnaires written by people who are experienced in writing questionnaires are that they aren't necessarily experienced in ME and in writing questionnaires that are ME friendly and designed to elicit the most accurate information with the least impact in the patient.

I would also suggest given the high degree of suspicion many ME patients have, especially those who've already had a thorough mauling by the system, the level of control and direction you're talking about would mean a fair few would walk away.

I completely respect you have more experience of drawing up questionnaires. I am saying that I believe questionnaire need to be designed and formulated specifically with ME patients, the level of cognitive dysfunction and their natural high levels of suspicion in mind.

Yes, I believe you do, to the extent that the must not be open to interpretation by either the person answering or the person gathering the information. They must be very clear and very unambiguous if you want those with cognitive impairment to be able to complete them honestly and accurately.

I would also point to Lenny Jason's questionnaire - as far as questionnaires go by far and away one of the better ones & written by a team familiar with ME. Did we like it? No. The layout and fonts were issues, we got confused about which question we were answering and so on. The questionnaire design itself needs to be built with severe ME patients in mind from.the ground up.