Patient led measure of outcomes

[Adrian]

An interesting idea. Sentiment analysis of free form descriptions by people of how they are?

It would require testing each LLM is different and its not something I've tried but generally I've found summarization really good - but on technical topics - and I'm only using small language models (~3 to 8b params) that I can run on my laptop.

 

[hotblack]

It would require testing each LLM is different and its not something I've tried but generally I've found summarization really good

Me too. But I’ve not got any experience with deeper analysis. Although know its used in quite a few areas.

Definitely interesting and could be good when there is lots of data from different sources to analyse too. It would be a lot more work to administer than what I was thinking, which was going for absolute minimum data and effort by everyone involved. Two different but potentially very valid approaches though.

 

[Kitty]

- Would picking 5 descriptors from Funcap 55 be easier/better than completely patient created descriptors or are these too restrictive?

I'd say the usefulness of personalised descriptors is that they're things you actually do, not things you have to imagine whether or not you could do.

Things you're motivated to do, either because they're a bodily need, or you do them every day because they've become a habit or you really enjoy them. Or things that need doing, and if you don't do them they either get added to the load of someone else in the household, or the omission impacts your dignity or quality of life (e.g. not changing bedding as often you want to).

So you start with things the person does every day unless something unusual happens. Every day is important, because it's an indication of their base level of functioning. How many days this week did they do them?

Then a couple of things they'd only expect to be able to do once a week or so because they trigger PEM. How often did they do them?

Then things they'd really like to be able to do, but can't without making themselves really unwell. Perhaps a couple of pre-specified things, and one option left blank for something they hadn't thought of. Have they done any of them?

They should all be the things of ordinary life: eating, toileting, catching up with online news/social media, listening to music, washing hair, dealing with laundry. The ambitions need to be ordinary as well, e.g. staying out of bed most of the day, meeting up with someone outside the house, doing a trip away from home.

It might be challenging to explain all this, but it's important to get away from the theoretical ability to do things as much as possible. If trial participants really want a treatment to be working, they're more likely to overestimate their capacity to do theoretical activities. The focus needs to be on things they actually did and how often they did them, because that will show whether they had more, less, or roughly the same capacity.

 

[Kiristar]

Some questions I have, and am unsure of the answers, would appreciate feedback

- Would picking 5 descriptors from Funcap 55 be easier/better than completely patient created descriptors or are these too restrictive?
- Should he question be can you do these things without significant negative impact? Or comfortably at the time? Or just did you do these things? How important is it to capture negative knock ons? Or would these simply reduce how often you did things so be captured by frequency measure?
- I like the idea of the patient measuring these 5 things weekly, it’s easy enough to then do a monthly total, what do people think? Would monthly work better or less well?
- Would we need the daily or weekly good/bad/average measure too? Does this give us extra useful data or is it too much to record?

Maybe we can organise our own trial of this framework? I could see people using whatever method they prefer, a spreadsheet or just a piece of paper. And not sharing any data but sharing what they like or don’t like, what works or doesn’t work, etc.

There’s no intervention to test, but we could learn how to best make the framework/process to work for different people. If we can get something that works for patients maybe some researchers would take it up?

I am severe and very foggy. The cognitive effort of having to develop my own questions and scales would be too hard for me. I can't even remember things over a week either same day is challenging.

I stopped visible as symptoms tracking was too onorous.

I know it doesn't work for everyone but the only outcome thing I track now is hrv trend and then 1 symptom ("did I have a bowel movement" and that only out the desperate need to avoid ending up in A&E! ).

Tbh I can't manage 3 things, five would be tough. Plus something like did I leave the house - I may be forced to for medical scan but with dire consequences. My relapses are so long I couldn't quantity them for noise.

Now rambling but guess I'm saying I'd rather answer a longer preset thing that someone with a better brain than mine co-produced with patients with better brains than mine.

When I did do funcap it made sense, picking a few from there might work but it would depend what the study needed I guess.

Edited to add paragraph spaces.

 

[Adrian]

Me too. But I’ve not got any experience with deeper analysis. Although know its used in quite a few areas.

Definitely interesting and could be good when there is lots of data from different sources to analyse too. It would be a lot more work to administer than what I was thinking, which was going for absolute minimum data and effort by everyone involved. Two different but potentially very valid approaches though.

I'm thinking with AI lots of tasks which could have proved difficult before could be doable now and automatable (although some work to automate). But if we are thinking to future trials we should have in mind these new capabilities in a design and it could lead to new ways of working.

 

[Hutan]

Some questions I have, and am unsure of the answers, would appreciate feedback

- Would picking 5 descriptors from Funcap 55 be easier/better than completely patient created descriptors or are these too restrictive?
- Should he question be can you do these things without significant negative impact? Or comfortably at the time? Or just did you do these things? How important is it to capture negative knock ons? Or would these simply reduce how often you did things so be captured by frequency measure?
- I like the idea of the patient measuring these 5 things weekly, it’s easy enough to then do a monthly total, what do people think? Would monthly work better or less well?
- Would we need the daily or weekly good/bad/average measure too? Does this give us extra useful data or is it too much to record?

Five seems like quite a lot - would three be good enough?

I think it should just be 'did you do them?' That cuts out the interpretation. I think any study still needs to be long enough to pick up deterioration due to overexertion. So, that covers part of the negative knock ons. I think the measure needs to be completed daily - that way there is an aspect of frequency, so that also covers part of the negative knock ons.

I like the idea of picking activities from Funcap.

 

[Jonathan Edwards]

Three is probably enough.

 

[Trish]

I assume this idea of each individual picking 3 activiies as an outcome measure would be for testing a treatment in a trial over a relatively short term, say up to a year. I guess it would be OK for that.

I think for longitiudinal studies of fluctuations of the illness over time and in relation to acitivity levels it would be better to use the whole 27 item version of FUNCAP.

 

[hotblack]

Lots of really useful and insightful feedback. Thank you everyone.

So maybe something like…

- Pick 3 activity descriptors that you feel best describe your current limitations and level of activity, you can choose from FUNCAP55 or write your own.

Try to pick a range which represents you best, with one you can usually do without much difficulty one less often and one you’re rarely able to do

- Each day (or week) record how many times you do each activity (without significant difficulty?)

- Each day (or week) record if you consider it a good, average or bad day/week for you

In this way you will get both weekly and monthly totals of ‘activities’ and ‘good/average/bad’ periods

You can use a spreadsheet, a piece of paper, or whatever method works for you (I/we could provide some ideas and templates to use, copy or print)

 

[hotblack]

I assume this idea of each individual picking 3 activiies as an outcome measure would be for testing a treatment in a trial over a relatively short term, say up to a year. I guess it would be OK for that.

Yes, very much that. Is there a statistically significant benefit to patients of the intervention.

All the other complexities in these questionnaires seem to try to get fine grained or subtle detail or discern if changes are present in different areas or in different groups of people occur or compare us with other conditions.

There may of course be objective or clinical measures which can be done independently, but the idea is a lightweight patient led way of answering ‘does this help me?’

 

[Utsikt]

Lots of really useful and insightful feedback. Thank you everyone.

So maybe something like…

- Pick 3 activity descriptors that you feel best describe your current limitations and level of activity, you can choose from FUNCAP55 or write your own.

Try to pick a range which represents you best, with one you can usually do without much difficulty one less often and one you’re rarely able to do

- Each day (or week) record how many times you do each activity (without significant difficulty?)

- Each day (or week) record if you consider it a good, average or bad day/week for you

In this way you will get both weekly and monthly totals of ‘activities’ and ‘good/average/bad’ periods

You can use a spreadsheet, a piece of paper, or whatever method works for you (I/we could provide some ideas and templates to use, copy or print)

How would you get around the issue of redistributing activities in order to do the things you measure more often? Or the opposite: not being able to do the things you measure because life happened and you had to use your energy on something else?

Maybe combining it with some kind of overall measurement of activity to see if you do more or less overall?

 

[hotblack]

Maybe combining it with some kind of overall measurement of activity to see if you do more or less overall?

I think that gets too complicated and difficult to measure. That’s one problem with these questionnaires, complexity as a result of trying to design for every possible ‘what if’. How would you measure activity? Most methods seem equally subjective and open to issues. Open to suggestions though!

How would you get around the issue of redistributing activities in order to do the things you measure more often? Or the opposite: not being able to do the things you measure because life happened and you had to use your energy on something else?

I’m not sure. Maybe we can’t account for every eventuality? Or maybe those things mentioned all balance out over time or enough payients? If a treatment/intervention has a meaningful, significant difference would it be picked up across a decent sized cohort and outweigh these effects?

Maybe we need to try it and find out.

 

[hotblack]

I’m planning on making a template and giving this a go for a few weeks to see what works and what doesn’t. This probably won’t be for a month or so, but I’ll share any templates I have then so others who want to can try themselves and also give feedback on what works or doesn’t for them.

If anyone has any more thoughts or feedback beforehand they’d be great to hear!

 

[MrMagoo]

Could people just wear a wearable and the data be fed into a bot/python script?
I’m just thinking about lessening the burden whilst maximising the data.

Personally I’ve been struggling lately with the difference between being active but in pain and forcing myself, vs not being active but in pain. Because both ways I’m in pain. And my mood can be low or high in either.

 

[Sasha]

Personalised outcome measures are perfectly valid (see Edwards, Isenberg and Snaith for a lupus trial using them)

There doesn't seem to be anyway to access that paper without paying. The only otherwise available info about the personalised outcome measures is in the abstract:

Patients were rated for improvement on a four point scale using individualised criteria. On three occasions patients improved to ‘ideal’, on 12 there was ‘useful’ improvement, on 11 the patient remained static, and on four occasions there was deterioration.

Is there any way for us to see a bit more about the development and use of the personalised outcomes?

 

[Sasha]

The discussion section of the paper is really interesting and very relevant. I’d quote it but copying from the png/pdf is giving terrible formatting issues which are a faff for me to clean up atm.

I'd be happy to clean that up if you'd be willing to send it to me. I'm very curious to see it!

 

[Sasha]

I assume this idea of each individual picking 3 activiies as an outcome measure would be for testing a treatment in a trial over a relatively short term, say up to a year. I guess it would be OK for that.

I think for longitiudinal studies of fluctuations of the illness over time and in relation to acitivity levels it would be better to use the whole 27 item version of FUNCAP.

Why do you think it would only be suitable as a short-term approach?

 

[Utsikt]

There doesn't seem to be anyway to access that paper without paying. The only otherwise available info about the personalised outcome measures is in the abstract:



Is there any way for us to see a bit more about the development and use of the personalised outcomes?

You can download the pdf from here:

A double blind controlled trial of methylprednisolone infusions in systemic lupus erythematosus using individualised outcome assessment - PMC

Twenty one patients with severe systemic lupus erythematosus (SLE) were treated with three daily infusions of either 100 mg or 1 g of methylprednisolone on a randomised double blind basis. Nine patients with unsatisfactory outcome subsequently ...

pmc.ncbi.nlm.nih.gov

 

[JemPD]

Sorry too ill to read whole thread but just wanted to say that these types of things are problematic because many of us don’t have lives that allow for doing the same every wk month etc.

I always find funcap so etc almost impossible to fill in, because the answer to whether I can bathe or not depends on whether I done the other things.

So for example the answer to “can you do x without crashing?” For many activities is “I don’t know how many times have I had ton get up to go to the loo today”, or “ I don’t know did I havesomeone bring me a meal or did I have to grab snacks from kitchen on way back from bathroom”

These types of scales work well for tracking improvements deterioration when lives allow routine. Many of us don’t have that. Unfortunately.

As an example, I said to carer last wk, I appear to be deteriorating but I hope it’s just that I had to do more unavoidable stuff like medical appointments etc the last couple yrs. plus amount of care support has fluctuated hugely & outside of my control. Leading to lots of crashes unavoidable rolling PEM.

It may be that candidates participants who have less predictable & regular/routine, those who are forced into push crash frequently would not be good candidates for a trial without objective measures.

Sorry don’t know the answer to the problem but just throwing it into the pot

 

[bobbler]

Thinking about how we can measure if treatments work. A lot of the questionnaires and scales seem to try to compare across subjects, which is really difficult for many of us, they don’t fit our experiences or severity. So people try to capture a range of experiences and we end up with huge long questionnaires. So how do we measure outcomes?

How about this
- Pick and describe in your own words 5 activities that you feel define your current limitations, include 2 things things you can only occasionally do
Examples: get to the toilet in the morning, sit up comfortably throughout the day, have a 5 minute conversation with someone , have a shower, walk to the car (I don’t know I haven’t done those last two for years but you get the idea, the usual sort of things we see on questionnaires, but defined by the patient)
- Count how many days you can do these per month, before and after interventions, without negative impact, record weekly
- Maybe add a measure of how many days are ‘good’ ‘average’ or ‘bad’ for you, record daily

This would be person specific but capture the changes which are relevant and/or important to them and how their ME/CFS affects them. It would be quicker than most things to record but I think would allow measurement of if an intervention has actually worked.

Probably needs some refinements but…thoughts?

I'm trying to have a think on actual examples of things I have tried. Exempting lots of 'no idea really if they do anything' stuff. I'm a cynical so and so and don't try things lightly too so this is a curated bunch.

B12 injections work for me. But I couldn't claim in more than certain symptoms ie not curing the ME/CFS or whatnot, although unlike nonsense like sleep hygiene or other therapy-based where they are getting you to just practice the task to be tested at the expense of everything else or something caffeine-like my kick me in the bum at another point I think there might be something it is fixing in me.

That is easily 'tested' if people believe me. Because it wears off really clearly at a certain point. And at a point say a week or two after that it's even more clear. It affects things like me being able to look at screens particularly TV etc. along with symptoms that are more sensations that would accumulate over longer times. The effect isn't always exact of how much or how great I feel each time I have one (I suspect I've 'used up' whatever before or have more to clear up whatever it does) but I'd know within x weeks if I'd been given a placebo.

That doesn't mean they work for everyone else either in the same symptoms or to the same extent. But what proof or test would I pick to find a group like me that others would 'OK' other than 'I can pick with constant accuracy when I've been on the placebo'?

I know ibuprofen works for certain injuries I have because if I leave it too long between doses then it is utterly stiff. Maybe that is measurable, certainly again I'm sure I'd notice a placebo.

But the question posed elsewhere about whether it helps PEM or ME/CFS I have no idea on even though theoretically I've done an experiment over years before and years after, but as we all know it is so situational re: how much PEM we have and remembering how bad something felt even last time vs this time is hard (points of reference, get used to pain, do other things to help, Noise + pain is worse) nevermind the timespans I'd have to be covering. And that's before I factor in adjustments I might have made and of course over this time my illness getting worse (but meaning I can do less etc).

I've had different doses of steroids which is really hard to report back on. Because it wasn't for ME, but would affect that and other things that add to my burden too. I know how much better I felt when I took a dose once. Obviously better. I also know how clear it was I needed something when coming off that dose, and then doing it even more slowly and it being clear there was an absolute minimum needed there. But the 'right' one is harder and more long-term.

When I had doses that might have been a level you could say were putting other things aside and just looking at some left-over for the ME/CFS I felt great. I'm not sure how much more I actually did, I still got myself stuck in silly situations flat on my back in awkward places but I also pulled off the little windows aping seeming normal a lot more convincingly and enjoyed that. I'm not sure I could have measured and compared how much I payed for it vs before. I probably felt stronger and had better posture when I was at my very good days. But also paid for them requisitely. I was just relieved to get that holiday of an hour here or there of 'less bad' or 'feeling normal doing x' (which any other person wouldn't think twice on like sitting in garden).

I think if you'd asked me a few months in I'd have said it was great, but 6months in perhaps it wasn't the right dose. I had a more debilitated number of months and had a virus of some sort which whilst I was flat on my back didn't feel as scarily ill as I would have done, but it was clear how fragile I was if I just chatted or scraped to the loo and had to sit and lean against a wall at certain points feeling white.

But again a lot of other things that were also significant changed at that point too. Including changes from major noise issues which nothing measures yet and dominated my health for years turning rest into exertion.