1. Sign our petition calling on Cochrane to withdraw their review of Exercise Therapy for CFS here.
    Dismiss Notice
  2. Guest, the 'News in Brief' for the week beginning 8th April 2024 is here.
    Dismiss Notice
  3. Welcome! To read the Core Purpose and Values of our forum, click here.
    Dismiss Notice

Thesis System and methods to determine ME/CFS & Long Covid disease severity using wearable sensor & survey data, 2023, Sun

Discussion in 'ME/CFS research' started by Tom Kindlon, Jul 11, 2023.

  1. Tom Kindlon

    Tom Kindlon Senior Member (Voting Rights)

    Messages:
    2,202
    Free fulltext:
    https://ccs.neu.edu/~ysun/publicati...ity-using-wearable-sensor-and-survey-data.pdf

    SYSTEM AND METHODS TO DETERMINE ME/CFS & LONG COVID DISEASE SEVERITY USING WEARABLE SENSOR & SURVEY DATA

    by Yifei Sun

    A Senior Thesis Submitted to the Faculty of The University of Utah In Partial Fulfillment of the Requirements for the Degree in Bachelor of Science in Computer Science

    Abstract

    Myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS) is a debilitating disease with high probability of misdiagnosis and significant unmet medical needs that affects as many as 2.5 million people in the U.S. and causes enormous burden for patients, their caregivers, the healthcare system and society. Between 84 to 91 percent of ME/CFS patients are not yet diagnosed [6, 19], and at least one-quarter of ME/CFS patients are house- or bedbound at some point in their lives [12, 13]. The impact of ME/CFS to the U.S. economy, is about $17 to $24 billion in medical bills and lost income from lost household and labor force productivity per year [7, 13].

    Current widely used diagnosis methods of ME/CFS and other diseases with similar clinical symptoms like Long COVID [6, 21] are highly dependent on patients’ self reporting [4, 5] and standardized survey, which are not optimal for medical diagnosis. In a joint study with The Bateman Horne Center (BHC) 1 , we designed and developed a system prototype that was able to stably collect terabytes of inertial measurement unit (IMU) time-series data, and analyzed multiple candidate parameters derived from them that could be used as reliable biomarkers for ME/CFS and other diseases with similar clinical symptoms.

    Utilizing our system prototype, MetaProcessor, we conducted grouped t-tests on data collected from the EndoPAT study group (55 recruited, 51 participated, 30 ME/CFS, 15 Long COVID, 6 healthy control) to evaluate the predictive power of Upright Position Time (UpTime), Hours of Upright Activity (HUA), and Steps/Day. Through statistical analysis, we were able to assert the following for ME/CFS versus healthy control: 1. UpTime yielded a low p-value of 0.00004, indicating a significant difference between the groups and demonstrating its potential as a reliable measure for differentiating ME/CFS from healthy control populations. 2. HUA had a p-value of less than 0.00004, suggesting it could also serve as a useful measure for distinguishing ME/CFS from healthy control groups. 3. Steps/Day, x-axis and y-axis, had p-values of 0.01059 and 0.08665, respectively, indicating that step count may be relevant for differentiating ME/CFS individuals from healthy controls, but step count alone may not be sufficient to reliably distinguish between these groups. In a linear regression analysis, we found a moderately positive correlation between UpTime and HUA with r 2 = 0.68. Overall, we can confidently conclude that UpTime is a superior overall predictor due to its objective nature and the lowest p-values observed across all groups.

     
  2. Hutan

    Hutan Moderator Staff Member

    Messages:
    26,839
    Location:
    Aotearoa New Zealand
    I note that this is a thesis done as part of a Bachelor's degree. It's a really worthwhile investigation. Well done to the student and their supervisors and clinical and patient collaborators.

    3 measures tested:

    Hours of upright activity (HUA) - standing, or sitting with feet on the floor - self-reported
    Screen Shot 2023-07-11 at 2.51.05 pm.png

    UpTime - time in the day spent with lower legs in an upright position - measured with the sensor
    Screen Shot 2023-07-11 at 2.55.35 pm.png

    Daily steps - number of steps (would be a cheaper measure than using a specialist sensor but may be more unreliable)

    They used MbientLab’s MetaMotionS (MMS), link - small, light weight, worn on the ankle of one leg, accurate, compatibility with various programming languages
    The battery lasts 7 days

    51 participants including ME/CFS (30), Long Covid (15) and healthy (6) cohorts - study of 1 week
    Data from 3 participants had to be discarded

    Collaboration with the Bateman Horne Center. Staff there were trained to brief the participants, and download the data and analyse it, adding metadata (e.g. date, person)
    They used some software to automate the download process, reducing user error.
    Cloud storage of data. There's lot of computer-y details.


    Results for Uptime percentage (sensor)
    Screen Shot 2023-07-11 at 3.20.06 pm.png
    The ME/CFS and Long Covid groups overlap; both are significantly different to the Controls.

    Results for Hours of Upright Activity (self report)

    chart looks similar to the one for Uptime
    Results for Step count
    ME/CFS cohort statistically different from Healthy controls, but Long Covid cohort isn't. There's a big overlap between the three cohorts. It suggests that step count may not be ideal as an ME/CFS outcome.
    (I'm not 100% sure the charts (Fig 3.3 and 3.4) are labelled and captioned right.)
     
    Last edited: Jul 11, 2023
    MSEsperanza, sebaaa, bobbler and 12 others like this.
  3. Hutan

    Hutan Moderator Staff Member

    Messages:
    26,839
    Location:
    Aotearoa New Zealand
    Accuracy of Hours of Upright Activity (self report)
    There certainly is a positive relationship between the self reported measure of hours of upright activity and the sensor-measured percentage of time with legs upright, but it's far from a perfect match. This isn't surprising, as self-reporting activity all day, every day is a demanding thing. The differences may not be all the fault of the self-reporting patients. For example sitting, as I currently am, with one leg bent with the foot on the floor and the other leg crossed over it with that shin resting on the knee of the upright bent leg, would result in a record of me being horizontal, when in fact my torso is upright.

    Anyway, this study suggests that using technology to generate an objective outcome is entirely feasible. Hopefully all research done at the Bateman Horne Center will use activity sensors. Unfortunately, we don't hear anything about how the patients found wearing the sensor.

    I note that the daily hours of upright activity reduced substantially over time, even within that one week. Any measurement of activity must be undertaken for a long time - we have been saying 'at least 3 months', to account for initial enthusiasm eventually being replaced with the reality of disease constraints.
     
    MSEsperanza, bobbler, Kitty and 9 others like this.
  4. Trish

    Trish Moderator Staff Member

    Messages:
    52,210
    Location:
    UK
    I'm not sure that's true. I would agree that it's no use as a diagnostic sign, but we already know that some people with mild ME/CFS would overlap with sedentary healthy people in number of steps. But it could still be a useful part of an objective outcome measure for a clinical trial, since it's the change over time that's relevant, not the starting value. I does however indicate a ceiling effect, since someone with mild ME who recovers to healthy sedentary wouldn't necessarily show significant improvement.

    I do agree that the hours of upright activity is a better measure for clinical trials, and with the importance of measuring for significant time, or better still throughout a trial. It's also a splendid demonstration that subjective estimates of activity levels don't correlate particularly well with objective data, and that people tend to over estimate. That could be very significant in a clinical trial.
     
    MSEsperanza, bobbler, Kitty and 7 others like this.
  5. Hutan

    Hutan Moderator Staff Member

    Messages:
    26,839
    Location:
    Aotearoa New Zealand
    Good point.

    As I said, I haven't yet understood the difference between the two charts of step counts, but here is Fig 3.3. You can see that the mean and standard error ranges for the ME/CFS and Long Covid participants fall almost entirely within those of the healthy participants. I'm not even sure that the healthy participants were matched for age or gender - and there was only six of them, so it's not a great sample. But I think that reported result probably is not atypical when the ME/CFS participants have mild ME/CFS.
    Screen Shot 2023-07-12 at 7.30.14 am.png

    I think the main trouble with step counts is indeed that ceiling effect. People with mild enough ME/CFS that they have the bandwidth to participate in a trial do seem to record a number of steps that, on average, falls within the bounds of normality. We saw that I think with Julia Newton's 2011 study, and most recently for Long Covid in that Knoop Netherlands Recover study, where they found that a large proportion of their participants did not have abnormal activity levels (I haven't checked either of those assertions, my memory might be wrong). Because, unless people are severe enough to not be participating in typical trials that require leaving home, they usually have to do a certain amount of steps to live, and so that essential activity is prioritised, at the expense of, well, everything else.

    People with mild ME/CFS who experience an improvement will not necessarily increase their step count. They might walk faster and with a bigger stride, they might do more cognitive activity, they might take on a sedentary job.

    I agree that step counts aren't useless, I only said that they aren't ideal. I think they probably are most useful for people with more severe ME/CFS, where their baseline step count really is very low. Perhaps distance walked would be a better measure obtainable from common consumer activity monitors than step counts?
     
    MSEsperanza, Michelle, Trish and 5 others like this.
  6. Kitty

    Kitty Senior Member (Voting Rights)

    Messages:
    5,347
    Location:
    UK
    As for step counts: I've run into major accuracy issues with pedometers worn on both the wrist and the leg.

    My ankle-worn pedometer thought that fidgeting was walking. I'm constantly shifting my legs around due to pain, and I'd record dozens of steps when I hadn't even stood up. More recently, my Apple Watch mistook bumping about in a wheelchair for walking, and walking as climbing stairs because of my odd gait. I'm not sure I'd trust ankle or arm-mounted devices, unless they were backed up by another sensor measuring something else, e.g. heart rate.

    The only accurate pedometer I've had was one of the mechanical hip-worn models used by hikers 40 years ago, where you had to measure your stride length and average it out over different terrains. These were good once correctly set up, but you needed to be able to calibrate them in the first place. It'd be totally impractical now that my gait includes small steps, off-balance staggering, and shuffling with both feet on the floor, randomly distributed through the day because the extent of leg numbness, weakness, and dizziness fluctuates so much.
     
    Michelle, Trish and Hutan like this.
  7. NelliePledge

    NelliePledge Moderator Staff Member

    Messages:
    13,254
    Location:
    UK West Midlands
    As a simple non scientist I think that whatever movement the step counter reacts to for that individual if a given intervention results in them moving say 10% 25% or 50% more in months following than in months before the intervention then that’s evidence of them being able to do more.
     
    Keela Too, Michelle, obeat and 3 others like this.
  8. Kitty

    Kitty Senior Member (Voting Rights)

    Messages:
    5,347
    Location:
    UK
    But for some people feeling better would mean less pain, which would obviously reduce their movement. They'd be able to sit and relax between doing things, instead of constantly being driven to move their legs and shift their sitting position, or get up and totter about for a couple of minutes' relief.

    I'm not sure how a study could accommodate patients whose pain is at its worst when they relax their muscles, unless the devices can discriminate between pain-derived and purposeful movement.
     
    voner, NelliePledge, Sean and 3 others like this.
  9. Trish

    Trish Moderator Staff Member

    Messages:
    52,210
    Location:
    UK
    Good point, @Kitty, I get restless legs sometimes at night. That would probably register on a leg worn step monitor as me getting up and going for a long walk. Mind you, I'm likely to do a lot less then next day as it's is still exercise beyond my limit.
     
    Sean, Hutan and Kitty like this.
  10. Kitty

    Kitty Senior Member (Voting Rights)

    Messages:
    5,347
    Location:
    UK
    I've been trapped in a never-ending circle of PEM with it for about the last 25 years, it drives me nuts. It only strikes me quite how unable I am to keep still when someone else comments on it, or if I stay in the caravan with my sis (where the sounds of moving about seem to be amplified, specially at night).

    I occasionally have a run of several hours when there's very little pain, and it's glorious to be able to just sit like a normal person!
     
    Sean, Trish and Hutan like this.
  11. Michelle

    Michelle Senior Member (Voting Rights)

    Messages:
    270
    I have severe ME/CFS and for the most part, step count variations can be pretty significant when you go from 600 to 50 (and vice versa). What's been interesting is that over much of the last year, I've been having an inexplicable improvement. I'm still pretty damn severe (completely homebound, about 600 steps/day) but now I can actually do some gardening or cooking or sewing. I'm still reveling in the novelty of sitting at my kitchen table or walking around my living room.

    What's interesting, however, is that this summer, though my average step count is better than last summer, I'm accomplishing less and suspect I'm spending more time in bed. One of the other metrics I used in addition to step count was time spent out of bed. It used to be easy to count because it was basically enough for my morning constitutional, my evening ablutions, and warming up some dinner--about 1-2 hours/day. Now it's much harder to keep track of. I have a clip on timer that I've thought of trying to use but I already know how that would work: I'd spend all day going "oh crap, I forgot to turn it off/on!" Indeed in thinking about the earlier thread with @sarahtyson , I can understand how she probably found that thread terribly frustrating as I think she was trying to lay out in this post why wearables at this time are still going to be problematic in the research setting but as someone with a full-time job and probably family obligations on top of that simply didn't have the time to spend on the granular details of those problems here on the forum. The research in the OP is extremely helpful in laying out what those problems are going to be while also showing potential technological ways forward. (And that's just movement; we all know cognition plays a huge role in functional capacity too.)

    I've always used a hip-worn Tri-axis model though to be sure, it's never seen any terrain other than my apartment. I've always set it to the very shortest stride length. There can be significant variations from day to day based on how I've felt rather than activity level, which I've always assumed had to do with the fact that I move differently when I feel better. I'm not constantly calculating and economizing movement. I seem to take something approaching a full stride rather than shuffling (and when I'm at my sickest, I'm practically dragging my legs). And it seems to not count my legs moving in bed, only when I'm vertical.
     
  12. Hoopoe

    Hoopoe Senior Member (Voting Rights)

    Messages:
    5,252
    I believe my average daily step count is similar to a sedentary healthy person, or close to it. I'm not sure because I haven't measured it.

    My entire day is structured to spare my physical energy reserves so that once per day I can go out for a longer walk or a short swim. It's also possible because my family does most of the household work.

    When I come back, I'm often visibly exhausted. I'm indeed always operating close to my limit.

    There are a lot of signs that indicate something is wrong with me but the daily step count can be deceptively normal or close to it. On its own it is insufficient.

    If I force myself to rest well beyond what I want to do, my fatigue can decrease a lot. So it can also be within the norm or close to it. It just depends how I allocate my energy.

    I can sometimes also allocate all my energy to socializing but that means giving up the daily walk. If I have to do significant household work, it means reducing or skipping the daily walk. And so on. Perhaps this is a pattern with diagnostic value.

    And once I have done my daily activity, whatever it is, the rest of the day will be at most low intensity activity.
     
    Last edited: Jul 12, 2023
  13. Hutan

    Hutan Moderator Staff Member

    Messages:
    26,839
    Location:
    Aotearoa New Zealand
    Thanks for your post. I think though that this study illustrates clearly that there is technology available that does a good job of distinguishing people with ME/CFS from healthy controls and that it is not at all problematic in the research or clinical setting. Yes, we need more research to fine tune the measures used here, and to investigate others such as acceleration and gait.

    This study also illustrates very clearly that patient reporting, even when data is collected daily and even when the reporting is of an objective thing like time spent upright, is inaccurate.

    I understand that @sarahtyson has a particular skillset that she wants to apply to the problem of quantifying patient outcomes, and perhaps some good may come of it. It is a shame though that she seems to have chosen to dismiss what was said here and push on with a project that is not as useful as it could be. It is a shame that the UK MEA isn't better informed when it comes to allocating scarce research dollars.

    We need a lot more wearable technology and a lot less patient reporting of activity levels and fatigue, both as outcomes in research, and in clinics for monitoring disease progress.
     
    obeat, SNT Gatchaman, Dolphin and 4 others like this.
  14. voner

    voner Senior Member (Voting Rights)

    Messages:
    222
    Dolphin likes this.
  15. Michelle

    Michelle Senior Member (Voting Rights)

    Messages:
    270
    I don't disagree with any of this but, to be fair to @sarahtyson, she might say the same thing about us. She might argue that we were dismissive of the technological and institutional infrastructure challenges that she outlined in this post in the 20 years she said she had been trying to incorporate wearables (there were other concerns with that study that she's involved with but which are not directly relevant to this thread so I'm not going to comment on them). The study in the OP here is very promising but it is only an undergraduate thesis. While we are all very rightly sick to our eyeballs of subjective questionnaires and they should absolutely be downgraded as tools to measure functional capacity (preferably to the darker recesses of Hell), that does not mean incorporating wearables is as simple a matter as we were insisting to her. I don't know her beyond what she posted here, but I assume that, like most of the researchers whose work we discuss here, she simply lacks the time to spend in extensively posting on her area of research.
     
    MSEsperanza, Trish, ahimsa and 2 others like this.
  16. Hutan

    Hutan Moderator Staff Member

    Messages:
    26,839
    Location:
    Aotearoa New Zealand
    I guess this is the difference between our viewpoints. I see wearables as currently adequate technologies that can be incorporated into research and clinical monitoring right now. They may not be perfect, but they are better than subjective questionnaires where people are asked to summarise their level of functioning or fatigue over the past two weeks/months. Members here routinely monitor their illness in a useful way with commercially available wearables.

    It's not like this is a new area. The fact that an undergraduate thesis can make a useful contribution illustrates the maturity of the technology with the various tools that are at hand to make things work.

    It's great if Sarah has previously tried to use wearables, and a desperate shame if her efforts have been frustrated. I am certainly not dismissive of the institutional obstacles that non-BPS ME/CFS researchers have faced. But clearly her experiences continue to colour her viewpoint now. Her assessment of wearables as 'promising but not suitable for use right now' is objectively wrong.

    See, for example this 2017 paper (5 years ago, there have been huge advances since then) about wearables in multiple sclerosis.
    Wearable biosensors to monitor disability in multiple sclerosis
    The problem of the Hawthorne effect is solved by having a long assessment time. And it's hard to see how the Hawthorne effect from wearables would be worse than that applying to retrospective participant reporting.

     
    Last edited: Jul 13, 2023
  17. Trish

    Trish Moderator Staff Member

    Messages:
    52,210
    Location:
    UK
    I think there is also the practical problem of getting patients to buy and use consistently wearables that can be expensive and inconvenient for people already struggling to keep their heads above water financially and in all aspects of life.

    As I understand it the primary purpose of Sarah Tyson's project is for patient record keeping to enable some consistency over time and to alert doctors to the range and severity of symptoms and levels of disability so they can give appropriate care. In that context, wearables, particularly in an NHS short of funding, are a bit of a luxury. They would be good to include, but are not so likely to be the core of patient data for some time to come in the NHS.

    On the other hand for research, I think the current and future generation of clinical trials should see them as an essential part of outcome measuring.
     
    RedFox, Michelle, obeat and 2 others like this.
  18. Hutan

    Hutan Moderator Staff Member

    Messages:
    26,839
    Location:
    Aotearoa New Zealand
    The cost of a basic consumer fitness tracker is pretty minimal (less than the cost to the government of an hour of individual CBT, I expect), and they often can run for two weeks without charging. They are much less hassle than trying to write down how much activity is done.

    I think they could be given to people with ME/CFS during the diagnosis process - wear the fitness tracker for a month, keep an activity diary and then the doctor has some good information to work out if you have PEM or sleep issues or if your heart rate is spiking at odd times, and the nurse has some good information to work out how to help you optimise activity levels.

    If the NHS really is that poor that it can't give people with ME/CFS a fitness tracker, they could just give people a strap, and get the tracker itself returned. But, like I said, the trackers don't cost much, and the NHS could negotiate a good bulk deal. And really? They can waste money on ineffectual CBT and mindfulness courses but they can't provide a fairly inexpensive tool that helps the person manage their illness? I would have thought that the NICE ME/CFS guideline shows that energy management is pretty much all that we have, in terms of ways to reduce symptoms. People with other diseases get thousands of dollars of medicines every year. I'm sure that there are mechanisms in the welfare system for disabled people to get financial assistance for equipment that helps them function.

    Maybe there are issues, but I think that working on addressing them is more useful than work that props up a monitoring system in clinics that relies entirely on subjective patient reporting.
     
  19. Trish

    Trish Moderator Staff Member

    Messages:
    52,210
    Location:
    UK
    I agree with you, but I doubt it works that way in the NHS. We'd need the NICE guideline to recommend them as a key part of management.
     
    Sean, RedFox, Michelle and 2 others like this.
  20. Michelle

    Michelle Senior Member (Voting Rights)

    Messages:
    270
    I don't think cost is the problem. My Tri-Axis pedometer, which seems to be more accurate than, say, a FitBit, is only about $25. I wear it clipped to my underwear 24/7, except when I'm bathing/showering. The one I've got now has lasted me since 2018. The only ongoing cost has been a 2032 battery once a year or so.

    Based on my reading of @sarahtyson 's post (again, here), the problems are more institutional in nature. Which wearable -- and brand of wearable--are you going to use? What, specifically, are you measuring and why? How do you know that a specific wearable is going to give you the information you're looking for--i.e. is that wearable validated for that use? These are just a few of the questions I could think of off the top of my head and there are probably many, many more that I wouldn't even think to ask because I don't design studies for a living. Studies like this one in the OP are absolutely vital to answering those preliminary questions so that wearables can be used in studies.

    It depends on what "suitable for use right now" means, though I suspect it's more institutional than technological. You are absolutely correct to keep pushing researchers like her to figure this out and it's a shame that we can't hear more specifically from her about how this works in the "real world" or compare and contrast how they are being used in other diseases like MS as you referenced above.
     
    MSEsperanza, obeat, Trish and 4 others like this.

Share This Page