A new "reasoning" framework for ME/CFS research

[mariovitali]

Dear All,

I would like to provide details about the existence of a new reasoning framework, available for ME/CFS research. In this post I will give a brief description and a post will follow with the hypotheses that have been generated.

This framework uses a number of Large Language Models (LLMs), Network Analysis, Information Retrieval methods and several other algorithms in order to generate causal hypotheses for ME/CFS and other syndromes.

As input, the framework uses text from related studies which usually exists in the "Results" section. The input data do not mention anything about ME/CFS, Long-COVID, "Infection-associated syndromes" or other related concepts. This is done so that LLMs are not biased with what they already "know" for ME/CFS and related syndromes. Additionally, a number of different triggers are given such as "organophosphate exposure"

Here is snapshot of the reasoning process. Notice how the reasoning uses given knowledge (e.g. FGF21 found to be elevated throughout more than one studies, Cholines found to be lower, organophosphate exposure is a trigger, ADHD and EDS can co-exist) :



The framework uses several reasoning engines to do this (shown above is just one reasoning engine, namely o3 from OpenAI). As soon as all LLMs give their output, another reasoning engine compares the answers and identifies commonalities, differences and novel information among those answers.

The great thing is that the reasoning process can be repeated using different layers of information : For example, in one version of the reasoning process no triggers are given, just the findings. In another version we may add patient symptoms or use different triggers. In this way we investigate how different layers of analysis change the identified cause(s) of the syndrome in question.

I am open to any suggestions as always and -if possible- your help regarding the information which is used for input. For example, one may argue that elevated FGF21 should not be used in the input.

Shortly in a separate post, the causal hypothesis found so far.

 

[Utsikt]

The input data do not mention anything about ME/CFS, Long-COVID, "Infection-associated syndromes" or other related concepts. This is done so that LLMs are not biased with what they already "know" for ME/CFS and related syndromes.

Out of curiosity, can’t the models just use proxies for ME/CFS and therefore still be biased?

Like how insurance companies still discriminate based on gender because «18-25 years old and first vehicle is a motorcycle» pretty much always means that the customer is male.

 

[mariovitali]

Out of curiosity, can’t the models just use proxies for ME/CFS and therefore still be biased?

Like how insurance companies still discriminate based on gender because «18-25 years old and first vehicle is a motorcycle» pretty much always means that the customer is male.

Which conditions are considered proxies to ME/CFS ? Fibromyalgia for example?

 

[Yann04]

Which conditions are considered proxies to ME/CFS ? Fibromyalgia for example?

Not necessarily a condition, just simply that words researchers use in the “results” section (“syndrome”, “fatigue”, “female predisposition”, “elusive pathophysiology”, “multisystem”, “viral persistence”), even if not immediately evocative of ME might get the LLM to activate lots of the same “circuits”/trails of thought it might have done if you just said ME.

But LLM’s are made on bias, you can never fully stomp it out. And I think the not mentioning ME or similar keywords is a pretty good first step to minimise the bias.

 

[Utsikt]

Which conditions are considered proxies to ME/CFS ? Fibromyalgia for example?

Essentially, anything that correlates with the use of the term «ME/CFS» or similar can be used as a proxy for the term «ME/CFS».

But LLM’s are made on bias, you can never fully stomp it out. And I think the not mentioning ME or similar keywords is a pretty good first step to minimise the bias.

It’s necessary, but I would say it’s far from adequate, as my insurance example illustrates.

 

[Utsikt]

I also wonder what the value added is here?

If the model can’t know about things we don’t know already, how can it be used to solve a puzzle if the puzzle requires pieces that we have not discovered yet, like an unknown mechanism or characteristic?

Does it inherently assume complete knowledge?

 

[Yann04]

It’s necessary, but I would say it’s far from adequate, as my insurance example illustrates.

I mean this framework isn’t doing anything like replacing researchers or outsourcing critical thinking really is it?

The way I understand it its just more seeing if the AI offers any sort of rabbitholes/different thought patterns or ways of exploring the results that can then be looked at. So really it doesn’t matter if 98% of it’s suggestions are useless in that this is being used to kind of dig for that 2% to kind of see if it can offer different perspectives that might be valuable for interpretation of findings.

Of course, if it’s in the hands of someone who reads everything it outputs as gospel, thats a problem.

 

[Creekside]

Personally, I think it's a wrong pathway, since I haven't noticed any symptoms I'd expect from body-wide mitochondrial dysfunction.

 

[Adrian]

I would like to provide details about the existence of a new reasoning framework, available for ME/CFS research.

Have you seen googles co-scientist Accelerating scientific breakthroughs with an AI co-scientist

 

[Utsikt]

The way I understand it its just more seeing if the AI offers any sort of rabbitholes/different thought patterns or ways of exploring the results that can then be looked at.

Yes, but if the data doesn’t contain the necessary information to create the solution, AI will never find it. And you end up just looking through loads of interpretations of what we already know.

The key question, to my mind, is: do we have reason to believe that we have enough information already, or are we looking for an unknown unknown?

 

[Creekside]

The key question, to my mind, is: do have reason to believe that we have enough information already, or are we looking for an unknown unknown?

I think I'd vote for unknown unknown. It's possible that at least part of the answer for ME exists in the present database, but if it's so off-track that humans haven't noticed it, it might be hard for an AI to spot it too. If, for example, there was a correlation between one cytokine, one protein fragment, and a pattern of brainwaves during sleep and vagus nerve signals while awake, that might be drowned out by millions of similar correlations that have nothing to do with ME.

Millions of monkeys at typewriters might produce the complete works of Shakespeare, but it would also take millions of educated humans to read each pile of papers to figure out which one was the desired result. The AI would need some way to test each correlation for possible validity, and would probably still produce millions of possibilities that "need more data".

I'm guessing we still don't have enough data on brainwaves during different periods and different situations and different severities to reveal correlations. Likewise for vagus nerve signals or other parts of the body that might be involved with ME, but aren't as popular as blood tests and mitochondria tests. Repeating post-exercise blood tests another thousand times won't help if the answer isn't in the blood.

 

[mariovitali]

@Utsikt regarding the following :

Yes, but if the data doesn’t contain the necessary information to create the solution, AI will never find it. And you end up just looking through loads of interpretations of what we already know.

What AI can do is to better connect the knowledge we already have, better than any human. As an example, Telcos wish to find subscribers that have a lot of social connections. If I give to you or any other human the Call Detail Records (CDRs) of thousands of callers you are unable to answer which of these callers is considered a "supernode". For this reason we use computers to make these connections for us.

In the same way, there are so many known interactions between medical concepts that we know about but we still do not know how these interactions lead to a given result. AI can definitely help us on that.

 

[Adrian]

Yes, but if the data doesn’t contain the necessary information to create the solution, AI will never find it. And you end up just looking through loads of interpretations of what we already know.

The key question, to my mind, is: do we have reason to believe that we have enough information already, or are we looking for an unknown unknown?

I think this is the big problem. AI may also not be cynical enough when summarizing papers unless you have prompts that explicitly ask to assess the methodology.

There are areas where AI could help such as drug repurposing which is where the google co-scientist is targeted. But I suspect there isn't enough research (with sufficient sample sizes) for this to be useful.

 

[Adrian]

I'm guessing we still don't have enough data on brainwaves during different periods and different situations and different severities to reveal correlations.

I have been wondering recently if there could be something in monitoring brainwaves (there are some available datasets for non-ME people). But I wondered in terms of monitoring mental activity. However, I'm not sure how cumbersome the sensors would be as they would need to be on several places on a head.

 

[Utsikt]

In the same way, there are so many known interactions between medical concepts that we know about but we still do not know how these interactions lead to a given result. AI can definitely help us on that.

If the data doesn’t contain the interaction we need to solve the problem, AI will never find it because AI can’t create information, only rearrange it.

In your example you already know that there probably are supernodes in the data so all you have to do is to classify the nodes!

 

[mariovitali]

If the data doesn’t contain the interaction we need to solve the problem, AI will never find it because AI can’t create information, only rearrange it.

In your example you already know that there probably are supernodes in the data so all you have to do is to classify the nodes!

(1) "If the data doesn’t contain the interaction" - This is an assumption. How do you know it doesn't ?

(2) "AI will never find it because AI can’t create information" - In other contexts AI does create information but we will talk about medical science here.

(3) "In your example you already know that there probably are supernodes in the data" - We do not know this apriori. There may be clusters of a given subset of subscribers that no supernode exists. Moreover, companies have different cutoff levels on the number of connections that characterise a supernode. So it is not that simple.


In ME/CFS and LongCOVID research we have found issues with Fibrinogen, P-Selectin and Collagen. What do they all have in common? They all require glycosylation. This is derived knowledge and if indeed some ME/CFS patients have glycosylation issues, it could explain a significant number of their symptoms. This concept generalisation was made by AI and not a single human throughout the years thought about it.

I hope it's all crystal clear now !

 

[Utsikt]

(1) "If the data doesn’t contain the interaction" - This is an assumption. How do you know it doesn't ?

How do you know that it does?

In ME/CFS and LongCOVID research we have found issues with Fibrinogen, P-Selectin and Collagen. What do they all have in common? They all require glycosylation. This is derived knowledge

Derived knowledge is always limited by the knowledge that it’s derived from.

 

[mariovitali]

@Utsikt I hoped that my examples would be obvious for you, clearly there weren't given your replies. It would be great if anyone else would like to comment so we have other opinions.

 

[Utsikt]

@Utsikt I hoped that my examples would be obvious for you, clearly there weren't given your replies.

It seems like you’re stuck on the semantics of «knowledge», so let’s stick to «data» instead.

Your example on glycosylation did not create any new data. It only highlighted existing data. Maybe it’s correct that nobody had explicitly mentioned or thought of the commonalities between Fibrinogen, P-Selectin and Collagen, but the characteristics of those were already in the data. So the AI model only rearranged the data. It didn’t discover the characteristics of Collagen, it just spotted a pattern between the already existing data.

My question from the start has been: how can this model help if the data that’s required to explain ME/CFS (or any other condition for that matter) isn’t already in the data that the model has access to?

What if ME/CFS is caused by a mechanism that we simply have not discovered yet?

In such a scenario - how can this model help, in your opinion?

 

[rvallee]

Yes, but if the data doesn’t contain the necessary information to create the solution, AI will never find it. And you end up just looking through loads of interpretations of what we already know.

The key question, to my mind, is: do we have reason to believe that we have enough information already, or are we looking for an unknown unknown?

I would be shocked if we don't have the data we need already. Probably have for decades.

The problem may actually be the opposite: too much data. Because the search space hasn't been restricted yet, it's still far too vast for human intelligence and labor. A lot of the key pieces are probably in patient testimonies, small bits of what makes scientific breakthroughs more than any other thing: "uh, that's unexpected".

But our current methods of working with this are about as completely inadequate as human labor working on manual switches would be able to route modern Internet traffic and make video streaming work. When you look at qualitative studies 'exploring' the patient experience, they only ever have at most a dozen participants, and even that is just too big, it always ends up being compressed into a bunch of useless output. Plus those studies can never be cross-referenced with objective laboratory data.

The scale of the work needed here is orders of magnitude higher than what modern medicine can do so far. But they can't figure out that this is the problem, or how to get there. These things usually just happen, low-hanging fruits, or happen much later once a discipline matures and the early manual labor gets automated.

There are likely a few missing pieces, but it's only possible to get there once the search space has been massively reduced, like knowing which square kilometer of space to dig for a treasure, compared to having to search the entire surface of the planet without a hint of where it could be.