A new "reasoning" framework for ME/CFS research

Take my black and white example above.

No matter how you rearrange black and white - you can’t get red. You can get an infinite amount of types of grey, but you can’t under any circumstances create red or blue or yellow.

If the part of human biology that is ME/CFS requires red, the AI will never be able to paint the right picture.

I’m not discussing if it can create a picture, but what kind of pictures it can create based the data it has available. The black and white ones are not new data per se, just rearranged data.

I have no said anything about «improving patients lives». I have spoken specifically about solving the mechanisms of ME/CFS, because your first post said «for ME/CFS research».

What happens when the hospital expands and you have an additional operating room? If you don’t give the model that data, it can’t account for it.

Or what happens if a new disease is discovered? Or a pandemic breaks out and you need completely different rules of prioritisation?

I’m not saying that this makes AI useless. Just that it strictly limits what AI can and can’t do, and we have to be aware of those limitations.

Last example: if you train an LLM on text that doesn’t contain any O’s, it will never be able to write the word «Open». Or if the text doesn’t contain any capital letters - it will never be able to write «ONE». Or if it doesn’t contain punctuation - it will never be able to write «…».

So you can’t expect it to write a letter using correct English grammar.

 

"rearranging" data can certainly create value. We don't know if we need any red in the picture. Maybe all of the needed data has already been generated and you only need to find the right connections - who knows? AI can certainly help with that. It might even be able to suggest experiments that could validate it's hypothesis, if it's trained on enough methodology. As you say, the model is only as good as the data is trained on. At this point there's no way to know if the current data is already good enough or not. From what I get, you sound like you're assuming the current data is not sufficient, while Mario is actively trying to use the data to see if it's sufficient.

Feels like you guys are talking on different levels - and taking it a bit too far.

 

I’m talking about what kind or value it can provide, and under which circumstances.

It seems very unlikely that whatever ME/CFS is, is a known unknown, especially considering the abysmal quality of most research.
Edit: and the complexity of human bodies.

I don’t think it’s taking it too far to question the fundamental assumptions of an approach, and how those assumptions limits and affects the results.

 

I completely agree that a general LLM cannot "solve ME/CFS" for the reasons already discussed above.

My interpretation of what you're saying is: We haven't gathered the data yet that would allow us to solve ME/CFS. Is that correct? In that case at least on a theorectical level it could still point us where not to look (on a practical level I do think this is currently impossible, at least in the general approach used here, for the reasons already discussed above), on a practical level there are of course other problems related to the data (for instance in relation to reliability), but of course you could also just call that a problem with data.

But I don't think your examples are quite bringing that across on a philosophical level. Humans use capital letters, presumably as useful signals in text. It's very possible that a (more sophisticated than currently existing) LLM can infere that as well even if that majority of the time it might introduce a signal that is different from our use of capital letters (of course it would also understand what a capital letter is and how to use it simply if your data set describes the notions without using them, but that's besides the point your making).

I see the colour example having a similar flaw: Humans cannot imagine a colour outside of the visible spectrum, so you could just as well argue that the solution to ME/CFS is written in ultraviolet we cannot ever get there. That is of course not true because we have understood colours in a different way, even if we cannot see them in from of our eyes directly. So in some sense this is also nothing else but rearranged data. These are all inferences a LLM can (theoretically!) also make and obviously it can conclude that if the solution is neither black, white, nor grey that it must be of an alternative form, which would already be a very valuable contribution. In fact this is the only contribution I currently see it would be possible be able to make but it would require a tremendous amount of work and might even still be outside of the scope of possibilities.

Like I have already said whether there are fundamental differences in these learning process between humans and computers is a highly debated question, but is not relevant to whether there can be use cases of it. Perhaps the father of it all, von Neumann once said "Young man, in mathematics you don't understand things. You just get used to them." and while you certainly can claim that I never understood this quote but rather just got used to it, I do like it to illustrate how similar things can often be, especially when it comes to use cases.

The hospital example is of course not a good one to bring across a use case, because it is not a life-saving intervention but a cost-cutting intervention. I suspect image recognition and solving inverse problems in cancer imaging is the more fruitful example here, albeit not of relevance to the discussion of a general LLM providing a solution to ME/CFS.

But I'll end the debate here, because I agree that I currently can't see any use of a very general LLM in this context, but I'll be extremely happy if someone manages to prove me wrong and I think all that can be said, has already been said.

 

My opinion is that we probably don’t because of the complexity of human bodies. But my opinions doesn’t matter - it’s @mariovitali that has to defend why he believes the opposite is true if this model is going to be able to be useful «for ME/CFS research» (I took that to mean trying to figure out what ME/CFS actually is).

I don’t understand how it could do that in theory, but it probably doesn’t matter if it can’t be done practically anyways.

In this context, it doesn’t matter if it can learn on the go when encountering something new, what matters is if it will encounter the necessary data to be able to create the words that are needed. In other words: have humans already generated the data we need to solve ME/CFS?

Black and white can be represented using values between 1 (white) and 0 (black), as a monochrome scale. You can’t ever represent red from those values, because you need a combination of e.g. RGB.

Humans know that all colour can be created through RGB, and all light through different wave lengths, but AI doesn’t if you only tell it how to create black and white through monochrome scales.

If you tell an AI how to make 100 different colours with RGB, or let it learn from RGB values, it might be able to make another 100 if prompted. But that would just be rearranging the data it already has. It didn’t create the knowledge of RGB as a concept - it was given it in its data.

And if you give a model only RGB values for black and white, certain algorithms might be able to learn to make other colours through exploring. But you still gave it the possibility to use RGB values.

So at the risk of repeating myself too many times: the question is: are we trying to make red with a monochrome black and white scale?

 

Even though my understanding is that all visible colours cannot be created through RGB, that is not the point. The point is that an AI, in the theoretical sense, can certainly come up with the concept of RGB, this does not require a large jump. Just as much as von Neumann was able to do math even if he himself claimed to have never jumped. Theoretically at least. But yes, for ME/CFS I agree that there is an absence of good data for any of this to currently have value in this very general setting. How big the theoretical possibility of jumps of AI vs humans is, is of course the more intriguing question.

At least currently in this model, I would tend to think that the amount of jumps is probably close to non existent, so I do see it becoming a debate on the data but I think everything that can be said has been said on that front.

 

No, it can’t unless it already has data that contains the properties of colour, either directly or indirectly.

If we don’t have data on how something works, and we don’t have data that can be used to deduce how something works, we can’t ever knowingly describe it (other than with pure randomness and infinite time, but that has no practical application).

It’s mathematically impossible due to a principle that I have been trying to remember the name of, but I can’t. So I don’t expect anyone to take my word for it.

It’s e.g. described in a book by particle physicist and AI researcher Inga Strümke, but I only have the hard copy so it’s difficult to search..

 

In all honesty I think it is quite a vague conclusion. Glycosylation is simply the process of conjugating proteins and carbohydrates, it’s a broad biological process with extremely wide applicability in various biological contexts. The connection between fibrinogen, collagen, and P-selectin is already quite obvious to any researcher with basic knowledge of the extra cellular matrix—glycosylation of the first two is what gives them rigidity for the ECM, and the latter is an anchoring transmembrane protein.

It’s a bit like saying “what’s the link between a peanut butter and jelly sandwich, a BLT, and a ham and cheese? AI told me that they’re all edible.”

The problem is also how integrated the ECM is in many many biological processes, such that a perturbation anywhere else is likely to have some degree of knock on effects in the ECM. Same goes with things like “proteasome” and “translation” which also come up often as “themes” if you just feed a large list of genes.

Glycosylation may very well be relevant somehow to ME/CFS, but the particular findings you point to don’t give any additional information beyond acknowledging that vague possibility. I suspect that if there were more details for why it might be particularly relevant in ME/CFS, it would already have registered in the minds of any decent researcher reading ME/CFS studies. And I say that even with full acknowledgment of how disappointing scientists in the field can be.

 

I think I found it:

ML assumes the the conditions for universal approximation are fulfilled, i.e. that there is a connection between the data and the problem we are trying to solve. If there is, ML can in theory solve the problem with neural networks if there are enough neurons.

But if the data is wrong, or if we don’t have data that is connected to the problem, or if we don’t have enough neurons, it’s impossible to solve it with ML.

If the data is almost good enough, ML might get us close enough that humans can make the leap. But that’s not a guarantee at all, neither the close enough or the leap.

So it’s still a question of:
Do we have correct data?
Do we have the right data for our problem?
Do we have enough neurons in the model?

 

I suspect we are talking past each other. Maybe it would be interesting to know the name of theorem you are talking about so that I understand what we are attempting to discussing, or at least me for that matter, but don't loose any energy over it.

 

Glycosylation is everywhere and its significance is the creation of a shape. Since every shape is different I don't see it as being helpful

I remain very sceptical about use of AI, other than as a literature source.

Working out how diseases work (which was my career) is a bit like working out how to invent an electric toothbrush. You have to dream up a way things could work and then see if they do. It is just that you are dreaming up how a disease works rather than a household appliance.

I am not aware of AI inventing anything. Maybe it has but what?

 

We posted at the same time. See my comment above

 

I think the main disconnect is between what an AI model theoretically could do, and what it actually ends up doing. Many of my rabbit holes are things that are, in an abstract sense, possible for an AI to figure out.

It would be extremely useful if e.g. AI was able to look at existing papers, pinpoint ones that used tissue samples that are likely to show a particular phenomenon that other literature hints at, screened the methods to see if any of them were likely to pick up on that phenomenon even if it wasn’t intended to be measured, and then determined if the rest of the methodology was adequate to actually draw any conclusions from.

Purely theoretically, that is something that a very well trained model might be able to do in specific circumstances.

But in all my experience, it never even comes close. The sheer amount of training and prompting needed to get something that detailed and useful basically requires me to already know what I’m looking for and keep pushing beyond the repetitive and generic output at every step. And anything more specific than generic overviews requires me to spend a lot of time verifying what the AI tells me, as it’s more likely to hallucinate those little details or pull them completely out of context.

I would really like to be gung-ho about AI for these purposes but the reality is that it’s just not even approximating the tasks that would be useful for ME/CFS research. It can give a decent impression of generating new knowledge if you’re assessing its superficial ability to find connecting words and concepts between published manuscripts. But my sense is you’d be much better off spending time training one average grad student than one AI model.

 

AI has essentially invented new tactics in chess that no humans have ever conceived of.

But chess is a problem where everyone works with the same correct data, so it’s niche in that sense.

 

Ah ok, yes you're talking about the universal approximation theorem, which I had vaguely mentioned a few post earlier ("one hidden layer can suffice as already proven since the 80's" which was the originally proved in a certain setting by Cybenko and later by Hornik, I can remember once having to learn the latter proof for an exam) which is just an existence results allowing for an approximation with the possibility of exploding weights and of course the number of neurons may depend on the approximating function and so forth. The results essentially state that for a given function (in fact things can get so ugly that the objects are not even objects that go by the name function anymore) there exists an approximating function. I have no idea how things behave for things that are a bit more sophicasted and which would perhaps be viewed as state of the art use today, I only know that Terry Lyons has an interest in this and has proved some approximation results as well.

But I don't really see how that is relevant to what I have been discussing nor to what @mariovitali is doing. The approximation theorem is a rather seperate result from problems of having the right data for ME/CFS, which much as others, I don't see existing.

 

I doubt it's necessary to involve AI in any of that and I wouldn't consider that an "invention" as it doesn't involve "jumps". Brute force would be entirely sufficient, as long as we don't care about energy resources, which as AI has shown, we clearly don't. The question is more so whether meaningful abstractions can be learned by getting a feel for other abstractions elsewhere. A biologist might say no, but perhaps a knot theorist would say yes, who knows. At least for current applications it seems largely hype accompanied with some very specific useful applications, which doesn't mean that it won't change a whole lot, not necessarily in positive ways.

 

That does not count because you know what the rules are. For inventing disease mechanisms you have to guess when the rules aren't.

When inventing a bicycle you have to realise that things balanced on a line do not necessarily fall over and you then have to work out how to contrive conditions where they don't.

 

Wouldn’t the UAT be one of the reasons that the lack of the right data is a problem in the first place?

Isn’t chess too complex to brute force with current computers, even without energy limits? So the point was that AI got us to that insight today, way earlier than we would have been able to do with other methods. But we’re off track, mostly because of my ramblings! Apologies for that.

I agree.

I agree that it’s not applicable for figuring out disease mechanisms. I used it as an example because you said «anything». I should probably have understood that it was intended as «anything in this context».

 

Sure, totally understand but I did not ask you about glycosylation itself. My question is : despite all research being done, are we sure that glycosylation works as expected in ME/CFS patients? Maybe again, this is a vague question so I will try to be more specific. Please have a look at the following research project and let me know if such research is another waste of money and effort, given your knowledge about glycosylation and its potential relevance to ME/CFS :


Title : Exploring the Glycome: Identifying Glycosylation-Related Biomarkers for severity stratification in Chronic Fatigue Syndrome

https://mrr.mecfs-research.org/en/projects/SGeaUqNxSw6EazXlnqpZVQ

Many Thanks for your time.

 

This thread gave me an odd idea: what about a project to build a model of a human in software, down to the sub-cellular level? It won't work properly because we're missing knowledge of some components and interactions, but the failures in that model could point to what's missing.