Why Chronic Illness Patients Feel Safer Talking to AI Than to Doctors

[rvallee]

I agree with the concerns but also recognise benefits. Do people think AI really carries more risk than any other "tool" that has ever been created?

What I see mostly is people who don't think that AI will improve enough, and are judging its final value entirely based on what they've seen so far. And I don't mean here, this is what's happening in general.

From that perspective, it kind of makes sense. Ironically, the same mistake psychosomatizers make: every model is coherent with itself when evaluated on what the model says. Current AI is still too limited. It has massively improved since the first loudly publicized blunders, but those blunders created a sort of checkpoint past which most people can't imagine it will get any better, even as it already has. But since it's currently unable to fully replace people, it will never be able to do that. Fallacious, but consistent with itself.

All of those issues go away the second a medical AI is as good as an average physician. At this point medical AIs will already be 100x better than the current system, because availability is a major component of performance, and instant 24/7 availability isn't just transformative, it's completely revolutionary. Which is also the point at which any AI is probably capable of replacing most non-manual jobs.

But it's not ready yet, therefore it will never reach that. Comically famous mistake. And ironically, not especially different from "we don't know what causes ME/CFS, therefore we will never find anything, and that means it doesn't actually exist". It's definitely funny how humans can be

 

[rvallee]

Engaging with human counsellors and therapists has likewise caused harm to some people. Engaging with the officially sanctioned medical system has widely caused harm though history, and I don't mean just as regard "illegal" illnesses that haven't been officially "medically sanctioned". Medical errors happen, which result in death or injury or harm. I've certainly been harmed by official medicine. I have a friend who died due to medical error. And in the latter two cases there has certainly been no accountability or responsibility.

This has been pretty damning as well. Seeing all the talk about how people have committed suicide because they were encouraged by AIs and how they should be banned as a result, and yet we know, for a fact, that it's common for sick people with disabling chronic illnesses to commit suicide because of the total lack of support and denial from health care, and... no one cares but us. And none of this is because of edge cases where someone went rogue, but is explicitly encouraged, even defended at all costs as perfectly good.

Biopsychosocial/psychosomatic ideology has killed and ruined far more lives than AIs have, and likely will unless it's used to commit atrocities by people in power, but is unlikely to happen entirely on its own, an AI alone doing that for its own motivations. And it goes completely unchallenged, the harm is completely denied.

Humans are so freaking weird.

 

[Creekside]

Do people think AI really carries more risk than any other "tool" that has ever been created?

One difference is the availability of the tool. You can buy lab equipment and do self-diagnostics or create your own pharmaceuticals, but neither those tools or the knowledge to use them is readily available. Medical specialists can be considered a tool for diagnosis and treatment, but are also not readily available (could be a multi-year wait). AI medical advice is at people's fingertips. Furthermore, doctors have some (flawed, inadequate) systems in place for quality control, while such systems for AIs have yet to be developed. There will be news reports about harm from these new tools, but that feedback leads to improvements. Weren't there deaths and injuries from airbags, anti-skid systems, and other such new safety features?

I don't see AI as being more risky; all new tools have risks and benefits and civilization simply has to adapt to them.

 

[Sly Saint]

Article on use of Chatgpt for medical issues:

Another draw to AI for some people is its perceived lack of judgment. ‘The most comforting thing is that it doesn’t judge me about my questions. It doesn’t roll its eyes or say “that’s an absurd thing to worry about”.’

Edward, 33, has spent years using the app for health advice, but says it’s his friend’s recent hospital stay for a heart attack that really tells the story of why it’s so useful.
‘My friend, in his late 70s, plugged the medical report into ChatGPT after he felt a bit disenchanted by the bed manner of the doctor,’ the AI advisor explains.

‘The app, however, was empathetic, told him exactly what the report was saying, and how severe it was in a gentle way.’

It’s not just patients embracing AI, either. More than half of GPs use it in their clinical practice, according to a survey by the Royal College of General Practitioners (RCGP) and the health organisation Nuffield Trust.

However, it’s worth pointing out that most use algorithmic systems for note-taking, rather than clinical judgement.

Dr Becks Fisher, the director of research and policy at the Nuffield Trust, says GPs like her ‘expect’ patients to come in armed with ChatGPT advice.

‘It’s very difficult to make broad generalisations about the accuracy of AI tools, partly because how useful they are will depend on what information the user prompts them with,’ she adds.

MSN

 

[NelliePledge]

From this I take that AI removes the tone of voice, eye rolling, laughing at suggestion I’ve received from in person GP interaction. The misleading of saying to your face we don’t do GET by CFS clinic staff. It wouldn’t remove the dismissal or downplaying inherent in the wording eg churning out BACME pacing up, emphasis on anxiety, avoidance etc.

 

[Trish]

The misleading of saying to your face we don’t do GET by CFS clinic staff. It wouldn’t remove the dismissal or downplaying inherent in the wording eg churning out BACME pacing up, emphasis on anxiety, avoidance etc.

I see no reason why AI will be any better on these recommendations. Who knows what nonsense it might generate, depending on thte quesitons the person asks. And I can do without the pseudo empathy from a machine.

 

[Arfmeister]

Let’s be clear if you know how to prompt AI well it will be more knowledgeable on ME/LC and PEM than +99% of the doctors out there.
Always ask for a reference papers (and let AI scavenge the internet for patient experiences)
And cross check S4ME forum for a second opinion.

Then hope to find that 1% doctor that’s willing to discuss these results and treatments with you.

With the speed things are now, I think doctors will start using AI more widely in less than 5 years.
- still the bottleneck is the amount of research and papers published that feed AI

 

[alex3619]

But, especially this early on in its development, there is way too much danger of it just being a more efficient version of spreading and reinforcing the same old ignorance and prejudice of living humans, with a superficial veneer of being more objective and neutral and authoritative.

GIGO still applies. Garbage in, garbage out. If the AI feeds on medical opinion then the quality of the opinion matters. As does the quality of studies. As does the volume of studies. If poor science dominates due to financial or other interests, it will have a tendency to bias the AI response. AI is a fantastic tool, and getting better, but it currently needs a lot of real world grounding by qualified people.

 

[Trish]

Let’s be clear if you know how to prompt AI well it will be more knowledgeable on ME/LC and PEM than +99% of the doctors out there.
Always ask for a reference papers (and let AI scavenge the internet for patient experiences)
And cross check S4ME forum for a second opinion.

Then hope to find that 1% doctor that’s willing to discuss these results and treatments with you.

With the speed things are now, I think doctors will start using AI more widely in less than 5 years.
- still the bottleneck is the amount of research and papers published that feed AI

While I agree that some S4ME members, scientists, doctors, and others sufficiently aware of the pitfalls, may be able to make positive use of some AI, I think the majority of people can get very badly misled.

But I think this thread topic is not really about whether AI adds value or makes people feel more understood and supported.

I think the real message is - doctors, do better. Listen to patients, admit when you don't know, offer practical supportive care, keep learning, be open minded, and so on.