How to read a paper involving artificial intelligence (AI), 2025, Dijkstra et al

How to read a paper involving artificial intelligence (AI)

Paul Dijkstra, Trisha Greenhalgh, Yosra Magdi Mekki, Jessica Morley

Abstract
This paper guides readers through the critical appraisal of a paper that includes the use of artificial intelligence (AI) in clinical settings for healthcare delivery. A brief introduction to the different types of AI used in healthcare is given, along with some ethical principles to guide the introduction of AI systems into healthcare. Existing publication guidelines for AI studies are highlighted. Ten preliminary questions to ask about a paper describing an AI based decision support algorithm are suggested.

Link (BMJ Medicine)
https://doi.org/10.1136/bmjmed-2025-001394

 

I almost stopped reading after their definition of AGI.

They also talk about how training data might be biased, but bias in AI refers to the difference between the model and the data, and not the difference between the data and the real (or ideal) world.

There is also no direct mention of Responsible AI (RAI) or Explainable AI (XAI).

 

Principle 3 seems kind of hard to follow, since we don't know how an AI makes the decisions it does. We don't know how humans do this either. I expect human doctors occasionally base decisions on information they remember incorrectly, or studies they misjudged the value of. I hope they judge AIs compared to human levels of accuracy, rather than some level that's also unreachable by humans.

I also hope they throw out principle 1 once AI reach the point of making fewer errors--and patient harm--than humans do.

 

Principle 3 is why we should not roll out AI on a massive scale like we do today.

Principle 1 can never go, we can’t abdicate from our own lives.

 

The issue with defining bias in AI as a deviation from «the real world» is that it doesn’t tell you anything about the source of the bias. It might not even have anything to do with the model at all. It’s like saying that a rifle has a bias if you missed the target, even though it might have happened for a number of reasons completely unrelated to the rifle.

You’ll also have a very hard time defining «the real world».

There are many sources of bias and it has a different meaning in different fields, but as far as I know, in AI, bias refers to the learning bias: the difference between the training data and the model. All of the other types of bias are not directly related to the inner workings of the model, so they are not «AI bias» per se. They are definitely relevant, though!

But the point I did at terrible job at making, is that an introduction to AI really should mention the types of bias that are more specific to AI, and not just mention bias in relation to the data that is used for training or input. It would be a bit like just warning people that a lathe can electrocute you if it isn’t grounded properly, instead of telling them that the massive spinning bit can rip the flesh off your bones.

 

Bit funny how literally none of those things apply to traditional health care when it comes to us. The idea is basically that AI should be so far above humans in terms of alignment with human values that it becomes unrecognizable. Health care is not actually aligned with those values in mind. It sometimes manages so, generically or by accident, and most professionals are definitely trying their best, but overall everything falls short of it.

And it may actually be a more realistic possibility, that machines could be better aligned with values that humans love to espouse, but rarely ever actually follow. But the idea here is that humans already do that, and that it will be very hard for machines to do, when the reality is the main reason it will be hard for machines to do this is because the very human nature that will train them. The problem isn't with machines, it's with the worst humans.

Although I guess #1 is already a thing, humans are in control of everything. Unfortunately those humans are not us, we have no control over anything that happens to us. Other people, strangers, do, with zero meaningful accountability. And it's truly awful. So awful in fact that with the right prompts, most LLMs already do better at it. Really, they do. Because it's easy, and simple. But human nature gets in the way.

I've accepted the idea that natural humans will never live up to human ideals. We are too imperfect, too weird. Either humanity evolves beyond biology, or we'll destroy ourselves anyway. We evolved as animals, and we remain so. Barbarians in clothes who can sometimes count accurately and do other things.