Cass report, quality of evidence in evidence-based medicine and double-blinding

I hope to keep discussion strictly on the methodology aspect, but I noticed how many of the issues overlap with the general problem of evidence-based medicine, the arbitrary application of standards, seemingly based entirely on preferences, and how very low quality evidence can influence real life clinical decisions.

I'll let Copilot/GPT4 summarize the issue broadly:

And I saw that the report apparently filtered out some studies that showed good outcomes on the basis of not being double-blinded, but kept some that showed negative outcomes that weren't double-blinded, confirmed by Copilot/GPT4:

I noticed this because of this cartoon, which by simply replacing a few words applies perfectly to, well, all of evidence-based medicine as applied to us for the last few decades:

Almost all of those issues apply to us, and conclusions clearly go whichever way the lead authors of a review want to.

To me all this shows is how completely arbitrary not only all of evidence-based medicine, but how utterly dysfunctional and broken the use of PROMs, how they become valid when people like their results and invalid when they don't. EBM is clearly mostly just opinions propped up by pseudoscience.

(Let's really keep this to methodology and evidence, this is a controversial topic and its relevance is strictly about how completely arbitrary the process of EBM is)

 

I'm reading Time to Think by Hannah Barnes which is an examination of the inside story of GIDS and goes into detail about the lack of evidence, poor methods, difficult work culture, mission creep etc. It's well written and balanced with detailed examination of the issues.

 

Guess who's going to lead the new Children and Young People’s Gender Dysphoria Research Oversight Board? Why, it's our old friend Sir Simon Wessely.

I am highly sceptical of the way the Cass Review tries to use NICE-like arguments around evidence quality, for the reasons rvallee points out, but also because of this. If you read closely, you'll find the same slippery use of referencing -- sources which don't actually back up the argument made, or which are reported on selectively. I think there was a conscious decision to appropriate what worked against GET in ME and apply it to another area in which people wanted to make unevidenced recommendations. They also didn't include trans people in the decision-making, unlike NICE, and didn't engage with charities (again, unlike NICE).

In this case, exploratory therapy also comes with some major ideological baggage and a lack of evidence. A refusal to ban "exploratory therapy", in particular, raises alarm bells much like GET does for us, or ABA does for people with autism. It is conversion therapy, and those using it have professed that they wish to cure or eradicate "transsexualism" (or similar terms), so we have to be cautious of the "GET is only pacing" approach being used here, too.

I welcome better quality research in the area, but not the way Cass uncritically accepts other therapies and approaches which are also unevidenced, and the strange ideological language used throughout.

I haven't read Barnes' book, although my own assessment of reading multiple reports was that GIDS' biggest failing was that it was under-resourced to deal with its huge waiting lists and this led to inconsistent treatment and record-keeping. In theory, replacing it with multiple new clinics is a lot better, as is better access to testing for neurodiversity (though ADHD and ASD services are also experiencing a crisis in funding and waiting list sizes). That part, I think, most people are agreed on. As always, though, the devil's in the details.

 

Listened to a brief stats podcast (BBC radio “More or Less”) about how they assessed the evidence (due to lack of RCTs).
It’s called the Newcastle-Ottawa Scale.
(Wikipedia has https://en.wikipedia.org/wiki/Newcastle–Ottawa_scale)
Here’s a random (did an image search) copy of the sorts of factors assessed in such a scale:


Does anyone have a stats background and understanding of this system?

Personally have not read the Cass report.

My impression is that it said that the evidence is poor (I.e. we need to do better in studying this stuff, something we on this forum find is all to common in far too much ‘medical research’). And that one of the reasons for that in this case is that we’ve not been studying the outcomes for these young people for long enough. As in the studies usually had a lack of long term followup.
If that is indeed the case, I’d agree with that as a criticism. It comes up a lot for pwME too (it’s easy for me to report a benefit in the short term but with longer followup we get a better picture of outcomes for me).

As for what they recommended, that’s another thing again.

 

This is an appropriate thing to do, for two reasons.

One is that safeguards like double blinding are there to avoid bias, which is nearly always bias towards a positive result. A negative result usually stands pretty well without double blinding.

The other is that the bar for noting risk of harms is always much lower than the bar for establishing benefit. That does not always apply if harms and benefits are pretty equivalent - as in influences on mortality in a cancer trial, plus or minus, but even there the principle of first do no harm is generally understood to be based on a human value judgment that causing additional harm outweighs causing benefit. In the situation here it is reasonably clear that an irreversible decision to alter physiology could produce a devastatingly bad effect life long, which probably does outweigh any speculative benefit of intervening early.

Harms cause guilt and guilt is one of the most distressing and long-lasting emotions.

 

One aspect I found troubling is that Cass wanted the NHS clinics providing gender-related care to provide the medical records of their patients to her preferred group of researchers at the University of York on a opt-out basis on the grounds that, because they planned to look at ~9000 records, "it is not feasible to get individual consent".

It doesn't seem at all infeasible to me - informed consent is far too important a principle to be discarded so readily. That the Government is apparently minded to insist that the various NHS clinics hand over the records seems to me to be setting an appalling precedent.

 

Usually anyway. Not for us, that's for sure. In fact even noting any harm is met with hostility, we're just being negative, taking away people's hope. This unequal application of standards is, well, non-standard, all over the place, arbitrary, motivated, biased, and so on.

 

Although that's not quite fair. With us they simply work around it by asserting that harm is impossible, and accordingly not recording incidences of harm related to the illness, only those like hospitalizations, heart arrests, injuries and other types of possible harm from exercise.

And ignoring a huge number of reports of harm outside of those trials.