Trial By Error: A Plea to Fiona Godlee on a Familiar Topic

MS: "This is a robust study because patient were allocated to one of the two treatments at random ensuring that any difference seen in outcome between these treatments, is not due to pre-existing differences in the patients."
So that's all that mattered then?

MS: "The main limitation is that, as it is not possible to hide which treatment they received from the patients, their self-ratings of fatigue and functioning could potentially be biased by their views on the treatment they received."
Maybe MS has been taking note of @dave30th's posts, along with everyone else striving to point this out to him.

MS: "However, differences in the school attendance a year later were also noted; it seems likely that these could be due to such a bias."
My guess (only a guess of course) is that the 'however' relates to the first sentence, with a slightly broken continuity.

Edit: x-posted with @Jonathan Edwards and @Hutan.

 

Mmm- the self reported school attendance and lots of long term follow up missing data.

I really wonder when the " penny drop" moment will happen .

 

Absolutely. I spoke to Christine and she was happy for me to quote this comment as she thought it covered the key safeguarding issue well. It's a solid argument to make.

Obviously, there's variation in local safeguarding policies, but I founded a young LGBT writers' programme, which means I have policies in place for young people and vulnerable people. Keeping secrets is not okay.

 

Do you know (and if so are you free to say) how Christine acquired the information behind her comment? Was it from children / parents / teachers she was personally in contact with?

 

When I wrote to her pointing out the outcome-swapping, retrospective registration, etc. she took another look and said she didn't see those changes made any difference, even though they were unfortunate. She actually misread the concerns and affirmed her earlier statement.

 

Here's what happened with the school absence records: In both the feasibility trial and full trial protocols, they promised to vet self-reported school attendance with the actual records. Self-reported school attendance is like self-reported physical function--it is not an "objective" measure. So vetting it against actual records makes sense. The primary outcome initially was school attendance at 6 months. They also planned to measure school attendance at other times, including 12 months, but those were secondary measures.

When they published the paper, all the school attendance findings were secondary outcomes and solely self-reported. The paper made no mention of official school records. The self-reported six-month school attendance results, part of the original primary outcome (the rest of the original primary outcome would have been the official school records at six months), had null results. There were positive results at 12 months, so they touted that. But that was always a secondary result and in any event it was not vetted against official records. Or it was, and the records show otherwise, and they chose not to mention it. Either way, they promised official records and didn't provide them. And didn't mention the absence of these records in the paper. That's not allowed.

What Bishop got wrong was when she looked at it again was that she said, well, the school attendance was positive, so the outcome-swapping didn't make a difference anyway. Of course, she got two things wrong: 1) only the 12-month self-reported school attendance was positive; six month self-reported school attendance had null results, and school attendance at 6 months was the original primary outcome, not at 12 months. 2) They did not provide the official school record results. So even if Bishop had been right and the 12-month results were the original primary outcome, she would have been wrong because those results are not credible unless vetted by the actual records. The fact that they did not provide the actual records suggests they did not get them or they did and they didn't include them, for obvious reasons. I wrote Bishop back about this, but she ignored it.

 

I agree it's important to keep all the issues in mind. There are so many things wrong with it!

 

This is the real goal, they believe in lies and hope to have them forced onto a captive population.

 

A Freudian slip?

That would be interesting to know: What exactly were the kids asked about their school attendance? (Also considering that it was a null result at 6 months)

(Apologies, haven't re-read the paper, do they display the questions there?)

 

It's from Bristol local authority's own safeguarding/child protection policies. She checked those with a colleague. However, it's a key point in many policies for protection of children and vulnerable people more generally, so the point should apply wherever the research is conducted.

The old policy for protecting vulnerable adults, for instance, was literally called 'No Secrets': https://en.m.wikipedia.org/wiki/No_Secrets_(adult_protection). This became part of the Care Act 2014 (i.e., it's now law).

Most policies talk about transparency, openness, etc. Children must be safe to talk about what's going on in their lives and adults should always be vigilant to maintain safety.

Adults are allowed to keep children's own secrets if appropriate (e.g., 'My mum has smelly farts,' which could be embarrassing or lead to bullying) but can't if they suggest a risk of abuse or an unsafe environment to the child or a vulnerable person ('I can't do my homework because we don't have any pencils at home,' for instance, might suggest neglect).

Adults should not ask kids to keep a secret, though, because that's a classic red flag for abuse. Obviously, there are some things parents will ask their kids not to tell people, which are private but not a risk to the child ('Auntie Sally is a lesbian, but she doesn't want anyone to know.'). But there shouldn't really be a situation where a teacher, doctor, researcher or trainer tells a child to keep a secret.

The Lightning Process falls into that secondary category because the 'trainers' are responsible adults with a duty of care. Researchers likewise have a duty of care and shouldn't let the trainers tell children to keep secrets about the process.

It is worrying, because how can you ensure the process is safe and that abuse isn't involved if no one in the room is allowed to talk about what happened? From a research perspective there's an additional problem in that you can't know which element, if any, is effective (or harmful).

ETA: I'm realising now you might have meant how did she know the process involves keeping secrets? If so, that's something that's been documented online by former participants.

 

Are we expected to believe that people with a specialist interest in ME/CFS at GOSH and UCL are unaware of the controversy surrounding the Lightning Process paper and of the caveat attached by the editor. If so, just how much reading do they do around the subject of their supposed specialism? It does not look good for them either way. Either they knew and yet gave their apparent approval. Or they did not, which may be even worse.

 

https://twitter.com/user/status/1129153503031693315

 

When considering what to provide NICE as testimony n the problems of clinical trails in ME/CFS I did not go in to the Lightning process specifically, other than to note that the apparent success of SMILE illustrates the problems with other trials.

I had forgotten the significant ethical problems with the method. However, I have noted the ethical problems with other therapist-delivered treatments. Anything that involves cognitive manipulation in the context of a trial of an unproven therapy raises serious issues. It probably raises issues even if the therapy is of proven value, but so far none of these therapies are of proven value and they certainly weren't when the trials were set up.

 

I guess that the 'controversy' for SMILE has been aired in social media rather than in published literature. I can imagine that clinicians and researchers in the field may not yet be aware of it. The situation is different from that for PACE where the controversy is in thepublishezd literature. People in the field should be aware of the weaknesses of SMILE but not necessarily the controversy. The editor's note sounds as if it is pretty hard to find.

 

Such a useful phrase!

 

Wouldn't serious clinicians want to know what their patients, or their parents, might be reading? SW always did. I would have thought that Dave's blogs would be on the essential list, if only to ensure that you don't become the subject of one.

 

But how would they know of the existence of David's blog unless a patient had brought it to their attention? Patients who are aware of the blog may well not mention it because they are afraid of sounding bolshy. Why would an ME clinician google 'virology blog'?

 

Because his name has often appeared in the general press in connection with criticism of Pace, and anyone who isn't abreast of that is not really up to date.