Randomized Trial of Ivabradine in Patients With Hyperadrenergic [POTS], 2021, Taub et al

[EndME]

@EndME can you define blinding as you understand it?

It's the definition we all have in our head (edit: I just saw that your definition was something along the lines of "effective blinding is when people only guess the allocation arm correctly as often one would expect by chance", this is definitely not the definition I had in mind and I don't think such a definition could ever make sense). The point that I see mattering that is being neglected is that any meaningful charaterisation of blinding carries information not about whether participants ever figure it out, but about why and when they do. So the procedure of blinding can be succesful even when informational blinding breaks due to efficacy effects. This is pretty much the textbook definition as well, at least as far as my biased eye and google tell me.

You want to avoid bias. Discovering your allocation because the treatment works is not bias, even if it can result in bias.

 

[EndME]

What this discussion however seems to illustrate to me is that short trials for POTS populations, that inevitiably aren't well characterised, with fast acting drugs can often not tell you whether your drugs work even if your trial is a perfectly triple-blinded placebo-controlled trial.

Since no objective measurements to characterise the illness (heart-rate or similar) currently exists you have to include objective measurements of disability and these are probably often hard to accurately be determined in short trials.

I'm unsure about dose-response studies. I guess it depends on the drug and what sort of effects you can expect from the drug and its different dosages.

I think what one actually looks for in such trials are then consistencies. Are the reported effects happening at the same time, what is reproducible ect? I do actually think that the cross-over design chosen here is very useful for exactly those reasons, because at least you gain one extra level of control, but I guess multiple cross-overs might be even more useful when you expect the drug to be fast-acting.

The alternative would be to use active placebos (say things reducing tachychardia) that are known to be ineffective. But for POTS things might become a bit tricky if your main subjective complaints in the population are somehow related to tachychardia, at that point you're probably going to have to better characterise the population?

Edit: I guess I just learned the same thing from the discussion that @Hutan had already written. I should probably be less blind!

 

[Jonathan Edwards]

So the procedure of blinding can be succesful even when informational blinding breaks due to efficacy effects. This is pretty much the textbook definition as well, at least as far as my biased eye and google tell me.

I think Utsikt is right here. The procedure of blinding is not the same as blinding being achieved. Blinding looks to have been broken. Textbook definitions are often simplistic in this sort of context.
Subjective outcomes become problematic as soon as blinding is broken by something other than that outcome itself. If heart rate was monitored and showed less tachycardia then the study provides sound evidence of effect. If the subjects report less tachycardia it is reasonable to assume that correlates well enough with actual heart rate to be valid. But all the other outcome measures are subject to bias because for them blinding has been broken by something else.

 

[EndME]

I think Utsikt is right here. The procedure of blinding is not the same as blinding being achieved. Blinding looks to have been broken. Textbook definitions are often simplistic in this sort of context.
Subjective outcomes become problematic as soon as blinding is broken by something other than that outcome itself. If heart rate was monitored and showed less tachycardia then the study provides sound evidence of effect. If the subjects report less tachycardia it is reasonable to assume that correlates well enough with actual heart rate to be valid. But all the other outcome measures are subject to bias because for them blinding has been broken by something else.

I'm a bit lost to what is being argued here. You seem to be concluding that blinding for the subjective outcome measures is broken on the basis of tachychardia observations. That very much fits the definition of what I proposed as blinding and why it's reasonable to think that the trial doesn't provide meaningful evidence of being of use for the treatment of POTS. Blinding here appears to have been broken by things other than the subjective outcomes measures itself.

If I understand @Uitsikt correctly they argued that a trial is never blinded if people can guess they received the treatment more often than one would expect by chance even if they only make that guess after having observed all possible effects. According to that definition, your Rituximab trial would not have been blinded study, because of course people observed effects and on the basis of this thought it was more likely that they received the drug.

 

[EndME]

This is the following example I hand in mind: A placebo-controlled study is conducted on a population suffering from low levels of an antibody. The placebo perfectly matches the treatment in terms of side-effects and treatment groups are randomly assigned without anybody knowing the labels. The treatment restores the antibody to normal level and starts working after 30 day, but participants don’t know this. At day 29 both groups report that there’s a 50% chance that they thought they are in either group. At day 31 the placebo group reports no changes in symptoms and no changes in guessing the allocation arm. At day 31 the treatment group reports drastic improvements in symptoms and these are correlated to a marker of antibody going up without them having access to seeing this. 80% of the treatment group now says that they believe to have received a drug. Roughly speaking I think both @Utsikt and I would say that this look like this was a successful trial, because objective and subjective measures line up. But I would also conclude this was a blinded trial whilst @Utikt argues it wasn’t (because things aren’t split 50%-50%), which is it?

(If I have misinterpreted what you said @Utsikt please excuse that, this is hypothetical example is merely meant to illustrate the problem)

 

[Utsikt]

If I understand @Uitsikt correctly they argued that a trial is never blinded if people can guess they received the treatment more often than one would expect by chance even if they only make that guess after having observed all possible effects. According to that definition, your Rituximab trial would not have been blinded study, because of course people observed effects and on the basis of this thought it was more likely that they received the drug.

Exactly!

And that has to be accounted for when interpreting the results.

Sometimes it doesn’t matter if the patients know which group they are in, because the relevant outcomes can’t reasonably be affected by them knowing.

Other times, the outcomes will probably be affected by the patients knowing. Like most subjective outcomes, and some objective ones depending on the context.

Everything else equal, a larger degree of blinding will always be better, but depending on the context, some degree of broken blinding might still be sufficient for the purposes of the trial.

 

[Jonathan Edwards]

According to that definition, your Rituximab trial would not have been blinded study, because of course people observed effects and on the basis of this thought it was more likely that they received the drug.

We may be arguing at cross purpose but I would say that the rituximab study had a blinded design but that blinding would effectively have been broken for those who achieved major improvement. I think the problem is a bit like the concept of a controlled trial. A trial can have a control but not be controlled in a meaningful sense. The semantic complexities are not the same of course but they may be of the same order. I think we have to work on the basis that these terms often aren't as unambiguous as we expect them to be.

For this particular trial I think the nuances are particularly problematic. But we may have come to the same conclusion.

 

[Utsikt]

Roughly speaking I think both @Utsikt and I would say that this look like this was a successful trial, because objective and subjective measures line up. But I would also conclude this was a blinded trial whilst @Utikt argues it wasn’t (because things aren’t split 50%-50%), which is it?

The trial was blinded in the sense that blinding measures were implemented.

But the participants in the active group were only successfully blinded until the effect of the drug kicked in on day 30.

So does that matter for how reliable the results are? In your example, I’d say it doesn’t because the objective markers line up and the participants had no way of knowing when the effect would occur so they couldn’t possibly have manifested themselves to feel better at the exact right moment. So we can reasonably conclude that the drug caused the observed change.

All blinding eventually breaks (we do that on purpose at the end) - that doesn’t make the trial unblinded in itself. I’d only call it unblinded (or not sufficiently blinded) if the blinding didn’t achieve its purpose of eliminating that source of bias.

 

[Hutan]

Yes, we are talking about the difference between a trial design being well blinded, and the effectiveness of a treatment resulting in a loss of blinding that potentially biases assessment of outcomes.

An example of the bias arising from people correctly guessing that they are in an arm with an effective treatment is when the treatment stops further damage but does not at least immediately fix damage caused in the past. And yet people, in the euphoria of feeling that one aspect of their illness is better, may report global improvement, including to symptoms that will not improve or will only improve much more slowly.

It seems likely that ivabradine reduces tachycardia. Beyond that we don't know whether any subjective reports of other aspects of QOL are attributable to that reduction in tachycardia (which is quite plausible) or to unblinded role-playing on the part of patients who know they are supposed to be appreciative.

From the data here, I'd say we only have evidence that ivabradine reduces heart rate (both resting heart rate and standing heart rate). There is no strong evidence that it corrected an abnormal increase in heart rate upon standing. That's mainly because there isn't much evidence of the participants having an abnormal increase in heart rate on standing, at baseline or in either treatment arm.

I think it's a bit questionable whether the average participant in the ivabradine arm with the increase in heart rate of 13 bpm would be able better able to correctly guess that things had improved than the average participant in the placebo arm with the increase in heart rate of 17 bpm. That's why I'm not sure that the question of blinding being broken matters all that much to this study.

The differences in heart rate are definitely significant, impressive, even. Whether they represent a meaningful benefit is another question, although from experience, that reduction alone is a clear benefit in itself, but there is definitely an objective improvement, and for sure the drops in heart rate are the kind that is noticeable when you are used to the higher, unstable, heart rates.

In this case, because the heart rate data suggests that most of the participants didn't have an orthostatic tachycardia problem, I don't think there is evidence that ivabradine is useful. I think the good p value reported there is comparing the baseline 21 bpm change with the ivabradine 13 bpm. The difference is much less impressive and possibly not significant if the outcomes in the two arms are compared (13 versus 17 for the placebo). Importantly both mean differences are normal responses to standing.

Rvallee, you say from experience that the sort of reduction seen is beneficial. But I don't think the mean increases in heart rate with 10 minutes of standing shown here would be causing symptoms - they are pretty normal.

A better analysis or at least an important analysis would have been to categorise the heart rate response to standing as within normal bounds (e.g. <30 bpm) or outside it, and report the percentage of people with an abnormal response before and after treatment.

It is possible that the data collected doesn't illustrate the participants' everyday problems with orthostatic tachycardia well. Perhaps they normally have problems in the afternoons and the assessment was done in the morning. Perhaps the stress and excitement of participating in the trial normalised their heart rate response. Against that though is the lack of an improvement in general health in the ivabradine arm over that achieved in the placebo arm.