Thanks
@Hilda Bastian for taking the time to offer up that urinary incontinence study.
There is certainly a lot going on in it.
Edit - I'll add here so people have an idea of the structure - two treatments were compared: physio (pelvic floor exercises) and surgery. Women were randomly allocated to the treatments but women allocated to physio were allowed to also have surgery (and about half of them chose to do that).
So, the study found that surgery was a lot better than physio and recommended that women be offered both treatments (rather than just physio) as first line treatments, and that they be given information on the expected outcomes of each. But how good was the information the study produced about the expected outcomes? Is physio even worth trying?
If we just consider the physio treatment (because there's a lot to talk about with just that), the paper noted that previous studies had reported rates of subjective 'success' (presumably 'cure' - as these figures are compared to surgery cure rates, but perhaps not) in the range of 53 to 97%. It's all rather odd, as the paper later mentions two studies of physio treatment with improvement rates of 33% and 43% - and surely improvement is a category that includes cure?
In this study, the rate of subjective cure in the women who were allocated to the physio group
and only had physio was only 16%.
However, this probably over-states the real rate of cure, because
half of the women who were allocated to the physio treatment actually gave up on physio and went and had surgery. It seems likely that the women allocated to the physio group who did not choose to have surgery were a whole lot more likely to be pleased with the outcome of their physio exercises than the ones who went on to have surgery. Indeed, of the 99 women allocated to physio who chose to have surgery, 90 of them had reported no improvement (so, not even some improvement, much less a cure) at their last assessment before surgery. So, from this study, we could expect the subjective cure rate from physio for women with the required level of urinary incontinence to be somewhat less than 16%, and maybe a whole lot less.
Also, of the women who were allocated to the physio group and underwent physio only (i.e. didn't also have surgery)
and who were lost to followup (i.e. who didn't contribute to the 12 month data), 76% (16 out of 21) reported no improvement at all in their last recorded assessment. So, the women who dropped out were very likely to have reported no improvement, and much less a cure. That adds further to the suggestion that the women who were allocated to the physio group
and who didn't choose to have surgery
and stayed around for the 12 month followup were quite a select group - who were a whole lot more likely to be pleased with the outcome than all the other women allocated to the physio group.
There was no control in this study, as it wasn't primarily trying to determine whether physio works. So we don't actually know if some of the 16% of women who received physio only and reported being cured might have reported being cured after 12 months with no treatment at all. I imagine there is some rate of spontaneous resolution of symptoms, especially for those women with an onset from giving birth. It looks as though obesity is a significant risk factor for urinary incontinence and so it seems possible that some women might lose weight and find that their symptoms resolved, independent of the pelvic floor exercises. I wouldn't be surprised if the physiotherapists actually prompted/helped some obese women to lose weight, so the reported improvement could arguably be from that rather than (or at least as well as) the pelvic floor exercises.
A problem with the subjective rating is that the physio treatment was presented as a skill to be learned. The paper talks about the treatment being given "depending on a number of things including 'adherence' and 'the ability of the women to learn to perform the muscle contractions'. So there would have been quite a bit of social pressure for the women to report a good outcome - many would want to be seen as having worked hard to do the exercises and as being able to learn what was required. It's quite possible to imagine that a woman might report a cure when there was still some urinary incontinence, thinking that probably every woman experiences some leakage under some circumstances.
Acting against an overly positive reporting of subjective outcomes is the possibility that many of the women in the study might just have wanted a fast track to the surgery. Some might even have felt that reporting no improvement from physio would make their case for surgery stronger.
Adding all that up, with the two subjective outcomes at 12 months the study focuses on (PGI self-report of improvement and a subjective report of cure (a negative response to "do you experience urine leakage related to physical activity, coughing or sneezing?")), it's very hard to say what impact the physio treatment had on urinary incontinence in this study.
Sadly, the objective outcome is a bit useless too.
There was no initial objective testing or even a requirement that the testing had been done at some earlier time.
Urodynamic testing to confirm the diagnosis was not mandatory for eligibility.
So, we don't actually know if the women in either treatment group really were in the moderate to severe category at the beginning of the study. It calls into question the objective cure rate - which was measured by a urodynamic test at 12 months - we don't know how many women would have failed the test at the beginning.
Among the women who only had physio, the rate of objective cure was higher than the rate of subjective cure (44% vs 16%). The paper suggests that women were able to withstand the clinical provocation cough test with a full bladder but 'still had stress urinary incontinence in everyday life in response to unexpected events'. Which sounds pretty likely to me, and calls into question the whole notion of 'objective cure'. An objective measure isn't automatically a useful measure. It also calls into question how you might compare cure rates between different studies, if some studies are using a test that doesn't actually measure cure.
There's a whole lot more. Perhaps most importantly, the paper does not really highlight the impact on treatment outcomes of the 49% of the women in the physio treatment who went on to have surgery. The data that is presented most prominently is an intention to treat analysis, with the result that, at first glance, the physio treatment (with about half of the participants also having had surgery) looks a whole lot more useful than it really was.
The paper is a bit of a dog's breakfast. There's some good stuff in there, but it's all mushed up with stuff that is a bit whiffy. I don't think it is a good example of subjective outcomes doing a good job of answering the question of how well a treatment works in an unblinded trial.
(I hope I haven't got too much wrong - I have run out of energy to carefully check everything. Some edits to improve readability.)
].
