Low-Dose Naltrexone restored TRPM3 ion channel function in Natural Killer cells from long COVID patients, 2025, Martini et al

[jnmaciuch]

This notable plot from Hanson is generated by Bonferroni adjustment. The distribution of the fold change of a lot of the metabolites on the x-axis is similar, but the q-value on the y-axis of the right-side plot is made zero by Bonferroni. ( I was so flabbergasted by these two plots that I taught myself multiple comparison and downloaded the data and reran the analysis, the plots are accurate. But I do find myself wondering what meaning is left after adjustment).

I remember also being confused why on earth they would use Bonferroni for an -omics analysis. You almost always want to use Benjamini-hochberg (FDR correction), because if you’re doing Bonferroni for so many analytes you’re basically throwing the baby out with the bath water.

Like @forestglip gets at, you only want to use Bonferroni when it’s really important to not report false positives. If you can tolerate a small proportion of false positives in the interest of not losing your true positives, which is the case in nearly all big data analyses, FDR is the way to go.

 

[Murph]

I remember also being confused why on earth they would use Bonferroni for an -omics analysis. You almost always want to use Benjamini-hochberg (FDR correction), because if you’re doing Bonferroni for so many analytes you’re basically throwing the baby out with the bath water.

Like @forestglip gets at, you only want to use Bonferroni when it’s really important to not report false positives. If you can tolerate a small proportion of false positives in the interest of not losing your true positives, which is the case in nearly all big data analyses, FDR is the way to go.

um, you know what, i think maybe it was benjamini hochberg. yep, they mention that. I mis-remembered.

 

[jnmaciuch]

um, you know what, i think maybe it was benjamini hochberg. yep, they mention that. I mis-remembered.

Looks like we both misremembered! I may have assumed it was Bonferroni because the p values were so negligible.

 

[DMissa]

fig 2a-c would benefit from showing the individual points, particularly 2a

 

[forestglip]

If you called your experiments different experiments and published them in separate papers, nobody would demand adjustment for multiple comparison. if they are done in one batch, it's expected.

For this part, I've thought about this same thing many times, and haven't developed an intuition for why it's done the way it is, and question it for the same reason you say. And apparently so have others:

To adjust, or not to adjust, for multiple comparisons (2025, Journal of Clinical Epidemiology)

The Bonferroni correction in particular, and controlling the FWER more generally, are beloved by that archetypal figure many of us can conjure from our publishing experience – “Reviewer 2” – whom we can readily imagine urging us to conduct a “proper analysis” adjusting for the multiple comparisons in our manuscript [3]. Though such a request comes from a good place, it is worth understanding why and when one might push back on it. Rothman was dismissive [4]. Many others have pointed out the logical difficulties that become apparent when we start asking where this wave of adjustment – this protection against making errors – is to end.

Should we apply a correction to all the results in all of the papers that contributed to the same programme of research? To all of the results in all of the papers that analysed data from the same routine database that we used? To all of the results in all of the papers in that issue of the journal where our paper is published? To all of the results in all of the papers in every issue of that journal ever published? To all of the results in all of the papers that we, personally, have published in our lifetime? None of these things seem to make any sense [1,3]. All of this militates against authors making adjustments to their P values for multiple testing in the more restricted domain of one paragraph of their results section.

 

[forestglip]

fig 2a-c would benefit from showing the individual points, particularly 2a

It was too much work for me to understand what it is they're exactly doing, but I got the impression figs 2D-F are the same data as 2A and 2B, just as dots. So LC in fig 2A would be the first column of dots in fig 2D. HC in 2A would be the first column in fig 2E. I might be wrong about this.