Multiple Testing Correction in BUSTED vs absREL

[NicMAlexandre]

Hi Sergei,

Could you point me to a preferred method and rationale for multiple testing correction using BUSTED and absREL. I am having trouble finding these resources.

[spond]

Dear @NicMAlexandre,

There really isn’t anything formal that I ever wrote. Your specific application matters quite a bit. Not knowing anything else, I would recommend FDR for BUSTED. aBSREL is a bit more complicated, because it does multiple testing correction at the level of branches for each run. You can layer another correction on top of that if you want, probably another FDR. Can you give me any details?

Best,
Sergei

[NicMAlexandre]

Hi Sergei, Thank you for the swift response: The aBSREL tests were run in exploratory mode, so not sure if that will have to be corrected within-gene for each branch, but BUSTED was specifically testing whether one specific branch is inconsistent with model that excludes positive selection (<40 gene of ~11k reject constrained model). We also ran FUBAR because, if a gene got flagged as consistent with positive selection in BUSTED, then I wanted to have the site-level tests calculated already.

…

On Tue, May 14, 2024 at 3:04 PM Sergei Pond ***@***.***> wrote: Dear @NicMAlexandre <https://github.com/NicMAlexandre>, There really isn’t anything formal that I ever wrote. Your specific application matters quite a bit. Not knowing anything else, I would recommend FDR for BUSTED. aBSREL is a bit more complicated, because it does multiple testing correction at the level of branches for each run. You can layer another correction on top of that if you want, probably another FDR. Can you give me any details? Best, Sergei — Reply to this email directly, view it on GitHub <#1707 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFB633ZXWBRS2IBNSBQWYADZCJU47AVCNFSM6AAAAABHUN6WT6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJRGA2DOOBQGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Best, Nicolas Alexandre PhD Candidate, Integrative Biology Whiteman Lab University of California - Berkeley ***@***.*** ***@***.***>

[spond]

Dear @NicMAlexandre,

aBSREL already does the within-gene correction for you, so that's not something you need to do after the fact. The number of test branches for BUSTED is immaterial for the purposes of per-gene correction (one gene = one test), only inasmuch as you would expect very low power to reject given a single foreground branch. In other words, you would expect the actual Type I error rate to be much lower than the nominal one (p-value cutoff). Doing a multiple test correction here is probably going to dilute everything out.

Would you happen to have a list of p-values (for the 11k genes) that BUSTED produced that you could share?

Best,
Sergei

[NicMAlexandre]

Thank you, that makes complete sense. I just direct messaged the P-values.