Continuing the discussion from Kruskal-Wallis test of alpha diversity:
Hi there @colinbrislawn is the Benjamini-Hochberg FDR correction also used to calculate q-values for the pairwise beta diversity PERMANOVA? Also, only p-values are shown for overall (non pair-wise) comparisons. Is there a parameter that can be passed along with the command to compute the q-values?
Great question! I know the Benjamini-Hochberg FDR correction is used to adjust the p-values to account for the multiple testing of permanova, but I'm not sure how it works on this specific script.
I'll let the qiime devs answer this one!
Colin
Yes. The --p-pairwise parameter toggles pairwise testing on, and the q-values are BH-FDR.
There is only a p-value shown because there is only one PERMANOVA test performed per visualizer (I suppose technically if you toggle on pairwise testing then there are multiple tests that should be corrected here). Is that your concern?
Or is your concern, e.g., that if you run PERMANOVA multiple times to examine different parameters then these p-values should be corrected? Since one PERMANOVA test is performed per command (excluding pairwise tests), there really is no way for QIIME2 to know how many tests have been performed (that is up to the user). So ultimately users who run this test multiple times have the responsibility to perform multiple test correction themselves. You can just plug a list of p-values into R p.adjust to do this on your own. You could also just do Bonferroni adjustment by hand — multiply each P value by the number of tests.
Multiple-test correction is probably really only going to be an issue with a large number of tests. If you have a handful of PERMANOVA tests, you are minimally inflating your type 1 error rate. But probably best to consult a real statistician if you want a real answer...
I hope that helps!
yes, this is very helpful. Thanks @Nicholas_Bokulich
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.