Understanding PERMANOVA QIIME2 Statistics for Beta Diversity

Hello,

I have had a look through the forum and these pages but I am still a bit confused so would appreciate some clarification on the PERMANOVA statistics.


I have run the following command: https://docs.qiime2.org/2020.2/plugins/available/diversity/beta-group-significance/
using the pairwise option.

To simplify, I have two groups called A and B:

I want to know what the statistics output means at the bottom of the output page
eg:

Group 1 Group 2 Sample size Permutations pseudo-F p-value q-value
A B 329 999 35.8374808 0.001 0.001666667

The answers previously on the forum say: " The pairwise permanova results at the bottom of the page compares between Group 1 and Group 2, and tests to see if they are statistically significantly different."

From my diagram above I am taking this to mean the statistics are referring to the difference between the within distances observed for A and the within distances observed for B (comparison 1) in the diagram. Is this correct?

If so, is it possible at all to perform PERMANOVA on comparisons 2 and 3 above or should I be using a different test (there was a Monte Carlo permutations 999 option in QIIME 1 previously but what should I be using in QIIME 2?). These values would be more informative to me because in theory the within distance could be very similar but the between distance very different meaning the samples form two distinct clusters but this might not be picked up if only comparison 1 was performed.

I have the plots for comparisons 1, 2 and 3 though.

Any advice or clarification much appreciated.

Thanks.

Hi @LVC,
This topic has a number of links to topics discussing PERMANOVA interpretation, etc, so I recommend taking a look here:

Great diagram! PERMANOVA is really testing all 3 comparisons — basically, a significant result is telling you that the within-group distances of at least one group are less than the between-group distances for all groups.

This is controlled by the --p-permutation parameter.

Good luck!

Thanks, that makes sense and I understand it now.

As for:

This is controlled by the --p-permutation parameter.

I meant that the Monte Carlo option reports the stats as:
comparison 1 - p-value 1
comparison 2 - p-value 2
comparison 3 - p-value 3
And I was wondering if it was possible to get a similar report from QIIME2 also with each comparison broken down instead of one overall p-value for all three.

I could perform a Kruskal-Wallis test on the dataset of permutations and look at each separately in addition to my PERMANOVA result but I am unsure if this is the best approach or whether there is a recommended alternative.

Thanks for your help and the links above.

No, I don’t think that’s an option. I think we did not implement it in QIIME 2 because technically a permutational t-test is not actually an appropriate test for comparing between-sample distances (due to assumptions of independence)

I am not sure that’s statistically appropriate. As far as I am aware, Kruskal-Wallis assumes that observations are independent, which between-sample distances are not.

beta-group-significance has a permdisp method available, this is the appropriate method for comparing within-group beta dispersions by looking at distance to the centroid of each group.

I hope that helps!

2 Likes

Yes, you are right, Kruskal-Wallis isn’t appropriate now I’ve looked into it.
Thanks again for all your help. I’ve got it sorted and understand it properly now I hope and the results are looking good.

2 Likes