Why is rarefying required with core-metrics but not when ran individually?

[Eman (Eman Khalaf)]

Hi @Mehrbod_Estaki,
I got it for writing the command 3 backticks (sorry for that).
I have another inquiry about alpha diversity (which is irrelevant to this topic). When applying core-metrics-phylogenetic , both alpha and beta diversity are calculated at a specific sampling depth, but when calculating different metrics of alpha diversity individually, the calculations are done on the entire feature table.
So, my question; why in core-metrics-phylogenetic, alpha diversity metrics are calculated using a rarefied table, however it is better to run it on the entire feature table.
I did it for a specific metric (observed otu) using both core-metrics-phylogenetic and qiime diversity alpha-phylogenetic and found difference in p-values.
Is there a different interpretation for alpha metrics calculated on rarefied and non-rarefied feature table.
Thank you

[Mehrbod_Estaki]

Hi @Eman

@Eman

I have another inquiry about alpha diversity (which is irrelevant to this topic).

Feel free to open new topics regarding new inquires. You're not bound to a single thread

I'm going to guess the developer's mind on this: core-metrics is meant to be a convenient way of running a bunch of typical diversity estimations at once and since dealing with the problem of uneven sampling depth is essential in this type of data (and the fact that there currently isn't another normalization method in qiime2) this action requires a rarefying depth as to avoid producing spurious results.
But to also give the user the flexibility and freedom to do these types of analysis without rarefying (ex benchmarking, education, exploring etc.), the stand alone version doesn't require this. That doesn't mean it is correct to run these metrics without some sort of normalization methods though! So if you are simply comparing your results from rarefied (core-metrics results) and non-normalized+non-rarefied (individually run) tests, I would say your non-rarefied data is unreliable and you should stick with the rarefied results. That being said there are some alternative options such as breakaway (alpha diversity estimator) and DEICODE (Beta diversity) in qiime2 that are meant to work without rarefying. There are further good discussions on the forum with regards to rarefying vs normalization vs using methods that don't require either. Have a read through some of those if you want to dig deeper into the topic.

[Eman (Eman Khalaf)]

Thanks a lot for your detailed answer.