Utility of rarefy function and rarefaction?

[sabahzero (Sabah Ul Hasan)]

I see there are already some discussions on this topic in the forum, but that the rarefy function still exists. Copy/pasting from original Github comment:

ENH: add rarefy with replacement

" I'm unsure if this may be too bold of a comment, but may rarefaction now be obsolete? I've been reading a good deal about the artifacts that can arise with rarefying these past few months and perhaps this is a function worth getting rid of as part of the surmounting statistical evidence?

ie Waste Not, Want Not: Why Rarefying Microbiome Data Is Inadmissible "

[Nicholas_Bokulich]

Hi @sabahzero,
Great question! I think it is too early to call rarefaction obsolete, though it is certainly on its way out the door. Your question is a replicate of some previous posts — see this response:

Rarefraction not advantageous to detect differences in beta-diversity

Hi @Alfred.Burian, Thanks for bringing this up! Without wading too far into this debate, I will point out that rarefying is not always inadmissible, particularly for unweighted UniFrac and other beta diversity methods based on presence/absence of features. This is actually also shown in Fig 4 of the article that you posted, though for other distance methods other normalization strategies perform better. Second, what you describe is the qiime diversity core-metrics-phylogenetic method, which r…

So it is still quite useful to expose functionality that a very large number of users still rely on.

Similarly, QIIME 2 has several methods for OTU clustering, which is still very popular (and useful for some applications), even though denoising methods are theoretically much better.

[sabahzero (Sabah Ul Hasan)]

Thank you for the quick and thorough reply, @Nicholas_Bokulich! These are helpful to read. I skimmed through some of them, but good to refresh.

Agree it’s popular, but is it correct statistically? That’s where I’m a bit uncertain and agree it may be on its way out.