diversity metrics and batch effects

[Nicholas_Bokulich]

Hi @yipinto,

@yipinto

I was thinking to filter batch specific features (with its pros and cons).

I think that approach is all cons, UNLESS if the contaminant features you are filtering are clearly contaminants, e.g., known reagent contaminants or other features that you can positively say will not be found in your samples.

@yipinto

I’m not sure if it safe to use percentile-normalized “counts” for diversity metrics.

cc: @cduvallet

@yipinto

Any other idea to deal with batch-effects?

See here for some other ideas:

Merging denoised outputs

Welcome back @Fabs! Yes indeed this looks like a typical batch effect issue. Note that batch effects will be most pronounced with Jaccard distance, since it is measuring the proportion of features that are NOT shared by each pair of samples and if you have even the most subtle differences between runs (say, the sequences could be identical but 1 nt longer in one run) then Jaccard distances will all == 1.0. The solution? The best you can do is attempt to standardize the processing between the…