My labmate @Lauren and I are trying to find an easier way to compile DADA2 denoising stats from many different DADA2 outputs. Is there any QIIME2 command available to merge denoising stats, similar to how one can merge feature tables and rep seqs? We tried using qiime metadata tabulate to merge the denoising-stats.qza files, but this did not work. It seemed to not like it because all of the columns are the same, and it basically wants to append new columns together.
The only other way we have thought to do this is to export the stats.tsv files from each of the qza files and then compile them in one large excel file, but this is a bit cumbersome.
Hi @cjone228,
The best way to do a customized merge like this would be to work with the stats objects in python. Something like this (untested!):
import qiime2 as q2, pandas as pd
dada2_stats_filepaths = ['path-to-file-1', 'path-to-file-2', 'etc']
stats = [q2.Artifact.load(f).view(q2.Metadata).to_dataframe() for f in dada2_stats_filepaths]
stats = pd.concat(stats)
stats.to_csv('concatenated-dada2-stats-filepath.tsv', sep='\t')
That will join your stats files into a larger TSV stats file (add more to the list, e.g., if you have multiple runs with different samples and want to view them as one).
If what you want instead is to add additional columns (e.g., if you ran dada2 multiple times on the same samples and want to see how many reads you get with different parameter settings) then it gets a little more complex. Replace the 5th line above (the one with pd.concat(stats)) to something like this (again, untested!!!):
for n, s in enumerate(stats):
s.columns = [c + '_' + str(n) for c in s.columns]
stats = pd.concat(stats, axis=1)
With a bit more finessing you could do something a bit nicer… python is a beautiful thing
But if python is well and truly out of your comfort zone right now, the next best thing will be to export the data and manually stitch it together using excel…
Thank you so much for this idea!! We are definitely not experienced with python but might try and play around with this. Would really like to learn python
