I am working with the same 16S rRNA gene amplicon and samples from the same sequencing batch, but the data were generated in two separate libraries (J1 and J2). The same barcode sequences were reused across libraries, so raw FASTQ files cannot be merged prior to demultiplexing.
Each library was therefore processed independently using Trimmomatic, followed by cutadapt demux-paired and cutadapt trim-paired. This resulted in two SampleData[PairedEndSequencesWithQuality] artifacts. After demultiplexing, sample IDs are already globally unique across libraries.
My goal is to jointly denoise all samples using DADA2, because denoising the two libraries separately and merging feature tables afterward may assign different ASV IDs to identical sequences due to library-specific error models.
Since I could not find a direct way in QIIME 2 to merge multiple demultiplexed paired-end artifacts prior to denoising, I am considering exporting each demultiplexed artifact to per-sample FASTQ files and re-importing all samples together via a single paired-end manifest, followed by one joint dada2 denoise-paired run.
Is this export → manifest re-import → joint denoise approach valid and recommended in QIIME 2 (v2024.5)? Are there any better alternatives or caveats I should be aware of?
First of all, thank you for providing a lot of details in the post. Your issue is well-described.
Yes, that is the right approach to handle different indexes with repeated barcodes within the sequencing run.
Library-specific error models may lead to minor differences between ASV counts, but the same ASVs will have the same IDs since IDs are assigned based on the sequences of resulted ASVs. Therefore, I would be more worried about the differences introduced during the primer removal step (make sure that cutadapt settings are identical across libraries). If you decide to run DADA2 separately, cutadapt and dada2 settings should be identical, then ASV IDs will match between batches.
It is what I usually do when I need to merge already imported demultiplexed datasets.
This approach is fine if you want to merge batches before dada2. I encountered some issues with cutadapt demultiplexing a long time ago and switched to other sostware for demultiplexing (sabre), so I demultiplex each index separately, and import samples by sequencing run for primer removal / dada2. In your case, exporting and reimporting samples will do the same thing.