# merging sequence runs with slight primer variation

**URL:** <https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142>\
**Category:** User Support\
**Tags:** 18s, dada2, merge, primers\
**Created:** [July 20, 2023, 5:34pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142 "2023-07-20T17:34:08Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![spongebob](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/90db22/32.png) [@spongebob](https://forum.qiime2.org/u/spongebob)\
**Post date:** [July 20, 2023, 5:34pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/1 "2023-07-20T17:34:08Z")

</div>

Hi all,

initial analysis done with qiime version: qiime2-2019.10

I have two data sets of 18S rRNA gene sequences from different sequence providers. The only difference is the reverse primer has an extra trailing TGA

```auto
Data set 1:
565F CCAGCASCYGCGGTAATTCC
948R ACTTTCGTTCTTGATYRA

Data set2:
565F = CCAGCASCYGCGGTAATTCC
948R-modified = ACTTTCGTTCTTGATYRATGA

```

For the initial analysis (after using cut-adapt for the different primers) i ran both datasets separately using DADA2 and used the same parameters for both for example:

```auto
qiime dada2 denoise-paired 
--i-demultiplexed-seqs dataset1_demux-paired-end.qza 
--p-trunc-len-f 270 
--p-trunc-len-r 270 
--p-max-ee-f 2 
--p-max-ee-r 5 
--p-n-threads 2 
--o-table dataset1_table-dada2.qza 
--o-representative-sequences dataset2_rep-seqs-dada2.qza 
--o-denoising-stats dataset1_denoising-stats.qza`

```

Each dataset is classified with the same PR2 eukaryotic database.

I thought it would be good to combine the datasets. I did this using the `qiime feature-table merge`and `qiime feature-table merge-seqs` commands. **However, i don't get a single overlapping ASV.** These data sets are from the same waters samples just separated by a different filter size.

I am wondering:

- If this is from an error in my analysis? could the extra TGA effect something? Does length effect dada2?

- Should i merge after cutadapt and before DADA2? from reading the forum i thought this was not needed.

- The merge is correct and the data is to be trusted?

Thanks for any help

---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [July 20, 2023, 10:41pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/2 "2023-07-20T22:41:30Z")

</div>



---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [July 20, 2023, 11:03pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/3 "2023-07-20T23:03:09Z")

</div>

Hi @spongebob,  
I think what's happening here is that the amplicons from the 565F/948R analysis are slightly longer than the amplicons from the 565F/948R-modified analysis. You should be able to see that difference if you look at the `qiime demux summarize` output for both of your `denoise-paired` input files. If that's the case, you shouldn't expect to see any overlapping ASVs, since that overlap is done based on exact matches, and the length difference will throw off the exact match.

What might work best here would be to cluster the ASVs in your merged table using the q2-vsearch plugin. You can try this using the [`qiime vsearch cluster-features-de-novo --p-perc-identity 1.0`](https://docs.qiime2.org/2023.5/plugins/available/vsearch/cluster-features-de-novo/), which will cluster your ASV sequences at 100% identity. I _think_ the way the percent identity calculation is performed, your ASVs will cluster if they only differ based on the three bases at the end, as terminal gaps are not counted as differences in the sequences.

Can you let us know how that works out either way? It'll be helpful to know if this works for the future.

One thing to be aware of is that those extra three bases in the 948R-modified primer will introduce biases across the runs. I suspect that the differences would be minor, but any time your primers differ at all primer biases are likely to show up.

Greg

---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [July 20, 2023, 11:03pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/4 "2023-07-20T23:03:16Z")

</div>



---

<div class="post-metadata">

**Author:** ![spongebob](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/90db22/32.png) [@spongebob](https://forum.qiime2.org/u/spongebob)\
**Post date:** [July 21, 2023, 5:03pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/5 "2023-07-21T17:03:34Z")

</div>

Attached are the outputs of the `qiime demux summarize` for each dataset. The only one of different length is the forward read on dataset 1:

 ![dataset_1SV](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/1/4/14eb05cb55910a6611389147d34c084b4bb449fc.png)

 ![dataset_2PF](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/2/5/2557016b1a297b7c3d9bf8569cb48c4ef782b56e.png)

Is this a cutadapt error? The reads are all 2\*300pb before using cutadapt for both datasets - so i know that something got trimmed. I ran as follows:

For dataset 1 i ran as:  
`qiime cutadapt trim-paired --i-demultiplexed-sequences dataset1_demux-paired-end.qza --p-cores 2 --p-front-f CCAGCASCYGCGGTAATTCC --p-front-r ACTTTCGTTCTTGATYRA --p-adapter-f TYRATCAAGAACGAAAGT --p-adapter-r GGAATTACCGCRGSTGCTGG --o-trimmed-sequences dataset1_demux-paired-end_PT.qza --verbose`

For dataset 2 i ran as:  
`qiime cutadapt trim-paired --i-demultiplexed-sequences dataset2_demux-paired-end.qza --p-cores 2 --p-front-f CCAGCASCYGCGGTAATTCC --p-front-r ACTTTCGTTCTTGATYRATGA --p-adapter-f TCATYRATCAAGAACGAAAGT --p-adapter-r GGAATTACCGCRGSTGCTG --o-trimmed-sequences dataset2_demux-paired-end_PT.qza`

If this is correct then the only other thing is that checking back on my logfiles the two datasets were run with different versions of qiime:

qiime2-2019.10 and qiime2-2021.4

If this does not make a difference then i'll run vsearch and let you know how it goes.

Any further insights much appreciated .

Thanks

---

<div class="post-metadata">

**Author:** ![colinvwood](https://forum.qiime2.org/user_avatar/forum.qiime2.org/colinvwood/32/17303_2.png) [@colinvwood](https://forum.qiime2.org/u/colinvwood)\
**Post date:** [July 24, 2023, 5:02pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/6 "2023-07-24T17:02:13Z")

</div>



---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [July 26, 2023, 8:48pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/7 "2023-07-26T20:48:19Z")

</div>

Hi @spongebob, The version of cutadapt that QIIME 2 uses changed between qiime2-2019.10 and qiime2-2021.4, so it's definitely possible that the trimming functionality works differently between the two versions. And, I would have expected the dataset2 reads to be shorter than the dataset1 reads post-cutadapt, since the dataset2 primer is longer.

I recommend re-running both with QIIME 2 2023.5, or at the very least running both through the same version of QIIME 2, and comparing these results again.

---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [July 26, 2023, 8:48pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/8 "2023-07-26T20:48:25Z")

</div>



---

<div class="post-metadata">

**Author:** ![spongebob](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/90db22/32.png) [@spongebob](https://forum.qiime2.org/u/spongebob)\
**Post date:** [August 4, 2023, 12:53am UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/9 "2023-08-04T00:53:00Z")

</div>

Double checked the primers and which dataset they go with - they are correct. I re-ran both datasets through qiime 2023.5.

 ![dataset1_precutadapt](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/6/0/60864e82b0dd6ff8b47257f11c598eba16ee5960.png)  
 ![dataset1_after_cutadapt](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/c/8/c868366ee6fd5ed92e9f266b048869deb25c2055.png)

 ![dataset2_precutadapt](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/7/9/79ebd97483dbbdada978cf66effc8fb9af8f6141.png)  
 ![dataset2_after_cutadapt](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/9/d/9dd2b6d9fe26ffc6cd8994be1bbb7a28502dba37.png)

I get the same lengths as before.

I ran dada2 with the same parameters for both datasets:

```auto
qiime dada2 denoise-paired 
--i-demultiplexed-seqs PrimerTrimed.qza 
--p-trunc-len-f 270 
--p-trunc-len-r 270 
--p-max-ee-f 2 
--p-max-ee-r 5 
--p-n-threads 4 
--o-table table.qza 
--o-representative-sequences rep-seqs.qza 
--o-denoising-stats denoising-stats.qza

```

I retained about 70-80% of the reads

I then tried to merge the ASVs ( using `qiime feature-table merge` and `qiime feature-table merge-seqs` commands) --- i get the same issues -- no overlapping ASVs.

I am now running vsearch to cluster the data.

A couple of questions:

1. I have already classified each dataset - can i filter that taxonomy file to get the ASVs classifications for my clustered data or will i need to run it again?

2. whats would be wrong with combining the forward and reverse reads from each dataset (after cutadapt) and then running dada2?

---

<div class="post-metadata">

**Author:** ![spongebob](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/90db22/32.png) [@spongebob](https://forum.qiime2.org/u/spongebob)\
**Post date:** [August 4, 2023, 1:26am UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/10 "2023-08-04T01:26:42Z")

</div>

here are the vsearch results:

I first merged the tables and the rep-seqs files:

```auto
qiime feature-table merge --i-tables PF_table.qza --i-tables SV_table.qza --o-merged-table PF_SV_merged_table.qza

qiime feature-table merge-seqs --i-data PF_rep_seqs.qza --i-data SV_rep_seqs.qza --o-merged-data PF_SV_merged_rep_seqs.qza 

```

Then i ran vsearch:

```auto
qiime vsearch cluster-features-de-novo --i-sequences PF_SV_merged_rep_seqs.qza --i-table PF_SV_merged_table.qza --p-perc-identity 1.0 --p-threads 14 --o-clustered-table PF_SVEX_clustered_table.qza --o-clustered-sequences PF_SVEX_clustered_rep_seqs.qza

```

Here is the summary before vsearch

 ![merged_tables_before_vsearch](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/f/8/f8d1c1f6b64ed419d6b7c67c68715007de552b75.png)

Here is the summary after vsearch

 ![after_vsearch](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/e/c/ec71cb5f870af24d0d94041b8f6e4d897e0d1a30.png)

So some ASVs do now overlap.........i need to think about what this means and if i should use this data or if i should keep my datasets separately

---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [August 4, 2023, 6:10pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/11 "2023-08-04T18:10:49Z")

</div>



---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [August 8, 2023, 9:06pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/12 "2023-08-08T21:06:33Z")

</div>

Interesting. I suspect the different primer length is resulting in slightly different ASV sequences, which is resulting in the behavior that you're seeing.

A couple of questions:

First, I am not sure what each of the tables you provided in your last message is exactly (i.e., which run, and why four tables for the two runs) - apologies if I'm missing something, just jumping back into this after a couple of weeks.

Second, I would want to know if the shared ASVs make up the majority of the ASVs in terms of relative abundance. If so, that suggests that this is probably working ok, and that you have a lot of low abundance ASVs (which isn't super uncommon). You could figure this out by [filtering the feature table, say to features that show up in at least 10 samples](https://docs.qiime2.org/2023.5/tutorials/filtering/#contingency-based-filtering), and then reassess the fraction of ASVs that are shared.

> [@spongebob](#):
>
> - I have already classified each dataset - can i filter that taxonomy file to get the ASVs classifications for my clustered data or will i need to run it again?

You should be able to use the existing `FeatureData[Taxonomy]`, as long as the ASV IDs haven't changed. You can always have extra feature ids in this file, relative to the feature ids in the table that you're working with.

> [@spongebob](#):
>
> - whats would be wrong with combining the forward and reverse reads from each dataset (after cutadapt) and then running dada2?

DADA2's error model assumes that all samples were sequenced on the same run, so it won't perform correctly if you've combined multiple sequencing runs.

---

<div class="post-metadata">

**Author:** ![spongebob](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/s/90db22/32.png) [@spongebob](https://forum.qiime2.org/u/spongebob)\
**Post date:** [August 14, 2023, 6:31pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/13 "2023-08-14T18:31:36Z")

</div>

4 tables as i posted the results of before and after running cutadapt - just to show cutadapt did work.

I played around with the shared ASVs and the amount shared makes sense for our data. So i think we are happy to move forward with it.

Thanks for the tips and help with this issue!

---

<div class="post-metadata">

**Author:** ![lizgehret](https://forum.qiime2.org/user_avatar/forum.qiime2.org/lizgehret/32/17128_2.png) [@lizgehret](https://forum.qiime2.org/u/lizgehret)\
**Post date:** [August 14, 2023, 7:51pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/14 "2023-08-14T19:51:03Z")

</div>



---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [August 14, 2023, 9:20pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/16 "2023-08-14T21:20:23Z")

</div>



---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [August 15, 2023, 2:32pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/19 "2023-08-15T14:32:39Z")

</div>

Hi @spongebob, Glad things seem to be looking good now.

I wanted to share a comment that @SoilRotifer shared with me directly:

> [@SoilRotifer](#):
>
> I noticed that their cutadapt command is missing `--p-discard-untrimmed`. This can be quite important as, if either primer is not detected and removed, the resulting sequences will remain within the output (_i.e._ sequences with untrimmed primers, and a slightly longer length). This can result in primer-pair / dataset specific ASV inflation, i.e. non-overlapping ASVs.

Let us know if you have questions about this.

---

<div class="post-metadata">

**Author:** ![system](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/3X/2/1/21af5fe23cb6f4579467c66a9ed94e55274ca7bd.svg) [@system](https://forum.qiime2.org/u/system)\
**Post date:** [September 15, 2023, 8:32pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/20 "2023-09-15T20:32:42Z")

</div>

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.

---

<div class="post-metadata">

**Author:** ![gregcaporaso](https://forum.qiime2.org/user_avatar/forum.qiime2.org/gregcaporaso/32/17769_2.png) [@gregcaporaso](https://forum.qiime2.org/u/gregcaporaso)\
**Post date:** [October 2, 2023, 4:32pm UTC](https://forum.qiime2.org/t/merging-sequence-runs-with-slight-primer-variation/27142/21 "2023-10-02T16:32:01Z")

</div>


