DADA2 simply won't work

[Liviacmg]

Hey guys,

I am facing an error on DADA2 similar to this other topic:

Q2-dada2: Error in isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) : Input must be a valid sequence table

Hello, I got the error below at the step of chimera detection. I wonder if this is that there were no sequences left for chimera detection at the end of that stage. Running qiime 2 Feb 2018 version. Let me know if you need any further info System versions Python version: 3.5.5 QIIME 2 release: 2018.2 QIIME 2 version: 2018.2.0 q2cli version: 2018.2.0 Installed plugins alignment 2018.2.0 composition 2018.2.0 cutadapt 2018.2.0 dada2 2018.2.0 deblur 2018.2.0 demux 2018.2.0 diversi…

Although, it doesn't seem to be the manifest file because I checked and everything is ok.

Also, I tried to run with the paired-end-sequences.qza without trimming 4 times with various parameters and it simply won't work, all the attempts resulted on "inputs non chimerics" 0 and table.qza super weird.

(I tried these before, with paired-end-demux.qza as input:

#non-chimeric inputs from "stats" returned 0:
--p-trim-left-f 0
--p-trim-left-r 1
--p-trunc-len-f 186
--p-trunc-len-r 154 \

#non-chimeric inputs from "stats" returned 0:
--p-trim-left-f 0
--p-trim-left-r 1
--p-trunc-len-f 204
--p-trunc-len-r 172 \

#non-chimeric inputs from "stats" returned 0:
--p-trim-left-f 0
--p-trim-left-r 1
--p-trunc-len-f 183
--p-trunc-len-r 180 \

#returned error:
--p-trim-left-f 0
--p-trim-left-r 1
--p-trunc-len-f 155
--p-trunc-len-r 140 \

Then, I used cutadapt to see if it would somehow help, but no. Now it only results on error...

I tried these:

#returned error:
--i-demultiplexed-seqs trimmed_sequences.qza
--p-trim-left-f 1
--p-trim-left-r 3
--p-trunc-len-f 201
--p-trunc-len-r 156 \

#returned error:
--i-demultiplexed-seqs trimmed_sequences.qza
--p-trim-left-f 0
--p-trim-left-r 1
--p-trunc-len-f 186
--p-trunc-len-r 154 \

Can anyone help me? I don't know what to do anymore. I am attaching one of my attempts too see if it helps somehow...

paired-end-demux.qzv (313.0 KB)

trimmed_sequences.qzv (317.8 KB)

table-dada2.qzv (404.4 KB)

table-dada2.qzv (404.4 KB)

I am using qiime2-2023.5, installed through conda.

qiime dada2 denoise-paired
--i-demultiplexed-seqs trimmed_sequences.qza
--p-trim-left-f 0
--p-trim-left-r 3
--p-trunc-len-f 201
--p-trunc-len-r 169
--o-table table.qza
--o-representative-sequences rep-seqs.qza
--o-denoising-stats denoising-stats.qza
--verbose

Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada.R --input_directory /tmp/tmp6nq2g8zb/forward --input_directory_reverse /tmp/tmp6nq2g8zb/reverse --output_path /tmp/tmp6nq2g8zb/output.tsv.biom --output_track /tmp/tmp6nq2g8zb/track.tsv --filtered_directory /tmp/tmp6nq2g8zb/filt_f --filtered_directory_reverse /tmp/tmp6nq2g8zb/filt_r --truncation_length 201 --truncation_length_reverse 169 --trim_left 0 --trim_left_reverse 3 --max_expected_errors 2.0 --max_expected_errors_reverse 2.0 --truncation_quality_score 2 --min_overlap 12 --pooling_method independent --chimera_method consensus --min_parental_fold 1.0 --allow_one_off False --num_threads 1 --learn_min_reads 1000000

R version 4.2.3 (2023-03-15)
Loading required package: Rcpp
DADA2: 1.26.0 / Rcpp: 1.0.10 / RcppParallel: 5.1.6
2) Filtering .....................................................................................................................
3) Learning Error Rates
202946082 total bases in 1009682 reads from 106 samples will be used for learning the error rates.
167607212 total bases in 1009682 reads from 106 samples will be used for learning the error rates.
3) Denoise samples .....................................................................................................................
.....................................................................................................................
5) Remove chimeras (method = consensus)
Error in isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) :
Input must be a valid sequence table.
Calls: removeBimeraDenovo -> isBimeraDenovoTable
3: stop("Input must be a valid sequence table.")
2: isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose)
1: removeBimeraDenovo(seqtab, method = chimeraMethod, minFoldParentOverAbundance = minParentFold,
allowOneOff = allowOneOff, multithread = multithread)
Traceback (most recent call last):
File "/usr/local/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 326, in denoise_paired
run_commands([cmd])
File "/usr/local/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
subprocess.run(cmd, check=True)
File "/usr/local/miniconda3/envs/qiime2-2023.5/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['run_dada.R', '--input_directory', '/tmp/tmp6nq2g8zb/forward', '--input_directory_reverse', '/tmp/tmp6nq2g8zb/reverse', '--output_path', '/tmp/tmp6nq2g8zb/output.tsv.biom', '--output_track', '/tmp/tmp6nq2g8zb/track.tsv', '--filtered_directory', '/tmp/tmp6nq2g8zb/filt_f', '--filtered_directory_reverse', '/tmp/tmp6nq2g8zb/filt_r', '--truncation_length', '201', '--truncation_length_reverse', '169', '--trim_left', '0', '--trim_left_reverse', '3', '--max_expected_errors', '2.0', '--max_expected_errors_reverse', '2.0', '--truncation_quality_score', '2', '--min_overlap', '12', '--pooling_method', 'independent', '--chimera_method', 'consensus', '--min_parental_fold', '1.0', '--allow_one_off', 'False', '--num_threads', '1', '--learn_min_reads', '1000000']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2cli/commands.py", line 468, in call
results = action(**arguments)
File "", line 2, in denoise_paired
File "/usr/local/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 274, in bound_callable
outputs = self.callable_executor(
File "/usr/local/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/qiime2/sdk/action.py", line 509, in callable_executor
output_views = self._callable(**view_args)
File "/usr/local/miniconda3/envs/qiime2-2023.5/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 339, in denoise_paired
raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

See above for debug info.

[SoilRotifer (Mike Robeson)]

Hi @Liviacmg,

I think the issue has to do with the fact that not many of your read pairs are merging, resulting in an empty, or nearly empty table. Your truncation values should take into account the overlap you'll need to merge the reads together. DADA2 defaults to requiring 12 bases of overlap between the forward and reverse read.

What amplicon region are you sequencing? What is the expected final length of your merged reads?

[Liviacmg]

Hi @SoilRotifer ,

The region is V3-V4 and I don't know the expected final length of my merged reads hehe where can I get this info?

[timanix (Timur Yergaliyev)]

Hello!
@SoilRotifer , I hope that it is ok that I will qiime in. Please correct me if I will mislead here.

@Liviacmg

The region is V3-V4 and I don't know the expected final length of my merged reads

With the V3-V4 region and after primers removal one can expect 2 batches of reads with lengths around 410 and 430. Add to it an overlapping region * 2, and you can calculate the lengths you need to merge reads. Quality plots of reverse reads are not so great and they definitely should be truncated. I am afraid, that they would not overlap after it. It looks like your dataset is the classic example of why V3-V4 should be sequenced with 2x300 instead of 2x250. In a similar situation, I ended up discarding reverse reads and analyzing only forward reads.

Best,

[SoilRotifer (Mike Robeson)]

To echo @timanix comments, I am currently in the same boat. I am analyzing V3V4 data, using 2x300. Even then, the quality was barely good enough to merge my reads. On average only ~50-60% of my reads are merging. I am considering analyzing my data with only the forward reads too.

[Liviacmg]

Hi @timanix ,

Thank you so much for your response!! I will only use forward reads then!! After several tries, the maximum I got was ~20% of merged reads, which was too low. Now should I treat this data as it was single-end? (only considering forward reads)

[colinvwood (Colin Wood)]

Yes, i.e. use denoise-single.

[Liviacmg]

Thank you @colinvwood !