Plugin error from dada2 using 18S Amplicon data

Adyasha_Panda · November 18, 2024, 3:16pm

Hello,
I am running qiime2 2024.10 version.
I am running this command for DADA2 de-noising, every time I am getting this error.
qiime dada2 denoise-paired --i-demultiplexed-seqs demux-paired-end.qza --p-trunc-len-f 230 --p-trunc-len-r 230 --p-n-threads 4 --p-n-reads-learn 1000 --o-table table.qza --o-representative-sequences rep-seqs.qza --o-denoising-stats denoising-stats.qza
Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Debug info has been saved to /tmp/qiime2-q2cli-err-ft74jgtp.log

When i ran this to check the log file, I got this :
cat /tmp/qiime2-q2cli-err-ft74jgtp.log
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada.R --input_directory /tmp/tmp7hkkeldb/forward --input_directory_reverse /tmp/tmp7hkkeldb/reverse --output_path /tmp/tmp7hkkeldb/output.tsv.biom --output_track /tmp/tmp7hkkeldb/track.tsv --filtered_directory /tmp/tmp7hkkeldb/filt_f --filtered_directory_reverse /tmp/tmp7hkkeldb/filt_r --truncation_length 230 --truncation_length_reverse 230 --trim_left 0 --trim_left_reverse 0 --max_expected_errors 2.0 --max_expected_errors_reverse 2.0 --truncation_quality_score 2 --min_overlap 12 --pooling_method independent --chimera_method consensus --min_parental_fold 1.0 --allow_one_off False --num_threads 4 --learn_min_reads 1000

R version 4.3.3 (2024-02-29)
Loading required package: Rcpp
DADA2: 1.30.0 / Rcpp: 1.0.13.1 / RcppParallel: 5.1.9
2) Filtering ........
3) Learning Error Rates
375360 total bases in 1632 reads from 1 samples will be used for learning the error rates.
Error rates could not be estimated (this is usually because of very few reads).
Error in getErrors(err, enforce = TRUE) : Error matrix is NULL.
6: stop("Error matrix is NULL.")
5: getErrors(err, enforce = TRUE)
4: dada(drps, err = NULL, errorEstimationFunction = errorEstimationFunction,
selfConsist = TRUE, multithread = multithread, verbose = verbose,
MAX_CONSIST = MAX_CONSIST, OMEGA_C = OMEGA_C, ...)
3: learnErrors(filts, nreads = nreads.learn, multithread = multithread)
2: withCallingHandlers(expr, warning = function(w) if (inherits(w,
classes)) tryInvokeRestart("muffleWarning"))
1: suppressWarnings(learnErrors(filts, nreads = nreads.learn, multithread = multithread))
Traceback (most recent call last):
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2_dada2/_denoise.py", line 353, in denoise_paired
run_commands([cmd])
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2_dada2/_denoise.py", line 38, in run_commands
subprocess.run(cmd, check=True)
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['run_dada.R', '--input_directory', '/tmp/tmp7hkkeldb/forward', '--input_directory_reverse', '/tmp/tmp7hkkeldb/reverse', '--output_path', '/tmp/tmp7hkkeldb/output.tsv.biom', '--output_track', '/tmp/tmp7hkkeldb/track.tsv', '--filtered_directory', '/tmp/tmp7hkkeldb/filt_f', '--filtered_directory_reverse', '/tmp/tmp7hkkeldb/filt_r', '--truncation_length', '230', '--truncation_length_reverse', '230', '--trim_left', '0', '--trim_left_reverse', '0', '--max_expected_errors', '2.0', '--max_expected_errors_reverse', '2.0', '--truncation_quality_score', '2', '--min_overlap', '12', '--pooling_method', 'independent', '--chimera_method', 'consensus', '--min_parental_fold', '1.0', '--allow_one_off', 'False', '--num_threads', '4', '--learn_min_reads', '1000']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/commands.py", line 530, in call
results = self._execute_action(
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/commands.py", line 602, in _execute_action
results = action(**arguments)
File "", line 2, in denoise_paired
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/sdk/action.py", line 299, in bound_callable
outputs = self.callable_executor(
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/sdk/action.py", line 570, in callable_executor
output_views = self._callable(**view_args)
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2_dada2/_denoise.py", line 366, in denoise_paired
raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

I am unable to understand this. Please help me in fixing this .
Thank you.

colinbrislawn · November 18, 2024, 3:23pm

Hello Adyasha,

Here are the docs for the plugin you are running:
https://docs.qiime2.org/2024.10/plugins/available/dada2/denoise-paired/

Here is the core of the error:

This plugin is designed for paired-end Illumina data. Are you still working with Oxford nanopore MinIon data, as you mentioned in your previous thread?

Adyasha_Panda · November 18, 2024, 3:54pm

Hello ,
No for this I'm using illumina 18S amplicon data .
It's a public dataset.
Project ID : PRJNA936461
I am trying to replicate the study using qiime2 pipeline.
Thank you.

colinbrislawn · November 18, 2024, 4:05pm

Okay!

Then the issue is likely due to the number of reads for training:

Try rerunning this with more reads for the learning step. I think 1 million is the default, so 1k is pretty low.

Let us know how it goes!

Adyasha_Panda · November 19, 2024, 5:11am

Hi, I Have tried running with more reads, but its throwing errors.

This is the summary of my demux-paired-end.qzv file

colinbrislawn · November 19, 2024, 11:59pm

Can you post the full error file here? I can see a few lines, but I think there will be more clues in the full file.

Thanks!

Adyasha_Panda · November 20, 2024, 11:02am

Hi,
These are the fastq files that I am using
-rw-r--r-- 1 adyasha adyasha 897241 Nov 19 15:05 SRR23517960_1.fastq.gz
-rw-r--r-- 1 adyasha adyasha 1027907 Nov 19 15:05 SRR23517960_2.fastq.gz
-rw-r--r-- 1 adyasha adyasha 666420 Nov 19 15:05 SRR23517961_1.fastq.gz
-rw-r--r-- 1 adyasha adyasha 746378 Nov 19 15:05 SRR23517961_2.fastq.gz
SRR23517962_1.fastq.gz
SRR23517962_2.fastq.gz
SRR23517963_1.fastq.gz
SRR23517963_2.fastq.gz
SRR23517964_1.fastq.gz
SRR23517964_2.fastq.gz
SRR23517965_1.fastq.gz
SRR23517965_2.fastq.gz
SRR23517966_1.fastq.gz
SRR23517966_2.fastq.gz
SRR23517967_1.fastq.gz
SRR23517967_2.fastq.gz

First I have used this command
qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path manifest.tsv
--output-path demux.qza
--input-format PairedEndFastqManifestPhred33V2

Then I have ran this command
qiime dada2 denoise-paired --i-demultiplexed-seqs demux.qza --p-trunc-len-f 250 --p-trunc-len-r 200 --o-table table.qza --o-representative-sequences rep-seqs.qza --o-denoising-stats denoising-stats.qza

this gave me this error :
Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Debug info has been saved to /tmp/qiime2-q2cli-err-x2m5r9j9.log

Here is the log file to the error -
cat /tmp/qiime2-q2cli-err-x2m5r9j9.log
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada.R --input_directory /tmp/tmp65t4hu0n/forward --input_directory_reverse /tmp/tmp65t4hu0n/reverse --output_path /tmp/tmp65t4hu0n/output.tsv.biom --output_track /tmp/tmp65t4hu0n/track.tsv --filtered_directory /tmp/tmp65t4hu0n/filt_f --filtered_directory_reverse /tmp/tmp65t4hu0n/filt_r --truncation_length 250 --truncation_length_reverse 200 --trim_left 0 --trim_left_reverse 0 --max_expected_errors 2.0 --max_expected_errors_reverse 2.0 --truncation_quality_score 2 --min_overlap 12 --pooling_method independent --chimera_method consensus --min_parental_fold 1.0 --allow_one_off False --num_threads 1 --learn_min_reads 1000000

R version 4.3.3 (2024-02-29)
Loading required package: Rcpp
DADA2: 1.30.0 / Rcpp: 1.0.13.1 / RcppParallel: 5.1.9
2) Filtering ........
3) Learning Error Rates
4191000 total bases in 16764 reads from 8 samples will be used for learning the error rates.
Error rates could not be estimated (this is usually because of very few reads).
Error in getErrors(err, enforce = TRUE) : Error matrix is NULL.
6: stop("Error matrix is NULL.")
5: getErrors(err, enforce = TRUE)
4: dada(drps, err = NULL, errorEstimationFunction = errorEstimationFunction,
selfConsist = TRUE, multithread = multithread, verbose = verbose,
MAX_CONSIST = MAX_CONSIST, OMEGA_C = OMEGA_C, ...)
3: learnErrors(filts, nreads = nreads.learn, multithread = multithread)
2: withCallingHandlers(expr, warning = function(w) if (inherits(w,
classes)) tryInvokeRestart("muffleWarning"))
1: suppressWarnings(learnErrors(filts, nreads = nreads.learn, multithread = multithread))
Traceback (most recent call last):
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2_dada2/_denoise.py", line 353, in denoise_paired
run_commands([cmd])
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2_dada2/_denoise.py", line 38, in run_commands
subprocess.run(cmd, check=True)
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/subprocess.py", line 526, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['run_dada.R', '--input_directory', '/tmp/tmp65t4hu0n/forward', '--input_directory_reverse', '/tmp/tmp65t4hu0n/reverse', '--output_path', '/tmp/tmp65t4hu0n/output.tsv.biom', '--output_track', '/tmp/tmp65t4hu0n/track.tsv', '--filtered_directory', '/tmp/tmp65t4hu0n/filt_f', '--filtered_directory_reverse', '/tmp/tmp65t4hu0n/filt_r', '--truncation_length', '250', '--truncation_length_reverse', '200', '--trim_left', '0', '--trim_left_reverse', '0', '--max_expected_errors', '2.0', '--max_expected_errors_reverse', '2.0', '--truncation_quality_score', '2', '--min_overlap', '12', '--pooling_method', 'independent', '--chimera_method', 'consensus', '--min_parental_fold', '1.0', '--allow_one_off', 'False', '--num_threads', '1', '--learn_min_reads', '1000000']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/commands.py", line 530, in call
results = self._execute_action(
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2cli/commands.py", line 602, in _execute_action
results = action(**arguments)
File "", line 2, in denoise_paired
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/sdk/action.py", line 299, in bound_callable
outputs = self.callable_executor(
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/qiime2/sdk/action.py", line 570, in callable_executor
output_views = self._callable(**view_args)
File "/home/adyasha/miniconda3/envs/qiime2-amplicon-2024.10/lib/python3.10/site-packages/q2_dada2/_denoise.py", line 366, in denoise_paired
raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Please help me where I am going wrong.

Thank you

colinbrislawn · November 20, 2024, 5:18pm

Here is the core of the error.

That's frustrating because the number of reads look good!

Can you post the plot of quality scores? I wonder if these are Illumina's new binned q-scores...

Adyasha_Panda · November 21, 2024, 4:15am

Hi,

This is the plot

colinbrislawn · November 21, 2024, 3:49pm

That's the problem! Those quality scores are not typical.

For comparison, check out the quality scores from the Atacama-soils tutorial
https://view.qiime2.org/visualization/?src=https://docs.qiime2.org/2024.10/data/tutorials/atacama-soils/demux-subsample.qzv

Has this data been preprocessed in some way?

Ideally, we want to import raw data directly from the Illumina sequencer and then perform all quality filtering and trimming within Qiime2 so we can automatically keep a record of our work!