Good afternoon,
I am trying to run simulated reads through the QIIME2 pipeline and am having issues denoising them with DADA2. The reads were created with CAMISIM (using only rRNA gene nucleotide sequences as the input genomes so as to simulate 16S amplicon sequencing.) Within CAMISIM, the reads were specifically created using either ART or WGSIM. If it helps in diagnosis, all reads from both sets were sucessfully mapped using the Kraken2 pipeline with the Greengenes 16S reference db, and I have never had an issue running the QIIME2 pipeline in my conda environment before when using real reads.
I am running q2cli version 2024.2.0 installed through conda.
The qiime demux summarize data for each read set are as follows:
ART with a variable error rate
WGSIM error rate set to p=0.00032/Phred=35
The initial issue with the ART reads appeared to be in the chimera removal step. When changing the setting to anything but "consensus", I instead received an error saying, "No features remain after denoising. Try adjusting your truncation and trim parameter settings." The error with the WGSIM reads seems to be in DADA2's error rate calculation. I tried playing with the error rate parameter in CAMISIM but I kept getting the same issue with all the WGSIM-derived read sets. The full commands and read outs can be found below. If y'all can help me get either set working I would be greatly appreciated. I'm comparing different analysis pipelines for 16S wastewater metagenomics and so far I have loved QIIME2, so I am really hoping to find a way to make this work so I can keep it in the paper. Any/all suggestions welcome.
Cheers,
Joe
ART
(qiime2-amplicon-2024.2) joeberta@JoesLaptop:~$ qiime dada2 denoise-paired --i-demultiplexed-seqs /mnt/d/qiime/16S/Trim/WB2_Kraken/default10art/WB2_Kraken_default10art_Trim.qza --p-trunc-len-f 0 --p-trunc-len-r 0 --p-chimera-method 'consensus' --verbose --p-n-threads 20 --o-table /mnt/d/qiime/16S/Denoise/WB2_Kraken/default10art/WB2_Kraken_default10art_FreqTable.qza --o-representative-sequences /mnt/d/qiime/16S/Denoise/WB2_Kraken/default10art/WB2_Kraken_default10art_RepSeqs.qza --o-denoising-stats /mnt/d/qiime/16S/Denoise/WB2_Kraken/default10art/WB2_Kraken_default10art_DadaStats.qza
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.
Command: run_dada.R --input_directory /tmp/tmpijk12tx7/forward --input_directory_reverse /tmp/tmpijk12tx7/reverse --output_path /tmp/tmpijk12tx7/output.tsv.biom --output_track /tmp/tmpijk12tx7/track.tsv --filtered_directory /tmp/tmpijk12tx7/filt_f --filtered_directory_reverse /tmp/tmpijk12tx7/filt_r --truncation_length 0 --truncation_length_reverse 0 --trim_left 0 --trim_left_reverse 0 --max_expected_errors 2.0 --max_expected_errors_reverse 2.0 --truncation_quality_score 2 --min_overlap 12 --pooling_method independent --chimera_method consensus --min_parental_fold 1.0 --allow_one_off False --num_threads 20 --learn_min_reads 1000000
Warning message:
package ‘optparse’ was built under R version 4.2.3
R version 4.2.2 (2022-10-31)
Loading required package: Rcpp
DADA2: 1.26.0 / Rcpp: 1.0.12 / RcppParallel: 5.1.6
2) Filtering .
3) Learning Error Rates
27094 total bases in 183 reads from 1 samples will be used for learning the error rates.
11143 total bases in 183 reads from 1 samples will be used for learning the error rates.
3) Denoise samples .
.
5) Remove chimeras (method = consensus)
Error in isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) :
Input must be a valid sequence table.
Calls: removeBimeraDenovo -> isBimeraDenovoTable
3: stop("Input must be a valid sequence table.")
2: isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose)
1: removeBimeraDenovo(seqtab, method = chimeraMethod, minFoldParentOverAbundance = minParentFold,
allowOneOff = allowOneOff, multithread = multithread)
Traceback (most recent call last):
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 350, in denoise_paired
run_commands([cmd])
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 37, in run_commands
subprocess.run(cmd, check=True)
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['run_dada.R', '--input_directory', '/tmp/tmpijk12tx7/forward', '--input_directory_reverse', '/tmp/tmpijk12tx7/reverse', '--output_path', '/tmp/tmpijk12tx7/output.tsv.biom', '--output_track', '/tmp/tmpijk12tx7/track.tsv', '--filtered_directory', '/tmp/tmpijk12tx7/filt_f', '--filtered_directory_reverse', '/tmp/tmpijk12tx7/filt_r', '--truncation_length', '0', '--truncation_length_reverse', '0', '--trim_left', '0', '--trim_left_reverse', '0', '--max_expected_errors', '2.0', '--max_expected_errors_reverse', '2.0', '--truncation_quality_score', '2', '--min_overlap', '12', '--pooling_method', 'independent', '--chimera_method', 'consensus', '--min_parental_fold', '1.0', '--allow_one_off', 'False', '--num_threads', '20', '--learn_min_reads', '1000000']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 520, in __call__
results = self._execute_action(
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 581, in _execute_action
results = action(**arguments)
File "<decorator-gen-73>", line 2, in denoise_paired
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self._callable_executor_(
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in _callable_executor_
output_views = self._callable(**view_args)
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 363, in denoise_paired
raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.
Plugin error from dada2:
An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.
See above for debug info.
Art-derived CAMISIM reads
(qiime2-amplicon-2024.2) joeberta@JoesLaptop:~$ # Art-derived CAMISIM reads
qiime dada2 denoise-paired \
--i-demultiplexed-seqs /mnt/d/qiime/16S/Trim/WB2_Kraken/default10art/WB2_Kraken_default10art_Trim.qza \
--p-trunc-len-f 0 --p-trunc-len-r 0 --p-chimera-method 'none' --verbose \
--p-n-threads 20 --o-table /mnt/d/qiime/16S/Denoise/WB2_Kraken/default10art/WB2_Kraken_default10art_FreqTable.qza \
--o-representative-sequences /mnt/d/qiime/16S/Denoise/WB2_Kraken/default10art/WB2_Kraken_default10art_RepSeqs.qza \
--o-denoising-stats /mnt/d/qiime/16S/Denoise/WB2_Kraken/default10art/WB2_Kraken_default10art_DadaStats.qza
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.
Command: run_dada.R --input_directory /tmp/tmpklho1lg7/forward --input_directory_reverse /tmp/tmpklho1lg7/reverse --output_path /tmp/tmpklho1lg7/output.tsv.biom --output_track /tmp/tmpklho1lg7/track.tsv --filtered_directory /tmp/tmpklho1lg7/filt_f --filtered_directory_reverse /tmp/tmpklho1lg7/filt_r --truncation_length 0 --truncation_length_reverse 0 --trim_left 0 --trim_left_reverse 0 --max_expected_errors 2.0 --max_expected_errors_reverse 2.0 --truncation_quality_score 2 --min_overlap 12 --pooling_method independent --chimera_method none --min_parental_fold 1.0 --allow_one_off False --num_threads 20 --learn_min_reads 1000000
Warning message:
package ‘optparse’ was built under R version 4.2.3
R version 4.2.2 (2022-10-31)
Loading required package: Rcpp
DADA2: 1.26.0 / Rcpp: 1.0.12 / RcppParallel: 5.1.6
2) Filtering .
3) Learning Error Rates
27094 total bases in 183 reads from 1 samples will be used for learning the error rates.
11143 total bases in 183 reads from 1 samples will be used for learning the error rates.
3) Denoise samples .
.
5) Remove chimeras (method = none)
6) Report read numbers through the pipeline
7) Write output
Traceback (most recent call last):
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 520, in __call__
results = self._execute_action(
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 581, in _execute_action
results = action(**arguments)
File "<decorator-gen-73>", line 2, in denoise_paired
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self._callable_executor_(
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in _callable_executor_
output_views = self._callable(**view_args)
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 367, in denoise_paired
return _denoise_helper(biom_fp, track_fp, hashed_feature_ids,
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 120, in _denoise_helper
_check_featureless_table(biom_fp)
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 46, in _check_featureless_table
raise ValueError("No features remain after denoising. Try adjusting "
ValueError: No features remain after denoising. Try adjusting your truncation and trim parameter settings.
Plugin error from dada2:
No features remain after denoising. Try adjusting your truncation and trim parameter settings.
See above for debug info.
WGSIM
(qiime2-amplicon-2024.2) joeberta@JoesLaptop:~$ # WGSIM-derived CAMISIM reads
qiime dada2 denoise-paired \
--i-demultiplexed-seqs /mnt/d/qiime/16S/Trim/WB2_Kraken/default10wgsim00032/WB2_Kraken_default10wgsim00032_Trim.qza \
--p-trunc-len-f 0 --p-trunc-len-r 0 --p-chimera-method 'none' --verbose \
--p-n-threads 20 --o-table /mnt/d/qiime/16S/Denoise/WB2_Kraken/default10wgsim00032/WB2_Kraken_default10wgsim00032_FreqTable.qza \
--o-representative-sequences /mnt/d/qiime/16S/Denoise/WB2_Kraken/default10wgsim00032/WB2_Kraken_default10wgsim00032_RepSeqs.qza \
--o-denoising-stats /mnt/d/qiime/16S/Denoise/WB2_Kraken/default10wgsim00032/WB2_Kraken_default10wgsim00032_DadaStats.qza
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.
Command: run_dada.R --input_directory /tmp/tmpcnkcdtpv/forward --input_directory_reverse /tmp/tmpcnkcdtpv/reverse --output_path /tmp/tmpcnkcdtpv/output.tsv.biom --output_track /tmp/tmpcnkcdtpv/track.tsv --filtered_directory /tmp/tmpcnkcdtpv/filt_f --filtered_directory_reverse /tmp/tmpcnkcdtpv/filt_r --truncation_length 0 --truncation_length_reverse 0 --trim_left 0 --trim_left_reverse 0 --max_expected_errors 2.0 --max_expected_errors_reverse 2.0 --truncation_quality_score 2 --min_overlap 12 --pooling_method independent --chimera_method none --min_parental_fold 1.0 --allow_one_off False --num_threads 20 --learn_min_reads 1000000
Warning message:
package ‘optparse’ was built under R version 4.2.3
R version 4.2.2 (2022-10-31)
Loading required package: Rcpp
DADA2: 1.26.0 / Rcpp: 1.0.12 / RcppParallel: 5.1.6
2) Filtering .
3) Learning Error Rates
48014241 total bases in 320204 reads from 1 samples will be used for learning the error rates.
Error rates could not be estimated (this is usually because of very few reads).
Error in getErrors(err, enforce = TRUE) : Error matrix is NULL.
6: stop("Error matrix is NULL.")
5: getErrors(err, enforce = TRUE)
4: dada(drps, err = NULL, errorEstimationFunction = errorEstimationFunction,
selfConsist = TRUE, multithread = multithread, verbose = verbose,
MAX_CONSIST = MAX_CONSIST, OMEGA_C = OMEGA_C, ...)
3: learnErrors(filts, nreads = nreads.learn, multithread = multithread)
2: withCallingHandlers(expr, warning = function(w) if (inherits(w,
classes)) tryInvokeRestart("muffleWarning"))
1: suppressWarnings(learnErrors(filts, nreads = nreads.learn, multithread = multithread))
Traceback (most recent call last):
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 350, in denoise_paired
run_commands([cmd])
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 37, in run_commands
subprocess.run(cmd, check=True)
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/subprocess.py", line 516, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['run_dada.R', '--input_directory', '/tmp/tmpcnkcdtpv/forward', '--input_directory_reverse', '/tmp/tmpcnkcdtpv/reverse', '--output_path', '/tmp/tmpcnkcdtpv/output.tsv.biom', '--output_track', '/tmp/tmpcnkcdtpv/track.tsv', '--filtered_directory', '/tmp/tmpcnkcdtpv/filt_f', '--filtered_directory_reverse', '/tmp/tmpcnkcdtpv/filt_r', '--truncation_length', '0', '--truncation_length_reverse', '0', '--trim_left', '0', '--trim_left_reverse', '0', '--max_expected_errors', '2.0', '--max_expected_errors_reverse', '2.0', '--truncation_quality_score', '2', '--min_overlap', '12', '--pooling_method', 'independent', '--chimera_method', 'none', '--min_parental_fold', '1.0', '--allow_one_off', 'False', '--num_threads', '20', '--learn_min_reads', '1000000']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 520, in __call__
results = self._execute_action(
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 581, in _execute_action
results = action(**arguments)
File "<decorator-gen-73>", line 2, in denoise_paired
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self._callable_executor_(
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in _callable_executor_
output_views = self._callable(**view_args)
File "/home/joeberta/miniforge-pypy3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 363, in denoise_paired
raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.
Plugin error from dada2:
An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.
See above for debug info.
