DADA2 Denoising CCS error in learning error rates

Hi @Oddant1,

I was trying to work on denoising step (a new set of samples). And it gives me following error.
I tried with solutions for OOM and didn't' work.
Looks like it is an error in dada2.
Your help is much appreciated.

  • Version of QIIME 2 you are running, and how it is installed (e.g. Virtualbox, conda, etc.)

qiime2-amplicon-2024.2
Virtual Machine: WSL, Ubuntu

  • What is the exact command or commands you ran? Copy and paste please.
    qiime dada2 denoise-ccs
    --i-demultiplexed-seqs reads_qzaMice/raw_readsMice.qza
    --p-min-len 300 --p-max-len 900
    --p-front TAGAGGAAGTAAAAGTCGTAA --p-adapter TCCTCCGCTTWTTGWTWTGC
    --p-n-threads 16
    --o-table dada2_outputMice/tableMice.qza
    --o-representative-sequences dada2_outputMice/representative_sequencesMice.qza
    --o-denoising-stats dada2_outputMice/statsMice.qza
    --verbose

  • What is the exact error message? If you didn't run the command with the --verbose flag, please re-run and copy-and-paste the results.

  1. Filtering The filter removed all reads: /tmp/tmpp726hcru/filt/sample-AR27_26_L001_R1_001.fastq.gz not written.
    Some input samples had no reads pass the filter.
    ................x...........
  2. Learning Error Rates
    55231 total bases in 98 reads from 27 samples will be used for learning the error rates.
    Error in dada(drps, err = NULL, errorEstimationFunction = errorEstimationFunction, :
    Invalid derep$quals matrix. Quality values must be positive integers.
    5: stop("Invalid derep$quals matrix. Quality values must be positive integers.")
    4: dada(drps, err = NULL, errorEstimationFunction = errorEstimationFunction,
    selfConsist = TRUE, multithread = multithread, verbose = verbose,
    MAX_CONSIST = MAX_CONSIST, OMEGA_C = OMEGA_C, ...)
    3: learnErrors(filts, nreads = nreads.learn, errorEstimationFunction = dada2:::PacBioErrfun,
    multithread = multithread, BAND_SIZE = BAND_SIZE)
    2: withCallingHandlers(expr, warning = function(w) if (inherits(w,
    classes)) tryInvokeRestart("muffleWarning"))
    1: suppressWarnings(learnErrors(filts, nreads = nreads.learn, errorEstimationFunction = dada2:::PacBioErrfun,
    multithread = multithread, BAND_SIZE = BAND_SIZE))
    Traceback (most recent call last):
    File "/home/sathya/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 467, in denoise_ccs
    run_commands([cmd])
    File "/home/sathya/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 37, in run_commands
    subprocess.run(cmd, check=True)
    File "/home/sathya/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['run_dada.R', '--input_directory', '/tmp/qiime2/sathya/data/80d190c9-1a54-405a-b81e-bc9bf7eef3c6/data', '--output_path', '/tmp/tmpp726hcru/output.tsv.biom', '--output_track', '/tmp/tmpp726hcru/track.tsv', '--removed_primer_directory', '/tmp/tmpp726hcru/nop', '--filtered_directory', '/tmp/tmpp726hcru/filt', '--forward_primer', 'TAGAGGAAGTAAAAGTCGTAA', '--reverse_primer', 'TCCTCCGCTTWTTGWTWTGC', '--max_mismatch', '2', '--indels', 'False', '--truncation_length', '0', '--trim_left', '0', '--max_expected_errors', '2.0', '--truncation_quality_score', '2', '--min_length', '300', '--max_length', '900', '--pooling_method', 'independent', '--chimera_method', 'consensus', '--min_parental_fold', '3.5', '--allow_one_off', 'False', '--num_threads', '16', '--learn_min_reads', '1000000', '--homopolymer_gap_penalty', 'NULL', '--band_size', '32']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/sathya/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 520, in call
results = self._execute_action(
File "/home/sathya/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 581, in _execute_action
results = action(**arguments)
File "", line 2, in denoise_ccs
File "/home/sathya/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self.callable_executor(
File "/home/sathya/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in callable_executor
output_views = self._callable(**view_args)
File "/home/sathya/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 476, in denoise_ccs
raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

See above for debug info.

Hi @Sathya-Am
Looks like this is your error:

Have you looked at your demux.qzv? are the values not positive integers?

No, I don't see any. But my interactive quality plot is like this.


and I got the similar looking plot when I did the previous sample set. It was due to out of range values as per my understanding.

Values are positive integers. I don't see any negative values.

Hi @Sathya-Am,
Do you have a warning at the top of that demux viz saying that the values are out of range?
This usually indicates that the sequences were imported wrong

This is the warning I get.
Danger: Some of the forward PHRED quality values are out of range. This is likely because an incorrect PHRED offset was chosen on import of your raw data. You can learn how to choose your PHRED offset during import in the importing tutorial.

I got the same warning earlier for another set. But, it didn't affect denoising.

Here I used ```
SingleEndFastqManifestPhred33V2
and tried with Phred64 earlier which generated the same warning.

@Sathya-Am,
Does the Phred64 attempt give you the same dada2 error?

For, this set o samples I didn't try the Phred64. I will try it and see what happens.

1 Like

Okay hopefully that helps! let me know what happens!

1 Like

Hi again,
When I use PHRED64 it gives me this error.
Decoded Phred score is out of range [0, 62].
But, PHRED33 works and generate the qza file.

Hi @Sathya-Am,

Okay! So it seems like that isn't the issue.

Just to confirm you are using Pacbio CCS sequences?

Yes, it is PacBio 16s CCS full length sequences. And I performed the same analysis for another sample set and I was able to proceed even though I see the same pattern and warning in interactive quality plot. I used PHRED33 for that too and tried with PHRED64 that didn't work.

Hi @cherman2 again, Mistake was mine. I was using the wrong length values. Now it is good.
Thank you for your time!

2 Likes