DADA2 error for 18S paired end reads

User Support
,
1

Hi
I have a similar problem. I am denoising 11 samples paired end reads for 18S amplicon. I am running the qiime2-amplicon-2024.5 installed natively in my macbook pro 2.3. GHz 8 core intel core i9 with 64GB RAM and 1TB hard disk. I am running it in the terminal.

Here is the demux.qzv:
18S-demux.qzv (310.1 KB)

This is the command I am using for DADA2 denoising:
qiime dada2 denoise-paired
--i-demultiplexed-seqs 18S-demux-paired-end.qza
--p-trunc-len-f 0
--p-trunc-len-r 0
--p-n-threads 0
--o-table 18S-table.qza
--o-representative-sequences 18S-rep-seqs.qza
--o-denoising-stats 18S-stats.qza
--verbose
It has been running for 10 hours and 36 minutes.
Here is the R learning error rate:
3) Learning Error Rates
415177372 total bases in 2773272 reads from 1 samples will be used for learning the error rates.

This is the third time I tried running DADA2 and it the two previous cases gave me the same error as the one @Adyasha_Panda is getting, i.e. "An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more."

The previous times I didn't use the --verbose command so I can't tell where it got stuck and I don't know how to access the stdout and stderror logs.
I have also tried the denoising the samples using q2galaxy with the same result. I think in this case was a memory capacity issue.
Any help would be greatly appreciated.
Happy Thanksgiving
Fernando

2

BTW here are the processes going on right now:
Processes: 519 total, 3 running, 516 sleeping, 3161 threads 10:46:34
Load Avg: 6.84, 7.70, 7.62 CPU usage: 19.93% user, 1.81% sys, 78.25% idle
SharedLibs: 529M resident, 90M data, 54M linkedit.
MemRegions: 674003 total, 15G resident, 303M private, 2875M shared.
PhysMem: 52G used (4447M wired), 12G unused.
VM: 48T vsize, 2318M framework vsize, 0(0) swapins, 0(0) swapouts.
Networks: packets: 1034996/986M in, 857148/280M out.
Disks: 1731146/32G read, 837958/26G written.

PID COMMAND %CPU TIME #TH #WQ #PORT MEM PURG CMPRS PGRP
4637 R 306.3 03:18:28 17/1 0 293 6825M 0B 0B 4635
518 Finder 15.6 10:20.51 9 7 807 98M 136K 0B 518
155 WindowServer 8.8 21:27.83 27 17 1798- 356M- 2304K 0B 155
5641 top 8.6 03:13.04 1/1 0 39 4804K 0B 0B 5641
663 Spotify Help 6.8 13:01.76 41 1 313 156M 0B 0B 637
3769 Terminal 3.0 00:40.79 12 7 305+ 54M- 16M+ 0B 3769
0 kernel_task 2.5 07:14.57 286/16 0 0 79M- 0B 0B 0
380 TouchBarServ 1.7 00:55.67 7 3 373+ 22M+ 2688K 0B 380
146 bluetoothd 0.8 00:21.95 4 2 205+ 4020K+ 0B 0B 146
468 Google Chrom 0.8 12:41.12 41 1 1177 302M 236K 0B 468
427 dorado_basec 0.6 01:06.49 55 0 77 47M 0B 0B 109
2514 RdrCEF Helpe 0.2 00:15.69 9 1 193 40M 0B 0B 2509
564 Google Chrom 0.2 02:51.45 24 1 305+ 59M+ 0B 0B 468
542 sharingd 0.1 00:12.47 3 1 265 11M 1728K 0B 542

The R error process is using between 250 and 320% CPU. I have a feeling that is where the problem is.
Thank you again
Regards
Fernando

3

Hello Fernando,

I think this may be a different issue, and if it is I'll split up these two threads to avoid bothering Adyasha.

That's good! That's means it's still running :hourglass_flowing_sand:

The previous times I didn't use the --verbose command so I can't tell where it got stuck and I don't know how to access the stdout and stderror logs.

If you do get the error again, then we will need to logs.
No logs -> no clues -> no idea! :person_shrugging:

From the terminal on your mac, try running
open .
If you were using WSL on Windows, the equlivent command is
explorer.exe .

That will open the current working directory so you can look at the files without using Linux terminal commands.

4

GM @colinbrislawn
I hope all is welll with you today.
Following on this thread, the dada2 run ended with this error:

"*R version 4.3.3 (2024-02-29) *
Loading required package: Rcpp
*DADA2: 1.30.0 / Rcpp: 1.0.12 / RcppParallel: 5.1.6 *
2) Filtering ...........
3) Learning Error Rates
415177372 total bases in 2773272 reads from 1 samples will be used for learning the error rates.
open .
*Error in derepFastq(fls[[i]], qualityType = qualityType) : *

  • Not all provided files exist.*
    5: stop("Not all provided files exist.")
    4: derepFastq(fls[[i]], qualityType = qualityType)
    3: learnErrors(filtsR, nreads = nreads.learn, multithread = multithread)
    *2: withCallingHandlers(expr, warning = function(w) if (inherits(w, *
  •   classes)) tryInvokeRestart("muffleWarning"))*

1: suppressWarnings(learnErrors(filtsR, nreads = nreads.learn, multithread = multithread))
Traceback (most recent call last):

  • File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2_dada2/_denoise.py", line 350, in denoise_paired*
  • run_commands([cmd])*
  • File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2_dada2/_denoise.py", line 37, in run_commands*
  • subprocess.run(cmd, check=True)*
  • File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/subprocess.py", line 528, in run*
  • raise CalledProcessError(retcode, process.args,*
    subprocess.CalledProcessError: Command '['run_dada.R', '--input_directory', '/var/folders/sh/bt1662zx00j4d45h9855jqy00000gn/T/tmp9oibcr5r/forward', '--input_directory_reverse', '/var/folders/sh/bt1662zx00j4d45h9855jqy00000gn/T/tmp9oibcr5r/reverse', '--output_path', '/var/folders/sh/bt1662zx00j4d45h9855jqy00000gn/T/tmp9oibcr5r/output.tsv.biom', '--output_track', '/var/folders/sh/bt1662zx00j4d45h9855jqy00000gn/T/tmp9oibcr5r/track.tsv', '--filtered_directory', '/var/folders/sh/bt1662zx00j4d45h9855jqy00000gn/T/tmp9oibcr5r/filt_f', '--filtered_directory_reverse', '/var/folders/sh/bt1662zx00j4d45h9855jqy00000gn/T/tmp9oibcr5r/filt_r', '--truncation_length', '0', '--truncation_length_reverse', '0', '--trim_left', '0', '--trim_left_reverse', '0', '--max_expected_errors', '2.0', '--max_expected_errors_reverse', '2.0', '--truncation_quality_score', '2', '--min_overlap', '12', '--pooling_method', 'independent', '--chimera_method', 'consensus', '--min_parental_fold', '1.0', '--allow_one_off', 'False', '--num_threads', '0', '--learn_min_reads', '1000000']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  • File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2cli/commands.py", line 520, in call*
  • results = self._execute_action(*
  • File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2cli/commands.py", line 581, in _execute_action*
  • results = action(*arguments)
  • File "", line 2, in denoise_paired*
  • File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 342, in bound_callable*
  • outputs = self.callable_executor(*
  • File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 576, in callable_executor*
  • output_views = self._callable(*view_args)
  • File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2_dada2/_denoise.py", line 363, in denoise_paired*
  • raise Exception("An error was encountered while running DADA2"*
    Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

  • An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.*

See above for debug info."

I don't know how to access the sdout and stderr files. I am not sure if this is the problem:
5: stop("Not all provided files exist.")
but I don't know what it means.
Thank you and Happy Thanksgiving.
Fernando