DADA2 error : Cannot allocate memory error in denoising step

User Support
1

I know this has been posted multiple times but I am getting this error in the denoising step if DADA 2

  1. Denoise remaining samples …Error in dada_uniques(names(derep[[i]]$uniques), unname(derep[[i]]$uniques), :
    Memory allocation failed.
    Calls: dada -> dada_uniques -> .Call
    Execution halted
    Warning message:
    system call failed: Cannot allocate memory
    Traceback (most recent call last):
    File “/home/basilin/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 152, in _denoise_single
    run_commands([cmd])
    File “/home/basilin/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 36, in run_commands
    subprocess.run(cmd, check=True)
    File “/home/basilin/miniconda3/envs/qiime2-2019.1/lib/python3.6/subprocess.py”, line 418, in run
    output=stdout, stderr=stderr)
    subprocess.CalledProcessError: Command ‘[‘run_dada_single.R’, ‘/tmp/qiime2-archive-45zo3p1n/b9a37e4e-ee08-47f3-8f07-b643d768e68e/data’, ‘/tmp/tmpd1qnfzkg/output.tsv.biom’, ‘/tmp/tmpd1qnfzkg/track.tsv’, ‘/tmp/tmpd1qnfzkg’, ‘294’, ‘0’, ‘2.0’, ‘2’, ‘Inf’, ‘consensus’, ‘1.0’, ‘0’, ‘1000’, ‘NULL’, ‘16’]’ returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/basilin/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2cli/commands.py”, line 274, in call
results = action(**arguments)
File “</home/basilin/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/decorator.py:decorator-gen-440>”, line 2, in denoise_single
File “/home/basilin/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/home/basilin/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 365, in callable_executor
output_views = self._callable(**view_args)
File “/home/basilin/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 187, in denoise_single
band_size=‘16’)
File “/home/basilin/miniconda3/envs/qiime2-2019.1/lib/python3.6/site-packages/q2_dada2/_denoise.py”, line 163, in _denoise_single
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

The command that I ran was:

qiime dada2 denoise-single
–i-demultiplexed-seqs demux/demultiplexed-seqs.qza /
–p-trim-left 0 /
–p-trunc-len 294 /
–p-n-threads 0 /
–p-n-reads-learn 1000 /
–o-representative-sequences rep-seqs-dada2.qza /
–o-table table-dada2.qza /
–o-denoising-stats stats-dada2.qza /
–verbose

2

Hello @basilin_benson,

Yep, this is a common dada2 error:

system call failed: Cannot allocate memory

I'm glad you did a 'literature review' about this error :slightly_smiling_face: What did you learn about this error? Do you want to mention how much memory or RAM your computer has?

Colin

3

My system has 16gb RAM. I have also tried reducing the number of learns but it still throws me the same error

4

Good morning,

You really have reduced the number of reads! Have you tried adjusting the threads? Passing –p-n-threads 0 will use the maximum number of threads. I wonder if using –p-n-threads 2 would also reduce memory usage. Have you tried it?

Colin

2 Likes
5

Yes I have tried with 2 thread count and also thread count of 4. Still got the same error.

6

That’s unexpected. 16GB is a good amount of RAM!

Is this a shared system being used by other people? Could you restart the computer and make sure you are the only one using it?

I should have asked this sooner, but are you running this on OSX/Linux or on a Windows VM? If you are using a VM, how much memory have you allocated to it?

Thanks for answering so many questions. I hope I can answer your question soon.
Colin

2 Likes
7

No This is not a shared system, its my personal system and Its running on Linux. Also I should mention that my data set has 11626811 sequences.

1 Like
8

I’m totally out of ideas. I think 16GB would be enough to process 11M reads, so I’m not sure what’s going on. Let’s see what the dada2 dev thinks: @benjjneb

While we wait to get dada2 working, you could try one of the other processing methods like deblur or vsearch.
https://docs.qiime2.org/2019.1/plugins/available/deblur/
https://docs.qiime2.org/2019.1/plugins/available/vsearch/

Good work. We’ll figure this out :+1:

Colin

1 Like
9

How many samples are in this dataset? And what is the sequencing technology?

My suspicion is that one very large or very noisy (or both) sample is exceeding the 16GB of memory even when dereplicated. This can happen when there are a very large number of unique sequences, as it is that number, not the raw number of reads, that controls the peak memory requirements.

3 Likes
10

There are 224 samples in the dataset and illumina sequencing was used. Also to perform demultiplexing of the sequence I have used cutadapt this is the command that I used:
qiime cutadapt demux-single
–i-seqs single_end_sequence/single_end_sequence.qza
–m-barcodes-file metadata.tsv
–m-barcodes-column BarcodeSequence
–p-error-rate 0
–o-per-sample-sequences demultiplexed-seqs.qza
–o-untrimmed-sequences untrimmed.qza
–verbose

11

I'm not sure... it doesn't seem like the samples are super-deep then so they should fit in memory easily.

Can you try one experiment, can you run dada2 with a short truncation length (e.g. –p-trunc-len 80) and see if that avoids the memory error?

12

I’m sorry for the late response. But it worked for me when I used qiime1 to extract the barcode sequences and perform the steps from moving pictures tutorial. I believe it could be a result of me using cutadapt to demultiplex the sequences, because that’s the only thing I did differently.

2 Likes
13

An off-topic reply has been split into a new topic: DADA2 errors (memory?)

Please keep replies on-topic in the future.

14

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.