Limited memory for QIIME2 run on VirtualBox

Technical Support
1

Hi,

I would like to ask for advise for my below problem:

(qiime2-2021.8) qiime2@qiime2core2021-8:~/MicrobiomeData$ qiime dada2 denoise-paired --i-demultiplexed-seqs raw_reads.qza --p-trim-left-f 0 --p-trim-left-r 0 --p-trunc-len-f 255 --p-trunc-len-r 200 --o-table table.qza --o-representative-sequences rep-seqs.qza --o-denoising-stats denoising-stats.qza --verbose
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /tmp/tmp6cexws17/forward /tmp/tmp6cexws17/reverse /tmp/tmp6cexws17/output.tsv.biom /tmp/tmp6cexws17/track.tsv /tmp/tmp6cexws17/filt_f /tmp/tmp6cexws17/filt_r 255 200 0 0 2.0 2.0 2 12 independent consensus 1.0 1 1000000

R version 4.0.5 (2021-03-31)
Loading required package: Rcpp
DADA2: 1.18.0 / Rcpp: 1.0.7 / RcppParallel: 5.1.4

  1. Filtering ......................................................................................................
  2. Learning Error Rates
    Error: cannot allocate vector of size 202.2 Mb
    Execution halted
    Warning message:
    system call failed: Cannot allocate memory
    Traceback (most recent call last):
    File "/home/qiime2/miniconda/envs/qiime2-2021.8/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 266, in denoise_paired
    run_commands([cmd])
    File "/home/qiime2/miniconda/envs/qiime2-2021.8/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 36, in run_commands
    subprocess.run(cmd, check=True)
    File "/home/qiime2/miniconda/envs/qiime2-2021.8/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
    subprocess.CalledProcessError: Command '['run_dada_paired.R', '/tmp/tmp6cexws17/forward', '/tmp/tmp6cexws17/reverse', '/tmp/tmp6cexws17/output.tsv.biom', '/tmp/tmp6cexws17/track.tsv', '/tmp/tmp6cexws17/filt_f', '/tmp/tmp6cexws17/filt_r', '255', '200', '0', '0', '2.0', '2.0', '2', '12', 'independent', 'consensus', '1.0', '1', '1000000']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/qiime2/miniconda/envs/qiime2-2021.8/lib/python3.8/site-packages/q2cli/commands.py", line 329, in call
results = action(**arguments)
File "", line 2, in denoise_paired
File "/home/qiime2/miniconda/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
outputs = self.callable_executor(scope, callable_args,
File "/home/qiime2/miniconda/envs/qiime2-2021.8/lib/python3.8/site-packages/qiime2/sdk/action.py", line 391, in callable_executor
output_views = self._callable(**view_args)
File "/home/qiime2/miniconda/envs/qiime2-2021.8/lib/python3.8/site-packages/q2_dada2/_denoise.py", line 279, in denoise_paired
raise Exception("An error was encountered while running DADA2"
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

See above for debug info.

It seems to me that the VirtualBox has limited memory. I need to run large data file (4-6 Gb per sequencing run), so I am not sure if I can proceed to use the QIIME2 on VirtualBox. I am very sorry, but I am new to this. Thank you very much for any advise!

1 Like
2

Welcome to the forum!

To increase amount of RAM available for a virtual machine you need to go to the settings in your Virtual box and increase amount of shared memory there, as well as number of CPUs.
On the machine with 32 Gb Virtual box allowed me to allocate only 16 gb and 6 CPUs from 12. So another solution will be to install Ubuntu as a second system (better on the separate drive) to get maximum performance. As an alternative, you can find a stronger machine or use remote clusters for the analysis.

3

Hi, @timanix . Thank you very much for your quick reply!

Will take a look on that one.

On a side note, is there any possibility there is something that is not right in my command? Or is it only because of the setup of the VirtualBox?

Thanks again!

4

The command should run on a stronger machine as it is.
I can not say if there is something wrong without additional information.
But you can go through this check-list:

  1. Are primers already removed?
  2. Do I keep overlapping region by chosen truncating parameters?
  3. Are my forward and reverse reads long enough to be truncated by chosen corresponding values?

Second and third questions may help you to troubleshoot if you lose a lot of reads after dada2 (if you are happy with the % of merged reads after dada2 - everything is fine as it is).

1 Like
5

Okay, understood. Thank you for the suggestion.

I will give it a try, thanks!

6

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.