Error encountered running DADA2 on AWS

[tvanlaar (Tricia Van Laar)]

Hello,

I am trying to analyze my data using AWS. I ran this command:

qiime dada2 denoise-single \
    --i-demultiplexed-seqs demux.qza \
    --p-trim-left 0 \
    --p-trunc-len 240 \
    --o-representative-sequences rep-seqs-dada2.qza \
    --o-table table-dada2.qza

and I ran into the following error:

Plugin error from dada2:

  An error was encountered while running DADA2 in R (return code -9), please inspect stdout and stderr to learn more.

Debug info has been saved to /tmp/qiime2-q2cli-err-cen8q4qj.log

The information from the log is below:
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_single.R /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-_dgmu_jm /tmp/tmp_bibtuxx/output.tsv.biom /tmp/tmp_bibtuxx 240 0 2.0 2 consensus 1.0 1 1000000

R version 3.3.2 (2016-10-31) 
Loading required package: Rcpp
'BiocParallel' did not register default BiocParallelParams:
  missing value where TRUE/FALSE needed
There were 50 or more warnings (use warnings() to see the first 50)
DADA2 R package version: 1.4.0 
1) Filtering Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2017.10/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 126, in denoise_single
    run_commands([cmd])
  File "/home/qiime2/miniconda/envs/qiime2-2017.10/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 35, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/qiime2/miniconda/envs/qiime2-2017.10/lib/python3.5/subprocess.py", line 398, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_single.R', '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-_dgmu_jm', '/tmp/tmp_bibtuxx/output.tsv.biom', '/tmp/tmp_bibtuxx', '240', '0', '2.0', '2', 'consensus', '1.0', '1', '1000000']' returned non-zero exit status -9

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2017.10/lib/python3.5/site-packages/q2cli/commands.py", line 218, in __call__
    results = action(**arguments)
  File "<decorator-gen-336>", line 2, in denoise_single
  File "/home/qiime2/miniconda/envs/qiime2-2017.10/lib/python3.5/site-packages/qiime2/sdk/action.py", line 220, in bound_callable
    output_types, provenance)
  File "/home/qiime2/miniconda/envs/qiime2-2017.10/lib/python3.5/site-packages/qiime2/sdk/action.py", line 355, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/qiime2/miniconda/envs/qiime2-2017.10/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 137, in denoise_single
    " and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code -9), please inspect stdout and stderr to learn more.

I am very new to using AWS and I would appreciate any help.

Thank you!

[ebolyen (Evan Bolyen)]

Thanks for the detailed output @tvanlaar!

Usually when DADA2 returns -9 it is because it ran out of memory. How large is demux.qza and what is your EC2 instance size (e.g t2.nano, m1.small, m1.medium, etc)?

Also are you using the official QIIME 2 AWS image, or one you made? I just double checked our official image for 2017.10 and it seems to work fine.

Thanks!

[tvanlaar (Tricia Van Laar)]

Thanks so much for the response. I'm running t1.micro (I'm on the free tier, so this may be the problem!) and my demux.qza file is 231.2 MB.

I am using the official QIIME2 AWS image for the newest update (2017.10).

Thanks again!

[thermokarst]

Hi @tvanlaar! Thanks for the info --- the micro instance is almost certainly the culprit here, that is a pretty puny resource (for context, this instance has the computing power of a high-end smartphone circa 2008...). I would look at either upgrading your instance to something a bit more substantial (maybe a t2.large?), or, you could think about installing QIIME 2 natively on your mac or linux laptop/desktop, or via docker or Virtualbox on your Windows laptop or desktop. Good luck!