I was able to reproduce the error you're seeing. It doesn't appear to be a memory issue -- the process used ~1.5GB memory on my system pretty consistently with 4 threads. Here's the full debug output I received:
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.
Command: run_dada_paired.R /tmp/tmpcldhjrvi/forward /tmp/tmpcldhjrvi/reverse /tmp/tmpcldhjrvi/output.tsv.biom /tmp/tmpcldhjrvi/filt_f /tmp/tmpcldhjrvi/filt_r 300 300 0 0 2.0 2 consensus 1.0 0 1000000
R version 3.3.1 (2016-06-21)
Loading required package: Rcpp
Warning messages:
1: multiple methods tables found for ‘arbind’
2: multiple methods tables found for ‘acbind’
3: replacing previous import ‘IRanges::arbind’ by ‘SummarizedExperiment::arbind’ when loading ‘GenomicAlignments’
4: replacing previous import ‘IRanges::acbind’ by ‘SummarizedExperiment::acbind’ when loading ‘GenomicAlignments’
5: multiple methods tables found for ‘left’
6: multiple methods tables found for ‘right’
DADA2 R package version: 1.4.0
1) Filtering .
2) Learning Error Rates
2a) Forward Reads
Initializing error rates to maximum possible estimate.
Sample 1 - 72607 reads in 40326 unique sequences.
selfConsist step 2
selfConsist step 3
selfConsist step 4
selfConsist step 5
selfConsist step 6
selfConsist step 7
selfConsist step 8
selfConsist step 9
selfConsist step 10
Warning message:
In dada(drpsF, err = NULL, selfConsist = TRUE, multithread = multithread) :
Self-consistency loop terminated before convergence.
2b) Reverse Reads
Initializing error rates to maximum possible estimate.
Sample 1 - 72607 reads in 40326 unique sequences.
selfConsist step 2
selfConsist step 3
selfConsist step 4
selfConsist step 5
selfConsist step 6
selfConsist step 7
selfConsist step 8
selfConsist step 9
selfConsist step 10
Warning message:
In dada(drpsR, err = NULL, selfConsist = TRUE, multithread = multithread) :
Self-consistency loop terminated before convergence.
3) Denoise remaining samples
4) Remove chimeras (method = consensus)
Error in isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) :
Input must be a valid sequence table.
Calls: removeBimeraDenovo -> isBimeraDenovoTable
In addition: Warning message:
In is.na(colnames(unqs[[i]])) :
is.na() applied to non-(list or vector) of type 'NULL'
Execution halted
Traceback (most recent call last):
File "/home/jairideout/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 179, in denoise_paired
run_commands([cmd])
File "/home/jairideout/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 35, in run_commands
subprocess.run(cmd, check=True)
File "/home/jairideout/miniconda3/envs/qiime2-2017.7/lib/python3.5/subprocess.py", line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_paired.R', '/tmp/tmpcldhjrvi/forward', '/tmp/tmpcldhjrvi/reverse', '/tmp/tmpcldhjrvi/output.tsv.biom', '/tmp/tmpcldhjrvi/filt_f', '/tmp/tmpcldhjrvi/filt_r', '300', '300', '0', '0', '2.0', '2', 'consensus', '1.0', '0', '1000000']' returned non-zero exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/jairideout/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2cli/commands.py", line 222, in __call__
results = action(**arguments)
File "<decorator-gen-138>", line 2, in denoise_paired
File "/home/jairideout/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py", line 201, in callable_wrapper
output_types, provenance)
File "/home/jairideout/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py", line 334, in _callable_executor_
output_views = callable(**view_args)
File "/home/jairideout/miniconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 194, in denoise_paired
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.
The key part of the debug log is:
4) Remove chimeras (method = consensus)
Error in isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) :
Input must be a valid sequence table.
Calls: removeBimeraDenovo -> isBimeraDenovoTable
This error message is being raised within the DADA2 R package. My guess is that all sequences are being filtered out at some previous step in the DADA2 processing pipeline, but after Googling around I haven't found any info/questions regarding the error message.
@benjjneb do you have any ideas? The .qza and q2-dada2 command are posted above -- it took several hours for the error to be raised when running locally with 4 threads, just a heads up.