Qiime dada2 denoise-paired end sequences

drmusk · August 21, 2017, 4:47pm

Hello colleagues

I am new to QIIME2.
I had illumina R1 and R2 files
I applied extract_barcodes.py in Qiime1 to get Reads1.fastq , Reads2.fastq and Barcodes.fastq
I converted them all to individual .gz files
and then imported them using
qiime tools import --type EMPPairedEndSequences --input-path emp-paired-end-sequences --output-path emp-paired-end-sequences.qza
After this I did demultiplexing using
qiime demux emp_paired and got demux.qza

Then I ran several option but most of them didn't work. The latest command that I ran is below which says

qiime dada2 denoise-paired --i-demultiplexed-seqs demux.qza --p-trunc-len-f 120 --p-trunc-len-r 120 --p-trim-left-f 0 --p-trim-left-r 0 --o-representative-sequences rep-seqs-dada2.qza --o-table table-dada2.qza

Plugin error from dada2:

An error was encountered while running DADA2 in R (return code 1),
please inspect stdout and stderr to learn more.

demux.qzv (281.0 KB)

I am also not able to open demux.qzv though I have tried several time on chrome, firefox

I have attach demux.qzv here.demux.qzv (281.0 KB)

drmusk · August 18, 2017, 2:38pm

Hello Dear Colleague

I am still confused regarding importing data.
Can you help me to construct the command
In the shared folder
I have three files as an output of extract_barcodes.py in the "processed_seqs" folder
Barcodes.FASTQ
Reads1.FASTQ
Reads2.FASTQ
First I define the path of Qiime2 to by command "cd" to go into abc folder
Shared_Folder/abc$

Shared_Folder/abc $ qiime tools import --type EMPPairedEndSequences --input-path processed_seqs --output-path emp-paired-end-sequences.qza

thermokarst · August 22, 2017, 2:12am

Hi @drmusk, thanks for writing!

It looks like your import went well, and the results from running demux summarize appear reasonable at first glance:

Have you made sure your Chrome or Firefox installations are up to date? You could share a screenshot of the error page you see when loading your files.

You can also view QIIME 2 Visualizations by running the following on the command line:

$ qiime tools view MY_VIZ.qzv

where MY_VIZ.qzv is the file you want to view.

For your issues with q2-dada2, please provide the detailed error message (by running with --verbose) or by attaching the detailed error log referenced at the end of the terse error output.

Let us know if you get stuck --- thanks!

PS - we collapsed your two posts into one topic because the questions were overlapping. In the future, please avoid cross-posting like that, it takes extra administrative time for us to resolve, which reduces our available time for developing QIIME 2. Thanks!

drmusk · August 22, 2017, 4:56pm

Dear Dr Dillon

Sorry, I will be cautious next time while posting.

Thanks for suggesting $ qiime tools view MY_VIZ.qzv
It helped.
I ran this command again with --verbose and got a big message which is attached herewith dada2 error message.txt (5.9 KB)

qiime dada2 denoise-paired --i-demultiplexed-seqs demux.qza --p-trunc-len-f 120 --p-trunc-len-r 120 --p-trim-left-f 0 --p-trim-left-r 0 --o-representative-sequences rep-seqs-dada2.qza --o-table table-dada2.qza --verbose


$ qiime dada2 denoise-paired --i-demultiplexed-seqs demux.qza --p-trunc-len-f 120 --p-trunc-len-r 120 --p-trim-left-f 0 --p-trim-left-r 0 --o-representative-sequences rep-seqs-dada2.qza --o-table table-dada2.qza --verbose
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.

Command: run_dada_paired.R /tmp/tmpyjf1k_v3/forward /tmp/tmpyjf1k_v3/reverse /tmp/tmpyjf1k_v3/output.tsv.biom /tmp/tmpyjf1k_v3/filt_f /tmp/tmpyjf1k_v3/filt_r 120 120 0 0 2.0 2 consensus 1.0 1 1000000

R version 3.3.1 (2016-06-21) 
Loading required package: Rcpp
Warning messages:
1: multiple methods tables found for ‘arbind’ 
2: multiple methods tables found for ‘acbind’ 
3: replacing previous import ‘IRanges::arbind’ by ‘SummarizedExperiment::arbind’ when loading ‘GenomicAlignments’ 
4: replacing previous import ‘IRanges::acbind’ by ‘SummarizedExperiment::acbind’ when loading ‘GenomicAlignments’ 
5: multiple methods tables found for ‘left’ 
6: multiple methods tables found for ‘right’ 
DADA2 R package version: 1.4.0 
1) Filtering ....................
2) Learning Error Rates
2a) Forward Reads
Initializing error rates to maximum possible estimate.
Sample 1 - 2787 reads in 1022 unique sequences.
Sample 2 - 1019 reads in 540 unique sequences.
Sample 3 - 778 reads in 434 unique sequences.
Sample 4 - 85 reads in 69 unique sequences.
Sample 5 - 3324 reads in 1296 unique sequences.
Sample 6 - 61 reads in 54 unique sequences.
Sample 7 - 1945 reads in 914 unique sequences.
Sample 8 - 1657 reads in 722 unique sequences.
Sample 9 - 1194 reads in 605 unique sequences.
Sample 10 - 1520 reads in 720 unique sequences.
Sample 11 - 592 reads in 343 unique sequences.
Sample 12 - 172 reads in 159 unique sequences.
Sample 13 - 70 reads in 53 unique sequences.
Sample 14 - 408 reads in 261 unique sequences.
Sample 15 - 1031 reads in 554 unique sequences.
Sample 16 - 349 reads in 217 unique sequences.
Sample 17 - 101 reads in 85 unique sequences.
Sample 18 - 194 reads in 117 unique sequences.
Sample 19 - 491 reads in 330 unique sequences.
Sample 20 - 54 reads in 52 unique sequences.
   selfConsist step 2 
   selfConsist step 3 
   selfConsist step 4 
   selfConsist step 5 
   selfConsist step 6 
   selfConsist step 7 


Convergence after  7  rounds.
2b) Reverse Reads
Initializing error rates to maximum possible estimate.
Sample 1 - 2787 reads in 1075 unique sequences.
Sample 2 - 1019 reads in 616 unique sequences.
Sample 3 - 778 reads in 641 unique sequences.
Sample 4 - 85 reads in 78 unique sequences.
Sample 5 - 3324 reads in 1240 unique sequences.
Sample 6 - 61 reads in 60 unique sequences.
Sample 7 - 1945 reads in 850 unique sequences.
Sample 8 - 1657 reads in 796 unique sequences.
Sample 9 - 1194 reads in 904 unique sequences.
Sample 10 - 1520 reads in 799 unique sequences.
Sample 11 - 592 reads in 392 unique sequences.
Sample 12 - 172 reads in 170 unique sequences.
Sample 13 - 70 reads in 68 unique sequences.
Sample 14 - 408 reads in 263 unique sequences.
Sample 15 - 1031 reads in 610 unique sequences.
Sample 16 - 349 reads in 249 unique sequences.
Sample 17 - 101 reads in 101 unique sequences.
Sample 18 - 194 reads in 128 unique sequences.
Sample 19 - 491 reads in 345 unique sequences.
Sample 20 - 54 reads in 54 unique sequences.
   selfConsist step 2 
   selfConsist step 3 
   selfConsist step 4 
   selfConsist step 5 


Convergence after  5  rounds.

3) Denoise remaining samples 
4) Remove chimeras (method = consensus)
Error in isBimeraDenovoTable(unqs[[i]], ..., verbose = verbose) : 
  Input must be a valid sequence table.
Calls: removeBimeraDenovo -> isBimeraDenovoTable
In addition: Warning message:
In is.na(colnames(unqs[[i]])) :
  is.na() applied to non-(list or vector) of type 'NULL'
Execution halted
Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 179, in denoise_paired
    run_commands([cmd])
  File "/home/qiime2/miniconda/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 35, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/qiime2/miniconda/envs/qiime2-2017.7/lib/python3.5/subprocess.py", line 398, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_paired.R', '/tmp/tmpyjf1k_v3/forward', '/tmp/tmpyjf1k_v3/reverse', '/tmp/tmpyjf1k_v3/output.tsv.biom', '/tmp/tmpyjf1k_v3/filt_f', '/tmp/tmpyjf1k_v3/filt_r', '120', '120', '0', '0', '2.0', '2', 'consensus', '1.0', '1', '1000000']' returned non-zero exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2017.7/lib/python3.5/site-packages/q2cli/commands.py", line 222, in __call__
    results = action(**arguments)
  File "<decorator-gen-270>", line 2, in denoise_paired
  File "/home/qiime2/miniconda/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py", line 201, in callable_wrapper
    output_types, provenance)
  File "/home/qiime2/miniconda/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py", line 334, in _callable_executor_
    output_views = callable(**view_args)
  File "/home/qiime2/miniconda/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 194, in denoise_paired
    " and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.

Plugin error from dada2:

  An error was encountered while running DADA2 in R (return code 1),
  please inspect stdout and stderr to learn more.

See above for debug info.

thermokarst · August 22, 2017, 5:17pm

Hi @drmusk, thanks for posting the error message. @benjjneb, I wonder if this is related to this bug on the DADA2 issue tracker?

benjjneb · August 22, 2017, 11:03pm

I suspect this one is due to none of the sequences being succesfully merged (hence the sequence table is empty and not "valid").

@drmusk can you clarify the primer set you are using? Is it the ~252 nucleotide V4 amplicon region? If it is, when the reads are truncated to 120/120, they won't overlap and none will merge.

Your sequencing data looks to be of good quality, so I would extend your truncation lengths singificantly (e.g. --p-trunc-len-f 220 and --p-trunc-len-r 200) and hopefully your problem will be solved.

drmusk · August 23, 2017, 5:00am

Dear @benjjneb Dear @thermokarst yes I was doing V4 region sequence analysis with 515F 806R primers.

--p-trunc-len-f 220 --p-trunc-len-r 220 served the job.

Feeling obliged sir

system · September 23, 2017, 11:00am

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.