I'm back....
I successfully imported my data into a qza with my manifest file:
AD_exp3_Arc_manifest-pe_sort.csv (10.9 KB)
(qiime2-2017.10) wsb255bioimac27:Fink_fermenter_Exp3 mel_local$ qiime tools import --type 'SampleData[PairedEndSequencesWithQuality]' --input-path AD_exp3_Arc_manifest-pe_sort.csv --output-path AD_exp3_Arc_pe-demux.qza --source-format PairedEndFastqManifestPhred33
And viewed it:
(qiime2-2017.10) wsb255bioimac27:Fink_fermenter_Exp3 mel_local$ qiime demux summarize --i-data AD_exp3_Arc_pe-demux.qza --o-visualization AD_exp3_Arc_pe-demux.qzv
Saved Visualization to: AD_exp3_Arc_pe-demux.qzv
AD_exp3_Arc_pe-demux.qzv (281.7 KB)
I decided on my dada2 cutoffs and ran dada2 with the following output:
(qiime2-2017.10) wsb255bioimac27:Fink_fermenter_Exp3 mel_local$ qiime dada2 denoise-paired --i-demultiplexed-seqs AD_exp3_Arc_pe-demux.qza --p-trim-left-f 15 --p-trunc-len-f 180 --p-trim-left-r 15 --p-trunc-len-r 180 --o-representative-sequences AD_exp3_Arc_rep-seqs.qza --o-table AD_exp3_Arc_table_dada2.qza --verbose --p-n-threads 0
Running external command line application(s). This may print messages to stdout and/or stderr.
The command(s) being run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.
Command: run_dada_paired.R /var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/forward /var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/reverse /var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/output.tsv.biom /var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/filt_f /var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/filt_r 180 180 15 15 2.0 2 consensus 1.0 0 1000000
R version 3.3.2 (2016-10-31)
Loading required package: Rcpp
There were 50 or more warnings (use warnings() to see the first 50)
DADA2 R package version: 1.4.0
1) Filtering ..Error in fastqPairedFilter(c(unfiltsF[[i]], unfiltsR[[i]]), c(filteredFastqF, :
Mismatched forward and reverse sequence files: 87112, 82484.
Execution halted
Traceback (most recent call last):
File "/Users/mel_local/miniconda2/envs/qiime2-2017.10/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 179, in denoise_paired
run_commands([cmd])
File "/Users/mel_local/miniconda2/envs/qiime2-2017.10/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 35, in run_commands
subprocess.run(cmd, check=True)
File "/Users/mel_local/miniconda2/envs/qiime2-2017.10/lib/python3.5/subprocess.py", line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['run_dada_paired.R', '/var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/forward', '/var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/reverse', '/var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/output.tsv.biom', '/var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/filt_f', '/var/folders/12/2j8hq03s52lbnstw5wh008k80000gq/T/tmp79pbfgms/filt_r', '180', '180', '15', '15', '2.0', '2', 'consensus', '1.0', '0', '1000000']' returned non-zero exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/mel_local/miniconda2/envs/qiime2-2017.10/lib/python3.5/site-packages/q2cli/commands.py", line 218, in __call__
results = action(**arguments)
File "<decorator-gen-338>", line 2, in denoise_paired
File "/Users/mel_local/miniconda2/envs/qiime2-2017.10/lib/python3.5/site-packages/qiime2/sdk/action.py", line 220, in bound_callable
output_types, provenance)
File "/Users/mel_local/miniconda2/envs/qiime2-2017.10/lib/python3.5/site-packages/qiime2/sdk/action.py", line 355, in _callable_executor_
output_views = self._callable(**view_args)
File "/Users/mel_local/miniconda2/envs/qiime2-2017.10/lib/python3.5/site-packages/q2_dada2/_denoise.py", line 194, in denoise_paired
" and stderr to learn more." % e.returncode)
Exception: An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.
Plugin error from dada2:
An error was encountered while running DADA2 in R (return code 1), please inspect stdout and stderr to learn more.
See above for debug info
Seeing the mismatch error I checked the forum and took the following actions:
-
Validated my .qza
(qiime2-2017.10) wsb255bioimac27:Fink_fermenter_Exp3 mel_local$ qiime tools validate AD_exp3_Arc_pe-demux.qza
Artifact AD_exp3_Arc_pe-demux.qza appears to be valid at level=max. -
Checked all of my paths - they were fine.
-
Checked all my sequence files for mismatched entries per fastq:
(qiime2-2017.10) wsb255bioimac27:Fink_Project_008 mel_local$ for f in *.fastq; do r=$(wc -l < $f | tr -d '[:space:]'); echo $r $f; done
504 BLANK_Arc_G06_S90_R1_001.fastq
504 BLANK_Arc_G06_S90_R2_001.fastq
504 BLANK_Arc_H06_S96_R1_001.fastq
504 BLANK_Arc_H06_S96_R2_001.fastq
329936 F1_0_5_Arc_S55_R1_001.fastq
329936 F1_0_5_Arc_S55_R2_001.fastq
348448 F1_0_Arc_S49_R1_001.fastq
348448 F1_0_Arc_S49_R2_001.fastq
402708 F1_101_5_Arc_S57_R1_001.fastq
402708 F1_101_5_Arc_S57_R2_001.fastq
331688 F1_128_5_Arc_S63_R1_001.fastq
331688 F1_128_5_Arc_S63_R2_001.fastq
33752 F1_155_5_Arc_S69_R1_001.fastq
33752 F1_155_5_Arc_S69_R2_001.fastq
337028 F1_16_Arc_S85_R1_001.fastq
337028 F1_16_Arc_S85_R2_001.fastq
128560 F1_182_5_Arc_S75_R1_001.fastq
128560 F1_182_5_Arc_S75_R2_001.fastq
363012 F1_1_Arc_S61_R1_001.fastq
363012 F1_1_Arc_S61_R2_001.fastq
363612 F1_24_Arc_S91_R1_001.fastq
363612 F1_24_Arc_S91_R2_001.fastq
87648 F1_2_Arc_S67_R1_001.fastq
87648 F1_2_Arc_S67_R2_001.fastq
276640 F1_41_5_Arc_S50_R1_001.fastq
276640 F1_41_5_Arc_S50_R2_001.fastq
399764 F1_42_5_Arc_S62_R1_001.fastq
399764 F1_42_5_Arc_S62_R2_001.fastq
401364 F1_42_Arc_S56_R1_001.fastq
401364 F1_42_Arc_S56_R2_001.fastq
237172 F1_43_5_Arc_S68_R1_001.fastq
237172 F1_43_5_Arc_S68_R2_001.fastq
290944 F1_45_5_Arc_S74_R1_001.fastq
290944 F1_45_5_Arc_S74_R2_001.fastq
379572 F1_49_5_Arc_S80_R1_001.fastq
379572 F1_49_5_Arc_S80_R2_001.fastq
231744 F1_4_Arc_S73_R1_001.fastq
231744 F1_4_Arc_S73_R2_001.fastq
433060 F1_57_5_Arc_S86_R1_001.fastq
433060 F1_57_5_Arc_S86_R2_001.fastq
416836 F1_65_5_Arc_S92_R1_001.fastq
416836 F1_65_5_Arc_S92_R2_001.fastq
463752 F1_89_5_Arc_S51_R1_001.fastq
463752 F1_89_5_Arc_S51_R2_001.fastq
308324 F1_8_Arc_S79_R1_001.fastq
308324 F1_8_Arc_S79_R2_001.fastq
540 F1_W_Arc_S78_R1_001.fastq
540 F1_W_Arc_S78_R2_001.fastq
315904 F2_0_5_Arc_S87_R1_001.fastq
315904 F2_0_5_Arc_S87_R2_001.fastq
291920 F2_0_Arc_S81_R1_001.fastq
291920 F2_0_Arc_S81_R2_001.fastq
299988 F2_101_5_Arc_S89_R1_001.fastq
299988 F2_101_5_Arc_S89_R2_001.fastq
370840 F2_128_5_Arc_S95_R1_001.fastq
370840 F2_128_5_Arc_S95_R2_001.fastq
337744 F2_155_5_Arc_S54_R1_001.fastq
337744 F2_155_5_Arc_S54_R2_001.fastq
79316 F2_16_Arc_S70_R1_001.fastq
79316 F2_16_Arc_S70_R2_001.fastq
326336 F2_182_5_Arc_S60_R1_001.fastq
326336 F2_182_5_Arc_S60_R2_001.fastq
465116 F2_1_Arc_S93_R1_001.fastq
465116 F2_1_Arc_S93_R2_001.fastq
288760 F2_24_Arc_S76_R1_001.fastq
288760 F2_24_Arc_S76_R2_001.fastq
377852 F2_2_Arc_S52_R1_001.fastq
377852 F2_2_Arc_S52_R2_001.fastq
362496 F2_41_5_Arc_S82_R1_001.fastq
362496 F2_41_5_Arc_S82_R2_001.fastq
410396 F2_42_5_Arc_S94_R1_001.fastq
410396 F2_42_5_Arc_S94_R2_001.fastq
280352 F2_42_Arc_S88_R1_001.fastq
280352 F2_42_Arc_S88_R2_001.fastq
396008 F2_43_5_Arc_S53_R1_001.fastq
396008 F2_43_5_Arc_S53_R2_001.fastq
407460 F2_45_5_Arc_S59_R1_001.fastq
407460 F2_45_5_Arc_S59_R2_001.fastq
276484 F2_49_5_Arc_S65_R1_001.fastq
276484 F2_49_5_Arc_S65_R2_001.fastq
233768 F2_4_Arc_S58_R1_001.fastq
233768 F2_4_Arc_S58_R2_001.fastq
116736 F2_57_5_Arc_S71_R1_001.fastq
116736 F2_57_5_Arc_S71_R2_001.fastq
99300 F2_65_5_Arc_S77_R1_001.fastq
99300 F2_65_5_Arc_S77_R2_001.fastq
293628 F2_89_5_Arc_S83_R1_001.fastq
293628 F2_89_5_Arc_S83_R2_001.fastq
416992 F2_8_Arc_S64_R1_001.fastq
416992 F2_8_Arc_S64_R2_001.fastq
484 F2_W_Arc_S84_R1_001.fastq
484 F2_W_Arc_S84_R2_001.fastq
166848 FD1_Arc_S66_R1_001.fastq
166848 FD1_Arc_S66_R2_001.fastq
254520 M1_Arc_S72_R1_001.fastq
254520 M1_Arc_S72_R2_001.fastq
The closest file with the mismatched sequence numbers per the error: Mismatched forward and reverse sequence files: 87112, 82484.
87648 F1_2_Arc_S67_R1_001.fastq
87648 F1_2_Arc_S67_R2_001.fastq
But they match?
Any other suggestions for getting dada2 to work?
Thanks!