Open reference clustering: exited with vsearch plugin error

Hi All,

I got a question coming up when trying to do open reference clustering.

I am using the reference database downloaded fromqiime2 data resources , Greengenes (16S rRNA)* 13_8 (most recent), and unzipped it as the image shows:image

$qiime tools import
--input-path rep_set/97_otus.fasta
--output-path 97_otus.qza
--type FeatureData[Sequence]

$qiime vsearch cluster-features-open-reference
--i-table table.qza
--i-sequences rep-seqs.qza
--i-reference-sequences 97_otus.qza
--p-perc-identity 0.97
--o-clustered-table table-or-97.qza
--o-clustered-sequences rep-seqs-or-97.qza
--o-new-reference-sequences new-ref-seqs-or-97.qza

The error information is:
Exited with exit code 1.

Resource usage summary:

CPU time :                                   1009.14 sec.
Max Memory :                                 2892 MB
Average Memory :                             2202.44 MB
Total Requested Memory :                     20480.00 MB
Delta Memory :                               17588.00 MB
Max Swap :                                   -
Max Processes :                              3
Max Threads :                                4
Run time :                                   1014 sec.
Turnaround time :                            1014 sec.

The output (if any) follows:

QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Plugin error from vsearch:

Command '['vsearch', '--usearch_global', '/tmp/tmps9v7jqlk', '--id', '0.97', '--db', '/tmp/qiime2-archive-eqjlg40q/b7394028-f7af-4d0c-941f-992d08e85cf2/data/dna-sequences.fasta', '--uc', '/tmp/tmpehh6ylc5', '--strand', 'plus', '--qmask', 'none', '--notmatched', '/tmp/tmp60y12dwb', '--threads', '1']' returned non-zero exit status 1

Debug info has been saved to /tmp/qiime2-q2cli-err-lw0rg3af.log

I don't understand what is wrong with the command but thinking maybe I was using the wrong reference, so I tried rep_set_aligned/97_otus.fasta to run the above command lines again. It seems it returned with the same error infomation:

Exited with exit code 1.

Resource usage summary:

CPU time :                                   2155.23 sec.
Max Memory :                                 2760 MB
Average Memory :                             2217.07 MB
Total Requested Memory :                     20480.00 MB
Delta Memory :                               17720.00 MB
Max Swap :                                   -
Max Processes :                              3
Max Threads :                                4
Run time :                                   2160 sec.
Turnaround time :                            2504 sec.

The output (if any) follows:

QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Plugin error from vsearch:

Command '['vsearch', '--usearch_global', '/tmp/tmps7q61sp2', '--id', '0.97', '--db', '/tmp/qiime2-archive-64egj9pt/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta', '--uc', '/tmp/tmph3wap4l6', '--strand', 'plus', '--qmask', 'none', '--notmatched', '/tmp/tmpzkey1h6n', '--threads', '1']' returned non-zero exit status 1

Debug info has been saved to /tmp/qiime2-q2cli-err-k8ligzzu.log

Is there a way to correct the command? or use some other way to solve this question?

Many thanks!

Shuqi

@Shuqi please post the full error message. You will need to re-run the command with the --verbose flag added, and post the entire output. Thanks!

Hi Nicholas, please see the full error message below:

Exited with exit code 1.

Resource usage summary:

CPU time :                                   1004.46 sec.
Max Memory :                                 2755 MB
Average Memory :                             2160.57 MB
Total Requested Memory :                     20480.00 MB
Delta Memory :                               17725.00 MB
Max Swap :                                   -
Max Processes :                              3
Max Threads :                                4
Run time :                                   1025 sec.
Turnaround time :                            1025 sec.

The output (if any) follows:

QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
vsearch v2.7.0_linux_x86_64, 189.2GB RAM, 16 cores

Reading file /tmp/qiime2-archive-6vfhdplr/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta

Fatal error: illegal character '-' on line 2 in FASTA file
Traceback (most recent call last):
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2cli/commands.py", line 274, in call
results = action(**arguments)
File "<decorator-gen-125>", line 2, in cluster_features_open_reference
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
output_types, provenance)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py", line 455, in callable_executor
outputs = self._callable(scope.ctx, **view_args)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_vsearch/_cluster_features.py", line 319, in cluster_features_open_reference
strand=strand, threads=threads)
File "<decorator-gen-446>", line 2, in cluster_features_closed_reference
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
output_types, provenance)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py", line 362, in callable_executor
output_views = self._callable(**view_args)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_vsearch/_cluster_features.py", line 256, in cluster_features_closed_reference
run_command(cmd)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_vsearch/_cluster_features.py", line 33, in run_command
subprocess.run(cmd, check=True)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/subprocess.py", line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['vsearch', '--usearch_global', '/tmp/tmpysz1hium', '--id', '0.97', '--db', '/tmp/qiime2-archive-6vfhdplr/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta', '--uc', '/tmp/tmpdkqc85fa', '--strand', 'plus', '--qmask', 'none', '--notmatched', '/tmp/tmp15qy_ehc', '--threads', '1']' returned non-zero exit status 1

Plugin error from vsearch:

Command '['vsearch', '--usearch_global', '/tmp/tmpysz1hium', '--id', '0.97', '--db', '/tmp/qiime2-archive-6vfhdplr/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta', '--uc', '/tmp/tmpdkqc85fa', '--strand', 'plus', '--qmask', 'none', '--notmatched', '/tmp/tmp15qy_ehc', '--threads', '1']' returned non-zero exit status 1

See above for debug info.
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command: vsearch --usearch_global /tmp/tmpysz1hium --id 0.97 --db /tmp/qiime2-archive-6vfhdplr/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta --uc /tmp/tmpdkqc85fa --strand plus --qmask none --notmatched /tmp/tmp15qy_ehc --threads 1

Looks like you have aligned sequences — you will need to import and work with the unaligned sequences in this plugin, instead.

Hi Matthew,

Could you explain more? Is the unaligned sequences you mentioned pointing to the 97_otus-rep-set.qza file? If this is the case, this artifact is imported from the folder rep_set/97_otus.fasta, which I suppose is an unaligned sequences database.

If you were pointing to my rep-seqs.qza instead, I have done nothing like alignment yet. The commands I used are listed below:

$qiime tools import
–type ‘SampleData[PairedEndSequencesWithQuality]’
–input-path casava-18-paired-end-demultiplexed
–input-format CasavaOneEightSingleLanePerSampleDirFmt
–output-path demux.qza

$qiime vsearch join-pairs
–i-demultiplexed-seqs demux.qza
–o-joined-sequences demux-joined.qza

$qiime demux summarize
–i-data demux-joined.qza
–o-visualization demux-joined.qzv

$qiiime quality-filter q-score-joined
–i-demux demux-joined.qza
–o-filtered-sequences demux-joined-filtered.qza
–o-filter-stats demux-joined-filter-stats.qza
–p-min-quality 19
–p-quality-window 3
–p-min-length 0.06
–p-max-ambiguous 3

$qiime2 vsearch dereplicated-sequences
–i-sequences demux-joined-filtered.qza
–o-dereplicated-table table.qza
–o-dereplicated-sequences rep-seqs.qza

The bolded rep-seqs.qza was the file I used in the flag of “$qiime vsearch cluster-features-open-reference”

shuqi

And, is the format of imported references database being "DNASequencesDirectoryFormat " correct? I mean what is the specific format of 85_otus.qza used in the clustering tutorial?

That file (and your input sequences) should be an unaligned fasta file. If you are sure you are using unaligned files, please check that file for a - character. That is usually only in aligned sequences (as it represents a gap).

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.