Open reference clustering: exited with vsearch plugin error

User Support
1

Hi All,

I got a question coming up when trying to do open reference clustering.

I am using the reference database downloaded fromqiime2 data resources , Greengenes (16S rRNA)* 13_8 (most recent), and unzipped it as the image shows:image

$qiime tools import
--input-path rep_set/97_otus.fasta
--output-path 97_otus.qza
--type FeatureData[Sequence]

$qiime vsearch cluster-features-open-reference
--i-table table.qza
--i-sequences rep-seqs.qza
--i-reference-sequences 97_otus.qza
--p-perc-identity 0.97
--o-clustered-table table-or-97.qza
--o-clustered-sequences rep-seqs-or-97.qza
--o-new-reference-sequences new-ref-seqs-or-97.qza

The error information is:
Exited with exit code 1.

Resource usage summary:

CPU time :                                   1009.14 sec.
Max Memory :                                 2892 MB
Average Memory :                             2202.44 MB
Total Requested Memory :                     20480.00 MB
Delta Memory :                               17588.00 MB
Max Swap :                                   -
Max Processes :                              3
Max Threads :                                4
Run time :                                   1014 sec.
Turnaround time :                            1014 sec.

The output (if any) follows:

QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Plugin error from vsearch:

Command '['vsearch', '--usearch_global', '/tmp/tmps9v7jqlk', '--id', '0.97', '--db', '/tmp/qiime2-archive-eqjlg40q/b7394028-f7af-4d0c-941f-992d08e85cf2/data/dna-sequences.fasta', '--uc', '/tmp/tmpehh6ylc5', '--strand', 'plus', '--qmask', 'none', '--notmatched', '/tmp/tmp60y12dwb', '--threads', '1']' returned non-zero exit status 1

Debug info has been saved to /tmp/qiime2-q2cli-err-lw0rg3af.log

I don't understand what is wrong with the command but thinking maybe I was using the wrong reference, so I tried rep_set_aligned/97_otus.fasta to run the above command lines again. It seems it returned with the same error infomation:

Exited with exit code 1.

Resource usage summary:

CPU time :                                   2155.23 sec.
Max Memory :                                 2760 MB
Average Memory :                             2217.07 MB
Total Requested Memory :                     20480.00 MB
Delta Memory :                               17720.00 MB
Max Swap :                                   -
Max Processes :                              3
Max Threads :                                4
Run time :                                   2160 sec.
Turnaround time :                            2504 sec.

The output (if any) follows:

QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
Plugin error from vsearch:

Command '['vsearch', '--usearch_global', '/tmp/tmps7q61sp2', '--id', '0.97', '--db', '/tmp/qiime2-archive-64egj9pt/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta', '--uc', '/tmp/tmph3wap4l6', '--strand', 'plus', '--qmask', 'none', '--notmatched', '/tmp/tmpzkey1h6n', '--threads', '1']' returned non-zero exit status 1

Debug info has been saved to /tmp/qiime2-q2cli-err-k8ligzzu.log

Is there a way to correct the command? or use some other way to solve this question?

Many thanks!

Shuqi

2

@Shuqi please post the full error message. You will need to re-run the command with the --verbose flag added, and post the entire output. Thanks!

3

Hi Nicholas, please see the full error message below:

Exited with exit code 1.

Resource usage summary:

CPU time :                                   1004.46 sec.
Max Memory :                                 2755 MB
Average Memory :                             2160.57 MB
Total Requested Memory :                     20480.00 MB
Delta Memory :                               17725.00 MB
Max Swap :                                   -
Max Processes :                              3
Max Threads :                                4
Run time :                                   1025 sec.
Turnaround time :                            1025 sec.

The output (if any) follows:

QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment.
vsearch v2.7.0_linux_x86_64, 189.2GB RAM, 16 cores

Reading file /tmp/qiime2-archive-6vfhdplr/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta

Fatal error: illegal character '-' on line 2 in FASTA file
Traceback (most recent call last):
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2cli/commands.py", line 274, in call
results = action(**arguments)
File "<decorator-gen-125>", line 2, in cluster_features_open_reference
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
output_types, provenance)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py", line 455, in callable_executor
outputs = self._callable(scope.ctx, **view_args)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_vsearch/_cluster_features.py", line 319, in cluster_features_open_reference
strand=strand, threads=threads)
File "<decorator-gen-446>", line 2, in cluster_features_closed_reference
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
output_types, provenance)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py", line 362, in callable_executor
output_views = self._callable(**view_args)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_vsearch/_cluster_features.py", line 256, in cluster_features_closed_reference
run_command(cmd)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_vsearch/_cluster_features.py", line 33, in run_command
subprocess.run(cmd, check=True)
File "/opt/conda/envs/qiime2-2018.8/lib/python3.5/subprocess.py", line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['vsearch', '--usearch_global', '/tmp/tmpysz1hium', '--id', '0.97', '--db', '/tmp/qiime2-archive-6vfhdplr/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta', '--uc', '/tmp/tmpdkqc85fa', '--strand', 'plus', '--qmask', 'none', '--notmatched', '/tmp/tmp15qy_ehc', '--threads', '1']' returned non-zero exit status 1

Plugin error from vsearch:

Command '['vsearch', '--usearch_global', '/tmp/tmpysz1hium', '--id', '0.97', '--db', '/tmp/qiime2-archive-6vfhdplr/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta', '--uc', '/tmp/tmpdkqc85fa', '--strand', 'plus', '--qmask', 'none', '--notmatched', '/tmp/tmp15qy_ehc', '--threads', '1']' returned non-zero exit status 1

See above for debug info.
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command: vsearch --usearch_global /tmp/tmpysz1hium --id 0.97 --db /tmp/qiime2-archive-6vfhdplr/57226a4a-a975-432a-8c19-8ef94f5e6acf/data/dna-sequences.fasta --uc /tmp/tmpdkqc85fa --strand plus --qmask none --notmatched /tmp/tmp15qy_ehc --threads 1

4

Looks like you have aligned sequences --- you will need to import and work with the unaligned sequences in this plugin, instead.

5

Hi Matthew,

Could you explain more? Is the unaligned sequences you mentioned pointing to the 97_otus-rep-set.qza file? If this is the case, this artifact is imported from the folder rep_set/97_otus.fasta, which I suppose is an unaligned sequences database.

If you were pointing to my rep-seqs.qza instead, I have done nothing like alignment yet. The commands I used are listed below:

$qiime tools import
--type 'SampleData[PairedEndSequencesWithQuality]'
--input-path casava-18-paired-end-demultiplexed
--input-format CasavaOneEightSingleLanePerSampleDirFmt
--output-path demux.qza

$qiime vsearch join-pairs
--i-demultiplexed-seqs demux.qza
--o-joined-sequences demux-joined.qza

$qiime demux summarize
--i-data demux-joined.qza
--o-visualization demux-joined.qzv

$qiiime quality-filter q-score-joined
--i-demux demux-joined.qza
--o-filtered-sequences demux-joined-filtered.qza
--o-filter-stats demux-joined-filter-stats.qza
--p-min-quality 19
--p-quality-window 3
--p-min-length 0.06
--p-max-ambiguous 3

$qiime2 vsearch dereplicated-sequences
--i-sequences demux-joined-filtered.qza
--o-dereplicated-table table.qza
--o-dereplicated-sequences rep-seqs.qza

The bolded rep-seqs.qza was the file I used in the flag of "$qiime vsearch cluster-features-open-reference"

shuqi

6

And, is the format of imported references database being "DNASequencesDirectoryFormat " correct? I mean what is the specific format of 85_otus.qza used in the clustering tutorial?

7

That file (and your input sequences) should be an unaligned fasta file. If you are sure you are using unaligned files, please check that file for a - character. That is usually only in aligned sequences (as it represents a gap).

8

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.