itsxpress vsearch error

ezgi · August 6, 2021, 11:59am

Dear Qiime2 community,

I am trying to run itsxpress plugin installed in Qiime2 2021.4 to analyse my ITS data produced by MiSeq. First I import the data, then I trim the primers with cutadapt and finally I run itsxpress:

#Importing the data
qiime tools import --type 'SampleData[PairedEndSequencesWithQuality]' --input-path $INP --input-format CasavaOneEightSingleLanePerSampleDirFmt --output-path
 demux-soil_ITS.qza

#Cutting the primers
qiime cutadapt trim-paired \
--i-demultiplexed-sequences demux-soil_ITS.qza \
--p-front-f GGGTGARTCATCRARTYTTTG \
--p-front-r CCTSCSCTTANTDATATGC \
--p-cores 12 \
--o-trimmed-sequences trimmed-seqs_ITS.qza \
--verbose

#itsxpress trimming:
qiime itsxpress trim-pair-output-unmerged --i-per-sample-sequences trimmed-seqs_ITS.qza --p-region ITS2 --p-taxa F --p-threads 12 --verbose --o-trimmed trim
med_demux_ITS.qza

However, I am facing with the error pasted below:

ERROR:root:Could not perform clustering with Vsearch. Error from Vsearch was:
 vsearch v2.13.3_linux_x86_64, 251.7GB RAM, 12 cores
https://github.com/torognes/vsearch



Fatal error: Files compressed with gzip are not supported
Traceback (most recent call last):
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/itsxpress/main.py", line 540, in cluster
    p2.check_returncode()
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/subprocess.py", line 448, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['vsearch', '--cluster_size', '/tmp/itsxpress_xgc13947/seq.fq.gz', '--centroids', '/tmp/itsxpress_xgc13947/rep.fa', '
--uc', '/tmp/itsxpress_xgc13947/uc.txt', '--strand', 'both', '--id', '0.995', '--threads', '12']' returned non-zero exit status 1.
Traceback (most recent call last):
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/q2cli/commands.py", line 329, in __call__
    results = action(**arguments)
  File "<decorator-gen-285>", line 2, in trim_pair_output_unmerged
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/qiime2/sdk/action.py", line 244, in bound_callable
    outputs = self._callable_executor_(scope, callable_args,
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/q2_itsxpress/_itsxpress.py", line 151, in trim_pair_output_unmerged
    results = main(per_sample_sequences=per_sample_sequences,
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/q2_itsxpress/_itsxpress.py", line 208, in main
    sobj.cluster(threads=threads, cluster_id=cluster_id)
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/itsxpress/main.py", line 543, in cluster
    raise e
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/itsxpress/main.py", line 540, in cluster
    p2.check_returncode()
  File "/hpc-home/miniconda3/envs/qiime2-2021.4/lib/python3.8/subprocess.py", line 448, in check_returncode
    raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['vsearch', '--cluster_size', '/tmp/itsxpress_xgc13947/seq.fq.gz', '--centroids', '/tmp/itsxpress_xgc13947/rep.fa', '
--uc', '/tmp/itsxpress_xgc13947/uc.txt', '--strand', 'both', '--id', '0.995', '--threads', '12']' returned non-zero exit status 1.

Plugin error from itsxpress:

  Command '['vsearch', '--cluster_size', '/tmp/itsxpress_xgc13947/seq.fq.gz', '--centroids', '/tmp/itsxpress_xgc13947/rep.fa', '--uc', '/tmp/itsxpress_xgc139
47/uc.txt', '--strand', 'both', '--id', '0.995', '--threads', '12']' returned non-zero exit status 1.

See above for debug info.

It looks like the problem is with vsearch. How can I fix it?

Thank you very much for your help!

thermokarst · August 10, 2021, 4:20pm

Hi there @ezgi!

Can you please run the following and report back the results?

qiime tools validate trimmed-seqs_ITS.qza

This'll help us confirm the validity of the input data you're using here.

Also, please activate your QIIME 2 environment and run the following:

conda list --revisions

I'm wondering how your env has a different version of vsearch from the one we shipped with it (looks like you have 2.13.3, we shipped 2.7.0).

Thanks!

ezgi · August 12, 2021, 2:27am

Hi thermokarst,

Thank you very much for your reply!

Here's the result of qiime tools validate:
Result trimmed-seqs_ITS.qza appears to be valid at level=max.

And here the vsearch-related part from conda list --revisions:
+q2-vsearch-2021.4.0 (qiime2/label/r2021.4/linux-64)
** +vsearch-2.7.0 (bioconda/linux-64)**

Since ITSxpress produced a vsearch-related error, I tried to install another version of vsearch in the qiime2 env, but still the run failed. This is the reason my env has a different version of vsearch.

Himm, what can be the reason for the error in this case?

Many thanks!

thermokarst · August 12, 2021, 2:28am

Thanks @ezgi!

Sorry, I actually need the whole output from that command - can you run it again?

Thanks!

ezgi · August 16, 2021, 1:45pm

Ahh, sorry! I am uploading the whole the whole output:
output.txt (20.5 KB)

Thanks again!

thermokarst · August 17, 2021, 11:45pm

No worries, thanks @ezgi!

Okay, things look good in your conda environment, unfortunately though that means that this other version of vsearch is leaking in some other way. Can you please run the following commands and copy-and-paste the entire results here? Thanks!

conda deactivate
which vsearch
conda activate qiime2-2021.4
which vsearch
env

This'll help us understand how your command line environment is configured, generally. The issue appears to be that somehow rather than using the version of vsearch provided with the conda environment for QIIME 2, you're instead using a different version of vsearch, which might indicate that there is an issue with how conda is set up on this machine.

ezgi · August 20, 2021, 10:35am

Thank you very much for your reply!

conda deactivate
which vsearch
~/mothur/vsearch

conda activate qiime2-2021.4
which vsearch
~/mothur/vsearch

And I am attaching the output of the "env" command:
output_env.txt (12.4 KB)

Many thanks!

thermokarst · August 20, 2021, 3:30pm

Ah ha! That's the problem right there - your QIIME 2 environment isn't using the copy of vsearch we have provided, instead it looks like you're using something from an installation of mothur!

The reason for this is your PATH environment variable - this is a list that tells your shell where to search for programs, here is yours (from the env command you ran above):

PATH=/hpc-home/USER/mothur:/hpc-home/USER/_park_otus/gg_13_8_otus/rdp_classifier_2.2/rdp_classifier-2.2.jar:/hpc-home/USER/Git-2.25.0/bin:/hpc-home/USER/mothur:/hpc-home/USER/miniconda3/envs/qiime2-2021.4/bin:/hpc-home/USER/miniconda3/condabin:/hpc-home/USER/_park_otus/gg_13_8_otus/rdp_classifier_2.2/rdp_classifier-2.2.jar:/hpc-home/USER/Git-2.25.0/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/nbi/software/production/bin:/nbi/software/production/libraries/bin:/nbi/software/testing/bin:/nbi/software/testing/libraries/bin:/ei/software/cb/bin:/opt/cis/bin:/opt/cis/help:/hpc-home/USER/.local/bin:/hpc-home/USER/bin:/nbi/software/production/bin:/nbi/software/production/libraries/bin:/nbi/software/testing/bin:/nbi/software/testing/libraries/bin:/ei/software/cb/bin:/opt/cis/bin:/opt/cis/help:/hpc-home/USER/.local/bin:/hpc-home/USER/bin

Your mothur directory is first, while your QIIME 2 directory is ~fifth! Usually when using conda, you set it up so that when a conda env is activated it is the first item in the list. You should talk to your sysadmin to see if they can help out with pointing you in the right direction, but right now you basically have different software tools mixing with each other, when they should be isolated.

Keep us posted!

ezgi · August 27, 2021, 1:43pm

Hello Matthew,

When I exported the PATH /hpc-home/USER/miniconda3/envs/qiime2-2021.4/bin before running the q2-itsxpress command, the problem is solved. Indeed it was because another installation of vsearch was interfering.. Thank you so much for your help!

Ezgi

system · September 27, 2021, 7:44pm

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.