Need help: plugin error from ITSxpress

Library Plugin Support ITSxpress
1

Dear All,

I am trying to use ITSxpress to extract ITS1 regions but keep getting a plugin error. I have soil samples sequenced with Illumina NovaSeq 6000 from Novogene amplified for ITS1 (Primers : ITS5-1737F and ITS2-2043R). I have the demultiplexed raw reads with primers and barcodes removed from Novogene. I followed the below steps so far:

  1. Turn paired end sequences to artifact:

    qiime tools import
    --type 'SampleData[PairedEndSequencesWithQuality]'
    --input-path Fastq_manifest.tsv
    --output-path paired-end-seq.qza
    --input-format PairedEndFastqManifestPhred33V2

  2. Use itsxpress to extract the ITS region from the reads

qiime itsxpress trim-pair-output-unmerged
--i-per-sample-sequences paired-end-seq.qza
--p-region ITS1
--p-taxa F
--p-cluster-id 1.0
--o-trimmed trimmed_seq.qza
--verbose

which gives the error:
ERROR:root:Could not perform read merging with vsearch. Error from vsearch was:
vsearch v2.22.1_macos_x86_64, 18.0GB RAM, 12 cores

Fatal error: Unable to open file for writing (/Users/KG1641/plant_soil_fungi/custom_tmp/itsxpress_j722j91c/seq.fq)
Traceback (most recent call last):
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/itsxpress/SeqSamplePaired.py", line 61, in _merge_reads
p1.check_returncode()
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/subprocess.py", line 460, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['vsearch', '--fastq_mergepairs', '/Users/KG1641/plant_soil_fungi/custom_tmp/qiime2/KG1641/data/46b111a1-560b-4bbb-a906-902ef4528473/data/FA13_10_L001_R1_001.fastq.gz', '--reverse', '/Users/KG1641/plant_soil_fungi/custom_tmp/qiime2/KG1641/data/46b111a1-560b-4bbb-a906-902ef4528473/data/FA13_60_L001_R2_001.fastq.gz', '--fastqout', '/Users/KG1641/plant_soil_fungi/custom_tmp/itsxpress_j722j91c/seq.fq', '--fastq_maxdiffs', '40', '--fastq_maxee', '2', '--threads', '1', '--fastq_allowmergestagger', '--fastq_qmax', '93']' returned non-zero exit status 1.
Traceback (most recent call last):
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
outputs = self.callable_executor(
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 576, in callable_executor
output_views = self._callable(**view_args)
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/itsxpress/q2_itsxpress.py", line 151, in trim_pair_output_unmerged
results = main(per_sample_sequences=per_sample_sequences,
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/itsxpress/q2_itsxpress.py", line 205, in main
sobj = _set_fastqs_and_check(
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/itsxpress/q2_itsxpress.py", line 79, in _set_fastqs_and_check
sobj._merge_reads(threads=threads,stagger=allow_staggered_reads)
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/itsxpress/SeqSamplePaired.py", line 65, in _merge_reads
raise e
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/itsxpress/SeqSamplePaired.py", line 61, in _merge_reads
p1.check_returncode()
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/subprocess.py", line 460, in check_returncode
raise CalledProcessError(self.returncode, self.args, self.stdout,
subprocess.CalledProcessError: Command '['vsearch', '--fastq_mergepairs', '/Users/KG1641/plant_soil_fungi/custom_tmp/qiime2/KG1641/data/46b111a1-560b-4bbb-a906-902ef4528473/data/FA13_10_L001_R1_001.fastq.gz', '--reverse', '/Users/KG1641/plant_soil_fungi/custom_tmp/qiime2/KG1641/data/46b111a1-560b-4bbb-a906-902ef4528473/data/FA13_60_L001_R2_001.fastq.gz', '--fastqout', '/Users/KG1641/plant_soil_fungi/custom_tmp/itsxpress_j722j91c/seq.fq', '--fastq_maxdiffs', '40', '--fastq_maxee', '2', '--threads', '1', '--fastq_allowmergestagger', '--fastq_qmax', '93']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2cli/commands.py", line 520, in call
results = self._execute_action(
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/q2cli/commands.py", line 581, in _execute_action
results = action(**arguments)
File "", line 2, in trim_pair_output_unmerged
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/action.py", line 356, in bound_callable
return results
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/context.py", line 195, in exit
self._scope.destroy()
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/sdk/context.py", line 269, in destroy
ctx.cache.garbage_collection()
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/core/cache.py", line 786, in garbage_collection
assert is_uuid4(data)
AssertionError

Plugin error from itsxpress:

See above for debug info.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/core/cache.py", line 240, in _exit_cleanup
cache.garbage_collection()
File "/opt/anaconda3/envs/qiime2-amplicon-2024.5/lib/python3.9/site-packages/qiime2/core/cache.py", line 786, in garbage_collection
assert is_uuid4(data)
AssertionError

I do not understand what is giving me the error. Any help and guidance from the community members to resolve this will be appreciated.

Version of ITSxpress : QIIME 2 Plugin 'itsxpress' version 2.1.1.dev3+g0ddb522.d20240917 (from package 'itsxpress' version 2.1.1.dev3+g0ddb522.d20240917)
Qiime 2 version : 2024.5.1

Thanks!

2

Hi @Kirandeep_Gill, I'm not sure what is going wrong here. From the error message, it looks like it might not be possible to write to disk, and that's causing the failure. That could be a result of you not having permissions to write to that location, or that the disk is full.

This appears to be the problematic directory:

Can you check whether you're able to write to this location, and that there is sufficient disk space available (I think a few GB would be safe).

Greg

3

Hi @gregcaporaso,
Thank you for your response. I tried a couple of things and was successful in resolving the issue. I checked for writing permissions to the custom_tmp folder and also had sufficient disk space. Finally what made it work for me was reinstalling vsearch and itsxpress again (this is what I think made it work). I had installed the newest version of itsxpress = 2.1.3. I removed that and installed version 2.1.1 and for vsearch, I installed version = 2.22.1

conda install -c bioconda vsearch=2.22.1
conda install -c bioconda -c conda-forge ITSxpress=2.1.1

Thanks!
Kirandeep

1 Like
4

Ok, glad you got it working @Kirandeep_Gill! I'll touch base with the developer and see if they're aware of this and if they would be able to update their installation instructions if necessary.

5

@Kirandeep_Gill, one question - how did you originally install q2-itsxpress? Was it using the installation instructions here?

6

Hi everyone,

I encountered an issue while working with ITSxpress 2.1.3 in QIIME 2 (2024.10). I successfully installed ITSxpress 2.1.3 after resolving version conflicts with hmmer caused by q2-fragment-insertion and sepp. I removed these plugins to allow the installation of ITSxpress 2.1.3. Additionally, since I use Apple Silicon, I configured the environment as osx-64 using:

conda config --env --set subdir osx-64

I followed the instructions from ITSxpress GitHub.

After installation, I ran the following command:

qiime itsxpress trim-pair-output-unmerged \
  --i-per-sample-sequences paired-end-demux.qza \
  --p-region ITS2 \
  --p-taxa F \
  --o-trimmed trimmed.qza \
  --p-threads 12

However, I encountered the following error:

Fatal error: Unable to open file for writing (/var/folders/fj/vnpgdy7573bg5z6c3h7wfrd00000gn/T/itsxpress_jolf67tr/seq.fq)
Traceback (most recent call last):
  File "...", line 61, in _merge_reads
    p1.check_returncode()
  ...
subprocess.CalledProcessError: Command '['vsearch', '--fastq_mergepairs', ...]' returned non-zero exit status 1.

Debugging Steps

  1. I tested the plugin with a demultiplexed paired-end artifact containing only one sample, and it worked without errors.
  2. When I used an artifact containing two samples, I encountered the same error again.

Solution

After some testing, I decided to downgrade to ITSxpress 2.1.1 without reinstalling vsearch. The same command worked successfully with two samples (--p-threads 12) and multiple samples (--p-threads 20):

qiime itsxpress trim-pair-output-unmerged \
  --i-per-sample-sequences paired-end-demux.qza \
  --p-region ITS2 \
  --p-taxa F \
  --o-trimmed trimmed.qza \
  --p-threads 20

Summary

  • ITSxpress 2.1.3 seems to have an issue when processing artifacts with multiple samples.
  • Downgrading to ITSxpress 2.1.1 resolved the issue.

While downgrading to ITSxpress 2.1.1 resolves the issue, I’d like to know if there’s a fix that allows ITSxpress 2.1.3 to be installed without needing to remove the q2-fragment-insertion and sepp plugins. Any insights or suggestions would be greatly appreciated.

Thanks!

1 Like
7

@Adam_Rivers, are you available to advise on this one?

8

Regarding the HMMER compatibility: I use the same version of HMMER and Vsearch that is specified by the current QIIME2 installation yaml file to prevent conflicts. If q2-fragment-insertion uses a later version of HMMER you may need to install it in a separate conda environment. It sounds like you solved that.

Regarding the Vsearch error: I did not see that error in our automated tests of the last two minor versions (2.1.2 and 2.1.3) but I'll see if I can replicate it on your data if you are able to share your paired-end-demux.qza file with me.

2 Likes
9

Hi,

Thank you for your response!

Vsearch Error

Thank you for offering to investigate the Vsearch error. I’ve attached the paired-end-demux.qza file for your review. The issue consistently occurs with multiple samples but works fine when I use a single sample in the artifact. Let me know if you need additional files or details about the commands I ran.

I appreciate your help in troubleshooting this :smiley:

10

I was able to reproduce the error with your data on Friday. I'll look into the casue this week.

1 Like
11

This was a bug introduced in v2.1.2. See Github Issue 60 It is now fixed in version 2.1.4.

1 Like
12

I see now that the QIIME 2 release 2024.11, increased their HMMERdependency to v3.4. I have updated my biocinda recipe to use HMMER v3.4 as well after testign it works.

3 Likes