ITS extraction error - Missing sequence for record beginning on line 17

Library Plugin Support ITSxpress
,
1

Dear Qiime2 user community,

I am contacting you with a question about ITS sequence extraction. In a state of absolute desperation, I am turning here for help.

I am learning to analyze the output from Illumina metabarcoding (2x250bp run) of ITS regions and I have encountered the following problem. Before using the DADA2 tool I wanted to extract ITS sequences, but when extracting ITS sequences from my .qza artefact with demultiplexed sequences I repeatedly encountered the following error:

qiime q2-itsxpress trim-pair
  --i-per-sample-sequences PL-02H_YT_demux_sequences.qza \
  --p-region ITS2  \
  --p-taxa F \
  --p-threads 80  \
  --o-trimmed trimmed-seqs.qza

->

Plugin error from q2-itsxpress:

/tmp/qiime2/svecka/processes/2657616-1733314924.09@svecka/tmp/q2-OutPath-hk518_dx/X13T_AAGGTACA_L001_R1_001.fastq.gz is not a(n) FastqGzFormat file:

Missing sequence for record beginning on line 17

Debug info has been saved to /tmp/qiime2-q2cli-err-4chc5196.log

I have seen that other users on this forum have encountered the same problem, but these threads did not help me solve the problem. My mapping file and the mentioned error log are enclosed.

I use (qiime2-amplicon-2024.10) q2cli version 2024.10.0 within miniconda3 on Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-125-generic x86_64).

Thank you very much for your valuable comments.

Karel

qiime2-q2cli-err-4chc5196.txt (45.3 KB)
mapping_file.txt (12.9 KB)

1 Like
2

Hello @Pardal_Oblackovy !

This suggests that you may have some corrupted read around line 17 on sample X13T forward reads. In this post a similar issue is addressed. Could you please run:


# If your FASTQ file is compressed
zcat path/to/fastq.gz | head -n 30

# Else
head path/to/fastq -n 30

and share the output with us? Maybe we can visually spot the issue :eye:

Best,

Sergio

1 Like
3

Hey Sergio @salias , thank you for your reply :slight_smile: I was thinikng the same thing before but i think reads seems to be OK... i guess...

(qiime2-amplicon-2024.2) svecka@svecserver:~/FRESH/FRESH_ITS/PL-02H/Red_test$ ls ~/FRESH/FRESH_ITS/PL-02H/Red_test/exported_reads/X13T_AAGGTACA_L001_R1_001.fastq.gz
/home/svecka/FRESH/FRESH_ITS/PL-02H/Red_test/exported_reads/X13T_AAGGTACA_L001_R1_001.fastq.gz
(qiime2-amplicon-2024.2) svecka@svecserver:~/FRESH/FRESH_ITS/PL-02H/Red_test$ zcat ~/FRESH/FRESH_ITS/PL-02H/Red_test/exported_reads/X13T_AAGGTACA_L001_R1_001.fastq.gz | head -n 30
@VH01079:52:AAFHJ7HM5:1:1101:36864:1227 1:N:0:CCAACATT+AAGAGAGC
GATGAAGAACGTAGCAAAATGCGATACCTGGTGTGAATTGCAGAATCCCGCGAACCATCGAGTCTTTGAACGCAAGTTGCGCCCGAGGCCACTCGGCCGAGGGCACGCCTGCCTGGGCGTCACGCCAAAACACGCTCCCAACCACTCTCAACGGGAATCGGGATGCGGCATCTGGTCCCTCGTCTCTCAAGGGACGGTGGACCGAAGATTGGGCTGCCGGCGTACCGCGCCGGACACAGCGCATGGTGGGCGTCCTCGCTTTATCAATGCAGTGCATCCGGCGCGCAGCTGGCATTATGGCCT
+
CCCC5CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCC*CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCC*C*CCCCCCC5CC5CCC5CC
@VH01079:52:AAFHJ7HM5:1:1101:45176:1284 1:N:0:CCAACATT+AAGAGAGC
AGCTGTTGCCGCTTCACTCGCCGTTACTAGGGGAATCCTTGTAAGTTTCTTCTCCTCCGCTTGTTTATATGCTTAAACTCAGCGGGTAGTGCCGCCTGACCTGGGGTCGCGATCAAAGCGACGTGCACTTCGTTTGCTGGGTCGTTCTGAGGCCATAAAGCCGGCTGCGCGTCAGATGCACTGCGTTGATAAAGCAAGGACGCCCACCATGCGCTGTGTCCGGCGCGGTACACCGGCAGCTCGATCTTCGATCCACCACCCCTAGCAAGACGAGGGACCAGTTCTTGCATCCCGATTCCCGATGA
+
CCCCCCCC5CCCCCCCCCCCCCC5CCCCCCCCCCC*CCCCCCC*CCCCCCC5CCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCC5CCC5CCCCC*CCCCCCCCCCCCCCCCCCCC*CCCCCCC*CCCCCCCCCCCCCCCCCCCCCCCCCCCCC*CC*CCCCCCCCCCCCCCCC*CCCCCCCCCCCCC5**CCCC55*CCCCCC*5**55555555*CCCCCC5CCCCCCCCC5C*CCCCCC5CCCCCCC5CCC5CC555C5*CCC5CCCCCC*CCCCCC*C*5CC*CC*C555C*5*555*55
@VH01079:52:AAFHJ7HM5:1:1101:25976:1663 1:N:0:CCAACATT+AAGAGAGC
CACAGCGGCTGGCAGTATTCTTCAATTCAAGCGCTAGTGCGGATTTTTTCTCCTCATTTTGGCGCCGAGCTGCTTGTTCGGCTCTTAGATCAGCCGATGCCTTCTCGGCAGCCGCATCACTACACCGAGCCAGCTCCTTGGCCTGGGCTAGCTCCGCCCGAAGAGCTTCAACGGCGGCAGCACCATCTGCACTCATGCACATTTCATATAAAACTCTTTTCGGCGTCAGGCCTAAATTACAAGCACATTGGTGTAAGGAATGCTCGAAATATACACCTT
+
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC*CCC*CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCC*CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC*CCCCCCC55CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCC
@VH01079:52:AAFHJ7HM5:1:1101:24726:2306 1:N:0:CCAACATT+AAGAGAGC
GATGAAGAACGCAGCGAAAGCTAGATATTGCGGAAGATGACATTGCATTGATCAACAGGCGTCTTGACGACTCGCAAGGTATATATGTCGAGCATTCCTTACACCAATGTACTTAGGCCTGATGCTGAAAAGGGTTTATATGAATTGTGCATGTTTGTAGATGGTGCTGCCGCCGTTGAAGCTCTTCGGGCGAAGCTAGCCCAGGCCAAGGAGCGGGCCCGGCGTAGTGATGCGGCTGCCGAGAAGGCATCGGCTGATCTAAGAGCCGAACAAGCAGCTCGGCAACAAAGTGAGCAGAAAATA
+
CCCC5CCCCCCCCCCCCCCCCCCC*CCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5C**CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
@VH01079:52:AAFHJ7HM5:1:1101:19178:2477 1:N:0:CCAACATT+AAGAGAGC
GATGAAGAACGCAGCGAAATGCGATACCTGGTGTGAATTGCAGAATCCCGCGAACCATCGAGTCTTTGAACGCAAGTTGCGCCCGAGGCCACTCGGCCGAGGGCACGCCTGCCTGGGCGTCACGCCAAAACACGCTCCCAACCACCCTCAACGGGAATCGGGATGCGGCATCTGGTCCCTCGTCTCTCAAGGGGCGGTGGACCGAAGATTGGGCTGCCGGTGTACCGCGCCGGACACAGCGCATGGTGGGCGTCCTCGCTTTATCAATGCAGTGCATCCAGCGCGCAGCTGGCATTATGGCCT
+
CCCC*CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC*CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC*CCCCCCC5CCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC*CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCC5CCCCC
@VH01079:52:AAFHJ7HM5:1:1101:52845:2590 1:N:0:CCAACATT+AAGAGAGC
GATGAAGAACGCAGCGAAAGCTGGATATTGCGGAGGATGACATTGCTTTGATCAACAGGCGTCTTGACGACTCGCAAGGTATATATTTTAAGCATTCCTTACACCAATATGCTTGTAATGTAAGCCTGACTCTGAAGAGAGTTTGATATGAAATGTGCATCAATGCAGATGGTGCCGCCGCCATGGAAGCCCTTCGGGCGGAGCTAGCCCGGGCCAAGGAGCAGGCCCGGTTTAGTAATGCGGCTGCCGAAAAGGCATCGGCTGAATTGAAGGCCCAACAAGCCGCGCGGCGCCAGGACAAGG
+
C55CCCC5C5CCCC5CC555C5*C5CCCCC5C5CCC*CCCCCCCC5CCCCCCCCCCC5CCCCCCCCCCC5CCCCC55CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC*CC*C5CCCCCCCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC*CCCCCCCCCCCCCCCCCCCCCCCCCC5CC5CCCC5CCCCC5C5CC*5CC5
@VH01079:52:AAFHJ7HM5:1:1101:38435:2799 1:N:0:CCAACATT+AAGAGAGC
GATGAAGAACGCAGTAAAGCTGGATACTGCGGCGGATGACATTGCATTGATCAACGGGCGTCTTGACGACTCGCAAGGTATATATTTCGAGCATTCCTTACACCACTGTGCTTGTAATTGAAGCCTAACGCGGAAGAGAGTTTTATATGAAATGTGCATGAATGCAGATGGTGCTGCCGCCGTTGAAGCTCTTCGGGTGGAGCTAGCCCGGGCCAAGGAGCAGGCCCGGTATAGTAATGCGGCTGCCGAAAAGGCATCGGCTGATCATAGAGCCGAACAAGCAGCTCGGCGCCAAAATGAGGG
+
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCC*CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCCCCCCCCCC5CCCCCCCCCCC5CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC5CCCCCC
@VH01079:52:AAFHJ7HM5:1:1101:48812:3101 1:N:0:CCAACATT+AAGAGAGC
GATGAAAGAACGTAGTGAAATGCGATACCTGGTGTGAATTGCAGAATCCCGCGAACCATCGAGTCTTTGAACGCAAGTTGCGCCCGAGGCCACTCGGCCGAGGGCACGCCTGCCTGGGCGTCACGCCAAAACACGCTCCCAACCACCCTCAACGGGAATCGGGATGCGGCATCTGGTCCCTCGTCTCTCAAGGGACGGTGGACCGAAGATTGGGCTGCCGGCGTACCGCGCCGGACACAGCGCATGGTGGGCGTCCTCGCTTTATCAATGCAGTGCATCCGGCGCGCAGCTGGCATTATGGCC

Thank you for you time <3

Karel

6

Hi! Sorry for the delay

Yes, the FASTQ file looks good, nothing strange... :thinking:

I think this is not likely to be the issue, but can we take a look to your manifest file? Maybe there is some path there pointing to another FASTQ which is actually causing the problem.

Best,

Sergio

7

Hello @salias , i tried to run same analysis on different dataset and i recieved similar error log:

(qiime2-amplicon-2024.2) svecka@svecserver:~/FRESH/FRESH_ITS/PL-01H/morning_glory_test/exported_sequences$

qiime q2-itsxpress trim-single \
  --i-per-sample-sequences ~/FRESH/FRESH_ITS/PL-01H/morning_glory_test/exported_sequences/merged_reads.qza \
  --p-region ITS2 \
  --p-taxa F \
  --p-threads 80 \
  --o-trimmed ~/FRESH/FRESH_ITS/PL-01H/morning_glory_test/exported_sequences/ITS2_trimmed_reads.qza

Plugin error from q2-itsxpress:

/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-ek8zhk98/NK2_AACGTGAT_L001_1_L001_R1_001.fastq.gz is not a(n) FastqGzFormat file:

Missing sequence for record beginning on line 13

Debug info has been saved to /tmp/qiime2-q2cli-err-341raxjk.log

Please find the Manifest here.

Thx a lot!
Qime2_metadata_template.txt (12.5 KB)

K.

9

Hi!

Thank you for the additional details. I set your commands with a monospaced font so I see them clearer.

As a quick recap, you tried two different datasets and obtained a similar error:

  1. Merged pared-end reads (as I see in the manifest), you run qiime q2-itsxpress trim-pair, everything works fine until sample X13T.
  2. Single-end reads (I assume), you run qiime q2-itsxpress trim-single -per-sam, sample NK2 fails.

I have a question that may (or may not) be related to the issue itself: why do you use qiime q2-itsxpress instead of qiime itsxpress?

Best,

Sergio

10

@salias Thank you for your reply again :slight_smile: When i use qiime itsxpress trim-pair --i-per-sample-sequences PL-02H_YT_demux_sequences.qza --p-region ITS2 --p-taxa F --p-threads 80 --o-trimmed trimmed-seqs.qza i got Error: QIIME 2 has no plugin/command named 'itsxpress'. Did you mean 'q2-itsxpress'? Its i whole chain of troubles :smiley:

11

I recognize this error from this post. You should be able to run ITSxpress using qiime itsxpress trim-pair instead of qiime q2-itsxpress trim-pair. If that is not the case, maybe there is some problem regarding how you installed ITSxpress. What was your procedure? Maybe solving this solves your original issue: something strange is going on if the proper command doesn't work.

The recommended installing method for ITSxpress, acording to the tutorial, is:

mamba activate qiime2-2024.2
mamba install -c bioconda itsxpress
qiime dev refresh-cache
qiime itsxpress # To check installation

Personally, what I do is directly open the YAML file of the QIIME 2 distribution I'm using, add the latest version¹ and install:

- itsxpress=2.1.3

Rember that, besides itsxpress, we don't need more plugins for running ITSxpress on QIIME 2. ITSxpress is already both standalone and a Q2 plugin.

Best,

Sergio

--

¹ 2.1.3 is the latest version as of December 10, 2024

12

Hello @salias, I tried to reinstall itsexpress according to your instructions and im afraid i met some problems which im not currently able to resolve :confused: PLS see my terminal history below to consider... I really aperciate your help <3

Greetings from Prague :slight_smile:

Karel

Welcome to Ubuntu 20.04 LTS (GNU/Linux 5.15.133.1-microsoft-standard-WSL2 x86_64)

System information as of Wed Dec 11 13:58:38 CET 2024

System load: 0.0 Processes: 9
Usage of /: 2.5% of 250.92GB Users logged in: 0
Memory usage: 5% IPv4 address for eth0: 172.23.31.27
Swap usage: 0%

351 updates can be installed immediately.
251 of these updates are security updates.
To see these additional updates run: apt list --upgradable

The list of available updates is more than a week old.
To check for new updates run: sudo apt update

This message is shown once once a day. To disable it please create the
/root/.hushlogin file.
root@112-C-KSV-NTB:~# ssh -p 22 svecka@10.150.112.145
svecka@10.150.112.145's password:
Welcome to Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-125-generic x86_64)

System information as of Wed Dec 11 12:58:46 PM UTC 2024

System load: 0.107421875 Temperature: 50.0 C
Usage of /: 90.6% of 914.35GB Processes: 673
Memory usage: 0% Users logged in: 2
Swap usage: 0% IPv4 address for enp134s0: 10.150.112.145

=> / is using 90.6% of 914.35GB

Expanded Security Maintenance for Applications is not enabled.

60 updates can be applied immediately.
To see these additional updates run: apt list --upgradable

2 additional security updates can be applied with ESM Apps.
Learn more about enabling ESM Apps service at https://ubuntu.com/esm

New release '24.04.1 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

Last login: Wed Dec 11 12:12:50 2024 from 10.150.112.33
(base) svecka@svecserver:~$ mamba activate qiime2-2024.2
Run 'mamba init' to be able to run mamba activate/deactivate
and start a new shell session. Or use conda to activate/deactivate.

(base) svecka@svecserver:~$ mamba install -c bioconda itsxpress

Looking for: ['itsxpress']

bioconda/linux-64 Using cache
bioconda/noarch Using cache
conda-forge/linux-64 Using cache
conda-forge/noarch Using cache
pkgs/r/noarch No change
pkgs/main/noarch No change
pkgs/r/linux-64 No change
pkgs/main/linux-64 No change

Pinned packages:

  • python 3.12.*

Transaction

Prefix: /home/svecka/miniconda3

All requested packages already installed

(base) svecka@svecserver:~ qiime dev refresh-cache qiime: command not found (base) svecka@svecserver:~ qiime itsxpress
qiime: command not found
(base) svecka@svecserver:~ (base) svecka@svecserver:~ conda activate qiime2-2024.2

EnvironmentNameNotFound: Could not find conda environment: qiime2-2024.2
You can list all discoverable environments with conda info --envs.

(base) svecka@svecserver:~$ conda env list

conda environments:

base * /home/svecka/miniconda3
eggnog-mapper /home/svecka/miniconda3/envs/eggnog-mapper
fresh_itsxpress /home/svecka/miniconda3/envs/fresh_itsxpress
my_env /home/svecka/miniconda3/envs/my_env
qiime2-2023.7 /home/svecka/miniconda3/envs/qiime2-2023.7
qiime2-amplicon-2024.10 /home/svecka/miniconda3/envs/qiime2-amplicon-2024.10
qiime2-amplicon-2024.2 /home/svecka/miniconda3/envs/qiime2-amplicon-2024.2
qiime2-env /home/svecka/miniconda3/envs/qiime2-env
qiime2-tiny-2023.9 /home/svecka/miniconda3/envs/qiime2-tiny-2023.9
sortmerna_env /home/svecka/miniconda3/envs/sortmerna_env
test_itsxpress /home/svecka/miniconda3/envs/test_itsxpress

(base) svecka@svecserver:~$ conda create -n qiime2-2024.2 -c qiime2/label/r2024.2 -c conda-forge -c bioconda python=3.8 qiime2
Retrieving notices: ...working... done
Channels:

  • qiime2/label/r2024.2
  • conda-forge
  • bioconda
  • defaults
    Platform: linux-64
    Collecting package metadata (repodata.json): done
    Solving environment: done

Package Plan

environment location: /home/svecka/miniconda3/envs/qiime2-2024.2

added / updated specs:
- python=3.8
- qiime2

The following packages will be downloaded:

package                    |            build
---------------------------|-----------------
bcrypt-4.2.0               |   py38h4005ec7_0         247 KB  conda-forge
cffi-1.17.0                |   py38heb5c249_0         235 KB  conda-forge
cryptography-43.0.0        |   py38hf793753_0         1.4 MB  conda-forge
h2-4.1.0                   |     pyhd8ed1ab_0          46 KB  conda-forge
hpack-4.0.0                |     pyh9f0ad1d_0          25 KB  conda-forge
hyperframe-6.0.1           |     pyhd8ed1ab_0          14 KB  conda-forge
libxml2-2.13.5             |       h0d44e9d_1         674 KB  conda-forge
lxml-5.3.0                 |   py38hd8ce619_0         1.3 MB  conda-forge
psutil-6.0.0               |   py38hfb59056_0         358 KB  conda-forge
pyparsing-3.1.4            |     pyhd8ed1ab_0          88 KB  conda-forge
python-3.8.20              |h4a871b0_2_cpython        21.1 MB  conda-forge
python_abi-3.8             |           5_cp38           6 KB  conda-forge
pyyaml-6.0.2               |   py38h2019614_0         181 KB  conda-forge
pyzmq-26.2.0               |   py38h6c80b9a_0         329 KB  conda-forge
qiime2-2024.2.0            |   py38h1da1538_0         657 KB  qiime2/label/r2024.2
setuptools-75.3.0          |     pyhd8ed1ab_0         761 KB  conda-forge
typing-extensions-4.12.2   |       hd8ed1ab_0          10 KB  conda-forge
urllib3-2.2.3              |     pyhd8ed1ab_0          96 KB  conda-forge
wheel-0.45.1               |     pyhd8ed1ab_0          62 KB  conda-forge
zipp-3.21.0                |     pyhd8ed1ab_0          21 KB  conda-forge
zstandard-0.23.0           |   py38h62bed22_0         396 KB  conda-forge
------------------------------------------------------------
                                       Total:        28.0 MB

The following NEW packages will be INSTALLED:

_libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
_openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-2_gnu
appdirs conda-forge/noarch::appdirs-1.4.4-pyh9f0ad1d_0
atpublic conda-forge/noarch::atpublic-5.0-pyhd8ed1ab_0
bcrypt conda-forge/linux-64::bcrypt-4.2.0-py38h4005ec7_0
bibtexparser conda-forge/noarch::bibtexparser-1.4.2-pyhd8ed1ab_0
brotli-python conda-forge/linux-64::brotli-python-1.1.0-py38h17151c0_1
bzip2 conda-forge/linux-64::bzip2-1.0.8-h4bc722e_7
ca-certificates conda-forge/linux-64::ca-certificates-2024.8.30-hbcca054_0
certifi conda-forge/noarch::certifi-2024.8.30-pyhd8ed1ab_0
cffi conda-forge/linux-64::cffi-1.17.0-py38heb5c249_0
charset-normalizer conda-forge/noarch::charset-normalizer-3.4.0-pyhd8ed1ab_0
cryptography conda-forge/linux-64::cryptography-43.0.0-py38hf793753_0
decorator conda-forge/noarch::decorator-4.4.2-py_0
dill conda-forge/noarch::dill-0.3.9-pyhd8ed1ab_0
flufl.lock conda-forge/noarch::flufl.lock-8.1.0-pyhd8ed1ab_0
globus-sdk conda-forge/noarch::globus-sdk-3.46.0-pyhd8ed1ab_0
h2 conda-forge/noarch::h2-4.1.0-pyhd8ed1ab_0
hpack conda-forge/noarch::hpack-4.0.0-pyh9f0ad1d_0
hyperframe conda-forge/noarch::hyperframe-6.0.1-pyhd8ed1ab_0
idna conda-forge/noarch::idna-3.10-pyhd8ed1ab_0
importlib_resourc~ conda-forge/noarch::importlib_resources-6.4.5-pyhd8ed1ab_0
keyutils conda-forge/linux-64::keyutils-1.6.1-h166bdaf_0
krb5 conda-forge/linux-64::krb5-1.21.3-h659f571_0
ld_impl_linux-64 conda-forge/linux-64::ld_impl_linux-64-2.43-h712a8e2_2
libblas conda-forge/linux-64::libblas-3.9.0-25_linux64_openblas
libcblas conda-forge/linux-64::libcblas-3.9.0-25_linux64_openblas
libedit conda-forge/linux-64::libedit-3.1.20191231-he28a2e2_2
libffi conda-forge/linux-64::libffi-3.4.2-h7f98852_5
libgcc conda-forge/linux-64::libgcc-14.2.0-h77fa898_1
libgcc-ng conda-forge/linux-64::libgcc-ng-14.2.0-h69a702a_1
libgfortran conda-forge/linux-64::libgfortran-14.2.0-h69a702a_1
libgfortran5 conda-forge/linux-64::libgfortran5-14.2.0-hd5240d6_1
libgomp conda-forge/linux-64::libgomp-14.2.0-h77fa898_1
libiconv conda-forge/linux-64::libiconv-1.17-hd590300_2
liblapack conda-forge/linux-64::liblapack-3.9.0-25_linux64_openblas
liblzma conda-forge/linux-64::liblzma-5.6.3-hb9d3cd8_1
liblzma-devel conda-forge/linux-64::liblzma-devel-5.6.3-hb9d3cd8_1
libnsl conda-forge/linux-64::libnsl-2.0.1-hd590300_0
libopenblas conda-forge/linux-64::libopenblas-0.3.28-pthreads_h94d23a6_1
libsodium conda-forge/linux-64::libsodium-1.0.18-h36c2ea0_1
libsqlite conda-forge/linux-64::libsqlite-3.47.2-hee588c1_0
libstdcxx conda-forge/linux-64::libstdcxx-14.2.0-hc0a3c3a_1
libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-14.2.0-h4852527_1
libuuid conda-forge/linux-64::libuuid-2.38.1-h0b41bf4_0
libxcrypt conda-forge/linux-64::libxcrypt-4.4.36-hd590300_1
libxml2 conda-forge/linux-64::libxml2-2.13.5-h0d44e9d_1
libxslt conda-forge/linux-64::libxslt-1.1.39-h76b75d6_0
libzlib conda-forge/linux-64::libzlib-1.3.1-hb9d3cd8_2
lxml conda-forge/linux-64::lxml-5.3.0-py38hd8ce619_0
ncurses conda-forge/linux-64::ncurses-6.5-he02047a_1
networkx conda-forge/noarch::networkx-3.1-pyhd8ed1ab_0
numpy conda-forge/linux-64::numpy-1.24.4-py38h59b608b_0
openssl conda-forge/linux-64::openssl-3.4.0-hb9d3cd8_0
pandas conda-forge/linux-64::pandas-1.5.3-py38hdc8b05c_1
paramiko conda-forge/noarch::paramiko-3.5.0-pyhd8ed1ab_0
parsl conda-forge/noarch::parsl-2023.12.4-pyhd8ed1ab_0
pip conda-forge/noarch::pip-24.3.1-pyh8b19718_0
psutil conda-forge/linux-64::psutil-6.0.0-py38hfb59056_0
pycparser conda-forge/noarch::pycparser-2.22-pyhd8ed1ab_0
pyjwt conda-forge/noarch::pyjwt-2.9.0-pyhd8ed1ab_1
pynacl conda-forge/linux-64::pynacl-1.5.0-py38h01eb140_3
pyparsing conda-forge/noarch::pyparsing-3.1.4-pyhd8ed1ab_0
pysocks conda-forge/noarch::pysocks-1.7.1-pyha2e5f31_6
python conda-forge/linux-64::python-3.8.20-h4a871b0_2_cpython
python-dateutil conda-forge/noarch::python-dateutil-2.9.0-pyhd8ed1ab_0
python_abi conda-forge/linux-64::python_abi-3.8-5_cp38
pytz conda-forge/noarch::pytz-2024.2-pyhd8ed1ab_0
pyyaml conda-forge/linux-64::pyyaml-6.0.2-py38h2019614_0
pyzmq conda-forge/linux-64::pyzmq-26.2.0-py38h6c80b9a_0
qiime2 qiime2/label/r2024.2/linux-64::qiime2-2024.2.0-py38h1da1538_0
readline conda-forge/linux-64::readline-8.2-h8228510_1
requests conda-forge/noarch::requests-2.32.3-pyhd8ed1ab_0
setproctitle conda-forge/linux-64::setproctitle-1.3.3-py38h01eb140_0
setuptools conda-forge/noarch::setuptools-75.3.0-pyhd8ed1ab_0
six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
tblib conda-forge/noarch::tblib-3.0.0-pyhd8ed1ab_0
tk conda-forge/linux-64::tk-8.6.13-noxft_h4845f30_101
tomlkit conda-forge/noarch::tomlkit-0.13.2-pyha770c72_0
typeguard conda-forge/noarch::typeguard-2.13.3-pyhd8ed1ab_0
typing-extensions conda-forge/noarch::typing-extensions-4.12.2-hd8ed1ab_0
typing_extensions conda-forge/noarch::typing_extensions-4.12.2-pyha770c72_0
tzlocal conda-forge/noarch::tzlocal-2.1-pyh9f0ad1d_0
urllib3 conda-forge/noarch::urllib3-2.2.3-pyhd8ed1ab_0
wheel conda-forge/noarch::wheel-0.45.1-pyhd8ed1ab_0
xz conda-forge/linux-64::xz-5.6.3-hbcc6ac9_1
xz-gpl-tools conda-forge/linux-64::xz-gpl-tools-5.6.3-hbcc6ac9_1
xz-tools conda-forge/linux-64::xz-tools-5.6.3-hb9d3cd8_1
yaml conda-forge/linux-64::yaml-0.2.5-h7f98852_2
zeromq conda-forge/linux-64::zeromq-4.3.5-h75354e8_4
zipp conda-forge/noarch::zipp-3.21.0-pyhd8ed1ab_0
zstandard conda-forge/linux-64::zstandard-0.23.0-py38h62bed22_0
zstd conda-forge/linux-64::zstd-1.5.6-ha6fb4c9_0

Proceed ([y]/n)? y

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

To activate this environment, use

$ conda activate qiime2-2024.2

To deactivate an active environment, use

$ conda deactivate

(base) svecka@svecserver:~ conda activate qiime2-2024.2 (qiime2-2024.2) svecka@svecserver:~ qiime itsxpress
qiime: command not found
(qiime2-2024.2) svecka@svecserver:~ qiime itsxpress --help qiime: command not found (qiime2-2024.2) svecka@svecserver:~ conda install -c bioconda itsxpress=2.1.3
Channels:

  • bioconda
  • conda-forge
  • defaults
    Platform: linux-64
    Collecting package metadata (repodata.json): done
    Solving environment: done

Package Plan

environment location: /home/svecka/miniconda3/envs/qiime2-2024.2

added / updated specs:
- itsxpress=2.1.3

The following packages will be downloaded:

package                    |            build
---------------------------|-----------------
lxml-5.2.2                 |   py38h81c7c4b_0         1.3 MB  conda-forge
pyzstd-0.16.1              |   py38h62bed22_0          91 KB  conda-forge
------------------------------------------------------------
                                       Total:         1.4 MB

The following NEW packages will be INSTALLED:

biopython conda-forge/linux-64::biopython-1.83-py38h01eb140_0
hmmer bioconda/linux-64::hmmer-3.1b2-3
icu conda-forge/linux-64::icu-73.2-h59595ed_0
itsxpress bioconda/noarch::itsxpress-2.1.3-pyhdfd78af_0
pyzstd conda-forge/linux-64::pyzstd-0.16.1-py38h62bed22_0
vsearch bioconda/linux-64::vsearch-2.22.1-h6a68c12_2
zlib conda-forge/linux-64::zlib-1.2.13-h4ab18f5_6

The following packages will be DOWNGRADED:

libsqlite 3.47.2-hee588c1_0 --> 3.46.0-hde9e2c9_0
libxml2 2.13.5-h0d44e9d_1 --> 2.12.7-hc051c1a_1
libzlib 1.3.1-hb9d3cd8_2 --> 1.2.13-h4ab18f5_6
lxml 5.3.0-py38hd8ce619_0 --> 5.2.2-py38h81c7c4b_0
python 3.8.20-h4a871b0_2_cpython --> 3.8.19-hd12c33a_0_cpython

Proceed ([y]/n)? y

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(qiime2-2024.2) svecka@svecserver:~ qiime --help qiime: command not found (qiime2-2024.2) svecka@svecserver:~ qiime2 --help
qiime2: command not found
(qiime2-2024.2) svecka@svecserver:~ qiime itsxpress --help qiime: command not found (qiime2-2024.2) svecka@svecserver:~

2 Likes
13

@salias, @Pardal_Oblackovy - note that there are newer install instructions for ITSxpress 2 on the project's GitHub page:

I would recommend starting here.

UPDATE: Sorry to bounce you back and forth between different installation instructions @Pardal_Oblackovy - ITSxpress will be moving to the new QIIME 2 Library soon, at which point the install instructions will be more centralized so finding the most recent ones shouldn't be an issue anymore.

4 Likes
15

Hello @gregcaporaso and @salias Hey guys :slight_smile: I reinstalled ITSxpress according to the latest instructions (GitHub - USDA-ARS-GBRU/itsxpress: Software to trim the ITS region of FASTQ sequences for amplicon sequencing analysis) and I'm afraid I'm doing something fundamentally wrong. See:

(Last login: Thu Dec 12 14:24:07 2024 from 10.150.112.33
(base) svecka@svecserver:~$ conda env create -n qiime2-amplicon-2024.5 --file https://data.qiime2.org/distro/amplicon/qiime2-amplicon-2024.5-py39-linux-conda.yml
activate qiime2-amplicon-2024.5
conda install -c bioconda -c conda-forge ITSxpress
qiime dev refresh-cache

CondaValueError: prefix already exists: /home/svecka/miniconda3/envs/qiime2-amplicon-2024.5

(base) svecka@svecserver:~ conda activate qiime2-amplicon-2024.5 (qiime2-amplicon-2024.5) svecka@svecserver:~ conda install -c bioconda -c conda-forge ITSxpress
Channels:

  • bioconda
  • conda-forge
  • defaults
    Platform: linux-64
    Collecting package metadata (repodata.json): done
    Solving environment: done

All requested packages already installed.

(qiime2-amplicon-2024.5) svecka@svecserver:~ qiime dev refresh-cache QIIME is caching your current deployment for improved performance. This may take a few moments and should only happen once per deployment. (qiime2-amplicon-2024.5) svecka@svecserver:~ qiime ITSxpress --info
Error: QIIME 2 has no plugin/command named 'ITSxpress'. Did you mean 'itsxpress'?
(qiime2-amplicon-2024.5) svecka@svecserver:~$ qiime itsxpress --info
Usage: qiime itsxpress [OPTIONS] COMMAND [ARGS]...
Try 'qiime itsxpress --help' for help.

Error: No such option: --info
(qiime2-amplicon-2024.5) svecka@svecserver:~$ qiime itsxpress
Usage: qiime itsxpress [OPTIONS] COMMAND [ARGS]...

Description: ITSxpress trims amplicon reads down to their ITS region.
ITSxpress is designed to support the calling of exact sequence variants
rather than OTUs. This newer method of sequence error-correction requires
quality score data from each sequence, so each input sequence must be
trimmed. ITSxpress makes this possible by taking FASTQ data, de-replicating
the sequences then identifying the start and stop sites using HMMSearch.
Results are parsed and the trimmed files are returned. The ITS 1, ITS2 or
the entire ITS region including the 5.8s rRNAgene can be selected. ALL
requires very long reads so it is not routinelyused with Illumina data.
ITSxpress uses the hmm models from ITSx so results are comparable.

Plugin website: GitHub - USDA-ARS-GBRU/q2_itsxpress: A plugin for qiime2 that runs ITSxpress
ITSxpress: GitHub - USDA-ARS-GBRU/itsxpress: Software to trim the ITS region of FASTQ sequences for amplicon sequencing analysis

Getting user support: Please post to the QIIME 2 forum for help with this
plugin: https://forum.qiime2.org

Options:
--version Show the version and exit.
--example-data PATH Write example data and exit.
--citations Show citations and exit.
--help Show this message and exit.

Commands:
trim-pair Trim paired-end reads, output merged reads for
use with Deblur
trim-pair-output-unmerged Trim paired-end reads, output unmerged reads for
use with Dada2
trim-single Trim single-end reads
(qiime2-amplicon-2024.5) svecka@svecserver:~$ qiime q2-itsxpress trim-single \

--i-per-sample-sequences ~/FRESH/FRESH_ITS/PL-01H/morning_glory_test/exported_sequences/merged_reads.qza
--p-region ITS2
--p-taxa F
--p-threads 80
FRES> --o-trimmed ~/FRESH/FRESH_ITS/PL-01H/morning_glory_test/exported_sequences/ITS2_trimmed_reads.qza
Error: QIIME 2 has no plugin/command named 'q2-itsxpress'. Did you mean 'itsxpress'?
(qiime2-amplicon-2024.5) svecka@svecserver:~$ qiime itsxpress trim-single --i-per-sample-sequences ~/FRESH/FRESH_ITS/PL-01H/morning_glory_test/exported_sequences/merged_reads.qza --p-region ITS2 --p-taxa F --p-threads 80 --o-tr
immed ~/FRESH/FRESH_ITS/PL-01H/morning_glory_test/exported_sequences/ITS2_trimmed_reads.qza
Plugin error from itsxpress:

Command '['vsearch', '--fastx_uniques', '/tmp/qiime2/svecka/data/799230a0-52d3-44ae-a056-4e9830066198/data/NK2_AACGTGAT_L001_1_L001_R1_001.fastq.gz', '--fastaout', '/tmp/itsxpress__aky9qfx/rep.fa', '--uc', '/tmp/itsxpress__aky9qfx/uc.txt', '--strand', 'both']' returned non-zero exit status 1.

Debug info has been saved to /tmp/qiime2-q2cli-err-mijx7gvg.log
(qiime2-amplicon-2024.5) svecka@svecserver:~$

qiime2-q2cli-err-mijx7gvg.txt (2.9 KB)
Thank you for your time and ha ve a nice weekend... Karel

16

Hi!

I'm not sure why this error is happening. With your Conda environment activated, could you share the output of these commands?

qiime tools validate ~/FRESH/FRESH_ITS/PL-01H/morning_glory_test/exported_sequences/merged_reads.qza
conda list --revisions

Best,

Sergio

22

A good catch by @colinbrislawn is that the issue could be disk space related:

ERROR:root:Could not perform dereplication with Vsearch. Error from Vsearch was:
vsearch v2.22.1_linux_x86_64, 754.6GB RAM, 64 cores
Fatal error: Unable to open FASTA output file for writing

Are you running QIIME 2 on your machine or on a cluster?

1 Like
23

Hey guys @salias @gregcaporaso @colinbrislawn, I'm sorry I didn't respond sooner. I've had a few days off. Anyway, I'm not sure what exactly the output of the command was supposed to be, but I'm attaching the output/terminla history below.

Environment should be running fine this time, but I'm worried that the problems could be caused by conflicting versions of Qiime (see att. pls)?

I'll do the analysis on our lab server and memory shoud be fine (HDD 3,4TB/ RAM 719GB avail). Thank you for your time again.
Karel_terminal_history_ITSxpress_troubleshooting_241218.txt (72.8 KB)

2 Likes
25

Hi @Pardal_Oblackovy ,

I'm consulting with other mods and then I'll get back to you. In the meantime, if disk space is not an issue, maybe there is some problem with your file quota? That would explain vsearch not being able to write to disk.

2 Likes
26

@salias There are no quotas or other "artificial" restrictions on our server but anyway i manually checked free space one more time:

du -ah /home/svecka | sort -rh | head -n 20

Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ubuntu--vg-ubuntu--lv 915G 832G 45G 95% /

and acutally is way smaller than i was expecting so i made more space in my home... so i increased it a bit:

(base) svecka@svecserver:/mnt/disk14TB/svecka/Vendy_Trans_2024/Vendy$ du -sh /home/svecka
204G /home/svecka

Quotas are disabled on our server... now i will try to rerun the analysis again.

THX :slight_smile:

2 Likes
27

Perfect, let us know if it worked!

Sergio

28

I am afraid I´m kind a trapped in some loop of errors. I thought i reinstalled th ITSxpress plugin correctly via Tutorial above into env (qiime2-amplicon-2024.2) because command:

qiime itsxpress trim-pair-output-unmerged --help

returned me this (as i was expecting):

Usage: qiime itsxpress trim-pair-output-unmerged [OPTIONS]

ITSxpress trimPairUnmerged takes the qza type PairedEndSquencesWithQuality.
The qza format must be the SingleLanePerSamplePairedEndFastqDirFmt and only
contain two fastq files. The function returns unmerged, trimmed reads for
use by Dada2. ...and so on.

But when i try run the analysis itself i got again the:

(qiime2-amplicon-2024.2) svecka@svecserver: qiime q2-itsxpress trim-pair
Error: QIIME 2 has no plugin/command named 'q2-itsxpress'. Did you mean 'itsxpress'?
(qiime2-amplicon-2024.2) svecka@svecserver: --i-per-sample-sequences PL-02H_YT_demux_sequences.qza \

--p-region ITS2
--p-taxa F
--p-threads 80
--o-trimmed trimmed-seqs.qza

I was testing also if running Qiime2 in screen (4.09.00) make any difference but no :frowning:

I am already very frustrated NGL... :smiley: Thank you for your help! @salias @gregcaporaso

2 Likes
29

Hello Pardál,

Thank you for your perseverance! You are so close!

This looks like a small typo causing this issue. Take a look:

qiime itsxpress trim-pair-output-unmerged --help

returned me this (as i was expecting):

(qiime2-amplicon-2024.2) svecka@svecserver: qiime q2-itsxpress trim-pair
Error: QIIME 2 has no plugin/command named 'q2-itsxpress'. Did you mean 'itsxpress'?

These are not the same!

qiime itsxpress trim-pair-output-unmerged --help
qiime q2-itsxpress trim-pair

In addition, this second command is missing the trailing slash, which causes this:

(qiime2-amplicon-2024.2) svecka@svecserver: --i-per-sample-sequences PL-02H_YT_demux_sequences.qza \

The linux terminal thinks that's a new command, which causes an error.

Take as much time as you need to review which commands are working and which ones are not. Let us know if you have more questions!

1 Like