Plugin error from itsxpress: Missing sequence for record beginning on line 13

[anethdavid]

Hello Everyone!

I am experiencing a problem with itsxpress.

I ran:
qiime itsxpress trim-pair-output-unmerged --i-per-sample-sequences qiime-its-obj1-ITS1/demux.qza --p-region ITS1 --p-taxa F --p-cluster-id 1.0 --o-trimmed qiime-its-obj1-ITS1/trimed-ITS1.qza

I got error message:

Plugin error from itsxpress:

/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-14p3_zfv/AM-ITS1F-37_S83_L001_R2_001.fastq.gz is not a(n) FastqGzFormat file:

Missing sequence for record beginning on line 13

Debug info has been saved to /tmp/qiime2-q2cli-err-3mfomwp_.log

The log file

Traceback (most recent call last):
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/plugin/model/file_format.py", line 24, in validate
    self._validate_(level)
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_types/per_sample_sequences/_format.py", line 279, in _validate_
    self._check_n_records(record_count_map[level])
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2_types/per_sample_sequences/_format.py", line 250, in _check_n_records
    % (i * 4 + 1))
qiime2.core.exceptions.ValidationError: Missing sequence for record beginning on line 13

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/q2cli/commands.py", line 311, in __call__
    results = action(**arguments)
  File "</home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/decorator.py:decorator-gen-277>", line 2, in trim_pair_output_unmerged
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py", line 231, in bound_callable
    output_types, provenance)
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/action.py", line 393, in _callable_executor_
    spec.qiime_type, output_view, spec.view_type, prov)
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/sdk/result.py", line 265, in _from_view
    result = transformation(view)
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/core/transform.py", line 68, in transformation
    self.validate(view)
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/core/transform.py", line 143, in validate
    view.validate('min')
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/plugin/model/directory_format.py", line 171, in validate
    getattr(self, field)._validate_members(collected_paths, level)
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/plugin/model/directory_format.py", line 101, in _validate_members
    self.format(path, mode='r').validate(level)
  File "/home/aneth.david/miniconda3/envs/qiime2-2019.4/lib/python3.6/site-packages/qiime2/plugin/model/file_format.py", line 29, in validate
    ) from e
qiime2.core.exceptions.ValidationError: /tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-14p3_zfv/AM-ITS1F-37_S83_L001_R2_001.fastq.gz is not a(n) FastqGzFormat file:

Missing sequence for record beginning on line 13

I did restart terminal and re-imported reads but I still get the same error.

I however have run itsxpress successfully on the same data only Tuesday this week, only difference was instead of ALL, I used ITS1 in option --p-region (Almost all reads were filtered out hence retrying fishing out ITS1 instead). I’m not sure what is the problem this time.

Please help!

Kind regards,
Aneth.

[Nicholas_Bokulich]

Hi @anethdavid,
What version of q2-itsxpress are you running? This error message appears to occur in older versions of the plugin when all reads are filtered out:

q2-ITSxpress generates empty sequences?

ITSxpress v1.7.2 fixes bug causing “Line 17” errors. @omera.WIS @einamart @bsen2018 @Nicholas_Bokulich @ebolyen Impact Fixes an error that occasionally caused come qiime itsxpress runs to crash. Fixes an error in the way that HMMER data was parsed that caused about 0.2% of reads to be incorrectly trimmed, including some to be trimmed to length 0 which was causing the Qiime issue. For more details see: https://github.com/USDA-ARS-GBRU/itsxpress/issues/8 How to fix To update make sure that bot…

Could you confirm you are running version 1.7.2 or later?

cc: @Adam_Rivers

[anethdavid]

Hi,

Thank you for getting back to me.

Version: QIIME 2 Plugin 'itsxpress' version 1.7.4 (from package 'q2-itsxpress' version 1.7.4)

And I did run itsxpress on the same dataset just this week without getting that error.

Best,
Aneth.

[anethdavid]

Also, I tried to run itsxpress on another dataset, I got the same error:

Plugin error from itsxpress:

/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-31n1ujfk/AM-ITS1F-32_S78_L001_R2_001.fastq.gz is not a(n) FastqGzFormat file:

Missing sequence for record beginning on line 9

Debug info has been saved to /tmp/qiime2-q2cli-err-fituf0xj.log

[Nicholas_Bokulich]

Hi @anethdavid,
I am not sure what is going on here — am hoping the developer @Adam_Rivers can provide some insight.
Thanks for your patience!

[Adam_Rivers]

Sorry, this is caused by an old version of ITSxpress.

The Qiime plugin q2-itsxpress and the stand alone version ITSxpress are different python packages, but the qiime plugin uses the stand alone package. To update it, you need to activate your conda environment then

conda update -c bioconda itsxpress

itsxprress should be 1.7.2 and q2-itsxpress should be 1.7.4.

We will be combining them with the next big update.

Please let me know if you continue to have problems, I’m happy to test ITSxpress on your data or have a closer look at your configuration.

[anethdavid]

Hi Adam,

Thanks for getting back to me, I have the latest versions of both:

QIIME 2 Plugin 'itsxpress' version 1.7.4 (from package 'q2-itsxpress' version 1.7.4)

@Adam_Rivers

itsxprress should be 1.7.2

How do I get itsxpress to 1.7.2?

And I don't understand why I get an error only with parameters ITS1 and ITS2 for option --p-region but not with parameter ALL, and its not even the same error, see below.

When I run this:
qiime itsxpress trim-pair-output-unmerged --i-per-sample-sequences demux-its-obj2.qza --p-region ITS1 --p-taxa F --p-cluster-id 1.0 --o-trimmed trimmed-its1-obj2

The outcome is:
Plugin error from itsxpress:
/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-m16rjyvj/AM-ITS1F-32_S78_L001_R2_001.fastq.gz is not a(n) FastqGzFormat file:

Missing sequence for record beginning on line 9

Debug info has been saved to /tmp/qiime2-q2cli-err-k_87b3qe.log

For ITS2
qiime itsxpress trim-pair-output-unmerged --i-per-sample-sequences demux-its-obj2.qza --p-region ITS2 --p-taxa F --p-cluster-id 1.0 --o-trimmed trimmed-ITS2-obj2

Outcome is:
Plugin error from itsxpress:
/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-f28zmd9t/AM-ITS1F-36_S82_L001_R1_001.fastq.gz is not a(n) FastqGzFormat file:

 Missing sequence for record beginning on line 17

Debug info has been saved to /tmp/qiime2-q2cli-err-9yhi1oq7.log

But for parameter ALL:
qiime itsxpress trim-pair-output-unmerged --i-per-sample-sequences demux-its-obj2.qza --p-region ALL --p-taxa F --p-cluster-id 1.0 --o-trimmed trimmed-ALL-obj2

Outcome is:
Saved SampleData[PairedEndSequencesWithQuality] to: trimmed-ALL-obj2.qza

What am I missing?

Please help.

Regards,
Aneth.

[Adam_Rivers]

You update ITSxpress (the stand-alone package) by running
conda update -c bioconda itsxpress

or if you have not installed it before:
conda install -c bioconda itsxpress

you can verify it was installed by running (within your conda environment)
conda list | grep "itsxpress"

Could you let me know the name and sequences of the primer pairs you used?
A few of the primer sets are reversed and we do not currently support them.

[anethdavid]

Hello Adam,

I have updated itsxpress again, will re-reun the analyses and give feedback.

@Adam_Rivers

Could you let me know the name and sequences of the primer pairs you used?

ITS1F: CTTGGTCATTTAGAGGAAGTAA
ITS4: TCCTCCGCTTATTGATATGC

Regards,
Aneth.

[anethdavid]

I updated and repeated the run, same error happened.

[Adam_Rivers]

You are using the primers ITS1F and ITS4 which span the ITS1, 5.8S and ITS2 regions. Did you sequence this with illumina 2x300 paired end reads or some other technology? If you sequenced this with Illumina your fragment size is too long to merge the reads so no reads are returned by ITSxpress. This causes ITSxpress to throw a number of errors, I intend to add clearer exception handling in future versions that better explain what the problem is.

If you have unmergeable reads, I would recommend you pick the ITS1 region then use only the forward reads. Reload your data using only the forward reads as single end data. then run ITSxpress in single end mode as:

qiime itsxpress  trim-single --i-per-sample-sequences F_samples.qza  --p-region ITS1 --p-taxa F --p-threads 2 --p-cluster-id 1.0 --o-trimmed F_samples_trimmed.qza

Dada2 has a single end mode qiime dada2 denoise-single that you can use.

[anethdavid]

Ah I see, this makes a lot of sense. That is why it works with option ALL but very few reads do remain.

Yes the sequencing was with Illumina, paired-end 300bp reads.

Many thanks, will give this suggestion a try.

Regards,
Aneth.