qiime vsearch merge-pairs Missing option and Command not found

yangyang · March 21, 2024, 9:05am

hello,
I'm using qiime vsearch merge-pairs to merge.
When I tray to perform command with this line of code:

#!/bin/bash
#PBS -q batch
#PBS -o merge.o
#PBS -e merge.e
source activate qiime2-amplicon-2024.2
cd /home/libofeng/yangyang990124/16s/skip/trim/
qiime vsearch merge-pairs \
 --i-demultiplexed-seqs reads_trimmed.qza \
 --o-merged-sequences reads_trimmed_joined.qza \
 --o-unmerged-sequences unmerged_reads_trimmed_joined.qza \
 --output-dir /home/libofeng/yangyang990124/16s/skip/merge/output

I got an error:

Plugin error from vsearch:

  Command '['vsearch', '--fastq_mergepairs', '/tmp/qiime2/yangyang990124/data/23a772fe-0e4c-4a4b-9da4-67bbdcc0e95e/data/P12_31_L001_R1_001.fastq.gz', '--reverse', '/tmp/qiime2/yangyang990124/data/23a772fe-0e4c-4a4b-9da4-67bbdcc0e95e/data/P12_71_L001_R2_001.fastq.gz', '--fastqout', '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-be8wbjwg/P12_23_L001_R1_001.fastq', '--fastqout_notmerged_fwd', '/tmp/q2-SingleLanePerSamplePairedEndFastqDirFmt-p80cg3xe/P12_23_L001_R1_001.fastq', '--fastqout_notmerged_rev', '/tmp/q2-SingleLanePerSamplePairedEndFastqDirFmt-p80cg3xe/P12_23_L001_R2_001.fastq', '--fastq_ascii', '33', '--fastq_minlen', '1', '--fastq_minovlen', '10', '--fastq_maxdiffs', '10', '--fastq_qmin', '0', '--fastq_qminout', '0', '--fastq_qmax', '41', '--fastq_qmaxout', '41', '--fasta_width', '0', '--threads', '1']' returned non-zero exit status 1.

Debug info has been saved to /tmp/qiime2-q2cli-err-ck_ka3bg.log

what can I do?
Thank you for your help.

Micro_Biologist · March 21, 2024, 11:35am

Hello!

Can you post the contents of your error log or rerun the command with the --verbose flag.

Also worth asking do you get the same error if running directly in the command line after activating the conda environment?

yangyang · March 21, 2024, 12:14pm

I have resubmitted this command several times, and got a new error

vsearch v2.22.1_linux_x86_64, 251.6GB RAM, 28 cores
https://github.com/torognes/vsearch

Merging reads 100%
    143334  Pairs
    139218  Merged (97.1%)
      4116  Not merged (2.9%)

Pairs that failed merging due to various reasons:
       110  too few kmers found on same diagonal
      4006  alignment score too low, or score drop too high

Statistics of all reads:
    224.50  Mean read length

Statistics of merged reads:
    419.48  Mean fragment length
      8.19  Standard deviation of fragment length
      0.34  Mean expected error in forward sequences
      0.37  Mean expected error in reverse sequences
      0.34  Mean expected error in merged sequences
      0.05  Mean observed errors in merged region of forward sequences
      0.05  Mean observed errors in merged region of reverse sequences
      0.09  Mean observed errors in merged region
Fatal error: FASTQ quality value (51) above qmax (41)
By default, quality values range from 0 to 41.
To allow higher quality values, please use the option --fastq_qmax 51
Traceback (most recent call last):
  File "/home/libofeng/yangyang990124/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 520, in __call__
    results = self._execute_action(
  File "/home/libofeng/yangyang990124/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2cli/commands.py", line 581, in _execute_action
    results = action(**arguments)
  File "<decorator-gen-707>", line 2, in merge_pairs
  File "/home/libofeng/yangyang990124/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 342, in bound_callable
    outputs = self._callable_executor_(
  File "/home/libofeng/yangyang990124/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/qiime2/sdk/action.py", line 566, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/libofeng/yangyang990124/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_vsearch/_merge_pairs.py", line 52, in merge_pairs
    _, merged, unmerged = _merge_pairs_w_command_output(
  File "/home/libofeng/yangyang990124/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_vsearch/_merge_pairs.py", line 164, in _merge_pairs_w_command_output
    run_command(cmd)
  File "/home/libofeng/yangyang990124/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/site-packages/q2_vsearch/_cluster_features.py", line 33, in run_command
    subprocess.run(cmd, check=True)
  File "/home/libofeng/yangyang990124/miniconda3/envs/qiime2-amplicon-2024.2/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['vsearch', '--fastq_mergepairs', '/tmp/qiime2/yangyang990124/data/23a772fe-0e4c-4a4b-9da4-67bbdcc0e95e/data/P12_31_L001_R1_001.fastq.gz', '--reverse', '/tmp/qiime2/yangyang990124/data/23a772fe-0e4c-4a4b-9da4-67bbdcc0e95e/data/P12_71_L001_R2_001.fastq.gz', '--fastqout', '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-b8czlbzm/P12_23_L001_R1_001.fastq', '--fastqout_notmerged_fwd', '/tmp/q2-SingleLanePerSamplePairedEndFastqDirFmt-9pi012_p/P12_23_L001_R1_001.fastq', '--fastqout_notmerged_rev', '/tmp/q2-SingleLanePerSamplePairedEndFastqDirFmt-9pi012_p/P12_23_L001_R2_001.fastq', '--fastq_ascii', '33', '--fastq_minlen', '1', '--fastq_minovlen', '10', '--fastq_maxdiffs', '10', '--fastq_qmin', '0', '--fastq_qminout', '0', '--fastq_qmax', '41', '--fastq_qmaxout', '41', '--fasta_width', '0', '--threads', '1']' returned non-zero exit status 1.
Plugin error from vsearch:

  Command '['vsearch', '--fastq_mergepairs', '/tmp/qiime2/yangyang990124/data/23a772fe-0e4c-4a4b-9da4-67bbdcc0e95e/data/P12_31_L001_R1_001.fastq.gz', '--reverse', '/tmp/qiime2/yangyang990124/data/23a772fe-0e4c-4a4b-9da4-67bbdcc0e95e/data/P12_71_L001_R2_001.fastq.gz', '--fastqout', '/tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-b8czlbzm/P12_23_L001_R1_001.fastq', '--fastqout_notmerged_fwd', '/tmp/q2-SingleLanePerSamplePairedEndFastqDirFmt-9pi012_p/P12_23_L001_R1_001.fastq', '--fastqout_notmerged_rev', '/tmp/q2-SingleLanePerSamplePairedEndFastqDirFmt-9pi012_p/P12_23_L001_R2_001.fastq', '--fastq_ascii', '33', '--fastq_minlen', '1', '--fastq_minovlen', '10', '--fastq_maxdiffs', '10', '--fastq_qmin', '0', '--fastq_qminout', '0', '--fastq_qmax', '41', '--fastq_qmaxout', '41', '--fasta_width', '0', '--threads', '1']' returned non-zero exit status 1.

Fatal error: FASTQ quality value (51) above qmax (41)
I use this command , but not work

#!/bin/bash
#PBS -q batch
#PBS -o m.o
#PBS -e m.e
source activate qiime2-amplicon-2024.2
cd /home/libofeng/yangyang990124/16s/skip/trim/
qiime vsearch merge-pairs \
 --i-demultiplexed-seqs reads_trimmed.qza \
 --o-merged-sequences reads_trimmed_joined.qza \
 --o-unmerged-sequences unmerged_reads_trimmed_joined.qza \
 --output-dir /home/libofeng/yangyang990124/16s/skip/merge/output \
 --fastq_qmax 51 \
 --verbose

result

 There was a problem with the command:
 (1/1?) No such option: --fastq_qmax

SoilRotifer · March 21, 2024, 12:16pm

Hi @yangyang,

I think this is the cause of the problem:

Assuming these reads were generated from an Illumina sequencer, the quality scores should never be above 41. How was the sequence data processed prior to the merging step?

yangyang · March 22, 2024, 7:19am

Thanks for your reply!
We stipulate that programs cannot be run on the login interface and must be submitted.
And I run the command --verbose. The log is a repetition of this paragraph

Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command: vsearch --fastq_mergepairs /tmp/qiime2/yangyang990124/data/e5ea6d14-81a9-4bdc-ac0c-d2ded718b607/data/N1_0_L001_R1_001.fastq.gz --reverse /tmp/qiime2/yangyang990124/data/e5ea6d14-81a9-4bdc-ac0c-d2ded718b607/data/N1_40_L001_R2_001.fastq.gz --fastqout /tmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-hqud9stb/N1_0_L001_R1_001.fastq --fastqout_notmerged_fwd /tmp/q2-SingleLanePerSamplePairedEndFastqDirFmt-h9cq_i8l/N1_0_L001_R1_001.fastq --fastqout_notmerged_rev /tmp/q2-SingleLanePerSamplePairedEndFastqDirFmt-h9cq_i8l/N1_0_L001_R2_001.fastq --fastq_ascii 33 --fastq_minlen 1 --fastq_minovlen 10 --fastq_maxdiffs 10 --fastq_qmin 0 --fastq_qminout 0 --fastq_qmax 41 --fastq_qmaxout 41 --fasta_width 0 --threads 1

What information can be obtained from it?

yangyang · March 22, 2024, 7:19am

thanks for your reply.
I download this data from NCBI, he said the Instrument was Illumina HiSeq 2500.
Before merge ,I import the data and cutadapt.
This is my code.
1.import

#!/bin/bash
#PBS -q batch
#PBS -o import.o
#PBS -e import.e
source activate qiime2-amplicon-2024.2
cd /home/libofeng/yangyang990124/16s/import/
qiime tools import \
 --type 'SampleData[PairedEndSequencesWithQuality]' \
 --input-path manifest.txt \
 --output-path /home/libofeng/yangyang990124/16s/skip/import/summary_pe.qza \
 --input-format PairedEndFastqManifestPhred33V2

qiime demux summarize \
  --i-data summary_pe.qza \
  --o-visualization summary_pe.qzv

2.cutadapt

#!/bin/bash
#PBS -q batch
#PBS -o trim.o
#PBS -e trim.e
source activate qiime2-amplicon-2024.2

cd /home/libofeng/yangyang990124/16s/skip/import/
 qiime cutadapt trim-paired \
   --i-demultiplexed-sequences summary_pe.qza \
   --p-cores 20 \
   --p-front-f CCTACGGGNGGCWGCAG \
   --p-front-r GACTACHVGGGTATCTAATCC \
   --p-discard-untrimmed \
   --p-no-indels \
   --o-trimmed-sequences /home/libofeng/yangyang990124/16s/skip/trim/reads_trimmed.qza

cd /home/libofeng/yangyang990124/16s/skip/trim/
qiime demux summarize \
   --i-data reads_trimmed.qza \
   --o-visualization reads_trimmed_summary.qzv

Finally, I use the reads_trimmed.qza (after cutadapt trim-paired) to merge.
Is there a problem somewhere? Looking forward to your reply.

colinbrislawn · March 28, 2024, 1:42pm

Hello @yangyang,

Are you still getting this error about fastq quality score ranges?

This could happen if the files in manifest.txt were processed or filtered before being imported into Qiime2.

This has happened to me; the sequencing core did their own filtering without telling me. When I asked them, they were able to send me the raw files.

Talking to your sequencing core would be good! Perhaps they can send you raw files too!

system · April 30, 2024, 2:12am

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.