Error whilst using v-search join pairs

I am attempting to join paired end reads using the v-search join pairs tool so that I can later perform Deblur to filter and clean up my data. I am doing 16S rRNA sequencing on a ~450bp amplified region.

The command I am running is

qiime vsearch join-pairs --i-demultiplexed-seqs Gya-cambodia-bacteria-paired-end-demux2.qza --o-joined-sequences Gya-cambodia-bacteria-paired-end-demux-joined2.qza --verbose

and I get the error message:

/home/lucas/miniconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_vsearch/_join_pairs.py:90: YAMLLoadWarning: calling yaml.load() without Loader=… is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
demultiplexed_seqs.metadata.pathspec)))[‘phred-offset’]
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command: vsearch --fastq_mergepairs /home/Public/Ps2/PsTmp/qiime2-archive-crik8_du/db07cc02-bfd6-429f-9655-49d098f7ad08/data/1-GYA_0_L001_R1_001.fastq.gz --reverse /home/Public/Ps2/PsTmp/qiime2-archive-crik8_du/db07cc02-bfd6-429f-9655-49d098f7ad08/data/1-GYA_1_L001_R2_001.fastq.gz --fastqout /home/Public/Ps2/PsTmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-r9u86guo/1-GYA_0_L001_R1_001.fastq --fastq_ascii 33 --fastq_minlen 1 --fastq_minovlen 10 --fastq_maxdiffs 10 --fastq_qmin 0 --fastq_qminout 0 --fastq_qmax 41 --fastq_qmaxout 41

vsearch v2.7.0_linux_x86_64, 141.5GB RAM, 16 cores

Merging reads 100%

Fatal error: More forward reads than reverse reads
Traceback (most recent call last):
File “/home/lucas/miniconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/commands.py”, line 327, in call
results = action(**arguments)
File “</home/lucas/miniconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/decorator.py:decorator-gen-131>”, line 2, in join_pairs
File “/home/lucas/miniconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 240, in bound_callable
output_types, provenance)
File “/home/lucas/miniconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 383, in callable_executor
output_views = self._callable(**view_args)
File “/home/lucas/miniconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_vsearch/_join_pairs.py”, line 57, in join_pairs
qmax, qmaxout)
File “/home/lucas/miniconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_vsearch/_join_pairs.py”, line 141, in _join_pairs_w_command_output
run_command(cmd)
File “/home/lucas/miniconda3/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_vsearch/_cluster_features.py”, line 33, in run_command
subprocess.run(cmd, check=True)
File “/home/lucas/miniconda3/envs/qiime2-2019.7/lib/python3.6/subprocess.py”, line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘vsearch’, ‘–fastq_mergepairs’, ‘/home/Public/Ps2/PsTmp/qiime2-archive-crik8_du/db07cc02-bfd6-429f-9655-49d098f7ad08/data/1-GYA_0_L001_R1_001.fastq.gz’, ‘–reverse’, ‘/home/Public/Ps2/PsTmp/qiime2-archive-crik8_du/db07cc02-bfd6-429f-9655-49d098f7ad08/data/1-GYA_1_L001_R2_001.fastq.gz’, ‘–fastqout’, ‘/home/Public/Ps2/PsTmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-r9u86guo/1-GYA_0_L001_R1_001.fastq’, ‘–fastq_ascii’, ‘33’, ‘–fastq_minlen’, ‘1’, ‘–fastq_minovlen’, ‘10’, ‘–fastq_maxdiffs’, ‘10’, ‘–fastq_qmin’, ‘0’, ‘–fastq_qminout’, ‘0’, ‘–fastq_qmax’, ‘41’, ‘–fastq_qmaxout’, ‘41’]’ returned non-zero exit status 1.

Plugin error from vsearch:

Command ‘[‘vsearch’, ‘–fastq_mergepairs’, ‘/home/Public/Ps2/PsTmp/qiime2-archive-crik8_du/db07cc02-bfd6-429f-9655-49d098f7ad08/data/1-GYA_0_L001_R1_001.fastq.gz’, ‘–reverse’, ‘/home/Public/Ps2/PsTmp/qiime2-archive-crik8_du/db07cc02-bfd6-429f-9655-49d098f7ad08/data/1-GYA_1_L001_R2_001.fastq.gz’, ‘–fastqout’, ‘/home/Public/Ps2/PsTmp/q2-SingleLanePerSampleSingleEndFastqDirFmt-r9u86guo/1-GYA_0_L001_R1_001.fastq’, ‘–fastq_ascii’, ‘33’, ‘–fastq_minlen’, ‘1’, ‘–fastq_minovlen’, ‘10’, ‘–fastq_maxdiffs’, ‘10’, ‘–fastq_qmin’, ‘0’, ‘–fastq_qminout’, ‘0’, ‘–fastq_qmax’, ‘41’, ‘–fastq_qmaxout’, ‘41’]’ returned non-zero exit status 1.

See above for debug info.

I am running qiime2-2019.7, through conda.

As a test to check that there is enough overlap between forward and reverse reads, for one of my indexes I uploaded all the data into Geneious and was able to successfully pair and merge forward and reverse reads into ~450bp long sequences.

I have tried to solve this issue using advice from similar posts but to no avail, any help would be greatly appreciated.

1 Like

Hi @lghuggins,

This is a guess, but are these the sequences from you rother post?

You might try setting your --p-truncqual which may trim your reads, but would get rid of the last 50 or so bases that seem to have issues.

Best,
Justine

ETA: You might also want to check whether the forward and reverse sequences

Best,
Justine

1 Like

Hi @jwdebelius,

Thanks for your advice, the sequences this time actually aren't from my previous post and I've rechecked to ensure that the imported data is separate forward and reverse reads which it is. See data quality plots here:

image

Before trying to join data using v-search I was attempting to join pairs and filter using DADA2 which I could not get to work either. Truncating reads to a variety of different lengths (both forward and reverse) including 200, 220, 240, 260 and 280 all failed to pair using DADA2. Would this not have had the same effect as the --p-truncqual command? If not which principal command would I run to complete --p-truncqual? V-search?

Looking at other posts that have had a similar issue to mine having more forward than reverse reads as alluded to by this message:

Fatal error: More forward reads than reverse reads

Is not actually the issue as there are almost never the same number of forward and reverse reads, when I've done v-search join pairs and DADA2 commands before the difference between numbers of forward and reverse reads can be within the range of thousands and the command still runs, so I'm not sure this is the true problem behind my command failing.

Any more help with this would be greatly appreciated.

I've been able to analyse the data as single-end reads thus far.

All the best,

Lucas

2 Likes

HI @lghuggins,

I certainly get errors in both vsearch in QIIME and vsearch alone when I try to work with reads that aren’t paired, but maybe you’re luckier than i am.

If you can’t get them to pair, my suggestion going forward would be to use single-end reads.

Best,
Justine

1 Like

Just to confirm,what is the principal command from which to do --p-truncqual command? V-search?

Cheers

Lucas

this is a parameter used by dada2

1 Like

A post was split to a new topic: Looking for help joining paired-end reads