cutadapt.seqio.FormatError: Reads are improperly paired

Dear all,

I am trying to combine sequence data from two runs A and B. Data in A and B are unmerged fastq read pairs R1 and R2. I am describing the problem breifly here, it’s an odd one. Despite my short post I would be extremely grateful to hear an opinion on this. Running Qiime 2017.12.

  • Processing for run A (completes successfully)

    • Spliting .fastq files by primers (to retain only 18S reads in mixed-loci libraries with identical barcodes: Tried this output during analysis A&B - fails
    • trimming using trimmomatic: Tried this output during analysis A&B - fails
    • qiime tools import
    • qiime2cli demux summarize
    • qiime2cli dada2 denoise-paired
  • Processing for run B (completes successfully)

    • qiime2cli tools import
    • qiime cutadapt trim-paired
    • qiime2cli demux summarize
    • qiime2cli metadata tabulate
    • qiime2cli dada2 denoise-paired
  • Processing for run A&B

    • qiime2cli tools import
    • qiime cutadapt trim-paired … fails - see error messages below - parts of R1 and R2 are “improperly paired”. Any one any ideas how this can happen, has this happened to anyone before.

I would be looking forward to hearing of your experiences and opinions.

Kind regards and thanks for working on Qiime,

Paul

Error messages

(a.k.a computer vomit)

++ qiime cutadapt trim-paired --i-demultiplexed-sequences /Users/paul/Documents/CU_combined/Zenodo/Qiime/040_18S_paired-end-import.qza --p-cores 2 --p-front-f GCGGTAATTCCAGCTCCAA --p-f
ront-r TTGGCAAATGCTTTCGC --p-error-rate 0.2 --o-trimmed-sequences /Users/paul/Documents/CU_combined/Zenodo/Qiime/042_18S_paired-end-trimmed.qza --verbose                               
Running external command line application. This may print messages to stdout and/or stderr.                                                                                             
The commands to be run are below. These commands cannot be manually re-run as they will depend on temporary files that no longer exist.                                                 
                                                                                                                                                                                        
Command: cutadapt --cores 2 --error-rate 0.2 --times 1 --overlap 3 -o /var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/q2-CasavaOneEightSingleLanePerSampleDirFmt-oi0k6eyg/CH00-0301_62_
L001_R1_001.fastq.gz -p /var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/q2-CasavaOneEightSingleLanePerSampleDirFmt-oi0k6eyg/CH00-0301_63_L001_R2_001.fastq.gz --front GCGGTAATTCCAGCTCC
AA -G TTGGCAAATGCTTTCGC /var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/qiime2-archive-zxavzj1i/59b9a374-550a-45f7-9467-5b22262183e3/data/CH00-0301_62_L001_R1_001.fastq.gz /var/folder
s/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/qiime2-archive-zxavzj1i/59b9a374-550a-45f7-9467-5b22262183e3/data/CH00-0301_63_L001_R2_001.fastq.gz                                               
                                                                                                                                                                                        
This is cutadapt 1.15 with Python 3.5.4                                                                                                                                                 
Command line parameters: --cores 2 --error-rate 0.2 --times 1 --overlap 3 -o /var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/q2-CasavaOneEightSingleLanePerSampleDirFmt-oi0k6eyg/CH00-0
301_62_L001_R1_001.fastq.gz -p /var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/q2-CasavaOneEightSingleLanePerSampleDirFmt-oi0k6eyg/CH00-0301_63_L001_R2_001.fastq.gz --front GCGGTAATTC
CAGCTCCAA -G TTGGCAAATGCTTTCGC /var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/qiime2-archive-zxavzj1i/59b9a374-550a-45f7-9467-5b22262183e3/data/CH00-0301_62_L001_R1_001.fastq.gz /var
/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/qiime2-archive-zxavzj1i/59b9a374-550a-45f7-9467-5b22262183e3/data/CH00-0301_63_L001_R2_001.fastq.gz                                        
Running on 2 cores                                                                                                                                                                      
Trimming 2 adapters with at most 20.0% errors in paired-end mode ...                                                                                                                    
ERROR: Traceback (most recent call last):                                                                                                                                               
  File "/Applications/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/cutadapt/pipeline.py", line 454, in run                                                                
    (n, bp1, bp2) = self._pipeline.process_reads()                                                                                                                                      
  File "/Applications/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/cutadapt/pipeline.py", line 282, in process_reads                                                      
ork/Frameworks/Metadata.framework/Versions/A/Supp
    for read1, read2 in self._reader:                                                                                                 
  File "/Applications/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/cutadapt/seqio.py", line 420, in __iter__                                                              
    "in file 1 does not match '{1}' in file 2.".format(r1.name, r2.name))                                                                                                               
cutadapt.seqio.FormatError: Reads are improperly paired. Read name 'M00314:113:000000000-AJW1N:1:1111:23432:12094 1:N:0:1' in file 1 does not match 'M00314:113:000000000-AJW1N:1:1101:1
8027:1086 2:N:0:1' in file 2.                                                                                                                                                           
                                                                                                                                                                                        
cutadapt: error: Reads are improperly paired. Read name 'M00314:113:000000000-AJW1N:1:1111:23432:12094 1:N:0:1' in file 1 does not match 'M00314:113:000000000-AJW1N:1:1101:18027:1086 2
:N:0:1' in file 2.                                                                                                                                                                      
Traceback (most recent call last):                                                                                                                                                      
onda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2cli/commands.py", line 224, in __call__                                                              │                          
    results = action(**arguments)                                                                                                                                                       
  File "<decorator-gen-358>", line 2, in trim_paired                                                                                                                                    
  File "/Applications/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 228, in bound_callable                                                     
    output_types, provenance)                                                                                                                                                           
  File "/Applications/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py", line 363, in _callable_executor_                                                
    output_views = self._callable(**view_args)                                                                                                                                          
  File "/Applications/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_cutadapt/_trim.py", line 172, in trim_paired                                                        
    run_commands(cmds)                                                                                                                                                                  
  File "/Applications/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_cutadapt/_trim.py", line 28, in run_commands                                                        
    subprocess.run(cmd, check=True)                                                                                                                                                     
  File "/Applications/miniconda3/envs/qiime2-2017.12/lib/python3.5/subprocess.py", line 398, in run                                                                                     
    output=stdout, stderr=stderr)                                                                                                                                                       
subprocess.CalledProcessError: Command '['cutadapt', '--cores', '2', '--error-rate', '0.2', '--times', '1', '--overlap', '3', '-o', '/var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/q2
-CasavaOneEightSingleLanePerSampleDirFmt-oi0k6eyg/CH00-0301_62_L001_R1_001.fastq.gz', '-p', '/var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/q2-CasavaOneEightSingleLanePerSampleDirFmt
-oi0k6eyg/CH00-0301_63_L001_R2_001.fastq.gz', '--front', 'GCGGTAATTCCAGCTCCAA', '-G', 'TTGGCAAATGCTTTCGC', '/var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/qiime2-archive-zxavzj1i/59b
9a374-550a-45f7-9467-5b22262183e3/data/CH00-0301_62_L001_R1_001.fastq.gz', '/var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/qiime2-archive-zxavzj1i/59b9a374-550a-45f7-9467-5b22262183e
3/data/CH00-0301_63_L001_R2_001.fastq.gz']' returned non-zero exit status 1                                                                                                             
                                                                                                                                                                                        
Plugin error from cutadapt:                                                                                                                                                             
                                                                                                                                                                                        
  Command '['cutadapt', '--cores', '2', '--error-rate', '0.2', '--times', '1', '--overlap', '3', '-o', '/var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/q2-CasavaOneEightSingleLanePerS
ampleDirFmt-oi0k6eyg/CH00-0301_62_L001_R1_001.fastq.gz', '-p', '/var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/q2-CasavaOneEightSingleLanePerSampleDirFmt-oi0k6eyg/CH00-0301_63_L001_R
2_001.fastq.gz', '--front', 'GCGGTAATTCCAGCTCCAA', '-G', 'TTGGCAAATGCTTTCGC', '/var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/qiime2-archive-zxavzj1i/59b9a374-550a-45f7-9467-5b222621
83e3/data/CH00-0301_62_L001_R1_001.fastq.gz', '/var/folders/jh/3gz6s81d2x7gjd30dcywtqpw0000gn/T/qiime2-archive-zxavzj1i/59b9a374-550a-45f7-9467-5b22262183e3/data/CH00-0301_63_L001_R2_0
01.fastq.gz']' returned non-zero exit status 1                                                                                                                                          
                                                                                                                                                                                        
See above for debug info.     

Hi @Paul_Czechowski

From your first two bullet points, it looks like you intend to use DADA2 for denoising, quality control, and read joining. If that is the case, then you will want to process these runs separately, as DADA2 works best when you process single runs at a time. Please see the FMT Tutorial for an example of processing and merging multiple runs.

As far as the cutadapt error, how are you importing these data? It isn’t clear to me from your post above what data you are processing (can you share your import commands?). The bulleted lists leave a lot to the imagination, there are any number of places where things good go awry!

Let us know what your plans are for analysis (e.g. are you planning on DADA2, or something else?), and keep us posted. Thanks! :t_rex:

Hello, thank you @thermokarst for your swift reply. I will try and import these things separately according to the tutorial, and then visit the tutorial you are mentioning. I think this will be the only way to get this right (efficiently).

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.