“EMP protocol” multiplexed paired-end fastq file

I’m confused about “EMP protocol” multiplexed paired-end fastq file. Are the barcode sequences in the forward and/or reverse file? If not, how does this software distinguish which sequences belong to which samples? Actually, my data contain a forward file, a reverse file and a sample-metadata file, but no barcode file here. Every barcode sequence is still in the forward and/or reverse file, tegother with my reads sequence…how should I cut away the barcode from my data?

Hi @laoshiren,

Neither actually. There will be a third file (something like Unnamed R3 or Index) which contains only the barcodes, but in the same order as the other reads. That allows QIIME to demultiplex.

It sounds like you need to use cutadapt demux-paired instead as your barcodes are within the reads.
Are your barcodes the same between the two directions (e.g. redundant) or are they combinatoric (often called dual-indexed reads)?

The are combinatoric. They are different between two directions. For example, in one of my data, the forward barcode is GGAATGAC, and the reverse barcode is CGTACGAC.

I know this common, but it seems like it can only demux the paired-end reads only with the forward barcode in the forward reads…But my forward reads have the farward barcode and the reverse reads also have reverse barcode (at the forefront of the reads, not inside)… What can I do?

I’m running ‘qiime cutadapt demux-paired’, but I got some error…
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2cli/commands.py”, line 274, in call
results = action(**arguments)
File “”, line 2, in demux_paired
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 362, in callable_executor
output_views = self._callable(**view_args)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_cutadapt/_demux.py”, line 136, in demux_paired
return _demux(seqs, forward_barcodes, error_rate, untrimmed)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_cutadapt/_demux.py”, line 109, in _demux
run_command(cmd)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_cutadapt/_demux.py”, line 33, in run_command
subprocess.run(cmd, check=True)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/subprocess.py”, line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘cutadapt’, ‘–front’, ‘file:/tmp/tmp0dwk8czi’, ‘–error-rate’, ‘0.1’, ‘-o’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-rkizuefd/{name}.1.fastq.gz’, ‘–untrimmed-output’, ‘/tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-la8mbalc/forward.fastq.gz’, ‘-p’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-rkizuefd/{name}.2.fastq.gz’, ‘–untrimmed-paired-output’, ‘/tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-la8mbalc/reverse.fastq.gz’, ‘/tmp/qiime2-archive-mw6bbrxu/20956bdb-6f4e-4982-b5e7-f2f7b909d336/data/forward.fastq.gz’, ‘/tmp/qiime2-archive-mw6bbrxu/20956bdb-6f4e-4982-b5e7-f2f7b909d336/data/reverse.fastq.gz’]’ returned non-zero exit status 1

Plugin error from cutadapt:

Command ‘[‘cutadapt’, ‘–front’, ‘file:/tmp/tmp0dwk8czi’, ‘–error-rate’, ‘0.1’, ‘-o’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-rkizuefd/{name}.1.fastq.gz’, ‘–untrimmed-output’, ‘/tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-la8mbalc/forward.fastq.gz’, ‘-p’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-rkizuefd/{name}.2.fastq.gz’, ‘–untrimmed-paired-output’, ‘/tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-la8mbalc/reverse.fastq.gz’, ‘/tmp/qiime2-archive-mw6bbrxu/20956bdb-6f4e-4982-b5e7-f2f7b909d336/data/forward.fastq.gz’, ‘/tmp/qiime2-archive-mw6bbrxu/20956bdb-6f4e-4982-b5e7-f2f7b909d336/data/reverse.fastq.gz’]’ returned non-zero exit status 1

See above for debug info.

Hey @laoshiren,

That’s a bummer, and it looks like you ran with --verbose? Was there anything above this traceback (unfortunately the tracebacks in this plugin are not the most helpful).

Also could you post your full command? Thanks!

Sorry scratch that last reply, I've been answering too many questions today and lost track of the thread here!

That is correct, QIIME 2 cannot handle combinatoric/dual-index barcodes. I think bcl2fastq can do this, or you could demultiplex some other way (another user recently used ea-utils for this and some elbow grease).


Once they are demultiplexed, it should be easy to use QIIME 2. Sorry we don't support dual-indexing yet!

Thank you for your reply. Here is my command:
qiime cutadapt demux-paired --i-seqs BarcodeInSequence.qza --m-forward-barcodes-file Forwrad_Map.tsv --m-forward-barcodes-column BarcodeSequence --o-per-sample-sequences per-sample-sequences.qza --o-untrimmed-sequences untrimmed-sequences.qza --verbose
That’s the complete log:
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command: cutadapt --front file:/tmp/tmp7n1ezihi --error-rate 0.1 -o /tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-0qh_kxcc/{name}.1.fastq.gz --untrimmed-output /tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-vpr2b_bp/forward.fastq.gz -p /tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-0qh_kxcc/{name}.2.fastq.gz --untrimmed-paired-output /tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-vpr2b_bp/reverse.fastq.gz /tmp/qiime2-archive-fumzim_f/2c5ca3f3-7e5e-4a47-ae12-691c9c7d598e/data/forward.fastq.gz /tmp/qiime2-archive-fumzim_f/2c5ca3f3-7e5e-4a47-ae12-691c9c7d598e/data/reverse.fastq.gz

This is cutadapt 1.17 with Python 3.5.5
Command line parameters: --front file:/tmp/tmp7n1ezihi --error-rate 0.1 -o /tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-0qh_kxcc/{name}.1.fastq.gz --untrimmed-output /tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-vpr2b_bp/forward.fastq.gz -p /tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-0qh_kxcc/{name}.2.fastq.gz --untrimmed-paired-output /tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-vpr2b_bp/reverse.fastq.gz /tmp/qiime2-archive-fumzim_f/2c5ca3f3-7e5e-4a47-ae12-691c9c7d598e/data/forward.fastq.gz /tmp/qiime2-archive-fumzim_f/2c5ca3f3-7e5e-4a47-ae12-691c9c7d598e/data/reverse.fastq.gz
Running on 1 core
Trimming 56 adapters with at most 10.0% errors in paired-end legacy mode …
cutadapt: error: Reads are improperly paired. Read name ‘FCBP2T9:1:1101:12597:1064#C_NGTCGAAC_CANGTAGT/1’ in file 1 does not match ‘FCBP2T9:1:1101:12597:1064#TNN_NGTCGAAC_CANGTAGT/2’ in file 2.
Traceback (most recent call last):
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2cli/commands.py”, line 274, in call
results = action(**arguments)
File “”, line 2, in demux_paired
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 231, in bound_callable
output_types, provenance)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 362, in callable_executor
output_views = self._callable(**view_args)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_cutadapt/_demux.py”, line 136, in demux_paired
return _demux(seqs, forward_barcodes, error_rate, untrimmed)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_cutadapt/_demux.py”, line 109, in _demux
run_command(cmd)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/site-packages/q2_cutadapt/_demux.py”, line 33, in run_command
subprocess.run(cmd, check=True)
File “/home/qiime2/miniconda/envs/qiime2-2018.8/lib/python3.5/subprocess.py”, line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘cutadapt’, ‘–front’, ‘file:/tmp/tmp7n1ezihi’, ‘–error-rate’, ‘0.1’, ‘-o’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-0qh_kxcc/{name}.1.fastq.gz’, ‘–untrimmed-output’, ‘/tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-vpr2b_bp/forward.fastq.gz’, ‘-p’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-0qh_kxcc/{name}.2.fastq.gz’, ‘–untrimmed-paired-output’, ‘/tmp/q2-MultiplexedPairedEndBarcodeInSequenceDirFmt-vpr2b_bp/reverse.fastq.gz’, ‘/tmp/qiime2-archive-fumzim_f/2c5ca3f3-7e5e-4a47-ae12-691c9c7d598e/data/forward.fastq.gz’, ‘/tmp/qiime2-archive-fumzim_f/2c5ca3f3-7e5e-4a47-ae12-691c9c7d598e/data/reverse.fastq.gz’]’ returned non-zero exit status 1

Hey @laoshiren,

I think this is the combinatoric barcodes (dual indexing) causing the issue. We don't have a mechanism to handle that in QIIME 2 yet so you'll need to demultiplex outside of QIIME 2 and then we can import your data normally.

Sorry for the inconvenience!

I see. Thank you very much.

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.