cutadapt fails. when removing primers

Huda_Ghori · June 16, 2020, 2:15pm

HI

I am using the command to cut amplicon primers from my sequences:
qiime cutadapt trim-paired
–i-demultiplexed-sequences SM_16S.qza
–p-cores 8
–p-front-f CCTACGGGNGGCWGCAG
–p-front-r GACTACHVGGGTATCTAATCC
–o-trimmed-sequences primer-trimmed-SM_16S.qza
–verbose

my job fails and gives the following out put

[8<----------] 00:00:01 35,585 reads @ 29.9 µs/read; 2.00 M reads/minuteTraceback (most recent call last):
File “/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2cli/commands.py”, line 328, in call
results = action(**arguments)
File “</home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/decorator.py:decorator-gen-461>”, line 2, in trim_paired
File “/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 245, in bound_callable
output_types, provenance)
File “/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py”, line 390, in callable_executor
output_views = self._callable(**view_args)
File “/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_cutadapt/_trim.py”, line 189, in trim_paired
run_commands(cmds)
File “/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_cutadapt/_trim.py”, line 30, in run_commands
subprocess.run(cmd, check=True)
File “/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/subprocess.py”, line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command ‘[‘cutadapt’, ‘–cores’, ‘8’, ‘–error-rate’, ‘0.1’, ‘–times’, ‘1’, ‘–overlap’, ‘3’, ‘–minimum-length’, ‘1’, ‘-o’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-3bs4gjhm/134-ear-D7_S135_L001_R1_001.fastq.gz’, ‘-p’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-3bs4gjhm/134-ear-D7_S135_L001_R2_001.fastq.gz’, ‘–front’, ‘CCTACGGGNGGCWGCAG’, ‘-G’, ‘GACTACHVGGGTATCTAATCC’, ‘/tmp/qiime2-archive-xc0bgpmp/f8f3e18e-ab62-4705-9000-3b1d56570a3b/data/134-ear-D7_S135_L001_R1_001.fastq.gz’, ‘/tmp/qiime2-archive-xc0bgpmp/f8f3e18e-ab62-4705-9000-3b1d56570a3b/data/134-ear-D7_S135_L001_R2_001.fastq.gz’]’ returned non-zero exit status 1.

Plugin error from cutadapt:

Command ‘[‘cutadapt’, ‘–cores’, ‘8’, ‘–error-rate’, ‘0.1’, ‘–times’, ‘1’, ‘–overlap’, ‘3’, ‘–minimum-length’, ‘1’, ‘-o’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-3bs4gjhm/134-ear-D7_S135_L001_R1_001.fastq.gz’, ‘-p’, ‘/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-3bs4gjhm/134-ear-D7_S135_L001_R2_001.fastq.gz’, ‘–front’, ‘CCTACGGGNGGCWGCAG’, ‘-G’, ‘GACTACHVGGGTATCTAATCC’, ‘/tmp/qiime2-archive-xc0bgpmp/f8f3e18e-ab62-4705-9000-3b1d56570a3b/data/134-ear-D7_S135_L001_R1_001.fastq.gz’, ‘/tmp/qiime2-archive-xc0bgpmp/f8f3e18e-ab62-4705-9000-3b1d56570a3b/data/134-ear-D7_S135_L001_R2_001.fastq.gz’]’ returned non-zero exit status 1.

See above for debug info.

I thought maybe my sequences did not import correctly so I imported it again but I am still getting the same output. I also validated my file and it says that the file is valid.

I do not understand the error.

Please help

sixvable · June 1, 2020, 7:41am

Hi @Huda_Ghori

Could you try this command to test if your seqeuences have been imported correct.

qiime demux summarize \
--i-data SM_16S.qza \
--o-visualization SM_16S.qzv

sixvable

thermokarst · June 1, 2020, 3:00pm

Hi @Huda_Ghori! Is there any more verbose output available? It looks like we are missing the first half of the log - can you please attach here - thanks!

PS - the first half of the cutadapt starts with a line or two like this:

This is cutadapt 2.8 with Python 3.6.7

Huda_Ghori · June 3, 2020, 2:29am

demux.qzv (337.4 KB)

HI thankyou for replying

this is my output from demux summarize command

thermokarst · June 3, 2020, 2:15pm

Hi @Huda_Ghori - I’m not really sure what @sixvable was asking for, but I am looking for the full log when running qiime cutadapt trim-paired with the --verbose flag. Thanks!

Huda_Ghori · June 9, 2020, 3:15pm

Hi @thermokarst

Here is the output

This is cutadapt 2.8 with Python 3.6.7
Command line parameters: --cores 8 --error-rate 0.1 --times 1 --overlap 3 --minimum-length 1 -o /tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-5d_0aq2w/132-gro-D7_S30_L001_R1_001.fastq.gz -p /tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-5d_0aq2w/132-gro-D7_S30_L001_R2_001.fastq.gz --front CCTACGGGNGGCWGCAG -G GACTACHVGGGTATCTAATCC /tmp/qiime2-archive-jqv7rlzr/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R1_001.fastq.gz /tmp/qiime2-archive-jqv7rlzr/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R2_001.fastq.gz
Processing reads on 8 cores in paired-end mode ...
Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2cli/commands.py", line 328, in __call__
    results = action(**arguments)
  File "</home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/decorator.py:decorator-gen-461>", line 2, in trim_paired
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_cutadapt/_trim.py", line 189, in trim_paired
    run_commands(cmds)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_cutadapt/_trim.py", line 30, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['cutadapt', '--cores', '8', '--error-rate', '0.1', '--times', '1', '--overlap', '3', '--minimum-length', '1', '-o', '/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-5d_0aq2w/132-gro-D7_S30_L001_R1_001.fastq.gz', '-p', '/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-5d_0aq2w/132-gro-D7_S30_L001_R2_001.fastq.gz', '--front', 'CCTACGGGNGGCWGCAG', '-G', 'GACTACHVGGGTATCTAATCC', '/tmp/qiime2-archive-jqv7rlzr/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R1_001.fastq.gz', '/tmp/qiime2-archive-jqv7rlzr/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R2_001.fastq.gz']' returned non-zero exit status 1.

Plugin error from cutadapt:

  Command '['cutadapt', '--cores', '8', '--error-rate', '0.1', '--times', '1', '--overlap', '3', '--minimum-length', '1', '-o', '/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-5d_0aq2w/132-gro-D7_S30_L001_R1_001.fastq.gz', '-p', '/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-5d_0aq2w/132-gro-D7_S30_L001_R2_001.fastq.gz', '--front', 'CCTACGGGNGGCWGCAG', '-G', 'GACTACHVGGGTATCTAATCC', '/tmp/qiime2-archive-jqv7rlzr/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R1_001.fastq.gz', '/tmp/qiime2-archive-jqv7rlzr/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R2_001.fastq.gz']' returned non-zero exit status 1.

See above for debug info.

thermokarst · June 9, 2020, 2:09pm

thermokarst · June 9, 2020, 3:17pm

Great, thanks for that, @Huda_Ghori!

It looks like maybe cutadapt is suffering from some kind of catastrophic failure - is it possible that you are specifying more cores than you have available on your system? What happens when you run with 4 (or 2)?

Keep us posted!

thermokarst · June 9, 2020, 3:17pm

Huda_Ghori · June 16, 2020, 2:12pm

Hi @thermokarst

I ran it again with 2 cores and it showed the same error :

This is cutadapt 2.8 with Python 3.6.7
Command line parameters: --cores 2 --error-rate 0.1 --times 1 --overlap 3 --minimum-length 1 -o /tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-r61gkbtw/132-gro-D7_S30_L001_R1_001.fastq.gz -p /tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-r61gkbtw/132-gro-D7_S30_L001_R2_001.fastq.gz --front CCTACGGGNGGCWGCAG -G GACTACHVGGGTATCTAATCC /tmp/qiime2-archive-l4uyj3tz/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R1_001.fastq.gz /tmp/qiime2-archive-l4uyj3tz/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R2_001.fastq.gz
Processing reads on 2 cores in paired-end mode ...
Traceback (most recent call last):
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2cli/commands.py", line 328, in __call__
    results = action(**arguments)
  File "</home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/decorator.py:decorator-gen-461>", line 2, in trim_paired
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 245, in bound_callable
    output_types, provenance)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/qiime2/sdk/action.py", line 390, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_cutadapt/_trim.py", line 189, in trim_paired
    run_commands(cmds)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/site-packages/q2_cutadapt/_trim.py", line 30, in run_commands
    subprocess.run(cmd, check=True)
  File "/home/ubuntu/miniconda3/envs/qiime2-2020.2/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['cutadapt', '--cores', '2', '--error-rate', '0.1', '--times', '1', '--overlap', '3', '--minimum-length', '1', '-o', '/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-r61gkbtw/132-gro-D7_S30_L001_R1_001.fastq.gz', '-p', '/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-r61gkbtw/132-gro-D7_S30_L001_R2_001.fastq.gz', '--front', 'CCTACGGGNGGCWGCAG', '-G', 'GACTACHVGGGTATCTAATCC', '/tmp/qiime2-archive-l4uyj3tz/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R1_001.fastq.gz', '/tmp/qiime2-archive-l4uyj3tz/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R2_001.fastq.gz']' returned non-zero exit status 1.

Plugin error from cutadapt:

  Command '['cutadapt', '--cores', '2', '--error-rate', '0.1', '--times', '1', '--overlap', '3', '--minimum-length', '1', '-o', '/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-r61gkbtw/132-gro-D7_S30_L001_R1_001.fastq.gz', '-p', '/tmp/q2-CasavaOneEightSingleLanePerSampleDirFmt-r61gkbtw/132-gro-D7_S30_L001_R2_001.fastq.gz', '--front', 'CCTACGGGNGGCWGCAG', '-G', 'GACTACHVGGGTATCTAATCC', '/tmp/qiime2-archive-l4uyj3tz/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R1_001.fastq.gz', '/tmp/qiime2-archive-l4uyj3tz/499a18b3-a677-43fd-aac3-4f92f92f894f/data/132-gro-D7_S30_L001_R2_001.fastq.gz']' returned non-zero exit status 1.

See above for debug info.

thermokarst · June 16, 2020, 2:15pm

I’m still not sure what the issue is here - cutadapt is failing catastrophically, with no traceback or error message (which is unusual for cutadapt). Would you be willing to share your files with me (you can send a download link to me in a direct message)? That would help with diagnosing if the issue is related to your computation environment, the data, or something else entirely.

Thanks!