Alignment error message

Hi there,
I have been trying to align my sequences by running the maaft command. It crops up with this error.

(qiime2-2017.11) qiime2@qiime2core2017-11:~/gbstrial2$ qiime alignment mafft \
>   --i-sequences rep-seqs.qza \
>   --o-alignment aligned-rep-seqs.qza
Plugin error from alignment:

  Command '['mafft', '--preservecase', '--inputorder', '--thread', '1', '/tmp/qiime2-archive-zibb0xaz/25fd7c9d-1d26-460b-82e0-b9b3b148d570/data/dna-sequences.fasta']' returned non-zero exit status 1

Debug info has been saved to /tmp/qiime2-q2cli-err-o2fknrg1.log

I double checked that my rep.seqs.qzv looked ok, which it did. I freed up some space on my laptop to check whether that was a problem and then attempted to run it again and got this error message. What can I do?

(qiime2-2017.11) qiime2@qiime2core2017-11:~/gbstrial2$ qiime alignment mafft \
> >   --i-sequences rep-seqs.qza \
> >   --o-alignment aligned-rep-seqs.qza

Usage: qiime alignment mafft [OPTIONS]

Error: Got unexpected extra arguments (rep-seqs.qza aligned-rep-seqs.qza)

Thank you!

1 Like

Hi @coralgal - can you please re-run that first command, and include the --verbose flag?

$ (qiime2-2017.11) qiime2@qiime2core2017-11:~/gbstrial2$ qiime alignment mafft \
  --i-sequences rep-seqs.qza \
  --o-alignment aligned-rep-seqs.qza \
  --verbose

It looks like when you copied-and-pasted your command to rerun, you picked up some other characters:

This is causing your command line shell to get confused (specifically, it is trying to redirect all of the output into a file named --i-sequences, which is not what you are trying to do...). In general, when writing out command line commands, you want to follow the old adage of "measure twice; cut once" --- double check that everything you have entered is actually what you meant to type.

Once we get a chance to peek at that error log we can come up with additional support suggestions!

Hey thermokast,
Thanks for your response!
Here is the error log:

(qiime2-2017.11) qiime2@qiime2core2017-11:~/gbstrial2$ qiime alignment mafft \
>   --i-sequences rep-seqs.qza \
>   --o-alignment aligned-rep-seqs.qza \
>   --verbose
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command: mafft --preservecase --inputorder --thread 1 /tmp/qiime2-archive-ty8b6q90/25fd7c9d-1d26-460b-82e0-b9b3b148d570/data/dna-sequences.fasta


inputfile = orig
27654 x 544 - 250 d
nthread = 1
stacksize: 8192 kb
generating a scoring matrix for nucleotide (dist=200) ... done
Gap Penalty = -1.53, +0.00, +0.00



Making a distance matrix ..
/home/qiime2/miniconda/envs/qiime2-2017.11/bin/mafft: line 2440:  3698 Killed                  "$prefix/disttbfast" -q $npickup -E $cycledisttbfast -V "-"$gopdist -s $unalignlevel $legacygapopt $mergearg -W $tuplesize $termgapopt $outnum $addarg $add2ndhalfarg -C $numthreadstb $memopt $weightopt $treeinopt $treeoutopt $distoutopt $seqtype $model -f "-"$gop -Q $spfactor -h $aof $param_fft $algopt $treealg $scoreoutarg < infile > pre 2>> "$progressfile"
Traceback (most recent call last):
  File "/home/qiime2/miniconda/envs/qiime2-2017.11/lib/python3.5/site-packages/q2cli/commands.py", line 218, in __call__
    results = action(**arguments)
  File "<decorator-gen-362>", line 2, in mafft
  File "/home/qiime2/miniconda/envs/qiime2-2017.11/lib/python3.5/site-packages/qiime2/sdk/action.py", line 220, in bound_callable
    output_types, provenance)
  File "/home/qiime2/miniconda/envs/qiime2-2017.11/lib/python3.5/site-packages/qiime2/sdk/action.py", line 355, in _callable_executor_
    output_views = self._callable(**view_args)
  File "/home/qiime2/miniconda/envs/qiime2-2017.11/lib/python3.5/site-packages/q2_alignment/_mafft.py", line 61, in mafft
    run_command(cmd, aligned_fp)
  File "/home/qiime2/miniconda/envs/qiime2-2017.11/lib/python3.5/site-packages/q2_alignment/_mafft.py", line 27, in run_command
    subprocess.run(cmd, stdout=output_f, check=True)
  File "/home/qiime2/miniconda/envs/qiime2-2017.11/lib/python3.5/subprocess.py", line 398, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['mafft', '--preservecase', '--inputorder', '--thread', '1', '/tmp/qiime2-archive-ty8b6q90/25fd7c9d-1d26-460b-82e0-b9b3b148d570/data/dna-sequences.fasta']' returned non-zero exit status 1

Plugin error from alignment:

  Command '['mafft', '--preservecase', '--inputorder', '--thread', '1', '/tmp/qiime2-archive-ty8b6q90/25fd7c9d-1d26-460b-82e0-b9b3b148d570/data/dna-sequences.fasta']' returned non-zero exit status 1

See above for debug info.

Cheers!

1 Like

Thanks @coralgal! This error looks like mafft is failing, but there isn’t any immediately useful information regarding why :frowning:. I don’t think we have seen this error before, and I didn’t see anything on the mafft changelog about this - would you be able to share your file rep-seqs.qza, that way one of us could try to reproduce this locally?

Hey there thermokast,

Here is a copy of the file

rep-seqs.qza (2.5 MB)

Cheers!

1 Like

Thanks for sharing @coralgal! This runs as expected on my machine --- I wonder if you are running out of memory while trying to perform this alignment? It looks like you might be using a QIIME 2 VirtualBox image --- how much memory is allocated on the machine?

If you have more available system memory, maybe you can send some more towards your VM's base memory? In the picture above, my VM has 2048 MB allocated toward base memory, but my system has up to 16384 MB available. VirtualBox recommends I could set this value up to ~12000 MB before running into issues on my host system (this is the green bar in the image). Maybe you can bump yours up a bit more?

Hello, Matthew.

I’m having the same problem with mafft as @coralgal and I tried bumping the memory used by the VM from 2084 MB to 4000 MB and mafft still returns the same problem. Should I try increasing the memory more (I can increase it around 5 GB before running into problems) or could I use another approach?

Hi @PabloCL — I think your only option is to increase the available memory. Are you running on a Mac or Linux machine? If so, you could try to do a native install, this will give you the most available memory. Another option is to use a cloud service, like AWS. Keep us posted! :t_rex:

Hello, Matthew.

I’m having the same problem, but in my case I tried to run mafft in a workstation with 128GB of RAM, I thought that ir’s enought to run it, but not.
Is it possible that error is caused by other issue?
Thanks in advance

Hi @cesteban - it certainly is possible — we would need to see the following info to start tracking down your problem, though:

  • What version of QIIME 2 are you currently running?
  • What is the exact command you are running? Copy and paste please.
  • What is the exact error you are seeing? Please copy and paste the results when run with the --verbose flag, or, attach the error log file reported at the end of the error message when run without the --verbose flag.

Thanks! :t_rex:

1 Like

Hello Matthew,
I updated to version qiime2-2018.2 and I got a similar error.
The commando that I used is:
qiime alignment mafft --p-n-threads 24 --i-sequences 0.mergeALLreads_gg_rep_set.fasta.qza --o-alignment 2.mergeALLreads_gg_rep_set_aligned.fasta.qza --verbose

Here, i attach the log file:

alignmentMAFFT_20180219_1136.log.txt (2.9 MB)

Thanks in advance

Thanks @cesteban! This is a tricky one - we aren’t the developers of MAFFT, so we aren’t experts on how it runs (and more specifically, how it fails!), so it might take us some time to understand what is happening. In the meantime, perhaps the MAFFT team has a support venue you can contact? As well, can you please send along your data - we might be able to experiment with some of the runtime options MAFFT exposes for low-mem environments. Feel free to send a link in a DM to myself, or any other Forum Admin. Thanks! :t_rex:

Hi Matthew,

Any update about this issue?

I am facing same problem with my study. I use a VM 2018.02, allocating 14Gb.

I have run:

qiime2-5_Chapter_1_the_Ithaca_experiment$ qiime alignment mafft
--i-sequences ref_hit_seq_ithaca.qza
--o-alignment 3_aligned-rep-seqs.qza
--p-n-threads -1
--verbose

Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.
Command: mafft --preservecase --inputorder --thread -1 /tmp/qiime2-archive-anck6zw_/d88b53b5-c8ca-4118-b4c6-fe65b78c5c69/data/dna-sequences.fasta
OS = linux
The number of physical cores = 1
inputfile = orig
67357 x 90 - 90 d
nthread = 1
stacksize: 8192 kb->13155 kb
generating a scoring matrix for nucleotide (dist=200) ... done
Gap Penalty = -1.53, +0.00, +0.00
Making a distance matrix ..
14001 / 67357 (thread 0)/home/qiime2/miniconda/envs/qiime2-2018.2/bin/mafft: line 2440: 2819 Killed "$prefix/disttbfast" -q $npickup -E $cycledisttbfast -V "-"$gopdist -s $unalignlevel $legacygapopt $mergearg -W $tuplesize $termgapopt $outnum $addarg $add2ndhalfarg -C $numthreadstb $memopt $weightopt $treeinopt $treeoutopt $distoutopt $seqtype $model -f "-"$gop -Q $spfactor -h $aof $param_fft $algopt $treealg $scoreoutarg < infile
pre 2>> "$progressfile"
Plugin error from alignment:
Command '['mafft', '--preservecase', '--inputorder', '--thread', '-1', '/tmp/qiime2-archive-anck6zw_/d88b53b5-c8ca-4118-b4c6-fe65b78c5c69/data/dna-sequences.fasta']' returned non-zero exit status 1

My rep-seq file is 705.3 Kb.

Thanks in advace!

Juanma

Hi @juanma!

Please see my comment above.

Dear @thermokarst

I am wondering whether you have found some solution regarding previous discussion? I have also got same type of error while doing alignment which is as follows;
qiime alignment mafft --i-sequences …/rep-seqs-dn-99.qza --o-alignment aligned_mafft-ref-seqs.qza --verbose --p-n-threads 24
Running external command line application. This may print messages to stdout and/or stderr.
The command being run is below. This command cannot be manually re-run as it will depend on temporary files that no longer exist.

Command: mafft --preservecase --inputorder --thread 24 /tmp/qiime2-archive-vclzld9_/74cea942-b3a9-4bc2-9b97-65d7cc42d988/data/dna-sequences.fasta

inputfile = orig
230857 x 580 - 200 d
nthread = 24
stacksize: 8192 kb->45089 kb
generating a scoring matrix for nucleotide (dist=200) … done
Gap Penalty = -1.53, +0.00, +0.00

Making a distance matrix …
18701 / 230857 (thread 23)/home/bxa015/miniconda3/envs/qiime2-2017.12/bin/mafft: line 2440: 10107 Killed “$prefix/disttbfast” -q $npickup -E $cycledisttbfast -V “-”$gopdist -s $unalignlevel $legacygapopt $mergearg -W $tuplesize $termgapopt
$outnum $addarg $add2ndhalfarg -C $numthreadstb $memopt $weightopt $treeinopt $treeoutopt $distoutopt $seqtype $model -f “-”$gop -Q $spfactor -h $aof $param_fft $algopt $treealg $scoreoutarg < infile > pre 2>> “$progressfile”
Traceback (most recent call last):
File “/home/bxa015/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2cli/commands.py”, line 224, in call
results = action(**arguments)
File “”, line 2, in mafft
File “/home/bxa015/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 228, in bound_callable
output_types, provenance)
File “/home/bxa015/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/qiime2/sdk/action.py”, line 363, in callable_executor
output_views = self._callable(**view_args)
File “/home/bxa015/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_alignment/_mafft.py”, line 61, in mafft
run_command(cmd, aligned_fp)
File "/home/bxa015/miniconda3/envs/qiime2-2017.12/lib/python3.5/site-packages/q2_alignment/mafft.py", line 27, in run_command
subprocess.run(cmd, stdout=output_f, check=True)
File “/home/bxa015/miniconda3/envs/qiime2-2017.12/lib/python3.5/subprocess.py”, line 398, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '[‘mafft’, ‘–preservecase’, ‘–inputorder’, ‘–thread’, ‘24’, '/tmp/qiime2-archive-vclzld9
/74cea942-b3a9-4bc2-9b97-65d7cc42d988/data/dna-sequences.fasta’]’ returned non-zero exit status 1

Plugin error from alignment:

Command ‘[‘mafft’, ‘–preservecase’, ‘–inputorder’, ‘–thread’, ‘24’, ‘/tmp/qiime2-archive-vclzld9_/74cea942-b3a9-4bc2-9b97-65d7cc42d988/data/dna-sequences.fasta’]’ returned non-zero exit status 1

Hi @badhikari (and others in the thread),

I don't think we have any progress to report.

One thing I did notice in your filename:

Were these sequences picked by closed-reference by chance? If so, the reference database almost certainly has a much better phylogeny than what you are going to get from MAFFT+FastTree.

I'll also plug @Stefan's plugin q2-fragment-insertion which could also be a good way to go.

At the end of the day, the goal is usually to get some kind of reasonable phylogeny (Phylogeny[Rooted]), so if MAFFT+FastTree isn't working, other strategies may work better (and be more meaningful).

Hope that helps!