I’m having trouble running the align_to_tree_mafft_fasttree pipeline using the artefact API in a jupyter notebook.
Here is a minimal example using the moving pictures example data which demonstrates the problem.
import os
import sys
from qiime2 import Artifact
from qiime2 import Metadata
from qiime2.plugins import dada2
from qiime2.plugins import demux
from qiime2.plugins import feature_table
from qiime2.plugins import phylogeny
os.chdir('/cluster/db/jabbott/qiime2_reanalysis/qiime2-moving-pictures-api')
try:
seqs = Artifact.import_data('EMPSingleEndSequences', 'emp-single-end-sequences',view_type='EMPSingleEndDirFmt')
except qiime2.plugin.ValidationError as e:
print('An error occured during import: %s' % e)
sys.exit(1)
except Exception as e:
print('An unexpected error has occured: %s' % e)
sys.exit(1)
metadata = Metadata.load('sample-metadata.tsv')
demuxed=demux.methods.emp_single(seqs=seqs,barcodes=metadata.get_column('BarcodeSequence'))
denoised=dada2.methods.denoise_single(demultiplexed_seqs=demuxed.per_sample_sequences,
trunc_len=110,trim_left=0,n_threads=1)
phyl=phylogeny.pipelines.align_to_tree_mafft_fasttree(sequences=denoised.representative_sequences)
which when run in a jupyter notebook results in the following being reported
/homes/jabbott/miniconda3/envs/qiime2-2018.11/bin/mafft: line 911: /dev/stderr: Permission denied
/homes/jabbott/miniconda3/envs/qiime2-2018.11/bin/mafft: line 1949: /dev/stderr: Permission denied
along with a stack trace in the jupyter window which has the underlying cause of:
CalledProcessError: Command '['mafft', '--preservecase', '--inputorder', '--thread', '1', '/tmp/3555549.1.m600.q/qiime2-archive-vmm8ryrx/69b619f8-7d85-4362-8aac-05d717bee055/data/dna-sequences.fasta']' returned non-zero exit status 1
If I export the notebook to a script and run it directly under the same environment, it runs correctly, I’m suspecting there is something strange going on with how jupyter is handling stderr.
This is with qiime2-2018.11 under CentOS 6.10, installed via conda. Jobs are run under Univa Grid Engine with jupyter started from a qrsh session, however the jupyter process is running under my uid so this shouldn’t (!) affect things…
Anyone have any ideas of how to work round this?
Many thanks
James