q2-picrust2 error in running full pipeline on qiime2-2019.4

I tried running dada2 output in q2-picrust2 plugin installed via conda in qiime2-2019.4 environment installed natively via conda on a system with 8 Gb ram and using 3 threads, there are about ~20,000 ASVs. The command runs completely (when in quiet mode) but no output directory is generated. Detailed error with verbose is given below, hsp method used mp.

(qiime2-2019.4) chandu@chandu-Inspiron-N4010:~$ qiime picrust2 full-pipeline --i-table /home/chandu/NPDF_qiime_z840/resultdada_f260_r220/table.qza --i-seq /home/chandu/NPDF_qiime_z840/resultdada_f260_r220/representative_sequences.qza --p-threads 3 --p-hsp-method mp --p-max-nsti 2 --o-ko-metagenome ko_dada2 --o-ec-metagenome ec_dada2 --o-pathway-abundance pathway_dada2 --verbose

20591 of 20591 sequence ids overlap between input table and FASTA.

Placing sequences onto reference tree
place_seqs.py --study_fasta /tmp/tmpb_e1cqb2/seqs.fna --ref_dir /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref --out_tree /tmp/tmpb_e1cqb2/picrust2_out/out.tre --threads 3 --intermediate /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs --chunk_size 5000 --print_cmds

Error running this command:
place_seqs.py --study_fasta /tmp/tmpb_e1cqb2/seqs.fna --ref_dir /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref --out_tree /tmp/tmpb_e1cqb2/picrust2_out/out.tre --threads 3 --intermediate /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs --chunk_size 5000 --print_cmds

Standard output of failed command:
“”

Standard error of failed command:
"hmmalign --trim --dna --mapali /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref/pro_ref.fna.gz --informat FASTA -o /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/query_align.stockholm /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref/pro_ref.hmm /tmp/tmpb_e1cqb2/seqs.fna
epa-ng --tree /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref/pro_ref.tre --ref-msa /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/ref_seqs_hmmalign.fasta --query /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/study_seqs_hmmalign.fasta --chunk-size 5000 -T 3 -m /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref/pro_ref.model -w /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/epa_out --filter-acc-lwr 0.5 --filter-max 20

Error running this command:
epa-ng --tree /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref/pro_ref.tre --ref-msa /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/ref_seqs_hmmalign.fasta --query /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/study_seqs_hmmalign.fasta --chunk-size 5000 -T 3 -m /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref/pro_ref.model -w /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/epa_out --filter-acc-lwr 0.5 --filter-max 20

Standard output of failed command:
"INFO Selected: Output dir: /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/epa_out/
INFO Selected: Query file: /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/study_seqs_hmmalign.fasta
INFO Selected: Tree file: /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref/pro_ref.tre
INFO Selected: Reference MSA: /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/ref_seqs_hmmalign.fasta
INFO Selected: Filtering by accumulated threshold: 0.5
INFO Selected: Maximum number of placements per query: 20
INFO Selected: Automatic switching of use of per rate scalers
INFO Selected: Preserving the root of the input tree
INFO Selected: Specified model file: /home/chandu/miniconda2/envs/qiime2-2019.4/lib/python3.6/site-packages/picrust2/default_files/prokaryotic/pro_ref/pro_ref.model
INFO Rate heterogeneity: GAMMA (4 cats, mean), alpha: 0.453141 (user), weights&rates: (0.25,0.0250674) (0.25,0.220229) (0.25,0.782933) (0.25,2.97177)
Base frequencies (user): 0.229585 0.22008 0.298596 0.251739
Substitution rates (user): 1.00319 2.79077 1.5301 0.87441 3.83966 1
INFO Selected: Reading queries in chunks of: 5000
INFO Selected: Using threads: 3
INFO ______ ____ ___ _ __ ______
/ // __ \ / | / | / // /
/ __/ / /
/ // /| | ______ / |/ // / __
/ /
/ // ___ |/_____// /| // // /
/_____//
/ /
/ |
| /
/ |
/ _
_/ (v0.3.5)
INFO Output file: /tmp/tmpb_e1cqb2/picrust2_out/intermediate/place_seqs/epa_out/epa_result.jplace
"

Standard error of failed command:

Maybe @gmdouglas can help out here? :qiime2:

Hey @Chandrashekar_M,

EPA-NG takes ~15GB of RAM to run with the default PICRUSt2 reference file. To still use the plugin you can use the alternative read placement step mentioned as the second option here: https://github.com/picrust/picrust2/wiki/q2-picrust2-Tutorial

Best,

Gavin

1 Like