Feature table summarize running long on large files

feature-table

(Emily Junkins) #1

Hi all,
I am running the following command on an originally 82M biom file converted to a 29M .qza feature table.

qiime feature-table summarize --i-table feature-table-allanimals.qza --o-visualization feature-table-allanimals.qzv --m-sample-metadata-file allanimals_mapping.txt

This has been taking quite some time. Is this to be expected?


(Nicholas Bokulich) #2

(Matthew Ryan Dillon) #3

What version of QIIME 2 are you running? We fixed a bug earlier this year with this visualizer that was causing it to run ridiculously slow. :t_rex:


(Matthew Ryan Dillon) #4

(Emily Junkins) #5

I am running version 2018.6.


(Matthew Ryan Dillon) #6

Hmm — that should have the latest fix then. Is it still running? If so, how long has it been running?


(Emily Junkins) #7

I have just killed it, but I let it run 3-4 days. I was having similar issues with summarize_taxa.py from Qiime1 (i was curious how it would handle it). I will try to run it on one of our larger servers, though I don’t see why the current workstation (64GB RAM) or the Qiime1 server (128 GB RAM) would be having problems.


(Emily Junkins) #8

I have just killed it, but I let it run 3-4 days. I was having similar issues with summarize_taxa.py from Qiime1 (i was curious how it would handle it). I will try to run it on one of our larger servers, though I don’t see why the current workstation (64GB RAM) or the Qiime1 server (128 GB RAM) would be having problems.


(Matthew Ryan Dillon) #9

Weird — would you mind sharing a link to the feature table in a a DM to me? I would like to investigate further…

:t_rex:


(Emily Junkins) #10

Here is a google link to this feature table https://drive.google.com/file/d/1Jxf7eWobSCc83W7X6KFxCaS5vT6MfQ8w/view?usp=sharing


(Matthew Ryan Dillon) #11

(Matthew Ryan Dillon) #12

Thanks @ejunkins! This ran for me in about 5 minutes on my laptop… One critical difference though is you didn’t provide metadata to me for testing, so I ran the following:

qiime feature-table summarize \
  --i-table feature-table-allanimals.qza \
  --o-visualization feature-table-allanimals.qzv

Does that run for you? If so, can you send me your metadata, too? Thanks! :qiime2: :t_rex:


(Matthew Ryan Dillon) #13