train classifier

[Sandy_Arguello]

Hello. In the part of "train the classifier" it throws me this error --Plugin error fron feature-classifler: Unable to allocate array with shape (268435456,) and data type float64--. What can be ? Help me please
I use this command:

qiime feature-classifier fit-classifier-naive-bayes
--i-reference-reads ref-seqs.qza
--i-reference-taxonomy silva-138-99-tax.qza
--o-classifier classifier.qza

[timanix (Timur Yergaliyev)]

@Sandy_Arguello

--Plugin error fron feature-classifler: Unable to allocate array with shape (268435456,) and data type float64

Hello!
I suppose that you are running out of memory.
As suggested here, you can try to provide additional parameter --p-classify--chunk-size 20000 or lower to your command to reduce amount of memory required

[timanix (Timur Yergaliyev)]

Hi @Sandy_Arguello !
Here is another tutorial you can check to prepare reference sequences and train a classifier. It is very detailed and clear.

Processing, filtering, and evaluating the SILVA database (and other reference sequence data) with RESCRIPt

Please consider this tutorial a living document, which may change based upon community feedback and ongoing plugin development. RESCRIPt RESCRIPt (REference Sequence annotation and CuRatIon Pipeline) is a python package and QIIME 2 plugin for formatting, managing, and manipulating sequence reference databases. This package was designed for compiling, manipulating, and evaluating sequence reference databases from SILVA, NCBI, Greengenes, GTDB, and other sources, and for construc…

[Sandy_Arguello]

Thanks, I will do it

[Sandy_Arguello]

I installed qiime2 on another computer with 12 gigs of ram, and now it gives me this error "Plugin error from feature-classifier:
Unable to allocate 6.77 GiB for an array with shape (20000, 45450) and data type int64 "
It looks similar to the previous one!
I entered the suggested command and there was no change

[timanix (Timur Yergaliyev)]

Hello!
It is highly possible that 12 gigs is not enough to train a classifier. Could you try to run it on a stronger machine or ask someone to do it for you? I think 32 Gb or 64 will work for you.

[Sandy_Arguello]

Thanks, I'll use another command for taxonomy.

[Nicholas_Bokulich]

Hi @Sandy_Arguello,
It looks like you are trying to train a classifier on the SILVA 138 database. This indeed can be a memory-intensive step — which is why we have already done it for you! You can download the pre-trained classifier and read more here: Data resources — QIIME 2 2021.4.0 documentation

(the tutorial that @timanix pointed you to explains the steps of how that classifier was created)

using the classifier to classify sequences is also memory-intensive, but a little bit less so... you can adjust the reads per batch to reduce memory load while running the classifier (search this forum for some more detailed troubleshooting advice)

Good luck!