Hi! When I used q2-feature-classifier to train a classifier, I found an error: Unable to allocate array with shape (20000, 5391) and data type int64. Does anyone know the causation and how to solve this problem? Thanks very much!
1 Like
Hello @Zhanzhan,
Welcome to the Qiime 2 forums! 
The "Unable to allocate array" error means you don't have enough memory / RAM on your computer or in your virtual box to train the classifier.
Training a taxonomic classifier takes a lot of RAM! How much RAM / memory do you have now? How large is your database?
Colin
2 Likes
Hello, @colinbrislawn!
Thank you for your reply! I used greegene database: 99_otus.fasta and 99_otu_taxonomy.txt to train a classifier. Here is my code to produce a V4 ref-seqs.qza: qiime feature-classifier extract-reads \
--i-sequences 99_otus.qza
--p-f-primer GTGTGYCAGCMGCCGCGGTAA
--p-r-primer CCGGACTACNVGGGTWTCTAAT
--p-min-length 100
--p-max-length 400
--o-reads ref-seqs.qza
and then I wanted to use ref-seqs.qza and ref-taxonomy.qza(99_otu_taxonomy.txt) to produce a classifier, but failed. My code:
qiime feature-classifier fit-classifier-naive-bayes
--i-reference-reads ref-seqs.qza
--i-reference-taxonomy ref-taxonomy.qza
--o-classifier classifier.qza
Plugin error from feature-classifier:
Unable to allocate array with shape (20000, 5391) and data type int64
My ram is so small and it's just 2G. And I want to update this device. Could you please give me advice about how much ram memory I need for training a classifier like this?
Ps:my biggest dataset is about 20G,130 samples, and this demands how much ram memory and how much disk memory? Thank you very much!
1 Like
Hello again,
I've edited your post a little bit. When you do a quote with > you need one extra line after to show the next text starts a new paragraph.
Back to Qiime 2 --> 
Thank you for telling me more. How large (GB or MB) is 99_otus.fasta? That is the main limitation with training a classifier.
My ram is so small and it’s just 2G. And I want to update this device. Could you please give me advice about how much ram memory I need for training a classifier like this?
Memory usage is hard to estimate because it depends on the size and complexity of the database. 
If you only have a small amount of memory, the best option is to use a pre-trained classifier. We got some good ones here: Data resources — QIIME 2 2023.9.2 documentation
Training a classifier takes a lot of RAM, but analyzing samples does not need as much RAM. If you could find a computer or VM with 8 GB of RAM should work well.
Thank you! 
Colin
1 Like
@colinbrislawn 
Thank you for your advice very much！Thanks for your editing!
The 99_otus.fasta is 270MB and after extraction and gzip, the 99-v4-ref-seqs.qza is about 8.82MB. The 99_otu_taxonomy.txt is 20.43MB and after importing qiime2, the ref-taxonomy.qza is 2.49MB. And how much ram memory does this demand? Thank you!
1 Like
Because 99_otus.fasta is only 270MB, I think 2 GB of RAM would be enough… but it looks like you do need more. Could you find a computer with 8 GB and try that?
Have you tried using any of the pre-trained classifiers?
Colin
@colinbrislawn Thank you Colin! I want to rent a 8GB ram ECS to analyze it and I used a V4 pre-classifier to perform taxonomy annotation and I got the results. But I found no significant differential abundance between two groups. Thank you again for your sincere advice!
1 Like
Thank you so much!
Feel free to open a new thread if you have more questions.
Colin
@colinbrislawn Thank you!
A post was split to a new topic: Virtual Box. Memory allocation error