# Memory Error using the pretrained silva 119 classifier
**URL:** https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956
**Category:** Silva
**Tags:** feature-classifier, pre-trained, silva
**Created:** [February 9, 2018, 2:31pm UTC](https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956 "2018-02-09T14:31:45Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@Vixer](https://forum.qiime2.org/u/Vixer)
#### Post date: [February 9, 2018, 2:31pm UTC](https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956/1 "2018-02-09T14:31:45Z")
Hello again!
I´m testing my reads using the pre-trained classifier "Silva 119 99% OTUs from 515F/806R region of sequences" since my primers are seem to work with it (f: GTGCCAGCMGCCGCGGTAA & r: GGACTACVSGGGTATCTAAT) and after few minutes into the process i get an error saying Memory Error. Then I use the "Greengenes 13\_8 99% OTUs from 515F/806R region of sequences" and the process runs just fine.
Is the error caused because of the size of the Silva file and is there a way to fix this error (I´m running qiime2 2017.11 in a virtual machine with 3 gigs RAM assigned to it, the computer has 8g of RAM).
---
### Author:  [@jairideout](https://forum.qiime2.org/u/jairideout)
#### Post date: [February 9, 2018, 5:35pm UTC](https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956/3 "2018-02-09T17:35:24Z")
Hi @Vixer! Take a look at [@BenKaehler's post](https://forum.qiime2.org/t/using-the-silva-pre-trained-classifier-consumes-more-than-256-gb-of-memory/1199/16) for details about pre-trained Silva classifier memory requirements, as well as some options you can try to reduce the memory requirements. It appears that the pre-trained Silva classifiers use a maximum of 11GB memory.
My hunch is that you'll need to either allocate more memory to your virtual machine, or run the command on a computer with more memory -- different chunk sizes and/or single-threaded mode will likely not work if you're limited to 3GB RAM. Thanks!
---
### Author:  [@Vixer](https://forum.qiime2.org/u/Vixer)
#### Post date: [February 12, 2018, 3:13pm UTC](https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956/5 "2018-02-12T15:13:20Z")
Thanks for asnwering. I uptaded to the lastest qiime2 version and allocated more RAM (now 4GB) and looking at the options; is the command --p-reads-per-batch the same as the --p-chunk-size in the one in the post you just linked (because with the --help command it doesnt appear any option named chunk-size)?
If so, I changed it from default to 1000 and it just took a little longer to send the error message. Also, can you give me a quick explanation about single-threaded mode and how modify it?
I´m gonna try the command on a computer with 16GB for now, I´ll post if something happens
Thanks!
---
### Author:  [@jairideout](https://forum.qiime2.org/u/jairideout)
#### Post date: [February 12, 2018, 5:17pm UTC](https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956/7 "2018-02-12T17:17:08Z")
The short answer is that 4GB RAM doesn't appear to be enough memory for the command to complete, either in single-job or multi-job mode. My guess is that you're running the command without using multiple jobs (which is the default behavior), so setting a different chunk size / reads-per-batch won't have any effect. I think you'll have better luck with the 16GB RAM computer. Let us know how it goes!
See below for specific answers to your questions -- apologies that my previous post wasn't clear about single vs multiple jobs and chunk size (looks like I led you on a bit of a wild goose chase!).
**Note:** I was using the word "single-threaded" in my previous post, but `classify-sklearn` actually uses multi-processing instead of multi-threading. There are technical differences between the two modes of parallelism, but the idea is roughly the same.
> [@Vixer](#):
>
> is the command --p-reads-per-batch the same as the --p-chunk-size in the one in the post you just linked (because with the --help command it doesnt appear any option named chunk-size)?
Sorry about that, it looks like `--p-chunk-size` was renamed to `--p-reads-per-batch` in the QIIME 2 `2017.9` release ([changelog notes](https://forum.qiime2.org/t/qiime-2-2017-9-release-is-now-live/1160)).
> [@Vixer](#):
>
> If so, I changed it from default to 1000 and it just took a little longer to send the error message. Also, can you give me a quick explanation about single-threaded mode and how modify it?
You're likely running the command in single-job mode unless you're using `--p-n-jobs` with a value other than `1`. If you're not including `--p-n-jobs` in the command, it will run in single-job mode by default. If you have a limited amount of memory, you'll want to run the command in single-job mode (in other words, you can omit the `--p-n-jobs` option altogether).
Single-job mode means that the command will run on a single CPU instead of processing the data in smaller parallel jobs. You can speed up the runtime of the command by specifying `-1` to use all CPUs, or a value greater than `1` to use the specified number of CPUs. However, that will use up more memory than running in single-job mode, so it won't help reduce memory requirements.
Let me know if you have any other questions about this, and sorry again for the confusion!
---
### Author:  [@Vixer](https://forum.qiime2.org/u/Vixer)
#### Post date: [February 15, 2018, 9:50pm UTC](https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956/9 "2018-02-15T21:50:17Z")
Hello again, I managed to get a laptop with 16 Gb of RAM and a i7 processor, I assigned 12Gb to the virtual machine and used a --p-reads-per-batch 1000 and 500 and I´m still getting the error.
Also, I forgot to add that I have 2,777 sequences in my rep-seqs.qza artifact.
Sadly, this is the best computer I´m able to get.
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [February 16, 2018, 3:06pm UTC](https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956/13 "2018-02-16T15:06:40Z")
Hi @Vixer,
> [@Vixer](#):
>
> Hello again, I managed to get a laptop with 16 Gb of RAM and a i7 processor, I assigned 12Gb to the virtual machine and used a --p-reads-per-batch 1000 and 500 and I´m still getting the error.
Unfortunately, SILVA classifiers take a ton of memory and even 12 GB may be insufficient... though reducing `reads-per-batch` should really make this work (that's how I get SILVA classifiers working on a laptop with 8 GB RAM).
My advice is to try one of the pre-trained [Greengenes classifiers](https://docs.qiime2.org/2018.2/data-resources/#taxonomy-classifiers-for-use-with-q2-feature-classifier) instead. These are much less memory-intensive and probably won't even require setting `reads-per-batch`. If that's still failing, I'd recommend monitoring memory use to see how much is actually being consumed.
Oh, one more thing: DO NOT attempt to run `classify-sklearn` in parallel. This will load multiple copies of the classifier into memory, depleting resources. Parallel classification is really only feasible/useful on high-performance computing clusters.
If all else fails, give `classify-consensus-blast` or `classify-consensus-vsearch` a try instead.
---
### Author:  [@Vixer](https://forum.qiime2.org/u/Vixer)
#### Post date: [February 19, 2018, 3:13pm UTC](https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956/15 "2018-02-19T15:13:10Z")
Oh, I was running classify-sklearn while using the SilvaDB. Just used the consensus blast with the rep-set99 from the silva 199 for qiime and just worked great!
Thanks a lot!
---
### Author:  [@system](https://forum.qiime2.org/u/system)
#### Post date: [March 22, 2018, 9:13pm UTC](https://forum.qiime2.org/t/memory-error-using-the-pretrained-silva-119-classifier/2956/16 "2018-03-22T21:13:21Z")
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.