# q2 moshpit eggnog-annotate: suggestion for multithreading with --p-n-threads

**URL:** https://forum.qiime2.org/t/q2-moshpit-eggnog-annotate-suggestion-for-multithreading-with-p-n-threads/30917
**Category:** Ideas and Suggestions
**Tags:** moshpit
**Created:** [July 23, 2024, 3:47pm UTC](https://forum.qiime2.org/t/q2-moshpit-eggnog-annotate-suggestion-for-multithreading-with-p-n-threads/30917 "2024-07-23T15:47:11Z")
**Posts on this page:** 3
**Page:** 1

<div class="post-metadata">

### Author: ![pietervanveelen](https://forum.qiime2.org/user_avatar/forum.qiime2.org/pietervanveelen/32/19014_2.png) [@pietervanveelen](https://forum.qiime2.org/u/pietervanveelen)
#### Post date: [July 23, 2024, 3:47pm UTC](https://forum.qiime2.org/t/q2-moshpit-eggnog-annotate-suggestion-for-multithreading-with-p-n-threads/30917/1 "2024-07-23T15:47:11Z")

</div>

Dear QIIME2 developers,

I am trying out q2-moshpit using qiime2-shotgun-2024.2 Linux deployment on a HPC.  
My capacity is capped at16 cpu/128 GB memory per node.

Most functions work well and relatively fast, whenever multithreading is possible. Using qiime moshpit eggnog-diamond-search on contigs assembled from a cleaned 50 million reads metagenomic dataset of an activated sludge sample works at an acceptable pace.

Particularly, the annotation function qiime moshpit eggnog-annotate is fairly slow (running about 15 minutes per 500 queries). I haven't checked the number of queries to go, but I am at 11000 and counting. The memory usage of this task seems low, conveniently output to stdout for each block of 500 queries (~1.5% used, 98.5% available). I was wondering if you could enable a --p-n-threads parameter for this function for multithreading?

Just an idea that may be worth considering.

As I am now dealing with 12 of such metagenomic samples sequenced, and 35 slightly larger sets of accompanying paired (ribo-depleted) metatranscriptomic data sets, I am hoping for optimized runtime.

Thanks guys (and thanks for the continuous development work in general over the years)

Cheers,  
Pieter

---

<div class="post-metadata">

### Author: ![misialq](https://forum.qiime2.org/user_avatar/forum.qiime2.org/misialq/32/9053_2.png) [@misialq](https://forum.qiime2.org/u/misialq)
#### Post date: [July 25, 2024, 1:53pm UTC](https://forum.qiime2.org/t/q2-moshpit-eggnog-annotate-suggestion-for-multithreading-with-p-n-threads/30917/3 "2024-07-25T13:53:46Z")

</div>

Hey @pietervanveelen, welcome to the forum!

Thanks for your question/suggestion! We introduced support for multiprocessing (through the `--p-num-cpus` parameter) in the `eggnog-annotate` action in the 2024.5 version of the Q2 metagenome distribution - maybe you could check it out there? Unless you were referring to something else?

Cheers,  
Michal

---

<div class="post-metadata">

### Author: ![pietervanveelen](https://forum.qiime2.org/user_avatar/forum.qiime2.org/pietervanveelen/32/19014_2.png) [@pietervanveelen](https://forum.qiime2.org/u/pietervanveelen)
#### Post date: [July 26, 2024, 2:31pm UTC](https://forum.qiime2.org/t/q2-moshpit-eggnog-annotate-suggestion-for-multithreading-with-p-n-threads/30917/4 "2024-07-26T14:31:08Z")

</div>

Dear @misialq, thanks for getting back promptly on my request.

Your suggestion nailed it, that's exactly what I was looking for. I continued with installing 2024.5 and will have try with the multiprocessing. A single CPU situation took more than two days to mapping all ~190K EggNog hits on the contigs of a single metagenome sample.

Cheers,  
Pieter