# Problem while trying to filter out unassigned taxa as well as bacteria
**URL:** https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738
**Category:** User Support
**Created:** [September 30, 2019, 12:52am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738 "2019-09-30T00:52:57Z")
**Posts on this page:** 15
**Page:** 1
### Author:  [@microme](https://forum.qiime2.org/u/microme)
#### Post date: [September 30, 2019, 12:52am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/1 "2019-09-30T00:52:57Z")
Since I had unassigned taxa and bacteria at the domain level (D\_0\_) i tried filtering these out using the command:
qiime taxa filter-table
--i-table table-demux.qza
--i-taxonomy taxonomy.qza
--p-exclude "D\_0\_\_Bacteria; **;** ; **;**"
--o-filtered-table table-filt.qza
However, taxa barplot still shows unassigned and D\_0\_Bacteria. [taxa-bar-plots1.qzv](https://forum.qiime2.org/uploads/short-url/mCqKv1uamSLP4vxgVfw34DyvfQS.qzv) (382.6 KB)
When I used "Unassigned; **;** ; **;**" I got an error message saying : All features were filtered, resulting in an empty table. Prior to this, I also had other contaminants such as- D\_0\_\_Bacteria;D\_1\_\_Cyanobacteria;D\_2\_\_Oxyphotobacteria;D\_3\_\_Chloroplast; **;**. But I removed such unwanted seqs. using qiime taxa filter-table
--i-table table-demux.qza
--i-taxonomy taxonomy.qza
--p-mode exact
--p-exclude "Unassigned; **;** ; **;** ,D\_0\_\_Bacteria; **;** ; **;** ,D\_0\_\_Bacteria;D\_1\_\_Cyanobacteria;D\_2\_\_Oxyphotobacteria;D\_3\_\_Chloroplast;\_\_,D\_0\_\_Bacteria;D\_1\_\_Cyanobacteria;D\_2\_\_Oxyphotobacteria;D\_3\_\_Chloroplast;D\_4\_\_Vicia faba (fava bean)"
--o-filtered-table table-filtered.qza
It seems trying to remove the
1. unassigned gives me an error saying empty feature table
2. D\_0\_bacteria does not remove this actually as seen from the taxa bar
(however from the taxa barplot I see that on removing the unassigned should not give an empty feature table).
What else can I do, could you please let me know?
I have already trained on the V4 region using SILVA 132 database and using my forward primers (single end reads).
Also please note: i have looked into the forums for similar such problems, especially [this](https://forum.qiime2.org/t/filtering-out-d-0-bacteria-from-feature-table/6696), and my above commands were based on this forum-yet this problem remains.
---
### Author:  [@ben](https://forum.qiime2.org/u/ben)
#### Post date: [September 30, 2019, 1:05am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/2 "2019-09-30T01:05:39Z")
I'm curious, what kind of samples are these. Ben
---
### Author:  [@microme](https://forum.qiime2.org/u/microme)
#### Post date: [September 30, 2019, 1:19am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/3 "2019-09-30T01:19:29Z")
These are low biomass samples from insects.
---
### Author:  [@ben](https://forum.qiime2.org/u/ben)
#### Post date: [September 30, 2019, 1:21am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/4 "2019-09-30T01:21:28Z")
Did you BLAST any of these sequences?
I primarily evaluate lung/bronchoalvolear lavage samples. They appear very similar to your samples. The unclassified ended up being eukaryotic DNA.
What I did was a strict filter with a qiime quality control step with 99% homology to Greengenes and this filtered out all non classified and bacterial DNA only classified at the kingdom level.
> [@unassigned taxa - oddly reversed between deblur and dada2 pipeline \*so confused\*](https://forum.qiime2.org/t/unassigned-taxa-oddly-reversed-between-deblur-and-dada2-pipeline-so-confused/11696/2):
>
> Yes, from my experience they are from eukaryotic DNA which for some reason has homology with the 16S primers. There is a quality control step, which requires a Vsearch filter to remove anything that doesn't hit with the 16S 97-99% homology with 16S which has dramatically cleaned up my samples. edit: The clustering methods in QIIME1 has a "sort of" built in filtering step. Remember the usearch in QIIME1 requires 97% match either open or closed clustering for it to "hit". DADA2 does not do this,…
Please see what this post did:
> [@Too many unassigned or only at kingdom level features](https://forum.qiime2.org/t/too-many-unassigned-or-only-at-kingdom-level-features/2934):
>
> Hi folks, I'm looking for some help in troubleshooting the source of error in a recent analysis I did that is turning up with large numbers of unassigned features and even more assignments only at the Kingdom level (as seen in the [barplots here](https://view.qiime2.org/visualization/?type=html&src=https%3A%2F%2Fdl.dropbox.com%2Fs%2Fwito6qx99gpf43i%2Ftaxa-bar-plots.qzv%3Fdl%3D1)) Summary: 16s, mouse colon, 2x300 paired-end, MiSeq run using V3-V4 primers (341F & 805R). q2 version 2017.12. I denoised using DADA2 with somewhat lenient truncating parameters based on the [QC plots](https://view.qiime2.org/visualization/?type=html&src=https%3A%2F%2Fdl.dropbox.com%2Fs%2Fo6pjqsxsxg2j70m%2Fdemux-summary.qzv%3Fdl%3D1) which can be found in the provenances. It ran success…
---
### Author:  [@microme](https://forum.qiime2.org/u/microme)
#### Post date: [September 30, 2019, 1:28am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/5 "2019-09-30T01:28:02Z")
Yes, I checked few of my seqs using BLAST, there was plentiful of host DNA. But I also assumed that I would be able to remove the host DNA using qiime taxa filter table. But seems this didn't work. And I used Silva at 97% homology.
"What I did was a strict filter with a qiime quality control step with 99% homology to Greengenes"
I was not using green genes because of the not so regular updating issues. But I think I might give this a try, now that I don't know how to proceed.
---
### Author:  [@ben](https://forum.qiime2.org/u/ben)
#### Post date: [September 30, 2019, 1:29am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/6 "2019-09-30T01:29:05Z")
Sorry, see the previous post, they did a quality control step with 99% homology, I'm sure SILVA will work as well. You can likely do a filter for hits.qza and misses.qza. Then filter your table/req-seq with that pass. Ben
---
### Author:  [@Mehrbod\_Estaki](https://forum.qiime2.org/u/Mehrbod_Estaki)
#### Post date: [September 30, 2019, 2:04am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/7 "2019-09-30T02:04:54Z")
Just as a suggestion, in the post linked I used 99% database because I was much more naive back then. As it was mentioned in that link you can achieve basically the same quality with much less time by using 88\_gg and reducing your identity thresholds to something like 65-85. The idea behind the positive filter is to just discard really foreign looking sequences. I doubt the choice between GG vs Silva in this step would really matter. I could be naive still though!
---
### Author:  [@ben](https://forum.qiime2.org/u/ben)
#### Post date: [September 30, 2019, 2:08am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/8 "2019-09-30T02:08:31Z")
Yeah, I would say that this quality control step probably takes the longest. My pipeline without the step would take 3-4 hours from import to taxonomy bar plot creation on a HPC, but the quality control step with 99% homology adds 4-5 hours to that run. Ben
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [September 30, 2019, 2:41am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/9 "2019-09-30T02:41:23Z")
> [@ben](#):
>
> pipeline without the step would take 3-4 hours from import to taxonomy bar plot creation on a HPC, but the quality control step with 99% homology adds 4-5 hours to that run.
this is because BLAST is not particularly fast and that is all this command is doing — using BLAST to align your query sequences against the reference sequences.
indeed this is the reason to use a small database, e.g., the 88% OTUs as @Mehrbod_Estaki proposes. The goal of this step is just to do a rough filter if you are trying to filter out host vs. bacterial DNA.
---
### Author:  [@ben](https://forum.qiime2.org/u/ben)
#### Post date: [October 7, 2019, 9:00pm UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/11 "2019-10-07T21:00:43Z")
> [@microme](#):
>
> –p-perc-identity 0.50
> –p-perc-query-aligned 0.50
Do 0.80 for both and see if you get better filtering. Ben
---
### Author:  [@microme](https://forum.qiime2.org/u/microme)
#### Post date: [October 21, 2019, 7:00pm UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/12 "2019-10-21T19:00:34Z")
Update and edit: [For the moderators: Please don't queue this. I was trying to edit this text but figured I had to either reply or delete this entire message. Out of the 2 i thought it was better to edit by replying by then by deleting the same previous post. I did this as I recently realized that i had included few of my user-id details in this deleted post, hence i have deleted those information. So now, the commands remain the same as that in the previous deleted post, with only removed personal ids. Apologies for the queueing notification.]
This time I used just a subset of samples instead of all my samples to try getting rid of the features that said: unassigned and D\_0\_bacteria.
My initial plot was [testtaxa-bar-plots.qzv](https://forum.qiime2.org/uploads/short-url/y5F3pCW9QKUUzeHyMe0YWPPXY7k.qzv) (331.5 KB)
After following the steps with hits.qza and misses.qza as mentioned [Too many unassigned or only at kingdom level features](https://forum.qiime2.org/t/too-many-unassigned-or-only-at-kingdom-level-features/2934)
As from the barplot, I see that many reads with unassigned taxa and bacteria were removed however, there are some that still remain. Any ideas on where can i go next with this?
Command used were:
qiime quality-control exclude-seqs
–i-query-sequences /test/repseqssingle.qza
–i-reference-sequences /test/SILVA\_132\_97\_16S-v4-v6-ref-seqs.qza
–p-method vsearch
–p-perc-identity 0.50
–p-perc-query-aligned 0.50
–p-threads 6
–o-sequence-hits /test/hits1.qza
–o-sequence-misses /test/misses1.qza
–verbose
qiime feature-table filter-features
–i-table /test/tabledada2single.qza
–m-metadata-file /test/misses1.qza
–o-filtered-table /test/no-miss-table-dada2.qza
–p-exclude-ids
qiime taxa barplot
–i-table /test/no-miss-table-dada2.qza
–i-taxonomy /test/taxonomyhits1.qza
–m-metadata-file /test/metadatatest.tsv
–o-visualization /test/taxa-barplot1-nomiss.qzv
Also, I lowered the perc-identity to 0.5 this time
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [October 21, 2019, 7:06pm UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/13 "2019-10-21T19:06:17Z")
Just for clarification were you able to resolve the issue using @ben's recommendation?
> [@ben](#):
>
> Do 0.80 for both and see if you get better filtering. Ben
---
### Author:  [@microme](https://forum.qiime2.org/u/microme)
#### Post date: [October 21, 2019, 7:13pm UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/14 "2019-10-21T19:13:04Z")
Nope! not with the silva. Acc. to a discussion with my colleague, I have been using RDP with an identity of 0.50. this problem seems to go away.The rest I am still trying to figure it out. Although I must mention that now i get archaea and there is no such situation with D\_0\_ or unclassified-then on using qiime taxa filter-table, removed them.
I am still trying to figure out the rest, incase i discover more, i will post here.
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [October 21, 2019, 7:15pm UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/15 "2019-10-21T19:15:45Z")
> [@microme](#):
>
> Although I must mention that now i get archaea and there is no such situation with D\_0\_ or unclassified-then on using qiime taxa filter-table, removed them.
Oh I see, so perhaps the RDP database you had been using was bacteria only, or at least missing the Archaea that you have in your samples. Please let us know when you have final resolution, or run into any more issues.
---
### Author:  [@system](https://forum.qiime2.org/u/system)
#### Post date: [November 22, 2019, 1:15am UTC](https://forum.qiime2.org/t/problem-while-trying-to-filter-out-unassigned-taxa-as-well-as-bacteria/11738/16 "2019-11-22T01:15:46Z")
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.