# Missing feature in feature-table filter-seqs output
**URL:** https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139
**Category:** User Support
**Tags:** filtering, metadata, feature-table
**Created:** [December 7, 2017, 11:26pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139 "2017-12-07T23:26:23Z")
**Posts on this page:** 9
**Page:** 1
### Author:  [@steff1088](https://forum.qiime2.org/u/steff1088)
#### Post date: [December 7, 2017, 11:26pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139/1 "2017-12-07T23:26:23Z")
Hi everybody,
after using `qiime feature-table filter-features`, generating a .tsv file that contained all the features **to retain** and applying `qiime feature-table filter-seqs ` I used the filtered data to obtain core metrics. However, the command resulted in a an error that indicated an inconsistency in my filtered feature table and the filtered sequences. It turned out that the command `qiime feature-table filter-seqs ` ignored the first line after the header in the .tsv file which was also the most abundant feature of my sample! Why did it do that?
Here the command I used to convert the .csv from the frequency per feature .qzv-page into the .tsv file including the information of which sequences to keep (frequency \< 5 in my case). All other sequences are correctly filtered (out), just one was thrown out (the one with highest frequency), which was consistent across different sample sets.
`$ echo '#FeatureID','Frequency' | cat - feature-frequency-detail.csv | tr "," "\\t" > features-to-retain.tsv`
The `qiime feature-table filter-seqs `command looked like this:
```
$qiime feature-table filter-seqs \
--i-data or-seqs.qza \
--m-metadata-file features-to-retain.tsv \
--o-filtered-data or-seqs_filtered.qza
```
Any suggestions or explanations would be great!
cheers,
steffen
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [December 8, 2017, 4:07pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139/3 "2017-12-08T16:07:24Z")
Hi @steff1088,
Would you mind sharing your `features-to-retain.tsv` and `or-seqs.qza` files to help us troubleshoot? You can send those directly to me in a direct message if you do not want your files to be shared openly on the forum.
Note that [in the future](https://github.com/qiime2/q2-feature-table/issues/152) (probably this month's release) it will be possible to filter `FeatureData[Sequence]` artifacts directly using a feature table (e.g., filtered feature table) as input, making it unnecessary to use this workaround for filtering sequences that are not found in a feature table.
Thanks!
---
### Author:  [@steff1088](https://forum.qiime2.org/u/steff1088)
#### Post date: [December 8, 2017, 5:10pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139/5 "2017-12-08T17:10:34Z")
Hi @Nicholas_Bokulich,
no problem, here the files:
[16S\_SJ1\_features-to-retain.tsv](https://cdck-file-uploads-global.s3.dualstack.us-west-2.amazonaws.com/flex002/uploads/qiime21/original/1X/cf8c1ba43c1bef34445c88530859f2d1dfd3579f.tsv) (70.1 KB)
I just noticed that it wont let me upload the .qza file which is 29 MB big. Can I send this as direct message?
-steffen
---
### Author:  [@Nicholas\_Bokulich](https://forum.qiime2.org/u/Nicholas_Bokulich)
#### Post date: [December 8, 2017, 5:56pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139/7 "2017-12-08T17:56:32Z")
Hi @steff1088,
It looks like the issue is that your `features-to-retain.tsv` file contains a `#` character at the start of the header line. This causes that line to be interpreted as a comment line instead of a header, and hence it is ignored; the next line (the missing feature!) winds up being interpreted as the header line, and hence is not retained in your filtered sequences! Remove the `#` in the first line and all is well.
I assume you are reading another forum post or tutorial describing how to filter sequences using a metadata file; if this guide states that a `#` character should be in the first line, please give me the link so that we can correct that information.
The pending issue that I linked to in my previous post will make the process of filtering sequences found in a feature table much easier and clearer! In the meantime (and until we have better documentation of these actions), thank you for your brave efforts! ⛵
---
### Author:  [@jairideout](https://forum.qiime2.org/u/jairideout)
#### Post date: [December 8, 2017, 6:24pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139/9 "2017-12-08T18:24:03Z")
> [@Nicholas\_Bokulich](#):
>
> It looks like the issue is that your features-to-retain.tsv file contains a # character at the start of the header line. This causes that line to be interpreted as a comment line instead of a header, and hence it is ignored; the next line (the missing feature!) winds up being interpreted as the header line, and hence is not retained in your filtered sequences! Remove the # in the first line and all is well.
Ouch, it looks like @steff1088 ran into the same metadata filtering issue reported here (other users have also run into this problem):
> [@Possible to provide warning when metadata doesn't have a header?](https://forum.qiime2.org/t/possible-to-provide-warning-when-metadata-doesnt-have-a-header/1450):
>
> In my rush to try out the new filter-seqs command I made a bit of a headache for myself by not having a header line in my metadata and losing my first sequence. I wasn't using a where clause, so just using the first column as ids. In this case, would it be possible to test the header line to see if it is a valid id in the data to be filtered and either retain it (perhaps too clever) or throw a warning? Thanks!
The issue stems from the fact that QIIME 2 doesn't have a required header for metadata files, making it impossible to detect and correct from the situation @steff1088 and @Chris_Hemmerich ran into 👎 I am working on overhauling metadata this month (including the file format), and this issue will be addressed then (no more unexpected filtering behavior!). I'll follow up here when the issue is fixed in the 2017.12 release!
---
### Author:  [@steff1088](https://forum.qiime2.org/u/steff1088)
#### Post date: [December 8, 2017, 6:30pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139/10 "2017-12-08T18:30:01Z")
Thank you @Nicholas_Bokulich and @jairideout.
This is exactly what was wrong. I falsely included the # symbol into my header line..
I checked and the feature does now appear in the filtered sequences.
Thanks for your quick help, I keep learning about qiime2 every day!
cheers,
steffen
---
### Author:  [@thermokarst](https://forum.qiime2.org/u/thermokarst)
#### Post date: [December 22, 2017, 5:50pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139/11 "2017-12-22T17:50:41Z")
[QIIME 2 2017.12 is now out](https://forum.qiime2.org/t/qiime-2-2017-12-release-is-now-live/2308), and it includes the ability to now optionally filter `FeatureData[Sequence]` using the feature ids found in a `FeatureTable`!
---
### Author:  [@system](https://forum.qiime2.org/u/system)
#### Post date: [January 22, 2018, 11:50pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139/12 "2018-01-22T23:50:48Z")
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.
---
### Author:  [@jairideout](https://forum.qiime2.org/u/jairideout)
#### Post date: [February 16, 2018, 5:01pm UTC](https://forum.qiime2.org/t/missing-feature-in-feature-table-filter-seqs-output/2139/14 "2018-02-16T17:01:46Z")
In the QIIME 2 `2018.2` release, Metadata now has a very minimal required header to avoid the issues reported in this topic.
There are a number of other changes to QIIME 2 Metadata in the `2018.2` release. See [this forum announcement](https://forum.qiime2.org/t/summary-of-changes-to-metadata-in-qiime-2-2018-2-release/2884) for details on what changed, as well as the updated [Metadata tutorial](https://docs.qiime2.org/2018.2/tutorials/metadata/). 🌞