Hello,
I'm attempting to create a heatmap of the core microbiome of my study subject. To do this I ran core-features and edited the feature table tsv to only include the core taxa that I want to include in my heatmap. However, when I convert this tsv file to a biom file and finally a feature table qza file, its lost one of the core bacterial ASVs. Does anyone know why this is happening or what I could do to prevent it?
Thanks in advance!
Hi @jessimya, We'll need some more detail to help with this. Could you provide the commands that you're running step by step, and ideally some example data that we could use to try to reproduce the error?
Sure!
So I edited the TSV of the tabulated-all-L6-table.qzv to omit all features not included in the core, shown in L6-core-taxa-metadata.tsv. The only way I could figure out to reupload the tsv as a qza was to convert it to a biom file first, for which I used the following script:
biom convert -i L6-core-taxa-metadata.tsv -o feature-table-core-L6_hdf5.biom --table-type="OTU table" --to-hdf5
qiime tools import
--input-path feature-table-core-L6_hdf5.biom
--type 'FeatureTable[Frequency]'
--input-format BIOMV210Format
--output-path feature-table-core-L6.qza
But when I use the following script to view the metadata:
qiime metadata tabulate
--m-input-file feature-table-core-L6.qza
--o-visualization tabulated-core-taxa-metadata.qzv
There are only 33 features shown instead of the 34 that are in the original feature table.
Please let me know if you need any more info!
tabulated-all-L6-table.qzv (1.4 MB)
L6-core-taxa-metadata.tsv (14.4 KB)
feature-table-core-L6_hdf5.biom (56.3 KB)
feature-table-core-L6.qza (23.6 KB)
tabulated-core-taxa-metadata.qzv (1.2 MB)
Additionally, somewhere along the upload process the axes are getting flipped so that the x is the sample ID and the y is the taxonomic group, instead of the other way around making it so I can't use my metadata with it
Hi @jessimya!
I ran this command to look at your biom table and we see the transposed axes and dropped row are are also present before using QIIME 2:
biom summarize-table -i ./feature-table-core-L6_hdf5.biom
Num samples: 33
Num observations: 113
Total count: 657,586
Table density (fraction of non-zero values): 0.821
Counts/sample summary:
Min: 1,254.000
Max: 268,818.000
Median: 4,295.000
Mean: 19,926.848
Std. dev.: 55,962.514
Sample Metadata Categories: None provided
Observation Metadata Categories: d__Bacteria;p__Proteobacteria;c__Gammaproteobacteria;o__Xanthomonadales;f__Rhodanobacteraceae;g__Luteibacter
Counts/sample detail:
d__Bacteria;p__Proteobacteria;c__Alphaproteobacteria;o__Rhizobiales;f__Beijerinckiaceae;g__1174-901-12: 1,254.000
d__Bacteria;p__Proteobacteria;c__Alphaproteobacteria;o__Micropepsales;f__Micropepsaceae;g__uncultured: 1,521.000
...
The reason for this is that biom expects OTUs to be the rows, while samples are the columns. This was kind of an "old-school" format, since it made it simple to add more observations by just adding another row. It's not especially natural however, since the OTUs are dependent on the samples so we prefer samples as rows in QIIME 2.
This may also be why you are ending up with the wrong number of samples as it the conversion to BIOM is slightly off.
You'll want your TSV to look like this:
#OTU ID S1 S2 S3
O1 0.0 0.0 1.0
O2 1.0 3.0 42.0
where O1, O2, etc are OTUs/Taxonomy and S1, S2, etc are samples.
Try transposing your table and removing resulting #q2:types column (formerly row) and renaming id to #OTU ID, then run the biom convert command again.
Hope that helps!
Thank you for your response!
I tried updating the tsv as you suggested
L6-core-taxa-metadata-fixed.tsv (27.5 KB)
but now when I try to convert it to a biom file I'm getting this error, sorry for the length. Do you have any advice on what's going wrong?
Thanks again!
Traceback (most recent call last):
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/biom/table.py", line 5163, in _extract_data_from_tsv
values = list(map(dtype, fields[1:-1]))
ValueError: could not convert string to float: ''
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/biom/parse.py", line 671, in load_table
table = parse_biom_table(fp)
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/biom/parse.py", line 415, in parse_biom_table
t = Table.from_tsv(file_obj, None, None, lambda x: x)
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/biom/table.py", line 5012, in from_tsv
t_md_name) = Table._extract_data_from_tsv(lines, **kwargs)
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/biom/table.py", line 5167, in _extract_data_from_tsv
raise TypeError(msg % (lineno, badidx+1, badval))
TypeError: Invalid value on line 35, column 34, value
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/jessie/miniconda3/envs/qiime2-2023.2/bin/biom", line 11, in
sys.exit(cli())
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/click/core.py", line 1130, in call
return self.main(*args, **kwargs)
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/biom/cli/table_converter.py", line 113, in convert
table = load_table(input_fp)
File "/home/jessie/miniconda3/envs/qiime2-2023.2/lib/python3.8/site-packages/biom/parse.py", line 673, in load_table
raise TypeError("%s does not appear to be a BIOM file!" % f)
TypeError: L6-core-taxa-metadata-fixed.tsv does not appear to be a BIOM file!
Hi @jessimya,
It looks like the original table is still in the TSV (so you have transpose + original stacked), so it results in a number of empty cells after line 36 (which is the error).
It should work if you delete everything after that point.
Oops, thanks! That fixed it and I was able to get my heatmap!
I'm using this heatmap to compare the skin and gut microbiome of my study species, and this figure lists all samples separately. Do you know if there's a way to collapse my table by the sample type so that I get a heatmap showing only two columns, all skin samples combined and all gut samples combined?
Thanks!
Hi @jessimya,
Yeah! It's qiime feature-table group. The only downside is you won't have very convenient metadata to go with it for heatmap (as the samples get collapsed into a new label), but with only two groups, you should be able to make that by hand again.
Thank you so much for all your help @ebolyen I really appreciate it!
Last thing (hopefully), I re-downloaded the tsv for the heatmap from the qza to look at the data, I'm trying to find the frequencies being shown, and I was hoping you could tell me a bit more about the two numbers at the end of each row.
I thought they might show the number of times that feature was seen in the skin and gut samples respectively, but the numbers aren't matching up with my data.
Thanks!
feature-table-core-sampletype.tsv (4.0 KB)
Hi @jessimya,
The number will depend on how you used feature-table group since it supports one of these three methods: sum, median-ceiling, mean-ceiling.
I presume you used sum, in which case it should be what you expect here. In what way does it seem to not match?
This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.