Error with biom convert: %s does not appear to be a BIOM file!

Hi everyone

I'm trying to convert a .tsv file into a BIOM file, for later convert in a .qza file. But I'm only getting error messages

This file is a decontaminated table exported from R: filtered-table-trim2-decon.tsv (536.9 KB)

Here is my code: biom convert -i filtered-table-trim2-decon.tsv -o filtered-table-trim2-decon.biom --table-type "Table" --to-hdf5

I already have tried "OTU table" or Table without the " ", but give me errors too.

Here is the error:

Traceback (most recent call last):
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/biom/parse.py", line 672, in load_table
    table = parse_biom_table(fp)
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/biom/parse.py", line 416, in parse_biom_table
    t = Table.from_tsv(file_obj, None, None, lambda x: x)
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/biom/table.py", line 4875, in from_tsv
    t_md_name) = Table._extract_data_from_tsv(lines, **kwargs)
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/biom/table.py", line 4990, in _extract_data_from_tsv
    md_name = header[-1]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/bin/biom", line 11, in <module>
    sys.exit(cli())
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/biom/cli/table_converter.py", line 114, in convert
    table = load_table(input_fp)
  File "/home/joaomiranda/miniconda3/envs/qiime2-2021.4/lib/python3.8/site-packages/biom/parse.py", line 674, in load_table
    raise TypeError("%s does not appear to be a BIOM file!" % f)
TypeError: filtered-table-trim2-decon.tsv does not appear to be a BIOM file!

Hi @joaomiranda,
I think the issue here is that you have a "Taxon" column in your table which is not obviously count data.

Taken from this issue before, can you try the following and see if it works?


biom convert \
-i filtered-table-trim2-decon.tsv \
-o filtered-table-trim2-decon.biom \
--to-hdf5 \
--table-type="Taxon table" \
--process-obs-metadata taxonomy

edit: updated the code chunk above to one that actually works.

Hello @Mehrbod_Estaki ,

Thanks for the prompt response!

convert_biom.py give me a "command not found", so I tried the same code but using biom convert

biom convert \
  -i filtered-table-trim2-decon.tsv \
  -o filtered-table-trim2-decon.biom \
  --to-hdf5 \
  --table-type="OTU table" \
  --process-obs-metadata Taxon

Now the error is: Error: Invalid value for '--process-obs-metadata': invalid choice: Taxon. (choose from sc_separated, naive, taxonomy)

So I tried to put quotes in Taxon ("Taxon") and runned the same code, resulted in the same error. When I replaced Taxon with taxonomy in the code, I received the error message that I posted above.

Hi @joaomiranda,
Sorry that thread was rather old so I updated the code-chunk in my original answer to one that works. I tested it with your table already and everything looks fine to me.

I even checked that it would be ok upon import into QIIME 2 and it worked for me:

qiime tools import \
  --input-path filtered-table-trim2-decon.biom \
  --type 'FeatureTable[Frequency]' \
  --input-format BIOMV210Format \
  --output-path filtered-table-trim2-decon.qza
1 Like

Thank you so much!

It worked perfectly well

1 Like

Hi @Mehrbod_Estaki ,

:exploding_head: I tried with another tsv file, with the same conditions, using the same code and gave me "%s does not appear to be a BIOM file!".

I don't understand this error, I just got this file from R and ajust the headers to look like the tsv model generated by qiime, the same as I did with the other file.

Do you think the error might be in the tsv file?

Here is the tsv file generated by qiime and the tsv that I'm trying to reconvert in biom:filtered-table-trim-gg.tsv (793.6 KB) filtered-table-trim-gg-decon.tsv (495.2 KB)

My code:

    biom convert \
    -i filtered-table-trim-gg-decon.tsv \
    -o filtered-table-trim-gg-decon.biom \
    --to-hdf5 \
    --table-type="Taxon table" \
    --process-obs-metadata taxonomy

edit: I don't know why my code is being posted without the backslash but I ran it in the right format

Hi @joaomiranda,

The reason why your new table is not working is because it does not fit the expected properties of a biom file. From a quick look, here's a few potential recommendations that I can see when I compared the QIIME 2 produced table vs the one you are trying to import.

  • make sure the first column is #OTU ID, yours currently reads Mean.blank and is the 2nd column. Your V1 column looks like a redundant index column to me, so just remove it
  • Your new table is not all count data which is what BIOM was designed for. You'll need to make sure you convert values like 10.73382 to counts, maybe by rounding them up?
  • Your taxonomy column does not have a header. You'll want to add Taxon there as your header?

There may be more issues, so you'll want to mirror the format you get our of QIIME 2 or read the documentations for BIOM to get a better sense of what may be wrong.

Good luck!

Update!

I checked my tsv table and it was out of qiime formatting. So I made the adjustments and runned the code: It worked.

So the conclusion about this forum is to check the table about formatting and execute the correct code according to the presence or absence of taxonomy data.

Thanks a lot!

Glad to hear it worked!
Just to clarify, QIIME 2 it self doesn't have any requirements regarding your biom conversion. That is handled by the biom package which is a separate tool and it has its own set of formatting requirements. But yes, always important to make sure formatting is correct and consistent!
Good luck with your analysis.