Error on importing a biom.json file

[amz (Amanda Zahorik)]

A colleague recently gave me a 16S dataset from a study that was done a couple of years ago. They already did their own read joining (this was paired end), quality filtering, and ASV clustering in UNOISE, and what they gave me was:

• a phylogenetic tree
• a fasta file of representative sequences
• a biom table in biom.json format

Importing the sequences and tree went smoothly, but the biom table is giving me trouble. Unsure of what format they were in, I looked at the qiime tools import list of available formats and only saw a couple of BIOM version formats that I thought would apply. I tried the BIOM v1.0 format and while qiime2 didn't complain that my file didn't match the specified format, it did give me a traceback error, which I've pasted below.

Here's the qiime2 command I used:

_qiime tools import \_
_--input-path asv.biom.json \_
_--type 'FeatureTable[Frequency]' \_
_--input-format BIOMV100Format \_
_--output-path asv-table.qza_

Here's the biom.json file (zipped):

asv.biom.zip (1.2 MB)

And here's the error message:

Traceback (most recent call last):

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/site-packages/q2cli/tools.py", line 140, in import_data

    view_type=input_format)

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/site-packages/qiime2/sdk/result.py", line 240, in import_data

    return cls._from_view(type_, view, view_type, provenance_capture)

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/site-packages/qiime2/sdk/result.py", line 265, in _from_view

    result = transformation(view)

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/site-packages/qiime2/core/transform.py", line 70, in transformation

    new_view = transformer(view)

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/site-packages/qiime2/core/transform.py", line 220, in wrapped

    file_view = transformer(view)

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/site-packages/q2_types/feature_table/_transformer.py", line 135, in _8

    data = _parse_biom_table_v100(ff)

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/site-packages/q2_types/feature_table/_transformer.py", line 46, in _parse_biom_table_v100

    table = biom.Table.from_json(json.load(fh))

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/json/__init__.py", line 268, in load

    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/json/__init__.py", line 319, in loads

    return _default_decoder.decode(s)

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/json/decoder.py", line 339, in decode

    obj, end = self.raw_decode(s, idx=_w(s, 0).end())

  File "/home/user/miniconda3/envs/qiime2-2018.11/lib/python3.5/json/decoder.py", line 357, in raw_decode

    raise JSONDecodeError("Expecting value", s, err.value) from None

json.decoder.JSONDecodeError: Expecting value: line 407103 column 2 (char 6432827)

An unexpected error has occurred:

  Expecting value: line 407103 column 2 (char 6432827)

Any help would be appreciated!

[ebolyen (Evan Bolyen)]

Hey @amz

This one's super weird actually (thanks for providing your table).

It turns out the table produced by UNOISE is not actually JSON (I mean it is, but it's mis-formated).
The JSON parser error you see is correct, and at the very end for the data section, there is a trailing comma which indicates more data should follow. However that is the end of the data, and we see a closing ]-bracket.

This means the data wasn't truncated, and you aren't missing anything, but the file is just written poorly.

Fortunately, it is super easy to fix, so I went ahead and removed the trailing comma for you. This file should work fine for import
fixed-asv.biom.json.zip (1.2 MB)

[amz (Amanda Zahorik)]

It worked! Thanks so much, @ebolyen!