Basic questions regarding classic OTU table in QIIME2

Dears users.

I have just finished the “Moving Pictures Tutorial” to get used to QIIME2. Very straighforward, although it was harder to find information on how to create a traditional OTU table like in QIIME1. Is there some rational behind this? I Have followed the tutorial by thermokarst (Exporting and modifying BIOM tables (e.g. adding taxonomy annotations)) - Many thanks for making this available.

Two questions

1)How do you convert the biom table to txt? I have tried the biom commands but for some reasons I´m not able to convert the biom table.

2)I have notice that OTU are not represented by numbers like in previous QIIME versions. I also curious regarding why this is done this way.

Thanks in advance,

Francisco

1 Like

Thank you for the feedback. I have raised an issue here to add this information to the documentation.

The reason behind this is that we describe conversion to biom format, which is fairly widely used at this point and has its own separate documentation for converting to tsv, aka “classic” OTU tables. But I agree, this would be useful to add to the QIIME2 docs.

Could you share your commands, and the full error message you are getting? (also search for that error on the forum — we get a lot of biom conversion errors reported here, even though technically these should be reported and supported elsewhere). Note that we might not be able to fix this issue — biom-format is not developed by the QIIME2 developers and we do not officially support biom on this forum — but we can see if we know the answer.

The Feature IDs (i.e., OTU names) all depend on what method is used to define features. In QIIME 1, the various OTU picking methods assigned OTUs arbitrary numbers or names. In QIIME 2, we recommend dada2 or deblur for denoising, which use the sequence variant (or by default its “hash”, a coded string of letters/numbers that represent that unique sequence), but OTU picking is also supported via q2-vsearch, which I believe might produce the same sorts of arbitrary numbers that QIIME 1 users are accustomed to. It all depends on the method.

It is unrelated to your question but important to note that for OTU pickers the OTU names are arbitrary and cannot be compared between OTU picking runs (with the exception of closed-reference OTU picking against the same reference sequences). The hash IDs used by dada2 and deblur are unique for each possible sequence, and hence allow comparison between datasets denoised using the same exact parameters (including primers and trimming lengths).

I hope that helps!

Dear Nicholas,

Thank you very much for the answers. After your explanation its pretty clear the advantage of a unique OTU ID instead of an arbitrary number.

Regarding the Biom convertion, I was following the “Moving Pictures” tutorial and have combined the sequence variant ID with the taxonomy, following a post of the forum (How to make classic otu table with QIIME2?). I got stuck on the final step when converting the biom format to txt.

The code is the following:

qiime tools export table.qza --output-dir exported
qiime tools export taxonomy.qza --output-dir exported

#####Change the first line of biom-taxonomy.tsv (i.e. the header) to this:

#OTUID
#taxonomy
#confidence

biom add-metadata -i exported/feature-table.biom -o table-with-taxonomy.biom --observation-metadata-fp exported/biom-taxonomy.tsv --sc-separated taxonomy

#convert to classic table

biom convert -i table-with-taxonomy.biom -o OTU_Table.txt --to-tsv --header-key taxonomy

What error are you getting when you run that command?

Thank you for the reply. The error is

TypeError: can only join an iterable

Is there more to the error? I will need the full error traceback to diagnose.

My guess is that something is going wrong with the previous command (add-metadata). The table might be empty, or the taxonomy annotations may be askew.

Actually, it looks like this problem has been addressed on the QIIME 1 forum: https://groups.google.com/forum/#!topic/qiime-forum/HVKRK6KV4Nw

Check out that topic — sounds like an issue with the taxonomy annotations. Let us know if fixing that solves your problem.

Good luck!

The whole error is -
Traceback (most recent call last):
File “/Users/franciscocoelho/anaconda/bin/biom”, line 11, in
sys.exit(cli())
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 716, in call
return self.main(*args, **kwargs)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 696, in main
rv = self.invoke(ctx)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 534, in invoke
return callback(*args, **kwargs)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 129, in convert
table_type, process_obs_metadata, tsv_metadata_formatter)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 185, in _convert
metadata_formatter=obs_md_fmt_f)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/table.py”, line 4096, in to_tsv
observation_column_name)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/table.py”, line 1364, in delimited_self
md_out = metadata_formatter(md.get(header_key, None))
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 36, in
‘sc_separated’: lambda x: '; '.join(x),
TypeError: can only join an iterable

I will check the QIIME 1 forum!

Thanks Nicholas!

Francisco

One more thing, if I remove the --header-key taxonomy from biom convert it does returns an OTU table, but without the taxonomy…

Francisco

That’s a pretty good indicator that the taxonomy is to blame (which that QIIME 1 issue would suggest).

You probably have a feature that is missing a taxonomy assignment, or is set to None.

Out of curiosity, do you get the same error if you use this command:

biom convert -i table-with-taxonomy.biom -o OTU_Table.txt --to-tsv --header-key taxonomy --tsv-metadata-formatter sc_separated

Thank´s Nicholas. Yes, the command returns the following error

Traceback (most recent call last):
File “/Users/franciscocoelho/anaconda/bin/biom”, line 11, in
sys.exit(cli())
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 716, in call
return self.main(*args, **kwargs)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 696, in main
rv = self.invoke(ctx)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 534, in invoke
return callback(*args, **kwargs)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 129, in convert
table_type, process_obs_metadata, tsv_metadata_formatter)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 185, in _convert
metadata_formatter=obs_md_fmt_f)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/table.py”, line 4096, in to_tsv
observation_column_name)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/table.py”, line 1364, in delimited_self
md_out = metadata_formatter(md.get(header_key, None))
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 36, in
‘sc_separated’: lambda x: '; '.join(x),
TypeError: can only join an iterable

Dear Nicholas,

I did found what was the mistake - when following the “How to make classic otu table with QIIME2?” post I had to rename the header of the taxonomy file - probably I didn´t do it well in the first time. I have redo it following the instructions of the post and was finally able to get the biom command working fine. Thank you very much for your support, it was indeed a problem with the taxonomy file.

Sincerely,

Francisco

2 Likes

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.