I have just finished the “Moving Pictures Tutorial” to get used to QIIME2. Very straighforward, although it was harder to find information on how to create a traditional OTU table like in QIIME1. Is there some rational behind this? I Have followed the tutorial by thermokarst (Exporting and modifying BIOM tables (e.g. adding taxonomy annotations)) - Many thanks for making this available.
Two questions
1)How do you convert the biom table to txt? I have tried the biom commands but for some reasons I´m not able to convert the biom table.
2)I have notice that OTU are not represented by numbers like in previous QIIME versions. I also curious regarding why this is done this way.
Thank you for the feedback. I have raised an issue here to add this information to the documentation.
The reason behind this is that we describe conversion to biom format, which is fairly widely used at this point and has its own separate documentation for converting to tsv, aka "classic" OTU tables. But I agree, this would be useful to add to the QIIME2 docs.
Could you share your commands, and the full error message you are getting? (also search for that error on the forum — we get a lot of biom conversion errors reported here, even though technically these should be reported and supported elsewhere). Note that we might not be able to fix this issue — biom-format is not developed by the QIIME2 developers and we do not officially support biom on this forum — but we can see if we know the answer.
The Feature IDs (i.e., OTU names) all depend on what method is used to define features. In QIIME 1, the various OTU picking methods assigned OTUs arbitrary numbers or names. In QIIME 2, we recommend dada2 or deblur for denoising, which use the sequence variant (or by default its "hash", a coded string of letters/numbers that represent that unique sequence), but OTU picking is also supported via q2-vsearch, which I believe might produce the same sorts of arbitrary numbers that QIIME 1 users are accustomed to. It all depends on the method.
It is unrelated to your question but important to note that for OTU pickers the OTU names are arbitrary and cannot be compared between OTU picking runs (with the exception of closed-reference OTU picking against the same reference sequences). The hash IDs used by dada2 and deblur are unique for each possible sequence, and hence allow comparison between datasets denoised using the same exact parameters (including primers and trimming lengths).
Thank you very much for the answers. After your explanation its pretty clear the advantage of a unique OTU ID instead of an arbitrary number.
Regarding the Biom convertion, I was following the “Moving Pictures” tutorial and have combined the sequence variant ID with the taxonomy, following a post of the forum (How to make classic otu table with QIIME2?). I got stuck on the final step when converting the biom format to txt.
Is there more to the error? I will need the full error traceback to diagnose.
My guess is that something is going wrong with the previous command (add-metadata). The table might be empty, or the taxonomy annotations may be askew.
The whole error is -
Traceback (most recent call last):
File “/Users/franciscocoelho/anaconda/bin/biom”, line 11, in
sys.exit(cli())
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 716, in call
return self.main(*args, **kwargs)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 696, in main
rv = self.invoke(ctx)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 534, in invoke
return callback(*args, **kwargs)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 129, in convert
table_type, process_obs_metadata, tsv_metadata_formatter)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 185, in _convert
metadata_formatter=obs_md_fmt_f)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/table.py”, line 4096, in to_tsv
observation_column_name)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/table.py”, line 1364, in delimited_self
md_out = metadata_formatter(md.get(header_key, None))
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 36, in
‘sc_separated’: lambda x: '; '.join(x),
TypeError: can only join an iterable
Thank´s Nicholas. Yes, the command returns the following error
Traceback (most recent call last):
File “/Users/franciscocoelho/anaconda/bin/biom”, line 11, in
sys.exit(cli())
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 716, in call
return self.main(*args, **kwargs)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 696, in main
rv = self.invoke(ctx)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 1060, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 889, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/click/core.py”, line 534, in invoke
return callback(*args, **kwargs)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 129, in convert
table_type, process_obs_metadata, tsv_metadata_formatter)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 185, in _convert
metadata_formatter=obs_md_fmt_f)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/table.py”, line 4096, in to_tsv
observation_column_name)
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/table.py”, line 1364, in delimited_self
md_out = metadata_formatter(md.get(header_key, None))
File “/Users/franciscocoelho/anaconda/lib/python3.5/site-packages/biom/cli/table_converter.py”, line 36, in
‘sc_separated’: lambda x: '; '.join(x),
TypeError: can only join an iterable
I did found what was the mistake - when following the “How to make classic otu table with QIIME2?” post I had to rename the header of the taxonomy file - probably I didn´t do it well in the first time. I have redo it following the instructions of the post and was finally able to get the biom command working fine. Thank you very much for your support, it was indeed a problem with the taxonomy file.