Metadata Index Error

Hi, I’m having a similar issue. I have a metadata file that passes both the qiime1 and qiime2 Keemi test. You can find it here. https://docs.google.com/spreadsheets/d/1OoGZh3tWZr2AVYCYexzq4fgtAxR8bJYSH8PWXFQwSiY/edit?usp=sharing

I updated to qiime 2 2017.9 but am still getting an error.
There was an issue with loading the file mapping_horses.tsv as metadata:

Non-string Metadata index values detected. There may be more errors
present in this metadata. Sample/feature metadata files can be
validated using Keemei: http://keemei.qiime.org

Hi @jessicalmetcalf! Thanks so much for writing, sorry to hear things aren't going well. It looks like you found a bug :bug:, related to some of the behind-the-scenes parsing of metadata in QIIME 2. To make a long story short, QIIME 2 currently uses pandas for parsing TSV metadata. It looks like one of your sample IDs is 'NA':

46 PM

and, pandas likes to interpret that as a special value representing a null/nan type, which is why you are seeing the Non-string Metadata index values detected error! In the meantime, you could change that sample ID somehow to not use the value 'NA' --- it looks like pandas won't do anything if you use the value 'na', for example.

Sorry for the inconvenience, and thanks for reporting this! :t_rex: :tada:

I ran into that problem when importing into R software. What is the easiest way to fix this in QIIME2? In QIIME1, I would use biom convert to classic otu, fix the name manually, and then convert back to biom.

Great question!

You have a few options:

  1. You could make the ID change at the import step — you could either rename the fastq.gz file (or files, if PE), or use one of the manifest formats to assign the new ID. This would probably be a bit of a pain, since it basically requires you to start over.
  2. You could export your feature table, which would give you the BIOM file — then you could do what you proposed above with the “classic otu,” then re-import the feature table. This might be the easiest option, especially if you are comfortable with biom, but it does “interrupt” your provenance tracking, which may or may not be of concern to you (any time you export/import, QIIME 2 isn’t able to keep track of these data).
  3. Use some existing QIIME 2 functionality to reindex!

a. First, create a new metadata column in your metadata file with the new sample IDs (I am just prefixing your existing IDs, for illustrative purposes):

#SampleID    ...   country    group   ...   SampleID-new
...
HS                                          sHS
NA                                          sNA
NI                                          sNI
PX                                          sPX
...

b. Next, we can use feature-table group to “hack” new sample IDs into place — basically we will “regoup” the feature table using our new SampleID-new column - because we aren’t creating new metadata groups (because the values here are still unique identifiers), we effectively just reindex our existing table. This means that the mode we specify can be anything, because we aren’t combing samples in any way, just remapping their IDs, so I will use sum for this example. Neat! :tada:

$ qiime feature-table group \
  --i-table table.qza \
  --p-axis sample \
  --m-metadata-file sample-metadata.tsv \
  --m-metadata-category SampleID-new \
  --p-mode sum \
  --o-grouped-table reindexed-table.qza

c. Now that the table has been reindexed, update your metadata to match the new IDs, and drop the unnecessary column:

#SampleID    ...   country    group   ...
...
sHS
sNA 
sNI
sPX
...

d. As a final check to make sure things worked out, you can summarize your feature table:

$ qiime feature-table summarize \
  --i-table reindexed-table.qza \
  --m-sample-metadata-file sample-metadata.tsv \
  --o-visualization reindexed-table.qzv

When you view that, you should see your new sample IDs in the sample summary!

Option 3 looks like a lot, but I just ran through the exercise with the Moving Pictures tutorial dataset and it took about 2 minutes (the longest part was creating a new metadata column). Hope that helps, and let us know if you get stuck or have any questions! :balloon:

Oops! Just chatted with @ebolyen about this — option three still won’t work for you, because of the initial issue. Option 3 would work generally for remapping IDs, but it won’t help you here. Sorry for my mistake!

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.

In the QIIME 2 2018.2 release, Metadata now supports IDs and column names that are NA; this name will no longer be interpreted as missing data.

There are a number of other changes to QIIME 2 Metadata in the 2018.2 release. See this forum announcement for details on what changed, as well as the updated Metadata tutorial. :sun_with_face: