Plugin error from diversity when run qiime diversity core-metrics-phylogenetic

[Yanwen_Gong (Yanwen)]

I am running the comment below:

qiime diversity core-metrics-phylogenetic
--i-phylogeny rooted-tree.qza
--i-table table.qza
--p-sampling-depth 2300
--m-metadata-file meta_data_google.tsv
--output-dir /pub/yanweng/kai/16s/maren_second_set/data/qiime/0902/core-metrics-results

And the the following error:
Plugin error from diversity:
'charmap' codec can't encode character '\u03b2' in position 3464: character maps to

Debug info has been saved to /scratch/4536597.1.bio/qiime2-q2cli-err-25l6q29x.log

I have used Keemei to validate the meta_data_google.tsv and it seems to be valid:

I found this relevant post but still cannot solve my issue: UnicodeEncodeError: 'charmap' codec can't encode character '\u03b2' in position 3518: character maps to <undefined> - #11 by thermokarst. I am running Qiime2 on the school HPC.

Thanks for you help!

[thermokarst]

Hi @Yanwen_Gong!

@Yanwen_Gong

\u03b2’

This translates to a unicode "β" character - is it possible one of the "B"s in your metadata is actually a beta? Please re-run with the --verbose flag and provide the full output of the command here.

[Yanwen_Gong (Yanwen)]

Hi @thermokarst, please see the output below:

/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/sklearn/metrics/pairwise.py:1575: DataConversionWarning: Data was converted to boolean for metric jaccard
  warnings.warn(msg, DataConversionWarning)
Traceback (most recent call last):
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/q2cli/commands.py", line 327, in __call__
    results = action(**arguments)
  File "</data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/decorator.py:decorator-gen-390>", line 2, in core_metrics_phylogenetic
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 240, in bound_callable
    output_types, provenance)
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 477, in _callable_executor_
    outputs = self._callable(scope.ctx, **view_args)
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/q2_diversity/_core_metrics.py", line 59, in core_metrics_phylogenetic
    metric='unweighted_unifrac', n_jobs=n_jobs)
  File "</data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/decorator.py:decorator-gen-482>", line 2, in beta_phylogenetic
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 240, in bound_callable
    output_types, provenance)
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/action.py", line 411, in _callable_executor_
    spec.qiime_type, output_view, spec.view_type, prov)
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/sdk/result.py", line 273, in _from_view
    provenance_capture=provenance_capture)
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/core/archive/archiver.py", line 316, in from_data
    Format.write(rec, type, format, data_initializer, provenance_capture)
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/core/archive/format/v5.py", line 21, in write
    provenance_capture)
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/core/archive/format/v1.py", line 26, in write
    prov_dir, [root / cls.METADATA_FILE, archive_record.version_fp])
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/core/archive/provenance.py", line 313, in finalize
    self.write_citations_bib()
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/core/archive/provenance.py", line 304, in write_citations_bib
    self.citations.save(str(self.path / self.CITATION_FILE))
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/qiime2/core/cite.py", line 71, in save
    bp.dump(db, f, writer=writer)
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/site-packages/bibtexparser/__init__.py", line 111, in dump
    bibtex_file.write(writer.write(bib_database))
  File "/data/apps/anaconda/3.6-4.3.1/envs/qiime2-2019.7/lib/python3.6/encodings/iso8859_15.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u03b2' in position 3464: character maps to <undefined>

Plugin error from diversity:

  'charmap' codec can't encode character '\u03b2' in position 3464: character maps to <undefined>

See above for debug info.

[thermokarst]

Hi @Yanwen_Gong, thanks, that information clears things up a bit. The issue appears to be related to how your system's locale is configured --- QIIME 2 expects UTF-8 system locale. You can confirm your system config by running the command env. You should probably coordinate with the computer's sysadmin to determine the best way to configure your locale. Keep us posted!

[Yanwen_Gong (Yanwen)]

Hi @thermokarst. I reset the local by "export LANG=“en_US.UTF-8” " before running qiime2, and everything works fine now.

Thanks for helping me troubleshoot.