Error without a display

I’m trying to generate these “.qzv” visualization “artifacts” on a headless system. I expected that the file would be created, regardless of the fact that the system is running headless. This was on “moving pictures” tutorial.

$ qiime demux summarize \
  --i-data demux.qza \
  --o-visualization demux.qzv

Plugin error from demux:

    Invalid DISPLAY variable

Debug info has been saved to /tmp/qiime2-q2cli-err-o_fl9irr.log.

$ cat /tmp/qiime2-q2cli-err-o_fl9irr.log
Traceback (most recent call last):
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2cli/commands.py", line 222, in __call__
  results = action(**arguments)
File "<decorator-gen-263>", line 2, in summarize
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py", line 201, in callable_wrapper
  output_types, provenance)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py", line 392, in _callable_executor_
  ret_val = callable(output_dir=temp_dir, **view_args)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_demux/_summarize/_visualizer.py", line 162, in summarize
  ax = sns.distplot(result, kde=False)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/seaborn/distributions.py", line 161, in distplot
  ax = plt.gca()
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/pyplot.py", line 950, in gca
  return gcf().gca(**kwargs)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/pyplot.py", line 586, in gcf
  return figure()
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/pyplot.py", line 535, in figure
  **kwargs)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 44, in new_figure_manager
  return new_figure_manager_given_figure(num, thisFig)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 51, in new_figure_manager_given_figure
  canvas = FigureCanvasQTAgg(figure)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 242, in __init__
  super(FigureCanvasQTAgg, self).__init__(figure=figure)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 66, in __init__
  super(FigureCanvasQTAggBase, self).__init__(figure=figure)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5.py", line 236, in __init__
  _create_qApp()
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5.py", line 144, in _create_qApp
  raise RuntimeError('Invalid DISPLAY variable')
RuntimeError: Invalid DISPLAY variable

$ tree
.
├── demux.qza
├── emp-single-end-sequences
│   ├── barcodes.fastq.gz
│   └── sequences.fastq.gz
├── emp-single-end-sequences.qza
└── sample-metadata.tsv

Hi @08-04! Looks like your installation of matplotlib needs to be configured to use an offscreen render for a headless system. Please give that a shot and let us know how it goes!

2 Likes

I’m not calling matplotlib, Qiime is; therefore it should really be handled by qiime. Everyone that tries to run this is going to have to edit their matplotlib config to get this to work.

Hi @08-04!

As you mentioned above, QIIME 2 is calling matplotlib, and it is matplotlib that is attempting to use a display, not QIIME 2 (which is why I suggested you edit your matplotlib config).

matplotlib default config varies from installation to installation, it depends on what libraries are available at install time (e.g. qt). Some environments (such as an academic HPC) might have their own matplotlib configs, so we generally steer clear from forcing all of our users to use our config for third party tools. The example I linked to above recommends using Agg as the backend, but there might be other users out there who prefer to use a different offscreen render (or a custom render even!). Similarly, matplotlib also supports per-project configs, which we also don't want to interfere with. Anyway, we will support some form of general QIIME 2 config in the future --- perhaps it is worth continuing this discussion then, to see if there is some way for us to provide a fallback config for tools like matplotlib. Thanks!

1 Like

FYI, the alpha diversity functions in the same tutorial produced the qzv files without an error using the default matplotlib configuration.

$ qiime diversity alpha-group-significance \
  --i-alpha-diversity core-metrics-results/faith_pd_vector.qza \
  --m-metadata-file sample-metadata.tsv \
  --o-visualization core-metrics-results/faith-pd-group-significance.qzv
$ qiime diversity alpha-group-significance \
  --i-alpha-diversity core-metrics-results/evenness_vector.qza \
  --m-metadata-file sample-metadata.tsv \
  --o-visualization core-metrics-results/evenness-group-significance.qzv
$ ls -l core-metrics-results/ | awk {'print$9'}

bray_curtis_distance_matrix.qza
bray_curtis_pcoa_results.qza
evenness-group-significance.qzv
evenness_vector.qza
faith-pd-group-significance.qzv
faith_pd_vector.qza
jaccard_distance_matrix.qza
jaccard_pcoa_results.qza
observed_otus_vector.qza
shannon_vector.qza
unweighted_unifrac_distance_matrix.qza
unweighted_unifrac_pcoa_results.qza
weighted_unifrac_distance_matrix.qza
weighted_unifrac_pcoa_results.qza

Thanks for the update @08-04! That visualization (alpha-group-significance) actually doesn’t use matplotlib, so your matplotlib configuration (or also the lack thereof) doesn’t matter! Those are d3 visualizations, which is a pretty great javascript-based library for interactive visualization. Because QIIME 2 is decentralized, and based on a plugin architecture, every plugin is able to define its own functionality using its own tools and libraries, so some plugins might use matplotlib, while others seaborn, or even d3. It is kind of like a choose-your-own-adventure!

1 Like

This topic was automatically closed 31 days after the last reply. New replies are no longer allowed.

We revisited this topic recently and decided that QIIME 2 will take a stronger stance on matplotlib configuration. The official builds of QIIME 2 now automatically set the backed to Agg. Check out the release notes to learn more about QIIME 2 2017.9! :tada:

1 Like