I'm trying to generate these ".qzv" visualization "artifacts" on a headless system. I expected that the file would be created, regardless of the fact that the system is running headless. This was on "moving pictures" tutorial.
$ qiime demux summarize \
--i-data demux.qza \
--o-visualization demux.qzv
Plugin error from demux:
Invalid DISPLAY variable
Debug info has been saved to /tmp/qiime2-q2cli-err-o_fl9irr.log.
$ cat /tmp/qiime2-q2cli-err-o_fl9irr.log
Traceback (most recent call last):
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2cli/commands.py", line 222, in __call__
results = action(**arguments)
File "<decorator-gen-263>", line 2, in summarize
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py", line 201, in callable_wrapper
output_types, provenance)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/qiime2/sdk/action.py", line 392, in _callable_executor_
ret_val = callable(output_dir=temp_dir, **view_args)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/q2_demux/_summarize/_visualizer.py", line 162, in summarize
ax = sns.distplot(result, kde=False)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/seaborn/distributions.py", line 161, in distplot
ax = plt.gca()
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/pyplot.py", line 950, in gca
return gcf().gca(**kwargs)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/pyplot.py", line 586, in gcf
return figure()
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/pyplot.py", line 535, in figure
**kwargs)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 44, in new_figure_manager
return new_figure_manager_given_figure(num, thisFig)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 51, in new_figure_manager_given_figure
canvas = FigureCanvasQTAgg(figure)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 242, in __init__
super(FigureCanvasQTAgg, self).__init__(figure=figure)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5agg.py", line 66, in __init__
super(FigureCanvasQTAggBase, self).__init__(figure=figure)
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5.py", line 236, in __init__
_create_qApp()
File "/home/user/anaconda3/envs/qiime2-2017.7/lib/python3.5/site-packages/matplotlib/backends/backend_qt5.py", line 144, in _create_qApp
raise RuntimeError('Invalid DISPLAY variable')
RuntimeError: Invalid DISPLAY variable
$ tree
.
βββ demux.qza
βββ emp-single-end-sequences
β βββ barcodes.fastq.gz
β βββ sequences.fastq.gz
βββ emp-single-end-sequences.qza
βββ sample-metadata.tsv
Hi @08-04! Looks like your installation of matplotlib needs to be configured to use an offscreen render for a headless system. Please give that a shot and let us know how it goes!
I'm not calling matplotlib, Qiime is; therefore it should really be handled by qiime. Everyone that tries to run this is going to have to edit their matplotlib config to get this to work.
As you mentioned above, QIIME 2 is calling matplotlib, and it is matplotlib that is attempting to use a display, not QIIME 2 (which is why I suggested you edit your matplotlib config).
matplotlib default config varies from installation to installation, it depends on what libraries are available at install time (e.g. qt). Some environments (such as an academic HPC) might have their own matplotlib configs, so we generally steer clear from forcing all of our users to use our config for third party tools. The example I linked to above recommends using Agg as the backend, but there might be other users out there who prefer to use a different offscreen render (or a custom render even!). Similarly, matplotlib also supports per-project configs, which we also don't want to interfere with. Anyway, we will support some form of general QIIME 2 config in the future --- perhaps it is worth continuing this discussion then, to see if there is some way for us to provide a fallback config for tools like matplotlib. Thanks!
Thanks for the update @08-04! That visualization (alpha-group-significance) actually doesn't use matplotlib, so your matplotlib configuration (or also the lack thereof) doesn't matter! Those are d3 visualizations, which is a pretty great javascript-based library for interactive visualization. Because QIIME 2 is decentralized, and based on a plugin architecture, every plugin is able to define its own functionality using its own tools and libraries, so some plugins might use matplotlib, while others seaborn, or even d3. It is kind of like a choose-your-own-adventure!
We revisited this topic recently and decided that QIIME 2 will take a stronger stance on matplotlib configuration. The official builds of QIIME 2 now automatically set the backed to Agg. Check out the release notes to learn more about QIIME 2 2017.9!