The QIIME 2 2017.10 release is now live! There are lots of exciting new changes packed into this release that are described below. Through the end of 2017 we will have monthly releases. We highly recommend staying up-to-date with the latest releases to get access to new features, bug fixes, documentation, etc. We will only provide user support for the latest monthly release of QIIME 2 as these monthly releases continue through the end of 2017.
QIIME 2 will succeed QIIME 1 on January 1, 2018. QIIME 1 will no longer be supported at that time, as development and support effort for QIIME will be focused entirely on QIIME 2. We recommend that existing QIIME users begin transitioning from QIIME 1 to QIIME 2 now.
Check out the QIIME 2 2017.10 docs for details on installing the latest QIIME 2 release, as well as tutorials and other resources. Get in touch on the QIIME 2 forum if you run into any issues!
We will provide official
conda builds on release day for native QIIME 2 installations. Virtual machine builds such as AWS and Docker will be released within the following week. We will post a follow-up announcement here when those builds become available.
Here’s the highlights of the release:
QIIME 2 Framework
Artifact.validate method for validating an artifact’s data. There are currently two levels of validation available:
- Added a new plugin developer API for defining minimal and maximal validation for file and directory formats. This validation API also gives the plugin developer an opportunity to describe why data is invalid, which was previously not possible with the
sniff API (binary yes/no answer only).
sniff has been deprecated in favor of this new validation API. See
q2_types.per_sample_sequences.FastqGzFormat for an example using the new API (i.e. define a
_validate_ method instead of
sniff). Existing formats with
sniff will have the sniffer used for all levels of validation, and will continue to provide binary yes/no answers for backwards-compatibility.
- Fixed a bug with the repr of
Range which made the start and end parameters indistinguishable. Thanks @hunter-cameron!
- Added a new action type called
Pipeline. This action type supports calling other actions and returning both Artifacts and Visualizations. Check out
core-metrics for a complete example, but minimally the API is as follows:
Plugin objects can now register a pipeline with
plugin.pipelines.register_function, it works the same as the other action registrations.
- Pipelines do not use function annotations and instead receive
Artifact objects as input and return
Visualization objects as output.
- Pipelines must have
ctx as their first parameter, it provides the following API:
ctx.get_action(plugin: str, action: str) this returns a “sub-action” that can be called like a normal Artifact API call.
ctx.make_artifact(type, view, view_type=None) this has the same behavior as
Artifact.import_data. It is wrapped by
ctx for pipeline book-keeping.
- Pipelines are available from the Artifact API via
qiime2.plugins.PLUGIN_NAME.pipelines or from
- Pipelines provide nested provenance. This is recorded as a new
pipeline action type and an
alias-of property. The
alias-of has a UUID which will point to a nested DAG which represents the actions that occurred within a pipeline.
- A long-standing resource leak of file descriptors has been fixed. Previously, if many transformers or format instances were created, a process would eventually run out of file descriptors.
- A new plugin containing methods for quality control of sequencing data. In the initial release, this plugin contains one method,
exclude_seqs, which aligns query sequences in a
FeatureData[Sequences] against a user-defined reference sequence set, and outputs separate files of sequences that hit or do not align to the reference sequences. This method is useful for excluding specific known contaminants, or removing host DNA from sequences.
- Check out the accompanying q2-quality-control tutorial!
Added a tutorial describing how to use the new
- Added a section to the q2-longitudinal tutorial showing how to use the new
- Added exercises to the FMT tutorial focused on longitudinal analyses.
- Added commands to the Moving Pictures tutorial to generate stats for
quality-filter q-score and
deblur denoise-16S outputs.
- Added a section to the Moving Pictures tutorial describing how to generate and interpret alpha rarefaction plots using
- Added a section to the Filtering tutorial explaining how to filter feature-tables and representative sequences via taxonomic assignments.
- Added two new methods,
filter-seqs, for filtering
FeatureData[Sequence] (respectively) based on taxonomic annotations in a provided
FeatureData[Taxonomy] artifact. Applications of these new methods include: removing features with specific annotations (e.g., chloroplast or mitochondrial 16S rRNA from a 16S dataset); retaining only features annotated as being part of a specific taxonomic group (e.g., the bacteroidetes); or removing features that were not able to be assigned taxonomy at sufficient depth for a downstream application (e.g., filtering all features that do not contain at least a phylum-level assignment).
core-metrics-phylogenetic are now QIIME 2 pipelines and additionally return Emperor plots for their beta-diversity metrics.
alpha-rarefaction now takes a set of metrics. (
metric has been renamed to
- Added new
mantel visualizer, which performs a Mantel test on two distance matrices. This new visualizer replaces
beta-correlation and provides a more general interface to the Mantel test. To reproduce the behavior of
beta-correlation, use the new
metadata distance-matrix command to create a distance matrix from a numeric metadata category, and then compare that distance matrix to another with
diversity mantel. Note: a
beta-correlation pipeline may be included in the future to essentially chain these two commands together.
- Added a new method,
distance-matrix, for computing a Euclidean distance matrix from a numeric metadata category.
- Added two new methods,
first-distances, for computing the magnitude of change in a user-defined metric between successive time points. There is also an accompanying section of the q2-longitudinal tutorial showing how to use these methods.
- Fixed a bug that caused
volatility charts to have inconsistently ordered labels.
- Added an optional parameter to
volatility that allows for a user to control the step-size/interval of the plot x-axis labels.
- Added a
qiime tools validate command for validating an artifact’s data. There is an option to control the level of validation (e.g. minimal vs. maximal validation).
List are now supported for input parameters.
rarefy performs better validation of sampling depths (depths must be greater than or equal to 1).
- A new method
subsample has been added which can randomly subsample (without replacement) features or samples from a table.
assign-ids now takes a
FeatureTable[Frequency] aligns it to the
Phylogeny[Rooted], and returns the newly aligned table.
balance-taxonomy now takes a
FeatureTable[Composition] instead of
FeatureTable[Balance]. The parameter was also renamed to
table instead of
balance-taxonomy now displays a proportion plot which displays relative abundance of individual taxa across two groups for either side of a balance.
- Regression outputs now report
r-squared (coefficient of determination) in addition to
r (correlation coefficient).
- The color palettes used by
classify-samples have been expanded (and renamed to be more intuitive).
- The “Model Accuracy” section of visualizations now include a link to download the table as a TSV file.
- Added more robust validation for many of the demultiplexed sequence formats, which will help users identify issues with their data (e.g. when importing) instead of receiving a binary yes/no answer about the data’s validity.
alignment mafft now raises a more meaningful error message when a process runs out of memory. This was fixed by updating the minimum required version of