The QIIME 2 2019.4 release is now available! Thanks to everyone involved for their hard work!
As a reminder, our next planned QIIME 2 release is scheduled for late July 2019 (QIIME 2 2019.7), but please stay tuned. Other proposed releases this year include:
- QIIME 2 2019.7
- QIIME 2 2019.10
Check out the QIIME 2 2019.4 docs for details on installing the latest QIIME 2 release, as well as tutorials and other resources. Get in touch on the QIIME 2 forum 2 if you run into any issues!
Virtual machine builds will be available sometime next week - watch this topic thread for an update!
Here’s the highlights of the release:
- QIIME 2 Framework
- @ChrisKeefe fixed a bug caused by invalid primitive arguments
. Expanded type mismatch error messages to better describe various invalid-input situations. 
- @ChrisKeefe fixed some pretty confusing language in the
SemanticTypedocstring. @Sleightly (GH)has added a new SDK utility namedbuild_graphtoqiime2.sdk.actiongraph. This will generate a directed NetworkX graph of QIIME 2 plugin actions and types. The graph is bipartite, with types on one conceptual side, and actions on the other. Edges between them indicate parameters.- The implementation of QIIME 2 types has been reworked by @ebolyen
. Some minor changes include:
Properties(['a', 'b'])=>Properties('a', 'b')Range(X, None)can be optionally written asStart(X)(it is converted to aRange)Range(None, X)can be optionally written asEnd(X)(also converted toRange)Choices({'a', 'b'})=>Choices('a', 'b')(this means order is now retained as well)
- Unions of primitive types are now possible, thanks to @ebolyen! This means you can create a plugin with a parameter such as:
Int % Start(0) | Str % Choices("ALL", "SOME", "NONE") - Intersection (
&) of types is now possible
however most intersections will result in a "bottom" type - thanks for that, @ebolyen! (This operator is primarily for implementing internal type solving algorithms.)
- real intesection:
Start(0) & End(10)=>Range(0, 10) - contradiction:
FeatureTable[Frequency] & Phylogeny[Rooted]=>UnionExp([])(an empty union is the smallest, emptiest, type in QIIME 2)
- real intesection:
- The framework now supports different output types based on the received input type
. This feature is called TypeMap(and it has a siblingTypeMatch, and was created by @ebolyen):- This allows expressions such as:
T, P, R = TypeMap({ (TypeA, Bool % Choices(True)): OutputA, (TypeB, Bool % Choices(False)): OutputB }) - For a direct example of TypeMatch, see implementation for feature-table filter-samples.
- This allows expressions such as:
- A utility to parse a type from the result of a
to_ast()call now exists:qiime2.sdk.util.type_from_ast. This andparse_typeare now significantly faster as loading plugins are no longer required to reconstruct the type.type_from_astis capable of completely reconstructing aTypeMap. - @thermokarst added a new utility to
qiime2.sdk.utilnamedparse_primitivewhich takes on the role of<sometype>.decode(val), however it can handle collection types of varying style. - Other type utilities exposed in
qiime2.sdk.utilinclude:is_semantic_type,is_primitive_type,is_metadata_type,is_visualization_type, andinterrogate_collection_type.
- @ChrisKeefe fixed a bug caused by invalid primitive arguments
- Library
- @thermokarst fixed a bug that prevented admins from editing plugins (due to an over-specified database rule).
- docs
- @Nicholas_Bokulich fixed a typo in one of the overview tutorial images.
- @Nicholas_Bokulich added an example using
q2-sample-classifier's newheatmappipeline to theq2-sample-classifiertutorial.
- dev-docs
- @ChrisKeefe busted out his editor's red pencil on the dev docs.
- @cduvallet updated the plugin development tutorial to reflect the latest versions of the packages used in the document!
- q2cli
- @epruesse cleaned up the
--versionoutput when looking at a plugin's details!
- @ebolyen put many hours of blood (
), sweat (
), and tears (
) into cleaning up the q2cli experience, changes include:
--cmd-confighas been removed (it was underdocumented and added a lot of complexity). We encourage users of QIIME 2 who need programmatic control to use the Python API instead which is significantly more flexible.--py-packageshas been removed fromqiime info(it was already broken). Useconda listinstead.
--output-dirand--o-<name>options now ensure the path is writable before executing the command.
--output-dirno longer creates the directory before running the command. This means no morerm -rf'ing because there was a typo in a different parameter.
- The underlying parser has been significantly improved. The following is now possible:
- Missing arguments will now be recognized instead of swallowing the next option (very confusing for beginners).
--helpis now "hyper-greedy", if it appears anywhere in the command, the help text will be shown (no matter how many errors exist otherwise).- Options that are repeatable may also now receive an arbitrary number of arguments:
can now also be written as:--p-something 1 --p-something 2 --p-something 3
This means that shell-expansion (like--p-something 1 2 3 # the original form is still supported*.qza) will become more useful:qiime feature-table merge --i-tables dada2-outputs/*.qza ...
- Missing arguments will now be recognized instead of swallowing the next option (very confusing for beginners).
- The CLI cache is now optimistically
stored in your conda environment. If q2cli is not running in a conda environment, then it will fall back to XDG_CONFIG_HOMEas before. This means that switching between multiple QIIME 2 installations will not constantly refresh the cache. difflibis now used to recommend options when a typo is made in a command.
- Error handling has been made significantly more consistent between plugin commands and builtins (such as
tools,dev, andinfo): - Errors are now enumerated. Parsing errors such as malformed commands will be re-parsed multiple times to collect as many errors as q2cli can (these are an incomplete list, and will be printed as
(1/5?)instead of(1/5))
- In order to support TypeMap, the
--helptext has a new layout system which includes the QIIME 2 type whenever relevant (this is especially helpful for ranged primitives). TypeMaps are indicated with a superscript number which will match between inputs and outputs. (In the future, a table may be a better way to represent this). - Help text is now colorized to improve readability
:
- @epruesse cleaned up the
https:/uploads/qiime21/original/2X/c/c83b5c662898d9addfee797017d1e087171042b8.webm
- q2-diversity
- @thermokarst fixed a bug
in the adonisvisualizer that prevented some metadata headers from working as expected. - @Nicholas_Bokulich fixed a bug in the
adonisvisualizer that caused an error if metadata values contained single quotes (e.g., apostrophes). - @Nicholas_Bokulich fixed a typo in the
adonisvisualizer help text.
- @thermokarst fixed a bug
- q2-longitudinal
- @Nicholas_Bokulich added a new
anovavisualizer to the plugin!
- The heatmap generated by
maturity-indexcan now be controlled to limit the number of important features displayed. (Thanks @Nicholas_Bokulich!)
- @Nicholas_Bokulich added a new
- q2-deblur
- @thermokarst added a new
left_trim_lenparameter to the denoising methods of this plugin!
- @thermokarst added a new
- q2-demux
- Fixed the per-sample sequence count table layout (that thing sure was bothering @ChrisKeefe!)
- @wasade brought Golay barcode correction to
emp-singleandemp-paired- the future is now!
- Fixed the per-sample sequence count table layout (that thing sure was bothering @ChrisKeefe!)
- q2-cutadapt
- @thermokarst added a new
minimum_lengthparameter totrim_singleandtrim_paired!
- The ability to
discard_untrimmedreads has been added totrim_singleandtrim_pairedby @thermokarst. - @thermokarst fixed a bug related to too many files being opened when demuxing datasets with many samples in it. Check out the new
batch_sizeparameter ondemux-pairedanddemux-singleto control how many samples are demultiplexed at a time!
- @ChrisKeefe & @thermokarst added support for dual-index demultiplexing! This functionality is enabled by preparing a second column in your Metadata file that contains the barcodes for the reverse reads. Check out the docs for more info!
- @thermokarst added a new
- q2-feature-classifier
- @BenKaehler & @Nicholas_Bokulich updated the default value for the
reads_per_batchparameter, in an effort to reduce memory usage!
- @BenKaehler & @Nicholas_Bokulich updated the default value for the
- q2-sample-classifier
- @Nicholas_Bokulich added a
heatmappipeline to display the (normalized) abundances of predictive features per sample or per group, usingFeatureData[Importance]scores output by supervised classifiers and regressors.
- @Nicholas_Bokulich updated the citation information to include the report describing this plugin in the Journal of Open Source Software.
- @Nicholas_Bokulich added a
- q2-feature-table
@gwarmstrong (GH)added a feature to allow disabling of clustering on both the sample and feature axes when creating aheatmap!
merge-taxanow supports multi-columnFeatureData[Taxonomy]artifacts, e.g., to preserve classification confidence scores in the merged output. (thanks @Nicholas_Bokulich)
- @turanoo overhauled the interactive even-sampling depth visualization --- it is now built with vega! There were several improvements/bugfixes related:
- Sample count summaries for each metadata group are now displayed in the plot
- SVG and PNG exports are enabled for the plot
- The visualization can be customized in the interactive vega editor
- A bug related to scrolling in the visualization's tabs was fixed
- As a result of TypeMap, it is now possible to filter out samples (
feature-table filter-samples) of feature tables with the variant types of:RelativeFrequency,PresenceAbsence, andComposition(Frequencyis of course still supported). The output type will match the input type provided.
- q2-quality-control
- @Nicholas_Bokulich fixed a bug in
evaluate-compositionthat caused linear regression R values of zero to be reported when observed and expected feature tables contain only a single observation at any taxonomic level. (the correct answer is no R value should be reported).
- @Nicholas_Bokulich fixed a bug in
- q2-dada2
- @benjjneb updated this plugin to use DADA2 1.10 - users can expect to see speed improvements from 2x to 20x!
! Check out the DADA2 docs for more details on this version of DADA2.
- @benjjneb updated this plugin to use DADA2 1.10 - users can expect to see speed improvements from 2x to 20x!
- q2-emperor
- This plugin now uses the latest version of emperor (
1.0.0-beta.19) - Miscellaneous bug fixes:
- Fixes a bug where re-centering the camera wouldn't re-center the view.
- Fixes a bug where two dimensional plots would fail to load.
- Fixes a bug where sample metadata would fail to load if only sample identifiers were included.
- Experimental support for exporting 2D vega plots has been added by @thermokarst ! This is currently only available for a subset of the available plot types, and can be accessed through the menu:
Any customization to the 3D plot should be inherited in the vega plot. For example, here is a customized 3D plot:
And here is the 2D vega version:
This functionality is still experimental --- please try it out and let us know how it goes for you!
- This plugin now uses the latest version of emperor (
- q2-types
- @thermokarst & @gregcaporaso revamped the FASTQ manifest formats! The new formats are TSV based, and produce files that are compatible with the Metadata specification.
Keep on QIIME-ing! 
