# \[Preview\] QIIME 2 2019.4 development changelog

**URL:** <https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148>\
**Category:** Developer Announcements\
**Tags:** preview, changelog\
**Created:** [February 1, 2019, 10:46pm UTC](https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148 "2019-02-01T22:46:57Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![thermokarst](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/t/8e7dd6/32.png) [@thermokarst](https://forum.qiime2.org/u/thermokarst)\
**Post date:** [February 1, 2019, 10:46pm UTC](https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148/1 "2019-02-01T22:46:57Z")

</div>

QIIME 2 2019.4 has been released!  
[Please see the official changelog for more details.](https://forum.qiime2.org/t/qiime-2-2019-4-is-now-available/9513)

> **Original content of post**
>
> # ❗ Important ❗
> 
> **The following is an early developer preview of the changes expected in 2019.4** This post is a live-document which will be updated throughout our development cycle. Any links will in this topic will be broken until the release is officially published. When we are ready for release, we’ll copy this changelog and create a new post in the [Announcements category](https://forum.qiime2.org/c/announcements).
> 
> ### Important Developer Dates (please keep an 👁 on this post, these 📅 might change):
> 
> - PRs must be submitted by: April 24, 2019
> - PRs must be merged by: April 26, 2019
> - Repo Freeze and Package Building: April 29, 2019
> - Release Day: April 30, 2019
> 
> Here’s the highlights of the release:
> 
> - [QIIME 2 Framework](https://github.com/qiime2/qiime2)
> - @ChrisKeefe fixed a bug caused by invalid primitive arguments 3️⃣🙅‍♂️. Expanded type mismatch error messages to better describe various invalid-input situations. 🗣
> - @ChrisKeefe fixed some pretty confusing language in the `SemanticType` docstring.
> - [`@Sleightly (GH)`](https://github.com/Sleightly/) has added a new SDK utility named `build_graph` to `qiime2.sdk.actiongraph`. This will generate a directed NetworkX graph of QIIME 2 plugin actions and types. The graph is bipartite, with types on one conceptual side, and actions on the other. Edges between them indicate parameters.
> - The implementation of QIIME 2 types has been reworked by @ebolyen 🥋 . Some minor changes include:
> - `Properties(['a', 'b'])` =\> `Properties('a', 'b')`
> - `Range(X, None)` can be optionally written as `Start(X)` (it is converted to a `Range`)
> - `Range(None, X)` can be optionally written as `End(X)` (also converted to `Range`)
> - `Choices({'a', 'b'})` =\> `Choices('a', 'b')` (this means order is now retained as well)
> 
> - Unions of primitive types are now possible, thanks to @ebolyen! This means you can create a plugin with a parameter such as: `Int % Start(0) | Str % Choices("ALL", "SOME", "NONE")`
> - Intersection (`&`) of types is now possible 🛑 however most intersections will result in a "bottom" type - thanks for that, @ebolyen! (This operator is primarily for implementing internal type solving algorithms.)
> - real intesection: `Start(0) & End(10)` =\> `Range(0, 10)`
> - contradiction: `FeatureTable[Frequency] & Phylogeny[Rooted]` =\> `UnionExp([])` (an empty union is the smallest, emptiest, type in QIIME 2)
> 
> - The framework now supports different output types based on the received input type 🍇 . This feature is called `TypeMap` (and it has a sibling `TypeMatch`, and was created by @ebolyen):
> - This allows expressions such as:
> 
> ```python
> T, P, R = TypeMap({
> (TypeA, Bool % Choices(True)): OutputA,
> (TypeB, Bool % Chocies(False)): OutputB
> })
> 
> ```
> 
> - For a direct example of TypeMatch, see implementation for [feature-table filter-samples](https://github.com/qiime2/q2-feature-table/blob/163e838ea29c3416fb5c14f1ef1cc2f2fc81b921/q2_feature_table/plugin_setup.py#L239).
> 
> - A utility to parse a type from the result of a `to_ast()` call now exists: `qiime2.sdk.util.type_from_ast`. This and `parse_type` are now significantly faster as loading plugins are no longer required to reconstruct the type. `type_from_ast` is capable of completely reconstructing a `TypeMap`.
> - @thermokarst added a new utility to `qiime2.sdk.util` named `parse_primitive` which takes on the role of `<sometype>.decode(val)`, however it can handle collection types of varying style.
> - Other type utilities exposed in `qiime2.sdk.util` include: `is_semantic_type`, `is_primitive_type`, `is_metadata_type`, `is_visualization_type`, and `interrogate_collection_type`.
> 
> - [Library](https://library.qiime2.org)
> - @thermokarst fixed a bug that prevented admins from editing plugins (due to an over-specified database rule).
> 
> - [docs](https://docs.qiime2.org/2019.4)
> - @Nicholas_Bokulich fixed a typo in one of the overview tutorial images.
> - @Nicholas_Bokulich added an example using `q2-sample-classifier`'s new `heatmap` pipeline to the `q2-sample-classifier` tutorial. 📑
> 
> - [dev-docs](https://dev.qiime2.org/latest/)
> - @ChrisKeefe busted out his editor's red pencil on the dev docs.
> - @cduvallet updated the plugin development tutorial to reflect the latest versions of the packages used in the document! 🗃
> 
> - [q2cli](https://docs.qiime2.org/2019.4/interfaces/q2cli/)
> - @epruesse cleaned up the `--version` output when looking at a plugin's details! 🌸
> - @ebolyen put many hours of blood (💉), sweat (💦), and tears (😢) into cleaning up the q2cli experience, changes include:
> - `--cmd-config` has been removed (it was underdocumented and added a lot of complexity). We encourage users of QIIME 2 who need programmatic control to use the Python API instead which is significantly more flexible.
> - `--py-packages` has been removed from `qiime info` (it was already broken). Use `conda list` instead. 🐉
> - `--output-dir` and `--o-<name>` options now ensure the path is _writable_ before executing the command. ✍
> - `--output-dir` no longer creates the directory before running the command. This means no more `rm -rf`'ing because there was a typo in a different parameter. 📂
> - The underlying parser has been significantly improved. The following is now possible:
> - Missing arguments will now be recognized instead of swallowing the next option (very confusing for beginners). 😮
> - `--help` is now "hyper-greedy", if it appears anywhere in the command, the help text will be shown (no matter how many errors exist otherwise).
> - Options that are repeatable may also now receive an arbitrary number of arguments:
> 
> ```bash
> --p-something 1 --p-something 2 --p-something 3
> 
> ```
> 
> can now also be written as:
> 
> ```bash
> --p-something 1 2 3 # the original form is still supported
> 
> ```
> 
> This means that shell-expansion (like `*.qza`) will become more useful:
> 
> ```bash
> qiime feature-table merge --i-tables dada2-outputs/*.qza ...
> 
> ```
> 
> - The CLI cache is now optimistically 🦋 stored in your conda environment. If q2cli is not running in a conda environment, then it will fall back to `XDG_CONFIG_HOME` as before. This means that switching between multiple QIIME 2 installations will not constantly refresh the cache.
> - `difflib` is now used to recommend options when a typo is made in a command. 🤷‍♀️
> - Error handling has been made significantly more consistent between plugin commands and builtins (such as `tools`, `dev`, and `info`):
> - Errors are now enumerated. Parsing errors such as malformed commands will be re-parsed multiple times to collect as many errors as q2cli can (these are an incomplete list, and will be printed as `(1/5?)` instead of `(1/5)`) 💯
> - In order to support TypeMap, the `--help` text has a new layout system which includes the QIIME 2 type whenever relevant (this is especially helpful for ranged primitives). TypeMaps are indicated with a superscript number which will match between inputs and outputs. (In the future, a table may be a better way to represent this).
> - Help text is now colorized to improve readability 🌈 :
> 
> https:/uploads/qiime21/original/2X/c/c83b5c662898d9addfee797017d1e087171042b8.webm
> 
> - [q2-diversity](https://docs.qiime2.org/2019.4/plugins/available/diversity/)
> - @thermokarst fixed a bug 🐛 in the `adonis` visualizer that prevented some metadata headers from working as expected.
> - @Nicholas_Bokulich fixed a bug in the `adonis` visualizer that caused an error if metadata values contained single quotes (e.g., apostrophes). 🦋
> - @Nicholas_Bokulich fixed a typo in the `adonis` visualizer help text. ⚗
> 
> - [q2-longitudinal](https://docs.qiime2.org/2019.4/plugins/available/longitudinal/)
> - @Nicholas_Bokulich added a new `anova` visualizer to the plugin! 🎉
> - The heatmap generated by `maturity-index` can now be controlled to limit the number of important features displayed. (Thanks @Nicholas_Bokulich!) 🚒
> 
> - [q2-deblur](https://docs.qiime2.org/2019.4/plugins/available/deblur/)
> - @thermokarst added a new `left_trim_len` parameter to the denoising methods of this plugin! 🚵‍♂️
> 
> - [q2-demux](https://docs.qiime2.org/2019.4/plugins/available/demux/)
> - Fixed the per-sample sequence count table layout (that thing sure was bothering @ChrisKeefe!) 🍥
> - @wasade brought Golay barcode correction to `emp-single` and `emp-paired` - the future is _now_! ⚾
> 
> - [q2-cutadapt](https://docs.qiime2.org/2019.4/plugins/available/cutadapt/)
> - @thermokarst added a new `minimum_length` parameter to `trim_single` and `trim_paired`! ✂
> - The ability to `discard_untrimmed` reads has been added to `trim_single` and `trim_paired` by @thermokarst. 🍇
> - @thermokarst fixed a bug related to too many files being opened when demuxing datasets with many samples in it. Check out the new `batch_size` parameter on `demux-paired` and `demux-single` to control how many samples are demultiplexed at a time! 🥦
> - @ChrisKeefe & @thermokarst added support for dual-index demultiplexing! This functionality is enabled by preparing a second column in your Metadata file that contains the barcodes for the reverse reads. Check out [the docs](https://docs.qiime2.org/2019.4/plugins/available/cutadapt/demux-paired/) for more info!
> 
> - [q2-feature-classifier](https://docs.qiime2.org/2019.4/plugins/available/feature-classifier/)
> - @BenKaehler & @Nicholas_Bokulich updated the default value for the `reads_per_batch` parameter, in an effort to reduce memory usage! 🧠
> 
> - [q2-sample-classifier](https://docs.qiime2.org/2019.4/plugins/available/sample-classifier/)
> - @Nicholas_Bokulich added a `heatmap` pipeline to display the (normalized) abundances of predictive features per sample or per group, using `FeatureData[Importance]` scores output by supervised classifiers and regressors. 🧯
> - @Nicholas_Bokulich updated the citation information to include the report describing this plugin in the Journal of Open Source Software.
> 
> - [q2-feature-table](https://docs.qiime2.org/2019.4/plugins/available/feature-table/)
> - [`@gwarmstrong (GH)`](https://github.com/gwarmstrong) added a feature to allow disabling of clustering on both the sample and feature axes when creating a `heatmap`! 🔥
> - `merge-taxa` now supports multi-column `FeatureData[Taxonomy]` artifacts, e.g., to preserve classification confidence scores in the merged output. (thanks @Nicholas_Bokulich) 👹
> - @turanoo overhauled the interactive even-sampling depth visualization --- it is now built with [vega](https://vega.github.io/)! There were several improvements/bugfixes related:
> - Sample count summaries for each metadata group are now displayed in the plot
> - SVG and PNG exports are enabled for the plot
> - The visualization can be customized in the interactive vega editor
> - A bug related to scrolling in the visualization's tabs was fixed
> 
> - As a result of TypeMap, it is now possible to filter out samples (`feature-table filter-samples`) of feature tables with the variant types of: `RelativeFrequency`, `PresenceAbsence`, and `Composition` (`Frequency` is of course still supported). The output type will match the input type provided.
> 
> - [q2-quality-control](https://docs.qiime2.org/2019.4/plugins/available/quality-control/)
> - @Nicholas_Bokulich fixed a bug in `evaluate-composition` that caused linear regression R values of zero to be reported when observed and expected feature tables contain only a single observation at any taxonomic level. (the correct answer is no R value should be reported). 🐛
> 
> - [q2-dada2](https://docs.qiime2.org/2019.4/plugins/available/dada2/)
> - @benjjneb updated this plugin to use DADA2 1.10 - users can expect to see speed improvements from 2x to 20x! 🏎 🔥 🍠! Check out the [DADA2 docs](https://benjjneb.github.io/dada2/) for more details on this version of DADA2.
> 
> - [q2-emperor](https://docs.qiime2.org/2019.4/plugins/available/emperor/)
> - This plugin now uses the latest version of emperor (`1.0.0-beta.19`)
> - Miscellaneous bug fixes: 🐛
> - Fixes a bug where re-centering the camera wouldn't re-center the view.
> - Fixes a bug where two dimensional plots would fail to load.
> - Fixes a bug where sample metadata would fail to load if only sample identifiers were included.
> 
> - Experimental support for exporting 2D [vega plots](https://vega.github.io/vega/) has been added! This is currently only available for a subset of the [available plot types](https://docs.qiime2.org/2019.4/plugins/available/emperor/plot/), and can be accessed through the menu:  
> ![image](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/2X/d/d26bad33adacce4729b6272d7210c6f049f07f39.png)  
> Any customization to the 3D plot should be inherited in the vega plot. For example, here is a customized 3D plot:  
> ![image](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/2X/f/f2748bbb42e39670a82141fbce97771578edb0ef.png)  
> And here is the 2D vega version:  
> ![visualization](https://forum-qiime2-org.s3.dualstack.us-west-2.amazonaws.com/original/2X/2/2c137ad8af6f9ffd8dd81d2d08e0c13c5af2c074.png)  
> This functionality is still experimental --- please try it out and let us know how it goes for you! 🎈
> 
> Happy QIIME-ing! 🌞

---

<div class="post-metadata">

**Author:** ![devonorourke](https://forum.qiime2.org/user_avatar/forum.qiime2.org/devonorourke/32/3110_2.png) [@devonorourke](https://forum.qiime2.org/u/devonorourke)\
**Post date:** [April 9, 2019, 11:25pm UTC](https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148/2 "2019-04-09T23:25:01Z")

</div>

> [@thermokarst](#):
>
> [q2-cutadapt](https://docs.qiime2.org/2019.4/plugins/available/cutadapt/)
> 
> - @thermokarst added a new `minimum_length` parameter to `trim_single` and `trim_paired` ! ✂

Still time to throw in a `--discard-untrimmed` parameter with this update @thermokarst? One of the things I realized was invaluable in my workflow was using the [linked adapters](https://cutadapt.readthedocs.io/en/stable/guide.html#linked-adapters) feature in Cutadapt; however, I only wanted to keep the reads that were framed by those 5' and 3' primers, and that required passing that flag (I think/hope!?).

Perhaps a future request for 2019.8?

Thanks for the improvements everyone!

---

<div class="post-metadata">

**Author:** ![thermokarst](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/t/8e7dd6/32.png) [@thermokarst](https://forum.qiime2.org/u/thermokarst)\
**Post date:** [April 10, 2019, 12:01am UTC](https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148/3 "2019-04-10T00:01:13Z")

</div>

> [@thermokarst](#):
>
> The ability to `discard_untrimmed` reads has been added to `trim_single` and `trim_paired` by @thermokarst. 🍇

💣

Ask, and you shall receive.

---

<div class="post-metadata">

**Author:** ![thermokarst](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/t/8e7dd6/32.png) [@thermokarst](https://forum.qiime2.org/u/thermokarst)\
**Post date:** [April 15, 2019, 4:12pm UTC](https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148/4 "2019-04-15T16:12:26Z")

</div>

Just a friendly reminder, PRs must be submitted by April 19, 2019 in order to be considered for the 2019.4 release. Keep us posted if you have incoming work that might not be on our radar - thanks! :qiime2:

---

<div class="post-metadata">

**Author:** ![thermokarst](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/t/8e7dd6/32.png) [@thermokarst](https://forum.qiime2.org/u/thermokarst)\
**Post date:** [April 17, 2019, 11:57pm UTC](https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148/5 "2019-04-17T23:57:11Z")

</div>

Scratch that, we are going to postpone the release by a few days so that we have time here to wrap up a few bigger projects:

- PRs must be submitted by: April 24, 2019
- PRs must be merged by: April 26, 2019
- Repo Freeze and Package Building: April 29, 2019
- Release Day: April 30, 2019

---

<div class="post-metadata">

**Author:** ![thermokarst](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/t/8e7dd6/32.png) [@thermokarst](https://forum.qiime2.org/u/thermokarst)\
**Post date:** [April 30, 2019, 11:13pm UTC](https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148/6 "2019-04-30T23:13:39Z")

</div>

Hey all - we are still wrapping up a few exciting new features, so we are going to postpone the release by a few days. We hope to get it out as soon as possible --- please stay tuned! 🦖

---

<div class="post-metadata">

**Author:** ![yoshiki](https://forum.qiime2.org/user_avatar/forum.qiime2.org/yoshiki/32/16_2.png) [@yoshiki](https://forum.qiime2.org/u/yoshiki)\
**Post date:** [May 3, 2019, 6:24pm UTC](https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148/8 "2019-05-03T18:24:45Z")

</div>



---

<div class="post-metadata">

**Author:** ![thermokarst](https://forum.qiime2.org/letter_avatar_proxy/v4/letter/t/8e7dd6/32.png) [@thermokarst](https://forum.qiime2.org/u/thermokarst)\
**Post date:** [May 3, 2019, 6:34pm UTC](https://forum.qiime2.org/t/preview-qiime-2-2019-4-development-changelog/8148/9 "2019-05-03T18:34:10Z")

</div>

Thanks for being patient --- we think the few extra days will be worth the wait! Head on over to the [2019.4 Announcement](https://forum.qiime2.org/t/qiime-2-2019-4-is-now-available/9513) to learn more about some of the exciting new features in this release!