[Preview] QIIME 2 2018.6 development changelog

Developer Discussion
,
1

QIIME 2 2018.6 has been release!
Please see the official changelog for more details.

Original content of post

:exclamation: Important :exclamation:

The following is an early developer preview of the changes expected in 2018.6 This post is a live-document which will be updated throughout our development cycle. Any links will in this topic will be broken until the release is officially published. When we are ready for release, we’ll copy this changelog and create a new post in the Announcements category.

Important Developer Dates:

  • PRs must be submitted by: 06/15
  • PRs must be merged by: 06/19
  • Release Day: 06/21

The QIIME 2 2018.6 release is now live! There are lots of exciting new changes packed into this release that are described below.

Our next planned QIIME 2 release is tentatively scheduled for July or August 2018 (QIIME 2 2018.7 / 2018.8), but please stay tuned.

Check out the QIIME 2 2018.6 docs for details on installing the latest QIIME 2 release, as well as tutorials and other resources. Get in touch on the QIIME 2 forum 2 if you run into any issues!

Virtual machine builds are also available today - check out the docs for info on getting set up!

:exclamation: Important :exclamation:

We recently discovered a bug related to the q2-cutadapt plugin’s trim-paired command that, in certain situations, can produce incorrect results. In particular, the forward and reverse reads are swapped, causing the underlying cutadapt command to apply the forward read trimming parameters on the reverse reads (and vice versa).

Check out this post for more detail:

Incorrect results produced by q2-cutadapt's trim-paired

Here’s the highlights of the release:

  • QIIME 2 Framework

    • For developers: fixed a bug where the util.duplicate function failed due to lack of ownership/write-permissions. util.duplicate now copies files if these permission errors prevent hard linking. Thanks @ChrisKeefe! :balloon:
    • For interface developers: a new utility actions_by_input_type has been added to qiime2.sdk.util which can list plugin-actions by an input semantic type. Thanks @antgonza! :ice_cream:

  • docs

    • Check out the new overview tutorial, intended to give a broad overview of the workflows employed in QIIME 2 for marker gene amplicon sequence data analysis, and different options therein! This guide is meant as a somewhat non-technical primer for beginners, as well as a handy desk reference for expert users who are untroubled by excessive use of emoji. Most importantly, it is chock full of BEAUTIFUL FLOWCHARTS. Enjoy! :scream: :birthday:
    • The q2-longitudinal tutorial has been updated to include details on using fit vs. residual plots to evaluate linear-mixed-effects model quality and identify outliers.
    • The q2-longitudinal tutorial has been updated to include recommendations on choosing random vs. fixed effects.
    • The q2-longitudinal tutorial has been endowed with a BEAUTIFUL FLOWCHART of its own to give an overview of this plugin’s actions. :twisted_rightwards_arrows:
    • Added a link to the draft developer docs - thanks @cduvallet! :tada:

  • q2cli

  • q2-composition

    • Upgraded to maintain compatibility with latest release of pandas, version 0.23 ! :panda_face:

  • q2-diversity

    • Raw alpha diversity and metadata values plotted by the alpha-group-significance visualizer can now be downloaded to TSV in the resulting visualization! :see_no_evil:
    • Raw alpha diversity and metadata values plotted by the alpha-correlation visualizer can now be downloaded to TSV in the resulting visualization! :hear_no_evil:
    • Raw pairwise distances and sample metadata values plotted in the beta-group-significance visualizer can now be downloaded in the resulting visualization! :speak_no_evil:

  • q2-feature-classifier

  • q2-gneiss

    • Fixed a minor typo in the correlation-clustering docstring.
    • Upgraded to maintain compatibility with latest release of statsmodels, version 0.9.0 ! :clamp:

  • q2-longitudinal

    • The linear-mixed-effects visualizer now generates fit vs. residual plots for assessing model quality and identifying outliers! See the updated q2-longitudinal tutorial for more details.
    • Upgraded to maintain compatibility with latest release of statsmodels, version 0.9.0 ! :construction_worker_man:
    • Fixed a bug in linear-mixed-effects that caused an error when attempting to use feature names beginning with numerals as a metric. Thanks @JenKelly for spotting and diagnosing this bug! :bug:

  • q2-phylogeny

  • q2-quality-control

    • Added a new visualizer, evaluate-taxonomy! This visualizer calculates precision/recall/F-measure for a list of observed taxonomy assignments, compared to a list of expected taxonomy assignments for those features, and creates a simple plot for p/r/f at each taxonomic level.
    • evaluate-composition now calculates Bray-Curtis and Jaccard distance between expected and observed results. :bar_chart:

  • q2-sample-classifier

    • Classifiers and regressors can now be saved as artifacts! Two new methods — fit_classifier and fit_regressor — allow users to train classification and regression models (similar to classify-samples and regress-samples), and save those models as artifacts for re-use. These methods also identify and select predictive features.
    • The new predict method allows users to predict metadata values for samples in a feature table, using a trained classifier (from fit_classifier or fit_regressor) as input! :robot:
    • Two new methods, classify-samples-ncv and regress-samples-ncv, support fully nested cross-validated sample prediction. These methods work similarly to classify-samples and regress-samples, except that metadata values are predicted for all input samples, and accuracy and feature importance scores are averaged across each iteration. Among other uses, these methods can be used to predict mislabeled or abnormal samples within a dataset. :bird:
    • Most actions now have the option to either ignore or raise an error (default) when samples present in the feature table are missing from the sample metadata. The result of ignoring is that these missing samples will be silently dropped from the analysis… :shushing_face:
    • Feature tables are no longer transformed to pandas.DataFrames. Short story for users: q2-sample-classifier actions should now be faster and more efficient! :rocket:
    • The new split_table method randomly subsamples a feature table into training and test sets at a specified ratio. Splitting can be stratified on a specific metadata column. This method is used for splitting feature tables prior to training predictive models.

  • q2-taxa

    • Fixed a UX bug by improving error transparency in the collapse method. Users are now notified if the Feature IDs in their FeatureTable[Frequency] are missing from their FeatureData[Taxonomy]. Thanks @ChrisKeefe! :champagne:

  • q2-types

    • A new transformer from BIOMV210Format to BIOMV100Format was added. Thanks @turanoo! :tada:
    • A new transformer from FastqManifestFormat to pandas.DataFrame was added. This should prevent future occurrences of paired-end file ordering issues.

  • q2-demux

    • Added a new sequence length 7-number-distribution table to summarize - this is helpful when working with pre-joined reads. Thanks @jakereps! :keyboard:

Happy QIIME-ing! :sun_with_face:

1 Like
6

Just a quick reminder - today is the last day for PR submissions for the 2018.6 cycle. If you have something you really want to get in before next Thursday, please let us know, we can discuss timing! Our goal is to merge PRs by 06/19, and cut the release on 06/21. We are tentatively shooting for the following release some time in August, but that might change - stay tuned!

1 Like