[Preview] QIIME 2 2017.12 development changelog

Developer Discussion
,
1

QIIME 2 2017.12 has been released!
Please see the official changelog for more details.

Original content of post

:exclamation: Important :exclamation:

The following is an early developer preview of the changes expected in 2017.12 This post is a live-document which will be updated throughout our development cycle. Any links will in this topic will be broken until the release is officially published. When we are ready for release, we'll copy this changelog and create a new post in the Announcements category.

Important Developer Dates:

  • PRs must be submitted by: 12/15
  • PRs must be merged by: 12/19
  • Release Day: 12/21

The QIIME 2 2017.12 release is now live! There are lots of exciting new changes packed into this release that are described below.

QIIME 2 2017.12 is the last QIIME 2 release before we end our support of QIIME 1. As a reminder, QIIME 2 will succeed QIIME 1 on January 1, 2018. QIIME 1 will no longer be officially supported at that time, as development and support effort for QIIME will be focused entirely on QIIME 2. To prepare for this transition, we have been working toward "feature parity" with QIIME 1 over the last year. We have made a huge amount of progress toward this goal, but there may still be some things that you want to do with QIIME 1 that are not possible with QIIME 2. If you identify things that you did with QIIME 1 but can't do with QIIME 2, please get in touch about them on the QIIME 2 forum. We'll be happy to help with work-arounds as we wrap up loose ends.

Because we still anticipate a lot of new features going into QIIME 2 over the next few months, we will continue to have monthly releases for at least the first quarter of 2018. We highly recommend staying up-to-date with the latest releases to get access to new features, bug fixes, documentation, etc. We will only provide user support for the latest monthly release of QIIME 2 as these monthly releases continue. Later in 2018, we expect to develop and implement a strategy for "long-term support" releases that we will support for longer periods of time.

Check out the QIIME 2 2017.12 docs for details on installing the latest QIIME 2 release, as well as tutorials and other resources. Get in touch on the QIIME 2 forum if you run into any issues!

We will provide official conda builds on release day for native QIIME 2 installations. Virtual machine builds such as AWS and Docker will be released within the following week. We will post a follow-up announcement here when those builds become available.

Here's the highlights of the release:

  • q2-cutadapt
    • This new plugin wraps cutadapt! :tada:
    • demux-single supports demultiplexing single-end reads, where the barcodes are still inline in the sequence.
    • demux-paired supports demultiplexing paired-end reads, where the barcodes are still inline and occur on the 5' end of the forward read.
    • trim-single & trim-paired support finding and removing adapters in demultiplexed reads.
  • q2-dada2
    • Upgraded dada2 to the latest, version 1.6! This version brings many performance enhancements, so if you previously experienced slow q2-dada2 performance, check out this new version and see for yourself!
    • Added addition statistics to stdout log (--verbose for users of q2cli) that indicates how many reads were filtered, denoised, merged, and non-chimeric.
  • q2-feature-classifier
    • In classify-sklearn, the reads-per-batch parameter is now automatically scaled based on the number of query sequences and n_jobs, making parallel jobs with this action much more efficient (without requiring user input). Autoscaling is set by default (when reads-per-batch == 0), but this parameter can still be defined by the user (e.g., to reduce batch size to address memory constraints). :brain:
  • q2-feature-table
    • Fixed a bug that accidentally allowed users to specify a negative rarefaction depth in the summarize visualization. Thanks @patthehat033! :tada:
    • Fixed a performance issue in the summarize visualization - tables with lots of features should now render much more quickly (orders of magnitude faster in our tests).
    • merge now accepts arbitrarily many tables provided via the new tables input (table1 and table2 have been removed as input).
    • merge-seq-data has been renamed to merge-seqs and now accepts arbitrarily many FeatureData[Sequence] artifacts via the new data input (data1 and data2 have been removed as input).
    • merge-taxa-data has been renamed to merge-taxa and now accepts arbitrarily many FeatureData[Taxonomy] artifacts via the new data input (data1 and data2 have been removed as input).
    • filter-seqs can now take a FeatureTable[Frequency] artifact as an optional input, in which case it filters features that are not present in the FeatureTable[Frequency] artifact (or, if exclude-ids is passed as well, it filters features that are present in the FeatureTable[Frequency] artifact).
  • QIIME 2 Framework
    • Variadic input is now supported in the framework for Artifact inputs. Plugin developers can use List[...] and Set[...] with a semantic type to indicate that they would like to receive 1 or more artifacts with that type.
      • Set is unordered but unique
      • List is ordered but not necessarily unique
      • View annotations (function annotations) for a method are unchanged. So for a registration of List[FeatureTable[Frequency]] the view annotation would still be biom.Table (or whatever you prefer). In the future we would like this to be typing.List[biom.Table] but there are some technical challenges preventing that at this time.
  • docs
    • The minimum disk space requirements recommendation for the VirtualBox images have been revised --- recent versions of QIIME 2 are larger than the versions that existed at the time.
    • Added a step to the VirtualBox install guide to demonstrate how to set up shared copy-and-paste between the host and guest machines.
    • Updated the VirtualBox install guide to make users aware of VirtualBox version requirements.
  • q2cli
    • Help text has been rearranged, this should make the types and descriptions more readable.
    • Variadic input is now supported for input artifacts. The help text will show MULTIPLE ARTIFACT PATH or MULTIPLE <TYPE> to indicate that the parameter can be provided multiple times.
  • q2-quality-filter
    • Removed the visualize-stats visualizer in favor of viewing QualityFilterStats as Metadata, which means users can now visualize these stats with the generic metadata tabulate visualizer! :tada:
    • Added support for SampleData[PairedEndSequencesWithQuality] as an additional input type to q-score - reverse reads are ignored and not included in the output (so the output is always SampleData[SequencesWithQuality]). This is similar to how dada2 denoise-single behaves - if you provide paired-end reads it only operates on the forward reads.
  • q2-emperor
    • Bug Fixes
      • Fix a problem where cubes would be rendered in a slightly different color from all the other shapes.
      • Fix a bug where plots would rotate slightly when switching tabs, or changing a visible attribute.
    • New Features
      • Add the ability to copy sample names to the clipboard. This can be done in one of two ways: by clicking on a sample and then clicking on the bottom left sample name legend, or by double-clicking a sample.
      • Add the ability to resize the plot. This can be done by clicking and dragging the line between the plot and the tabs space. Double clicking will toggle the visibility of the menu. See this animated gif:
      • emperor
        emperor982×697 427 KB
      • New shapes are now available in the Shapes tab.
      • Add the ability to create animated PCoA plots via the Animations tab. This features is useful to visualize how multiple samples change over a gradient. This tutorial explains how to use this functionality, and this this paper explains the motivation for this feature.
  • q2-demux
    • A bug in summarize has been fixed in interactive tab where the shortest global read would be reported in the legend instead of the shortest sampled read. The plot was not impacted by this bug.
  • q2-longitudinal
    • Updated plugin to cite the preprint for the upcoming q2-longitudinal paper.
  • q2-diversity
    • Fixed a bug which would crash core-metrics-phylogenetic when n-jobs was set to any value other than 1.
    • beta-rarefaction now supports jackknifed PCoA plots with Emperor, and sample clustering (with support calculation) via neighbor joining and UPGMA. Two new required parameters were added to that end: metadata (same as always) and clustering-method (accepts "nj" or "upgma").
  • q2-vsearch

Happy QIIME-ing! :sun_with_face: