[Preview] QIIME 2 2018.11 development changelog

Developer Discussion
,
1

QIIME 2 2018.11 has been released!
Please see the official changelog for more details.

Original content of post

:exclamation: Important :exclamation:

The following is an early developer preview of the changes expected in 2018.11 This post is a live-document which will be updated throughout our development cycle. Any links will in this topic will be broken until the release is officially published. When we are ready for release, we’ll copy this changelog and create a new post in the Announcements category.

Important Developer Dates (please keep an :eye: on this post, these :calendar: might change):

  • PRs must be submitted by: 11/01
  • PRs must be merged by: 11/06
  • Release Day: 11/08

Here’s the highlights of the release:

  • QIIME 2 Framework
    • A pre-print announcing :qiime2: has been published! If you use :qiime2: in your research, please cite this pre-print until a peer-reviewed article is accepted for publication (the provenance citation for the framework has been updated, so if you use this version of QIIME 2 you will automatically have the preprint citation embedded in provenance!). Stay tuned for more details! :partying_face:
    • Artifacts now have checksums stored in the archive — this will help preserve data integrity of QIIME 2 artifacts from now 'til the end of days. :derelict_house: For example, here is what happens when someone tampers with a Visualization (in this example, foo.qzv):
      47529769-8f262500-d85d-11e8-9c95-91ce9feeb170
      47529769-8f262500-d85d-11e8-9c95-91ce9feeb170715×196 20.3 KB
    • Type-mismatch errors have been improved by @ChrisKeefe:
      32%20AM
      32%20AM760×204 21.8 KB
  • QIIME 2 Library
    • An all new site for sharing and learning about QIIME 2 Plugins. We have many exciting plans for this site, so please stay tuned. In the meantime, check out the announcement for more details.
    • Several new plugins have been developed in the last few months. Rather than announce them here, we encourage you to check out the Library to learn all about them!
  • q2cwl
    • An all-new prototype interface for automatically generating CWL tools from QIIME 2 actions. This interface is not currently distributed as part of the "Core Distribution" for now, but please check out the README for more info!
  • docs
  • dev-docs
    • Newly released developer documentation is available at https://dev.qiime2.org. We still have a lot of work to do on API docs, but this is a good first step!
  • q2cli
    • qiime tools validate can now validate QZV files! Hurrah! :confetti_ball:
  • q2-feature-table
    • @ChrisKeefe - Added sequence lengths and descriptive stats including a seven-number summary to feature-table tabulate-seqs :bar_chart:
    • The latest version of biom is now used (2.1.7).
    • @sdbrown added subsampling with replacement capability to feature-table rarefy. :partying_face:
  • q2-phylogeny
    • IQ-TREE associated enhancements. By @SoilRotifer! :man_dancing::microbe::
      • --p-fast: This resembles the fast search of FastTree. Available only in iqtree.
      • Added access to single branch test methods: SH-aLRT via --p-alrt, aBayes via --p-abayes, and local bootstrap test via --p-lbp. These methods can be used as an alternative or in conjunction with bootstrapping approaches. Available in both iqtree and iqtree-ulfrafast-bootstrap, and all can be used simultaneously.
      • --p-n-runs: This allows multiple independent phylogenetic inferences to take place. The single best tree from these runs will be retained. Available in both iqtree and iqtree-ulfrafast-bootstrap.
      • --p-bnni: This option reduces the risk of overestimating branch supports via ultrafast bootstrap due to severe model violations. Available only through iqtree-ultrafast-bootstrap.
  • q2-feature-classifier
    • extract-reads: Added min_length and max_length parameters to support filtering simulated amplicons based on size.
    • classify-consensus-vsearch can now allow maxaccepts=0 as a parameter setting; this will cause VSEARCH to accept all database hits for performing consensus taxonomy assignment.
    • The classify-consensus-vsearch and classify-consensus-blast method and parameter descriptions have been updated to clarify how these operate, especially regarding the misleading behavior of BLASTn max_target_seqs (maxaccepts).
  • q2-sample-classifier
    • split-table: fixed a bug to support unstratified feature table splitting.
    • metatable: added a new method for converting a metadata file into a feature table, and optionally merging it with an existing feature table.
    • q2-sample-classifier is now published! read all about it. :newspaper:
  • q2-longitudinal
    • linear-mixed-effects can now accept R-style formulae for specifying fixed effects/interactions. This action can still accept fixed effects as a comma-separated list for the formula-adverse.
    • pairwise-distances: fixed a bug whereby metadata categories present in the metric sample metadata column but not represented by the samples in the distance matrix would be displayed as an empty column in the output boxplots.
  • q2-emperor
    • Adds an option to ignore missing samples (--p-ignore-missing-samples) when samples are not included in the metadata file but are present in the coordinates file. Thanks to @mortonjt.
    • Fixes a bug where the Red-Yellow-Blue colormap showed as a grayscale.
    • Fixes a compatibility problem with newer versions of the Jupyter notebook environment.
  • q2-diversity
    • beta-phylogenetic-alt has been removed and is superseded by beta-phylogenetic, which now uses striped UniFrac (this is a faster implementation of vanilla UniFrac). :film_strip:
    • Canberra-Adkins has joined the family of available beta diversity metrics. This is a normalized variant of the Canberra metric.
  • q2-fragment-insertion
    • This plugin is now a part of the core :qiime2: distribution! :christmas_tree:
    • Check out this tutorial for examples on how to use this plugin!
  • q2-gneiss
    • ilr-phylogenetic can once again accept FeatureTable[Composition] artifacts.
  • q2-types
    • Manifest format validation has been updated and improved! :balloon:
      • Validation now provides detailed error messages about what went wrong and why when QIIME 2 runs into problems importing.
      • Whitespace is now stripped from fields, which has caused many problems in the past for users trying to get their data loaded up.
      • This format now verifies that the files referenced actually exist at the locations indicated in the manifest, prior to importing --- reducing the number of painful import errors! :skull:
  • q2-demux
    • subsample-single and subsample-paired methods have been added to this plugin, which allow for subsampling SampleData[SequencesWithQuality] and SampleData[PairedEndSequencesWithQuality] into smaller datasets --- useful for preparing tutorial data, or for asking for help here on the Forum.

Happy QIIME-ing! :sun_with_face:

3 Likes
2

Two notes:

One, we are bit behind on updating the changelog, rest assured, work is being done!

Two, PRs to official QIIME 2 repos need to be submitted by the end of the day on November 1st — if you need a hand with anything or have questions, please let us know!

6

Ran into some technical difficulties today that slowed the release down, but rest assured, it is coming along quite nicely. Expect a follow-up sometime on Friday, unless some catastrophic failure occurs. :qiime2: :fire:

1 Like