[Preview] QIIME 2 2022.8 development changelog

Developer Discussion
,
1

QIIME 2 2022.8 has been released!
Please see the official changelog for more details.

Original content of post

:exclamation: Important :exclamation:

The following is an early developer preview of the changes expected in 2022.8

This post is a live-document which will be updated throughout our development cycle. Any links will in this topic will be broken until the release is officially published. When we are ready for release, we’ll copy this changelog and create a new post in the Announcements category .

Important Developer Information

Dates (please keep an :eye: on this post, these :calendar: might change):

  • PRs must be submitted by: August 11th, 2022
  • PRs must be merged by: August 18th, 2022
  • Repo Freeze and Package Building: August 23rd, 2022
  • Release Day: August 26th, 2022

Developer Project Board:

QIIME 2 2022.8 Project on Github

Exciting Announcements!

  • Community Distribution! :package: :mailbox_with_mail:

    • NEW! In our continued efforts to support plugin developers and the QIIMEmunity (pun intended), we have added a new distribution to our infrastructure that includes all active, currently maintained community plugins :electric_plug: :bulb: This allows for all of these plugins to undergo regular integration tests so that we can quickly triage and debug any issues that may arise due to dependency conflicts or other unforeseen :beetle: :bug: !
    • Please note that we are still beta testing this distribution, so there is no formal release but we wanted to share the exciting news with all of you :nerd_face: :newspaper: To install, simply run the following command:
    conda create -n qiime2-community-2022.8-BETA \
  -c conda-forge -c bioconda \
  -c https://packages.qiime2.org/qiime2/2022.8/passed/community/ \
  qiime2-community
    • If you are a developer and you would like to see your plugin added to the community distribution, please get in touch with us! This is our goal for this project - to have a distribution with all of our community plugins available all in one place :slightly_smiling_face:

  • q2galaxy

    • NEW! We're very excited to announce that QIIME 2 is now officially supported in Galaxy release 22.05, which is now available! What this means is that you can upload QIIME 2 artifacts/visualizations and Galaxy will recognize our file format :card_file_box: This does not include any QIIME 2 actions, which is the next exciting announcement. :hushed:

    • We now have a Galaxy toolshed repository for QIIME 2 actions which can be used by any Galaxy instance running release 22.05 with Docker support. This makes it easy to install the latest QIIME 2 release directly into your Galaxy server straight from the official Galaxy Toolshed :milky_way: :hammer_and_wrench:

    • Please note that the toolshed will be made available along with the virtual machines on our usual post-release timeline of 1-2 weeks after release day :calendar:

:exclamation: BREAKING CHANGES :exclamation:

  • q2-feature-classifier

    @Nicholas_Bokulich completed a large refactor for improved readability and efficiency that includes the following new features:

    1. Separated blast + vsearch aligners from the consensus classification step. Now we have thin wrappers for database searching, as well as a separate action for generating consensus annotations from BLAST6Format search results :boom: :fire:
    2. classify-consensus-* methods are now pipelines, linking together the separate actions mentioned above :plunger:
    3. The Search results are output as part of the classify-consensus-* pipelines for convenient post-mortem :skull_and_crossbones:
    4. New + improved unit tests :test_tube: :chart_with_upwards_trend:

  • q2studio

    • DEPRECIATION NOTICE: q2studio will be removed from our user documentation and will no longer be maintained after the 2022.11 release. It is being phased out and will be replaced with a fully supported release of q2galaxy! :milky_way: :star2:

Here are the highlights of the release:

  • QIIME 2 Framework

    • @ebolyen added support for all INSDC missing values to be processed as NULL within QIIME 2 Metadata :mag_right: :thinking:
    • @misialq added support for typing.Union in view types by selecting the first available transformer. This adds the ability for transformers to solve to a typing type like Union which will make paired-end visualizations simpler :robot: :microbe:
    • @ebolyen fixed a long-standing annoyance for plugin developers by allowing for dictionaries to be registered outputs! :nerd_face: :books: Additionally, Evan added sorting for sets provided to Choices so that interfaces which cache the types will maintain a consistent ordering. :1234:

  • q2-cutadapt

    • @hyphaltip added multi-core support to demux-paired with the --p-cores parameter :apple: This speeds up demux dramatically based on the number of cores selected! :exploding_head: :rocket:

  • q2-dada2

    • @jordanrabasco submitted a huge refactor in q2-dada2 that combines all of the R scripts into one file for single, paired end, and ccs reads - for easier readability and reduced code duplication! :partying_face: :raised_hands:t3: :dna:
    • @leasi added the allowOneOff argument in the runBimeraDenovo() method, which enables users to allow for one mismatch or indel in bimera detection when set to True :space_invader: :dizzy:

  • q2-demux

    • @lizgehret and @Keegan-Evans added more concise error handling for metadata file handling when running qiime demux emp-single to include the specific column where any metadata.tsv (or sequences.qza) errors occur :page_with_curl: :interrobang:

  • q2-diversity

    • @yoshiki added support for sub-sampling with replacement in phylogenetic-core-metrics, which is useful for metagenomic studies where there's frequently over 1M counts per sample :100:
    • @ChrisKeefe updated the Halko citation and added a DOI field for a number of other citations :memo:

  • q2-diversity-lib

    • @ChrisKeefe added some Usage API examples on Faith's Phylogenetic Diversity, Observed Features, Pielou's Evenness, and Shannon's Entropy :mag: :chart_with_upwards_trend:
    • @wasade added new citations for in-press UniFrac and optimized Faith's PD papers :newspaper_roll: :bookmark:
    • @wasade made an improvement to how alpha diversity metrics are calculated. These calculations are now made either directly from sparse data, or operate on partitions to minimize memory use on dense conversions. :brain: :thinking:

  • q2-FMT (Available in the community distribution as a beta release)

    • NEW! Developed by @ebolyen and @lizgehret and currently maintained by @cherman2, this plugin provides tools for analyzing the efficacy of fecal microbiota transplants :poop: :microbe: The current methods that are supported are as follows:

      1. group_timepoints: Produce timepoint and reference distributions for use in statistical calculations, based on specified group data over discrete timepoints. Both alpha and beta diversity data are supported for this method.
      2. engraftment pipeline: Applies group timepoints to Fecal Microbiota Transplant data and then applies statistical tests (Wilcoxon signed rank test or Mann–Whitney U test) on alpha or beta diversity metrics to assess engraftment success.

  • q2-longitudinal

    • @lizgehret fixed a :bug: in pairwise-distances and pairwise-distances that raised an error if samples in the metadata but not the data were missing group data. Samples not present in the data (e.g., an alpha diversity vector or distance matrix), they wouldn't be included in the analysis, so missing group data is irrelevant for those samples. :chart_with_upwards_trend:

  • q2-metadata

    • @gregcaporaso added an action that supports the generation of shuffled metadata groups. This allows users to create one or more sample metadata columns by shuffling the values in an actual metadata column. The shuffled metadata columns can be useful for assessing whether an analysis of the corresponding actual metadata column is producing an interesting signal or should be considered noise. For example, if a differential abundance test identifies 10 features of interest with an actual metadata column, and 5-15 features of interest with a shuffled variant of that column, the actual test results should likely be considered unreliable. :card_file_box: :computer:

  • q2-stats (Available in the community distribution as a beta release)

    • NEW! Currently in development by @ebolyen and @lizgehret this plugin provides statistical tools, methods, and visualizers that can be used on a variety of different data. :chart_with_upwards_trend: :bar_chart: :computer:

  • q2-types

    • @Nicholas_Bokulich added a transformer that allows for BLAST6Format data to be transformed into QIIME 2 Metadata :boom: :firecracker:

Documentation:

  • @thermokarst published new developer tutorials on the following:

    • How to publish a plugin using Library :books: :open_book: :worm:
    • An overview of our packaging infrastructure :package: :spider_web:

  • @ChrisKeefe published a new developer tutorial on the usage API and how to write basic usage examples :nerd_face:

  • @ChrisKeefe published a new user tutorial on the Artifact API :gem: :sparkles:

5 Likes