[Preview] QIIME 2 2024.10 development changelog

Developer Discussion
,
1

:exclamation: Important :exclamation:

The following is an early developer preview of the changes expected in 2024.10

This post is a live-document which will be updated throughout our development cycle. Any links will in this topic will be broken until the release is officially published. When we are ready for release, we’ll copy this changelog and create a new post in the Announcements category.

Developer Project Board:

QIIME 2 2024.10 Project on Github

:bangbang: Important: Interface Changes in QIIME 2 20XX.REL :bangbang:

Here are the highlights of the release:

QIIME 2 View Updates :mag:

Distribution Updates :package:

Framework Updates :qiime2:

  • @Oddant1 made it so cache validation will now return false on things that aren't directories instead of erroring :file_cabinet:
  • @Oddant1 made parallel pipelines tolerant of mixed artifacts and proxies in a collection :abcd: :1234:
  • @Oddant1 made it so actions will now show up with their real names when run in parallel :railway_track:
  • @Oddant1 made the highthroughput executor the default instead of threadpool :hotsprings:
  • @Oddant1 made the QIIME 2 duplicate util detect when os.link is unsupported and copy instead :scroll:
  • @Oddant1 made it so Collection[Visualizaion] usage variables are properly typed.
  • @lizgehret fixed a :lady_beetle: in Artifact.import_data() where checksums weren't being stored when the view_type was a sub-class of FormatBase :abacus:
  • @yoshiki updated the jupyter server extension command (formerly jupyter serverextension) :ringer_planet:
  • @Oddant1 improved support for collections of visualizations and collections in general in the usage API :closed_book:

Interface Updates :tv:

  • q2cli

    • @Oddant1 added the --use-cache flag to all actions not just pipelines :money_with_wings:
    • @Oddant1 added plugin level citations to the --citation flag on actions :clapper:
    • @Oddant1 added information about the current default parallel config to the output of the qiime info command :gear:
    • @Oddant1 improved support for collections of visualizations and collections in general in the cli usage API :books:

  • q2galaxy

    • @bebatut added a bio.tools xref to rendered tools for QIIME 2, which will add support some new ontology operations (grouping and filtering tools in Galaxy).

Plugin Updates :electric_plug:

  • q2-dada2

    • @colinvwood changed the denoise-ccs --p-adapter parameter to optional to support a wider set of input data.

  • q2-diversity

    • @lizgehret fixed a :beetle: in the alpha-rarefaction visualization (due to an updated version of pandas) that removed all of the sequencing depth data :exploding_head:
    • @lizgehret fixed a :bug: in the bioenv visualizer that was dropping samples with rows containing any empty values. It now drops sparse columns instead :bar_chart: Thanks to @jacobs for reporting this! :raised_hands:t3:

  • q2-diversity-lib

    • @lizgehret fixed a :bug: in the faith-pd output that left #SampleID as the index name in the resultant alpha diversity vector. Thanks to @Mehrbod_Estaki for reporting this! :pray:

  • q2-fmt

    • @cherman2 added a to-baseline parameter in fmt group-timepoints and fmt cc. :poop: This new parameter will allow users to investigate engraftment by comparing diversity metrics to a baseline microbiome to illustrate microbial shifts away from baseline after FMT. :chart_with_downwards_trend:
    • @cherman2 refactored fmt peds-heatmap to encode sample size! This will help users determine which individuals/features have the best engraftment while considering population size of the engraftment! :bar_chart:
    • @cherman2 refactored group-timepoints so that it now accepts an additional group parameter (like a treatment group :dark_sunglasses: ). This means you can investigate the engraftment extent of any groups of interest and compare engraftment extent between your groups!
    • @cherman2 add an action called peds-simulation, this action randomizes the relationships between donors and recipients, to test whether the PEDS score between a recipient and their actual donor issignificantly higher than PEDS scores between other recipients paired with random donors. :star_struck: This method attempts to quantify the extent to which indicator features that are unique to a given donor are transferred to their recipients, as opposed to features that are not indicative of any specific donor. Note: PEDS Monte Carlo simulation may have dependency issues between samples and the simulated background distribution that can make the test overly conservative. This can be fixed by filtering down to a single timepoint before running this method. Additionally if there are many baseline timepoints, the global test may be too conservative and this can be addressed by filtering out baseline samples prior to running this method. :+1:

  • q2-longitudinal

    • @Oddant1 added tests to the volatility plot as part of an ongoing effort to test that visualizations are rendering correctly :chart_with_upwards_trend:

  • q2-types

    • @Sann5 added the ReferenceDB[HMMER] semantic type to allow for orthologue prediction with a HMMER database through eggNOG in q2-moshpit :metal:
    • @misialq refactored some of the existing GenomeData types and added some new types that will be utilized in q2-moshpit as follows:
      • Removed SampleData[BLAST6] in favor of a new type: SampleData[Orthologs]
      • Removed OG and KEGG types as they are unused in the current methods.
      • Added SampleData[Orthologs] and GenomeData[Orthologs] to store orthologs produced by EggNOG+Diamond.
      • Added GenomeData[DNASequence] to represent collections of genome sequences fetched from various sources.

  • q2-vizard

    • @lizgehret developed a new suite of visualization tools (included in the amplicon distribution) that can be used to visualize your metadata! :framed_picture: The following new visualizations are now available:
      • 2D scatterplot (qiime vizard scatterplot-2d)

Documentation Updates :open_book:

  • User Documentation
    • @peterjc fixed some outdated references of our continuous integration system to now accurately reflect our use of Github Actions :gear:
    • @lizgehret updated the installation instructions to now use a direct URL for the environment file. :open_book: This removes the need to install wget or download the file to your local machine! :tada:
    • @jphagen made improvements to the Cancer Microbiome Intervention Tutorial: creating a section for provenance replay in the tutorial index, adding CLI as the default driver, updating the taxonomic classifier link :crab:

QIIME 2 Library :books:

2 Likes
2

Our current development process for tracking and assigning incoming issues and pull requests can be found here :cowboy_hat_face:

1 Like