[Preview] QIIME 2 2021.11 development changelog

[lizgehret]

QIIME 2 2021.11 has been released!
Please see the official changelog for more details.

Original content of post

Important

The following is an early developer preview of the changes expected in 2021.11

This post is a live-document which will be updated throughout our development cycle. Any links will in this topic will be broken until the release is officially published. When we are ready for release, we’ll copy this changelog and create a new post in the Announcements category.

---

Important Developer Information

Dates (please keep an on this post, these might change):

• PRs must be submitted by: 2021-11-05T07:00:00Z
• PRs must be merged by: 2021-11-16T07:00:00Z
• Repo Freeze and Package Building: 2021-11-22T07:00:00Z
• Release Day: 2021-11-23T07:00:00Z

Developer Project Board: 2021.11 · GitHub

---

BREAKING CHANGES

Please update scripts, workflows, etc. as appropriate. Stuck? Reach out here on the forum for help.

---

Here are the highlights of the release:

• QIIME 2 Framework

  • We are really excited to formally announce the release of the new QIIME 2 Usage API! This is a set of tools targeting developers and documentation writers , that will allow them to write interface-agnostic "usage examples" - snippets of code that different QIIME 2 interfaces can turn into meaningful interface-specific examples! Speaking of examples, here is a simple one, this demonstrates how to merge two feature tables using q2-feature-table:

    q2cli:

      qiime feature-table merge \
        --i-tables feature-table1.qza feature-table2.qza \
        --o-merged-table merged-table.qza
    

    Python 3 API:

    import qiime2.plugins.feature_table.actions as feature_table_actions
    
    merged_table, = feature_table_actions.merge(
        tables=[feature_table1, feature_table2],
    )
    

    The Usage API code that generated both of the interface-specific examples above is here:

    # ft1_factory and ft2_factory are functions defined elsewhere that
    # create data for this example.
    
    def feature_table_merge_example(use):
        feature_table1 = use.init_artifact('feature_table1', ft1_factory)
        feature_table2 = use.init_artifact('feature_table2', ft2_factory)
    
        merged_table, = use.action(
            use.UsageAction('feature_table', 'merge'),
            use.UsageInputs(tables=[feature_table1, feature_table2]),
            use.UsageOutputNames(merged_table='merged_table'),
        )
    

    So with one code snippet, plugin developers and tutorial authors can now write examples that users will be able to review and use in the interface of their choosing!
    We ported the Moving Pictures Tutorial to this new system, here is a quick preview of what users can expect to see:

    
    

    If you're interested in learning more, we have brand new developer documents. Also, here is the Moving Pictures Tutorial source, as well as an example of how to write and register Usage examples within an existing QIIME 2 Plugin.

• docs
  • A brand new Usage API enabled version of the Moving Pictures tutorial is now available as a beta preview - in future release we plan to update the remaining tutorials as well!
• q2cli
  • @ebolyen adjusted the terrible dark blue text to a nicer light blue color to reduce the incidence of eye bleeds amongst users of the default dark terminal color scheme.
  • @Keegan-Evans and @lizgehret added assert-result-type, assert-result-data, and some waffles to the dev tools. At-least two of those things anyways.
  • q2cli now has full support for Usage API - you can see Usage examples for any actions that have any (which admittedly isn't many, but more soon!) by running --help on your favorite command. An example:

    qiime feature-table merge --help
    ...
    Examples:
      # ### example: basic
      qiime feature-table merge \
        --i-tables feature-table1.qza feature-table2.qza \
        --o-merged-table merged-table.qza
    
      # ### example: three tables
      qiime feature-table merge \
        --i-tables feature-table1.qza feature-table2.qza feature-table3.qza \
        --p-overlap-method sum \
        --o-merged-table merged-table.qza
    

  • You can also save example data (--example-data) to run these commands with!

    qiime feature-table merge --example-data example_data
    Saved FeatureTable[Frequency] to: example_data/basic/feature-table1.qza
    Saved FeatureTable[Frequency] to: example_data/basic/feature-table2.qza
    Saved FeatureTable[Frequency] to: example_data/three-tables/feature-table1.qza
    Saved FeatureTable[Frequency] to: example_data/three-tables/feature-table2.qza
    Saved FeatureTable[Frequency] to: example_data/three-tables/feature-table3.qza
    

    Now you have data that can be used to "test out" the examples that are shown in the help text!
• q2-demux
  • @wasade added a test for barcode trimming with the reverse complement and moved the trimming logic to occur prior to the reverse complement.
• q2-diversity
  • @twollhoewer added two awesome new ordination techniques: t-SNE and UMAP

[thermokarst]

Hey all, we are going to push back this release to the second half of November - this'll give us some additional time to get some new features all set for you. Thanks for your patience!