Interface changes coming in 2024.2, and a test build of the current QIIME 2 distributions

Announcements
1

:boom: This December the QIIME 2 team opted to forego the planned 2023.11 release in favor of reducing our long queue of pull requests, and working on some new functionality. :boom:

For users who are interested in getting the latest changes (but can tolerate some potential instability as well as some of the interface changes listed below), you can use the Prepared Environment Files that will be linked below shortly. We're very interested in testing and feedback on the Updates listed below, so please let us know if anything isn't working for you. These environment files do not constitute a QIIME 2 release, and you do not need to upgrade your QIIME 2 deployment(s) at this time if you don't want to. All of the changes in these environments will be included in the next planned QIIME 2 release, 2024.2 (scheduled for February 2024).

:bangbang: Important: Upcoming interface changes in QIIME 2 2024.2 :bangbang:

In the 2024.2 release the following interface changes will go into effect:

  • All existing action parameters that configure the number of threads or cpus to be used by an action will have their names and input domains standardized. For example, --p-num-threads, --p-n-jobs, --p-threads, will all be renamed to a single parameter and will all share the same input domain (i.e., --p-). The updated parameter name and domain will be announced at the time of the release. This will support consistency in how actions are parallelized in QIIME 2.
  • The merge-pairs method in the q2-vsearch plugin will output unmerged reads via the --o-unmerged-sequences option. Calls to qiime vsearch merge-pairs will now require that either --o-unmerged-sequences or --output-dir be provided.
  • The summarize method in the feature-table plugin will no longer accept the FeatureTable[RelativeFrequency] semantic type as input. Upon review of the contents we realize that much of the information presented (e.g., summarized feature frequencies) didn't make sense (they weren't wrong, there just wasn't a meaningful interpretation).
  • The subsample method in the feature-table plugin will be renamed to avoid confusion with the rarefy and *-rarefaction methods. We will report on the new name in the 2024.2 release announcements (we're still discussing options).

Prepared Environment Files

Below you'll find links to the resolved environment files representing the most recent versions of the amplicon, shotgun, and tiny distributions.

  • [Amplicon distribution] COMING SOON
  • Shotgun distribution
  • [Tiny distribution] COMING SOON

These can be installed by following the usual QIIME 2 install instructions, but substituting these environment files for the ones that are typically downloaded in that process. Again, these environment files do not constitute a QIIME 2 release, and you do not need to upgrade your QIIME 2 deployment(s) at this time if you don't want to.

Updates

The following updates for the 2024.2 release are already in place, and you'll have access to these if you install QIIME 2 from the environment files linked above. These notes will be expanded on for the 2024.2 release.

  • qiime2

    • @Oddant1 Fixed a bug that allowed a cache to be created in an existing directory :card_file_box:
    • @Oddant1 Added support for more parsl features in the parsl config :gear:

  • q2-assembly

    • @Oddant1 Added parallel support to the assemble-megahit, index-contigs, and map-reads-to-contigs actions :world_map:

  • q2cli

    • @Oddant1 Added the qiime tools cache import command that allows data to be imported directly into an artifact in the cache on the cli :luggage:
    • @Oddant1 Fixed a bug making it impossible to start the path to keyed collection member inputs with ~ :key:
    • @ebolyen and @colinvwood exposed the --validate-level option on the import and cache-import commands, allowing users to customize the extent to which imported data is validated. This is useful for importing exceptionally large datasets for which full validation is time-prohibitive. The same parameter is exposed in the python API as well. Passing --validate-level min to qiime tools import can be used to speed up imports relative to the default (--validate-level max) for formats where minimal validators exist. If you'd like to see support for a minimal validator where one doesn't exist, please file an issue on q2-types. In your issue, please include details on the runtime of your qiime tools import command with and without the --validate-level min parameter.

  • q2-cutadapt

    • vaamb Exposed the --p-cut option on the demux-single and demux-paired commands to support libraries with poly-N tails :dna:
    • vaamb Added support for anchored adapters to the demux-single and demux-paired commands for higher confidence and faster demultiplexing :running_man:

  • q2-demux

    • @Oddant1 Fixed a bug causing NaNs to appear in the demux summarize viz :hash:

  • q2-feature-table

    • @isaiah-ghost Added the summarize-plus pipeline that returns feature frequencies, sample frequencies, and the summarize viz. This represents some experimental changes we're making to the widely used summarize action in q2-feature-table, and this is ready for you to experiment with. Please note that b/c this new visualizer is experimental, we may still make some interface changes. If you'd rather wait for things to stabilize, the summarize action continues to work mostly as it did before, but see our note above about the qiime feature-table summarize interface change coming in 2024.2. :heavy_plus_sign:

  • q2-moshpit

    • @Oddant1 Added parallel support for the classify-kraken2 action :space_invader:

  • q2-vsearch

    • @colinvwood exposed the --o-unmerged-sequences output to allow users to retain the unmerged paired-end reads as an artifact for further analysis.
    • vaamb exposed the --p-strand option to the cluster-features-de-novo command to add support for mixed orientation reads.

  • q2-quality-control

    • @jordenrabasco fixed a bug in decontam-identify that disallowed special characters in users' metadata.

Happy 2024! :sparkler:

As always, thanks so much to the QIIME 2 users and developers for all you that do, in research, education, and beyond, to make the world a better place! We hope you get to relax and spend time with your loved ones over the holidays. :hugs:

We're looking forward to :qiime2:'ing with you in 2024! :tada: :tada: :tada:

10 Likes