[Preview] QIIME 2 2020.6 development changelog

q2d2 · March 5, 2020, 8:13pm

QIIME 2 2020.6 has been released!
Please see the official changelog for more details.

Original content of post

Important

The following is an early developer preview of the changes expected in 2020.6

This post is a live-document which will be updated throughout our development cycle. Any links will in this topic will be broken until the release is officially published. When we are ready for release, we’ll copy this changelog and create a new post in the Announcements category.

Important Developer Information

Dates (please keep an on this post, these might change):

PRs must be submitted by: June 9, 2020
PRs must be merged by: June 20, 2020
Repo Freeze and Package Building: June 25, 2020
Release Day: June 26, 2020

Developer Project Board: https://github.com/orgs/qiime2/projects/19

BREAKING CHANGES

q2-diversity
- core_metrics_phylogenetic: The n_jobs parameter has been renamed n_jobs_or_threads to accommodate a speedy new implementation of unifrac now being used
- core_metrics & core_metrics_phylogenetic: The phrase observed_features is replacing observed_otus, as "observed features" better describes the different ways in which users work with non-taxonomic features in QIIME 2 (an OTU is a feature, but a feature isn't necessarily an OTU)
- core_metrics & core_metrics_phylogenetic: n_jobs and n_jobs_or_threads no longer accept zero or negative integers as arguments. Users should pass auto to those parameters to use all available physical cores

Please update scripts, workflows, etc. as appropriate. Stuck? Reach out here on the forum for help. Want to test this prior to the next release of QIIME 2? Check on the "development environment" instructions here:

https://dev.qiime2.org/latest/quickstart/

Here’s the highlights of the release:

QIIME 2 Framework
- @thermokarst, @andrewsanchez, and @David-Rod worked up further developing the Usage API, which will allow plugin developers to showcase usage examples of their plugin actions, in an interface agnostic way!
  This included:
  - Enabling source tracking of initialized data
  - Adding a method to initialize data collections
- @thermokarst improved some of the developer documentation in the test harness.
- @Oddant1 increased level of validation of filenames to prevent creation of artifacts with incorrectly named data files.
docs
- @ChrisKeefe showed us how to peek at an Artifact in the Moving Pictures Tutorial
- @ChrisKeefe added a section to the Importing Tutorial for multiplexed fastq files with barcodes in sequences.
- @ChrisKeefe clarified the behaviors of qiime tools view and q2view
- @thermokarst added Native installation instructions for Windows users leveraging the Windows Subsystem for Linux
- @thermokarst added some rough guidelines for selecting an installation method that best fits your computational needs
- @thermokarst added examples of q2-demux's subsampling and sample filtering tools to the Atacama tutorial
- @Oddant1 Changed the wording of the suggestion to use the vega editor for the q2-longitudinal volatility plot to reflect the current implementation
- Several small typos were fixed, including one by @hmaru (GH)!
data-resources
- @Nicholas_Bokulich also added SILVA 138 release sequence and taxonomy files formatted for use with QIIME 2. Data were prepared and formatted using RESCRIPt (co-developed with @SoilRotifer)! See that tutorial for more details on how these files were generated, and a pipeline to reproduce this process!
- @Nicholas_Bokulich added new pre-trained classifiers for Greengenes 13_8 release and SILVA 138 release (the latest versions of both), trained using scikit-learn 0.23.1. These classifiers were built and tested in a new way, using RESCRIPt!
q2cli
- @thermokarst fixed up an error message: a tiny typo can cause a surprising amount of confusion!
- @andrewsanchez merged in the first iteration of the Usage API CLI driver . This should be considered as sort of "pre-release," for developers until the Usage API and usage examples are a bit more mature. Stay tuned for much more action in this realm in future releases!
QIIME 2 Studio
- @andrewsanchez fixed a bug that cropped up when trying to import data!
q2-feature-classifier
- @cherman2 (GH) helped remind us why we don't have multithreading options for the blast classifier - thanks for the research and code comments!
- @adamovanja (GH) added a trim-right option to the extract-reads action, allowing simulated reads to be trimmed at the 3' ends. This is useful for, e.g., emulating trimming options of reads coming from q2-dada2, to generate optimally trimmed reference sequences for classifier training as described here. Thank you adamovanja for fixing this issue, and @Sean_McKenzie for bringing it to our attention!
- @Nicholas_Bokulich improved the docs for extract-reads !
- @Nicholas_Bokulich trained new feature-classifiers using scikit-learn 0.23.1
q2-phylogeny
- @ebolyen added a new method, robinson-foulds, for comparing two phylogenetic trees!
q2-composition
- @thermokarst fixed a bug in ancom! The visualization had a propensity for (accidentally) attempting to divide by zero - never a good plan!
- @Oddant1 Removed the link to the vega editor on the ancom plot pending a rework of the plot for vega 5
q2-cutadapt
- @Oddant1 and @thermokarst added support for demultiplexing mixed-orientation single-indexed reads.
q2-feature-table
- @David-Rod added some usage examples to the merge method!
- @jwdebelius added a new method for re-naming features or samples in a table
- @thermokarst fixed a bug that caused summarize visualizations to sometimes display distorted column widths
- @Oddant1 Removed the link to the vega editor on the summarize plot pending a rework of the plot for vega 5
q2-alignment
- @gregcaporaso added a new action, mafft-add, which allows for the addition of unaligned sequences to an existing alignment
q2-quality-filter
- @thermokarst merged q-score and q-score-joined into the same method (q-score), and deprecated the q-score-joined method. This method will be removed in the fall 2020 release of QIIME 2.
q2-quality-control
- @Nicholas_Bokulich (aka Doc Bok) added a new filter_reads method that supports filtering SampleData[SequencesWithQuality] and SampleData[PairedEndSequencesWithQuality] artifacts (a.k.a. FASTQ sequence data) by alignment to a reference database. How might you use this? To filter human reads and other non-target (or junk) sequences prior to denoising or dereplication.
- @gwarmstrong transitioned alpha_phylogenetic and alpha_rarefaction to use Stacked Faith, a speed- and memory-optimized implementation previous available in alpha_phylogenetic_alt. alpha_phylogenetic_alt will be removed in the next release.
q2-diversity
- @patrickimran (GH) added a new jensenshannon beta diversity metric!
- @ChrisKeefe wired up q2-diversity-lib to perform alpha- and beta-diversity calculations for core-metrics and core-metrics-phylogenetic. See Breaking Changes above for related API changes.
- @Oddant1 fixed a bug allowing 0 variance numerical columns to slip past the filter in bioenv causing errors.
q2-diversity-lib
- @ChrisKeefe - this plugin calculates alpha- and beta-diversity measures, expands the number of available semantic types where appropriate, and provides measure-specific citation information.
q2-types
- @thermokarst added new transformers for working with per-sample-sequences in fun and interesting ways (dev alert!)
- @gregcaporaso added more aligned sequence transformers.
q2-vsearch
- @thermokarst fixed up a performance bug in the cluster-features-closed-reference method, resulting in potentially significant speedups for some use-cases.
- @angrybee (gh) added an awesome new visualizer fastq_stats for distilling single-end or paired-end sequences into FASTQ file statistics using vsearch!
- @Oddant1 added the ability for all vsearch methods to accept sequences shorter than 32nt
- @Oddant1 set --fasta_width 0 on invoked vsearch commands to prevent fasta records from being split across multiple lines
q2-longitudinal
- @Oddant1 removed the link to the vega editor from the volatility plot pending a rework of the plot for vega 5
q2-demux
- @Oddant1 and @thermokarst reworked summarize visualization to better inform the user of missing reads
q2-dada2
- @benjjneb added pseudo-pooling to all DADA2 workflows, allowing users to increase sensitivity to rare variants in samples, particularly singletons, as long as they were observed in other samples independently.
- @ebolyen squashed a bug where feature-tables containing only one feature were being mis-diagnosed as empty.
q2-sample-classifier
- @Nicholas_Bokulich gave the test suite some much needed TLC with a generous refactor.
q2-emperor
- @yoshiki and @antgonza updated to the latest release of Emperor 1.0.1 .
- @yoshiki fixed an issue where biplot arrows would incorrectly show while visualizing a parallel plot.
- @fedarko fixed an issue where continuous and divergent colormaps would be partially used in categorical data.
- @fedarko fixed a bug with sorting numeric and non-numeric values.
- @yoshiki fixed an issue where axes settings wouldn't load correctly from the Python API.
- @fedarko and @yoshiki updated various dependencies.
- Added the ability to select a group of samples and save the sample names to the users' clipboard. To do this, users need to hold shift and select the samples they are interested in:
  
  notes1319×952 238 KB

q2d2 · May 15, 2020, 10:41pm

Hi all - we are pushing this release back by a month (please see revised dates, above) - things have been pretty busy over here at Q2HQ, and we could use a little bit of breathing room to ensure we get the best release possible out the door for y'all. Thanks for understanding!

thermokarst · June 8, 2020, 2:48pm

Today is the last day to submit PRs for consideration in the 2020.6 release cycle. If you're cookin' something up and it isn't on our radar yet, please let us know about it, that way we can get it worked into our release plan - thanks!