QIIME 2 2019.10 is now available!

[q2d2]

The QIIME 2 2019.10 release is now available! Thanks to everyone involved for their hard work!

As a reminder, our next planned QIIME 2 release is scheduled for late February 2020 (QIIME 2 2020.2), but please stay tuned for updates.

Check out the QIIME 2 2019.10 docs for details on installing the latest QIIME 2 release, as well as tutorials and other resources. Get in touch on the QIIME 2 Forum if you run into any issues!

Virtual machine builds will be available sometime within the next week - watch this topic thread for an update!

Here’s the highlights of the release:

• QIIME 2 Framework
  • @Oddant1 added a view method to the TextFileFormat and BinaryFileFormat classes, which allows for easier type conversion (especially for developers!).
  • @Nicholas_Bokulich updated the citation info for the canonical QIIME 2 citation (Bolyen et al., 2019 Nature Biotechnol)
  • @David-Rod upgraded our core distribution to use the latest (and greatest) version of Pandas !
  • @Oddant1 implemented a highly sought after feature - Metadata constructed via the Artifact API now gets all of its whitespace stripped off ! This is really useful for the dozens of transformers that allow for various semantic types to be "viewed" as Metadata!
  • @ebolyen fixed a bug that prevented MetadataColumnvalues from being decoded properly - this was mostly an issue for q2studio.
  • @thermokarst made some changes to a framework utility that uses the networkx package --- QIIME 2 is now compatible with the latest release of this tool!
• docs
  • @Nicholas_Bokulich updated the tutorials to reflect changes to the feature-table heatmap API (primarily the parameter rename from metadata to sample-metadata).
  • @SoilRotifer added the tutorial Phylogenetic inference with q2-phylogeny. This is the latest version of his Community Tutorial of the same name
  • @BenKaehler added a nifty new section to the Parkinson's Mice Tutorial that demonstrates how to create a bespoke classifier.
  • @ChrisKeefe found a pesky typo in the Parkinson's Mice Tutorial dataset , but fortunately those days are behind us!
  • @thermokarst created a new "SEPP Refs" section of the Data Resources page on the QIIME 2 Docs - these handy little databases are for use with q2-fragment-insertion!
  • @gregcaporaso re-wrote the User Glossary of QIIME 2 terms - if you have ever puzzled over our definitions of words like "Artifact" or "Pipeline" - this is the page for you!
• Library
  • @ebolyen, chief drop-shadow design guru for the QIIME 2 team did what he does best: beefed up the drop shadow on the Library!
• q2cli
  • @David-Rod fixed a bug that caused help text generation to slow down significantly! For the average user this wasn't much of a problem, but, if you're in the business of building QIIME 2 docs, this slowdown was significant.
  • @Oddant1 fixed a bug that prevented exporting to a local path. We think Rumpelstiltskin was involved - but that's neither here nor there - @Oddant1 got us sorted out!
  • @ebolyen fixed a bug that caused things to explode when saving files into a non-existent directory!
  • @Oddant1 improved the error message presented when an invalid primitive value is identified --- previously the error message was pretty opaque, now it actually tells you what is wrong!
• q2-feature-classifier
  • @Oddant1 exposed new n_jobs and batch_size parameters in the extract_reads method allowing for parallelization of the method.
  • @Oddant1 changed extract-reads so it will now ensure that min-len is greater than trunc-len minus trim-left before executing.
• q2-sample-classifier
  • @Oddant1 added the heatmap visualizer to the party known as the classify-samples pipeline.
  • @Nicholas_Bokulich fixed a bug in confusion-matrix that caused ROC plots to fail on unstratified, imbalanced data.
  • @Nicholas_Bokulich added cividis to the list of available colormaps.
  • @Oddant1 added the ability to adjust color scale of the confusion-matrix heatmap.
  • @David-Rod made some updates to the transformers defined in this plugin to allow it to work with the latest versions of Pandas !
  • @Nicholas_Bokulich added a new drop_all_unique parameter to the metatable action - this prevents metadata with all unique columns from ruining the party
  • @ChrisKeefe fixed a bug in the heatmap visualizations produced in this plugin that caused some cells to be trimmed, which looked pretty weird!
• q2-feature-table
  • @Nicholas_Bokulich added cividis to the list of available colormaps for heatmap.
  • @Nicholas_Bokulich updated heatmap to accept both sample-metadata and feature-metadata, enabling sample/feature labeling along each axis.
• q2-longitudinal
  • @Nicholas_Bokulich added cividis to the list of available colormaps. Wow, @Nicholas_Bokulich was really excited about this colormap!
  • @Nicholas_Bokulich made the plot-feature-volatility visualization a lot more performant by building-in a simple "filter" system - by default only the top 100 most important features will be displayed. There are a few other mechanisms for filtering importances - check out the docs here!
• q2-gneiss
  • @Oddant1 created a new ignore_missing_samples parameter in the gradient_clustering method that, well, allows for any missing samples in the metadata to be, ahem ignored.
  • @ebolyen made some updates to the test suite defined in this plugin to allow it to work with the latest versions of Pandas !
• q2-types
  • @David-Rod removed white space and comment support from TaxonomyFormat and TSVTaxonomyFormat. These elements generally caused more problems than they solved, so we decided to part ways with those aspects of the formats.
  • @Oddant1 made it so DNAFASTAFormat disallows duplicate records, disallows whitespace at the start of an ID, and a few other housekeeping elements.
  • @David-Rod made some updates to the tests defined in this plugin to allow it to work with the latest versions of Pandas !
  • @David-Rod beefed up the validation for TSVTaxonomyFormat - users will be presented with much clearer error messages whenever there is a problem importing or creating one of these files!
• q2-deblur
  • @Oddant1 Made it a new warning to prevent allowing underscores in sample ids (which deblur can't handle.
  • @EmFord (GH) got rid of a leftover license file lurking in the source code for this plugin!
• q2-emperor
  • This release is using the latest release version of emperor (1.0.0-beta.20).
  • Miscellaneous bug fixes: and performance improvements :
    • Improve error messages for missing or mismatching feature metatdata.
    • For large plots fix a problem where hidden objects would still be clickable.
    • Fixes a bug where axes orientation would be incorrectly loaded from a settings file.
    • Daniel Hakim: Improved performance for animations with a large number of timepoints.
  • New features :
    • Add a search bar to every tab to narrow down the values you are interested in for any metadata column.
    • Xuhan Yang: Adds a list of colors to the color selector based on the currently selected color palette.
    • Daniel Hakim: Add the ability to visualize multiple dimensions using parallel plots. Go to the Axes tab and click on Parallel or Scatter to change the plot type.
    • The following animated GIF showcases all three new features:
      
      
• q2-diversity
  • @tomasz implemented a new intersect_ids feature in the alpha-correlation visualizer. This parameter allows for filtering out IDs in sample metadata or the SampleData[AlphaDiversity] that aren't found in both inputs (which would previously terminate as an error).
  • @yoshiki got the core-metrics/core-metrics-phylogenetics pipelines outfitted with the ability to subsample-with-replacement while rarefying!
  • @ebolyen fixed a bug in the beta-group-significance visualization that caused some of the boxplots and PDFs to not show up. This was primarily caused by having any kind of non-alphanumeric characters in the metadata used to compute the visualization:
• q2-vsearch
  • @colinbrislawn hooked us up with a sweet new feature in join-pairs --- the ability to specify how many execution threads with which to get the job done! Neat!
• q2-taxa
  • @David-Rod took a magnifying glass to the barplot visualizer in this plugin - lo and behold, a simple, yet important typo was revealed - an error used the word "feature" when it should've used the word "sample". Funny how such a small change in language can yield such different results!
• q2-fragment-insertion
  • @thermokarst & @Stefan brought SILVA 128 to the sepp method, and there was much rejoicing! This method no longer contains a "default" reference (it used to be GreenGenes) - now users can grab a handy SEPP database reference from the Docs (pick your favorite - SILVA or GG!). In the future we hope to expose new methods to support importing and prepping any database, stay tuned ! For a glimpse of this new action, well, uh, in action, check out the Parkinson's Mouse Tutorial!
• q2-dada2
  • @Oddant1 updated the DADA2StatsFormat to include a few new computed columns, that way you don't have to do the mental math to figure out the percentage of reads merged. Phew!

Happy QIIME-ing!

[jwdebelius (Justine Debelius)]

Congratulations to all of you! These are amazing!!! I cant wait to use the new release.

[HebaHussein-1981 (Heba Hussein)]

Thanks for the update. Is it mandatory to update 201.7 qiime? is it mandatory to update each update? and do we need to uninstall or delete the old version first?

[q2d2]

Wow! We have a special treat for you - VMs are fresh out of the oven ! Head on over to the docs for more details!

[ben (benjamin w.)]

You can delete the conda environment, but it's generally recommended to go to the new version. I don't think it's necessary to move to the update just yet, but it may be necessary if you use a plug-in that's been updated. Ben

@HebaHussein-1981

Thanks for the update. Is it mandatory to update 201.7 qiime? is it mandatory to update each update? and do we need to uninstall or delete the old version first?

[sanda]

Congratulations to all. If new version is released then do I need to repeat my analysis using new version?

Regards

[yitkheng]

Thank you for the updates.

[Deni_Ribicic]

Congrats guys, and great effort!
Looking forward to novelties.

[jwdebelius (Justine Debelius)]

@ben,

Its funny because one of the things I love about conda is how many qiime 2 versions i have. I like to work on a single version per project (unless there’s a good reason to switch, like a particular update) and so I keep an enviroment for each project until I’m done and then save that with the analysis scripts. That way, even if its not the most up to date (and it wont be when I publish because there’s always a lag between analysis and publication) I know exactly when I did things. (I actually name my environments differently based on each project.)

So, my advice to @HebaHussein-1981 and @sanda would be to keep whatever environment you’ve been working on for this analysis unless there’s a good reason to update (bug, compatibility, new plugin, critical new feature) and install the latest version so you have the shiny features for your next analysis.

Best,
Justine

[sanda]

@jwdebelius, Thank you so much for your sincere suggestions. I appreciate that. Now I am enjoying using Qiime2. Kudos to Qiime team @ben.
Many thanks and regards

[Eman (Eman Khalaf)]

Hi all,
Congrats on the new release and thanks for your continuous support to the microbiome people. I have a question in this regard; I was working in a project since last year and for some reason publication was postponed. I updated qiime environment many times (started with V2018. and currently I have qiime2-2109.4. If I installed the new version, should I re-do all the analyses (will I find significant differences in results?) . In general, I used to install new versions but I use it for new running projects.
Thanks
Eman

[HebaHussein-1981 (Heba Hussein)]

thanks for the advice. or this analysis, I use only 6 samples to practice the steps and will run same codes within two days on my whole ~200 samples. So, in this case for my ~200, is it recommended to use the current version as I know the codes or is it better to update so in the publication, I mention it is the latest version? Will be difference in the commands themselves in the update?

[HebaHussein-1981 (Heba Hussein)]

thanks for the advice. for this current analysis as you know, I use only 6 samples to practice the steps and will run same codes within two days on my whole ~200 samples. So, in this case for my ~200, is it recommended to use the current version as I know the codes or is it better to update so in the publication, I mention it is the latest version? Will be difference in the commands themselves in the update?

[ben (benjamin w.)]

@HebaHussein-1981

I would say, if it is not too costly for parallel runs, there’s no issue running them in parallel conda environments. However, unless there was critical plug-in update, there’s not really a hurry to move to the new version. So, I would compete the analysis with the version you’re using (2019.7), and then run and compare results with 2019.10. Ben

[thedam (Damian Loska)]

where can I find a robust 16S rRNA end-to-end tutorial for analysing pair-end data, that includes new Qimme2 version?

[Susie]

Hi all,

I would like to share some experience during analyzed the data through QIIME2. I was attending the NIH QIIME2 workshop in 2018 that was very useful ( I learned a lot ) and I had one issue when I analyzed the data and I asked Evan Bolyen in the workshop , he helped me to solve the issue and be able to run the QIIME2 in my computer. In addition, I asked online help , Nicholas and Mathew were guide me very well to be able run the QIIME2 without any issue. I hope my experience very useful for you and good luck. ~Suad

[sanda]

Hi @thedam,
As a beginner, I suggest you to go through “Moving pictures tutorial” and “Atacama soil microbiome tutorial”. It helps a lot. All information are very clear, only you need to choose parameters according to your sequencing details. You can also find in Moving pictures tutorial that on what basis and how to choose parameters.
Best wishes

[jwdebelius (Justine Debelius)]

Hi @HebaHussein-1981,

The change log describes what has changed and the tutorials and help documentation are always a good place to start for whether or not the commands are different! You may still need to pay attention to paths and be careful how you work, but that’s not a QIIME specific issue.

Best,
Justine

[Eman (Eman Khalaf)]

Hi @ben,
While reading the tutorial of the new version, I found a recommendation to use single-end seq since the reads are too short:
“Our samples were amplified using the EMP 515f-806r primers and sequenced on an Illumina MiSeq with a 2x150bp kit. The hypervariable region covered by the primers we used is 290bp long, so with 150bp reads our sequences will be slightly too short to be able to do paired-end analysis downstream. Therefore, we’re going to work with single-end sequences”

Actually, the company that does sequencing for our samples is currently using the same primer set 515f-806r and all of our running projects and future projects will be treated the same. Does that mean it is not recommended to follow the analysis script of paired-end?
Thanks
Eman