I have a issue with my diversity analysis in the 2019.4 release. my jaccard plot in the 2019.1 release, it had many distinct clusters representing my treatments, but in the 2019.4 release, my plot is a dogs breakfast with no distinctive clustering (all treatments are mixed and overlapping with all the other treatments). I did not see any other forum topics discussing this issue
Both of the qiime2 releases were installed as environments in Miniconda3 v4.7.5.
Hey @BuildableDuck,
To the best of my knowledge, nothing should have changed here.
Would you be able to share the two emperor plots? The provenance information would be useful to compare to make sure nothing funny happened in the middle on accident.
1 Like
Here are my plots, from what I can see in the provenance, the time taken to complete each task varied a little at some of the steps, and the version of qiime is different; the 4clust plot is the one using the 2019.1 release. both of the plots were generation using the same script with only a different version of qiime.
jaccard_emperor_starfield.qzv (860.4 KB) 4clustjaccard_emperor.qzv (858.8 KB)
Thank you for sharing, @BuildableDuck! Your source manifests are different (different filenames and different content) - can this difference be explained by any potential changes there? Maybe you could share those files, or, your demux summaries? Otherwise, I agree, the rest of the workflow looks to be the same to me. Keep us posted! 
The names of the Manifest are indead different, but the sequences imported are the same (the different name is only so i didnt overwrite the test on our cluster). the contents of the deblur stats also looks the same.
I have attached the manifests and deblur stats.
2019-07-01_Experment_1_STARFIELD_TEST.d.denoising-stats.qzv (212.7 KB) 2019-07-01_Experment_1_STARFIELD_TEST.Manifest.qiime2.tsv (55.1 KB) 2019-07-02_Experment_1_4cluster_TEST.d.denoising-stats.qzv (213.0 KB) 2019-07-02_Experment_1_4cluster_TEST.Manifest.qiime2.tsv (54.7 KB)
Have you run diversity beta-rarefaction? I am wondering if your even sampling depth of 5000 is just too low to be stable with these data... (but, that feels like a bit of a reach). To be frank, I am a bit stumped. @ebolyen?
I just ran this (not 100% sure what the most correct parameters are) and found that this also shows the starfield/cloud affect from the earlier empeor plots. Could there be an error in beta rarefaction itself?
qiime diversity beta-rarefaction
--i-table STARFIELD/4cluster.qza
--p-metric jaccard
--p-clustering-method nj
--m-metadata-file
--p-sampling-depth 50000/5000
--p-iterations 5
--i-phylogeny rooted-tree.qza
--o-visualization beta-rarefaction.qzv
2019-07-01_Experment_1_STARFIELD_TEST.beta-rarefaction5k.qzv (850.8 KB) 2019-07-01_Experment_1_STARFIELD_TEST.beta-rarefaction50k.qzv (837.0 KB) 2019-07-02_Experment_1_4cluster_TEST.beta-rarefaction5k.qzv (848.8 KB) 2019-07-02_Experment_1_4cluster_TEST.beta-rarefaction50k.qzv (834.9 KB)
Alrighty, it appears there was an issue with the installation of the environment. I had reinstalled the environment as a "release_test" and used by starfield.qza for a jaccard emperor plot. it appears normal. I will rerun the full pipeline and if its still fine, ill declare the problem solved 
Fustrating though it is, there was an issue with the qiime2-2019.4 installation. I set up a fresh install, reran all of the commands and the plots look the same within reason. Thanks for the help anyway, hopefully this thread will help shed light on similar issues with other peeps.
Can you tell us what was wrong with the old install? Or how it got to be that way?
Unfortunately, I can't rightly say. I had a look at the directory sizes and the faulty installation had a total size of only 44MB whereas the fresh environment was 4.2GB. Conda doesn't leave logs, so either it dropped the ball or the channels required for the installation were not available at the time (strangely, I do not recall any errors during the installation either).
1 Like