Reporting issues/bugs ongoing list?

First, I’m really liking Qiime2! Great job to you all. :slight_smile: I’m going through analyses and I occasionally run into minor things that don’t seem worthy of opening up an issue or I’m not sure where to report it.

The pairwise-distances.qzv link is broken here:
https://docs.qiime2.org/2017.9/tutorials/longitudinal/?highlight=longitudinal

So trying to figure out where to report that led me to:

That says I should contact you via Qiime2 website, but then were do I go from there?

Do you guys have an ongoing list of bugs and suggestions made by users? Seems like that’d be nice. I just thought I would recommend that because it doesn’t seem like there is anywhere where you guys openly invite bug reports from your users. I realize bug reports are often annoying, incorrect or already in the queue, but at the same time you don’t want them to go unreported if a user finds them.

Thanks @Jennifer_Fouquier

For this particular bug, I’ve opened an issue to address this (thanks for reporting!).

You are absolutely right: even if it’s minor, it still needs to be reported somewhere but it can often be confusing for users to know where to report these issues (and if users do try to post is on github, that could lead to duplicate and cross-posting, or reporting non-bugs, etc)

IMO, the forum is the best place for users to report any and all issues they are having, so that the developers can diagnose and decide the appropriate place to raise an issue, but @admins might have a different opinion on this.

The QIIME 2 documentation does actually note where to report issues on this page

2 Likes

Yes, the forum sounds good. Maybe even one special thread on the forum for reporting small issues. I guess my hesitation is that it seems like overkill to open up an entire thread for posting a very small issue and I’m guessing others feel the same way.

Maybe you could add the bug reporting link to this page: https://forum.qiime2.org/faq for clarity. I was reading the forum notes, not the Qiime2 documentation. Thanks @Nicholas_Bokulich!

I agree, in general we'd prefer that users and developers post to the forum first, and then we can create bug reports on the relevant GitHub issue tracker(s). The reason is that sometimes what seems like a simple bug report requires some back-and-forth to diagnose the problem, and sometimes these "bugs" end up not being bugs at all. The forum seems best suited for these types of conversations, which can then be migrated to issues when necessary. Also, users and developers often have trouble identifying the appropriate issue tracker to report their problem, and we end up having to migrate the issue anyways, which takes time and clutters the issue tracker.

The issue with one big forum thread (or one big GitHub issue) is that it'll be an ever-expanding list, making it hard to identify discrete tasks, which tasks are dependent or blocked by other tasks, which are complete, etc. Also, if conversation is necessary for certain issues reported in the thread, there will be potentially many conversations happening at the same time. I think the monolithic thread would become spaghetti pretty quickly. We've tried similar approaches in the past and it doesn't work well -- usually what happens is the monolithic issue is conveniently ignored :slight_smile:

2 Likes

Oops, that page tells users to post bug reports on the QIIME 2 framework's issue tracker, which is usually the last place we want those issues filed :grin: I created an issue to point users to the forum on that page. Thanks @Nicholas_Bokulich and @Jennifer_Fouquier!

2 Likes

Cool, I get why you don’t like the long running list. I guess when I’ve used them in the past they were written by our outreach person and then a developer relocated them to their own issue so it was still controlled more in-house than by users. I understand! :slight_smile:

Maybe you need to just make a new section/category on the main forum page for bugs and issues. Sorry I’m waaaayyy overcomplicating this, but it just doesn’t feel like there’s a welcoming spot on the forum page either to report issues or bugs. The word ‘issue’ is only listed under ‘ideas and suggestions’ and ‘bug’ is mentioned zero times. That’s part of why I’ve gone to Github a few times. :grin:

We don't have a separate category for bug/issue reports because there is often back-and-forth between devs and users in order to determine if the issue is actually a bug or something else. We thought it'd be simpler to keep a single User Support category that is reserved for any user questions about QIIME 2 (including possible bugs), and then we can migrate topics onto issue trackers as necessary.

I think most of the confusion here is that our Community docs text is outdated (see my previous post for the bug report), and the User Support category description isn't clear. We'll fix the Community docs text in the upcoming 2017.11 release, and in the meantime I fixed the User Support description to include bug reports:

image

With these doc fixes in place, do you think this makes it clearer where users should seek help and post bug reports?


A little tangential but may be of interest to you: We also have a special pending-development tag on the forum that we use to track forum topics requiring some sort of action on our end (e.g. bug fix, new feature, fixing docs, etc.). We use this tag to follow up on relevant forum topics when the feature/bug fix/etc. is available in a release. Thus, we use the pending-development tag to indicate which topics have generated a bug report, feature request, etc., and also as a mechanism to follow up with users when the issue is resolved.

3 Likes

Yes, I think that helps a lot just mentioning ‘bug’ somewhere in the description. Since something like the broken link I reported isn’t really ‘user support’, I didn’t think I should post there. I know that I lean towards reporting bugs, but a lot of users don’t understand how important reporting a bug really is so this should help them too.

Re: pending-development tag… sounds good. I will try to follow along with the development a bit more. Always learning! Thank you!

3 Likes

Totally agree, getting bug reports from users is so important to the improvement of the project. Thanks for helping us make the bug reporting process clearer to users!

No worries, the pending-development tag isn't something we've documented anywhere, we're just using it internally for bookkeeping. @thermokarst thought this would be a good opportunity to mention the tag in case it is useful to you or other readers in the future! :rocket:

2 Likes

The QIIME 2 2017.11 release has the q2-longitudinal tutorial pairwise-distances output fixed, and the QIIME 2 community page now directs users to the QIIME 2 Forum for bug reports, feature requests, etc. :tada:

2 Likes