How not to use a bug tracker

Friday, January 6. 2012

From time to time suggestions are made that the PostgreSQL project should use trackers to manage bugs and possibly feature requests. I have a lot of sympathy with these suggestions. But there has always been lots of pushback, along with significant disagreement about which tracker to use. Having done a bunch of work years ago to make Bugzilla platform independent, I have some fondness for it, but others hate it with a passion that seems way out of proportion to the perceived evil, so it's probably out of the question, if we ever did decide to use some sort of tracker.

Meanwhile, I encountered the Perl community's use of a tracker yesterday. David Wheeler encouraged me to file a bug about the recently discovered misbehaviour of a documented piece of the perl API. Accordingly, I ran perlbug and it told me that it had sent the bug report. Later he asked me if I had done so, as he couldn't find the report. First black mark. Then I went and looked at the tracker's web interface. What I saw was just horrible. For perl 5 there are 259 "new" bugs, (the oldest 8 years old, which isn't "new" in my book) and 1149 open bugs. And my bug, which their own program told me had been successfully submitted, didn't seem to be there. What's the point in having such a system? It's worse than useless. Trackers require effort to maintain. As I remarked to David, it's no wonder that there is significant resistance to using them when we have horrible examples like this one.

Update

Apparently my bug is sitting in a moderation queue. So I'm glad it's not lost anyway.

Posted by Andrew Dunstan in PostgreSQL at 20:34 | Comments (9) | Trackbacks (0)

Trackbacks

Trackback specific URI for this entry

No Trackbacks

Comments

Display comments as (Linear | Threaded)

Github Issues provides a really excellent mechanism for discussing issues and integrating with patches and pull requests. If you haven't seen it before, it's definitely worth taking a look at. 
 
One really great example of it being used effectively is the Sequel project for Ruby. See here: https://github.com/jeremyevans/sequel/issues

#1 Peter van Hardenberg on 2012-01-06 20:58 (Reply)

Yeah. My point was that no matter what tracker you're using, if you don't properly triage the bugs, and assign them and fix them or close them it's hopeless. That demands resources. Open bugs for something like Perl (or Postgres for that matter) should number in the handfuls, not in the thousands.

#1.1 Andrew Dunstan on 2012-01-06 21:15 (Reply)

But how can you know for sure how many open bugs are there without a bug tracker? 
I just did a sampling. Bugzilla says there's 2057 open bugs for Firefox (that have the word "firefox") in them. Subversion's bug tracker says there are 689 open bugs. Python's tracker lists over 3000 open bugs. Debian (http://bugs.debian.org/release-critical/) shows "release critical" bugs that number in the 1000-2500 range between releases.

#2 Joe on 2012-01-07 16:20 (Reply)

I'm in favor of using a tracker if it's used effectively. Keeping bugs open is not effective use. Bugs need to be confirmed and fixed or closed. 
 
I spent a part of my career as a development manager where a considerable part of my responsibility was making sure that the list of open bugs was kept down. If the tracker is used as a graveyard of bug reports it's almost worse than useless, IMNSHO. 
 
Take a look at https://bugzilla.mozilla.org/show_bug.cgi?id=312164 and tell me why it's still open after three years of no action whatsoever.

#2.1 Andrew Dunstan on 2012-01-07 18:19 (Reply)

Yes, that Mozilla bug is somewhat amazing but perhaps typical. It got reported under FF 1.5, then re-reported three years later against 3.x, someone crossed it to another bug that supposedly got closed and then got re-reported. I recall something similar with TBird that had at least a couple of "same as" indicators. 
Unless someone proactively keeps an eye on the tracker, at least before a release or as a post mortem, this bug accumulation tends to happen in most places.

#2.1.1 Joe on 2012-01-07 20:15 (Reply)

Oh, and commercial organizations are not immune from this disease, either. This RHEL5 bug for example should clearly be closed: https://bugzilla.redhat.com/show_bug.cgi?id=555744

#2.2 Andrew Dunstan on 2012-01-07 19:37 (Reply)

Hi! I'm the project lead for perl 5, and I thought I'd date show my face around here. <img src="/andrew/templates/default/img/emoticons/wink.png" alt=";-)" style="display: inline; vertical-align: bottom;" class="emoticon" /> 
 
First: I'm also one of the moderators for submissions to perlbug, and I don't recall seeing that bug come in. Obviously, it still hasn't hit the tracker. I can ask the admins on the inbound mailserver whether they have logs of it showing up, but I wonder whether you would mind also checking your outbound mail queue. Once in a while, a half-configured MTA is to blame. 
 
As for the tracker itself, I think that it's definitely a mess, but I don't think the answer (for us) is to not have it. There are a lot of things in play. 
 
The "new" bug category is definitely a problem. These are tickets where nobody ever responded to an initial bug report. Most of these are in response to poor discipline long ago combined with a lack of current interest in cleaning up the ancient mess. For example, in 2005, someone could report a bug that on VMS, opening a file named "-" after closing STDIN causes problem xyz. All the non-VMS experts ignore the ticket as out of bounds, and the actual VMS experts miss its arrival, or can't reproduce it and forget to get back to it, or whatever. Most users are used to only watching for new tickets arriving, so the old one can then languish. 
 
This problem is getting better, as we have a few absolutely heroic contributors who have been churning through the back catalog. I think that within a month or two, we'll have fully cleared the backlog, and I think that bugs get cleared much more aggressively now. 
 
As for open bugs, I don't really think the large bug count is a problem. Many of these record problems that are real, but rarely encountered, or technical issues that we know about but don't yet know how to address. Long-standing tickets that end with "nobody can reproduce this, can you please send a report of X?" with no response are a problem, but long-standing tickets that "everybody knows that buffered IO interferes with tied encoding layers if compiled with experimental pseudothread emulation" are a good thing. 
 
From the real experience of using rt.perl.org, I can tell you that it's not worthless. It's got a lot of value. It definitely has problems, too, though. I think we're getting a number of those solved. As you say, though, having a bug tracker has a big cost, and recognizing that up front is important. Some people do think that a bug tracker is entirely a value-adding system, with no maintenance cost, and that's a naive and dangerous position. It takes a lot of work, but if you really do the work, you can get a lot of value out of it, and I think we do. 
 
Finally, I am concerned about the missing ticket, because I'd hate to think that it's one of many. With your help, I would like to really track it down.

#3 Ricardo Signes (Homepage) on 2012-01-09 06:29 (Reply)

I'll follow up privately about the missing bug, which might well have been my fault. 
I am not arguing not to have a tracker. I am saying that if you have one you need to devote substantial effort to managing it. On this point we seem to be in agreement. 
Having a large number of open bugs can be a problem, IMNSHO, because it can mean that they just get lost in the noise. Bugs in the "we know it's a problem but we don't know how to fix it" category need to be segregated from the "we haven't got around to fixing it yet" category. 
And no bugs at all should stay in the "new" state for long. Weeks at most, not months or years, I would say. 
As for RT, I found searching it somewhat primitive, but that's a whole other topic. 
I'm glad to hear that the Perl community is addressing some of these issues. I'm a long time Perl user (taught myself back in '93 on perl 4.036, IIRC) and still use it extensively, e.g. see the PostgreSQL Build Farm code at https://github.com/PGBuildFarm - it's all Perl. 
Thanks for taking the time to comment on this.

#3.1 Andrew Dunstan on 2012-01-09 19:21 (Reply)

Agreed. When we were discussing a Postgres bug tracker, I would ask to be shown an open source project where the bug tracker isn't just a dumping ground --- that stopped the conversation because no one had a good reply to the question.

#4 Bruce Momjian (Homepage) on 2012-01-14 22:59 (Reply)

Add Comment

Name
Email
Homepage
In reply to
Comment	Enclosing asterisks marks text as bold (word), underscore are made via _word_. Standard emoticons like :-) and ;-) are converted to images. Wiki format allowed
	Remember Information? Subscribe to this entry