I had interesting discussions with Andy Lindeberg and jlaska today about the semantics of our current Bugzilla - and particularly triage - workflow.
We were thinking about the NEW / ASSIGNED dichotomy, and Andy and I came to the conclusion that it doesn't really make a lot of sense.
Right now, Bugzappers uses it one way, Andy uses it another for anaconda. In the Bugzappers process, ASSIGNED just means, really, Triaged: you set a bug to ASSIGNED once the triage process is complete.
For Anaconda, Andy sets bugs to ASSIGNED once they're assigned to the correct anaconda maintainer (whether or not the rest of triage is yet fully completed).
Neither case is really terribly problematic, but they also don't make a whole lot of sense, really. Having 'triage completed' as a status is a bit arguable; it's not quite in the flow of the statuses, as a bug could be at a point 'beyond' ASSIGNED but not yet have been triaged, for instance. And it's also just semantically wrong - the word 'assigned' does not mean 'triaged'.
In the anaconda case, we just thought about it and decided the use of ASSIGNED isn't really _achieving_ anything: you can tell whether the bug's been assigned or not just by looking at who it's assigned to.
We sort of came to the conclusion that it'd probably make the most sense to have a 'Triaged' keyword that's used to affirm that a bug's been triaged (in fact there already is one, but it's not officially used in our current process), and abandon the distinction between NEW and ASSIGNED. Ideally, the ASSIGNED state would just be removed, but we could keep it and just note in our workflow / policy that it's not really used to mean anything in particular.
This could, I think, be implemented quite quickly if desired; I think we could rig something up to set all bugs in ASSIGNED state (except anaconda ones) to have the 'Triaged' keyword, that shouldn't be impossible.
We'd be interested in thoughts - negative, positive, whatever - on the idea. Thanks!
Adam Williamson said the following on 07/16/2009 04:41 PM Pacific Time:
I had interesting discussions with Andy Lindeberg and jlaska today about the semantics of our current Bugzilla - and particularly triage - workflow.
We were thinking about the NEW / ASSIGNED dichotomy, and Andy and I came to the conclusion that it doesn't really make a lot of sense.
Right now, Bugzappers uses it one way, Andy uses it another for anaconda. In the Bugzappers process, ASSIGNED just means, really, Triaged: you set a bug to ASSIGNED once the triage process is complete.
For Anaconda, Andy sets bugs to ASSIGNED once they're assigned to the correct anaconda maintainer (whether or not the rest of triage is yet fully completed).
Neither case is really terribly problematic, but they also don't make a whole lot of sense, really. Having 'triage completed' as a status is a bit arguable; it's not quite in the flow of the statuses, as a bug could be at a point 'beyond' ASSIGNED but not yet have been triaged, for instance. And it's also just semantically wrong - the word 'assigned' does not mean 'triaged'.
In the anaconda case, we just thought about it and decided the use of ASSIGNED isn't really _achieving_ anything: you can tell whether the bug's been assigned or not just by looking at who it's assigned to.
We sort of came to the conclusion that it'd probably make the most sense to have a 'Triaged' keyword that's used to affirm that a bug's been triaged (in fact there already is one, but it's not officially used in our current process), and abandon the distinction between NEW and ASSIGNED. Ideally, the ASSIGNED state would just be removed, but we could keep it and just note in our workflow / policy that it's not really used to mean anything in particular.
This could, I think, be implemented quite quickly if desired; I think we could rig something up to set all bugs in ASSIGNED state (except anaconda ones) to have the 'Triaged' keyword, that shouldn't be impossible.
We'd be interested in thoughts - negative, positive, whatever - on the idea. Thanks!
This has been discussed at length before. I believe the initial reluctance was changing the set process for a handful of people that wanted to do things differently and then trying to keep track of that. One major reason for not changing the state definitions was that the Fedora usage of the bug states was the same as RHEL.
At one time I thought the "keyword" approach was a good idea and it still seems to make a lot of good sense... I can't remember why we didn't move forward with it, but as you explain, it seems like a good compromise.
John
On Thu, 2009-07-16 at 18:56 -0700, John Poelstra wrote:
We'd be interested in thoughts - negative, positive, whatever - on the idea. Thanks!
This has been discussed at length before. I believe the initial reluctance was changing the set process for a handful of people that wanted to do things differently and then trying to keep track of that. One major reason for not changing the state definitions was that the Fedora usage of the bug states was the same as RHEL.
At one time I thought the "keyword" approach was a good idea and it still seems to make a lot of good sense... I can't remember why we didn't move forward with it, but as you explain, it seems like a good compromise.
I think it feels like a bigger change than it is. In fact, I think the case may well be the same for RHEL; RHEL seems to use ASSIGNED in the same way Fedora does (i.e. it's abused to mean 'Triaged').
That makes it a bit double-edged - it's not a big change to make, but it's also not a big deal not to change...
On Thu, 2009-07-16 at 19:06 -0700, Adam Williamson wrote:
On Thu, 2009-07-16 at 18:56 -0700, John Poelstra wrote:
We'd be interested in thoughts - negative, positive, whatever - on the idea. Thanks!
This has been discussed at length before. I believe the initial reluctance was changing the set process for a handful of people that wanted to do things differently and then trying to keep track of that. One major reason for not changing the state definitions was that the Fedora usage of the bug states was the same as RHEL.
At one time I thought the "keyword" approach was a good idea and it still seems to make a lot of good sense... I can't remember why we didn't move forward with it, but as you explain, it seems like a good compromise.
I think it feels like a bigger change than it is. In fact, I think the case may well be the same for RHEL; RHEL seems to use ASSIGNED in the same way Fedora does (i.e. it's abused to mean 'Triaged').
That makes it a bit double-edged - it's not a big change to make, but it's also not a big deal not to change...
The old semantics of ASSIGNED before the triage process was that the package maintainer looked at the bug, and agreed that he will be (or already is) working on it.
The question is whether the old semantics was useful or not.
2009/7/17 Tomas Mraz tmraz@redhat.com:
The old semantics of ASSIGNED before the triage process was that the package maintainer looked at the bug, and agreed that he will be (or already is) working on it.
It would be nice if there's a sign that the package maintainer had a look at the bug at all, like VIEWED or whatever. Maybe automatically set. Would give the reporter and triager a better feeling.
On Fri, 2009-07-17 at 08:13 +0000, Thomas Janssen wrote:
2009/7/17 Tomas Mraz tmraz@redhat.com:
The old semantics of ASSIGNED before the triage process was that the package maintainer looked at the bug, and agreed that he will be (or already is) working on it.
It would be nice if there's a sign that the package maintainer had a look at the bug at all, like VIEWED or whatever. Maybe automatically set. Would give the reporter and triager a better feeling.
Then we could use the ASSIGNED state for this meaning if triaged bugs were marked with the triaged keyword.
On Fri, 2009-07-17 at 10:35 +0200, Tomas Mraz wrote:
On Fri, 2009-07-17 at 08:13 +0000, Thomas Janssen wrote:
2009/7/17 Tomas Mraz tmraz@redhat.com:
The old semantics of ASSIGNED before the triage process was that the package maintainer looked at the bug, and agreed that he will be (or already is) working on it.
It would be nice if there's a sign that the package maintainer had a look at the bug at all, like VIEWED or whatever. Maybe automatically set. Would give the reporter and triager a better feeling.
Then we could use the ASSIGNED state for this meaning if triaged bugs were marked with the triaged keyword.
I don't think setting it automatically just based on the maintainer having opened the bug at some point is a good idea. We could allow its use for that purpose manually, though, sure. With my developer hat on I probably wouldn't ever use it myself (it doesn't tell me anything useful), but others have different attitudes of course :)
John Poelstra, Thu, 16 Jul 2009 18:56:03 -0700:
This has been discussed at length before. I believe the initial reluctance was changing the set process for a handful of people that wanted to do things differently and then trying to keep track of that. One major reason for not changing the state definitions was that the Fedora usage of the bug states was the same as RHEL.
At one time I thought the "keyword" approach was a good idea and it still seems to make a lot of good sense... I can't remember why we didn't move forward with it, but as you explain, it seems like a good compromise.
This has been actually flamed to death and I (and others) spent endless hours defending the current status quo. The main reason for this has been that we wanted to minimize changes in our bugzilla, not rocking the boat unnecessarily, etc. Also, the meaning of ASSIGNED was somehow understood and we didn't want to change the world for bugzilla users too much ... it is already too complicated.
Moreover, I am not sure what are actual benefits of any change. The word "ASSIGNED" is just a label which covers much richer set of meanings, and I am not sure any other word would be able to cover completely the state the bug should be in.
Moreover, I don't believe for a second that conversion from ASSIGNED to something else would be that simple as you would like to see it ... there are tons of tiny shifts in meanings of individual components which could be lost and make even bigger mess than it is.
So, if the only reason for this is to include anaconda to the crowd, then I would say, it's better to make some huge exception for them, than to make earthquake which would make a life more difficult for everybody.
Matěj
On Thu, 2009-07-16 at 16:41 -0700, Adam Williamson wrote:
And it's also just semantically wrong - the word 'assigned' does not mean 'triaged'.
It can also be a bit annoying when there are multiple maintainers for a package, and it gets "assigned" to the main maintainer. Personally I'd prefer e.g. renaming "new" to "unconfirmed", and "assigned" to "confirmed". Whether or not an additional state of "assigned" is then necessary is fair moot to me. I'd prefer to avoid flag-city with an explosion of possible flag-combinations, and have a simple state pipeline.
C.
On Fri, 2009-07-17 at 09:06 +0100, Caolán McNamara wrote:
On Thu, 2009-07-16 at 16:41 -0700, Adam Williamson wrote:
And it's also just semantically wrong - the word 'assigned' does not mean 'triaged'.
It can also be a bit annoying when there are multiple maintainers for a package, and it gets "assigned" to the main maintainer. Personally I'd prefer e.g. renaming "new" to "unconfirmed", and "assigned" to "confirmed".
That doesn't work either, though, for the same objection - it's not 'confirmed', it's triaged. We don't require reproduction as part of the triage process.
I think the only sensible way to rename the states would be NEW and TRIAGED, but I'd rather have a keyword for Triaged than have it be a state, personally.
Whether or not an additional state of "assigned" is then necessary is fair moot to me. I'd prefer to avoid flag-city with an explosion of possible flag-combinations, and have a simple state pipeline.
Yep, me too.
On Fri, 2009-07-17 at 07:19 -0700, Adam Williamson wrote:
On Fri, 2009-07-17 at 09:06 +0100, Caolán McNamara wrote:
On Thu, 2009-07-16 at 16:41 -0700, Adam Williamson wrote:
And it's also just semantically wrong - the word 'assigned' does not mean 'triaged'.
It can also be a bit annoying when there are multiple maintainers for a package, and it gets "assigned" to the main maintainer. Personally I'd prefer e.g. renaming "new" to "unconfirmed", and "assigned" to "confirmed".
That doesn't work either, though, for the same objection - it's not 'confirmed', it's triaged. We don't require reproduction as part of the triage process.
Well, now I think about it, we could have UNCONFIRMED and CONFIRMED, and a 'Triaged' flag, as those two definitions are different; you can have a bug that's been confirmed but not triaged, and vice versa. And that may actually be useful information. Yay, complications!
On Fri, 2009-07-17 at 07:19 -0700, Adam Williamson wrote:
We don't require reproduction as part of the triage process.
Fair enough, in that case IMO whether a bug is triaged or not doesn't really have sufficient mojo to push it into any states except closed->duplicate, closed->notabug and needinfo. So being triaged sounds to me more of an attribute, whack it in a keyword/flag then :-)
C.
If it is useful to indicate whether or not a bug has been reproduced, there could be three states: NEW, TRIAGED, and CONFIRMED. It could work in a one-dimensional fashion (no keywords) if CONFIRMED implies TRIAGED. It seems like if someone is going to all the work of confirming a bug *and* they have permission to change the bug's state, they might as well triage it as well.
This additional state only becomes useful if there are people actively looking in Bugzilla trying to reproduce bugs that have already been triaged. Is that actually the case? Or are there so many untriaged bugs that no one really cares whether the "extra credit" work of reproduction has been done on all of the triaged bugs for a specific component?
-B.
On Fri, 2009-07-17 at 13:43 -0400, Christopher Beland wrote:
If it is useful to indicate whether or not a bug has been reproduced, there could be three states: NEW, TRIAGED, and CONFIRMED. It could work in a one-dimensional fashion (no keywords) if CONFIRMED implies TRIAGED. It seems like if someone is going to all the work of confirming a bug *and* they have permission to change the bug's state, they might as well triage it as well.
This additional state only becomes useful if there are people actively looking in Bugzilla trying to reproduce bugs that have already been triaged. Is that actually the case? Or are there so many untriaged bugs that no one really cares whether the "extra credit" work of reproduction has been done on all of the triaged bugs for a specific component?
-B.
Two classes of people, although for some groups the same person plays both roles. Triaging can mean that the triager thinks enough information has been provided, and it's on the right component. However the issue may still need to be reproduced.
It's of my opinion that we should try to avoid adding too many interim steps here, or states, and try to keep things as simple as possible. Bugzilla is already way way way too complicated for people to file bugs and manage bugs. A simple keyword to mark whether it has been triaged or not is probably best, something that the triagers can use to avoid looking at the same bugs twice. Bonus points for avoiding bugzilla spam when that keyword gets set, hard to do outside of the xmlrpc api though.
On Fri, 2009-07-17 at 13:43 -0400, Christopher Beland wrote:
If it is useful to indicate whether or not a bug has been reproduced, there could be three states: NEW, TRIAGED, and CONFIRMED. It could work in a one-dimensional fashion (no keywords) if CONFIRMED implies TRIAGED. It seems like if someone is going to all the work of confirming a bug *and* they have permission to change the bug's state, they might as well triage it as well.
I don't think we can assume that. If, say, I'd seen an issue reported twice but not yet with sufficient info, I might set it to CONFIRMED, but it's not yet been triaged.
This additional state only becomes useful if there are people actively looking in Bugzilla trying to reproduce bugs that have already been triaged. Is that actually the case? Or are there so many untriaged bugs that no one really cares whether the "extra credit" work of reproduction has been done on all of the triaged bugs for a specific component?
It's not necessarily about 'trying to reproduce', there are many cases where we know multiple people are experiencing a bug just from duplicate reports or follow-up comments.
On Fri, 2009-07-17 at 11:50 -0700, Adam Williamson wrote:
If, say, I'd seen an issue reported twice but not yet with sufficient info, I might set it to CONFIRMED, but it's not yet been triaged.
What would have to happen is that the person setting "CONFIRMED" would mark the other bugs as duplicates, do whatever small remaining triage work there is to do, request the missing info and set the bug to CONFIRMED+NEEDINFO.
This violates the "it's not finished being triaged if it's missing info from the reporter" rule, but I never really liked that rule, anyway. 8)
-B.
On Fri, 2009-07-17 at 16:03 -0400, Christopher Beland wrote:
On Fri, 2009-07-17 at 11:50 -0700, Adam Williamson wrote:
If, say, I'd seen an issue reported twice but not yet with sufficient info, I might set it to CONFIRMED, but it's not yet been triaged.
What would have to happen is that the person setting "CONFIRMED" would mark the other bugs as duplicates, do whatever small remaining triage work there is to do, request the missing info and set the bug to CONFIRMED+NEEDINFO.
This violates the "it's not finished being triaged if it's missing info from the reporter" rule, but I never really liked that rule, anyway. 8)
Unless you're setting the new Triaged flag, I don't see how it violates that rule. But we're getting ahead of ourselves, we haven't even decided what changes we want to *ask* for yet. We should probably discuss this at the next meeting.
Bugs without sufficient information have not been triaged. Ensuring sufficient information is present in the report is one of the most important parts of triage.
On Fri, 2009-07-17 at 13:12 -0700, Adam Williamson wrote:
On Fri, 2009-07-17 at 16:03 -0400, Christopher Beland wrote:
On Fri, 2009-07-17 at 11:50 -0700, Adam Williamson wrote:
If, say, I'd seen an issue reported twice but not yet with sufficient info, I might set it to CONFIRMED, but it's not yet been triaged.
What would have to happen is that the person setting "CONFIRMED" would mark the other bugs as duplicates, do whatever small remaining triage work there is to do, request the missing info and set the bug to CONFIRMED+NEEDINFO.
This violates the "it's not finished being triaged if it's missing info from the reporter" rule, but I never really liked that rule, anyway. 8)
Unless you're setting the new Triaged flag, I don't see how it violates that rule. But we're getting ahead of ourselves, we haven't even decided what changes we want to *ask* for yet. We should probably discuss this at the next meeting.
Well, I was trying to clearly articulate an alternative that would *not* involve a Triaged flag, only the three states NEW, TRIAGED, and CONFIRMED.
I probably won't be at the meeting, and I don't have a strong opinion about whether to use a flag or a state. The upside of using a flag is that "triaged" and "confirmed" can be indicated separately. The downside is that certain UI stuff that's geared toward states would need to be improved to also indicate flags in order to maintain convenience for triagers. (Which might be useful to do anyway, since NEEDINFO visibility is convenient for triagers.)
The upside of using only states is that there are fewer permutations to worry about. The downside is that procedurally, any bug that is confirmed-but-not triaged would just need to be triaged on the spot when it was marked CONFIRMED.
Bugs without sufficient information have not been triaged. Ensuring sufficient information is present in the report is one of the most important parts of triage.
Well, I agree that *requesting* missing information is an important part of triage. I suppose there are some bugs that you need an explanation from the reporter before you can make heads or tails of them. But normally, once the checklist has been completed and a request for any missing info has been filed, I pretty much consider the triage over. Either the reporter will add the requested info and the package maintainer can take a look, or they won't and it will continue to be marked NEEDINFO. If they unset the NEEDINFO flag without supplying the requested info, or if their reply indicates further triage action is needed (e.g. it's now clear the bug is reported against the wrong component) I'll get an email, so in the meantime there's no need for the bug to show up on my "not triaged" list. I assume it's clear to maintainers they might not be able to take action on bugs marked NEEDINFO, though in many cases they can.
-B.
On Fri, 2009-07-17 at 17:59 -0400, Christopher Beland wrote:
Well, I was trying to clearly articulate an alternative that would *not* involve a Triaged flag, only the three states NEW, TRIAGED, and CONFIRMED.
Ah, I see - sorry, I'd lost the thread.
I probably won't be at the meeting, and I don't have a strong opinion about whether to use a flag or a state. The upside of using a flag is that "triaged" and "confirmed" can be indicated separately. The downside is that certain UI stuff that's geared toward states would need to be improved to also indicate flags in order to maintain convenience for triagers. (Which might be useful to do anyway, since NEEDINFO visibility is convenient for triagers.)
Yeah, lack of visibility in search results (for e.g.) is a bit of a pain with keywords. (I'd rather use a keyword than a flag, here; flag doesn't fit the use case).
The upside of using only states is that there are fewer permutations to worry about. The downside is that procedurally, any bug that is confirmed-but-not triaged would just need to be triaged on the spot when it was marked CONFIRMED.
Bugs without sufficient information have not been triaged. Ensuring sufficient information is present in the report is one of the most important parts of triage.
Well, I agree that *requesting* missing information is an important part of triage. I suppose there are some bugs that you need an explanation from the reporter before you can make heads or tails of them. But normally, once the checklist has been completed and a request for any missing info has been filed, I pretty much consider the triage over. Either the reporter will add the requested info and the package maintainer can take a look, or they won't and it will continue to be marked NEEDINFO. If they unset the NEEDINFO flag without supplying the requested info, or if their reply indicates further triage action is needed (e.g. it's now clear the bug is reported against the wrong component) I'll get an email, so in the meantime there's no need for the bug to show up on my "not triaged" list. I assume it's clear to maintainers they might not be able to take action on bugs marked NEEDINFO, though in many cases they can.
Well, I find that as a matter of course on many bugs, you have to go through a round or two of interaction with the reporter to make sure the information is fully available. But overall I don't really disagree, you're right that requesting info is the important step.
On Thu, 2009-07-16 at 16:41 -0700, Adam Williamson wrote:
I had interesting discussions with Andy Lindeberg and jlaska today about the semantics of our current Bugzilla - and particularly triage - workflow.
So, this was discussed at the meeting this morning, and there was some significant movement. This email is intended to summarize the current status and provide some options for discussion.
One concern brought up was bug mail spam. I am currently testing exactly what changes on a bug produce an email, and what don't. I'll report on this soon.
Aside from that, Jesse Keating and Josh Boyer stated that teams they work with use the ASSIGNED state with a different meaning from the official one defined in the bug workflow page. For them, ASSIGNED is used to mean that a specific person has accepted the bug and will work on it soon.
The proposal to use a Triaged keyword to indicate that a bug has been triaged would certainly address this issue.
Another option proposed during the meeting was that the ON_DEV state should be used to indicate this, as its official definition is much closer. The bug workflow page states: "This optional state is used to show that someone is working on fixing this bug. This is especially useful if there exists a team of maintainers for a package."
An option not discussed during the meeting but which I should add, is that no state is actually needed to indicate this. AFAIK, all cases where this usage is pertinent involve components owned by teams. When bugs are first filed on these components, they are assigned to a mailing list which represents the entire group. I contend that the simple act of changing the assignment from the mailing list to an individual developer is enough to indicate that the bug has been accepted by that individual developer; no state change is needed to make this clear. Searching can also be done on this distinction: you can search for bugs filed on a given component and assigned to a specific developer, bugs filed on a given component and assigned to the mailing list for that component, or for bugs filed on a given component and *not* assigned to the mailing list (hence, by implication, assigned to _any_ specific developer). That gives you all the search granularity you can achieve with a state.
Matej Cepl objects to the proposed change on the grounds that it achieves nothing (in light of the fact that there are other ways to address the 'problem' which do not involve any change to our current workflow definition, or mass changes to Bugzilla) and is a disruptive and potentially damaging change to make to a large number of existing bugs. In his words:
"Moreover, I don't believe for a second that conversion from ASSIGNED to something else would be that simple as you would like to see it ... there are tons of tiny shifts in meanings of individual components which could be lost and make even bigger mess than it is.
So, if the only reason for this is to include anaconda to the crowd, then I would say, it's better to make some huge exception for them, than to make earthquake which would make a life more difficult for everybody."
I think we can accept Matej's framing: the question is whether changing the official definitions, and possibly adjusting all current bugs to use the keyword, provides a significant enough advantage to outweigh the disadvantage of disruption and potential errors introduced in current bugs.
I present three options:
1) Leave everything unchanged, both in Bugzilla and in the workflow definition. Encourage components which currently use ASSIGNED in a non-standard way to use ON_DEV or no state change, as discussed above.
2) Change practice for Bugzappers going forward: from some date forward, all triaged bugs should have the Triaged keyword set rather than being changed to ASSIGNED. Do not change anything about any existing bugs.
3) Change practice for Bugzappers going forward, as in proposal 2), and also attempt to convert existing bugs by adding the 'Triaged' keyword to all bugs in ASSIGNED status on all components which are currently actively triaged.
On Tue, 2009-07-21 at 09:33 -0700, Adam Williamson wrote:
On Thu, 2009-07-16 at 16:41 -0700, Adam Williamson wrote:
I had interesting discussions with Andy Lindeberg and jlaska today about the semantics of our current Bugzilla - and particularly triage - workflow.
So, this was discussed at the meeting this morning, and there was some significant movement. This email is intended to summarize the current status and provide some options for discussion.
One concern brought up was bug mail spam. I am currently testing exactly what changes on a bug produce an email, and what don't. I'll report on this soon.
Aside from that, Jesse Keating and Josh Boyer stated that teams they work with use the ASSIGNED state with a different meaning from the official one defined in the bug workflow page. For them, ASSIGNED is used to mean that a specific person has accepted the bug and will work on it soon.
The proposal to use a Triaged keyword to indicate that a bug has been triaged would certainly address this issue.
Another option proposed during the meeting was that the ON_DEV state should be used to indicate this, as its official definition is much closer. The bug workflow page states: "This optional state is used to show that someone is working on fixing this bug. This is especially useful if there exists a team of maintainers for a package."
An option not discussed during the meeting but which I should add, is that no state is actually needed to indicate this. AFAIK, all cases where this usage is pertinent involve components owned by teams. When bugs are first filed on these components, they are assigned to a mailing list which represents the entire group. I contend that the simple act of changing the assignment from the mailing list to an individual developer is enough to indicate that the bug has been accepted by that individual developer; no state change is needed to make this clear. Searching can also be done on this distinction: you can search for bugs filed on a given component and assigned to a specific developer, bugs filed on a given component and assigned to the mailing list for that component, or for bugs filed on a given component and *not* assigned to the mailing list (hence, by implication, assigned to _any_ specific developer). That gives you all the search granularity you can achieve with a state.
Matej Cepl objects to the proposed change on the grounds that it achieves nothing (in light of the fact that there are other ways to address the 'problem' which do not involve any change to our current workflow definition, or mass changes to Bugzilla) and is a disruptive and potentially damaging change to make to a large number of existing bugs. In his words:
"Moreover, I don't believe for a second that conversion from ASSIGNED to something else would be that simple as you would like to see it ... there are tons of tiny shifts in meanings of individual components which could be lost and make even bigger mess than it is.
So, if the only reason for this is to include anaconda to the crowd, then I would say, it's better to make some huge exception for them, than to make earthquake which would make a life more difficult for everybody."
I think we can accept Matej's framing: the question is whether changing the official definitions, and possibly adjusting all current bugs to use the keyword, provides a significant enough advantage to outweigh the disadvantage of disruption and potential errors introduced in current bugs.
I present three options:
- Leave everything unchanged, both in Bugzilla and in the workflow
definition. Encourage components which currently use ASSIGNED in a non-standard way to use ON_DEV or no state change, as discussed above.
- Change practice for Bugzappers going forward: from some date forward,
all triaged bugs should have the Triaged keyword set rather than being changed to ASSIGNED. Do not change anything about any existing bugs.
- Change practice for Bugzappers going forward, as in proposal 2), and
also attempt to convert existing bugs by adding the 'Triaged' keyword to all bugs in ASSIGNED status on all components which are currently actively triaged.
I would like to point out again for the record that it seems to me that the primary reason bugzappers are altering the bug, in any way, is to mark that they've been there, done that, and to avoid having duplicate effort by other triagers on the same bug. I know it's not the only reason, but it seems to be the most significant.
It's also worth pointing out that while there is an "official" meaning of ASSIGNED, it's meaning has been contrived by the bugzappers crowd whether or not the entire maintainer set agrees with it, and I think it's abundantly clear that not all maintainers agree.
Given that we have a primary concern of marking the bug as having been triaged, and we know already that the use of ASSIGNED is controversial, I believe it is in the best interest of bugzappers to find a different way of marking the bug, such as a keyword. This accomplishes the primary (and other) goals, and completely side steps the arguments of bug states. This allows maintainers and teams to manage their own states as they see fit and still provides the information to them and other zappers that the bug has indeed been triaged.
I appreciate that there will be churn, and a period of time where both state and a keyword would indicate triaged or not. I also appreciate that this issue should have been discussed better from the onset rather than months and months into the effort. However as often with Fedora, you don't get your complaints until you start acting upon your decisions, and it's always good to review your decisions after those effected by them have had a chance to witness first hand what is going on. We have had swaths of packages/packagers completely opt-out of the triage process due to disagreements of how things are going, and instead of just accepting that and moving on, I'm trying to reach a happy medium where triage efforts can cover more packages and packagers, and those packagers are more willing to let the work be done, as they /do/ see value in it. It really comes down to the execution, which can be adjusted.
If it were just about one team, I'd agree, let that team do their own thing, zappers move on ahead. It isn't though and that's why I've taken an interest in seeing this work out. Complaints have bubbled up and been over heard, and I'm representing those complaints, whether the complainers want me to or not (:
Anyway that's my pitch, take it as you will.
On Tue, 2009-07-21 at 10:26 -0700, Jesse Keating wrote:
I would like to point out again for the record that it seems to me that the primary reason bugzappers are altering the bug, in any way, is to mark that they've been there, done that, and to avoid having duplicate effort by other triagers on the same bug. I know it's not the only reason, but it seems to be the most significant.
This is not the theory; the primary reason for marking bugs triaged is intended to be for the benefit of developers. Whether or not it works this way in practice.
It's also worth pointing out that while there is an "official" meaning of ASSIGNED, it's meaning has been contrived by the bugzappers crowd whether or not the entire maintainer set agrees with it, and I think it's abundantly clear that not all maintainers agree.
I believe it was not originally 'contrived', but rather based on RHEL practice. The ASSIGNED state has been defined to mean 'the bug has been triaged' since at least May 2008, when the Wiki page was imported from the old system:
https://fedoraproject.org/w/index.php?title=BugZappers/BugStatusWorkFlow&...
On Tue, 2009-07-21 at 10:40 -0700, Adam Williamson wrote:
On Tue, 2009-07-21 at 10:26 -0700, Jesse Keating wrote:
I would like to point out again for the record that it seems to me that the primary reason bugzappers are altering the bug, in any way, is to mark that they've been there, done that, and to avoid having duplicate effort by other triagers on the same bug. I know it's not the only reason, but it seems to be the most significant.
This is not the theory; the primary reason for marking bugs triaged is intended to be for the benefit of developers. Whether or not it works this way in practice.
The benefit to the developer would be in properly assigning the bug to the right component, or getting enough useful information out of the reporter. Such things should be evident without any other marking by a triager. Either there was value added to the bug and visible to the maintainer, or there was nothing more to add. In either case, the value to the maintainer is there whether the triager has left footprints or not, so it seems that the actual value of the /footprint/ is to prevent multiple attempts at triaging.
It's also worth pointing out that while there is an "official" meaning of ASSIGNED, it's meaning has been contrived by the bugzappers crowd whether or not the entire maintainer set agrees with it, and I think it's abundantly clear that not all maintainers agree.
I believe it was not originally 'contrived', but rather based on RHEL practice. The ASSIGNED state has been defined to mean 'the bug has been triaged' since at least May 2008, when the Wiki page was imported from the old system:
https://fedoraproject.org/w/index.php?title=BugZappers/BugStatusWorkFlow&...
-- Adam Williamson Fedora QA Community Monkey IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org http://www.happyassassin.net
Based on RHEL is still contrived, because in the RHEL world, red tape and upper management /assigns/ bugs to people to work on, whether they want to or not. In the Fedora world, maintainers have to accept bugs to work on. There is a big difference between 'You are going to work on this' and 'I am going to work on this'. In fact, using the RHEL meaning, I would take offense as a maintainer if suddenly the triage team is dictating to me what bugs I will work on.
Suffice to say there are plenty of maintainers who do not come from a RHEL background, and who have a long history in Fedora and who have developed their own way of using bug states to deal with their software. Entirely too much energy has been spent on trying to argue about what bug states mean when at the end of the day it really really really doesn't matter to the triage efforts. Triage can accomplish all of their goals without ever touching the bug state. Why spend time and effort fighting over something that clearly doesn't matter?
On Tue, 2009-07-21 at 11:13 -0700, Jesse Keating wrote:
This is not the theory; the primary reason for marking bugs triaged is intended to be for the benefit of developers. Whether or not it works this way in practice.
The benefit to the developer would be in properly assigning the bug to the right component, or getting enough useful information out of the reporter. Such things should be evident without any other marking by a triager. Either there was value added to the bug and visible to the maintainer, or there was nothing more to add. In either case, the value to the maintainer is there whether the triager has left footprints or not, so it seems that the actual value of the /footprint/ is to prevent multiple attempts at triaging.
No, the idea is to allow developers to pay attention mainly to issues that have been triaged. To do this they need to know which are which. But there's not much point arguing about this, as all proposals allow it to happen.
Triage can accomplish all of their goals without ever touching the bug state. Why spend time and effort fighting over something that clearly doesn't matter?
Because it's the status quo, and changing it requires either significant upheavals in existing bugs, or a period of confusion where multiple methods are in use. Either of these is a 'cost', so we have to demonstrate a sufficient benefit to make it worthwhile.
On Tue, 2009-07-21 at 12:07 -0700, Adam Williamson wrote:
The benefit to the developer would be in properly assigning the bug to the right component, or getting enough useful information out of the reporter. Such things should be evident without any other marking by a triager. Either there was value added to the bug and visible to the maintainer, or there was nothing more to add. In either case, the value to the maintainer is there whether the triager has left footprints or not, so it seems that the actual value of the /footprint/ is to prevent multiple attempts at triaging.
No, the idea is to allow developers to pay attention mainly to issues that have been triaged. To do this they need to know which are which. But there's not much point arguing about this, as all proposals allow it to happen.
Sure, I always pictured triage as a added bonus one might get on some bugs, but never going to be to the point where as maintainers we'd just ignore untriaged bugs.
Triage can accomplish all of their goals without ever touching the bug state. Why spend time and effort fighting over something that clearly doesn't matter?
Because it's the status quo, and changing it requires either significant upheavals in existing bugs, or a period of confusion where multiple methods are in use. Either of these is a 'cost', so we have to demonstrate a sufficient benefit to make it worthwhile.
It's only been the "status quo" for a very short period of time in relation to the lifespan of Fedora itself, and while there is a one time cost, it's IMHO lower than an ongoing cost of frustration and anger over continued quibbling over bug state meanings.
Adam Williamson said the following on 07/21/2009 09:33 AM Pacific Time:
I present three options:
- Leave everything unchanged, both in Bugzilla and in the workflow
definition. Encourage components which currently use ASSIGNED in a non-standard way to use ON_DEV or no state change, as discussed above.
- Change practice for Bugzappers going forward: from some date forward,
all triaged bugs should have the Triaged keyword set rather than being changed to ASSIGNED. Do not change anything about any existing bugs.
Based on the 1.5 years I've been involved in this process and tried to help make it better, I think this is the best solution and provides the least amount of potential "disruption" going forward.
I disagree with the anecdotal evidence that has been presented about the disruptive affects of the BugZappers and the magnitude of this problem that this perceived problem that has caused resulting in some groups to opt-out. That being said I think it is time to try out option #2 and move forward.
Indicate that a bug has been triaged by using the "Triaged" keyword on a go-forward basis.
- Change practice for Bugzappers going forward, as in proposal 2), and
also attempt to convert existing bugs by adding the 'Triaged' keyword to all bugs in ASSIGNED status on all components which are currently actively triaged.
Adam Williamson, Tue, 21 Jul 2009 09:33:08 -0700:
Matej Cepl objects to the proposed change on the grounds that it achieves nothing
Just for the record: I feel like I am totally not getting this discussion, so I will rather abstain from it (as told on IRC), and I accept in advance whatever conclusion will be. Just let me know in advance so I can fix my scripts to follow new guidelines.
Matěj
2009/7/22 Matej Cepl mcepl@redhat.com:
Adam Williamson, Tue, 21 Jul 2009 09:33:08 -0700:
Matej Cepl objects to the proposed change on the grounds that it achieves nothing
Just for the record: I feel like I am totally not getting this discussion, so I will rather abstain from it (as told on IRC), and I accept in advance whatever conclusion will be. Just let me know in advance so I can fix my scripts to follow new guidelines.
Matěj
I'm also not quite sure, if I understand the discussion at all. Maybe someone can be so nice to correct me if I'm wrong.
The main problem is the spam that the developer reached from bugzilla while triaging of the bugs is ongoing by the bugzappers.
But how what will the change of "move away from ASSIGNED to Triaged keyword if the triage is done" change here? The amount of outgoing mail is the same, since it's the same amount of comments and needinfo that is needed to collect whole needed information form the reporter.
So, if anyone can please so kind to explain me the relation of changed the state from ASSIGNED to keyword Triaged in the relation to the mail spam. Thanks for your patience.
What make sense for me is, to changed the behaviour to use the Triaged keyword to mark the bug as "done" for the triaged afford and leave it as NEW. The maintainer can now ASSIGN the bug to a list or himself. Witch will result in one more stage. This make it for my opinion easier to see if the bug is currently under development by the maintainer.
The work flow maybe can look like this: Bug is NEW, triage get started, if the triage is done, set the "Triaged" keyword, leave the bug as NEW. Now the maintainer need to ASSIGN the bug (to himself or a list), not the bugzapper any more.
-- Regards, Niels
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
Niels Haase, Wed, 22 Jul 2009 13:15:52 +0200:
But how what will the change of "move away from ASSIGNED to Triaged keyword if the triage is done" change here? The amount of outgoing mail is the same, since it's the same amount of comments and needinfo that is needed to collect whole needed information form the reporter.
That's actually the only part which makes sense ... go to https://bugzilla.redhat.com/userprefs.cgi?tab=email and make Bugzilla not to send you email on change of Keyword field.
Matěj
2009/7/22 Matej Cepl mcepl@redhat.com:
Niels Haase, Wed, 22 Jul 2009 13:15:52 +0200:
But how what will the change of "move away from ASSIGNED to Triaged keyword if the triage is done" change here? The amount of outgoing mail is the same, since it's the same amount of comments and needinfo that is needed to collect whole needed information form the reporter.
That's actually the only part which makes sense ... go to https://bugzilla.redhat.com/userprefs.cgi?tab=email and make Bugzilla not to send you email on change of Keyword field.
Right, but this will prevent the maintainer only from _one_ mail. The mails before that, with NEEDINFO to reporter and the response from them will not changed. I'm confused :)
-- Regards, Niels
Matěj
-- fedora-test-list mailing list fedora-test-list@redhat.com To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-test-list
On Wed, 2009-07-22 at 13:15 +0200, Niels Haase wrote:
I'm also not quite sure, if I understand the discussion at all. Maybe someone can be so nice to correct me if I'm wrong.
The main problem is the spam that the developer reached from bugzilla while triaging of the bugs is ongoing by the bugzappers.
But how what will the change of "move away from ASSIGNED to Triaged keyword if the triage is done" change here? The amount of outgoing mail is the same, since it's the same amount of comments and needinfo that is needed to collect whole needed information form the reporter.
So, if anyone can please so kind to explain me the relation of changed the state from ASSIGNED to keyword Triaged in the relation to the mail spam. Thanks for your patience.
What make sense for me is, to changed the behaviour to use the Triaged keyword to mark the bug as "done" for the triaged afford and leave it as NEW. The maintainer can now ASSIGN the bug to a list or himself. Witch will result in one more stage. This make it for my opinion easier to see if the bug is currently under development by the maintainer.
The work flow maybe can look like this: Bug is NEW, triage get started, if the triage is done, set the "Triaged" keyword, leave the bug as NEW. Now the maintainer need to ASSIGN the bug (to himself or a list), not the bugzapper any more.
The mail from comments soliciting information and getting information aren't necessarily the concern I was seeing. Mostly I was seeing questions of the value of looking at a bug and doing nothing but putting a comment that says "triaged" and setting the status to ASSIGNED, and nothing else. That was the biggest issue. This is where a keyword can be set without adding unnecessary comments in the bug, nor changing the status when the use of status may be disagreed upon.
On Wed, 2009-07-22 at 13:15 +0200, Niels Haase wrote:
I'm also not quite sure, if I understand the discussion at all. Maybe someone can be so nice to correct me if I'm wrong.
The main problem is the spam that the developer reached from bugzilla while triaging of the bugs is ongoing by the bugzappers.
No, it's not. That's probably a less important issue. The main sticking point is that several development groups have said they use ASSIGNED to mean that a bug has been accepted for work by a particular member of the development group, which differs significantly from the 'official' definition of ASSIGNED, in the triage workflow. This means those groups cannot really work with the current triage workflow. That's the main problem we're trying to address.
What make sense for me is, to changed the behaviour to use the Triaged keyword to mark the bug as "done" for the triaged afford and leave it as NEW. The maintainer can now ASSIGN the bug to a list or himself. Witch will result in one more stage. This make it for my opinion easier to see if the bug is currently under development by the maintainer.
The work flow maybe can look like this: Bug is NEW, triage get started, if the triage is done, set the "Triaged" keyword, leave the bug as NEW. Now the maintainer need to ASSIGN the bug (to himself or a list), not the bugzapper any more.
This is exactly one of the current proposals, yes. One of the main questions being whether we do this only going forward, or try to 'fix' existing bugs to have the Triaged keyword where appropriate.
On Wed, 2009-07-22 at 09:44 +0000, Matej Cepl wrote:
Adam Williamson, Tue, 21 Jul 2009 09:33:08 -0700:
Matej Cepl objects to the proposed change on the grounds that it achieves nothing
Just for the record: I feel like I am totally not getting this discussion, so I will rather abstain from it (as told on IRC), and I accept in advance whatever conclusion will be. Just let me know in advance so I can fix my scripts to follow new guidelines.
Understood. I wasn't trying to set you up as the Bad Guy or anything - just trying to fully summarize the current status of the debate, including everyone who's declared their positions.
Adam Williamson, Wed, 22 Jul 2009 13:19:40 -0700:
Understood. I wasn't trying to set you up as the Bad Guy or anything - just trying to fully summarize the current status of the debate, including everyone who's declared their positions.
No, this wasn't reaction on your (very kind, actually) email here ... the discussion on IRC made me feel completely out-of-whack.
Matěj
On Tue, 2009-07-21 at 09:33 -0700, Adam Williamson wrote:
I present three options:
- Leave everything unchanged, both in Bugzilla and in the workflow
definition. Encourage components which currently use ASSIGNED in a non-standard way to use ON_DEV or no state change, as discussed above.
- Change practice for Bugzappers going forward: from some date
forward, all triaged bugs should have the Triaged keyword set rather than being changed to ASSIGNED. Do not change anything about any existing bugs.
- Change practice for Bugzappers going forward, as in proposal 2),
and also attempt to convert existing bugs by adding the 'Triaged' keyword to all bugs in ASSIGNED status on all components which are currently actively triaged.
Sorry for the lateness of this email!
As discussed at the meeting last week, consensus seems to be roughly in favor of option #2 above. So, let's make that the formal proposal.
Are there any serious objections to proceeding with option #2 - going forward, bugs will be marked as having been triaged with the use of a keyword, rather than changing status from NEW to ASSIGNED?
If not, we'll push ahead and make that the new official policy. Thanks!
On Mon, 2009-08-03 at 09:48 -0700, Adam Williamson wrote:
- Change practice for Bugzappers going forward: from some date
forward, all triaged bugs should have the Triaged keyword set rather than being changed to ASSIGNED. Do not change anything about any existing bugs.
Sorry for the lateness of this email!
As discussed at the meeting last week, consensus seems to be roughly in favor of option #2 above. So, let's make that the formal proposal.
Are there any serious objections to proceeding with option #2 - going forward, bugs will be marked as having been triaged with the use of a keyword, rather than changing status from NEW to ASSIGNED?
If not, we'll push ahead and make that the new official policy. Thanks!
So, resurrecting this thread zombie one last time:
As discussed in the past few weeks' meetings, we went ahead and implemented option 2) initially. However, Matej pointed out a problem with that; it made it impossible to search Bugzilla for all triaged or un-triaged bugs in a given component across all releases. So, to solve this, we made a change. All Fedora 10-12 bugs in ASSIGNED status have had the Triaged keyword added. From now on, when triaging bugs in Fedora 11 and 12, please _both_ set the ASSIGNED status _and_ add the Triaged keyword when the bug is triaged. For Rawhide (and, later, Fedora 13 onwards) bugs, please _only_ add the Triaged keyword, do not change the status of the bug.
When searching for bugs, use the Triaged keyword in your searches if you want to search for triaged or un-triaged bugs. This way, it should always give accurate results.
Based on the announcements to the list, I have updated
https://fedoraproject.org/wiki/BugZappers/How_to_Triage#How_to_Mark_as_Triag...
to the below text. Just wanted to make sure this is accurate and desirable.
-B.
---
How to Mark as Triaged
If you have completed the entire checklist for a bug, you should mark it as triaged so that others will not duplicate your efforts.
For Fedora 12:
* If the bug now has all the information the maintainer needs, mark it as ASSIGNED. If you are waiting on information from the reporter, leave it as NEW and check the "Need information from" box and set the pulldown appropriately. * Either way, add "Triaged" to the Keywords field.
For Rawhide, Fedora 13, and later releases:
* Add "Triaged" to the Keywords field. Do not change the bug's status. If information is missing from the report, check the "Need information from" box and set the pulldown appropriately.
Closing Bugs
Many bugs are fixed upstream or unintentionally during a seemingly unrelated rewrite. If someone who has experienced the bug can confirm that is is fixed by an update to the version of Fedora is it reported against, mark it CLOSED/ERRATA.
Normally when a package maintainer intentionally fixes a bug, this is recorded in the build system, and Bodhi should automatically close the bug when the update goes live. (But this does not happen for Rawhide bugs.)