I got feedback from Adam and Ben today; so the following changes have been made:
I have added a little paragraph at the beginning to say what a last minute blocker bug is. I used freeze as the time anchor rather than a meeting since that seems to be the most firm time constraint we work to. Perhaps the review meetings could be anchored to the freeze as well. There might be some merit to showing the critical meetings in the schedule list that gets published.
I changed "team" to "stakeholder groups"
I removed "proposed" from places where it really didn't add anything.
Have a Great Day!
Pat (tablepc)
On Wed, 2019-07-24 at 10:04 -0400, pmkellly@frontier.com wrote:
I got feedback from Adam and Ben today; so the following changes have been made:
I have added a little paragraph at the beginning to say what a last minute blocker bug is. I used freeze as the time anchor rather than a meeting since that seems to be the most firm time constraint we work to. Perhaps the review meetings could be anchored to the freeze as well. There might be some merit to showing the critical meetings in the schedule list that gets published.
I changed "team" to "stakeholder groups"
I removed "proposed" from places where it really didn't add anything.
Have a Great Day!
Pat (tablepc)
Thanks Pat! For future drafts, can you please just send them as plain text in line? It makes it more convenient to read them quickly. For the record, here is Pat's proposal:
" 3 Last Minute Blocker Bugs
3.1 A Last Minute Blocker Bug occurs when a bug is nominated as a blocker bug seven (7) days or less before a scheduled freeze for either a Beta or Final release 3.2 The stakeholder groups must agree according to the two prior paragraphs in this section, that a Last Minute Blocker Bug has the character of a Blocker Bug before being process per this section.
3.3 A Last Minute Blocker Bug shall be evaluated for being delayed to the next release cycle according to the criteria below. A Last Minute Blocker Bug that meets any of the listed criteria may be delayed to the next release cycle as a Blocker Bug if the stakeholder groups agree. Other Last Minute Blocker Bugs must be corrected before the current cycle Final Release.
3.3.1 Any bug that can not be fixed in a reasonable amount of time considering the current Release Cycle or due to complexity or resource constraints.
3.3.2 Any bug in non-vital system operation or release package installed application operation.
4 Delaying the current Release:
4.1 Delaying the current release cycle must take into account all of the following criteria.
4.1.1 Consider if the cause of the delay should have been caught earlier in the current cycle. What changes in process could help moving such discoveries earlier in the cycle?
4.1.2 Adding the current proposed delay to any prior delays, is the total delay becoming unacceptable in regard to Fedora release policy?
4.1.3 Will the current proposed delay enable other desirable work to be completed for the release?
4.1.4 Impact on down stream projects."
I will follow up with a new draft of my own, as discussed at the meeting last week.
On Fri, 2019-08-09 at 18:26 -0700, Adam Williamson wrote:
On Wed, 2019-07-24 at 10:04 -0400, pmkellly@frontier.com wrote:
I got feedback from Adam and Ben today; so the following changes have been made:
I have added a little paragraph at the beginning to say what a last minute blocker bug is. I used freeze as the time anchor rather than a meeting since that seems to be the most firm time constraint we work to. Perhaps the review meetings could be anchored to the freeze as well. There might be some merit to showing the critical meetings in the schedule list that gets published.
I changed "team" to "stakeholder groups"
I removed "proposed" from places where it really didn't add anything.
Have a Great Day!
Pat (tablepc)
Thanks Pat! For future drafts, can you please just send them as plain text in line? It makes it more convenient to read them quickly. For the record, here is Pat's proposal:
*snip*
OK, so here is a new draft based on a kind of merge of Pat's recent drafts and my earlier drafts. For the record, my previous draft was 555 words, Pat's last draft is 258 words (without counting the paragraph numbers and without any wikitext like mine has), and this is 445 words. I think Pat's draft left out some necessary connective tissue, though (like what exactly this concept is *for*, and details on how exactly the review should be handled, and it kinda smooshed together the 'last minute' and 'difficult to fix' concepts), so I don't think I can cut much more.
+++++++++++ DRAFT START +++++++++++
=== Exceptional cases ===
Generally speaking, any bug that is agreed to be a violation of the [[Fedora Release Criteria|release criteria]] should be accepted as a blocker bug for the next relevant milestone release. However, bearing in mind the [[Fedora_Release_Life_Cycle|Fedora life cycle's]] emphasis on '''both''' time '''and''' quality, in some cases we may make an exception. There are two main categories of bug that may be 'exceptional':
# '''Last minute blocker bugs''' - bugs proposed as blockers 7 days or fewer before the scheduled [[Go_No_Go_Meeting]] for a milestone release (Beta or Final) can be considered under this policy, as there are some circumstances in which we believe it is not sensible to delay an otherwise-impending release to fix a bug which would usually be accepted as a blocker if discovered earlier. In these circumstances, the bug can instead be accepted as a blocker for the ''next'' milestone release. # '''Difficult to fix blocker bugs''' - bugs which it may not be practical to fix within a reasonable time frame for the release to be made (due to e.g. complexity or resource constraints)
The stakeholder groups must first agree, following the procedures described above, that the bug violates the release criteria and so would otherwise be accepted as a blocker bug for the imminent release.
After that, the stakeholder groups may separately make a decision as to whether to invoke this policy and consider delaying the blocker status to a future milestone release. Anyone attending the meeting (or otherwise taking part in the discussion, if it is being done outside of a meeting) can suggest that this evaluation be done. In making the decision, the following factors can be considered:
* How prominently visible the bug will be * How severe the consequences of the bug are * How many users are likely to encounter the bug * Whether the bug could or should have been proposed earlier in the cycle * Whether the current stable release is affected by the bug * Whether delaying the release may give us an opportunity to carry out other desirable work * Possible effects of the expected delay on Fedora itself and also to downstream projects * Whether an additional delay to fix the bug, combined with any prior delays in the cycle, results in the total delay becoming unacceptable in regard to the [[Fedora_Release_Life_Cycle]]
In almost all 'exceptional' cases, the bug should be accepted as a blocker either for the very next milestone release, or for the equivalent milestone for the next release (if it would not violate the criteria for the very next milestone). For very complex '''difficult to fix''' cases, a longer extension may be needed.
+++++++++++ DRAFT END +++++++++++
On Sat, Aug 10, 2019 at 4:04 AM Adam Williamson adamwill@fedoraproject.org wrote:
+++++++++++ DRAFT START +++++++++++
=== Exceptional cases ===
Generally speaking, any bug that is agreed to be a violation of the [[Fedora Release Criteria|release criteria]] should be accepted as a blocker bug for the next relevant milestone release. However, bearing in mind the [[Fedora_Release_Life_Cycle|Fedora life cycle's]] emphasis on '''both''' time '''and''' quality, in some cases we may make an exception. There are two main categories of bug that may be 'exceptional':
# '''Last minute blocker bugs''' - bugs proposed as blockers 7 days or fewer before the scheduled [[Go_No_Go_Meeting]] for a milestone release (Beta or Final) can be considered under this policy, as there are some circumstances in which we believe it is not sensible to delay an otherwise-impending release to fix a bug which would usually be accepted as a blocker if discovered earlier. In these circumstances, the bug can instead be accepted as a blocker for the ''next'' milestone release. # '''Difficult to fix blocker bugs''' - bugs which it may not be practical to fix within a reasonable time frame for the release to be made (due to e.g. complexity or resource constraints)
The stakeholder groups must first agree, following the procedures described above, that the bug violates the release criteria and so would otherwise be accepted as a blocker bug for the imminent release.
After that, the stakeholder groups may separately make a decision as to whether to invoke this policy and consider delaying the blocker status to a future milestone release. Anyone attending the meeting (or otherwise taking part in the discussion, if it is being done outside of a meeting) can suggest that this evaluation be done. In making the decision, the following factors can be considered:
- How prominently visible the bug will be
- How severe the consequences of the bug are
- How many users are likely to encounter the bug
- Whether the bug could or should have been proposed earlier in the
cycle
- Whether the current stable release is affected by the bug
- Whether delaying the release may give us an opportunity to carry out
other desirable work
- Possible effects of the expected delay on Fedora itself and also to
downstream projects
- Whether an additional delay to fix the bug, combined with any prior
delays in the cycle, results in the total delay becoming unacceptable in regard to the [[Fedora_Release_Life_Cycle]]
In almost all 'exceptional' cases, the bug should be accepted as a blocker either for the very next milestone release, or for the equivalent milestone for the next release (if it would not violate the criteria for the very next milestone). For very complex '''difficult to fix''' cases, a longer extension may be needed.
+++++++++++ DRAFT END +++++++++++
That's very well written and I don't have any concerns about its wording. I'm a bit hesitant whether 7 is the right number, but we can try it and see.
Whether this new policy is a good idea, that's a separate question. The idealist in me cries every time we sacrifice quality. And this policy will probably result in more bugs being waved compared to the past. However, I feel it's better to have the rules formalized than to wave such bugs without any real grounds and feel like cheating on our policies every time we actually need waive something. So yeah, I guess I have no objections to this being a part of our release criteria.
On Mon, 2019-08-12 at 22:39 +0200, Kamil Paral wrote:
On Sat, Aug 10, 2019 at 4:04 AM Adam Williamson adamwill@fedoraproject.org wrote:
+++++++++++ DRAFT START +++++++++++
=== Exceptional cases ===
Generally speaking, any bug that is agreed to be a violation of the [[Fedora Release Criteria|release criteria]] should be accepted as a blocker bug for the next relevant milestone release. However, bearing in mind the [[Fedora_Release_Life_Cycle|Fedora life cycle's]] emphasis on '''both''' time '''and''' quality, in some cases we may make an exception. There are two main categories of bug that may be 'exceptional':
# '''Last minute blocker bugs''' - bugs proposed as blockers 7 days or fewer before the scheduled [[Go_No_Go_Meeting]] for a milestone release (Beta or Final) can be considered under this policy, as there are some circumstances in which we believe it is not sensible to delay an otherwise-impending release to fix a bug which would usually be accepted as a blocker if discovered earlier. In these circumstances, the bug can instead be accepted as a blocker for the ''next'' milestone release.
[snip]
That's very well written and I don't have any concerns about its wording. I'm a bit hesitant whether 7 is the right number, but we can try it and see.
I should've called that up for discussion more prominently, I guess. Yes, I just picked that number out of the air, it's absolutely up for debate. Do you think it should be a bigger number or a smaller number? :) I wondered about whether to build in some fudge space here - say the number's a guideline and we can go beyond it if we think it's a good idea - but worried that might be a bit too messy.
Whether this new policy is a good idea, that's a separate question. The idealist in me cries every time we sacrifice quality. And this policy will probably result in more bugs being waved compared to the past. However, I feel it's better to have the rules formalized than to wave such bugs without any real grounds and feel like cheating on our policies every time we actually need waive something.
FWIW my intent would be to use this no more often than we currently do this in a non-formalized way. But of course it's possible that having it written down will make us use it more, that's a risk indeed.
On Mon, Aug 12, 2019 at 11:40 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Mon, 2019-08-12 at 22:39 +0200, Kamil Paral wrote:
On Sat, Aug 10, 2019 at 4:04 AM Adam Williamson <
adamwill@fedoraproject.org>
wrote:
+++++++++++ DRAFT START +++++++++++
=== Exceptional cases ===
Generally speaking, any bug that is agreed to be a violation of the [[Fedora Release Criteria|release criteria]] should be accepted as a blocker bug for the next relevant milestone release. However, bearing in mind the [[Fedora_Release_Life_Cycle|Fedora life cycle's]] emphasis on '''both''' time '''and''' quality, in some cases we may make an exception. There are two main categories of bug that may be 'exceptional':
# '''Last minute blocker bugs''' - bugs proposed as blockers 7 days or fewer before the scheduled [[Go_No_Go_Meeting]] for a milestone release (Beta or Final) can be considered under this policy, as there are some circumstances in which we believe it is not sensible to delay an otherwise-impending release to fix a bug which would usually be accepted as a blocker if discovered earlier. In these circumstances, the bug can instead be accepted as a blocker for the ''next'' milestone release.
[snip]
That's very well written and I don't have any concerns about its wording. I'm a bit hesitant whether 7 is the right number, but we can try it and
see.
I should've called that up for discussion more prominently, I guess. Yes, I just picked that number out of the air, it's absolutely up for debate. Do you think it should be a bigger number or a smaller number? :) I wondered about whether to build in some fudge space here - say the number's a guideline and we can go beyond it if we think it's a good idea - but worried that might be a bit too messy.
I thought about it and 7 days probably sounds OK, considering it also includes the weekend. I think I wouldn't increase it, but consider decreasing it if people think it's the right way to go.
On Tue, Aug 13, 2019 at 8:46 AM Kamil Paral kparal@redhat.com wrote:
I thought about it and 7 days probably sounds OK, considering it also includes the weekend. I think I wouldn't increase it, but consider decreasing it if people think it's the right way to go.
7 is definitely higher than I would have proposed. I'm not opposed to it, but I was thinking something closer to 3 days. That would make it, in practice, "if it's not up for discussion on the last regular blocker review meeting before Go/No-Go, then it doesn't count." The only bug that's really made me mad in the years
On Mon, Aug 12, 2019 at 4:39 PM Kamil Paral kparal@redhat.com wrote:
Whether this new policy is a good idea, that's a separate question. The idealist in me cries every time we sacrifice quality. And this policy will probably result in more bugs being waved compared to the past. However, I feel it's better to have the rules formalized than to wave such bugs without any real grounds and feel like cheating on our policies every time we actually need waive something. So yeah, I guess I have no objections to this being a part of our release criteria.
I agree with all points. It's better to explicitly say "we may choose to go ahead under some circumstances" than to do rules-lawyering to find loopholes that allow us to say "no, we really did follow the criteria, this isn't a blocker because $reasons".
I'm +1 to Adam's draft as-is. We can always revise it again later if experience finds flaws for us.
On Wed, 2019-08-14 at 13:13 -0400, Ben Cotton wrote:
On Tue, Aug 13, 2019 at 8:46 AM Kamil Paral kparal@redhat.com wrote:
I thought about it and 7 days probably sounds OK, considering it also includes the weekend. I think I wouldn't increase it, but consider decreasing it if people think it's the right way to go.
7 is definitely higher than I would have proposed. I'm not opposed to it, but I was thinking something closer to 3 days. That would make it, in practice, "if it's not up for discussion on the last regular blocker review meeting before Go/No-Go, then it doesn't count." The only bug that's really made me mad in the years
Wow, let's get into details :) So, go/no-go is on a Thursday. Last blocker meeting is Monday. So the proposed date in my draft is "the Thursday before go/no-go". Ben's proposed 3 days would be "the Monday before go/no-go", same day as the final blocker review meeting.
Once a bug is proposed as a blocker, these are the *minimum* required things that have to happen if it actually is a blocker:
* At least 3 stakeholder group members need to vote on blocker status * Someone needs to fix the bug and submit an update (packager, or a provenpackager) * Someone needs to run a compose with the fix included * That compose needs to be tested
If we say 3 days, then we're committing to doing all of that for a blocker that's proposed the Sunday before the go/no-go - one day before the final blocker review meeting. Which, I mean...yeah, we have time to do all that. Juuust barely. :)
I dunno, it's hard to be sure what number is right exactly.
On Wed, Aug 14, 2019 at 1:26 PM Adam Williamson adamwill@fedoraproject.org wrote:
If we say 3 days, then we're committing to doing all of that for a blocker that's proposed the Sunday before the go/no-go - one day before the final blocker review meeting. Which, I mean...yeah, we have time to do all that. Juuust barely. :)
I mean, it seems like we do that now anyway. I'm definitely okay with you saying "yes, but the status quo is still kinda terrible." And I'm okay with finding a bug and not being able to fix it in time for it to be in the RC. What I don't want happening is for a bug to pop up just before the meeting so that we end up spending half of of the go/no-go meeting trying to reproduce the bug report.
I dunno, it's hard to be sure what number is right exactly.
Only because there's no right answer. :-) I'm fine with choosing 7 days as a starting point. If that's the only area of disagreement on this proposal, then I'd say you've done pretty well.
I have the same doubts about the 7. Well, nobody is sure ;) +1 on my side, great work Pat & Adam
On August 14, 2019 7:33:03 PM GMT+02:00, Ben Cotton bcotton@redhat.com wrote:
On Wed, Aug 14, 2019 at 1:26 PM Adam Williamson adamwill@fedoraproject.org wrote:
If we say 3 days, then we're committing to doing all of that for a blocker that's proposed the Sunday before the go/no-go - one day
before
the final blocker review meeting. Which, I mean...yeah, we have time
to
do all that. Juuust barely. :)
I mean, it seems like we do that now anyway. I'm definitely okay with you saying "yes, but the status quo is still kinda terrible." And I'm okay with finding a bug and not being able to fix it in time for it to be in the RC. What I don't want happening is for a bug to pop up just before the meeting so that we end up spending half of of the go/no-go meeting trying to reproduce the bug report.
I dunno, it's hard to be sure what number is right exactly.
Only because there's no right answer. :-) I'm fine with choosing 7 days as a starting point. If that's the only area of disagreement on this proposal, then I'd say you've done pretty well.
Julen Landa Alustiza jlanda@fedoraproject.org
On Wed, Aug 14, 2019 at 11:26 AM Adam Williamson adamwill@fedoraproject.org wrote:
If we say 3 days, then we're committing to doing all of that for a blocker that's proposed the Sunday before the go/no-go - one day before the final blocker review meeting. Which, I mean...yeah, we have time to do all that. Juuust barely. :)
I dunno, it's hard to be sure what number is right exactly.
Consistent with the existing wording, any bug proposed during this period has the smell of being "a last minute bug". It's a valid position that consideration needs to save it from the policy, rather than whether the policy applies. The category applies by virtue of the bug being proposed during the period, there is a bias from the outset.
Ergo, in fact you have more bugs with this implicit bias with a 7 day period, than a 3 day period. If it's a three day period, the last Monday blocker means those blockers are blockers. That's rather last minute, good chance they aren't getting fixed and we have to slip. If this is 7 days, well only the really meaningful ones would cause a slip, otherwise they are blockers for the next round.
In some ways this reminds me of printing newspapers. Those folks have to print *another* paper tomorrow, so they can't fix all mistakes. They simply run out of time, there really is a hard and fast deadline. Every day. And I like anything that reduces the testing stress of the final week to finding the really nasty bugs that we really shouldn't ship with that can't easily be fixed with an update or just plain look bad.
Another option here, is applying the last minute blocker to the Release Target #1 week, and not to the Preferred Target week. Of even, if it's a 3 day period apply it to the Preferred Target week; and if 7 days apply to Target #1 week.
On Wed, 2019-08-14 at 14:26 -0600, Chris Murphy wrote:
Another option here, is applying the last minute blocker to the Release Target #1 week, and not to the Preferred Target week. Of even, if it's a 3 day period apply it to the Preferred Target week; and if 7 days apply to Target #1 week.
Hmm, that's an interesting point, I was kinda ignoring the whole 'preferred target' malarkey with this policy. Indeed the go/no-go meetings are at least initially scheduled in relation to the 'preferred target' dates.
Still, I don't want to go making the policy more complicated again...
Hmm. Since we've had a proposal for 7 days and a proposal for 3 days, why not just split the difference and say 5 days? That would be the Saturday before go/no-go: so anything proposed the week before isn't covered, but from the weekend on, we're in 'last minute' territory. How does that sound?
On Wed, Aug 14, 2019 at 4:48 PM Adam Williamson adamwill@fedoraproject.org wrote:
Hmm. Since we've had a proposal for 7 days and a proposal for 3 days, why not just split the difference and say 5 days? That would be the Saturday before go/no-go: so anything proposed the week before isn't covered, but from the weekend on, we're in 'last minute' territory. How does that sound?
That works for me, although I'm just as happy with going for the 7-day option. After all, the time frame is when we *can* choose to ignore the bug, we're not obligated to.
On Wed, Aug 14, 2019 at 11:14 AM Ben Cotton bcotton@redhat.com wrote:
On Tue, Aug 13, 2019 at 8:46 AM Kamil Paral kparal@redhat.com wrote:
I thought about it and 7 days probably sounds OK, considering it also includes the weekend. I think I wouldn't increase it, but consider decreasing it if people think it's the right way to go.
7 is definitely higher than I would have proposed. I'm not opposed to it, but I was thinking something closer to 3 days. That would make it, in practice, "if it's not up for discussion on the last regular blocker review meeting before Go/No-Go, then it doesn't count." The only bug that's really made me mad in the years
Go on then!
On Mon, Aug 12, 2019 at 4:39 PM Kamil Paral kparal@redhat.com wrote:
Whether this new policy is a good idea, that's a separate question. The idealist in me cries every time we sacrifice quality. And this policy will probably result in more bugs being waved compared to the past. However, I feel it's better to have the rules formalized than to wave such bugs without any real grounds and feel like cheating on our policies every time we actually need waive something. So yeah, I guess I have no objections to this being a part of our release criteria.
I agree with all points. It's better to explicitly say "we may choose to go ahead under some circumstances" than to do rules-lawyering to find loopholes that allow us to say "no, we really did follow the criteria, this isn't a blocker because $reasons".
I'm +1 to Adam's draft as-is. We can always revise it again later if experience finds flaws for us.
Agreed. The question I had about collapsing the two categories is probably academic, and probably doesn't simplify anything.
On Wed, Aug 14, 2019 at 3:57 PM Chris Murphy lists@colorremedies.com wrote:
On Wed, Aug 14, 2019 at 11:14 AM Ben Cotton bcotton@redhat.com wrote:
only bug that's really made me mad in the years
Go on then!
Whoops! I was going to say the only bug that's really made me mad in the year I've been FPgM is the one that got nominated as a blocker within a few mintues of the Go/No-Go meeting starting (I don't recall if it was before or after, I just know that it was pretty close and I got sad)
On Wed, Aug 14, 2019 at 2:09 PM Ben Cotton bcotton@redhat.com wrote:
On Wed, Aug 14, 2019 at 3:57 PM Chris Murphy lists@colorremedies.com wrote:
On Wed, Aug 14, 2019 at 11:14 AM Ben Cotton bcotton@redhat.com wrote:
only bug that's really made me mad in the years
Go on then!
Whoops! I was going to say the only bug that's really made me mad in the year I've been FPgM is the one that got nominated as a blocker within a few mintues of the Go/No-Go meeting starting (I don't recall if it was before or after, I just know that it was pretty close and I got sad)
Oh I've seen a few of those same day blocker bugs. They are a surprise, but they never make me sad. :-D It's Kamil's sad panda that make me sad!
On Fri, Aug 9, 2019 at 8:04 PM Adam Williamson adamwill@fedoraproject.org wrote:
# '''Last minute blocker bugs''' - bugs proposed as blockers 7 days or fewer before the scheduled [[Go_No_Go_Meeting]] for a milestone release (Beta or Final) can be considered under this policy, as there are some circumstances in which we believe it is not sensible to delay an otherwise-impending release to fix a bug which would usually be accepted as a blocker if discovered earlier. In these circumstances, the bug can instead be accepted as a blocker for the ''next'' milestone release. # '''Difficult to fix blocker bugs''' - bugs which it may not be practical to fix within a reasonable time frame for the release to be made (due to e.g. complexity or resource constraints)
Are these collapsible? That is, ostensibly the reason to trigger a last minute blocker bug exception, would be that it can't be fixed in say, 12 hours. Right? So chances are it's somehow difficult to fix, or possibly difficult to fix and test (perhaps mainly for regressions).
A "difficult to fix and confidently test within remaining time constraints" single category and exception?
On Mon, 2019-08-12 at 16:11 -0600, Chris Murphy wrote:
On Fri, Aug 9, 2019 at 8:04 PM Adam Williamson adamwill@fedoraproject.org wrote:
# '''Last minute blocker bugs''' - bugs proposed as blockers 7 days or fewer before the scheduled [[Go_No_Go_Meeting]] for a milestone release (Beta or Final) can be considered under this policy, as there are some circumstances in which we believe it is not sensible to delay an otherwise-impending release to fix a bug which would usually be accepted as a blocker if discovered earlier. In these circumstances, the bug can instead be accepted as a blocker for the ''next'' milestone release. # '''Difficult to fix blocker bugs''' - bugs which it may not be practical to fix within a reasonable time frame for the release to be made (due to e.g. complexity or resource constraints)
Are these collapsible? That is, ostensibly the reason to trigger a last minute blocker bug exception, would be that it can't be fixed in say, 12 hours. Right? So chances are it's somehow difficult to fix, or possibly difficult to fix and test (perhaps mainly for regressions).
A "difficult to fix and confidently test within remaining time constraints" single category and exception?
In theory, yeah. In practice, I feel like if we tried to be clever and just word it that way we'd wind up having to explain why an apparently simple fix that popped up at the last minute was taken under this "difficult to fix" clause, and then I think the text would wind having more or less the same bits as it does now, only in a less clear construction. But...if you think it can be done, take a cut at it! Post a draft :)
On Fri, 2019-08-09 at 19:04 -0700, Adam Williamson wrote:
On Fri, 2019-08-09 at 18:26 -0700, Adam Williamson wrote:
On Wed, 2019-07-24 at 10:04 -0400, pmkellly@frontier.com wrote:
I got feedback from Adam and Ben today; so the following changes have been made:
I have added a little paragraph at the beginning to say what a last minute blocker bug is. I used freeze as the time anchor rather than a meeting since that seems to be the most firm time constraint we work to. Perhaps the review meetings could be anchored to the freeze as well. There might be some merit to showing the critical meetings in the schedule list that gets published.
I changed "team" to "stakeholder groups"
I removed "proposed" from places where it really didn't add anything.
Have a Great Day!
Pat (tablepc)
Thanks Pat! For future drafts, can you please just send them as plain text in line? It makes it more convenient to read them quickly. For the record, here is Pat's proposal:
*snip*
OK, so here is a new draft based on a kind of merge of Pat's recent drafts and my earlier drafts. For the record, my previous draft was 555 words, Pat's last draft is 258 words (without counting the paragraph numbers and without any wikitext like mine has), and this is 445 words. I think Pat's draft left out some necessary connective tissue, though (like what exactly this concept is *for*, and details on how exactly the review should be handled, and it kinda smooshed together the 'last minute' and 'difficult to fix' concepts), so I don't think I can cut much more.
OK, so based on the follow-ups here, I went ahead and merged this draft, only with '7 days' changed to '5 days':
https://fedoraproject.org/w/index.php?title=QA%3ASOP_blocker_bug_process&...
Thanks folks!
+++++++++++ DRAFT START +++++++++++
=== Exceptional cases ===
Generally speaking, any bug that is agreed to be a violation of the [[Fedora Release Criteria|release criteria]] should be accepted as a blocker bug for the next relevant milestone release. However, bearing in mind the [[Fedora_Release_Life_Cycle|Fedora life cycle's]] emphasis on '''both''' time '''and''' quality, in some cases we may make an exception. There are two main categories of bug that may be 'exceptional':
# '''Last minute blocker bugs''' - bugs proposed as blockers 7 days or fewer before the scheduled [[Go_No_Go_Meeting]] for a milestone release (Beta or Final) can be considered under this policy, as there are some circumstances in which we believe it is not sensible to delay an otherwise-impending release to fix a bug which would usually be accepted as a blocker if discovered earlier. In these circumstances, the bug can instead be accepted as a blocker for the ''next'' milestone release. # '''Difficult to fix blocker bugs''' - bugs which it may not be practical to fix within a reasonable time frame for the release to be made (due to e.g. complexity or resource constraints)
The stakeholder groups must first agree, following the procedures described above, that the bug violates the release criteria and so would otherwise be accepted as a blocker bug for the imminent release.
After that, the stakeholder groups may separately make a decision as to whether to invoke this policy and consider delaying the blocker status to a future milestone release. Anyone attending the meeting (or otherwise taking part in the discussion, if it is being done outside of a meeting) can suggest that this evaluation be done. In making the decision, the following factors can be considered:
- How prominently visible the bug will be
- How severe the consequences of the bug are
- How many users are likely to encounter the bug
- Whether the bug could or should have been proposed earlier in the
cycle
- Whether the current stable release is affected by the bug
- Whether delaying the release may give us an opportunity to carry out
other desirable work
- Possible effects of the expected delay on Fedora itself and also to
downstream projects
- Whether an additional delay to fix the bug, combined with any prior
delays in the cycle, results in the total delay becoming unacceptable in regard to the [[Fedora_Release_Life_Cycle]]
In almost all 'exceptional' cases, the bug should be accepted as a blocker either for the very next milestone release, or for the equivalent milestone for the next release (if it would not violate the criteria for the very next milestone). For very complex '''difficult to fix''' cases, a longer extension may be needed.
+++++++++++ DRAFT END +++++++++++
Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net _______________________________________________ test mailing list -- test@lists.fedoraproject.org To unsubscribe send an email to test-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/test@lists.fedoraproject.org
On Fri, Aug 30, 2019 at 3:11 AM Adam Williamson adamwill@fedoraproject.org wrote:
On Fri, 2019-08-09 at 19:04 -0700, Adam Williamson wrote:
On Fri, 2019-08-09 at 18:26 -0700, Adam Williamson wrote:
On Wed, 2019-07-24 at 10:04 -0400, pmkellly@frontier.com wrote:
I got feedback from Adam and Ben today; so the following changes
have
been made:
I have added a little paragraph at the beginning to say what a last minute blocker bug is. I used freeze as the time anchor rather than
a
meeting since that seems to be the most firm time constraint we work
to.
Perhaps the review meetings could be anchored to the freeze as well. There might be some merit to showing the critical meetings in the schedule list that gets published.
I changed "team" to "stakeholder groups"
I removed "proposed" from places where it really didn't add anything.
Have a Great Day!
Pat (tablepc)
Thanks Pat! For future drafts, can you please just send them as plain text in line? It makes it more convenient to read them quickly. For the record, here is Pat's proposal:
*snip*
OK, so here is a new draft based on a kind of merge of Pat's recent drafts and my earlier drafts. For the record, my previous draft was 555 words, Pat's last draft is 258 words (without counting the paragraph numbers and without any wikitext like mine has), and this is 445 words. I think Pat's draft left out some necessary connective tissue, though (like what exactly this concept is *for*, and details on how exactly the review should be handled, and it kinda smooshed together the 'last minute' and 'difficult to fix' concepts), so I don't think I can cut much more.
OK, so based on the follow-ups here, I went ahead and merged this draft, only with '7 days' changed to '5 days':
https://fedoraproject.org/w/index.php?title=QA%3ASOP_blocker_bug_process&...
Thanks folks!
I'd like to revive this topic. Yesterday [1], the last minute blocker policy was misused (at least in my eyes) to ignore the workstation oversize bug [2], which was already accepted as an automatic blocker. I believe it was an inappropriate solution to the situation, and last minute blocker policy directly contradicts automatic blocker policy, i.e. they can't be used together, and the latter automatically beats the former.
As I feel it (and would like to have it), "automatic blockers" imply they are such core and basic issues that they are non-questionable and non-waivable (except by FESCo, which is itself part of the same policy and marked to have godly powers). If you read the list of automatic blockers [3], those are broken composes, dead-on-arrival images, incorrect checksums, broken dependencies, and *oversize images*. I don't think anyone but FESCo should be able to say "go" in that case, regardless of when the problem was reported (even minutes before the final meeting). I'd really like automatic to mean automatic, without any consideration.
As a result, I propose that we add a note to the automatic blockers policy description that sounds something like this: "Automatic blockers can't un-accepted (waived), with the exception of FESCo."
An alternative option would be to exclude just "last minute blocker bugs" exception, but keep "difficult to fix" exception: "Automatic blockers can't be subject to last minute blocker bugs exception policy." This would allow people in blocker review/Go-NoGo meeting to delay an automatic blocker because it was difficult to fix, but would not allow them to delay it because it was reported too late ("delay" here can mean from F31 to F32, not just Beta to Final). For any other reason, the issue would have to be raised to FESCo.
I think the first proposal is better, but the second version works for me as well.
Anything that we don't deem "absolutely essential" should not be part of the automatic blocker list. If we're OK with not keeping the maximum image size during Beta, we should not have it there. The same applies to anything else in the list.
Don't misunderstand my email. I'm glad that F31 Beta is go. And I'm also not sold on the idea of delaying Beta release because Workstation image is 50 MB over size. But I believe we shouldn't sacrifice our policies consistency to achieve that goal. That's why I want to clarify our policies. And then we can talk about dealing with this particular situation in a better way (by excluding Beta image sizes from automatic blockers, or perhaps keeping them just for release-blocking optical images, or bumping the Workstation maximum size, improving our QA processes, etc).
Thanks, Kamil
[1] https://meetbot.fedoraproject.org/fedora-meeting-1/2019-09-12/f31-beta-go_no... [2] https://bugzilla.redhat.com/show_bug.cgi?id=1751438 [3] https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process#Automatic_blockers
On Fri, Sep 13, 2019 at 5:54 AM Kamil Paral kparal@redhat.com wrote:
I'd like to revive this topic. Yesterday [1], the last minute blocker policy was misused (at least in my eyes) to ignore the workstation oversize bug [2], which was already accepted as an automatic blocker. I believe it was an inappropriate solution to the situation, and last minute blocker policy directly contradicts automatic blocker policy, i.e. they can't be used together, and the latter automatically beats the former.
This is compelling. We don't even vote on automatic blockers. Anyone can fill in the whiteboard.
As I feel it (and would like to have it), "automatic blockers" imply they are such core and basic issues that they are non-questionable and non-waivable (except by FESCo, which is itself part of the same policy and marked to have godly powers). If you read the list of automatic blockers [3], those are broken composes, dead-on-arrival images, incorrect checksums, broken dependencies, and *oversize images*. I don't think anyone but FESCo should be able to say "go" in that case, regardless of when the problem was reported (even minutes before the final meeting). I'd really like automatic to mean automatic, without any consideration.
As a result, I propose that we add a note to the automatic blockers policy description that sounds something like this: "Automatic blockers can't un-accepted (waived), with the exception of FESCo."
An alternative option would be to exclude just "last minute blocker bugs" exception, but keep "difficult to fix" exception: "Automatic blockers can't be subject to last minute blocker bugs exception policy." This would allow people in blocker review/Go-NoGo meeting to delay an automatic blocker because it was difficult to fix, but would not allow them to delay it because it was reported too late ("delay" here can mean from F31 to F32, not just Beta to Final). For any other reason, the issue would have to be raised to FESCo.
I think the first proposal is better, but the second version works for me as well.
Anything that we don't deem "absolutely essential" should not be part of the automatic blocker list. If we're OK with not keeping the maximum image size during Beta, we should not have it there. The same applies to anything else in the list.
Don't misunderstand my email. I'm glad that F31 Beta is go. And I'm also not sold on the idea of delaying Beta release because Workstation image is 50 MB over size. But I believe we shouldn't sacrifice our policies consistency to achieve that goal. That's why I want to clarify our policies. And then we can talk about dealing with this particular situation in a better way (by excluding Beta image sizes from automatic blockers, or perhaps keeping them just for release-blocking optical images, or bumping the Workstation maximum size, improving our QA processes, etc).
I agree policies become diluted and vague, when they aren't followed. If there's potential for judgement call, write in a tolerance. e.g. Images can be up to 5% oversize for beta.
Otherwise, even hockey has more consistent rules with fewer exceptions. And the decision really comes down to who makes the most persuasive argument at that moment, rather than a set of policies that provides the guidance.
On Fri, 2019-09-13 at 13:53 +0200, Kamil Paral wrote:
As I feel it (and would like to have it), "automatic blockers" imply they are such core and basic issues that they are non-questionable and non-waivable (except by FESCo, which is itself part of the same policy and marked to have godly powers). If you read the list of automatic blockers [3], those are broken composes, dead-on-arrival images, incorrect checksums, broken dependencies, and *oversize images*. I don't think anyone but FESCo should be able to say "go" in that case, regardless of when the problem was reported (even minutes before the final meeting). I'd really like automatic to mean automatic, without any consideration.
FWIW, I respectfully disagree with this. The key factor for whether something goes on the 'automatic' blockers list is *whether it could possibly be ambiguous*, not how waivable it is.
It happens to be the case that *most* things that are really non- ambiguous are terribly critical things we would never postpone under the last-minute policy - like an image simply failing to boot. That kinda makes sense if you think about it: catastrophic things tend to be very clear. But it's also possible for something to be very clear but not necessarily critical. To me, the size limits - at least, ones which don't match any significant media size - are that. Size limits are on the automatic blocker list because there's really no need for three teams to debate and vote on whether one number is bigger than another, not because exceeding the size limit is a catastrophic bug.
Note also that under the last-minute blocker policy as written we're supposed to *first* decide that the issue is a blocker - i.e. accept it in principle - *then* decide to push that status to a later milestone. i.e. the policy effectively applies to bugs that are accepted as blockers. The process isn't that we reject the bug as a blocker under the last-minute blocker policy: the process is that we accept it but agree that moving it to the next milestone is acceptable. So, I don't think the incompatibility between the two policies is that bad at all. I think we could simply clarify that the last-minute blocker policy can also be applied to bugs that have already been accepted under the 'automatic blocker' policy, so long as they were *proposed* within the 5 day time limit.
To me this was actually a very appropriate bug for the last-minute policy: it's a not-terribly-critical problem that, through process failure, was only raised at the last minute even though it was known about for ages. I feel bad for not noticing it had been failing for two months, but I don't feel bad about applying the last-minute blocker policy to it. I don't think releasing a Beta with a Workstation live image that's 2.04GB in size instead of 1.99GB in size is going to cause Fedora huge problems.
On Fri, Sep 13, 2019 at 10:43 AM Adam Williamson adamwill@fedoraproject.org wrote:
On Fri, 2019-09-13 at 13:53 +0200, Kamil Paral wrote:
As I feel it (and would like to have it), "automatic blockers" imply they are such core and basic issues that they are non-questionable and non-waivable (except by FESCo, which is itself part of the same policy and marked to have godly powers). If you read the list of automatic blockers [3], those are broken composes, dead-on-arrival images, incorrect checksums, broken dependencies, and *oversize images*. I don't think anyone but FESCo should be able to say "go" in that case, regardless of when the problem was reported (even minutes before the final meeting). I'd really like automatic to mean automatic, without any consideration.
FWIW, I respectfully disagree with this. The key factor for whether something goes on the 'automatic' blockers list is *whether it could possibly be ambiguous*, not how waivable it is.
While I agree that automatic blockers are more about their unambiguous nature, it is also unambiguous that image size is presently a beta requirement. That it can be waived, and also that it was overwhelmingly waived, very strongly suggests a) it should not be a beta requirement; b) the present maximum size is incorrect and needs updating; c) there's some kind of unwritten fudge factor, maybe 5% oversize for beta is OK but 100% wouldn't be.
Essentially, no one found it intuitively compelling or desirable to actually block release, even though it's unquestionably a blocker. Since the bug is not blocking the release tells us it is in fact not a release blocking bug. It's a blocker in name only, in practice it's not. It's a contradiction, and thus hilarious.
Anyway, the Workstation working group will revisit the maximum size issue on Monday, and perhaps there won't be a contradiction on release day. :D
https://pagure.io/fedora-workstation/issue/104
On Fri, 2019-09-13 at 12:34 -0600, Chris Murphy wrote:
On Fri, Sep 13, 2019 at 10:43 AM Adam Williamson adamwill@fedoraproject.org wrote:
On Fri, 2019-09-13 at 13:53 +0200, Kamil Paral wrote:
As I feel it (and would like to have it), "automatic blockers" imply they are such core and basic issues that they are non-questionable and non-waivable (except by FESCo, which is itself part of the same policy and marked to have godly powers). If you read the list of automatic blockers [3], those are broken composes, dead-on-arrival images, incorrect checksums, broken dependencies, and *oversize images*. I don't think anyone but FESCo should be able to say "go" in that case, regardless of when the problem was reported (even minutes before the final meeting). I'd really like automatic to mean automatic, without any consideration.
FWIW, I respectfully disagree with this. The key factor for whether something goes on the 'automatic' blockers list is *whether it could possibly be ambiguous*, not how waivable it is.
While I agree that automatic blockers are more about their unambiguous nature, it is also unambiguous that image size is presently a beta requirement. That it can be waived, and also that it was overwhelmingly waived, very strongly suggests a) it should not be a beta requirement; b) the present maximum size is incorrect and needs updating; c) there's some kind of unwritten fudge factor, maybe 5% oversize for beta is OK but 100% wouldn't be.
Let's be clear about what we mean by 'waiving'. Actually I'd like to avoid using that word at all (yes I know I did it once, I edited my mail to take them out but missed one).
We are *not* 'waiving' the criterion. We are *not* deciding the bug isn't a blocker. We are deciding to mark it as blocking the next milestone instead of the current one.
Essentially, no one found it intuitively compelling or desirable to actually block release, even though it's unquestionably a blocker. Since the bug is not blocking the release tells us it is in fact not a release blocking bug. It's a blocker in name only, in practice it's not. It's a contradiction, and thus hilarious.
But this would be the case for any bug we apply the 'last minute' policy to. There is nothing specific to automatic blockers in anything you write, yet we agreed the policy anyway, and AFAIR you +1ed it.
On Fri, Sep 13, 2019 at 1:10 PM Adam Williamson adamwill@fedoraproject.org wrote:
Let's be clear about what we mean by 'waiving'. Actually I'd like to avoid using that word at all (yes I know I did it once, I edited my mail to take them out but missed one).
We are *not* 'waiving' the criterion. We are *not* deciding the bug isn't a blocker. We are deciding to mark it as blocking the next milestone instead of the current one.
OK yeah so it's a delayed blocker. That underscores the ISO maximum size is probably a soft requirement for beta.
Essentially, no one found it intuitively compelling or desirable to actually block release, even though it's unquestionably a blocker. Since the bug is not blocking the release tells us it is in fact not a release blocking bug. It's a blocker in name only, in practice it's not. It's a contradiction, and thus hilarious.
But this would be the case for any bug we apply the 'last minute' policy to. There is nothing specific to automatic blockers in anything you write, yet we agreed the policy anyway, and AFAIR you +1ed it.
Yes, but that's only half of the hilarity. I also -1'd blocking on ISO size.
I'm just noting the contrast between highly objective nature of automatic blockers, and the subjectivity injected by the last minute policy. I really hadn't considered that particular combination.
On Fri, Sep 13, 2019 at 6:44 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Fri, 2019-09-13 at 13:53 +0200, Kamil Paral wrote:
As I feel it (and would like to have it), "automatic blockers" imply they are such core and basic issues that they are non-questionable and non-waivable (except by FESCo, which is itself part of the same policy
and
marked to have godly powers). If you read the list of automatic blockers [3], those are broken composes, dead-on-arrival images, incorrect checksums, broken dependencies, and *oversize images*. I don't think
anyone
but FESCo should be able to say "go" in that case, regardless of when the problem was reported (even minutes before the final meeting). I'd really like automatic to mean automatic, without any consideration.
FWIW, I respectfully disagree with this. The key factor for whether something goes on the 'automatic' blockers list is *whether it could possibly be ambiguous*, not how waivable it is.
It happens to be the case that *most* things that are really non- ambiguous are terribly critical things we would never postpone under the last-minute policy - like an image simply failing to boot. That kinda makes sense if you think about it: catastrophic things tend to be very clear. But it's also possible for something to be very clear but not necessarily critical. To me, the size limits - at least, ones which don't match any significant media size - are that. Size limits are on the automatic blocker list because there's really no need for three teams to debate and vote on whether one number is bigger than another, not because exceeding the size limit is a catastrophic bug.
There are two ways how to understand automatic blockers - that they are based on bug seriousness, or obviousness. I clearly use the former one, and you the latter one. I guess that's why our views differ.
Note also that under the last-minute blocker policy as written we're supposed to *first* decide that the issue is a blocker - i.e. accept it in principle - *then* decide to push that status to a later milestone. i.e. the policy effectively applies to bugs that are accepted as blockers. The process isn't that we reject the bug as a blocker under the last-minute blocker policy: the process is that we accept it but agree that moving it to the next milestone is acceptable. So, I don't think the incompatibility between the two policies is that bad at all. I think we could simply clarify that the last-minute blocker policy can also be applied to bugs that have already been accepted under the 'automatic blocker' policy, so long as they were *proposed* within the 5 day time limit.
You've lawyered me here (I should have expected that). I wanted to argue that the delaying is supposed to happen before accepting it as a blocker, but no, you're correct.
What I see mildly annoying with this approach is that it bring uncertainty into the process. Before, an accepted blocker was an accepted blocker, and you could count on that and adjust your priorities appropriately. That means working on the fixes as a developer, or debugging it more as a QA. Now, a blocker can be delayed (postponed to a much later date) and so the dependability on the "AcceptedBlocker" label is suddenly no longer what it used to be.
To me this was actually a very appropriate bug for the last-minute policy: it's a not-terribly-critical problem that, through process failure, was only raised at the last minute even though it was known about for ages. I feel bad for not noticing it had been failing for two months, but I don't feel bad about applying the last-minute blocker policy to it. I don't think releasing a Beta with a Workstation live image that's 2.04GB in size instead of 1.99GB in size is going to cause Fedora huge problems.
I don't disagree here, but that's why I wanted to discuss the process separately from this particular issue. For me, automatic blockers were about bug criticality, and therefore my view was that we are too strict in this particular requirement for Beta release and should resolve it in a different way (e.g. by weakening the criterion).
On Mon, 2019-09-16 at 16:11 +0200, Kamil Paral wrote:
On Fri, Sep 13, 2019 at 6:44 PM Adam Williamson adamwill@fedoraproject.org wrote:
On Fri, 2019-09-13 at 13:53 +0200, Kamil Paral wrote:
As I feel it (and would like to have it), "automatic blockers" imply they are such core and basic issues that they are non-questionable and non-waivable (except by FESCo, which is itself part of the same policy
and
marked to have godly powers). If you read the list of automatic blockers [3], those are broken composes, dead-on-arrival images, incorrect checksums, broken dependencies, and *oversize images*. I don't think
anyone
but FESCo should be able to say "go" in that case, regardless of when the problem was reported (even minutes before the final meeting). I'd really like automatic to mean automatic, without any consideration.
FWIW, I respectfully disagree with this. The key factor for whether something goes on the 'automatic' blockers list is *whether it could possibly be ambiguous*, not how waivable it is.
It happens to be the case that *most* things that are really non- ambiguous are terribly critical things we would never postpone under the last-minute policy - like an image simply failing to boot. That kinda makes sense if you think about it: catastrophic things tend to be very clear. But it's also possible for something to be very clear but not necessarily critical. To me, the size limits - at least, ones which don't match any significant media size - are that. Size limits are on the automatic blocker list because there's really no need for three teams to debate and vote on whether one number is bigger than another, not because exceeding the size limit is a catastrophic bug.
There are two ways how to understand automatic blockers - that they are based on bug seriousness, or obviousness. I clearly use the former one, and you the latter one. I guess that's why our views differ.
Well, here's what I wrote when I proposed the automatic blocker process in the first place:
"There's a few types of blocker bug that are basically no-brainers; it doesn't make a lot of sense to waste time in blocker meetings discussing them, and more importantly, sometimes they show up and we want to quickly accept them as blockers and get the fixes in, but we have to try and track down three people to vote +1 before they can be accepted."
So, it was all about saving time in blocker meetings and being able to accept obvious blockers quickly.
Here's the full mail:
https://lists.fedoraproject.org/pipermail/test/2013-February/113840.html
Note also that under the last-minute blocker policy as written we're supposed to *first* decide that the issue is a blocker - i.e. accept it in principle - *then* decide to push that status to a later milestone. i.e. the policy effectively applies to bugs that are accepted as blockers. The process isn't that we reject the bug as a blocker under the last-minute blocker policy: the process is that we accept it but agree that moving it to the next milestone is acceptable. So, I don't think the incompatibility between the two policies is that bad at all. I think we could simply clarify that the last-minute blocker policy can also be applied to bugs that have already been accepted under the 'automatic blocker' policy, so long as they were *proposed* within the 5 day time limit.
You've lawyered me here (I should have expected that).
You really should ;)
I wanted to argue that the delaying is supposed to happen before accepting it as a blocker, but no, you're correct.
What I see mildly annoying with this approach is that it bring uncertainty into the process. Before, an accepted blocker was an accepted blocker, and you could count on that and adjust your priorities appropriately. That means working on the fixes as a developer, or debugging it more as a QA. Now, a blocker can be delayed (postponed to a much later date) and so the dependability on the "AcceptedBlocker" label is suddenly no longer what it used to be.
That's a fair point, but I would say it's a bit limited, as it only applies to the five days before go/no-go. Still, it's true that the 'automatic blocker' process makes this different. As we wrote/considered the 'last minute blocker' policy, without thinking about 'automatic blockers', you could say that the time frame during which a bug was accepted as a blocker but under consideration for 'last minute' status would be very short - as in, a few minutes - and it would never actually be marked in Bugzilla, as all this would happen during a meeting. But now we know that's not quite true, as this exact scenario can happen - it can be automatically accepted and tagged in Bugzilla, then come up for 'last minute' consideration during the next meeting. So the time frame expands from 'a few minutes' to 'a few days'.
To me this was actually a very appropriate bug for the last-minute policy: it's a not-terribly-critical problem that, through process failure, was only raised at the last minute even though it was known about for ages. I feel bad for not noticing it had been failing for two months, but I don't feel bad about applying the last-minute blocker policy to it. I don't think releasing a Beta with a Workstation live image that's 2.04GB in size instead of 1.99GB in size is going to cause Fedora huge problems.
I don't disagree here, but that's why I wanted to discuss the process separately from this particular issue. For me, automatic blockers were about bug criticality, and therefore my view was that we are too strict in this particular requirement for Beta release and should resolve it in a different way (e.g. by weakening the criterion).
I still consider automatic blockers to be about obviousness (not criticality), but I'm open to different resolutions here anyway, if we want to come up with something really clear.
On Mon, Sep 16, 2019 at 5:06 PM Adam Williamson adamwill@fedoraproject.org wrote:
I still consider automatic blockers to be about obviousness (not criticality), but I'm open to different resolutions here anyway, if we want to come up with something really clear.
For the record, we talked about this yesterday in QA meeting [1], and the majority opinion was that last minute blocker policy is applicable to automatic blockers. Adam promised to slightly clarify that in the SOP.
[1] search for "Automatic blockers vs. 'last minute' blockers" at https://meetbot.fedoraproject.org/teams/fedora-qa/fedora-qa.2019-09-16-15.01...