All,
I happened to be going through the upgradepath results for F15 today and I'm seeing a lot of builds with stale tags (~50 so far and I'm not done). If we re-enable comments before these builds are taken care of, we're going to be adding a lot more comments to old builds again.
This is why spot-checking (like I did before) isn't enough, I suppose.
Anyhow, I sent an email to lmacken with my initial list and I'm planning to write a script to check for stale tags tomorrow.
Tim
All,
I happened to be going through the upgradepath results for F15 today and I'm seeing a lot of builds with stale tags (~50 so far and I'm not done). If we re-enable comments before these builds are taken care of, we're going to be adding a lot more comments to old builds again.
These are bad news. If lmacken doesn't take care of that soon, I'm afraid we will have to postpone re-enabling of bodhi comments. The other approach is to write an explanation to the announcement, explain how to recognize stale koji tag from some autoqa bug and ask people to ignore comments for stable builds for now.
What do you think?
On 04/08/2011 01:01 AM, Kamil Paral wrote:
All,
I happened to be going through the upgradepath results for F15 today and I'm seeing a lot of builds with stale tags (~50 so far and I'm not done). If we re-enable comments before these builds are taken care of, we're going to be adding a lot more comments to old builds again.
These are bad news. If lmacken doesn't take care of that soon, I'm afraid we will have to postpone re-enabling of bodhi comments. The other approach is to write an explanation to the announcement, explain how to recognize stale koji tag from some autoqa bug and ask people to ignore comments for stable builds for now.
What do you think?
I guess that I'm for a bit of both, more of the waiting as long as reasonable to make sure that the tags are taken care of.
Obviously, I'd rather see the stale tags taken care of before we enable bodhi comments again. Maybe the Fedora community is better about this, but the thought process I've seen elsewhere is like "Why is AutoQA commenting this old build? It must be a broken piece of SW, I'm not sure that we can trust it" regardless of whether or not an explanation is sent over email.
If we can keep the signal/noise ratio up as high as possible, that would be awesome.
On the bright side, all of the builds that I've found so far are old (most were pushed to stable 2011-03-06 to 2011-03-08) so I'm pretty sure that the root cause has been taken care of in bodhi and koji. We just need to make sure that everything was cleaned up.
For now, let's see what lmacken says and hold off on any decision to enable or not enable comments until Monday. I assume that we'll hear back from him today at some point and he has been very responsive to this kind of stuff so far.
Tim
On 04/08/2011 08:07 AM, Tim Flink wrote:
On 04/08/2011 01:01 AM, Kamil Paral wrote:
All,
I happened to be going through the upgradepath results for F15 today and I'm seeing a lot of builds with stale tags (~50 so far and I'm not done). If we re-enable comments before these builds are taken care of, we're going to be adding a lot more comments to old builds again.
These are bad news. If lmacken doesn't take care of that soon, I'm afraid we will have to postpone re-enabling of bodhi comments. The other approach is to write an explanation to the announcement, explain how to recognize stale koji tag from some autoqa bug and ask people to ignore comments for stable builds for now.
What do you think?
I guess that I'm for a bit of both, more of the waiting as long as reasonable to make sure that the tags are taken care of.
Obviously, I'd rather see the stale tags taken care of before we enable bodhi comments again. Maybe the Fedora community is better about this, but the thought process I've seen elsewhere is like "Why is AutoQA commenting this old build? It must be a broken piece of SW, I'm not sure that we can trust it" regardless of whether or not an explanation is sent over email.
If we can keep the signal/noise ratio up as high as possible, that would be awesome.
On the bright side, all of the builds that I've found so far are old (most were pushed to stable 2011-03-06 to 2011-03-08) so I'm pretty sure that the root cause has been taken care of in bodhi and koji. We just need to make sure that everything was cleaned up.
For now, let's see what lmacken says and hold off on any decision to enable or not enable comments until Monday. I assume that we'll hear back from him today at some point and he has been very responsive to this kind of stuff so far.
Tim
As an update, I talked to lmacken today and he's looking into the stale koji tags even though he's on PTO today.
Looking at the upgradecheck results, I don't think that all the tags have been fixed yet but I will send out more information when I have it.
If anyone is interested in what I was running, the results and crappy script I was running are up at http://tflink.fedorapeople.org/tag_reports/
When I last ran it, there were 106 builds with stale tags that would be commented improperly if we re-enable bodhi comments.
Tim
On Fri, Apr 08, 2011 at 04:17:59PM -0600, Tim Flink wrote:
On 04/08/2011 08:07 AM, Tim Flink wrote:
On 04/08/2011 01:01 AM, Kamil Paral wrote:
All,
I happened to be going through the upgradepath results for F15 today and I'm seeing a lot of builds with stale tags (~50 so far and I'm not done). If we re-enable comments before these builds are taken care of, we're going to be adding a lot more comments to old builds again.
These are bad news. If lmacken doesn't take care of that soon, I'm afraid we will have to postpone re-enabling of bodhi comments. The other approach is to write an explanation to the announcement, explain how to recognize stale koji tag from some autoqa bug and ask people to ignore comments for stable builds for now.
What do you think?
I guess that I'm for a bit of both, more of the waiting as long as reasonable to make sure that the tags are taken care of.
Obviously, I'd rather see the stale tags taken care of before we enable bodhi comments again. Maybe the Fedora community is better about this, but the thought process I've seen elsewhere is like "Why is AutoQA commenting this old build? It must be a broken piece of SW, I'm not sure that we can trust it" regardless of whether or not an explanation is sent over email.
If we can keep the signal/noise ratio up as high as possible, that would be awesome.
On the bright side, all of the builds that I've found so far are old (most were pushed to stable 2011-03-06 to 2011-03-08) so I'm pretty sure that the root cause has been taken care of in bodhi and koji. We just need to make sure that everything was cleaned up.
For now, let's see what lmacken says and hold off on any decision to enable or not enable comments until Monday. I assume that we'll hear back from him today at some point and he has been very responsive to this kind of stuff so far.
Tim
As an update, I talked to lmacken today and he's looking into the stale koji tags even though he's on PTO today.
So, I'm pretty sure the last bodhi update fixed the underlying tagging issues, but there are still many builds that are stable with unnecessary testing/pending/candidate tags. The tag checking script that I wrote in bodhi queries koji for the 'latest' packages in each tag, as opposed to everything with that tag, and thus many went unnoticed. I updated the script and am running it now to clean up the rest.
luke
On Fri, Apr 08, 2011 at 06:25:47PM -0400, Luke Macken wrote:
On Fri, Apr 08, 2011 at 04:17:59PM -0600, Tim Flink wrote:
On 04/08/2011 08:07 AM, Tim Flink wrote:
On 04/08/2011 01:01 AM, Kamil Paral wrote:
All,
I happened to be going through the upgradepath results for F15 today and I'm seeing a lot of builds with stale tags (~50 so far and I'm not done). If we re-enable comments before these builds are taken care of, we're going to be adding a lot more comments to old builds again.
These are bad news. If lmacken doesn't take care of that soon, I'm afraid we will have to postpone re-enabling of bodhi comments. The other approach is to write an explanation to the announcement, explain how to recognize stale koji tag from some autoqa bug and ask people to ignore comments for stable builds for now.
What do you think?
I guess that I'm for a bit of both, more of the waiting as long as reasonable to make sure that the tags are taken care of.
Obviously, I'd rather see the stale tags taken care of before we enable bodhi comments again. Maybe the Fedora community is better about this, but the thought process I've seen elsewhere is like "Why is AutoQA commenting this old build? It must be a broken piece of SW, I'm not sure that we can trust it" regardless of whether or not an explanation is sent over email.
If we can keep the signal/noise ratio up as high as possible, that would be awesome.
On the bright side, all of the builds that I've found so far are old (most were pushed to stable 2011-03-06 to 2011-03-08) so I'm pretty sure that the root cause has been taken care of in bodhi and koji. We just need to make sure that everything was cleaned up.
For now, let's see what lmacken says and hold off on any decision to enable or not enable comments until Monday. I assume that we'll hear back from him today at some point and he has been very responsive to this kind of stuff so far.
Tim
As an update, I talked to lmacken today and he's looking into the stale koji tags even though he's on PTO today.
So, I'm pretty sure the last bodhi update fixed the underlying tagging issues, but there are still many builds that are stable with unnecessary testing/pending/candidate tags. The tag checking script that I wrote in bodhi queries koji for the 'latest' packages in each tag, as opposed to everything with that tag, and thus many went unnoticed. I updated the script and am running it now to clean up the rest.
Ok, all of the stray tags in your list have been cleaned up.
I updated bodhi's tagcheck script to find all stale tags, and also to utilize koji multiCalls to drastically speed things up. I ran your script again, and it doesn't complain about any stale tags, and the 'missing tags' are all false positives.
luke
On 04/10/2011 05:20 PM, Luke Macken wrote:
On Fri, Apr 08, 2011 at 06:25:47PM -0400, Luke Macken wrote:
On Fri, Apr 08, 2011 at 04:17:59PM -0600, Tim Flink wrote:
On 04/08/2011 08:07 AM, Tim Flink wrote:
On 04/08/2011 01:01 AM, Kamil Paral wrote:
All,
I happened to be going through the upgradepath results for F15 today and I'm seeing a lot of builds with stale tags (~50 so far and I'm not done). If we re-enable comments before these builds are taken care of, we're going to be adding a lot more comments to old builds again.
These are bad news. If lmacken doesn't take care of that soon, I'm afraid we will have to postpone re-enabling of bodhi comments. The other approach is to write an explanation to the announcement, explain how to recognize stale koji tag from some autoqa bug and ask people to ignore comments for stable builds for now.
What do you think?
I guess that I'm for a bit of both, more of the waiting as long as reasonable to make sure that the tags are taken care of.
Obviously, I'd rather see the stale tags taken care of before we enable bodhi comments again. Maybe the Fedora community is better about this, but the thought process I've seen elsewhere is like "Why is AutoQA commenting this old build? It must be a broken piece of SW, I'm not sure that we can trust it" regardless of whether or not an explanation is sent over email.
If we can keep the signal/noise ratio up as high as possible, that would be awesome.
On the bright side, all of the builds that I've found so far are old (most were pushed to stable 2011-03-06 to 2011-03-08) so I'm pretty sure that the root cause has been taken care of in bodhi and koji. We just need to make sure that everything was cleaned up.
For now, let's see what lmacken says and hold off on any decision to enable or not enable comments until Monday. I assume that we'll hear back from him today at some point and he has been very responsive to this kind of stuff so far.
Tim
As an update, I talked to lmacken today and he's looking into the stale koji tags even though he's on PTO today.
So, I'm pretty sure the last bodhi update fixed the underlying tagging issues, but there are still many builds that are stable with unnecessary testing/pending/candidate tags. The tag checking script that I wrote in bodhi queries koji for the 'latest' packages in each tag, as opposed to everything with that tag, and thus many went unnoticed. I updated the script and am running it now to clean up the rest.
Ok, all of the stray tags in your list have been cleaned up.
I updated bodhi's tagcheck script to find all stale tags, and also to utilize koji multiCalls to drastically speed things up. I ran your script again, and it doesn't complain about any stale tags, and the 'missing tags' are all false positives.
luke
Thanks for taking care of that. We really appreciate it on such short notice and especially when you were on PTO.
Looking through some of the upgradepath results, things are looking much better to me.
Yeah, I wasn't sure if those were really missing tags or not but figured that it was better to include information that might need to be ignored than to leave it out.
I'll have to look into multiCalls, though. It feels like the koji calls in autoqa take way too long when I'm running them locally. Or maybe I'm just impatient :)
Thanks again,
Tim
autoqa-devel@lists.fedorahosted.org
-
Kamil Paral -
Luke Macken -
Tim Flink