#370: bad test result --------------------+------------------------------------------------------- Reporter: till | Owner: Type: defect | Status: new Priority: major | Milestone: Future tasks Component: core | Keywords: --------------------+------------------------------------------------------- This test result is bad:
http://autoqa.fedoraproject.org/results/150039-autotest/qa02.qa.fedoraprojec... /youtube-dl-2011.08.0.html
It was for this update: https://admin.fedoraproject.org/updates/youtube-dl-2011.08.04-1.fc14
But I created a Fedora 15 and 16 update with the same version at the same time: https://admin.fedoraproject.org/updates/youtube-dl-2011.08.04-1.fc16
#370: upgradepath: bad test result ----------------------+----------------------------------------------------- Reporter: till | Owner: kparal Type: defect | Status: assigned Priority: critical | Milestone: Hot issues Component: tests | Resolution: Keywords: | ----------------------+----------------------------------------------------- Changes (by kparal):
* status: new => assigned * component: core => tests * summary: bad test result => upgradepath: bad test result * priority: major => critical * milestone: Future tasks => Hot issues * owner: => kparal
Comment:
Thanks for the report, investigating...
#370: upgradepath: bad test result ----------------------+----------------------------------------------------- Reporter: till | Owner: kparal Type: defect | Status: assigned Priority: critical | Milestone: Hot issues Component: tests | Resolution: Keywords: | ----------------------+----------------------------------------------------- Comment (by kparal):
First of all, I did not investigate the 4 days old report, I worked with the today's one:
http://autoqa.fedoraproject.org/results/153561-autotest/10.5.124.164/upgrade... /youtube-dl-2011.08.0.html
I suspect the reason will be the same though.
The problem is in Bodhi. It reports these two updates were pushed into stable: * https://admin.fedoraproject.org/updates/youtube-dl-2011.08.04-1.fc14 * https://admin.fedoraproject.org/updates/youtube-dl-2011.08.04-1.fc15
But they were not: {{{ $ koji list-tag-history --build youtube-dl-2011.08.04-1.fc14 Thu Aug 4 23:41:16 2011: youtube-dl-2011.08.04-1.fc14 tagged into dist-f14-updates-candidate by till Thu Aug 4 23:42:40 2011: youtube-dl-2011.08.04-1.fc14 tagged into dist-f14-updates-testing-pending by bodhi Fri Aug 5 02:24:58 2011: youtube-dl-2011.08.04-1.fc14 untagged from dist-f14-updates-candidate by bodhi Fri Aug 5 02:24:58 2011: youtube-dl-2011.08.04-1.fc14 tagged into dist-f14-updates-testing by bodhi [still active] Fri Aug 5 02:30:44 2011: youtube-dl-2011.08.04-1.fc14 untagged from dist-f14-updates-testing-pending by bodhi Fri Aug 5 02:49:04 2011: youtube-dl-2011.08.04-1.fc14 tagged into dist-f14-updates-pending by bodhi [still active]
$ koji list-tag-history --build youtube-dl-2011.08.04-1.fc15 Thu Aug 4 23:36:27 2011: youtube-dl-2011.08.04-1.fc15 tagged into dist-f15-updates-candidate by till Thu Aug 4 23:42:29 2011: youtube-dl-2011.08.04-1.fc15 tagged into dist-f15-updates-testing-pending by bodhi Fri Aug 5 02:25:26 2011: youtube-dl-2011.08.04-1.fc15 untagged from dist-f15-updates-candidate by bodhi Fri Aug 5 02:25:26 2011: youtube-dl-2011.08.04-1.fc15 tagged into dist-f15-updates-testing by bodhi [still active] Fri Aug 5 02:30:44 2011: youtube-dl-2011.08.04-1.fc15 untagged from dist-f15-updates-testing-pending by bodhi Fri Aug 5 02:33:00 2011: youtube-dl-2011.08.04-1.fc15 tagged into dist-f15-updates-pending by bodhi [still active] }}}
When querying Bodhi, it reports invalid results: {{{
bodhi.query(package='youtube-dl', release='F15', limit=1,
request='stable') Bunch(num_items=0, tg_flash=None, title='0 updates found', updates=[]) }}} This should have returned the 'youtube-dl-2011.08.04-1.fc15' update.
I'll report this to Bodhi Trac. Upgradepath test is working fine, but it is being fed by invalid data.
#370: upgradepath: bad test result ----------------------+----------------------------------------------------- Reporter: till | Owner: kparal Type: defect | Status: assigned Priority: critical | Milestone: Hot issues Component: tests | Resolution: Keywords: | ----------------------+----------------------------------------------------- Changes (by kparal):
* cc: lmacken (added)
Comment:
I have reported Bodhi bug here: https://fedorahosted.org/bodhi/ticket/627
#370: upgradepath: bad test result ----------------------+----------------------------------------------------- Reporter: till | Owner: kparal Type: defect | Status: assigned Priority: critical | Milestone: Hot issues Component: tests | Resolution: Keywords: | ----------------------+----------------------------------------------------- Comment (by till):
Replying to [comment:2 kparal]:
First of all, I did not investigate the 4 days old report, I worked with
the today's one:
http://autoqa.fedoraproject.org/results/153561-autotest/10.5.124.164/upgrade... /youtube-dl-2011.08.0.html
I suspect the reason will be the same though.
The problem is in Bodhi. It reports these two updates were pushed into
stable:
I'll report this to Bodhi Trac. Upgradepath test is working fine, but it
is being fed by invalid data.
You have identified another oddity with these updates, but in the original failed test AutoQA did not find the update for F16:
| [FAIL] f16 + f16-updates | Latest package: youtube-dl-2011.03.29-1.fc16 | Latest pending package: None | Error: Tested package must be less than or equal to the latest (pending) package.
But the F16 update did not have bad tags afaics.
#370: upgradepath: bad test result ----------------------+----------------------------------------------------- Reporter: till | Owner: kparal Type: defect | Status: assigned Priority: critical | Milestone: Hot issues Component: tests | Resolution: Keywords: | ----------------------+----------------------------------------------------- Comment (by kparal):
OK, I worked on that issue too. See the time stamps:
{{{ Test ended 2011-08-05 01:05:57 UTC Fri Aug 5 02:49:04 2011: youtube-dl-2011.08.04-1.fc14 tagged into dist-f14-updates-pending by bodhi [still active] Fri Aug 5 22:21:50 2011: youtube-dl-2011.08.04-1.fc16 tagged into f16 -updates-pending by bodhi [still active] }}}
Subtract two hours from the last two lines, since it's printed in my localtime. Now it makes sense. There was 20 hours delay between push- to-F14 request and push-to-F16 request. Upgradepath was run in the mean time. The FAIL result is correct, but it should have been changed to PASS later. It was not, because of the Bodhi bug I reported.
#370: upgradepath: bad test result ----------------------+----------------------------------------------------- Reporter: till | Owner: kparal Type: defect | Status: assigned Priority: critical | Milestone: Hot issues Component: tests | Resolution: Keywords: | ----------------------+----------------------------------------------------- Comment (by till):
Replying to [comment:5 kparal]:
Subtract two hours from the last two lines, since it's printed in my
localtime. Now it makes sense. There was 20 hours delay between push- to-F14 request and push-to-F16 request. Upgradepath was run in the mean time. The FAIL result is correct, but it should have been changed to PASS later. It was not, because of the Bodhi bug I reported.
ok. Then it seems that there is another problem with Bodhi that caused the delay, because I submitted the updates at the same time with only one submission to Bodhi. And Bodhi created the updates with only 9 seconds delay: {{{ bodhi - 2011-08-04 21:42:33 This update has been submitted for testing by till. }}} {{{ bodhi - 2011-08-04 21:42:42 This update has been submitted for testing by till. }}}
#370: upgradepath: bad test result ----------------------+----------------------------------------------------- Reporter: till | Owner: kparal Type: defect | Status: assigned Priority: critical | Milestone: Hot issues Component: tests | Resolution: Keywords: | ----------------------+----------------------------------------------------- Comment (by kparal):
Replying to [comment:6 till]:
ok. Then it seems that there is another problem with Bodhi that caused
the delay, because I submitted the updates at the same time with only one submission to Bodhi.
Not really. You submitted them into *testing* simultaneously. That's correct. But you enabled karma auto-push for *stable*. Upgradepath runs only for stable requests. And for stable repo the fc14 update has requested the push earlier than fc16 update. Bodhi is correct and upgradepath (failing it) is correct. It just should've been re-calculated in 20 hours, and it was not.
The best solution would probably be if Bodhi could file an update for multiple releases as a "set" and then karma auto-push would be triggered for all releases at once, or not at all. That would be an improvement. But I don't see any new bug in Bodhi apart from the one I reported.
#370: upgradepath: bad test result ----------------------+----------------------------------------------------- Reporter: till | Owner: kparal Type: defect | Status: assigned Priority: critical | Milestone: Hot issues Component: tests | Resolution: Keywords: | ----------------------+----------------------------------------------------- Comment (by till):
Replying to [comment:7 kparal]:
Replying to [comment:6 till]:
ok. Then it seems that there is another problem with Bodhi that caused
the delay, because I submitted the updates at the same time with only one submission to Bodhi.
Not really. You submitted them into *testing* simultaneously. That's
correct. But you enabled karma auto-push for *stable*. Upgradepath runs only for stable requests. And for stable repo the fc14 update has requested the push earlier than fc16 update. Bodhi is correct and upgradepath (failing it) is correct. It just should've been re-calculated in 20 hours, and it was not.
Ok, thank you for the investigation.
The best solution would probably be if Bodhi could file an update for
multiple releases as a "set" and then karma auto-push would be triggered for all releases at once, or not at all. That would be an improvement. But I don't see any new bug in Bodhi apart from the one I reported.
My suggestion would be to check for upgradepath for packages which reach autokarma before they are automatically moved to stable and only move them to stable when the upgradepath test was passed. This way a Fedora 16 update with enough karma can be moved to stable automatically without the Fedora 15 update having enough karma.
#370: upgradepath: bad test result ----------------------+----------------------------------------------------- Reporter: till | Owner: kparal Type: defect | Status: closed Priority: critical | Milestone: Hot issues Component: tests | Resolution: fixed Keywords: | ----------------------+----------------------------------------------------- Changes (by kparal):
* status: assigned => closed * resolution: => fixed
Comment:
Luke Macken found the problem in Bodhi and manually fixed it:
https://fedorahosted.org/bodhi/ticket/627
Closing this ticket, no change needed in AutoQA.
autoqa-devel@lists.fedorahosted.org
-
fedora-badges