----- "Josef Skladanka" <jskladan(a)redhat.com> wrote:
Hi gang,
according to #228 <
https://fedorahosted.org/autoqa/ticket/228>,
and discussions with kparal, I've rewritten the koji watcher,
so it can check also the -pending tags in koji.
For the -pendig tags, we're using quite a different querying model
based on parsing the tagHistory (e.g.
# koji list-tag-history --tag='dist-f14-updates-pending'
)
The benefit of this solution is, that we can catch the situations
like
1) package Foo is built at date XYZ. It gains tag
dist-f14-updates-pending
That should read dist-f14-updates-candidate I guess.
2) koji-watcher founds out "ha, new package, let's test
it"
3) tests are OK
4) package Foo gains dist-f14-updates-testing-pending tag
5) we'd like to run tests like depcheck on it, but because the 'built
at'
date, which the actual watcher checks is not altered, we miss the
change
But parsing the tag history, we only check for new 'events' in the
tag
history (i.e. 'package foo got the -pending tag few minutes ago'),
independently
on the build time -> win :)
The drawback is, that at the moment, querying for the tag history
takes time
because koji sends the whole tag history (a lot of data). I'm
discussing a minor
change which would allow us to specify "give me the history which is
newer than
date XYZ" (as is currently used for fetching new builds) with
jkeating.
Because of this, the non-pending repos are still handled the 'old'
way.
....
We also found us in need of 'batch' scheduling - e.g. we don't want to
run
depcheck for every package built, but we'd like just to inform autoqa
"hey, there is new stuff in dist-f14-updates-pending tag, run stuff".
So there is new watch-koji-builds-batch watcher.
The thing is, that it is not really a watcher as such, only the
hook.py
file. The reason to do this is, that querying koji is time-consuming
operation,
and there is no need to do it twice, to get the same data. So there
are two
different 'schedule jobs' methods in the watch-koji-build hook, and
one
schedules 'per package' jobs, and the second does it as a 'batch'.
(note that the batch one is not yet tested - I just wanted to post you
the
patch, so you can see the new 'concept'. The per-package part is
tested,
and considered to be working).
Comments are more than welcomed - looking forward to hearing from
you!
(master)$ git apply koji_watcher.patch
fatal: git apply: bad git-diff - expected /dev/null on line 4
Gosh, do I have git somehow broken today? (Or is it my hands?)
Edit: Ah, CRLF characters confused git. Good to know. Dos2unix fixed that.
It's too much new code to check for correctness, but I tried to run that,
and I have a few tracebacks for you :)
[root@aqd autoqa]# /usr/share/autoqa/post-koji-build/watch-koji-builds.py --dryrun
File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 376
exit status = 0
^
SyntaxError: invalid syntax
[root@aqd autoqa]# /usr/share/autoqa/post-koji-build/watch-koji-builds.py --dryrun
...
Traceback (most recent call last):
File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 459, in
<module>
exit_status = watcher.run()
File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 434, in
run
exit_status_2.append(self.schedule_jobs_batch(new_builds))
NameError: global name 'exit_status_2' is not defined
[root@aqd autoqa]# /usr/share/autoqa/post-koji-build/watch-koji-builds.py --dryrun
...
Traceback (most recent call last):
File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 459, in
<module>
exit_status = watcher.run()
File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 434, in
run
exit_status.append(self.schedule_jobs_batch(new_builds))
File "/usr/share/autoqa/post-koji-build/watch-koji-builds.py", line 404, in
schedule_jobs_batch
print " ".join(harnescall)
TypeError: sequence item 4: expected string, list found
After correcting these problems, here is the sample output:
http://fpaste.org/mOtd/
This approach comes from my and Josef's discussion and I think it's the best
thing we can currently do to accommodate depcheck-style tests into our
framework.