On Mon, 2010-11-15 at 12:16 -0500, Kamil Paral wrote:
This is an interesting problem. We want to post comments from AutoQA
into Bodhi. But some of our tests are designed in such a way that they
test a particular package several (dozen) times. For example depcheck
will test all packages in -updates-pending every time this tag is updated
(new package is added/removed).
If some package fails depcheck, we probably don't want to add 10 comments
to its Bodhi page saying "depcheck failed" every single day.
So we devised a solution where we won't post duplicate comments. If
the first comment says "depcheck fails", we won't post any new comment
until it says something else.
But that has a catch - we also want to add a URL to the full results
of the test. If we don't post duplicate comments, the maintainer will
see just the original URL. But the reason of the failure might have
changed. How do we tell the maintainer what's wrong when we don't post
updated URL?
One silly idea I've come up with is that we will post duplicate comments,
but only once in a few days. For example we will post "depcheck failed
(URL)" again only after at least two days from the previous "depcheck
failed (URL)".
That sounds much better than posting a comment each time that update
fails (which would be *many* times a day until the issue is resolved).
Ideas?
I would need to see what the test output for depcheck looks like ... I
have no idea right now. However, one thing I'm using now might be
helpful. I wrote a wrapper around the 'repoclosure' command that
auto-files bugs into bugzilla for repoclosure issues. The wrapper
script hashes the repoclosure output for each failure and attempts to
make it unique enough to prevent filing duplicate bugs. It works well
enough that I've been happy with the result. Using this hashing, we
could detect when the failure reason has changed, and add a new bodhi
comment accordingly.
You can see bugs that were filed by the script at
http://bit.ly/cTpTLt .
I don't have the script published anywhere, I'm still working on how
best to communicate this sample script (autoqa-devel or a blog post
etc...).
Thanks,
James