On Fri, 2017-02-24 at 12:03 -0500, Randy Barlow wrote:
Hello! I went trawling through Bodhi's issue tracker today to try
to
find some ideas for Outreachy this summer. I came up with a few:
* Integrate Bodhi's backend with ResultsDB so that updates can be
rejected for failing tests.
This one and the next are why I CC'd Adam Williamson, as there are a
few things about it that are unknown to me. I believe that ResultsDB
doesn't know which tests should or shouldn't block a build from going
out to users, just which tests passed/failed.
This is correct, and jskladan is very strongly of the opinion that it
shouldn't.
AFAIK, there isn't yet
a database that says which tests need to pass to go, but Fedora does
need something like that.
This is also broadly true, AIUI. Ralph probably has a better overview
of where he wants to go with this sort of thing.
We have the same issue with compose gating. It is not an easy issue to
solve. What you wind up building is basically a true test case
management system, and they always tend to be awful. There is also a
substantial problem of who owns (and is responsible for) keeping the
information up to date and in sync with the actual test systems.
AIUI, right now Bodhi can *already* reject updates for failing
automated tests, but the update submitter has to manually declare which
tests the update will block on at the time of creation.
If we went this route, we could either have
an intern participate in creating such a database, or we could try to
create it before they start and then have them integrate Bodhi with
it and ResultsDB so updates can be rejected automatically.
* Rework how ResultsDB results are displayed on Updates.
Bodhi currently has trouble displaying all the ResultDB results on
Updates. For large updates, Bodhi resorts to only showing failures as
it would crush the web browser otherwise. It also doesn't display
them in a particularly useful/attractive manner.
This one would probably involve a lot of JS/CSS/HTML work, which to
be honest isn't my strong suit. That might not make me the best
mentor for it.
https://github.com/fedora-infra/bodhi/issues/983
All I have to say on this one is that the problem is only going to get
bigger, because what I'm working on right now is having openQA test
updates and report those results to ResultsDB. So Bodhi's going to have
more results to show, very soon. Dunno what other input you're looking
for from me here.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net