10:06 a.m.
I'm trying to wrap up the work to get rpminspect running on all builds
coming out of koji and I've run into some questions about the messages
I'm emitting and what should be in resultsdb.
Do error messages (koji-build.test.error) belong in resultsdb?
If an error condition is reached, should both a completion
(koji-build.test.complete) and an error message be emitted? I'm unclear
if exiting with an error condition constitutes completion or not.
I'd appreciate input that folks have on this.
Tim
10:52 a.m.
Hi Tim,
On Wed, Feb 5, 2020 at 5:09 PM Tim Flink <tflink(a)redhat.com> wrote:
I'm trying to wrap up the work to get rpminspect running on all
builds
coming out of koji and I've run into some questions about the messages
I'm emitting and what should be in resultsdb.
Do error messages (koji-build.test.error) belong in resultsdb?
I think they do.
If an error condition is reached, should both a completion
(koji-build.test.complete) and an error message be emitted? I'm unclear
if exiting with an error condition constitutes completion or not.
AFAIK, it's either "error" or "complete". Not both.
"error" when CI
couldn't finish testing from some reason and "complete" when CI finished
testing successfully. Test failures still count as "complete".
Thanks,
Michal
I'd appreciate input that folks have on this.
Tim
_______________________________________________
CI mailing list -- ci(a)lists.fedoraproject.org
To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org
12:19 p.m.
On Wed, 5 Feb 2020 17:52:56 +0100
Michal Srb <msrb(a)redhat.com> wrote:
Hi Tim,
On Wed, Feb 5, 2020 at 5:09 PM Tim Flink <tflink(a)redhat.com> wrote:
> I'm trying to wrap up the work to get rpminspect running on all
> builds coming out of koji and I've run into some questions about
> the messages I'm emitting and what should be in resultsdb.
>
> Do error messages (koji-build.test.error) belong in resultsdb?
>
I think they do.
Can you elaborate a bit more on why you think that's the case. I'm more
of the mind that they don't belong in resultsdb because "error" as
defined in the messaging standard isn't a test result - it's an
execution state. I can't see an error state ever being used in a gating
decision and thus don't see why errors should be in resultsdb.
That being said, I'm not so strongly attached to not putting them in
resultsdb.
FWIW, it appears as if the all-packages pipeline's error messages are
not stored in resultsdb:
https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/ci_r...
Lines 32-36 are for production.
> If an error condition is reached, should both a completion
> (koji-build.test.complete) and an error message be emitted? I'm
> unclear if exiting with an error condition constitutes completion
> or not.
AFAIK, it's either "error" or "complete". Not both.
"error" when CI
couldn't finish testing from some reason and "complete" when CI
finished testing successfully. Test failures still count as
"complete".
That logic makes sense but there is also some sense in saying that a
run is complete if it errors out.
Is that what other systems/pipelines are doing? The all-packages
pipeline seems to send out both when an error condition is reached. I'm
mostly not sure how other systems are reacting to the messages sent out
by CI systems.
Tim
Thanks,
Michal
>
> I'd appreciate input that folks have on this.
>
> Tim
> _______________________________________________
> CI mailing list -- ci(a)lists.fedoraproject.org
> To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:
> https://fedoraproject.org/wiki/Mailing_list_guidelines List
> Archives:
> https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org
1:01 p.m.
On Wed, 2020-02-05 at 11:19 -0700, Tim Flink wrote:
Can you elaborate a bit more on why you think that's the case.
I'm
more
of the mind that they don't belong in resultsdb because "error" as
defined in the messaging standard isn't a test result - it's an
execution state. I can't see an error state ever being used in a
gating
decision and thus don't see why errors should be in resultsdb.
I agree with you. However, I do think execution states would be good to
store somewhere, so that UIs can show users the state of the tests as
they are running (queued, running, error). It makes sense to me not to
store them in the same place as results.
1:47 p.m.
On Wed, 2020-02-05 at 14:01 -0500, Randy Barlow wrote:
On Wed, 2020-02-05 at 11:19 -0700, Tim Flink wrote:
> Can you elaborate a bit more on why you think that's the case. I'm
> more
> of the mind that they don't belong in resultsdb because "error" as
> defined in the messaging standard isn't a test result - it's an
> execution state. I can't see an error state ever being used in a
> gating
> decision and thus don't see why errors should be in resultsdb.
I agree with you. However, I do think execution states would be good to
store somewhere, so that UIs can show users the state of the tests as
they are running (queued, running, error). It makes sense to me not to
store them in the same place as results.
We've kicked around the idea of an "execdb" before, I think...
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net
2:03 p.m.
On Wed, 2020-02-05 at 11:47 -0800, Adam Williamson wrote:
We've kicked around the idea of an "execdb" before, I
think...
Yeah, or it could just be a new table/API on resultsdb, to avoid
complexities of starting and deploying a whole new project.
The data would have a different schema than test results, which is why
I advocate for at least giving it its own API and database table.
3:06 p.m.
On Wed, 05 Feb 2020 15:03:19 -0500
Randy Barlow <bowlofeggs(a)fedoraproject.org> wrote:
On Wed, 2020-02-05 at 11:47 -0800, Adam Williamson wrote:
> We've kicked around the idea of an "execdb" before, I think...
Yeah, or it could just be a new table/API on resultsdb, to avoid
complexities of starting and deploying a whole new project.
I'd push back on this pretty hard. Resultsdb is as useful as it is
because it's so simple and "dumb". Adding features on top of what's
already there starts to defeat that utility.
I'm personally of the mind that two small, simple services are easier to
manage than one larger, more complex service.
Taskotron's execdb is a good example in my mind. Most of the
dev/maintenance time that we ever put into that service was the
frontend and how the data was presented to human users. That work would
have to be done no matter where the backend functionality landed and
keeping that functionality separate made it easier to maintain both
codebases.
That being said, I have no real dog in this fight as someone else would
end up doing the implementation.
The data would have a different schema than test results, which is
why
I advocate for at least giving it its own API and database table.
It sounds like we're of the same mind on this. I'm an advocate of
keeping results and execution status separate because the primary
audience/consumer for them are not looking for the same thing.
Anyhow, there may well be plans for something like this already.
Hopefully someone will chime in if that's the case.
Tim
2:45 p.m.
On Wed, 05 Feb 2020 14:01:04 -0500
Randy Barlow <bowlofeggs(a)fedoraproject.org> wrote:
On Wed, 2020-02-05 at 11:19 -0700, Tim Flink wrote:
> Can you elaborate a bit more on why you think that's the case. I'm
> more
> of the mind that they don't belong in resultsdb because "error" as
> defined in the messaging standard isn't a test result - it's an
> execution state. I can't see an error state ever being used in a
> gating
> decision and thus don't see why errors should be in resultsdb.
I agree with you. However, I do think execution states would be good
to store somewhere, so that UIs can show users the state of the tests
as they are running (queued, running, error). It makes sense to me
not to store them in the same place as results.
Back in the Taskotron days, we had a dashboard to show the execution
status of jobs separate from test results called execdb.
I'm under the impression that there's a dashboard project in RH that
accomplishes a similar purpose. It will be open sourced if it hasn't
been already but I don't know what the plans are around using it.
Tim
1:56 a.m.
On Wed, Feb 5, 2020 at 7:20 PM Tim Flink <tflink(a)redhat.com> wrote:
On Wed, 5 Feb 2020 17:52:56 +0100
Michal Srb <msrb(a)redhat.com> wrote:
> Hi Tim,
>
>
> On Wed, Feb 5, 2020 at 5:09 PM Tim Flink <tflink(a)redhat.com> wrote:
>
> > I'm trying to wrap up the work to get rpminspect running on all
> > builds coming out of koji and I've run into some questions about
> > the messages I'm emitting and what should be in resultsdb.
> >
> > Do error messages (koji-build.test.error) belong in resultsdb?
> >
>
> I think they do.
Can you elaborate a bit more on why you think that's the case. I'm more
of the mind that they don't belong in resultsdb because "error" as
defined in the messaging standard isn't a test result - it's an
execution state. I can't see an error state ever being used in a gating
decision and thus don't see why errors should be in resultsdb.
That being said, I'm not so strongly attached to not putting them in
resultsdb.
FWIW, it appears as if the all-packages pipeline's error messages are
not stored in resultsdb:
https://infrastructure.fedoraproject.org/cgit/ansible.git/tree/roles/ci_r...
Lines 32-36 are for production.
Michal answer is correct for the current situation, the error topic is
stored in resultsdb and it is also used by Greenwave.
the all-packages pipeline is also sends messages using the new format
(lines 37-40), but still sends old format as well as there are
components that are gating on old testcase name.
Here is an example of error message sent by the pipeline
https://apps.fedoraproject.org/datagrepper/id?id=2020-5f7780c8-fbd8-46c8-...
And you can see it shows in bodhi under "Automated Tests" with
testcase name "fedora-ci.koji-build.tier0.functional"
https://bodhi.fedoraproject.org/updates/FEDORA-2020-803c9175b9
> > If an error condition is reached, should both a completion
> > (koji-build.test.complete) and an error message be emitted? I'm
> > unclear if exiting with an error condition constitutes completion
> > or not.
>
> AFAIK, it's either "error" or "complete". Not both.
"error" when CI
> couldn't finish testing from some reason and "complete" when CI
> finished testing successfully. Test failures still count as
> "complete".
That logic makes sense but there is also some sense in saying that a
run is complete if it errors out.
Is that what other systems/pipelines are doing? The all-packages
pipeline seems to send out both when an error condition is reached. I'm
mostly not sure how other systems are reacting to the messages sent out
by CI systems.
Tim
> Thanks,
> Michal
>
>
> >
> > I'd appreciate input that folks have on this.
> >
> > Tim
> > _______________________________________________
> > CI mailing list -- ci(a)lists.fedoraproject.org
> > To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines:
> > https://fedoraproject.org/wiki/Mailing_list_guidelines List
> > Archives:
> > https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org
_______________________________________________
CI mailing list -- ci(a)lists.fedoraproject.org
To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org
1542
days inactive
1543
days old
8 comments
5 participants
participants (5)
-
Adam Williamson -
Bruno Goncalves -
Michal Srb -
Randy Barlow -
Tim Flink