Hi everybody and sorry for the rant, but I must vent somewhere.

executed at some point and the results are available in Bodhi [1]. Therefore, I'd like to execute similar test on my machine prior pushing anything anywhere. So lets give rpminspect a shot: ~~~ $ rpminspect rpminspect: Please specify a configuration file using '-c' or supply ./rpminspect.yaml ~~~ Hm, bummer. So how is this configured in Bodhi? Lets look at Jenkins [2]. Hm, nice, but how is it executed? No clue. So lets try the main task [3]. Ha, console [4], there must be the "rpminspect" command, right? But no, there is non. There is just 11 pages of nonsense. So looking once again, ok, kubernetes, there is used some container. Lets check what the rpminspect-pipeline stands for, opening its repository [5]. FMF, that sounds familiar, lets open the rpminspect.fmf [6]. Hm, /usr/local/bin/rpminspect_runner.sh, what is that? How can I get it? Ah, there is image repository and the file [7], which as again more then 5 pages of gibberish. There finally some `rpminspect` call on line 155 [8], but then I could continue with localization of the config file. This is really awful and I suffer every time I have to do something like this. I am really frustrated. I really don't understand, why the command can't be visible as far as at the console [4]. Comparing e.g. with GH actions, I have opened cpython repository for the first time in my life and on 5 clicks, I know the test suite is executed via `make buildbottest TESTOPTS="-j4 -uall,-cpu"`. Do I want to try to test rpminspect at this moment? No, No, No and no again, because this is not CI, this is just torture. This is so intimidating. So, would it be so big problem to have the several rpminspect calls which are certainly done somewhere at some point visible on the Jenkins console [4]? /rant Vít P.S. Once again sorry and please no offense. [1] https://bodhi.fedoraproject.org/updates/FEDORA-2021-62cd39359e [2] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... [3] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... [4] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... [5] https://github.com/fedora-ci/rpminspect-pipeline [6] https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runn... [7] https://github.com/fedora-ci/rpminspect-pipeline/blob/master/rpminspect.fmf [8] https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runn... _______________________________________________ CI mailing list -- ci(a)lists.fedoraproject.org To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

Dne 07. 01. 22 v 21:46 Miroslav Vadkerti napsal(a): > Hi, > > On Fri, Jan 7, 2022 at 6:43 PM Vít Ondruch <vondruch(a)redhat.com&gt; wrote: > > Hi everybody and sorry for the rant, but I must vent somewhere. > > As usual, sorry for the bad experience, I guess the problems boils > down to missing proper documentation for the test exposed to the > end-user :( Eh, don't start me on documentation ;) The rpminspect-pipeline [5] contains non. But I still think that I should not need any documentation. The logs should be comprehensive on itself and should contain every information. Actually, I have to correct my statement. The problem with current logs is that they targets the CI developer instead of targeting CI users. I understand that the CI is complex and CI developers needs to know some information, but I think this approach should be reconsidered. The CI should be done for CI users. > > To run it exactly as CI does you would need to run: > >    $ git clone https://github.com/fedora-ci/rpminspect-pipeline >    $ cd rpminspect-pipeline >    $ tmt run -e PREVIOUS_TAG=f36-updates -e TASK_ID='79796333' -e > RPMINSPECT_PROFILE_NAME=rawhide -e > REPOSITORY_URL=https://src.fedoraproject.org/rpms/ruby -e > CONFIG_BRANCH=rawhide -e DEFAULT_RELEASE_STRING=fc36 -e > GIT_COMMIT=5d3688479ab299cbd5657f9a503463a04d963d65 -e DEBUG=off I am looking once again at the Jenkins console [4] and the only thing from your list I can see there is the first line. And that is the first and last useful information in the log. Why the remaining lines are not there? But also, I'd like to see in the log how `tmt` executes the rpminspect. After all, I am not working with Koji build locally and what not. Last but not least, seeing something as a `rpminspect-image`, this seems to be over-engineered. If I submitted change to that project, I'd expect that simple merge would propagate my changes to CI. But that is probably not true, because the new image would need to be built. Or if CI is using such image, then it should also build it. BTW neither the rpminspect-image is mentioned in the log. > > We should expose this information in all `tmt` based tests, sorry it > is still not there :( it is this RH issue > https://issues.redhat.com/browse/TFT-641 > <https://issues.redhat.com/browse/TFT-641> > > Now why is running rpminspect in CI more complex than just > run`rpminspect-fedora OLD_BUILD NEW_BUILD`, I guess that is the > question to @Michal Srb <mailto:msrb@redhat.com> or @Cantrell, David > <mailto:dcantrell@redhat.com> ...I do not exactly know > > Best regards, > /M > > PS: command to run for rpminspect for Fedora is `rpminspect-fedora` > not plain `rpminspect`. Why to remember this, when I can look at how CI does it, right? BTW while this might put rpminspect into the spotlight, a bit earlier I tried to understand how rpmlint [0] used to be executed and the experience was not really better. Vít [0] https://bodhi.fedoraproject.org/updates/FEDORA-2020-0d62cd7900 > > So I am preparing updated Ruby package. I know that there is > rpminspect executed at some point and the results are available > in Bodhi [1]. Therefore, I'd like to execute similar test on my > machine prior pushing anything anywhere. So lets give rpminspect > a shot: > > ~~~ > > $ rpminspect > rpminspect: Please specify a configuration file using '-c' or > supply ./rpminspect.yaml > > ~~~ > > Hm, bummer. So how is this configured in Bodhi? Lets look at > Jenkins [2]. Hm, nice, but how is it executed? No clue. > > So lets try the main task [3]. Ha, console [4], there must be the > "rpminspect" command, right? But no, there is non. There is just > 11 pages of nonsense. > > So looking once again, ok, kubernetes, there is used some > container. Lets check what the rpminspect-pipeline stands for, > opening its repository [5]. FMF, that sounds familiar, lets open > the rpminspect.fmf [6]. Hm, /usr/local/bin/rpminspect_runner.sh, > what is that? How can I get it? Ah, there is image repository and > the file [7], which as again more then 5 pages of gibberish. > There finally some `rpminspect` call on line 155 [8], but then I > could continue with localization of the config file. > > This is really awful and I suffer every time I have to do > something like this. I am really frustrated. I really don't > understand, why the command can't be visible as far as at the > console [4]. > > Comparing e.g. with GH actions, I have opened cpython repository > for the first time in my life and on 5 clicks, I know the test > suite is executed via `make buildbottest TESTOPTS="-j4 -uall,-cpu"`. > > Do I want to try to test rpminspect at this moment? No, No, No > and no again, because this is not CI, this is just torture. This > is so intimidating. > > So, would it be so big problem to have the several rpminspect > calls which are certainly done somewhere at some point visible on > the Jenkins console [4]? > > /rant > > > Vít > > > P.S. Once again sorry and please no offense. > > > [1] https://bodhi.fedoraproject.org/updates/FEDORA-2021-62cd39359e > > [2] > https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... > > [3] > https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... > > [4] > https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... > > [5] https://github.com/fedora-ci/rpminspect-pipeline > > [6] > https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runn... > > [7] > https://github.com/fedora-ci/rpminspect-pipeline/blob/master/rpminspect.fmf > > [8] > https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runn... > > _______________________________________________ > CI mailing list -- ci(a)lists.fedoraproject.org > To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure > > > > -- > Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE > IRC mvadkert #tft #tmt #osci :: Mobile +420 773 944 252 > Remote Czech Republic :: Red Hat Czech s.r.o > > > > _______________________________________________ > CI mailing list --ci(a)lists.fedoraproject.org > To unsubscribe send an email toci-leave(a)lists.fedoraproject.org > Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives:https://lists.fedoraproject.org/archives/list/ci@lists.fedorapro... > Do not reply to spam on the list, report it:https://pagure.io/fedora-infrastructure

Hi everybody and sorry for the rant, but I must vent somewhere. So I am preparing updated Ruby package. I know that there is rpminspect executed at some point and the results are available in Bodhi [1]. Therefore, I'd like to execute similar test on my machine prior pushing anything anywhere. So lets give rpminspect a shot: ~~~ $ rpminspect rpminspect: Please specify a configuration file using '-c' or supply ./rpminspect.yaml ~~~ Hm, bummer. So how is this configured in Bodhi? Lets look at Jenkins [2]. Hm, nice, but how is it executed? No clue. So lets try the main task [3]. Ha, console [4], there must be the "rpminspect" command, right? But no, there is non. There is just 11 pages of nonsense. So looking once again, ok, kubernetes, there is used some container. Lets check what the rpminspect-pipeline stands for, opening its repository [5]. FMF, that sounds familiar, lets open the rpminspect.fmf [6]. Hm, /usr/local/bin/rpminspect_runner.sh, what is that? How can I get it? Ah, there is image repository and the file [7], which as again more then 5 pages of gibberish. There finally some `rpminspect` call on line 155 [8], but then I could continue with localization of the config file. This is really awful and I suffer every time I have to do something like this. I am really frustrated. I really don't understand, why the command can't be visible as far as at the console [4]. Comparing e.g. with GH actions, I have opened cpython repository for the first time in my life and on 5 clicks, I know the test suite is executed via `make buildbottest TESTOPTS="-j4 -uall,-cpu"`.

again, because this is not CI, this is just torture. This is so intimidating. So, would it be so big problem to have the several rpminspect calls which are certainly done somewhere at some point visible on the Jenkins console [4]?

/rant Vít P.S. Once again sorry and please no offense. [1] https://bodhi.fedoraproject.org/updates/FEDORA-2021-62cd39359e [2] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... [3] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... [4] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... [5] https://github.com/fedora-ci/rpminspect-pipeline [6] https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runn... [7] https://github.com/fedora-ci/rpminspect-pipeline/blob/master/rpminspect.fmf [8] https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runn... _______________________________________________ CI mailing list -- ci(a)lists.fedoraproject.org To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

Dne 10. 01. 22 v 10:30 Michal Srb napsal(a): >Hi there, > >On Fri, Jan 7, 2022 at 6:43 PM Vít Ondruch <vondruch(a)redhat.com&gt; wrote: > > Hi everybody and sorry for the rant, but I must vent somewhere. > > So I am preparing updated Ruby package. I know that there is > rpminspect executed at some point and the results are available in > Bodhi [1]. Therefore, I'd like to execute similar test on my > machine prior pushing anything anywhere. So lets give rpminspect a > shot: > > ~~~ > > $ rpminspect > rpminspect: Please specify a configuration file using '-c' or > supply ./rpminspect.yaml > > ~~~ > > Hm, bummer. So how is this configured in Bodhi? Lets look at > Jenkins [2]. Hm, nice, but how is it executed? No clue. > > So lets try the main task [3]. Ha, console [4], there must be the > "rpminspect" command, right? But no, there is non. There is just > 11 pages of nonsense. > > So looking once again, ok, kubernetes, there is used some > container. Lets check what the rpminspect-pipeline stands for, > opening its repository [5]. FMF, that sounds familiar, lets open > the rpminspect.fmf [6]. Hm, /usr/local/bin/rpminspect_runner.sh, > what is that? How can I get it? Ah, there is image repository and > the file [7], which as again more then 5 pages of gibberish. There > finally some `rpminspect` call on line 155 [8], but then I could > continue with localization of the config file. > > This is really awful and I suffer every time I have to do > something like this. I am really frustrated. I really don't > understand, why the command can't be visible as far as at the > console [4]. > > Comparing e.g. with GH actions, I have opened cpython repository > for the first time in my life and on 5 clicks, I know the test > suite is executed via `make buildbottest TESTOPTS="-j4 -uall,-cpu"`. > > >Yes, this is how CI usually works. You have some scripts that you >want to execute in some environment and maybe, at the end, you want >to retrieve/archive a few test-specific files/directories that you >know were created during the run. Easy-peasy. > >We had trouble with the "environment" part in the past, when >OpenStack was not very reliable. But instead of fixing just that one >problem (credit card would do the trick), we went further and >introduced more opinionated layers. In hindsight, I think we went a >little overboard. > > Do I want to try to test rpminspect at this moment? No, No, No and > no again, because this is not CI, this is just torture. This is so > intimidating. > > So, would it be so big problem to have the several rpminspect > calls which are certainly done somewhere at some point visible on > the Jenkins console [4]? > > >The problem is that Jenkins is actually not running anything -- it >delegates everything. So maintainers will never find any useful >information in Jenkins logs. Jenkins doesn't really know what >rpminspect command will be executed several layers down. And it >currently doesn't have access to the raw output of the test run, the >only thing that is returned back through all the layers is the >captured stdout/stderr from tmt-executed tests (so we cannot just >dump the command to a file in a lower layer and then print it to the >Jenkins console later -- that file wouldn't be available in >Jenkins). The situation will be much better when the whole workdir >is available to the upper layers (Miro shared a JIRA link in the >other email in this thread). > >However, you're not completely out of luck here. rpminspect prints >the command that was used to produce the test results as part of its >"diagnostics" inspection [1]. >And there you can see: > >Details: >/usr/bin/rpminspect -c /usr/share/rpminspect/fedora.yaml >--format=json --release=fc36 --profile=rawhide ruby-3.0.3-154.fc36 >ruby-3.0.3-155.fc36 Oh, wow. Please, can this be extracted somewhere visible? Nice place would be right bellow the "Test Result : tests" in the main landing page. Actually, I would not mind to have the whole "diagnostics" section visible on the first sight. Also, having no real experiences with rpminspect, I might just guess that diagnostics might be the first section reported by rpminspect, so I'd suggest against alphabetical sorting of the results.

Vít > >That is the command that actually runs in CI. > >Thanks, >Michal > >[1]: https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... > > /rant > > > Vít > > > P.S. Once again sorry and please no offense. > > > [1] https://bodhi.fedoraproject.org/updates/FEDORA-2021-62cd39359e > > [2] > https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... > > [3] > https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... > > [4] > https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-... > > [5] https://github.com/fedora-ci/rpminspect-pipeline > > [6] > https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runn... > > [7] > https://github.com/fedora-ci/rpminspect-pipeline/blob/master/rpminspect.fmf > > [8] > https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runn... > > _______________________________________________ > CI mailing list -- ci(a)lists.fedoraproject.org > To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: > https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure > > >_______________________________________________ >CI mailing list --ci(a)lists.fedoraproject.org >To unsubscribe send an email toci-leave(a)lists.fedoraproject.org >Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines >List Archives:https://lists.fedoraproject.org/archives/list/ci@lists.fedorapro... >Do not reply to spam on the list, report it:https://pagure.io/fedora-infrastructure

_______________________________________________ CI mailing list -- ci(a)lists.fedoraproject.org To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure