On Thu, Jul 19, 2018 at 9:08 PM Igor Gnatenko < ignatenkobrain(a)fedoraproject.org&gt; wrote: > I'm trying to have shared tests between all 400+ rust packages. > I have few problems though: > * There is no package NEVRA in test environment so the shared test > doesn't know what to test - it is necessary to know which test to run. We trigger tests for dist-git. I think about dist-git as a SRPM. SRPM can produce many RPMs. CI pipeline: install all RPMS from Koji build and run tests from /tests for all installed RPMs. The same approach is for manual testing. Conclusion: test-environment-system has installed RPMS.

On Fri, Jul 20, 2018 at 1:54 PM Igor Gnatenko < ignatenkobrain(a)fedoraproject.org&gt; wrote: > On Fri, Jul 20, 2018 at 1:44 PM Andrei Stepanov <astepano(a)redhat.com&gt; > wrote: > >> On Thu, Jul 19, 2018 at 9:08 PM Igor Gnatenko < >> ignatenkobrain(a)fedoraproject.org&gt; wrote: >> >>> I'm trying to have shared tests between all 400+ rust packages. >>> >>> I have few problems though: >>> >>> * There is no package NEVRA in test environment so the shared test >>> doesn't know what to test - it is necessary to know which test to run. >>> >> >> We trigger tests for dist-git. >> I think about dist-git as a SRPM. >> SRPM can produce many RPMs. >> CI pipeline: install all RPMS from Koji build and run tests from /tests >> for all installed RPMs. >> The same approach is for manual testing. >> Conclusion: test-environment-system has installed RPMS. >> > > Just want to mention that this won't work if your subpackages conflict > (e.g. curl and curl-minimal. > Right, and I would ask contra@ team or other guys for clarification, how CI pipeline does care about above moment?

> > Tests from /tests dir check test-environment-system as a whole. >> > >> https://fedoraproject.org/wiki/CI/Standard_Test_Interface doesn't >> define NEVRA. >> Reason: STI tests are for all test-environment-system (all installed >> packages). >> /tests directory from dist-git knows that packages goes from dist-git/pkg >> /tests/tests.yml is written for each SRPM package. >> /tests/tests.yml knows that it will run against: SRPM + distro version. >> /tests/tests.yml for Fedora 27 and for Fedora 28 can are different files. >> If you have common tests, you have different /tests/tests.yml where you >> can specify exactly that this tests are for this SRPM and for Branch 28 >> (for example). >> > > This is wwhat I have done so far, however test runner has all this > information, why not to simply pass it to the tests rather than asking > every maintainer to do that manually. > > >> * No dist-git sources of a package - while trying to workaround previous >>> problem I have found that dist-git doesn't get into the environment so I >>> can't do something like dnf builddep >>> >>> Please help me ;) >>> >> >> >> We should decide: if tests can exist independently without dist-git. >> For now, we presume they can. >> This is current state. >> If yes. There is nothing to do. >> > > Why would they do? > > If you think that: "parent dir for /tests must be contents from dist-git branch" , then: please file a issue at https://pagure.io/standard-test-roles/issues That I can announce discussion about this changes to STI to our team. Please remember, this change should be consistent for all dist-git repos. > If no. We should tight together dist-git branch + /tests as one entity. >> And copy /tests to test-environment system together with dist-git. >> >> If we decide to change this, than we need to make correspondent changes >> to STI. >> https://fedoraproject.org/wiki/CI/Standard_Test_Interface >> >> >> >>> >>> P.S> I'm using standard-test-basic role. >>> >>> Thanks in advance! >>> -- >>> >>> -Igor Gnatenko >>> >> _______________________________________________ >>> CI mailing list -- ci(a)lists.fedoraproject.org >>> To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org >>> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html >>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >>> List Archives: https://lists.fedoraproject.org/archives/list/ci@lists. >>> fedoraproject.org/message/ESOHHMZ37AZIVVOPAWQNTRBXDEYKWAF6/ >>> >> _______________________________________________ >> CI mailing list -- ci(a)lists.fedoraproject.org >> To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org >> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: https://lists.fedoraproject.org/archives/list/ci@lists. >> fedoraproject.org/message/45YUVGGKW3BSFRAFPRGAZ2PKPTFHXLTP/ >> > -- > > -Igor Gnatenko > _______________________________________________ > CI mailing list -- ci(a)lists.fedoraproject.org > To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: https://lists.fedoraproject.org/archives/list/ci@lists. > fedoraproject.org/message/QCO7WV2N5LHUBPN62PZMEGBJL2SKUI33/ > _______________________________________________ CI mailing list -- ci(a)lists.fedoraproject.org To unsubscribe send an email to ci-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists. fedoraproject.org/message/EWUIYSJKUZ6DPX3YGJ2ZEXUZFOEFC3PU/