On Fri, Jun 05, 2020 at 04:10:20PM +0200, Tomas Orsava wrote: > Hi, > I think it would be useful to have a standard way of disabling the > running of tests during RPM build (in the %check section of a spec > file). > > I see a lot of packages already having %bcond's or other macro > definitions to archieve this, but each package has their own way, > there's no real standard. Thus you have to first look into the spec, > locate the appropriate %bcond or macro name and only then you can > disable the tests. > > I would like to propose two approaches: > > (a) Add a *SHOULD* rule to the guidelines that specifies what is the > preferred way to conditionalize the tests. > > (b) Or, if that's too strong, mention in the guidelines the common > methods that are being used (e.g. %bcond tests and %bcond check) so > that new packagers have something to use. What's the motivation for disabling tests globally?

I have some packages where tests fail on particular architectures at particular times, and what I do there is (a) file a BZ (b) surround the %check section with %ifarch/%ifnarch and a comment linking to the bug, and this seems to me a practical and lightweight approach that requires no special support in the toolchain. Also rpmbuild itself can happily disable tests, just add the --nocheck flag.

On Fri, Jun 05, 2020 at 04:10:20PM +0200, Tomas Orsava wrote: > Hi, > I think it would be useful to have a standard way of disabling the > running of tests during RPM build (in the %check section of a spec > file). > > I see a lot of packages already having %bcond's or other macro > definitions to archieve this, but each package has their own way, > there's no real standard. Thus you have to first look into the spec, > locate the appropriate %bcond or macro name and only then you can > disable the tests. > > I would like to propose two approaches: > > (a) Add a *SHOULD* rule to the guidelines that specifies what is the > preferred way to conditionalize the tests. > > (b) Or, if that's too strong, mention in the guidelines the common > methods that are being used (e.g. %bcond tests and %bcond check) so > that new packagers have something to use. What's the motivation for disabling tests globally?

I have some packages where tests fail on particular architectures at particular times, and what I do there is (a) file a BZ (b) surround the %check section with %ifarch/%ifnarch and a comment linking to the bug, and this seems to me a practical and lightweight approach that requires no special support in the toolchain. Also rpmbuild itself can happily disable tests, just add the --nocheck flag.

On Fri, Jun 05, 2020 at 04:38:03PM +0200, Miro Hrončok wrote: > On 05. 06. 20 16:26, Richard W.M. Jones wrote: >> On Fri, Jun 05, 2020 at 04:10:20PM +0200, Tomas Orsava wrote: >>> Hi, >>> I think it would be useful to have a standard way of disabling the >>> running of tests during RPM build (in the %check section of a spec >>> file). >>> >>> I see a lot of packages already having %bcond's or other macro >>> definitions to archieve this, but each package has their own way, >>> there's no real standard. Thus you have to first look into the spec, >>> locate the appropriate %bcond or macro name and only then you can >>> disable the tests. >>> >>> I would like to propose two approaches: >>> >>> (a) Add a *SHOULD* rule to the guidelines that specifies what is the >>> preferred way to conditionalize the tests. >>> >>> (b) Or, if that's too strong, mention in the guidelines the common >>> methods that are being used (e.g. %bcond tests and %bcond check) so >>> that new packagers have something to use. >> What's the motivation for disabling tests globally? > Bootstrapping mostly. For the RISC-V bootstrap we used rpmbuild directly (before Koji and its dependencies had been ported), and added --nocheck. However once Koji was working we built packages properly with checks enabled. How often do we bootstrap Fedora from scratch? Wholly new architectures are rare. Are there other events that require bootstrapping from scratch?

On 6/5/20 4:46 PM, Richard W.M. Jones wrote: > On Fri, Jun 05, 2020 at 04:38:03PM +0200, Miro Hrončok wrote: >> On 05. 06. 20 16:26, Richard W.M. Jones wrote: >>> On Fri, Jun 05, 2020 at 04:10:20PM +0200, Tomas Orsava wrote: >>>> Hi, >>>> I think it would be useful to have a standard way of disabling the >>>> running of tests during RPM build (in the %check section of a spec >>>> file). >>>> >>>> I see a lot of packages already having %bcond's or other macro >>>> definitions to archieve this, but each package has their own way, >>>> there's no real standard. Thus you have to first look into the spec, >>>> locate the appropriate %bcond or macro name and only then you can >>>> disable the tests. >>>> >>>> I would like to propose two approaches: >>>> >>>> (a) Add a *SHOULD* rule to the guidelines that specifies what is the >>>> preferred way to conditionalize the tests. >>>> >>>> (b) Or, if that's too strong, mention in the guidelines the common >>>> methods that are being used (e.g. %bcond tests and %bcond check) so >>>> that new packagers have something to use. >>> What's the motivation for disabling tests globally? >> Bootstrapping mostly. > For the RISC-V bootstrap we used rpmbuild directly (before Koji and > its dependencies had been ported), and added --nocheck.  However once > Koji was working we built packages properly with checks enabled. > > How often do we bootstrap Fedora from scratch?  Wholly new > architectures are rare.  Are there other events that require > bootstrapping from scratch? Not necessarily bootstrapping from scratch, this is useful for bootstrapping of anything in Fedora.

Fod example, Python now releases on a yearly schedule, and bootstrapping it is a huge undertaking involving thousands of components. And most importantly, the reason tests are disabled during bootstrapping is missing dependencies. Those have to be conditionalized by some %bcond or macro, and `--nocheck` doesn't help. Tomas _______________________________________________ packaging mailing list -- packaging(a)lists.fedoraproject.org To unsubscribe send an email to packaging-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproje...

Le mardi 09 juin 2020 à 12:08 +0200, Vít Ondruch a écrit : > Just FTR, we have bootstrapping guidelines > https://docs.fedoraproject.org/en-US/packaging-guidelines/#bootstrapping > Those suffer from 1. the horrible bcond logic inversion that trips pretty much everyone all the time.

2. the fact you can not ask koji or mock for a bootstrapped build, you have to change the spec manually

That won't be different for what was the original question here, i.e. conditionally disable tests. bconds are what we have for better or worse. And really, this seems about bootstrapping not disabling tests, which are not completely different, but nobody can objects bootstrapping, while disabling tests might be good just to improve build speed and nothing else. That should never happen in production environment IMO.

The proposal was to optionally disable test. When somebody asked why, the answer was bootstrapping. But we know how to handle bootstrapping. So shouldn't somebody spend time changing the test conditionals to bootstrapping conditionals, because that seems to be the use case?

Le mardi 09 juin 2020 à 14:56 +0200, Nicolas Mailhot a écrit : > Le mardi 09 juin 2020 à 14:35 +0200, Vít Ondruch a écrit : >> The proposal was to optionally disable test. When somebody asked >> why, >> the answer was bootstrapping. But we know how to handle >> bootstrapping. >> So shouldn't somebody spend time changing the test conditionals to >> bootstrapping conditionals, because that seems to be the use case? > One use case is bootstrapping. Another is just getting things to > build > till you have the time to investigate if a new test failure is an > actual problem or upstream being careless as usual. There are > probably > other use cases out there Another fun case: someone broke the dep of a component used in unit tests. Fixing the component requires rebuilding the dep. Except, the dep uses the component itself in its own unit tests… There are boundless possibilities for fun and profit there (well profit, not so sure actually)