On 20. 12. 21 18:41, Ben Cotton wrote: > https://fedoraproject.org/wiki/Changes/SetBuildFlagsBuildCheck > > == Summary == > Call %set_build_flags macro automatically at the beginning of the > %build and %check phases of RPM builds in Fedora Linux.  This will > ensure that the compiler flag environment variables are set for every > RPM build. Should we also do this in %install for completeness? As in "just in case".

* Ben Cotton: > This change will be implemented by updating the %__spec_build_pre and > %__speck_check_pre macros in redhat-rpm-config to include > %set_build_flags. This will set these environment variables > automatically before the %build and %check sections. See the proposed > [https://src.fedoraproject.org/fork/tstellar/rpms/redhat-rpm-config/c/a397... > implementation] for more details. Would you please add a clear opt-out mechanism, and document the behavior and the mechanism in buildflags.md? I can't tell right now how much is going to break because of this, but I think it's worth a try.

What do you think the best place is to document this? I was thinking in the Packaging Guidelines: https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags

* Ben Cotton: > This change will be implemented by updating the %__spec_build_pre and > %__speck_check_pre macros in redhat-rpm-config to include > %set_build_flags. This will set these environment variables > automatically before the %build and %check sections. See the proposed > [https://src.fedoraproject.org/fork/tstellar/rpms/redhat-rpm-config/c/a397... > implementation] for more details. Would you please add a clear opt-out mechanism, and document the behavior and the mechanism in buildflags.md?

I can't tell right now how much is going to break because of this, but I think it's worth a try. Thanks, Florian

Since you are digging into this and AFAIK you are involved with toolchains, this reminds me this dreaded issue: https://bugzilla.redhat.com/show_bug.cgi?id=1284684 In short, various languages, such as Ruby embeds the build time options and reuse them for build of extensions. And I wonder, would it be possible to generalize this e.g. into some tool, which would set the environment variables and would be usable outside of rpmbuild?

Hi Tom, Since you are digging into this and AFAIK you are involved with toolchains, this reminds me this dreaded issue: https://bugzilla.redhat.com/show_bug.cgi?id=1284684 In short, various languages, such as Ruby embeds the build time options and reuse them for build of extensions. And I wonder, would it be possible to generalize this e.g. into some tool, which would set the environment variables and would be usable outside of rpmbuild?

Also, Fedora sets all these flags for purpose, but we won't let our users to reuse them. So on top of my previous question, I wonder if we set these flags on the right place and if there would not be better to set them more broadly then just for RPMs. Vít Dne 20. 12. 21 v 18:41 Ben Cotton napsal(a): > https://fedoraproject.org/wiki/Changes/SetBuildFlagsBuildCheck > > == Summary == > Call %set_build_flags macro automatically at the beginning of the > %build and %check phases of RPM builds in Fedora Linux.  This will > ensure that the compiler flag environment variables are set for every > RPM build. > > > == Owner == > * Name: [[User:tstellar| Tom Stellard]] > * Email: <tstellar(a)redhat.com&gt; > > > == Detailed Description == > The %set_build_flags macro exports common environment variables used > for building packages: > * CFLAGS > * CXXFLAGS > * FFLAGS > * FCFLAGS > * LDFLAGS > * LT_SYS_LIBRARY_PATH > * CC > * CXX > > > These environment variables are set to the compiler flags defined in > the system RPM configuration.  This macro is currently implicitly > called when packages use some of the build system helper macros, like > %configure, %cmake, and %meson.  However, not all packages use these > macros and so some packages do not use the correct compiler flags as > required by the Fedora packaging guidelines[1]. > > This change will be implemented by updating the %__spec_build_pre and > %__speck_check_pre macros in redhat-rpm-config to include > %set_build_flags.  This will set these environment variables > automatically before the %build and %check sections.  See the proposed > [https://src.fedoraproject.org/fork/tstellar/rpms/redhat-rpm-config/c/a397... > implementation] for more details. > > The purpose for making this change in both the %build and %check > sections is because sometimes test code gets built in the %check > sections for unit tests and this will ensure that the application code > and its tests are built with the same set of flags. > > This change should have no impact on packages that already use > %set_build_flags either directly or indirectly through another macro. > It also won't impact any package that currently sets these environment > variables or modifies any of the %{build*_flags} macros in their > %build or %check sections. > > [1] https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags > > > == Benefit to Fedora == > This change will ensure that more packages are built using the correct > compiler flags, and bring them in compliance with the Fedora packaging > guidelines.  It will also help improve the security of the > distribution as many of the compiler flags help defend against common > security attacks. > > > == Scope == > * Proposal owners: > ** Make the necessary changes to redhat-rpm-config. > ** Help debug any issues uncovered by this change during the mass rebuild. > * Other developers: > ** Report bugs to the proposal owner. > > * Release engineering: [https://pagure.io/releng/issue/10482 #10482] > * Policies and guidelines: N/A (not needed for this Change) > * Trademark approval: N/A (not needed for this Change) > * Alignment with Objectives: > > > == How To Test == > This change will be tested by rebuilding packages as part of the mass rebuild. > > > == User Experience == > This change will make some packages less susceptible to security exploits. > > > == Contingency Plan == > > * Contingency mechanism: The proposal owner will revert the change in > redhat-rpm-config > * Contingency deadline: Beta Freeze > * Blocks release? No > == Documentation == > None needed. > > _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

Dne 21. 12. 21 v 21:56 Tom Stellard napsal(a): > On 12/21/21 01:42, Vít Ondruch wrote: >> Hi Tom, >> >> Since you are digging into this and AFAIK you are involved with toolchains, this reminds me this dreaded issue: >> >> https://bugzilla.redhat.com/show_bug.cgi?id=1284684 >> >> In short, various languages, such as Ruby embeds the build time options and reuse them for build of extensions. And I wonder, would it be possible to generalize this e.g. into some tool, which would set the environment variables and would be usable outside of rpmbuild? >> > > > I think the only way to really generalize this is for the upstream projects to > make it easier for distros to manually specify the flags for extensions rather > than automatically taking the flags from the compiler invocation used to build > the interpreter. I think this is limited POV. The issue is that the languages are actually providing services to their extensions. IOW the languages are doing a lot of probing for their build and they provides these results for their extensions, therefore the extensions don't need to do so much probing. And that is reasonable IMO.

The problem is that everything is designed to be build on single system, which is not the case for binary distribution. Moreover, the binary distribution is using some flags for its build, but it does not offer any generic way to reuse these flags for builds done outside of the packaging environment. IOW if I install gcc on my system, it won't be using all the hardening and other flags Fedora itself is using for its build and that is something which should be improved IMO.

Vít > > > - Tom > >> Also, Fedora sets all these flags for purpose, but we won't let our users to reuse them. So on top of my previous question, I wonder if we set these flags on the right place and if there would not be better to set them more broadly then just for RPMs. >> >> >> Vít >> >> >> Dne 20. 12. 21 v 18:41 Ben Cotton napsal(a): >>> https://fedoraproject.org/wiki/Changes/SetBuildFlagsBuildCheck >>> >>> == Summary == >>> Call %set_build_flags macro automatically at the beginning of the >>> %build and %check phases of RPM builds in Fedora Linux.  This will >>> ensure that the compiler flag environment variables are set for every >>> RPM build. >>> >>> >>> == Owner == >>> * Name: [[User:tstellar| Tom Stellard]] >>> * Email: <tstellar(a)redhat.com&gt; >>> >>> >>> == Detailed Description == >>> The %set_build_flags macro exports common environment variables used >>> for building packages: >>> * CFLAGS >>> * CXXFLAGS >>> * FFLAGS >>> * FCFLAGS >>> * LDFLAGS >>> * LT_SYS_LIBRARY_PATH >>> * CC >>> * CXX >>> >>> >>> These environment variables are set to the compiler flags defined in >>> the system RPM configuration.  This macro is currently implicitly >>> called when packages use some of the build system helper macros, like >>> %configure, %cmake, and %meson.  However, not all packages use these >>> macros and so some packages do not use the correct compiler flags as >>> required by the Fedora packaging guidelines[1]. >>> >>> This change will be implemented by updating the %__spec_build_pre and >>> %__speck_check_pre macros in redhat-rpm-config to include >>> %set_build_flags.  This will set these environment variables >>> automatically before the %build and %check sections.  See the proposed >>> [https://src.fedoraproject.org/fork/tstellar/rpms/redhat-rpm-config/c/a397... >>> implementation] for more details. >>> >>> The purpose for making this change in both the %build and %check >>> sections is because sometimes test code gets built in the %check >>> sections for unit tests and this will ensure that the application code >>> and its tests are built with the same set of flags. >>> >>> This change should have no impact on packages that already use >>> %set_build_flags either directly or indirectly through another macro. >>> It also won't impact any package that currently sets these environment >>> variables or modifies any of the %{build*_flags} macros in their >>> %build or %check sections. >>> >>> [1] https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags >>> >>> >>> == Benefit to Fedora == >>> This change will ensure that more packages are built using the correct >>> compiler flags, and bring them in compliance with the Fedora packaging >>> guidelines.  It will also help improve the security of the >>> distribution as many of the compiler flags help defend against common >>> security attacks. >>> >>> >>> == Scope == >>> * Proposal owners: >>> ** Make the necessary changes to redhat-rpm-config. >>> ** Help debug any issues uncovered by this change during the mass rebuild. >>> * Other developers: >>> ** Report bugs to the proposal owner. >>> >>> * Release engineering: [https://pagure.io/releng/issue/10482 #10482] >>> * Policies and guidelines: N/A (not needed for this Change) >>> * Trademark approval: N/A (not needed for this Change) >>> * Alignment with Objectives: >>> >>> >>> == How To Test == >>> This change will be tested by rebuilding packages as part of the mass rebuild. >>> >>> >>> == User Experience == >>> This change will make some packages less susceptible to security exploits. >>> >>> >>> == Contingency Plan == >>> >>> * Contingency mechanism: The proposal owner will revert the change in >>> redhat-rpm-config >>> * Contingency deadline: Beta Freeze >>> * Blocks release? No >>> == Documentation == >>> None needed. >>> >>> >> >> _______________________________________________ >> devel mailing list -- devel(a)lists.fedoraproject.org >> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org >> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines >> List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org >> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure >> > _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

On 12/22/21 01:56, Vít Ondruch wrote: > > Dne 21. 12. 21 v 21:56 Tom Stellard napsal(a): >> On 12/21/21 01:42, Vít Ondruch wrote: >>> Hi Tom, >>> >>> Since you are digging into this and AFAIK you are involved with >>> toolchains, this reminds me this dreaded issue: >>> >>> https://bugzilla.redhat.com/show_bug.cgi?id=1284684 >>> >>> In short, various languages, such as Ruby embeds the build time >>> options and reuse them for build of extensions. And I wonder, would >>> it be possible to generalize this e.g. into some tool, which would >>> set the environment variables and would be usable outside of rpmbuild? >>> >> >> >> I think the only way to really generalize this is for the upstream >> projects to >> make it easier for distros to manually specify the flags for >> extensions rather >> than automatically taking the flags from the compiler invocation >> used to build >> the interpreter. > > > I think this is limited POV. The issue is that the languages are > actually providing services to their extensions. IOW the languages > are doing a lot of probing for their build and they provides these > results for their extensions, therefore the extensions don't need to > do so much probing. And that is reasonable IMO. > I wasn't suggesting modifying the extension flags directly.  What I meant was that we should be able to specify a set of flags for extensions to use when we build python, for example.  And then extensions would pick up those flags up the same way they do now via a config file, header, etc.

> The problem is that everything is designed to be build on single > system, which is not the case for binary distribution. > > Moreover, the binary distribution is using some flags for its build, > but it does not offer any generic way to reuse these flags for builds > done outside of the packaging environment. IOW if I install gcc on my > system, it won't be using all the hardening and other flags Fedora > itself is using for its build and that is something which should be > improved IMO. > I don't think it would be too difficult to install a spec file (not an RPM spec file, a gcc spec file) that contains the default Fedora flags.  Then users could build with gcc -spec=fedora-flags to get the same set of flags.  clang has a similar feature and could do the same thing.

I don't think we should change the compiler defaults to match what Fedora does, though. This causes too many headaches for developers who are trying to support multiple distros.

-Tom > > Vít > > >> >> >> - Tom >> >>> Also, Fedora sets all these flags for purpose, but we won't let our >>> users to reuse them. So on top of my previous question, I wonder if >>> we set these flags on the right place and if there would not be >>> better to set them more broadly then just for RPMs. >>> >>> >>> Vít >>> >>> >>> Dne 20. 12. 21 v 18:41 Ben Cotton napsal(a): >>>> https://fedoraproject.org/wiki/Changes/SetBuildFlagsBuildCheck >>>> >>>> == Summary == >>>> Call %set_build_flags macro automatically at the beginning of the >>>> %build and %check phases of RPM builds in Fedora Linux. This will >>>> ensure that the compiler flag environment variables are set for every >>>> RPM build. >>>> >>>> >>>> == Owner == >>>> * Name: [[User:tstellar| Tom Stellard]] >>>> * Email: <tstellar(a)redhat.com&gt; >>>> >>>> >>>> == Detailed Description == >>>> The %set_build_flags macro exports common environment variables used >>>> for building packages: >>>> * CFLAGS >>>> * CXXFLAGS >>>> * FFLAGS >>>> * FCFLAGS >>>> * LDFLAGS >>>> * LT_SYS_LIBRARY_PATH >>>> * CC >>>> * CXX >>>> >>>> >>>> These environment variables are set to the compiler flags defined in >>>> the system RPM configuration.  This macro is currently implicitly >>>> called when packages use some of the build system helper macros, like >>>> %configure, %cmake, and %meson.  However, not all packages use these >>>> macros and so some packages do not use the correct compiler flags as >>>> required by the Fedora packaging guidelines[1]. >>>> >>>> This change will be implemented by updating the %__spec_build_pre and >>>> %__speck_check_pre macros in redhat-rpm-config to include >>>> %set_build_flags.  This will set these environment variables >>>> automatically before the %build and %check sections.  See the >>>> proposed >>>> [https://src.fedoraproject.org/fork/tstellar/rpms/redhat-rpm-config/c/a397... >>>> >>>> implementation] for more details. >>>> >>>> The purpose for making this change in both the %build and %check >>>> sections is because sometimes test code gets built in the %check >>>> sections for unit tests and this will ensure that the application >>>> code >>>> and its tests are built with the same set of flags. >>>> >>>> This change should have no impact on packages that already use >>>> %set_build_flags either directly or indirectly through another macro. >>>> It also won't impact any package that currently sets these >>>> environment >>>> variables or modifies any of the %{build*_flags} macros in their >>>> %build or %check sections. >>>> >>>> [1] >>>> https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags >>>> >>>> >>>> == Benefit to Fedora == >>>> This change will ensure that more packages are built using the >>>> correct >>>> compiler flags, and bring them in compliance with the Fedora >>>> packaging >>>> guidelines.  It will also help improve the security of the >>>> distribution as many of the compiler flags help defend against common >>>> security attacks. >>>> >>>> >>>> == Scope == >>>> * Proposal owners: >>>> ** Make the necessary changes to redhat-rpm-config. >>>> ** Help debug any issues uncovered by this change during the mass >>>> rebuild. >>>> * Other developers: >>>> ** Report bugs to the proposal owner. >>>> >>>> * Release engineering: [https://pagure.io/releng/issue/10482 #10482] >>>> * Policies and guidelines: N/A (not needed for this Change) >>>> * Trademark approval: N/A (not needed for this Change) >>>> * Alignment with Objectives: >>>> >>>> >>>> == How To Test == >>>> This change will be tested by rebuilding packages as part of the >>>> mass rebuild. >>>> >>>> >>>> == User Experience == >>>> This change will make some packages less susceptible to security >>>> exploits. >>>> >>>> >>>> == Contingency Plan == >>>> >>>> * Contingency mechanism: The proposal owner will revert the change in >>>> redhat-rpm-config >>>> * Contingency deadline: Beta Freeze >>>> * Blocks release? No >>>> == Documentation == >>>> None needed. >>>> >>>> >>> >>> _______________________________________________ >>> devel mailing list -- devel(a)lists.fedoraproject.org >>> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org >>> Fedora Code of Conduct: >>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ >>> List Guidelines: >>> https://fedoraproject.org/wiki/Mailing_list_guidelines >>> List Archives: >>> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org >>> Do not reply to spam on the list, report it: >>> https://pagure.io/fedora-infrastructure >>> >> > > _______________________________________________ > devel mailing list -- devel(a)lists.fedoraproject.org > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure >

On 20/12/2021 18:41, Ben Cotton wrote: > Call %set_build_flags macro automatically at the beginning of the > %build and %check phases of RPM builds in Fedora Linux. This will > ensure that the compiler flag environment variables are set for every > RPM build. +1 for the %build section, but I doubt it is really needed for %check.

Also, owners of this proposal should use their proven packager abilities to remove any existing %set_build_flags from all Fedora packages (Rawhide branch).

On 28/12/2021 13:17, Neal Gompa wrote: > It is. There are plenty of packages that compile for tests. I have > one, for example (libeconf). I think all compilation should be done in %build.

> Why? If the package is being maintained as a single spec across > multiple branches, you'd have to ask them to condition it, which is > hard to do. Removing it would break it for packagers maintaining a > single spec across branches, all for avoiding a redundant call. It'd > be better to advise packagers to only use this for older Fedora and > EPEL and let them fix it. I agree. I think %set_build_flags can be removed after Fedora 35 reaches EOL.