F38 proposal: Perl: Replace versioned MODULE_COMPAT_ requires by
macro (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/Perl_replace_MODULE_COMPAT_by_macro
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
A ''perl(:MODULE_COMPAT_%(eval "<nowiki>`%{__perl}
-V:version`</nowiki>"; echo $version))'' run-time dependency will be
replaced with a new ''%perl_require_compat'' macro in all Perl spec
files.
The macro will expand differently based on architecture specificity of
the binary packages. That will significantly shrink an amount of Perl
packages required to be rebuilt with each Perl upgrade.
== Owner ==
* Name: [[User:jplesnik| Jitka Plesnikova]]
* Email: <jplesnik(a)redhat.com>
=== Completed items ===
=== Items in progress ===
* Add `%perl_require_compat` macro to ''perl-srpm-macros'' in F38
* Add `%perl_require_compat` macro to ''perl-srpm-macros'' in F37
* Add `%perl_require_compat` macro to ''perl-srpm-macros'' in F36
* Add `%perl_require_compat` macro to ''perl-srpm-macros'' in F35
* Add `%perl_require_compat` macro to ''perl-srpm-macros'' in RHEL 9
* Update [[Packaging:Perl | Fedora Packaging Guidelines for Perl]]
* Replace ''perl(:MODULE_COMPAT_XXX)'' by `%perl_require_compat`
run-time in all F38 spec files (3259)
== Detailed Description ==
The list of packages that need to be rebuilt with the new major
version of Perl is determined according to the dependency on
''perl(:MODULE_COMPAT_XXX)'' now.
In Fedora, all Perl modules run-require the versioned
''perl(:MODULE_COMPAT_XXX)'' provided by ''perl-libs'' now.
However, only multi-arch packages need to have a dependency on the
particular version of Perl it was built against, or on a newer version
of Perl that provides backward compatibility with it. For those
packages, we need to ensure that the packages will use the right
version of ''libperl.so'' for the Perl used during the rebuild.
The noarch packages don't need to be rebuilt against each new major
version of Perl, they only have to require non-versioned ''perl-libs''
which includes all directories used by all Perl modules.
The macro ''%perl_require_compat'' will evaluate the run-require based
on ''%{_target_cpu}''. The macro will be defined in the rpm
''perl-srpm-macros'' and the definition is:
`%perl_require_compat %[ "%{_target_cpu}" == "noarch" ? "perl-libs" :
"%{!?perl_version:perl-libs}%{?perl_version:perl(:MODULE_COMPAT_%{perl_version})}"
]`
The macro ''%perl_requires_compat'' will evaluate to the correct
value. There is a known, yet harmless, imperfection: The macro will
evaluate to ''perl(:MODULE_COMPAT_XXX)'' even in noarch subpackages of
a multi-arch main package. In this case, I recommend to use `Requires:
%perl_require_compat`. The purpose of the change is a matter of
simplifying the rebuild, where it depends if the source package
produces at least one multi-arch binary package. Not on whether it
makes a noarch subpackage next to it.
If you don't want to see this imperfection you could use directly
`Requires: perl-libs`.
The macro will work for all supported Fedoras and EPEL 9.
For EPEL 8 and older, it won't work, because the
[https://rpm-software-management.github.io/rpm/manual/macros.html
Expression Expansion] is not implemented there. In these versions,
''perl(:MODULE_COMPAT_%(eval "<nowiki>`%{__perl}
-V:version`</nowiki>"; echo $version))'' must be used.
== Benefit to Fedora ==
It will simplify the rebuild and reduce the number of packages which
have to be rebuild from 3259 to approximately 600. It should currently
be enough to rebuild only multi-arch packages and those that are part
of the Perl itself (dual-life packages). Here we need to ensure that
the packages will use the right ''libperl.so'' for the Perl used. The
new syntax will also be shorter and clearer for packagers.
== Scope ==
* Proposal owners:
** Submit Fedora Packaging Guidelines for Perl update to Fedora
Packaging Committee.
** Update and rebuild perl-srpm-macros source package.
** Add ''%perl_require_compat'' to ''perl-srpm-macros'' package in
older Fedoras.
** Replace Requires for ''perl(:MODULE_COMPAT_XXX)'' with
''%perl_require_compat'' in all spec files.
* Other developers: Get familiar with new Fedora Packaging Guidelines for Perl.
* Release engineering: No action needed.
[https://pagure.io/releng/issues #Releng issue number] <!-- REQUIRED
FOR SYSTEM WIDE CHANGES -->
* Policies and guidelines: N/A (not needed for this Change) <!--
REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- Do the packaging guidelines or other documents need to be updated
for this feature? If so, does it need to happen before or after the
implementation is done? If a FPC ticket exists, add a link here.
Please submit a pull request with the proposed changes before
submitting your Change proposal. -->
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives:
== Upgrade/compatibility impact ==
N/A
== How To Test ==
All multi-arch packages which use the macro should run-require
''perl(:MODULE_COMPAT_%{perl_version})'' and the ''noarch'' packages
should run-require ''perl-libs'' except the case listed in '''Detailed
Description'''.
== User Experience ==
There should not be any remarkable change in user experience.
== Dependencies ==
This change will affect 3259 source packages and all binary noarch
packages. The rebuild of affected packages will be done by mass
rebuild of Fedora 38. There is no dependency on other Fedora changes.
== Contingency Plan ==
* Contingency mechanism: The change will be reverted.
* Contingency deadline: Before Mass Rebuild.
* Blocks release? No.
== Documentation ==
== Release Notes ==
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
1 year, 5 months
Should the policy documents better reflect real package maintenance
practice?
by Gordon Messmer
In the wild, I often see Fedora described as a "semi-rolling" release.
As a policy matter, the distribution promises to be mostly stable, but I
find it increasingly hard to honestly present it as such.
As a couple of quick examples, I'd point out that in Fedora 35, Blender
updated from 2.93 (an LTS version) to version 3. In Fedora 36, Emacs
updated from version 27 to 28. I've read in the KDE Matrix channel that
KDE will be updated in Fedora 36 to 5.26, even though it has already
been updated from 5.24 -> 5.25 (my reading of the KDE update policy is
that Fedora used to update all releases with every KDE release, but
decided to stop). Firefox and Thunderbird get updates in most releases,
even when they contain API-breaking changes (those really should have
an explicit exception, IMHO.) I could offer more, but my point is
simply that examples of updates in prominent packages isn't hard to find.
That's not necessarily to object to those changes (though I did have to
do some minor fixes after the emacs update, and I grumbled quietly), and
I don't want to disrupt users getting new features if that's what
everyone actually wants. But, it does bother me that the documentation
doesn't seem to reflect reality. I think that the documentation should
offer users a realistic expectation of what they'll get from Fedora.
Does anyone else feel like the documentation should be updated, or am I
making too much of this?
1 year, 5 months
perl-Prima license change
by Petr Pisar
perl-Prima-1.67 in Rawhide will change a license from
BSD-2-Clause AND BSD-3-Clause AND BSD-4-Clause AND MIT-open-group AND HPND-sell-variant AND TCL AND ImageMagick AND LGPL-2.0-or-later AND AGPL-3.0-or-later
to
BSD-2-Clause AND BSD-3-Clause AND BSD-4-Clause AND MIT-open-group AND HPND AND HPND-sell-variant AND TCL AND ImageMagick AND LGPL-2.0-or-later AND AGPL-3.0-or-later
-- Petr
1 year, 5 months
Question about git signed tags
by Bob Hepple
Here's a question from one of my upstream devels. Not sure I understand
exactly what he's asking but I thought I'd post here in the hope that
someone can enlighten him (and me!).
"... Arch supports signed git tags. I'm hoping Fedora does too.
I'm thinking of dropping this cumbersome process (i.e: signing and pushing
the .sig and .tar.gz) for the next release. Simply sign the tag and create
a release out of it. Can you please do a bit of research on your side to
see if that's possible?
Also, for your consideration, git now supports ssh-based signatures
<https://blog.dbrgn.ch/2021/11/16/git-ssh-signatures/>. I won't stop using
PGP because I think distros don't support this very well but just so you
know."
If we _do_ support "signed git tags" how do we code for it in the spec
file? Presently I have this:
Source0: %{url}/releases/download/v%{version}/%{name}-%{version}.tar.gz
Source1: %{url}/releases/download/v%{version}/%{name}-%{version}.tar.gz.sig
Source2: 6A6B35DBE9442683.gpg
...
%prep
%gpgverify -k 2 -s 1 -d 0
%autosetup -p1
Thanks
Bob
1 year, 5 months
Schedule for Tuesday's FESCo Meeting (2022-11-29)
by Stephen Gallagher
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Following is the list of topics that will be discussed in the
FESCo meeting Tuesday at 17:00UTC in #fedora-meeting on
irc.libera.chat.
To convert UTC to your local time, take a look at
http://fedoraproject.org/wiki/UTCHowto
or run:
date -d '2022-11-29 17:00 UTC'
Links to all issues to be discussed can be found at:
https://pagure.io/fesco/report/meeting_agenda
= Discussed and Voted in the Ticket =
#2899 Change: Reproducible builds: Clamp build mtimes to $SOURCE_DATE_EPOCH
https://pagure.io/fesco/issue/2899
APPROVED (+7, 0, -0)
#2893 Change: Ruby 3.2
https://pagure.io/fesco/issue/2893
APPROVED (+5, 0, -0)
= Followups =
#2817 Change proposal: Add -fno-omit-frame-pointer to default compilation flags
https://pagure.io/fesco/issue/2817
#2870 Change proposal: Replace DNF with DNF5
https://pagure.io/fesco/issue/2870
#2895 Election interview questions: FESCo (F37)
https://pagure.io/fesco/issue/2895
= New business =
= Open Floor =
For more complete details, please visit each individual
issue. The report of the agenda items can be found at
https://pagure.io/fesco/report/meeting_agenda
If you would like to add something to this agenda, you can
reply to this e-mail, file a new issue at
https://pagure.io/fesco, e-mail me directly, or bring it
up at the end of the meeting, during the open floor topic. Note
that added topics may be deferred until the following meeting.
-----BEGIN PGP SIGNATURE-----
iQGzBAEBCAAdFiEEhQqd0dvyrMxvxJSRRduFpWgobREFAmOFGjMACgkQRduFpWgo
bRGnPAv/Rqe68KFCUH/8C7ELyZ3OUs9+O2hlG8X0z+npxDcVDg4e96h+zi6zobp7
P+9ColvjFDZPuuVJbLih23PlPkur9y0s4F3X9jdGHP8YdGftBphkAEPrU7BnvJRL
tfKEVyvUIWo99z8h6MVMKPU2mLdZCxzUssWnOD+beWwcppgaTTzeXG6WXjcbOSWt
3n3u7k5PygqYS0O9RgaSG7saaSXrYZm21COy/A0g3ZpPxLH/dw8AfdA/sJchZBAR
ejyjIYfa5nBWrbvaRkmaG4AyyoYF1Sn++LQPKxaKLrYaM9SaAjRau8Yv7VfU/MIj
Fm3UbVNY08yRe/lWWd2zHMmcNwsclVTe/02iniLnArgO1O+GLKYEcA/PTzmwJ/zY
AinzAR9G0g5RFVtlIpxggRFWHCT1ztIdH0hPweNqWrINQIKZsyMGkj4QkqcqBppH
l5JA6hZ0ZqUi/FS5hvamVXM4OnyprFepVgjuLOqHKJ7Zk2QA+mB5veCi4WpbFjLx
MK1b9HJv
=+Mby
-----END PGP SIGNATURE-----
1 year, 5 months
F37 proposal: Add -fno-omit-frame-pointer to default compilation
flags (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/fno-omit-frame-pointer
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Fedora will add -fno-omit-frame-pointer to the default C/C++
compilation flags, which will improve the effectiveness of profiling
and debugging tools.
== Owner ==
* Name: [[User:daandemeyer| Daan De Meyer]], [[User:Dcavalca| Davide
Cavalca]], [[ Andrii Nakryiko]]
* Email: daandemeyer(a)fb.com, dcavalca(a)fb.com, andriin(a)fb.com
== Detailed Description ==
Credits to Mirek Klimos, whose internal note on stacktrace unwinding
formed the basis for this change proposal (myreggg(a)gmail.com).
Any performance or efficiency work relies on accurate profiling data.
Sampling profilers probe the target program's call stack at regular
intervals and store the stack traces. If we collect enough of them, we
can closely approximate the real cost of a library or function with
minimal runtime overhead.
Stack trace capture what’s running on a thread. It should start with
clone - if the thread was created via clone syscall - or with _start -
if it’s the main thread of the process. The last function in the stack
trace is code that CPU is currently executing. If a stack starts with
[unknown] or any other symbol, it means it's not complete.
=== Unwinding ===
How does the profiler get the list of function names? There are two parts of it:
# Unwinding the stack - getting a list of virtual addresses pointing
to the executable code
# Symbolization - translating virtual addresses into human-readable
information, like function name, inlined functions at the address, or
file name and line number.
Unwinding is what we're interested in for the purpose of this
proposal. The important things are:
* Data on stack is split into frames, each frame belonging to one function.
* Right before each function call, the return address is put on the
stack. This is the instruction address in the caller to which we will
eventually return — and that's what we care about.
* One register, called the "frame pointer" or "base pointer" register
(RBP), is traditionally used to point to the beginning of the current
frame. Every function should back up RBP onto the stack and set it
properly at the very beginning.
The “frame pointer” part is achieved by adding push %rbp, mov
%rsp,%rbp to the beginning of every function and by adding pop %rbp
before returning. Using this knowledge, stack unwinding boils down to
traversing a linked list:
https://i.imgur.com/P6pFdPD.png
=== Where’s the catch? ===
The frame pointer register is not necessary to run a compiled binary.
It makes it easy to unwind the stack, and some debugging tools rely on
frame pointers, but the compiler knows how much data it put on the
stack, so it can generate code that doesn't need the RBP. Not using
the frame pointer register can make a program more efficient:
* We don’t need to back up the value of the register onto the stack,
which saves 3 instructions per function.
* We can treat the RBP as a general-purpose register and use it for
something else.
Whether the compiler sets frame pointer or not is controlled by the
-fomit-frame-pointer flag and the default is "omit", meaning we can’t
use this method of stack unwinding by default.
To make it possible to rely on the frame pointer being available,
we'll add -fno-omit-frame-pointer to the default C/C++ compilation
flags. This will instruct the compiler to make sure the frame pointer
is always available. This will in turn allow profiling tools to
provide accurate performance data which can drive performance
improvements in core libraries and executables.
== Feedback ==
=== Potential performance impact ===
* Meta builds all its libraries and executables with
-fno-omit-frame-pointer by default. Internal benchmarks did not show
significant impact on performance when omitting the frame pointer for
two of our most performance intensive applications.
* Firefox recently landed a change to preserve the frame pointer in
all jitted code
(https://bugzilla.mozilla.org/show_bug.cgi?id=1426134). No significant
decrease in performance was observed.
* Kernel 4.8 frame pointer benchmarks by Suse showed 5%-10%
regressions in some benchmarks
(https://lore.kernel.org/all/20170602104048.jkkzssljsompjdwy@suse.de/T/#u)
Should individual libraries or executables notice a significant
performance degradation caused by including the frame pointer
everywhere, these packages can opt-out on an individual basis as
described in https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags.
=== Alternatives to frame pointers ===
There are a few alternative ways to unwind stacks instead of using the
frame pointer:
* [https://dwarfstd.org DWARF] data - The compiler can emit extra
information that allows us to find the beginning of the frame without
the frame pointer, which means we can walk the stack exactly as
before. The problem is that we need to unwind the stack in kernel
space which isn't implemented in the kernel. Given that the kernel
implemented it's own format (ORC) instead of using DWARF, it's
unlikely that we'll see a DWARF unwinder in the kernel any time soon.
The perf tool allows you to use the DWARF data with
--call-graph=dwarf, but this means that it copies the full stack on
every event and unwinds in user space. This has very high overhead.
* [https://www.kernel.org/doc/html/v5.3/x86/orc-unwinder.html ORC]
(undwarf) - problems with unwinding in kernel led to creation of
another format with the same purpose as DWARF, just much simpler. This
can only be used to unwind kernel stack traces; it doesn't help us
with userspace stacks. More information on ORC can be found
[https://lwn.net/Articles/728339 here].
* [https://lwn.net/Articles/680985 LBR] - New Intel CPUs have a
feature that gives you source and target addresses for the last 16 (or
32, in newer CPUs) branches with no overhead. It can be configured to
record only function calls and to be used as a stack, which means it
can be used to get the stack trace. Sadly, you only get the last X
calls, and not the full stack trace, so the data can be very
incomplete. On top of that, many Fedora users might still be using
CPUs without LBR support which means we wouldn't be able to assume
working profilers on a Fedora system by default.
To summarize, if we want complete stacks with reasonably low overhead
(which we do, there's no other way to get accurate profiling data from
running services), frame pointers are currently the best option.
== Benefit to Fedora ==
Implementing this change will provide profiling tools with easy access
to stacktraces of installed libraries and executables which will lead
to more accurate profiling data in general. This in turn can be used
to implement optimizations to core libraries and executables which
will improve the overall performance of Fedora itself and the wider
Linux ecosystem.
Various debugging tools can also make use of the frame pointer to
access the current stacktrace, although tools like gdb can already do
this to some degree via embedded dwarf debugging info.
== Scope ==
* Proposal owners: Put up a PR to change the rpm macros to build
packages by default with -fno-omit-frame-pointer by default.
* Other developers: Review and merge the PR implementing the Change.
* Release engineering: [https://pagure.io/releng/issues #Releng issue
number]. A mass rebuild is required.
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact ==
This should not impact upgrades in any way.
== How To Test ==
# Build the package with the updated rpm macros
# Profile the binary with `perf record -g <binary>`
# Inspect the perf data with `perf report -g 'graph,0.5,caller'`
# When expanding hot functions in the perf report, perf should show
the full call graph of the hot function (at least for all functions
that are part of the binary compiled with -fno-omit-frame-pointer)
== User Experience ==
Fedora users will be more likely to have a streamlined experience when
trying to debug/profile system executables/libraries. Tools such as
perf will work out of the box instead of requiring to users to provide
extra options (e.g. --call-graph=dwarf/LBR) or requiring users to
recompile all relevant packages with -fno-omit-frame-pointer.
== Dependencies ==
The rpm macros for Fedora need to be adjusted to include
-fno-omit-frame-pointer in the default C/C++ compilation flags.
== Contingency Plan ==
* Contingency mechanism: The new version can be released without every
package being rebuilt with fno-omit-frame-pointer. Profiling will only
work perfectly once all packages have been rebuilt but there will be
no regression in behavior if not all packages have been rebuilt by the
time of the release. If the Change is found to introduce unacceptable
regressions, the PR implementing it can be reverted and affected
packages can be rebuilt.
* Contingency deadline: Final freeze
* Blocks release? No
== Documentation ==
* Original proposal for in-kernel DWARF unwinder (rejected):
https://lkml.org/lkml/2017/5/5/571
== Release Notes ==
Packages are now compiled with frame pointers included by default.
This will enable a variety of profiling and debugging tools to show
more information out of the box.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
1 year, 5 months
Updating asio in rawhide (and possibly F37) to 1.24.0
by Julian Sikorski
Dear maintainers,
I have updated Fedora asio package from the current 1.16.1 to 1.24.0. I
have rebuilt the seven current dependencies and with the exception of
OpenSceneGraph, all build fine against the updated asio package. While
OpenSceneGraph failed to build, the failure does not seem to have to do
anything with asio.
I have created a side tag so that the rebuilds can be coordinated:
f38-build-side-56604. I do not have provenpacker privileges which means
you have to request the rebuilds yourself. You can start once the asio
build [1] finishes. Thank you for your cooperation in advance.
Best regards,
Julian
[1] https://koji.fedoraproject.org/koji/taskinfo?taskID=90781633
1 year, 5 months
