FESCo wants to know what you use i686 packages for
by David Cantrell
Hi,
Our most recent FESCo meeting involved discussing the proposal to drop i686
builds of jdk8,11,17 from Fedora 37 onward. The topic quickly changed to the
larger question of "what do people use i686 packages for?"
Rather than guess, we wanted to ask the community what you use i686 packages
for in Fedora. There are no wrong answers here. We are seeking information.
Why? Since the removal of the i686 kernel in Fedora, we want to reduce the
number of i686 packages provided in the repo. As time marches on, the ability
to build a lot of things for i686 becomes unrealistic or even impossible.
Remember it goes beyond providing builds...providing support, bug fixes, and
security fixes for those packages too. Maybe some things using i686 packages
now can move to x86_64 packages. We do not know yet, but a goal is to figure
out what packages, if anything, can drop their i686 builds.
NOTE: Nothing is changing now. We are in an information gathering phase.
~~~~~~~~~~~~~~~~~~~~~~~~
If you use i686 packages for something now, please respond to this thread.
Thanks,
--
David Cantrell <dcantrell(a)redhat.com>
Red Hat, Inc. | Boston, MA | EST5EDT
2 weeks, 5 days
RISC-V -- are we ready for more, and what do we need to do it?
by Matthew Miller
Hi all! I just got back from Open Source Summit, several of the talks I
found interesting were on RISC-V -- a high-level one about the
organizational structure, and Drew Fustini's more technical talk.
In that, he noted that there's a Fedora build *, but it isn't an official
Fedora arch. As I understand it, the major infrastructure blocker is simply
that there isn't server-class hardware (let alone hardware that will build
fast enough that it isn't a frustrating bottleneck).
So, one question is: if we used, say, ARM or x86_64 Amazon cloud instances
as builders, could we build fast enough under QEMU emulation to work? We
have a nice early advantage, but if we don't keep moving, we'll lose that.
But beyond that: What other things might be limits? Are there key bits of
the distro which don't build yet? Is there a big enough risc-v team to
respond to arch-specific build failures? And, do we have enough people to do
QA around release time?
* see http://fedora.riscv.rocks/koji/
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
3 weeks, 5 days
"rescue" boot entry files are not updated on OS upgrades
by Chris Murphy
Summary----------
Most all Fedora variants (except Cloud) have a GRUB menu entry
containing the word "rescue". This kernel+initramfs pair are never
updated for the life of a Fedora installation. And they quickly become
stale as a Fedora installation ages. This kernel's modules are
eventually deleted, and if selected at boot time, the typical user
experience is a dracut shell.
Basic background-------------
(skip this section if you know how it works)
During a new installation, a single kernel version is installed. e.g.
vmlinuz-5.17.0-0.rc4.96.fc36.x86_64 which is then duplicated as e.g.
vmlinuz-0-rescue-3a86878de5d649a983916543ece7bb7e.
Each of those (identical) kernels has an initramfs file:
initramfs-5.17.0-0.rc4.96.fc36.x86_64.img
initramfs-0-rescue-3a86878de5d649a983916543ece7bb7e.img
The sole difference is the first one is a smaller host-only initramfs,
the second one is a larger no host-only initramfs created with `dracut
-N`. The bigger one just contains a bunch of extra kernel modules and
dracut scripts, ostensibly to make it more likely to boot a system
with some change in hardware that the host-only initramfs doesn't
contain. The size of this rescue initramfs is around 100 MiB, with the
common day to day "host only" initramfs being around 33 MiB. [1]
As the system is updated, additional kernel versions are installed.
dnf.conf contains installonly_limit=3, which results in a maximum of
three kernel versions being installed at a time. Once a fourth kernel
is installed, the first kernel and its modules are removed from
/usr/lib/modules. The rescue kernel+initramfs pair are never updated
or upgraded, even during system upgrades.
Observations------------
This has been discussed by the Workstation working group [2] but since
this functionality is present in all of Fedora, we're moving the
discussion for greater visibility.
There's two separate complaints, if you will: (a) that the
kernel+initramfs pair are never update or upgraded for the life of the
installation; and (b) that even during one release cycle, the user
experience when booting the rescue entry, changes, i.e. when the
matching /usr/lib/modules for the rescue entry are present early on,
you do get a full runtime behavior, you will get to a graphical
environment. But then once the version matched /usr/lib/modules are
removed, you get a completely different behavior when booting the
rescue entry.
An important note from that ticket from Justin Forbes, the Fedora
kernel maintainer: " Remember, the only real purpose of the rescue
kernel is to get your system out of something completely unusable. It
isn't meant to be a full runtime."
Questions------------
* Considering the very narrow purpose of the entry, maybe the current
behavior is adequate?
* Does the rescue entry reliably get users to a dracut prompt, rather
than indefinite hang? I don't know whether it does.
* Is there any way to improve the situation without increasing the
risk that the rescue entry becomes totally non-functional?
* The chosen kernel version needs to be based on one that is known
to boot. Currently we know the kernel+initramfs pair work because it's
the same version used to boot the installation media when doing the
initial provisioning. We don't actually know an updated replacement
"no host-only" initramfs will work until it's tried. Is it possible to
automate this? And is it worth the risk, or even figuring out how to
assess the risk?
* At Flock 2021, Zbyszek proposed "Building Initrd Images from
RPMs" to reduce the complexity of building initramfs, maybe there's a
role for it here? More: https://www.youtube.com/watch?v=GATg_bqmASc
* What happens if we accept some scope creep, and go for many
improvements that make the extra work worth it?
* What about the unsigned nature of the initramfs? Should we be
creating initramfs's in Fedora infra and signing them?
* Stuff a graphical rescue environment into the initramfs? (This
might be ten leaps too far, but it's intended to encourage thinking
with a vivid imagination.)
[1] both values from a recent Fedora 36 Workstation installation
[2] https://pagure.io/fedora-workstation/issue/259
--
Chris Murphy
1 month
Planning to start unifying native and mingw packages
by Sandro Mani
Hi
Following recent discussions and to reduce the maintenance burden, I'm
planning to start merging native and mingw packages. Initially, I'll be
looking at these packages where I maintain both variants:
eigen3 mingw-eigen3
enchant2 mingw-enchant2
freeimage mingw-freeimage
gdal mingw-gdal
GeographicLib mingw-GeographicLib
geos mingw-geos
giflib mingw-giflib
gtkspell3 mingw-gtkspell3
gtkspellmm30 mingw-gtkspellmm30
jxrlib mingw-jxrlib
leptonica mingw-leptonica
libgeotiff mingw-libgeotiff
libimagequant mingw-libimagequant
libkml mingw-libkml
librttopo mingw-librttopo
libspatialite mingw-libspatialite
libwebp mingw-libwebp
openjpeg2 mingw-openjpeg2
OpenSceneGraph mingw-OpenSceneGraph
osgearth mingw-osgearth
podofo mingw-podofo
proj mingw-proj
python-pillow mingw-python-pillow
qtspell mingw-qtspell
shapelib mingw-shapelib
svg2svgt mingw-svg2svgt
tesseract mingw-tesseract
uriparser mingw-uriparser
I'm performing test builds here [1]. Once I've got them all building
there, if there are no objections, I plan to push to F37 and retire all
the corresponding mingw repos.
Sandro
[1] https://copr.fedorainfracloud.org/coprs/smani/mingw-unified-spec/builds/
1 month
Orphaned X11 packages
by Adam Jackson
The following packages, previously owned by xgl-maint, are now up for grabs:
xorg-x11-xfs
xorg-sgml-doctools
xorg-x11-drv-v4l
xorg-x11-xsm
xorg-x11-twm
xorg-x11-drv-sisusb
xorg-x11-xdm
xorg-x11-docs
Upstream development on all of these is pretty much nil, so if you're
serious about picking up any of these you may also wish to take on the
module upstream:
https://gitlab.freedesktop.org/xorg
- ajax
1 month, 1 week
F37 proposal: Add -fno-omit-frame-pointer to default compilation
flags (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/fno-omit-frame-pointer
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Fedora will add -fno-omit-frame-pointer to the default C/C++
compilation flags, which will improve the effectiveness of profiling
and debugging tools.
== Owner ==
* Name: [[User:daandemeyer| Daan De Meyer]], [[User:Dcavalca| Davide
Cavalca]], [[ Andrii Nakryiko]]
* Email: daandemeyer(a)fb.com, dcavalca(a)fb.com, andriin(a)fb.com
== Detailed Description ==
Credits to Mirek Klimos, whose internal note on stacktrace unwinding
formed the basis for this change proposal (myreggg(a)gmail.com).
Any performance or efficiency work relies on accurate profiling data.
Sampling profilers probe the target program's call stack at regular
intervals and store the stack traces. If we collect enough of them, we
can closely approximate the real cost of a library or function with
minimal runtime overhead.
Stack trace capture what’s running on a thread. It should start with
clone - if the thread was created via clone syscall - or with _start -
if it’s the main thread of the process. The last function in the stack
trace is code that CPU is currently executing. If a stack starts with
[unknown] or any other symbol, it means it's not complete.
=== Unwinding ===
How does the profiler get the list of function names? There are two parts of it:
# Unwinding the stack - getting a list of virtual addresses pointing
to the executable code
# Symbolization - translating virtual addresses into human-readable
information, like function name, inlined functions at the address, or
file name and line number.
Unwinding is what we're interested in for the purpose of this
proposal. The important things are:
* Data on stack is split into frames, each frame belonging to one function.
* Right before each function call, the return address is put on the
stack. This is the instruction address in the caller to which we will
eventually return — and that's what we care about.
* One register, called the "frame pointer" or "base pointer" register
(RBP), is traditionally used to point to the beginning of the current
frame. Every function should back up RBP onto the stack and set it
properly at the very beginning.
The “frame pointer” part is achieved by adding push %rbp, mov
%rsp,%rbp to the beginning of every function and by adding pop %rbp
before returning. Using this knowledge, stack unwinding boils down to
traversing a linked list:
https://i.imgur.com/P6pFdPD.png
=== Where’s the catch? ===
The frame pointer register is not necessary to run a compiled binary.
It makes it easy to unwind the stack, and some debugging tools rely on
frame pointers, but the compiler knows how much data it put on the
stack, so it can generate code that doesn't need the RBP. Not using
the frame pointer register can make a program more efficient:
* We don’t need to back up the value of the register onto the stack,
which saves 3 instructions per function.
* We can treat the RBP as a general-purpose register and use it for
something else.
Whether the compiler sets frame pointer or not is controlled by the
-fomit-frame-pointer flag and the default is "omit", meaning we can’t
use this method of stack unwinding by default.
To make it possible to rely on the frame pointer being available,
we'll add -fno-omit-frame-pointer to the default C/C++ compilation
flags. This will instruct the compiler to make sure the frame pointer
is always available. This will in turn allow profiling tools to
provide accurate performance data which can drive performance
improvements in core libraries and executables.
== Feedback ==
=== Potential performance impact ===
* Meta builds all its libraries and executables with
-fno-omit-frame-pointer by default. Internal benchmarks did not show
significant impact on performance when omitting the frame pointer for
two of our most performance intensive applications.
* Firefox recently landed a change to preserve the frame pointer in
all jitted code
(https://bugzilla.mozilla.org/show_bug.cgi?id=1426134). No significant
decrease in performance was observed.
* Kernel 4.8 frame pointer benchmarks by Suse showed 5%-10%
regressions in some benchmarks
(https://lore.kernel.org/all/20170602104048.jkkzssljsompjdwy@suse.de/T/#u)
Should individual libraries or executables notice a significant
performance degradation caused by including the frame pointer
everywhere, these packages can opt-out on an individual basis as
described in https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags.
=== Alternatives to frame pointers ===
There are a few alternative ways to unwind stacks instead of using the
frame pointer:
* [https://dwarfstd.org DWARF] data - The compiler can emit extra
information that allows us to find the beginning of the frame without
the frame pointer, which means we can walk the stack exactly as
before. The problem is that we need to unwind the stack in kernel
space which isn't implemented in the kernel. Given that the kernel
implemented it's own format (ORC) instead of using DWARF, it's
unlikely that we'll see a DWARF unwinder in the kernel any time soon.
The perf tool allows you to use the DWARF data with
--call-graph=dwarf, but this means that it copies the full stack on
every event and unwinds in user space. This has very high overhead.
* [https://www.kernel.org/doc/html/v5.3/x86/orc-unwinder.html ORC]
(undwarf) - problems with unwinding in kernel led to creation of
another format with the same purpose as DWARF, just much simpler. This
can only be used to unwind kernel stack traces; it doesn't help us
with userspace stacks. More information on ORC can be found
[https://lwn.net/Articles/728339 here].
* [https://lwn.net/Articles/680985 LBR] - New Intel CPUs have a
feature that gives you source and target addresses for the last 16 (or
32, in newer CPUs) branches with no overhead. It can be configured to
record only function calls and to be used as a stack, which means it
can be used to get the stack trace. Sadly, you only get the last X
calls, and not the full stack trace, so the data can be very
incomplete. On top of that, many Fedora users might still be using
CPUs without LBR support which means we wouldn't be able to assume
working profilers on a Fedora system by default.
To summarize, if we want complete stacks with reasonably low overhead
(which we do, there's no other way to get accurate profiling data from
running services), frame pointers are currently the best option.
== Benefit to Fedora ==
Implementing this change will provide profiling tools with easy access
to stacktraces of installed libraries and executables which will lead
to more accurate profiling data in general. This in turn can be used
to implement optimizations to core libraries and executables which
will improve the overall performance of Fedora itself and the wider
Linux ecosystem.
Various debugging tools can also make use of the frame pointer to
access the current stacktrace, although tools like gdb can already do
this to some degree via embedded dwarf debugging info.
== Scope ==
* Proposal owners: Put up a PR to change the rpm macros to build
packages by default with -fno-omit-frame-pointer by default.
* Other developers: Review and merge the PR implementing the Change.
* Release engineering: [https://pagure.io/releng/issues #Releng issue
number]. A mass rebuild is required.
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact ==
This should not impact upgrades in any way.
== How To Test ==
# Build the package with the updated rpm macros
# Profile the binary with `perf record -g <binary>`
# Inspect the perf data with `perf report -g 'graph,0.5,caller'`
# When expanding hot functions in the perf report, perf should show
the full call graph of the hot function (at least for all functions
that are part of the binary compiled with -fno-omit-frame-pointer)
== User Experience ==
Fedora users will be more likely to have a streamlined experience when
trying to debug/profile system executables/libraries. Tools such as
perf will work out of the box instead of requiring to users to provide
extra options (e.g. --call-graph=dwarf/LBR) or requiring users to
recompile all relevant packages with -fno-omit-frame-pointer.
== Dependencies ==
The rpm macros for Fedora need to be adjusted to include
-fno-omit-frame-pointer in the default C/C++ compilation flags.
== Contingency Plan ==
* Contingency mechanism: The new version can be released without every
package being rebuilt with fno-omit-frame-pointer. Profiling will only
work perfectly once all packages have been rebuilt but there will be
no regression in behavior if not all packages have been rebuilt by the
time of the release. If the Change is found to introduce unacceptable
regressions, the PR implementing it can be reverted and affected
packages can be rebuilt.
* Contingency deadline: Final freeze
* Blocks release? No
== Documentation ==
* Original proposal for in-kernel DWARF unwinder (rejected):
https://lkml.org/lkml/2017/5/5/571
== Release Notes ==
Packages are now compiled with frame pointers included by default.
This will enable a variety of profiling and debugging tools to show
more information out of the box.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
2 months, 1 week
fedpkg clone fails with Permission denied (publickey).
by Richard Shaw
Long story short I lost my home directory where I do all of my packager
activities (separate from my main user) so I'm setting things up from
scratch.
I created new ssh keys and uploaded the public key to
admin.fedoraproject.org and pasted into pagure.io. It's been over an hour
and I'm still getting:
$ fedpkg clone hamlib
Cloning into 'hamlib'...
hobbes1069(a)pkgs.fedoraproject.org: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Could not execute clone: Failed to execute command.
I've also updated my API tokens, which is STILL not well documented. I
pasted them in the appropriate spot in "/etc/rpkg/fedpkg.conf" which isn't
real intuitive.
Thanks,
Richard
2 months, 1 week
F38 Proposal: SPDX License Phase 1 (Self-Contained Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/SPDX_Licenses_Phase_1
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Transition from Fedora's short name of licenses to standardized
[https://spdx.org/licenses/ SPDX license]
[https://spdx.dev/specifications/ formula].
== Owner ==
* Name: [[User:msuchy| Miroslav Suchý]]
* Name: [[User:jlovejoy| Jilayne Lovejoy]]
* Name: [[User:ngompa| Neal Gompa]]
* Name: [[User:dcantrell| David Cantrell]]
* Name: [[User:rfontanaref| Richard Fontana]]
* Name: [[User:mattdm| Matthew Miller]]
<!-- Include you email address that you can be reached should people
want to contact you about helping with your change, status is
requested, or technical issues need to be resolved. If the change
proposal is owned by a SIG, please also add a primary contact person.
-->
* Email: msuchy(a)redhat.com, dcantrell(a)redhat.com, jlovejoy(a)redhat.com,
ngompa13(a)gmail.com, rfontana(a)redhat.com
== Detailed Description ==
In the past, Fedora decided to use short names for licenses. Although
we documented the short names very well. The identifiers were never
standard. In the meantime, SPDX identifiers become standard, and
[https://wiki.spdx.org/view/Business_Team/Adoption other SW vendors
start using it].
In this phase, we want to provide documentation and tooling to allow
maintainers to begin using SPDX license ids instead of the old Fedora
short names. This move is opt-in. There will be
[[Changes/SPDX_Licenses_Phase_2|Phase 2]], where we identify the
remaining packages and help them to migrate to the SPDX formula.
== Feedback ==
Ancient [https://lists.fedoraproject.org/archives/list/legal@lists.fedoraproject.o...
feedback from SPDX organization].
Summary from [https://lists.fedoraproject.org/archives/search?q=spdx&page=1&mlist=legal...
fedora-legal mailing list]: we want this to happen, but this is big
scope and likely will happen over more than one release.
Summary from packaging-committee:
* [https://pagure.io/packaging-committee/pull-request/971#]: older PR
to change packaging guidelines
* [https://pagure.io/packaging-committee/pull-request/1142]: present
PR that needs more updating
Summary from devel-list: TBD
== Benefit to Fedora ==
The use of a standardized identifier for license will align Fedora
with other distributions. And allows efficient and reliable
identification of licenses.
== Scope ==
* Proposal owners (things sorted by done/todo and by priorities):
** Miroslav Suchý: license-fedora2spdx - done
** Jilayne Lovejoy: map rest of Fedora licenses to SPDX ids - done
** David Cantrell: create machine-readable format and new repo - done
** David Cantrell: merge mapping of Fedora licenses to SPDX ids to new
data format/repo - done
** Richard Fontana & Jilayne Lovejoy: review update all licensing info
and legal pages in wiki - in process
** Jilayne Lovejoy & Richard Fontana: create and populate new Docs
pages for legal and licensing info - in process
** Miroslav Suchy - create
[https://gitlab.com/fedora/legal/fedora-license-data
fedora-license-data package] (with data from rpminspect-data-fedora) -
TODO
** David Cantrell: separate licenses from rpminspect-data-fedora
[https://bugzilla.redhat.com/show_bug.cgi?id=2077914 BZ 2077914] -
TODO
** Miroslav Suchý: allow `license-validate` to use spdx - TODO
** David Cantrell: generate from license data to new Docs page similar
to [https://fedoraproject.org/wiki/Licensing:Main#Software_License_List
Licensing:Main]
** SOMEBODY: create a webhook that updates Docs page after the merge
to fedora-license-data - TODO
** Jilayne Lovejoy: prepare PR for updates to packaging guidelines -
in the process [https://pagure.io/packaging-committee/pull-request/1142]
** SOMEBODY: help maintainers who want to change license string to
SPDX identifiers proactively.
* Out of Scope: In this phase, we do not target to move **all**
packages to SPDX identifiers. That will be done in
[[Changes/SPDX_Licenses_Phase_2|Phase 2]]. In
[[Changes/SPDX_Licenses_Phase_2|Phase 2]] we will identify the
remaining packages and open BZ or PR.
* Other developers:
Early adopters can migrate their License tag to the SPDX identifiers.
Proposal owners will gather feedback and will work on potential
problems.
We want to have all bits ready so that maintainers can start changing
the spec files just after Fedora 37 branching (summer 2022).
* Release engineering:
* Policies and guidelines: Licensing page, packaging guidelines has to
be altered.
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives:
== Upgrade/compatibility impact ==
License strings are not used anything in run time. This change will
not affect the upgrade or runtime of Fedora.
During the transition period, developer tools like rpminspect,
licensecheck, etc. may produce false negatives. And we have to define
a date where we flip these tools from old Fedora's short names to the
SPDX formula.
== How To Test ==
Users should not need any testing. These steps are for package maintainers:
* Fetch your license string from `License` tag in SPEC file.
* Test that your current Fedora's short name is correct. E.g.
$ license-validate -v 'MIT or GPLv1'
Approved license
* Convert license string to SPDX formula:
$ license-fedora2spdx 'MIT or GPLv1'
Warning: more options how to interpret MIT. Possible options:
['Adobe-Glyph', 'MIT-CMU', 'MIT-CMU', 'HPND', 'HPND', 'no-spdx-yet
(MIT license (also X11))', 'SGI-B-2.0', 'SGI-B-2.0', 'SMLNJ',
'MIT-enna', 'MIT-feh', 'mpich2']
mpich2 or GPL-1.0-only
In this example, the short name `GPLv1` can be converted straight to
`GPL-1.0-only`. But short name `MIT` stands for several licenses with
different [https://spdx.org/licenses/ SPDX identifiers]. You have to
examine what license is package actually using. `license-fedora2spdx`
will try to convert the formula and use one of the options but without
any heuristics. You need to manually review the license.
You can check if SPDX formula is correct using:
$ license-validate -v --file FIXME "MIT-CMU or GPL-1.0-only"
== User Experience ==
Users should be able to use standard software tools that audit
licenses. E.g. for Software Bills of Materials.
== Dependencies ==
No other dependencies.
== Contingency Plan ==
* Contingency mechanism: In this first phase, if something goes wrong,
we can 'git revert' each change in dist-git. It is expected that in
the first phase, there will be only a few packages altered. It may be
a few hundred, but it is still doable to revert.
* Contingency deadline: Beta freeze. But it is expected that not all
packages will be converted by that time and the change will continue
in the next release.
* Blocks release? No. This change has no impact on runtime of any package.
== Documentation ==
N/A (not a System Wide Change)
== Release Notes ==
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
4 months
