Does ccache ever help with kernel mock build?
by Julian Sikorski
Hello,
has anyone successfully managed to get ccache to speed up kernel
mockbuilds, be it SRPMS from kernel-ark or from dist-git? I gave it a
brief shot but saw no difference, if anything the build was slower.
The machine I am building on has 32 GB RAM which is not quite enough to
fit everything on a ramdisk, and the drive I am building to is an NVME
SSD, "only" PCI Express 3.0. CPU is a Ryzen 5 5600x. A baseonly builds
takes about 25 minutes, which is somewhat annoying when one has to
bisect something.
If ccache can speed this up, how big does it need to be? I understand
the default 4 GB is nowhere near enough but what would be needed to
actually help? Thanks!
Best regards,
Julian
5 days, 20 hours
[SPDX] Mass license change GPL+ to GPL-1.0-or-later
by Miroslav Suchý
Hi.
I am going to do the mass change of the license from GPL+ to GPL-1.0-or-later
The proposed diff is in attachment.
Affected packages:
afpfs-ng
alex4
arc
atomorun
bib2html
buffer
catdoc
colorgcc
curlftpfs
cvsps
dia-gnomeDIAicons
docbook-utils
e2tools
echolinux
flpsed
fortune-firefly
freeze
gambas3
gemdropx
gnome-mime-data
gphotofs
graphlcd-base
gt5
hwinfo
jed
launchy
logserial
machineball
meterbridge
mirrormagic
moon-buggy
mussh
netgen
nethogs
R-servr
nss_updatedb
nullmodem
p0rn-comfort
perl-HTTP-Recorder
perl-Image-Xpm
perl-Nagios-NSCA
perl-Test-AutoLoader
perl-WWW-OrangeHRM-Client
php-lukasreschke-id3parser
procinfo
pwgen
qucs
R-biglm
redhat-menus
rocksndiamonds
rpld
rubber
safecopy
setserial
sgml-common
shippy
SolarModel
six
stripesnoop
system-config-rootpassword
tcpflow
trophy
tse3
tunctl
tuxanci
vdr-epg2vdr
vdr-remote
vdr-scraper2vdr
vdr-screenshot
vdr-skinenigmang
wipe
xdialog
xl2tpd
xosview
xteddy
zisofs-tools
Unless somebody stop me, I will do this change directly in dist-git after a week.
--
Miroslav Suchy, RHCA
Red Hat, Manager, Packit and CPT, #brno, #fedora-buildsys
1 week
pipenv removal in F40
by Mattia Verga
I vaguely remember that pipenv retirement was briefly discussed here on
the ML, yet I was surprised that F40 doesn't have pipenv anymore.
IMO, this would have been announced more prominently as a self contained
change, as I expect more python developers to find out this too late.
Also, the official guide on
https://developer.fedoraproject.org/tech/languages/python/pipenv.html
should have been updated as well.
Mattia
2 weeks, 1 day
F35 Change: Debuginfod By Default (Self-Contained Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/DebuginfodByDefault
== Summary ==
Fedora users / developers who need to debug/trace distro binaries can
make use of the recently activated elfutils-debuginfod servers to
automatically fetch debugging data and source code, instead of having
to use `# sudo dnf` commands.
== Owner ==
* Name: [[User:fche| Frank Ch. Eigler]]
* Email: fche(a)redhat.com
== Detailed Description ==
Numerous fedora debugging-type tools have built-in capabilities to use
the [https://sourceware.org/elfutils/Debuginfod.html debuginfod]
protocol to fetch debuginfo/source code automatically. We would like
to activate a setting so that Fedora's debuginfod servers are
automatically used, rather than requiring hand-editing individual
users' dot files.
== Feedback ==
There has existed
[https://www.spinics.net/lists/fedora-devel/msg279002.html broad
interest] in a Fedora debuginfod server since the project was proposed
/ announced in 2020, and several distros already operate public
servers of this nature. Some of the distros configure their
installations by default to talk to those servers, some do not.
Turning on this by default has some limited privacy implications.
Some Debian users have
[https://lists.debian.org/debian-devel/2021/02/msg00262.html expressed
concerns] that this facility "calls home" during debugging, so it may
expose a limited amount of information about what a user is debugging.
The information is limited to the build-id and source code file names
of programs being debugged, and is only sent to the servers if their
machine lacks locally installed debuginfo. Whether this should be
opt-in or opt-out and how has been resolved there via an install-time
query to the sysadmin. In contrast, on OpenSUSE Tumbleweed, it is
[https://build.opensuse.org/request/show/879926 simply defaulted-on],
and we have heard of no controversy.
We anticipate discussing this topic on the mailing list, and noting it
in the release notes either way.
== Benefit to Fedora ==
This will improve developers' experience.
It may reduce download server burden, as only individual
ELF/DWARF/source files are downloaded rather than entire `-debuginfo`
and `-debugsource` RPMs.
It would help Fedora catch up to other distros who put up `debuginfod`
servers already. :-)
== Scope ==
* Proposal owners:
The work could consist one extra parameter in the `elfutils.spec`
`%configure`. Its effect is to arrange for the
`elfutils-debuginfod-client` RPM
to install an `/etc/profile.d` file that sets the `DEBUGINFOD_URLS`
environment variable automatically to
`https://debuginfod.fedoraproject.org/`. (At the time of this
writing, the _staging_ server is getting ready for testing:
`https://debuginfod.stg.fedoraproject.org/`.)
* Other developers: None - relevant code has been previously upstreamed!
* Release engineering: None - our team is operating the
`debuginfod[.stg].fedoraproject.org` VMs.
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact ==
None.
Note that these servers index all active Fedora releases (32-), all
architectures, so users of those versions can already set
`DEBUGINFOD_URLS` manually to take advantage.
== How To Test ==
* Install `elfutils-debuginfod-client`
* Open arbitrary fedora binary via gdb.
** Admire the immediate downloading of debuginfo and source code.
* Run `eu-stack -v -p $pid` for an arbitrary process.
** Admire the immediate downloading of debuginfo to give precise file:line data.
== User Experience ==
Primarily: users running debuggers, profilers, tracing tools on
internet-capable machines can work immediately, without switching to
privileged users and fragile manual `dnf` commands to install this
data.
== Dependencies ==
The `debuginfod` servers at `fedora-infra` need to be up.
== Contingency Plan ==
* Contingency mechanism: change the elfutils-debuginfod-client subrpm
to not set the default `DEBUGINFOD_URLS` environment variable for all
users. In the case of a server outage, the debugger tools revert to
debuginfo-less operation, prior to this feature.
* Contingency deadline: shortly before freeze
* Blocks release? No
== Documentation ==
There is upstream documentation in the debugging tools as well as
associated with the client code / cli tooling. What our Release Notes
would focus on however is the _automatic activation_ of this facility
via the environment variable.
== Release Notes ==
TBD.
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
2 weeks, 5 days
future of dual booting Windows and Fedora, redux
by Chris Murphy
Summary: Windows 10/11 increasingly enables Bitlocker (full disk encryption) out of the box with the encryption key sealed in the TPM. Two different issues result:
1. Fedora's installer, Anaconda, can't resize Bitlocker volumes. We could use better documentation to help the user perform the volume resize in Windows, before proceeding to booting our installation media. The documentation probably should be explicitly referenced by the Windows version of Fedora Media Writer.
2. The Bitlocker encryption key is unsealed only if the boot chain measurement by the TPM matches the expected values in a TPM PCR. When shim+GRUB are in the boot chain, as is the case in our default dual boot installation, the measurements are wrong, and this means the GRUB menu entry to boot Windows won't work. The user is dropped to a Windows Bitlocker recovery page. If they have their backup encryption key handy, it will work but to say the least this condition is unexpected and not user friendly - not least of which is many users won't have this backup key handy since they didn't take the action to enable Bitlocker in the first place.
The bug report for this is https://bugzilla.redhat.com/show_bug.cgi?id=2049849
It was a Fedora 36 final release blocking bug, but considered a "difficult to fix" exceptional case, moving it to Fedora 37 final. Some of the options for consideration:
a. Fix GRUB by giving it the ability to modify UEFI NRAM "bootnext" value, so that instead of chainloading the Windows bootloader from GRUB, GRUB will modify the system NVRAM such that the next boot (only) will directly boot the Windows bootloader. Thus far there's no interest by GRUB upstream. Whereas systemd-boot has implemented it.
b. Add a user space utility modifies system NVRAM such that the next boot (only) will directly boot the Windows bootloader. And also remove the Windows boot entry in GRUB, on UEFI systems. (It would be retained on BIOS systems.)
c. Change the release criterion.
https://fedoraproject.org/wiki/Fedora_36_Final_Release_Criteria#Windows_d...
Current: The installer must be able to install into free space alongside an existing clean Windows installation and install a bootloader which can boot into both Windows and Fedora.
Replacement: The installer must be able to install into free space alongside an existing clean Windows installation, install and configure a bootloader that will boot Fedora.
d. Consider the problem sufficiently difficult to fix that we need an extension to the exceptional case allowance, and wave the bug for another release.
Thoughts?
--
Chris Murphy
3 weeks, 6 days
LLVM Packaging Ideas for Fedora 41
by Tom Stellard
Hi,
After each Fedora release we do a retrospective with the LLVM package maintainers
and talk about how we can improve the LLVM packages[1] in Fedora. We've come up
with some ideas for Fedora 41 that we'd like to share to raise awareness and
get feedback. Right now these are just ideas, and we plan to write up a formal
change proposal once we have decided which of these we are going to implement:
* Spec file merge. We plan to merge the clang, compiler-rt, and libomp packages
in with llvm and have them be sub-packages of the llvm package. This will allow
us to use the build configuration recommended by upstream and also make it possible
to optimize the packages using Profile-Guided Optimizations (PGO).
* Build compat packages (e.g. llvm18) as early as possible. When we package a new
major release of llvm, we create a compat package so that packages that aren't
compatible with the new version can still use the old version. In the past, we've
waited to introduce the compat packages until the new version of LLVM was ready
(typically during the Beta Freeze). However, this proved to be an issue this
release for packages the were ready to switch to the compat packages early in
the release cycle, but then had to wait for Beta freeze.
* Switch to python-style compat/main packages. In order to make the packaging more
consistent between the main package (e.g. llvm) and the compat package (e.g. llvm18),
we would retire the un-versioned dist-git for llvm, and create a new versioned dist-git
for each new release (e.g. llvm19, llvm20, llvm21 etc.). We would then designate one
of these as the 'main version', and that version would produce binary rpms that look
like the current main package (i.e. llvm-libs instead of llvm19-libs).
* Invert the order of compat/main packages. Instead of having the compat package be
the old version, and the main package be the new version, we would have the compat package
be newer and the main package be older. This would allow us to introduce a new version of
llvm without impacting other packages that depend on the main version of LLVM.
If anyone has any feedback on these ideas we'd like to hear it and are happy to discuss
these more.
Thanks,
Tom
[1] LLVM Packages are: llvm, clang, compiler-rt, libomp, lld, lldb, llvm-test-suite, libclc,
llvm-bolt, libcxx, mlir, flang, python-lit, and polly.
1 month
F41 Change Proposal - Reproducible Package Builds (System-Wide)
by Aoife Moloney
Wiki - https://fedoraproject.org/wiki/Changes/ReproduciblePackageBuilds
Discussion.fpo -
https://discussion.fedoraproject.org/t/f41-change-proposal-reproducible-p...
== Summary ==
A post-build cleanup is integrated into the RPM build process so that
common causes of build irreproducibility in packages are removed,
making most of Fedora packages reproducible.
== Owner ==
* Name: Davide Cavalca
* Name: Neil Hanlon
* Name: [[User:churchyard|Miro Hrončok]]
* Name: [[User:zbyszek|Zbigniew Jędrzejewski-Szmek]]
* Email: dcavalca(a)fedoraproject.org
* Email: neil at shrug.pw
* Email: mhroncok at redhat.com
* Email: zbyszek at in.waw.pl
== Detailed Description ==
As of 2023 there is an active effort to implement
[https://docs.fedoraproject.org/en-US/reproducible-builds/
Reproducible builds] in Fedora. Reproducible builds will allow our
users to be able to independently verify that the RPMs have not been
tampered with (either maliciously or via hardware/software fault):
someone can do an independent rebuild of a package and confirm that
they get identical binaries when building with the same versions of
the compiler and other tools. This Change allows us to move forward in
this direction by removing the common sources of irreproducibility.
[https://github.com/keszybz/add-determinism add-determinism] is a Rust
program which, as its name suggests, adds determinism to files that
are given as input by attempting to standardize metadata contained in
binary or source files to ensure consistency and clamping to
$SOURCE_DATE_EPOCH in all instances. `add-determinism` is the "Fedora
version" of [https://salsa.debian.org/reproducible-builds/strip-nondeterminism
strip-nondeterminism] from the Debian project. Since
strip-nondeterminism is written in perl, it is undesirable for use in
Fedora, as we don't want to pull perl in the buildroot for every
package.
It's worth noting that this Change does not intend to impose any
specific reproducibility requirements on Fedora packages. Once this
Change is implemented and we have been through a mass rebuild and can
verify that the common causes of irreproducibility have indeed been
removed, we can consider further steps. But that will be at least one
release later.
This change does add a small amount of time to the processing of RPMs
at the end of a build. Accordingly, packages containing large
quantities or sizes of files can be slower, but this effect is not
expected to be noticeable. `add-determinism` takes steps to ensure it
does not interfere with other buildroot post processors like
`mangle-shebangs`, `python-hardlink`, `python-bytecompile`. It
defaults to not doing any modifications in case it doesn't understand
the input file or there are any other problems.
A mechanism to opt-out will be provided: to either completely disable
the postprocessing step or to disable specific "handlers" (i.e.
implementations of cleanup for specific file types, for example static
archives). See [https://github.com/keszybz/add-determinism/blob/main/rpm/macros.build-rep...
macros.build-reproducibility].
=== Related Changes ===
* [https://fedoraproject.org/wiki/Changes/ReproducibleBuildsClampMtimes
Clamp build mtimes to SOURCE_DATE_EPOCH]
* [https://fedoraproject.org/wiki/Changes/RPM-4.20 RPM 4.20] — this
pulls in changes to `%autosetup -S git` which removed a source of
irreproducibility.
== Feedback ==
== Benefit to Fedora ==
Adding determinism (i.e., removing non-determinsim) enables the Fedora
community to have confidence that, if given the same source code,
build environment, build instructions, and metadata from the build
artifacts, any party can recreate copies of the artifacts that are
identical except for the signatures and some parts of metadata.
Reproducibility of builds leads to packages of higher quality. It
turns out that quite often those irreproducible bits are caused by an
error or sloppiness in the code. In particular, any dependence on
architecture in noarch packages is almost always unwanted and/or a
bug. Test builds that check reproducibility will expose such
instances.
Reproducibility of builds makes it easier to develop packages: when a
small change is made and a package is rebuilt (in the same
environment), then with a reproducible package, the only difference is
directly caused by the change. If the package is different every time
it is rebuilt, making a comparison is much harder.
Build reproducibility for noarch ''sub''packages solves the problem
where package builds on different architectures are different, causing
mock to reject the whole build. In particular, this issue occurs for
[https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix...
pyc files]. This will now be solved without requiring opt-in from
individual packages.
== Scope ==
* Proposal Owners:
** Integrate `add-determinism` as a BuildRoot Policy script
** Add a dependency on `marshalparser` to `python3` (probably
conditionalized on `rpm-build`)
* Other Developers:
** Test their packages with the additional phase, report problems
** Potentially integrate changes to packages to enable reproducibility
* Release Engineering: Ideally we want this to happen before the mass
rebuild, but that is not strictly required.
* Policies and Guidelines: Fedora Packaging Guidelines should be
updated to include information on the add-determinism BuildRoot
Policy. User documentation should be amended to include instructions
on how to verify reproducibility for a given package, and what
packages are known to be non-reproducible, and how to opt-out.
* Trademark approval: N/A (not needed for this Change)
* Alignment with Community Initiatives: All software and requests are
consistent with the decision process and similar across other groups
in Fedora. The Fedora Reproducibility Working group began at Flock
2023 in Cork.
== Upgrade/compatibility impact ==
No impact is expected.
== How To Test ==
To test on the level of individual files:
* install `add-determinism`
* call `SOURCE_DATE_EPOCH=… add-determinism -v ./path/to/file`
To test package builds:
* build a local copy of `redhat-rpm-config` with
https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/293
* install `add-determinism`
* build packages ;)
(This can be done on a normal system or in a mock chroot.)
== User Experience ==
No impact is expected.
== Dependencies ==
== Contingency Plan ==
* Contingency mechanism:
** In case of major problems, disable the change in `redhat-rpm-config`.
** In case of problems with specific packages, opt-out by setting a macro.
* Contingency deadline: No limit really.
* Blocks release? No.
== Documentation ==
* [https://docs.fedoraproject.org/en-US/reproducible-builds/ Fedora
Reproducible Builds]
* [https://github.com/keszybz/add-determinism/blob/main/rpm/macros.build-rep...
add-determinism macros.build-reproducibility]
* [https://github.com/keszybz/add-determinism/tree/main?tab=readme-ov-file#b...
add-determinism README]
== Release Notes ==
Fedora package builds are now more deterministic, bringing the
distribution closer to the goal of achieving fully reproducible builds
for all of its packages.
--
Aoife Moloney
Fedora Operations Architect
Fedora Project
Matrix: @amoloney:fedora.im
IRC: amoloney
--
_______________________________________________
devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org
To unsubscribe send an email to devel-announce-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedora...
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
1 month
