F38 proposal: Add Fedora Auto Firstboot Services to desktop variants
(System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/AutoFirstBootServices
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Add {{package|fedora-autofirstboot}} to desktop variants to run a
predetermined set of tasks on first boot after post installation,
notably installing codecs and cleaning up installer packages from the
installed system.
== Owner ==
* Name: [[User:Ngompa| Neal Gompa]]
* Email: ngompa13(a)gmail.com
== Detailed Description ==
{{package|fedora-autofirstboot}} is a collection of scripts that
invoke on firstboot of a freshly installed system to run a set of
predetermined tasks. It also provides a framework for third-parties to
introduce their own firstboot tasks to run through this framework. The
initial services included are to install OpenH264 and remove Anaconda.
== Benefit to Fedora ==
The main benefit is to smooth out the new user experience for new
Fedora Linux installations. In particular, we can deal with a
long-standing sticking point that Anaconda remains installed on the
user's machine when it is not useful to do so.
== Scope ==
* Proposal owners:
** Add {{package|fedora-autofirstboot}} to the desktop kickstarts
** Add a preset to {{package|fedora-release}} for
<code>fedora-autofirstboot.service</code>
* Other developers: N/A (not needed for this Change)
* Release engineering: [https://pagure.io/releng/issue/11148 #11148]
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A
== Upgrade/compatibility impact ==
This will have no impact on upgraded systems, since the firstboot
condition is not true in that case.
== How To Test ==
# Install Fedora Workstation, KDE, etc.
# Reboot into installed environment
# Check to see <code>openh264</code> is installed and
<code>anaconda-core</code> is not.
== User Experience ==
The first boot will be slightly slower because of these tasks running,
though they should happily run in the background as other services
start up, so it should not be noticeable.
== Dependencies ==
The main dependency is {{package|fedora-release}}, though we will need
to ensure all {{package|udisks2}} plugins get pulled in as
dependencies for {{package|gnome-disks}} and {{package|blivet-gui}} so
they don't get uninstalled when Anaconda is.
== Contingency Plan ==
* Contingency mechanism: Remove {{package|fedora-autofirstboot}} from
the kickstarts
* Contingency deadline: Final freeze
* Blocks release? No
== Documentation ==
There is not currently much documentation in
[https://pagure.io/fedora-autofirstboot the upstream project], though
contributions are welcome.
== Release Notes ==
Fedora Linux now ships with a framework for setting up first-boot
services and uses this to install multimedia software and remove the
installer software from the system after installation.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
1 year, 2 months
F37 proposal: RPM Macros for Build Flags (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/RPMMacrosForBuildFlags
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Create a corresponding macro for each compiler flag in the
redhat-rpm-config macro file and create "extra flag" macros to make it
easier for packages to add and remove compiler flags.
== Owner ==
* Name: [[User:tstellar| Tom Stellard]]
* Email: <tstellar(a)redhat.com>
== Detailed Description ==
The macros file in the redhat-rpm-config package contains a list of
default compiler flags for packages to use when compiling C, C++, and
Fortran packages. There is currently no standard way to remove or add
to the set of default flags. Most packages use a combination of echo
and sed to remove unwanted flags or add new ones. Some examples:
compiler-rt:
[https://src.fedoraproject.org/rpms/compiler-rt/blob/rawhide/f/compiler-rt...
global optflags %(echo %{optflags} -D_DEFAULT_SOURCE)]
julia:
[https://src.fedoraproject.org/rpms/julia/blob/rawhide/f/julia.spec#_267
%global optflags %(echo %{optflags} | sed 's/-Wp,-D_GLIBCXX_ASSERTIONS
//')]
This change will add new macros which will make it easier for packages
to add and remove their own compiler flags. This strategy is already
used to some extent with feature macros like %{_lto_cflags},
%{_hardening_cflags}, etc, but these new macros will give packagers
even more fine-grained control over the options.
The proposed macros for adding new flags are:
%_pkg_extra_cflags
%_pkg_extra_cxxflags
%_pkg_extra_fflags
%_pkg_extra_ldflags
These will be added to %{build_cflags}, %{build_cxxflags},
%{build_fflags}, and %{build_ldflags} respectively to allow packges to
add their own flags to the default list: e.g.
%build_cflags %{optflags} %{_pkg_extra_cflags}
The proposed new macros to represent existing flags are:
%_flag_fstack_protector_strong -fstack-protector-strong
%_flag_z_now -Wl,-z,now
%_flag_z_defs -Wl,-z,defs
%_flag_flto_auto -flto=auto
%_flag_ffat_lto_objects -ffat-lto-objects
%_flag_o -O2
%_flag_f_exceptions -fexceptions
%_flag_g -g
%_flag_grecord_gcc_switches -grecord-gcc-switches
%_flag_pipe -pipe
%_flag_wall -Wall
%_flag_werror_format_security -Werror=format-security
%_flag_fortify_source -Wp,-D_FORTIFY_SOURCE=2
%_flag_glibcxx_assertions -Wp,-D_GLIBCXX_ASSERTIONS
%_flag_asynchronous_unwind_tables -fasynchronous-unwind-tables
%_flag_fstack_clash_protection -fstack-clash-protection
%_flag_fcf_protection -fcf-protection
%_flag_mbranch_protection_standard -mbranch-protection=standard
With these new macros, the examples from above could be re-written as:
compiler-rt: %global _pkg_extra_cflags -D_DEFAULT_SOURCE
julia: %global _flag_glibcxx_assertions %{nil}
For more details see the
[https://src.fedoraproject.org/fork/tstellar/rpms/redhat-rpm-config/c/0ee9...
Prototype Implementation].
In addition to adding these new macros, the packaging guidelines will
be updated to require that all new flags added to redhat-rpm-config
have their own RPM macro.
== Benefit to Fedora ==
* It will provide a standard way to disable existing compiler flags or
enable new ones that is more simple and robust than the existing echo
+ sed solution.
* It will make it easier to determine which packages disable or add
compiler flags by doing a simple grep of the spec files.
* It will make it easier for toolchain developers to experiment with
adding new flags to the distribution as this can be done with a simple
macro definition instead of patching redhat-rpm-config.
== Scope ==
* Proposal owners:
** Proposal owners will update the redhat-rpm-config package and add
the new macros.
** Proposal owners will test the changes to ensure that the correct
flags are still being used.
* Other developers:
** Other developers may, but are not required to, update their
packages to use the new macros.
* Release engineering: [https://pagure.io/releng/issues/10819 #10819]
* Policies and guidelines:
** The Fedora packaging policy will be updated to require that new
flags added to redhat-rpm-config come with their own RPM macro.
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives:
== Upgrade/compatibility impact ==
None.
== How To Test ==
* This can be tested by inspecting the value of the %{build_cflags},
%{build_cxxflags}, %{build_fflags}, and %{build_ldflags} and ensuring
they are the same before and after the change.
* This can be tested by modifying some of the new macros in a spec
file and ensuring that the changes appear in the appropriate macro
mentioned above.
== User Experience ==
This is a change for developers and will have no impact to the user experience.
== Dependencies ==
None.
== Contingency Plan ==
* Contingency mechanism: (What to do? Who will do it?) Change owner
will revert the update to redhat-rpm-config.
* Contingency deadline: Mass Rebuild
* Blocks release? N/A (not a System Wide Change), No
== Documentation ==
None.
== Release Notes ==
None.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
1 year, 2 months
F37 Change Proposal: MAC Address Policy none (System-Wide Change)
by Vipul Siddharth
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
The `systemd-udev` package installs
`"/usr/lib/systemd/network/99-default.link"`,
which sets `Link.MACAddressPolicy=persistent`. This proposal is to
change it to set `Link.MACAddressPolicy=none` to stop changing the MAC address.
This is particularly important for bridge and bond devices.
This change can either only apply to bridge/bond devices, or to
various software devices. That is to be discussed.
== Owner ==
* Name: [[User:thaller|Thomas Haller]] (NetworkManager)
* Email: <thaller(a)redhat.com>
* Name: [[User:dustymabe| Dusty Mabe]] (Fedora CoreOS)
* Email: <dmabe(a)redhat.com>
== Detailed Description ==
On Fedora, udev by default changes the MAC address of a wide range of
software devices.
This was introduced by systemd 242 in early 2019 (Fedora 31), when
`MACAddressPolicy=` was
extended to affect more types of devices.
With `MACAddressPolicy=persistent` udev's aim is to provide a stable
MAC address, otherwise
the kernel will assign a random one. However, that can cause problems:
Firstly, software devices are always created by some tool that has
plans for the device. The tool may not
expect that udev is going to change the MAC address and races against
that. The best solution
for the tool is to set the MAC address when creating an interface.
This will prevent
udev from changing the MAC address according to the MACAddressPolicy.
Otherwise, the tool should wait for udev to initialize the device to
avoid the race. In theory, a
tool is always advised to wait for udev to initialize the device.
However, if it were not for MACAddressPolicy,
in common scenarios udev doesn't do anything relevant for software
devices to make that necessary.
Secondly, for interface types bridge and bond, an unset MAC address
has a special meaning
to the kernel and the MAC address of the first port is used. If udev
changes the MAC
address, that no longer works.
Now the generated MAC address is not directly discoverable as it is
based on `/etc/machine-id`
([https://www.man7.org/linux/man-pages/man5/machine-id.5.html
machine-id(5)]), among
other data. Even if there were a tool to easily calculate the MAC
address, it could be cumbersome
to use it without logging into the machine first. The MAC address can
directly affect the
assigned IP address, for example when using DHCP. When booting a new
virtual machine, the user might
know the MAC address of the (virtual) "physical" interfaces. When
bonding/bridging those
interfaces, the bond/bridge would get one of the well known MAC
addresses. `MACAddressPolicy=persistent`
interferes with that.
The goal of persistent policy is to provide a stable MAC address. Note
that if the
tool or user who created the interface would want a certain MAC address, they
have all the means to set it already. That applies regardless whether
the tool is
iproute2, NetworkManager, systemd-networkd. Neither NetworkManager nor
systemd-networkd
rely on udev's MACAddressPolicy for setting the MAC address. This
behavior is mostly
useful for plain `ip link add`, but it's unclear which real world user
wants this behavior.
Of course, the user is welcome to configure the MAC address in any way
they want. Including,
dropping a link file that sets `MACAddressPolicy=persistent`. The
problem is once udev sets a MAC address,
it cannot be unset. Which makes this problematic to do by default.
While Fedora inherited this behavior from upstream systemd, RHEL-9
does not follow this behavior
([https://gitlab.com/redhat/centos-stream/rpms/systemd/-/blob/c8953519504bf...
centos9],
[https://bugzilla.redhat.com/show_bug.cgi?id=1921094 rh#1921094]). For
RHEL-8, this doesn't
apply because the systemd there is too old to change the MAC address
of most software devices.
This could be either implemented by patching
`/usr/lib/systemd/network/99-default.link`
to have a different policy, or by dropping a link file with higher
priority. In the latter
case, that override could be shipped either by udev or even by NetworkManager.
Another option is to change the scope of this proposal to only change to
`MACAddressPolicy=none` for the device types where this causes the most issues
(bridge/bond/team).
== Feedback ==
This was also discussed on upstream systemd mailing list
[https://lists.freedesktop.org/archives/systemd-devel/2022-May/047893.html
here].
The upstream systemd maintainers' opinion is that the current udev
behavior is desirable, but accepts that distributions (or network
stacks such as NetworkManager) may choose to change the default
depending on their needs
([https://lists.freedesktop.org/archives/systemd-devel/2022-May/047943.html
reference]).
The goal here is to find out what the Fedora community thinks is the
most appropriate default.
The RHEL-9 bug is [https://bugzilla.redhat.com/show_bug.cgi?id=1921094
[rh#1921094]].
== Benefit to Fedora ==
Pros:
- Consistent behavior with RHEL8 and RHEL9.
- Avoid race of udev and the tool that creates the interface.
- Bridge and bond interfaces can get the MAC addresses from their first port.
In the case of `MACAddressPolicy=none` for a bond (or bridge) the bond will
get a MAC related to one of its physical NIC devices. In the case of
provisioning
new systems (especially in a large datacenter) information about the server
can be used to configure the network environment (DHCP, Firewall, etc) before
the server is ever even powered on. This does mean that you may have to update
your network environment configuration if you replace a NIC (con), but that you
won't have to update your network environment configuration if you re-install
your server (pro), which is what you'd have to do now with
`MACAddressPolicy=persistent`.
Cons:
- Deviate from upstream systemd.
It is desirable that RHEL and Fedora behaves similar. A possible outcome
could be the current behavior stays and RHEL 10 would change behavior. On the
other hand, different distributions (or even Fedora spins) have different
uses and needs. Deviating might be fine. In the same vein, there is also
a desire to stay close to upstream systemd behavior. But the uses of systemd
project go beyond Fedora/RHEL, so deviating here may also be fine.
== Scope ==
* Proposal owners:
The main goal of this request is to generate productive discussion and
find the desired behavior.
The implementation/changes are either way very simple.
* Other developers:
Other projects that wish a certain MAC address are welcome to
set it for their devices. Including using udev's MACAddressPolicy.
* Release engineering:
Not needed for this change.
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives:
== Upgrade/compatibility impact ==
After the change, the MAC address for the affected device types changes.
== How To Test ==
1) Create a software device two times, for example `ip link add type
bridge`. Note
that the MAC address is either stable or random, depending on the
`MACAddressPolicy=`.
2) Note that if the software device has the MAC address set initially,
udev does not
change it (`ip link add address aa:aa:aa:aa:aa:aa type bridge`). That depends on
`/sys/class/net/$dev/addr_assign_type`.
3) Create a bridge/bond interface without setting the MAC address.
Note that if `MACAddressPolicy=none`,
the MAC address is random at first. Note that attaching the first port
will update the controller's MAC address.
On the other hand, with `MACAddressPolicy=persistent`, the MAC address
of the controller is fixed
and not inherited from the port.
4) Run
ip monitor link &
while : ; do
ip link del xxx
ip link add name xxx type dummy \
&& ip link set xxx addr aa:00:00:00:00:00 \
&& ip link show xxx | grep -q aa:00:00:00:00:00 \
|| break
done
to reproduce the race between a simple tool and udev changing the MAC address.
== User Experience ==
Bond/bridge devices would again get assigned the MAC address of the
first NIC added to the device.
If we chose to not limit the scope of this change to just
bonds/bridges then all software devices would get randomly assigned
MAC addresses.
== Dependencies ==
None.
== Contingency Plan ==
If the change is rejected, nothing needs to be done. The change
itself will be simple to implement.
Contingency deadline: beta freeze
Blocks release? No
== Documentation ==
TODO.
== Release Notes ==
--
Vipul Siddharth
He/His/Him
FPgM team member
1 year, 2 months
HEADS-UP: Upcoming retirement of long-term-unused packages for Rust crates
by Fabio Valentini
Hi all,
I've been collecting data about the dependency graph of Rust packages
in Fedora for over a year now, and I would like to start the process
of removing some accumulated cruft. In particular, I've been keeping
track of which packages for *library* packages (i.e. they ship only
source code but no binaries) have been leaf packages.
List of Rust library-only packages, which no other package in Fedora
has depended on, for over a year (365 days+):
- rust-compiletest_rs
- rust-constant_time_eq
- rust-conv
- rust-counted-array
- rust-dbus-tokio
- rust-defmac
- rust-dialoguer
- rust-enumset
- rust-failure-tools
- rust-fake-simd
- rust-fbthrift_codegen_includer_proc_macro
- rust-fdlimit
- rust-hamcrest2
- rust-html2pango
- rust-imgref
- rust-ioctl-rs
- rust-lipsum
- rust-listenfd
- rust-loggerv
- rust-lzw
- rust-macro-attr
- rust-mdl
- rust-mktemp
- rust-mnt
- rust-newtype_derive
- rust-odds
- rust-osstrtools
- rust-parse_cfg
- rust-permutate
- rust-piper
- rust-podio
- rust-proc-quote-impl
- rust-process_path
- rust-progress-streams
- rust-protoc-rust
- rust-quickersort
- rust-rand_jitter
- rust-rand_os
- rust-read_input
- rust-relay
- rust-rustc_tools_util
- rust-rustdoc-stripper
- rust-rustfilt
- rust-safe-transmute
- rust-scoped-tls-hkt
- rust-serde-pickle
- rust-serial-core
- rust-sluice
- rust-smallstr
- rust-spinning_top
- rust-spmc
- rust-ssh-key-dir
- rust-stb_truetype
- rust-string_cache_shared
- rust-strings
- rust-sudo_plugin
- rust-sxd-document
- rust-synom
- rust-sysctl
- rust-tabwriter
- rust-take
- rust-timerfd
- rust-tower-test
- rust-tower-util
- rust-ucd-util
- rust-unic-ucd-category
- rust-url_serde
- rust-urlocator
- rust-utf8-ranges
- rust-watchman_client
Some of these packages are dependencies of things that will be worked
on at some point in the future (for example, packaging of GStreamer
plugins that are written in Rust), but others look very much like
accumulated cruft.
If you see a package on this list that you would like to keep for some
reason, please speak up, and I will exclude it from future dependency
graph analysis. Otherwise I will soon start retiring packages that
have been unused for over a year.
The packages that would be in the list above, but which I *know* will
get some use soon, are:
- rust-curve25519-dalek
- rust-gstreamer-audio
- rust-gstreamer-editing-services
- rust-gstreamer-player
I will probably start the cleanup process with packages for crates
that no longer have any dependent crates listed on the crates.io
registry (which is a good indicator that they are indeed obsolete),
and then continue with crates for which the longest amount of time has
passed since the last upstream release (which is "more than 5 years
ago" for some crates ...).
Fabio
1 year, 2 months
Web Assembly on Fedora: interested in a Fedora SIG to work on this?
by Michael Dawson
As Web Assembly (WASM) gains momentum we’d like to create a SIG as a place to collaborate to ensure that Fedora is a great platform to both build and run WASM workloads. This includes looking at the toolchains needed to build WASM as a target and the runtimes needed to run WASM. It will provide a place to bring together efforts across different ecosystems (nodejs, rust, compiler toolchains, etc.) as well as a place where people can provide self-help when building and running WASM workloads.
If you are interested in a WASM Sig please let us know.
1 year, 2 months
FESCo wants to know what you use i686 packages for
by David Cantrell
Hi,
Our most recent FESCo meeting involved discussing the proposal to drop i686
builds of jdk8,11,17 from Fedora 37 onward. The topic quickly changed to the
larger question of "what do people use i686 packages for?"
Rather than guess, we wanted to ask the community what you use i686 packages
for in Fedora. There are no wrong answers here. We are seeking information.
Why? Since the removal of the i686 kernel in Fedora, we want to reduce the
number of i686 packages provided in the repo. As time marches on, the ability
to build a lot of things for i686 becomes unrealistic or even impossible.
Remember it goes beyond providing builds...providing support, bug fixes, and
security fixes for those packages too. Maybe some things using i686 packages
now can move to x86_64 packages. We do not know yet, but a goal is to figure
out what packages, if anything, can drop their i686 builds.
NOTE: Nothing is changing now. We are in an information gathering phase.
~~~~~~~~~~~~~~~~~~~~~~~~
If you use i686 packages for something now, please respond to this thread.
Thanks,
--
David Cantrell <dcantrell(a)redhat.com>
Red Hat, Inc. | Boston, MA | EST5EDT
1 year, 3 months
F38 proposal: Reproducible builds: Clamp build mtimes to
$SOURCE_DATE_EPOCH (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/ReproducibleBuildsClampMtimes
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
The `%clamp_mtime_to_source_date_epoch` RPM macro will be set to `1`.
When an RPM package is built, mtimes of packaged files will be clamped
to `$SOURCE_DATE_EPOCH` which is already set to the date of the latest
`%changelog` entry. As a result, more RPM packages will be
reproducible: The actual modification time of files that are e.g.
modified in the `%prep` section or built in the `%build` section will
not be reflected in the resulting RPM packages. Files in RPM packages
will have mtimes that are independent of the time of the actual build.
== Owner ==
* Name: [[User:Churchyard|Miro Hrončok]], [[User:Zbyszek|Zbigniew
Jędrzejewski-Szmek]]
* Email: mhroncok at redhat.com, zbyszek at in.waw.pl
== Detailed Description ==
This change exists to make RPM package builds more reproducible. A
common problem that prevents [https://reproducible-builds.org/ build
reproducibility] is the mtime (modification times) of the packaged
files.
Suppose we package an RPM package of software called `skynet` in
version `1.0`. Upstream released this version at datetime A. A Fedora
packager creates the RPM package at datetime B. Unfortunately, the
packager needs to patch the sources in the RPM `%prep` section. When
the build runs at datetime C, the modification datetime of the patched
file is set to C. When the build runs again in an otherwise identical
environment at datetime D, the modification datetime of the patched
file is set to D. As a result, the build is not bit-by-bit
reproducible, because the datetime of the build is saved in the
resulting package.
Patching is not necessary to make this happen. When a source file is
compiled into a binary file, the modification datetime is also set to
the datetime of the build. In practice, the modification datetime of
many files packaged in RPM packages is dependent on when the package
was actually built.
To eliminate this problem, we propose to clamp build mtimes to
`$SOURCE_DATE_EPOCH`. RPM build in Fedora already sets the
`$SOURCE_DATE_EPOCH` environment variable based on the latest
`%changelog` entry because the `%source_date_epoch_from_changelog`
macro is set to `1`. We will also set the
`%clamp_mtime_to_source_date_epoch` macro to `1`. As a result, when
files are packaged to the RPM package, their modification datetimes
will be clamped to `$SOURCE_DATE_EPOCH` (to the latest changelog entry
datetime). Clamping means that all files which would otherwise have a
modification datetime higher than `$SOURCE_DATE_EPOCH` will have the
modification datetime changed to `$SOURCE_DATE_EPOCH`; files with
mtime lower (or equal) to `$SOURCE_DATE_EPOCH` will retain the
original mtimes.
This functionality is already implemented in RPM. We will enable it by
setting `%clamp_mtime_to_source_date_epoch` to `1`.
=== Non-goal ===
We do not aim to make all Fedora packages reproducible (at least not
as part of this change proposal). We just eliminate one problem that
we consider the biggest blocker for reproducible builds.
=== Python bytecode ===
When Python bytecode cache (a `.pyc` file) is built, the mtime of the
corresponding Python source file (`.py`) is included in it for
invalidation purposes. Since the `.pyc` file is created before RPM
clamps the mtime of the `.py` file, the mtime stored in the `.pyc`
file might be higher than the corresponding mtime of the `.py` file.
With the previous example, if `skynet` is written in Python:
# `skynet.py` is modified in `%prep` and hence has mtime set to the
time of the build
# `skynet.pyc` is generated in `%install` and the mtime of `skynet.py`
is saved in it
# RPM clamps the mtime of `skynet.py`
# `skynet.pyc` is considered invalid by Python on runtime, as the
stored and actual mtime of `skynet.py` don't match
To solve this, we will modify Python to clamp the stored mtime to
`$SOURCE_DATE_EPOCH` as well (when building RPM packages). Upstream
Python chooses to invalidate bytecode cache based on hashes instead of
mtimes when `$SOURCE_DATE_EPOCH` is set, but that could cause
performance issues for big files, so Fedora's Python already deviates
from upstream behavior when building RPM packages. To avoid
accidentally breaking the behavior when
`%clamp_mtime_to_source_date_epoch` is not set to `1`, RPM macros and
buildroot policy scripts for creating the Python bytecode cache will
be modified to unset `$SOURCE_DATE_EPOCH` when
`%clamp_mtime_to_source_date_epoch` is not set to `1`.
This behavior might be proposed upstream if it turns out to be
superior to the current upstream choice, in case we
[https://discuss.python.org/t/14594 won't redesign the bytecode-source
relationship entirely] instead.
=== Opting out ===
Packages broken by this new behavior can unset
`%clamp_mtime_to_source_date_epoch` but packagers are encouraged to
fix the problem instead.
== Feedback ==
Enabling this RPM feature was
[https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/126
proposed as a pull request] to {{package|redhat-rpm-config}} in April
2021. It received good feedback with the exception of the following:
* it was said the change needs to be coordinated with the Python maintainers
* it was said the change should be done via a change process for
better coordination and exposure
We believe that by proposing this via the change process and planning
for the changes needed in Python, both issues are addressed.
== Benefit to Fedora ==
We believe that many RPM packages will become reproducible and others
will be more reproducible than before. The benefits of reproducible
builds are better explained at https://reproducible-builds.org/
== Scope ==
* Proposal owners:
** Propose a PR for {{package|redhat-rpm-config}} (set
`%clamp_mtime_to_source_date_epoch` to `1`, possibly only when
`%source_date_epoch_from_changelog` is set)
** Propose a PR for {{package|python-rpm-macros}} (unset
`$SOURCE_DATE_EPOCH` while creating `.pyc` files iff
`%clamp_mtime_to_source_date_epoch` is not `1`)
** Propose a PR for
[https://src.fedoraproject.org/rpms/python3.11/blob/b2d80045f9/f/00328-pyc...
the Python's bytecode invalidation mode patch] for all Python versions
that have it
** Backport (the new portion of) the patch to older Pythons
({{package|python2.7}}, {{package|python3.6}} and PyPys)
** Test everything together in Copr and deploy it if it works.
** Optional: Run some reproducibility tests before and after this
change and produce some statistics.
* Other developers:
** Test their packages with the new behavior, report problems, and
opt-out if really needed.
* Release engineering: N/A (not needed for this Change)
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives: N/A (not needed for this Change)
== Upgrade/compatibility impact ==
Nothing anticipated.
== How To Test ==
The change owners plan to perform a mass rebuild in Copr to see if
this breaks anything significantly.
If it actually works as anticipated, they also plan to run some
reproducibility tests and hopefully produce some statistics before and
after this change.
Other packages can test by building their packages and verifying they
still work as expected and no packaged files have higher mtimes than
the last `%changelog` entry.
To verify if this change has landed, run: `rpm --eval
'%clamp_mtime_to_source_date_epoch'` on Fedora 38. The result should
be `1`.
== User Experience ==
Users of Fedora Linux on their machines should not be impacted at all.
Users who build RPM packages atop Fedora will be impacted by this
change the same way Fedora is.
== Dependencies ==
* RPM needs to support this (it already does)
* RPM needs to set `$SOURCE_DATE_EPOCH` (it already does)
== Contingency Plan ==
* Contingency mechanism: The change owners or
{{package|redhat-rpm-config}} maintainers or proven packagers will
revert the change in {{package|redhat-rpm-config}}. That should be
enough to undo anything as the changes in Python should be dependent
on that. If not enough, revert everything.
* Contingency deadline: Ideally, we should do this before the Mass
Rebuild. Technically, we can land it any time before the Beta Freeze,
but it would not change all the packages, which is a bit messy. *
Blocks release? No <
== Documentation ==
This page is the documentation.
== Release Notes ==
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
1 year, 3 months
F39 proposal: Replace DNF with DNF5 (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/ReplaceDnfWithDnf5
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
Make DNF5 the new default packaging tool. The change will replace DNF,
LIBDNF, and DNF-AUTOMATIC with the new DNF5 and new Libdnf5 library.
It is a second step after
https://fedoraproject.org/wiki/Changes/MajorUpgradeOfMicrodnf.
== Owner ==
* Name: [[User:jmracek| Jaroslav Mracek]]
* Email: jmracek(a)redhat.com
== Detailed Description ==
The new DNF5 will provide a significant improvement in user
experiences and performance. The replacement is the second step in
upgrade of Fedora Software Management stack. Without the change there
will be multiple software management tool (DNF5, old Microdnf,
PackageKit, and DNF) based on different libraries (libdnf, libdnf5),
providing a different behavior, and not sharing a history. We can also
expect that DNF will have only limited support from upstream. The DNF5
development was announced on
[https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.o...
Fedora-Devel] list in 2020.
=== New DNF5 Features ===
* Fully featured package manager without requirement of Python
** Smaller system
** Faster
** Replace DNF and Microdnf
* Unified behavior of in the software management stack
** New Libdnf5 plugins (C++, Python) will be applicable to DNF5, Dnf5Daemon
*** DNF4 plugins were not applicable for PackageKit and Microdnf (e.g.
versionlock, subscription-manager), therefore PackageKit behaves
differently in comparison to DNF
** Shared configurations
*** In DNF4 not all configuration is honored by PackageKit and Microdnf
** DNF/YUM was developed for decades with impact of multiple styles
and naming conventions (options, configuration, options, commands)
* New Daemon
** The new daemon can provide an alternative to PackageKit for RPMs
(only one backend of PackageKit) if it will be integrated into Desktop
** Support of Modularity and Comps group
* Performance improvement
** Loading of repositories
** Advisory operations
** RPM queries
*** Name filters with a case-insensitive search (the `repoquery` command)
** Smart sharing of metadata between dnf5 and daemon
*** Reduce disk and downloads requirements
*** Currently, DNF, Microdnf, and PackageKit use their own cache
*** Optional, may be not available for Fedora 39
* Decrease of a maintenance cost in the long term
** Shared plugins
** Removal of functional duplicates
* Fully integrated Modularity in LIBDNF5 workflows
** The Modularity is supported in DNF and LIBDNF but it is not fully
integrated. Integration was not possible due to limitation of
compatibility with other tools (PackageKit)
** Fully integrated Modularity required changes in the library workflow
=== Major codebase improvements ===
*Reports in structure
** DNF reports a lot of important information only in logs
* Removal of duplicated implementation
** LIBDNF evolved from LIBHIF (PackageKit library) and HAWKEY (DNF
library). The integration was never finished, therefore LIBDNF still
contains duplicated functionality.
** decrease of the code maintenance cost in future
* Unify Python bindings
** Formal Libdnf provides two types of Python bindings
*** CPython (hawkey)
*** SWIG (libdnf)
** Maintaining and communication between both bindings requires a lot
of resources
** Binding unification was not possible without breaking API compatibility
* SWIG bindings
** With SWIG we can generate additional bindings without spending huge resources
** Code in particular languages will be very similar to each other
* Separation of system state from history DB and `/etc/dnf/module.d`
** In dnf-4 the list of userinstalled packages and list of installed
groups along with the lists of packages installed from them is
computed as an aggregation of transaction history. In dnf5 it will be
stored separately, having multiple benefits, among them that the
history database will serve for informational purposes only and will
not define the state of the system (it gets corrupted occasionally
etc.).
** Data stored in `/etc/dnf/module.d` were not supposed to be user
modifiable and their format is not sufficient (missing information
about installed packages with installed profiles)
*** Content of `/etc/dnf/module.d` will be moved into the System State
== Feedback ==
== Benefit to Fedora ==
== Scope ==
* Proposal owners:
DNF5 is still in the development and some of the features or options
are not yet available. We still have to finish the implementation of
Modularity, storing internal data related to History and System State,
and also documentation and man pages. DNF5 can be tested from
repository with upstream nightly builds -
https://copr.fedorainfracloud.org/coprs/rpmsoftwaremanagement/dnf5-unstable/.
The project's github repository is here -
https://github.com/rpm-software-management/dnf5/
* Other developers:
* Release engineering: [https://pagure.io/releng/issues #Releng issue number]
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives:
== Upgrade/compatibility impact ==
The new DNF5 will obsolete `dnf`, `yum`, `dnf-automatic`, `yum-utils`,
and DNF plugins (core and extras). python3-dnf and LIBDNF (`libdnf`,
`python3-hawkey`) will be obsoleted by `fedora-obsolete-packages`.
=== Compatibility ===
The new DNF5 will provide a symlink to `/usr/bin/dnf` therefore users
will see the replacement as an upgrade of DNF with limited but
documented syntax changes. The DNF5 will provide some compatible
aliases of commands and options to improve adoption of the DNF5.
== How To Test ==
Install `dnf5` package from
https://copr.fedorainfracloud.org/coprs/rpmsoftwaremanagement/dnf5-unstable/
== User Experience ==
* Improved progress bars
* Improved transaction table
* Transaction progress reports including scriptlets reports
* Support of local rpm for transaction operation
* Great bash completion (better then DNF has)
* New commands and options that are only available with `DNF`
== Dependencies ==
There is a long list of dependent packages
=== dnf ===
auter
calamares
copr-builder
cpanspec
dnf-plugin-diff
dnfdragora
etckeeper-dnf
fedora-review
fedora-upgrade
kiwi-systemdeps-core
libdnf-plugin-subscription-manager
lpf
mock
osbuild
perl-CPAN-Plugin-Sysdeps
policycoreutils-devel
rbm
subscription-manager
supermin
system-config-language
=== python3-dnf ===
anaconda-core
dnf-plugin-ovl
dnfdaemon
fedora-easy-karma
fedora-review
lorax
mock-core-configs
module-build-service
modulemd-tools
needrestart
pungi
python3-bodhi-client
python3-dnf-plugin-cow
python3-dnf-plugin-flunk_dependent_remove
python3-imgcreate
python3-libreport
retrace-server
system-config-language
=== libdnf ===
PackageKit
copr-builder
gnome-software-rpm-ostree
libdnf-plugin-subscription-manager
libdnf-plugin-swidtags
libdnf-plugin-txnupd
=== python3-hawkey ===
mock-core-configs
modulemd-tools
python3-rpmdeplint
retrace-server
== Contingency Plan ==
* Contingency mechanism: (What to do? Who will do it?)
* Contingency deadline:
* Blocks release?
== Documentation ==
none
== Release Notes ==
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
1 year, 3 months
