devel June 2024

devel@lists.fedoraproject.org

152 participants
187 discussions

Execute RPM dependency generators on the .spec file which ships them
by Vít Ondruch 27 Sep '24

27 Sep '24

I am still playing with idea of executing RPM dependency generators on the .spec file which ships them. I have created RPM RFE ticket quite a while ago here: https://github.com/rpm-software-management/rpm/issues/782 So for no luck with having this implemented. However, I think there is reasonable way. It would be enough to have some generic attr file around. I have create small test package exploring this idea and you can find it in my Copr repo: https://copr.fedorainfracloud.org/coprs/vondruch/rpm-local-generator/ https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generato… On the same place, you can also find package leveraging this concept: https://copr-dist-git.fedorainfracloud.org/cgit/vondruch/rpm-local-generato… Would somebody else benefit from this idea/package or can somebody spot some issue with it? Thx for feedback. Vít

5 13

gdk-pixbuf removing several icon loaders
by Michael Catanzaro 14 Sep '24

14 Sep '24

Hi, gdk-pixbuf 2.42.11 has dropped support for several uncommon image formats. This is causing several applications to crash in Fedora rawhide [1][2]. (The change also got backported to F40 and F39, but I've reverted it there.) Benjamin Gilbert has proposed reenabling the removed loaders [3], but this is not likely to be accepted upstream. So he's currently planning to package the removed loaders for Fedora in a separate package. You'll be able to depend on these if needed to avoid crashing, but please do so only if you really need to, since the goal of removing the extra loaders is to reduce attack surface. (Unfortunately gdk-pixbuf is a fairly risky dependency: many applications require it, but it's not very safe.) Most applications should use modern image formats instead. Michael [1] https://bugzilla.redhat.com/show_bug.cgi?id=2276464 [2] https://bugzilla.redhat.com/show_bug.cgi?id=2276661 [3] https://gitlab.gnome.org/GNOME/gdk-pixbuf/-/merge_requests/169 [4] https://src.fedoraproject.org/rpms/gdk-pixbuf2/pull-request/4#comment-198909

4 6

GRUB2 rebase (from 2.06 to 2.12) landing soon on rawhide
by Leo Sandoval 04 Sep '24

04 Sep '24

Hi team, We (the Red Hat bootloader team) are completing the work of rebasing/reviewing/testing current rawhide patches, from GRUB2 2.06 to 2.12, so at some point in the near future these would land finally into rawhide. Once everything is in place, we would like your help to start testing the package and report any boot issue found. If there is any concern on this work, please let me know. Regards, Leo

6 16

SPDX Statistics - Hulk edition
by Miroslav Suchý 29 Aug '24

29 Aug '24

Hot news: SPDX v3 has been published. The biggest change for us is that license expression allows lowercase operators (and, or, with). This got into the specification because of your (Fedora maintainers) feedback! And there is new terminal AdditionRef-* which can be used for custom exceptions. We have no use for this now. It will take some time untill a tooling will accomodate this change. For fedora-license-data you can track it in https://gitlab.com/fedora/legal/fedora-license-data/-/issues/514 Two weeks ago we had: > * 23943spec files in Fedora > > * 30600license tags in all spec files > > * 10639 tags have not been converted to SPDX yet > > * 4689 tags can be trivially converted using `license-fedora2spdx` > > * Progress: 65,23% ░░░░░░████ 100% > > ELN subset: > > 94 out of 2394 packages are not converted yet (progress 96.07%) > Today we have: * 23990spec files in Fedora * 30640license tags in all spec files * 10589 tags have not been converted to SPDX yet * 4656 tags can be trivially converted using `license-fedora2spdx` * Progress: 65,44% ░░░░░░████ 100% ELN subset: 90 out of 2394 packages are not converted yet (progress 96.23%) Graph of these data with the burndown chart: https://docs.google.com/spreadsheets/d/1QVMEzXWML-6_Mrlln02axFAaRKCQ8zE807r… The list of packages needed to be converted is here: https://pagure.io/copr/license-validate/blob/main/f/packages-without-spdx-f… List by package maintainers is here https://pagure.io/copr/license-validate/blob/main/f/packages-without-spdx-f… List of packages from ELN subset that needs to be converted: https://pagure.io/copr/license-validate/blob/main/f/eln-not-migrated.txt No license has been added to fedora-license-data. Therefore no release and no update of documentation. New projection when we will be finished is 2025-04-25 (+19 days from last report). Pure linear approximation. If your package does not have neither git-log entry nor spec-changelog entry mentioning SPDX and you know your license tag matches SPDX formula, you can put your package on ignore list https://pagure.io/copr/license-validate/blob/main/f/ignore-packages.txt Either pull-request or direct email to me is fine. Why Hulk? On today's date at 1962 Marvel published first comics about The Hulk. Stan Lee (the writer) stated that the Hulk's creation was inspired by a combination of Frankenstein and Dr. Jekyll and Mr. Hyde. https://en.wikipedia.org/wiki/Hulk http://tonsoffacts.com/32-fun-interesting-facts-hulk/ Miroslav

8 11

Heads-up: llhttp 9.0.1 coming to Rawhide
by Ben Beasley 18 Aug '24

18 Aug '24

In one week, 2023-08-29, I plan to update llhttp from 8.1.1 to 9.0.1 in F40/Rawhide[1]. This update contains some breaking API changes and breaks the ABI, bumping the SONAME version[2][3]. Most significantly, the compile-time LLHTTP_STRICT_MODE option is replaced with three new runtime flags. The sole dependent package is python-aiohttp; I will rebuild it myself in a side tag, using a patch for llhttp 9.x support backported from an unreleased upstream commit[4]. I don’t plan to apply the python-aiohttp patch and the llhttp 9.0.1 update to F39 at this time, with the understanding that this will constrain future python-aiohttp updates in F39: I would prefer to avoid getting ahead of upstream. I might reconsider this if an upstream release of python-aiohttp using llhttp 9.x happens by the end of the beta freeze, or if someone convinces me that keeping llhttp at 8.1.1 is the wrong approach. Input is welcome. [1] https://src.fedoraproject.org/rpms/llhttp/pull-request/15 [2] https://github.com/nodejs/llhttp/releases/tag/release%2Fv9.0.0 [3] https://github.com/nodejs/llhttp/releases/tag/release%2Fv9.0.1 [4] https://src.fedoraproject.org/rpms/python-aiohttp/pull-request/27

1 1

Porting Fedora for the LoongArch architecture.
by 孙海勇 17 Aug '24

17 Aug '24

Hi everyone, I am Sun Haiyong, from China. I want to port Fedora for the LoongArch architecture. LoongArch is a RISC ISA released by Loongson Technology Corporation Limited, and has supported a series of (Binutils, GCC, Linux, Glibc, LLVM, QEMU, etc.) core open source projects. Currently, there are many linux distributions that can run on LoongArch machines, they are OpenEuler, OpenAnolis, UOS, Kylin. I am good at cross-compiling operating systems and often build Linux systems using something like LFS or CLFS. I have built Linux distributions using rpm package management from scratch several times since 2015 (some systems are not publicly available): 1 Fedora 21, 28, 32 based on MIPS64EL architecture; 2 CentOS 7 based on MIPS64EL architecture; 3 CentOS 7 based on Power8 architecture; 4 CentOS 8.3 based on LoongArch architecture; 5 OpenEuler 2109 based on LoongArch architecture. And I have published a book on porting Fedora systems to new architectures. I want to add LoongArch to the official Fedora support architecture, and I've been doing so for some time, here's some of what I've done so far： To verify the feasibility of building a LoongArch architecture branch for Fedora, I have used the software version from the rawhide git repository, and have now compiled a large number of base packages and built a temporary repository that can be accessed at https://mirrors.wsyu.edu.cn/fedora/ I have compiled and generated more than 45,000 installable rpm files (of course there are a lot of perl, Python, rust and texlive files), and the number is still expanding, the scope of the package is enough to build a LiveCD system, for which I have built LXDE, MATE, WorkStation ( Gnome3) of the LiveCD and the installation of the ISO, you can get in the following address: https://github.com/fedora-remix-loongarch/releases-info Of course, there are still a lot of problems with LoongArch's Fedora system, for example, some software is not yet fully supported by the upstream community, but I believe the power of the community can gradually improve them, so I am sending out an email here to get more people to support this new LoongArch architecture. I have recruited some developers who are interested in this and they are： Wu Xiaotian Chen Huacai Shi Pujin Si Yanteng Chen Feiyang Of course, there are many other users who are interested in Fedora systems. I'm currently a newbie in the Fedora community, so I need help from community developers, and would like someone to guide me on what to do next, such as what would be a better time to submit necessary patches to packages in the Fedora repository, how to develop in a collaborative manner, what other systems to be used for management, etc. In short, any information would be useful. Could I get help here? :) Again, thanks for reading this email. Best Regards Sun Haiyong

12 18

F41 Change Proposal: Switch to DNF5 (system-wide)
by Aoife Moloney 14 Aug '24

14 Aug '24

Wiki - https://fedoraproject.org/wiki/Changes/SwitchToDnf5 This is a proposed Change for Fedora Linux. This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == Change the default package manager from dnf to dnf5. == Owner == * Name: [[User:jkolarik| Jan Kolarik]] * Email: jkolarik(a)redhat.com * Name: [[User:jmracek| Jaroslav Mracek]] * Email: jmracek(a)redhat.com == Detailed Description == This proposal will implement several topics, which are outlined below. === Provider of the dnf command === This change proposes to switch the current provider of the /usr/bin/dnf symlink from dnf to dnf5. Currently, the symlink target is /usr/bin/dnf-3, provided by the dnf sub-package, python3-dnf. Upon implementation of this change, the symlink will point to /usr/bin/dnf5, provided by the dnf5 package. === Prepare the upgrade path === The dnf5 package, serving as the new provider of the /usr/bin/dnf symlink, will obsolete the dnf package starting with Fedora 41. Upon the release of this dnf5 package, upgrading the system or installing dnf5 will replace the existing dnf package on the system. Additionally, the dnf5 package will provide a /usr/bin/yum symlink for backwards compatibility and the dnf-automatic command will be obsoleted. === Feature parity with dnf === We aim to cover the majority of use cases available in the existing dnf package. However, there are some features that may not be implemented in time. Nevertheless, we plan to deliver them at a later stage. ==== Plugins ==== The progress of implementing plugins to match the current set from the dnf-plugins-core package is tracked [https://github.com/rpm-software-management/dnf5/issues/389 upstream]. Among the missing plugins, we still plan to implement: * debuginfo-install plugin * reposync plugin ==== Modularity ==== As support for modularity was retired in Fedora 39, dnf5 currently only implements a basic feature set for listing and enabling/disabling modules. === Background service support === A new daemonized service, dnf5daemon, utilizing the D-Bus interface, is prepared for clients as a sub-package. This will serve as an alternative or replacement for the PackageKit layer. Integration of dnf5daemon support into the default Fedora user interface, GNOME Software, is currently in progress === Documentation of API changes === The public interface has undergone significant changes to enhance the user experience and remove unused and obsolete code components. To facilitate user migration to the new CLI and API interfaces, a [https://dnf5.readthedocs.io/en/latest/changes.html guide] was prepared covering all differences compared to the interface provided by the existing dnf package, along with examples of typical use cases. === Deployment tasks === During the deployment of the dnf5 package manager as the new default, several adjustments need to be made both to the infrastructure and the dnf5 package itself. Some of these adjustments are detailed [[#Release engineering|below]]. To ensure synchronization and address all necessary changes, we've established an upstream tracking [https://github.com/rpm-software-management/dnf5/issues/1057 issue]. == Feedback == As this is the second iteration of such a proposal, we've gathered a lot of feedback from various sources during the first attempt to accept this change. === FESCo inputs === A [https://pagure.io/fesco/issue/3039 ticket] discussing the reasons why the contingency mechanism was invoked for the first attempt of the proposal was opened by FESCo. It includes a list of items that are either incomplete or in progress. Below is a list of issues from the ticket that are still unresolved or require clarification on their current status. Other items not mentioned below are considered completed. ==== Switch in ELN ==== This should not block the proposal, as the current plan is to target RHEL 11. Integration can occur there after the proposal is implemented for Fedora 41. ==== Aligning configuration with the current state in dnf ==== All overrides to match the current state of dnf configuration will be provided to the Fedora release project, see [[#Apply downstream configuration overrides|below]]. ==== System upgrade and offline transactions ==== The implementation work has been completed and is already present upstream. We anticipate extensive testing during the summer, and we also plan to organize testing days for this purpose. ==== Dropping the Snapper plugin ==== In dnf5, we've adopted a new approach for implementing functionality that was previously handled by the Snapper plugin in dnf. We're introducing the Actions plugin, which offers more capabilities than the Snapper plugin, including support for running external applications before or after transactions and interacting with the dnf5 configuration. [https://dnf5.readthedocs.io/en/latest/libdnf5_plugins/actions.8.html Here] is the documentation for the Actions plugin, which includes examples of how to emulate the behavior of the Snapper plugin. ==== Messages from RPM scriptlets ==== An issue with output from RPM scriptlets is the potential length, coupled with the absence of a standardized policy for distinguishing between important and unimportant messages. Currently, all messages are logged in the dnf5 log files, with differentiation based on their originating scriptlets, representing an improvement over dnf. Additionally, in case of transaction errors, the scriptlet output is included in the standard output. Furthermore, there is already a resolved [https://pagure.io/koji/issue/4009 ticket] in the infrastructure to incorporate logs on the builders. === Testing days === We've already held several dnf5 test days for Fedora [https://fedoraproject.org/wiki/Test_Day:2023-03-14_Fedora_38_DNF_5 38], [https://fedoraproject.org/wiki/Test_Day:2023-08-11_Fedora_39_DNF_5 39], and recently also [https://fedoraproject.org/wiki/Test_Day:2024-03-15_Fedora_40_DNF_5 40]. We've made efforts to document all reported issues in our upstream tracking system, and major issues should now be resolved. Some of these issues were related to user documentation, improving command-line outputs, and enhancing overall user experience. These topics are next on our priority list after completing the functionality for mandatory commands and plugins. === Fedora QA scenarios === We've started a discussion [https://discussion.fedoraproject.org/t/requirements-for-dnf5-in-fedora-41 thread] on the requirements for accepting this proposal from a QA perspective. A list of relevant test cases and criteria has been mentioned, which we'll review to ensure we've covered everything on our end. === Fedora CI readiness === The dnf project is also deployed in the CI pipeline. We've initiated communications with this team to ensure that all dnf functionality used there is either already implemented in dnf5 or can be addressed through an alternative dnf5 method. We've already received some feedback from the first iteration of the proposal. === Tracking issue upstream === When implementing the first iteration of this proposal, we created an upstream [https://github.com/rpm-software-management/dnf5/issues/635 ticket] to track all bugs or lack of needed functionality. All items have been addressed, and only several known deployment issues remain, which need to be managed at the time of the next switch. == Benefit to Fedora == The new dnf5 will significantly improve the user experience and performance. Detailed descriptions of individual areas are provided below. === Reduced footprint === The dnf5 package is a fully-featured package manager that doesn't require Python dependencies. It also reduces the number of software management tools in Fedora by replacing both the dnf and microdnf packages. The installation size of the dnf5 stack in an empty container is approximately 60% smaller than the dnf installation. Currently, dnf, microdnf, and PackageKit use their own cache, leading to significant metadata redundancy. With dnf5 and dnf5daemon, which share metadata, this redundancy will be eliminated. === Enhanced performance === Loading and downloading repository metadata now occur concurrently. Package query operations, including processing numerous command-line arguments, have been significantly accelerated. === Lowered maintenance costs === Many functional duplicates in dnf were eliminated during the development of the new dnf5 package manager. This was partly because the integration of the original PackageKit and dnf libraries into the original libdnf library was never completed. Plugins are now included in the same package as the core functionality. === Unified user experience === Consistent user experience is offered to users across servers, workstations, and containers, as dnf5 is the sole package manager deployed there. Existing dnf, yum, and microdnf commands will be linked to dnf5, while compatibility aliases for essential use cases will be provided to facilitate migration. Configuration files will be shared among dnf5 components. API users will encounter unified code style and naming conventions. Various scripting language interfaces are now provided from a single source using SWIG bindings (formerly CPython and SWIG). == Scope == === Proposal owners === The remaining work on the proposal can be divided into several sections. ==== Feature implementation ==== ===== [https://github.com/rpm-software-management/dnf5/issues/1052 System upgrade] ===== This command is essential for upgrading the system to the next release. While the implementation is already completed, we plan to conduct extensive testing, including community participation, to minimize the risk of issues occurring in production. ===== [https://github.com/rpm-software-management/dnf5/issues/140 History command] ===== The functionality related to manipulating transaction history has not yet been implemented. However, following the completion of the system upgrade functionality, it is currently our top priority. Due to the significant overlap between the functionality in the history command and the system upgrade functionality, we anticipate its readiness shortly thereafter. ===== [https://github.com/rpm-software-management/dnf5/issues/169 GNOME Software support] ===== The integration of dnf5 support, particularly dnf5daemon, into GNOME Software is currently underway. Developers from both DNF5 and GNOME Software are closely connected and regularly synchronize the progress of their work. ==== Documentation ==== While our current priority is achieving full coverage of [https://dnf5.readthedocs.io/en/latest/ user and API documentation], there may still be some undocumented parts of the code. Please don't hesitate to [https://github.com/rpm-software-management/dnf5/issues/new report] any such issues upstream, and we'll endeavor to address them promptly. ==== Early access for developmental branch users ==== The intention is to implement the proposed changes in the Rawhide developmental branch prior to the date specified in the Contingency Deadline section, targeting the transition period between March and April. It is anticipated that certain mandatory items for the regular release, as outlined in previous sections, may not be completed by this time. ===== System upgrade ===== This functionality is unnecessary as Rawhide operates on a rolling release model. ===== GNOME Software ===== Rawhide users will continue to utilize the current PackageKit backend connected to the existing libdnf interface until the integration of dnf5 is finalized. These libraries can coexist with the new dnf5 package on the same system. For more details, see [[#Different system state|below]]. === Other developers === The following components are already prepared for transition to dnf5: * [https://github.com/ansible/ansible/issues/78898 Ansible] * [https://github.com/rpm-software-management/dnf5/issues/66 Lorax] * [https://github.com/rpm-software-management/mock/issues/894 Mock] Below is a list of dependencies that have not yet integrated with dnf5. ==== Anaconda ==== Migration to the dnf5 API will not be implemented at this time. After discussion with the Anaconda team, it was determined that there isn't sufficient capacity and time to complete the work on schedule. Therefore, the existing dnf4 Python bindings from the python3-dnf package will continue to be used for now. ==== GNOME Software ==== This integration is already addressed in the section above, as it involves collaborative efforts. === Release engineering === ==== Building with dnf5 ==== The Fedora infrastructure has been utilizing dnf5 for building Fedora 40+ chroots since before the Fedora 40 mass rebuilds began. This implementation was based on the system-wide change outlined [https://fedoraproject.org/wiki/Changes/BuildWithDNF5 here]. ==== Apply downstream configuration overrides ==== Starting with dnf5, distro-specific overrides of the default configuration values are implemented using [https://dnf5.readthedocs.io/en/latest/dnf5.conf.5.html#drop-in-configuratio… drop-in directories]. Options with different values upstream compared to the current state of dnf in Fedora will be filed in a pull request against the [https://src.fedoraproject.org/rpms/fedora-release Fedora release project]. ==== Update of kickstarts for image composes ==== The dnf team will prepare a pull request to include the packages for the new default dnf5 package manager in the [https://pagure.io/fedora-kickstarts Fedora Kickstarts project]. ==== Update of package groups definitions ==== The dnf team will prepare a pull request to include the packages for the new default dnf5 package manager in the [https://pagure.io/fedora-comps Fedora Comps project]. ==== Update of KIWI image descriptions ==== The dnf team will prepare a pull request to include the packages for the new default dnf5 package manager in the [https://pagure.io/fedora-kiwi-descriptions Fedora KIWI descriptions project]. == Upgrade/compatibility impact == === Running the upgrade === The dnf5 package will be installed on the system to provide /usr/bin/dnf, replacing the existing dnf package as it becomes obsolete. Additionally, the dnf automatic tool will be replaced by the new dnf5 automatic plugin. Below is an example of performing this upgrade transaction. <pre> $ sudo dnf upgrade Last metadata expiration check: 0:01:09 ago on Wed 13 Mar 2024 05:48:25 AM EDT. Dependencies resolved. ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: dnf5-plugin-automatic x86_64 5.1.14-1.fc41 rawhide 120 k replacing dnf-automatic.noarch 4.19.0-1.fc40 Upgrading: dnf-data noarch 4.19.0-1.fc41 rawhide 40 k python3-dnf noarch 4.19.0-1.fc41 rawhide 551 k Installing group/module packages: dnf5 x86_64 5.1.14-1.fc41 rawhide 599 k replacing dnf.noarch 4.19.0-1.fc40 replacing yum.noarch 4.19.0-1.fc40 Installing dependencies: fmt x86_64 10.2.1-3.fc40 rawhide 125 k libdnf5 x86_64 5.1.14-1.fc41 rawhide 991 k libdnf5-cli x86_64 5.1.14-1.fc41 rawhide 229 k Installing weak dependencies: bash-completion noarch 1:2.11-15.fc41 rawhide 360 k Transaction Summary ================================================================================ Install 6 Packages Upgrade 2 Packages </pre> === Binaries and symlinks === The existing dnf binaries will remain on the system and will be available at /usr/bin/dnf-3, which refers to the Python 3 version to which dnf was migrated in the past. Additionally, they will be accessible at /usr/bin/dnf4, which is a symlink to the dnf-3 script and denotes the major version of the dnf binary. This binary naming convention is also used for /usr/bin/dnf5, which will become the target of the /usr/bin/dnf symlink. Below is the output of the tree command showing the expected file structure after the upgrade. ==== Before upgrade ==== <pre> $ tree /usr/bin/ -P dnf* /usr/bin/ ├── dnf -> dnf-3 ├── dnf-3 └── dnf4 -> dnf-3 </pre> ==== After upgrade ==== <pre> $ tree /usr/bin/ -P dnf* /usr/bin/ ├── dnf -> dnf5 ├── dnf-3 ├── dnf4 -> dnf-3 └── dnf5 </pre> === Different system state === Though the RPM DB, which contains the database of installed packages, remains the singular source in the system, the transactional history and installed packages reasons in dnf and dnf5 is not shared, and they now use different formats. Transactions performed in dnf will not be visible in dnf5, and vice versa. Therefore, when concurrently using dnf and dnf5 on the system, packages installed by one of them as dependencies will appear as user-installed to the other one, potentially leading to them not being auto-removed later. While the history database is not migrated to dnf5, when running a transaction in dnf5 for the first time, an attempt is made to convert and load the existing system state from dnf. This should preserve information about the reasons for installed packages and prevent them from being treated as user-installed, requiring manual removal from the system instead of being seen as dependencies of explicitly removed packages. === Compatibility === While the majority of CLI use cases for managing packages should remain the same, the dnf5 API has undergone significant changes. To ease adoption, dnf5 will provide compatibility aliases for commands and options. Additionally, user output will differ, and we plan to offer machine-readable support for most commands. For more details, please refer to the section above. Applications encountering difficulties with dnf5 adoption can continue using the existing dnf CLI and API provided by the python3-dnf and libdnf packages. These libraries can be used in parallel on the system, but modifying installed software is not recommended due to differences in system state, as mentioned in the [[#Documentation of API changes|previous]] section. == How To Test == === Copr repository === [https://copr.fedorainfracloud.org/coprs/rpmsoftwaremanagement/dnf5-testing/ A testing Copr repository] has been set up with dnf5 already deployed as the default package manager. Instructions on how to proceed are provided alongside. === Quay containers === We have also prepared Fedora container images with the dnf5 stack preinstalled on [https://quay.io/repository/rpmsoftwaremanagement/fedora-dnf5 quay.io], making it easy to test in isolation with Podman. === Side-tag for testing === Before pushing the new dnf5 into the Rawhide compose, we plan to announce the prepared side-tag containing this new package publicly. This will allow interested parties to test it against their software. === Testing days === We are planning at least two iterations of testing days before dnf5 is delivered into Fedora 41. One will focus on testing the overall functionality of dnf5, with an emphasis on parts that were not tested during previous testing days. The other iteration will center around testing the system upgrade functionality. === Communication channels === Community feedback, including bug reports, issues, or feature requests, is highly encouraged. Our primary communication channel is [https://github.com/rpm-software-management/dnf5 upstream], where you can report [https://github.com/rpm-software-management/dnf5/issues issues], participate in [https://github.com/rpm-software-management/dnf5/discussions discussions], or even propose [https://github.com/rpm-software-management/dnf5/pulls pull requests]. We are happy to review them. Issues specific to a particular version of the dnf5 package can also be reported through the [https://bugzilla.redhat.com/enter_bug.cgi?product=Fedora&component=dnf5 Bugzilla] tracking system, which we also monitor. == User Experience == === Faster query processing === The processing of package metadata is now significantly faster. Executing commands such as repoquery to list packages available in repositories is now twice as fast compared to dnf. Similarly, operations like listing dependencies or parsing numerous command-line arguments are notably expedited, potentially saving users seconds to tens of seconds in waiting time for the results. === Consolidated and streamlined API === The API for managing packages, working with repositories, and solving package dependencies is now consolidated into a single component, providing a unified solution. The original dnf API underwent a review process, during which unused workflows and obsolete methods were removed, while improving usability for users. === Enhanced command-line outputs === Transaction tables now offer more detailed information, verbose scriptlet outputs are redirected and organized by package name into log files, individual commands come with their own man pages, bash completion has been enhanced, and numerous other improvements have been made. == Dependencies == === Owned by our team === ==== dnf-plugins-core ==== Installed plugins will persist on the system and remain functional using the dnf4 binary from the python3-dnf package. With the exception of the system upgrade plugin, all essential plugins are now implemented in dnf5 and are provided by the dnf5-plugins package or dnf5 directly. ==== dnf-plugins-extras ==== Installed plugins will persist on the system and remain functional using the dnf4 binary from the python3-dnf package. Porting the functionality to dnf5 is currently of low priority, and its implementation depends more on community involvement. The functionality of the Snapper plugin is already covered by the dnf5 actions plugin. === Requirements on dnf === This is the most critical category of packages as issues connected with them can potentially disrupt the system upgrade path. Here, we can split components into two groups. The first group utilizes dnf from the command line interface (CLI), and thus, they either need to adjust to the new syntax and behavior of dnf5 or switch to utilizing the existing dnf4 binary directly. The second group consists of components providing plugins for the dnf command. These plugins will remain functional using the binary from python3-dnf, requiring packaging changes to depend on this package instead. <pre> auter calamares copr-builder cpanspec dnfdragora etckeeper-dnf fedora-review fedora-upgrade kiwi-systemdeps-core libdnf-plugin-subscription-manager lpf mock osbuild perl-CPAN-Plugin-Sysdeps rbm rpmdistro-repoquery supermin system-config-language </pre> A tracking [https://github.com/rpm-software-management/dnf5/issues/418 issue] was created in advance to inform maintainers about the planned switch to dnf5. === Requirements on python3-dnf === Here, we can also categorize components into two groups. The first group consists of components providing plugins for the dnf command. These plugins will remain functional using the binary from python3-dnf, requiring packaging changes to depend on this package instead. The second group utilizes the dnf’s Python API, and they should not be directly affected by the change. However, testing is still necessary, and it is strongly recommended to consider porting to the dnf5 API. <pre> anaconda-core copr-builder dnf-plugin-diff dnf-plugin-ovl dnfdaemon fedora-easy-karma fedora-review fedrq lorax mock-core-configs module-build-service modulemd-tools needrestart policycoreutils-devel pungi python3-dnf-plugin-cow python3-dnf-plugin-flunk_dependent_remove python3-dnf-plugin-perfmetrics python3-imgcreate python3-libreport retrace-server subscription-manager system-config-language </pre> === Requirements on libdnf === These packages utilize the libdnf API and should not be directly impacted by the change. However, testing is still necessary. Tools that modify system software, such as PackageKit, may exhibit different behavior when used alongside dnf5 to manage the same system, as previously described. Additionally, it is strongly recommended to consider porting to the dnf5 API in this context. <pre> PackageKit copr-builder libdnf-plugin-subscription-manager libdnf-plugin-swidtags libdnf-plugin-txnupd </pre> === Requirements on python3-hawkey === These components utilize unsupported Hawkey Python bindings and should not be directly impacted by the change. However, thorough testing is necessary. Again, it is strongly recommended to consider porting to the dnf5 API in this context. <pre> mock-core-configs modulemd-tools python3-rpmdeplint retrace-server </pre> == Contingency Plan == === Contingency mechanism === To revert to the previous state with dnf as the default package manager on the system, the following steps would be necessary: * Packaging changes in dnf5 to remove the obsoletion of dnf and provider of the /usr/bin/dnf symlink. * Untagging the candidate dnf5 package from the compose. * Components that adapted to the dnf5 CLI must synchronize in this process and revert the changes to use dnf4 again. Proactive communication will be conducted similarly to how components were informed about the dnf5 migration. === Contingency deadline === Branch Fedora Linux 41 from Rawhide === Blocks release? === No == Documentation == === [https://dnf5.readthedocs.io/en/latest/index.html Official documentation] === === [https://github.com/rpm-software-management/dnf5 Upstream project] === === [https://dnf5.readthedocs.io/en/latest/changes.html Changes between dnf and dnf5] === === [https://dnf5.readthedocs.io/en/latest/dnf_daemon/dnf5daemon_dbus_api.8.html D-Bus API documentation] === === [https://github.com/rpm-software-management/dnf5/blob/main/CONTRIBUTING.md Contributing guide for developers] === == Release Notes == -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney -- _______________________________________________ devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org To unsubscribe send an email to devel-announce-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedorapr… Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

8 10

F41 Change Proposal - Python Built with gcc -03 (self-contained)
by Aoife Moloney 13 Aug '24

13 Aug '24

Wiki - https://fedoraproject.org/wiki/Changes/Python_built_with_gcc_O3 Discussion.fpo - https://discussion.fedoraproject.org/t/f41-change-proposal-python-built-wit… This is a proposed Change for Fedora Linux. This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == Instead of [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags Fedora's default `-O2` compiler flag], we will use `-O3` to build CPython. This only impacts the interpreter and Python standard library, not any 3rd party extension modules built as RPM or on developer machines. This aligns with the way Python is built upstream. According to our performance measurements, it makes Python significantly faster (pyperformance geometric mean: 1.04x faster). == Owner == * Name: [[User:churchyard|Miro Hrončok]] * Email: mhroncok(a)redhat.com == Detailed Description == We will replace the `-O2` compiler flag with `-O3` when building the python3.13 package. This change may be backported to older Pythons if desired. [[Changes/Python3.13|Python 3.13 should be the main Python version in Fedora 41+]]. The [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_compiler_flags Fedora packaging guidelines] about compiler flags explicitly say: ''> Overriding these flags for performance optimizations (for instance, `-O3` instead of `-O2`) is generally discouraged. If you can present benchmarks that show a significant speedup for this particular code, this could be revisited on a case-by-case basis.'' This change proposal presents such benchmarks and a case for Python to use `-O3`. This change is limited to CPython interpreter and extension modules from the Python standard library only thanks to [[Changes/Python_Extension_Flags_Reduction]] (since Fedora 39). Other Python extension modules will remain bulidng as before, e.g. in RPM packages, they will still be built with `-O2`, unless Fedora changes that globally. The extension modules built with `-O2` still work with Python built with `-O3`. == Feedback == == Benefit to Fedora == Upstream already builds Python with `-O3` by default. Fedora's Python built with `-O3` is faster (1.04x): {| class="wikitable sortable" |+ Benchmark with python3.12-3.12.2-3.fc41 |- ! Benchmark !! -O2 !! -O3 !! Change !! Significance |- | 2to3 || 465 ms || 446 ms || 1.04x faster || Significant (t=21.72) |- | async_generators || 853 ms || 784 ms || 1.09x faster || Significant (t=36.61) |- | async_tree_cpu_io_mixed || 1.19 sec || 1.11 sec || 1.08x faster || Significant (t=13.38) |- | async_tree_cpu_io_mixed_tg || 1.17 sec || 1.09 sec || 1.08x faster || Significant (t=18.69) |- | async_tree_eager || 202 ms || 189 ms || 1.07x faster || Significant (t=7.99) |- | async_tree_eager_cpu_io_mixed || 727 ms || 664 ms || 1.09x faster || Significant (t=18.56) |- | async_tree_eager_cpu_io_mixed_tg || 633 ms || 558 ms || 1.13x faster || Significant (t=24.53) |- | async_tree_eager_io || 1.72 sec || 1.68 sec || 1.03x faster || Significant (t=6.13) |- | async_tree_eager_io_tg || 1.65 sec || 1.62 sec || 1.02x faster || Significant (t=4.65) |- | async_tree_eager_memoization || 437 ms || 422 ms || 1.04x faster || Significant (t=5.09) |- | async_tree_eager_memoization_tg || 330 ms || 322 ms || 1.03x faster || Significant (t=2.60) |- | async_tree_eager_tg || 137 ms || 125 ms || 1.09x faster || Significant (t=16.94) |- | async_tree_io || 1.64 sec || 1.60 sec || 1.02x faster || Significant (t=9.49) |- | async_tree_io_tg || 1.65 sec || 1.61 sec || 1.02x faster || Not significant |- | async_tree_memoization || 895 ms || 871 ms || 1.03x faster || Significant (t=3.73) |- | async_tree_memoization_tg || 848 ms || 836 ms || 1.01x faster || Not significant |- | async_tree_none || 718 ms || 700 ms || 1.03x faster || Significant (t=6.90) |- | async_tree_none_tg || 686 ms || 659 ms || 1.04x faster || Significant (t=13.11) |- | asyncio_tcp || 757 ms || 748 ms || 1.01x faster || Not significant |- | asyncio_tcp_ssl || 2.58 sec || 2.56 sec || 1.01x faster || Not significant |- | asyncio_websockets || 419 ms || 418 ms || 1.00x faster || Not significant |- | bench_mp_pool || 10.7 ms || 10.7 ms || 1.00x faster || Not significant |- | bench_thread_pool || 1.62 ms || 1.61 ms || 1.01x faster || Not significant |- | chameleon || 12.2 ms || 12.0 ms || 1.02x faster || Not significant |- | chaos || 113 ms || 105 ms || 1.07x faster || Significant (t=46.23) |- | comprehensions || 37.4 us || 35.1 us || 1.07x faster || Significant (t=49.72) |- | coroutines || 42.4 ms || 41.4 ms || 1.02x faster || Significant (t=18.68) |- | coverage || 109 ms || 104 ms || 1.05x faster || Significant (t=33.91) |- | create_gc_cycles || 1.84 ms || 1.79 ms || 1.02x faster || Significant (t=5.50) |- | crypto_pyaes || 141 ms || 127 ms || 1.11x faster || Significant (t=86.61) |- | dask || 766 ms || 769 ms || 1.00x slower || Not significant |- | deepcopy || 619 us || 614 us || 1.01x faster || Not significant |- | deepcopy_memo || 71.3 us || 68.3 us || 1.04x faster || Significant (t=26.58) |- | deepcopy_reduce || 5.62 us || 5.56 us || 1.01x faster || Not significant |- | deltablue || 5.76 ms || 5.49 ms || 1.05x faster || Significant (t=7.97) |- | django_template || 62.8 ms || 59.7 ms || 1.05x faster || Significant (t=27.05) |- | docutils || 4.38 sec || 4.29 sec || 1.02x faster || Significant (t=11.25) |- | fannkuch || 706 ms || 667 ms || 1.06x faster || Significant (t=75.80) |- | float || 144 ms || 137 ms || 1.05x faster || Significant (t=24.66) |- | gc_traversal || 5.73 ms || 5.81 ms || 1.01x slower || Not significant |- | generators || 56.0 ms || 58.2 ms || 1.04x slower || Significant (t=-16.25) |- | genshi_text || 40.8 ms || 39.5 ms || 1.03x faster || Significant (t=17.64) |- | genshi_xml || 88.2 ms || 86.3 ms || 1.02x faster || Significant (t=6.96) |- | go || 223 ms || 217 ms || 1.03x faster || Significant (t=19.92) |- | hexiom || 10.3 ms || 9.76 ms || 1.05x faster || Significant (t=42.15) |- | html5lib || 109 ms || 108 ms || 1.01x faster || Not significant |- | json_dumps || 17.4 ms || 16.3 ms || 1.06x faster || Significant (t=45.38) |- | json_loads || 44.2 us || 42.3 us || 1.04x faster || Significant (t=27.71) |- | logging_format || 12.9 us || 12.4 us || 1.04x faster || Significant (t=9.81) |- | logging_silent || 176 ns || 174 ns || 1.01x faster || Not significant |- | logging_simple || 11.4 us || 11.0 us || 1.03x faster || Significant (t=9.94) |- | mako || 19.2 ms || 18.1 ms || 1.06x faster || Significant (t=54.89) |- | mdp || 4.46 sec || 4.33 sec || 1.03x faster || Significant (t=30.14) |- | meteor_contest || 189 ms || 167 ms || 1.13x faster || Significant (t=60.31) |- | nbody || 157 ms || 153 ms || 1.03x faster || Significant (t=4.34) |- | nqueens || 153 ms || 140 ms || 1.09x faster || Significant (t=63.60) |- | pathlib || 32.9 ms || 32.6 ms || 1.01x faster || Not significant |- | pickle || 18.6 us || 16.0 us || 1.16x faster || Significant (t=23.88) |- | pickle_dict || 45.8 us || 44.6 us || 1.03x faster || Significant (t=16.51) |- | pickle_list || 6.86 us || 6.59 us || 1.04x faster || Significant (t=19.65) |- | pickle_pure_python || 515 us || 505 us || 1.02x faster || Not significant |- | pidigits || 285 ms || 284 ms || 1.00x faster || Not significant |- | pprint_pformat || 2.72 sec || 2.54 sec || 1.07x faster || Significant (t=40.28) |- | pprint_safe_repr || 1.34 sec || 1.25 sec || 1.08x faster || Significant (t=58.43) |- | pyflate || 738 ms || 724 ms || 1.02x faster || Not significant |- | python_startup || 15.5 ms || 15.3 ms || 1.01x faster || Not significant |- | python_startup_no_site || 11.2 ms || 11.0 ms || 1.01x faster || Not significant |- | raytrace || 549 ms || 514 ms || 1.07x faster || Significant (t=45.37) |- | regex_compile || 245 ms || 233 ms || 1.05x faster || Significant (t=13.30) |- | regex_dna || 269 ms || 268 ms || 1.00x faster || Not significant |- | regex_effbot || 4.83 ms || 4.95 ms || 1.03x slower || Significant (t=-12.52) |- | regex_v8 || 33.7 ms || 33.1 ms || 1.02x faster || Not significant |- | richards || 75.7 ms || 71.9 ms || 1.05x faster || Significant (t=18.30) |- | richards_super || 85.2 ms || 81.4 ms || 1.05x faster || Significant (t=31.25) |- | scimark_fft || 662 ms || 587 ms || 1.13x faster || Significant (t=71.10) |- | scimark_lu || 199 ms || 190 ms || 1.04x faster || Significant (t=26.77) |- | scimark_monte_carlo || 123 ms || 117 ms || 1.05x faster || Significant (t=37.45) |- | scimark_sor || 217 ms || 210 ms || 1.04x faster || Significant (t=10.68) |- | scimark_sparse_mat_mult || 8.51 ms || 7.42 ms || 1.15x faster || Significant (t=62.99) |- | spectral_norm || 196 ms || 183 ms || 1.07x faster || Significant (t=95.78) |- | sqlalchemy_declarative || 239 ms || 234 ms || 1.02x faster || Significant (t=4.81) |- | sqlalchemy_imperative || 33.1 ms || 33.4 ms || 1.01x slower || Not significant |- | sqlglot_normalize || 197 ms || 187 ms || 1.05x faster || Significant (t=39.81) |- | sqlglot_optimize || 97.1 ms || 91.3 ms || 1.06x faster || Significant (t=47.14) |- | sqlglot_parse || 2.29 ms || 2.18 ms || 1.05x faster || Significant (t=14.70) |- | sqlglot_transpile || 2.79 ms || 2.67 ms || 1.04x faster || Significant (t=11.76) |- | sqlite_synth || 3.97 us || 3.90 us || 1.02x faster || Not significant |- | sympy_expand || 833 ms || 802 ms || 1.04x faster || Significant (t=19.41) |- | sympy_integrate || 34.7 ms || 33.8 ms || 1.03x faster || Significant (t=9.99) |- | sympy_str || 511 ms || 489 ms || 1.04x faster || Significant (t=18.17) |- | sympy_sum || 286 ms || 278 ms || 1.03x faster || Significant (t=14.46) |- | telco || 12.6 ms || 11.7 ms || 1.08x faster || Significant (t=9.31) |- | tomli_loads || 3.91 sec || 3.56 sec || 1.10x faster || Significant (t=46.29) |- | tornado_http || 213 ms || 212 ms || 1.01x faster || Not significant |- | typing_runtime_protocols || 214 us || 196 us || 1.09x faster || Significant (t=24.74) |- | unpack_sequence || 70.5 ns || 66.0 ns || 1.07x faster || Significant (t=8.58) |- | unpickle || 24.3 us || 22.0 us || 1.10x faster || Significant (t=10.67) |- | unpickle_list || 7.44 us || 8.61 us || 1.16x slower || Significant (t=-45.10) |- | unpickle_pure_python || 390 us || 360 us || 1.08x faster || Significant (t=37.48) |- | xml_etree_generate || 160 ms || 145 ms || 1.10x faster || Significant (t=44.33) |- | xml_etree_iterparse || 189 ms || 180 ms || 1.05x faster || Significant (t=20.16) |- | xml_etree_parse || 275 ms || 257 ms || 1.07x faster || Significant (t=20.58) |- | xml_etree_process || 106 ms || 98.6 ms || 1.08x faster || Significant (t=46.73) |- | Geometric mean || || || 1.04x faster || |} Generated by `pyperformance run -o Ox.json` and `pyperformance compare -O table O2.json O3.json` on Fedora 40 x86_64 with rawhide-built Python, python3.12-3.12.2-3.fc41, on Lenovo X1 Carbon 3rd gen. The benchmark was performed on Python 3.12 because it uses 3rd party Python packages lacking support for Python 3.13. Once it is possible to run such a benchmark for Python 3.13, we will do so. The benchmark was performed on x86_64. Until somebody presents a contradicting benchmark (or gives explicit reason for us to measure it), we believe the change makes sense on all architectures. == Scope == * Proposal owners: ** Change python3.13 to build with `-O3` instead of `-O2` ** Backport the change to older Pythons if desired * Other developers: no action expected, report bugs when found * Release engineering: no action expected * Policies and guidelines: this change is following the spirit of the guidelines * Trademark approval: not needed for this Change * Alignment with Community Initiatives: faster Python, happier users, more contributors? == Upgrade/compatibility impact == None expected. == How To Test == To verify this change has landed, inspect the build.log of python3.13. It should be built with `gcc ... -O3`. To test this change, test Fedora as you would normally do and assert there are no regressions. Run benchmarks, and report slowdowns if found. == User Experience == Faster Python, faster Fedora. == Dependencies == * [[Changes/Python_Extension_Flags_Reduction]] landed in Fedora 39 * [[Changes/Python3.13]] is expected to land in Fedora 41. If not, we will apply this on Python 3.12. == Contingency Plan == * Contingency mechanism: revert the change, rebuild Python * Contingency deadline: Final Freeze * Blocks release? No == Documentation == N/A (not a System Wide Change) == Release Notes == -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney -- _______________________________________________ devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org To unsubscribe send an email to devel-announce-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel-announce@lists.fedorapr… Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

6 9

Following up on: Three steps we could take to make supply chain attacks a bit harder
by Gordon Messmer 31 Jul '24

31 Jul '24

Now that I've packaged the GEF gdb extension (package "gdb-gef" in Rawhide) which provides the "got-audit" command, I wanted to ask again whether there's interest in monitoring critical packages for signs of GOT tampering. The "got-audit" gdb command examines a running process to print a list of symbols, and the file backing the area of memory where each symbol points. For example: accept(a)GLIBC_2.2.5 : /usr/lib64/libc.so.6 access(a)GLIBC_2.2.5 : /usr/lib64/libc.so.6 AES_encrypt(a)OPENSSL_3.0.0 : /usr/lib64/libcrypto.so.3 AES_set_encrypt_key(a)OPENSSL_3.0.0 : /usr/lib64/libcrypto.so.3 ... EVP_PKEY_set1_RSA(a)OPENSSL_3.0.0 : /usr/lib64/liblzma.so.5 :: ERROR EVP_PKEY_set1_RSA not exported by /usr/lib64/liblzma.so.5 In addition to printing the mapping, the command will print an error if it determines that two libraries are both exporting a matching symbol (e.g., if EVP_PKEY_set1_RSA appeared in libcrypto.so and in liblzma.so), and it will print an error if it determines that a symbol has been resolved to a file that doesn't export that symbol (e.g., if EVP_PKEY_set1_RSA points to liblzma.so, but liblzma.so doesn't export a symbol of that name). In the case of the xz-utils attack, the got-audit command would have logged an error due to the latter check, as illustrated above. (As a topic for later: the tirpc library exports functions with the same name as functions that appear in libc, so the behavior of erroring on duplicate symbols needs to be rationalized. Maybe an exemption for libtirpc.so? Are there other libraries that do this?) I have a proof-of-concept test implemented for the openssh package, in this pull request: https://src.fedoraproject.org/rpms/openssh/pull-request/73 If we are interested in building out protection from this class of attack, there are a number of questions I have about the best way to do that: The current implementation compares the result to an expected result. While I like the idea of keeping historical records, I'm not sure I can justify the developer overhead of updating the expected result file as it evolves due to updates in the program and in the libraries it uses. It would be simpler to simply watch for errors in the output. If we only watch for errors in the output, it would be much easier to convert this into a generic test and run it against more daemons. The proof-of-concept only audits symbols in the binary, but shared objects have their own set of dynamic symbols loaded from shared libraries. The output is quite large if we log the full set, and more importantly it will change much more often, but if we only look for errors, we can use "got-audit --all" to also watch symbol resolution within the shared objects. But it's probably important to mention that if we only watch for errors in the output, the test could very easily invisibly break. If it stopped working properly and printed nothing, we might go on thinking that we had mitigations in place when the test had actually become a NOP. Is this work interesting? Should I continue working on it?

5 11

2FA policy for provenpackagers is now active
by Zbigniew Jędrzejewski-Szmek 30 Jul '24

30 Jul '24

Proven packagers, we changed [2,3] the FESCo policy document [1] for provenpackagers to say: "Provenpackagers SHOULD have two-factor-authentication (2FA) enabled for their FAS accounts." This is not enforced or checked, but please take steps to conform to the policy if you haven't yet. [1] https://docs.fedoraproject.org/en-US/fesco/Provenpackager_policy/ [2] It's not visible on the web yet, because antora is doing its thing … slowly. [3] https://pagure.io/fesco/issue/3186 Zbyszek

29 73

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

devel June 2024