packaging June 2020

packaging@lists.fedoraproject.org

14 participants
13 discussions

RPM-level auto release and changelog bumping - Fedora 33 System-Wide Change proposal
by Ben Cotton 13 Aug '22

13 Aug '22

https://fedoraproject.org/wiki/Changes/rpm_level_auto_release_and_changelog… == Summary == redhat-rpm-config will be updated so users of the auto framework get automated release and changelog bumping. == Owner == * Name: [[User:nim| Nicolas Mailhot]] * Email: <nicolas.mailhot at laposte.net> == Detailed Description == This is a system-wide change because all packages build with redhat-rpm-config, but it only concerns packages that opted to use this part of redhat-rpm-config (auto framework). The change will make those packages auto-bump and auto-changelog at the rpm level, in an infrastructure-independent way. == Benefit to Fedora == Autobumping removes a huge packager shore and makes timestamping in changelogs more reliable. == Scope == * Proposal owners: The feature is coded and works at the rpm level. Unfortunately, mock filters away the srpms containing the bump state, so it does not work in upper layers. * Other developers: The feature requires buy-in by mock developers (and probably koji developers) to lift the restrictions that block it above the rpm level. Also, it requires a mechanism to pass the user name and email that will be used in bumped changelogs (defining two variables in ~/.rpmmacros is sufficient at rpm level) * Mock issue: https://github.com/rpm-software-management/mock/issues/599 * Release engineering: https://pagure.io/releng/issue/9567 * Policies and guidelines: maybe eventually if things work out on the technical level * FPC issue: https://pagure.io/packaging-committee/issue/998 * Trademark approval: N/A (not needed for this Change) == Upgrade/compatibility impact == This is a pure build tooling update, it changes how things are built not what is built. == How To Test == A redhat-rpm-config packages with the changes and some example packages are available in https://copr.fedorainfracloud.org/coprs/nim/refactoring-forge-patches-auto-… Since the mock/copr layer is currently blocking the feature, you need to install the redhat-rpm-config and forge macro packages available in this repo locally. Afterwards you can take any of the example packages in the repo and rebuild them with rpmbuild -ba to your heart content, and see the releases bump and the changelogs being updated accordingly. To get beautiful changelogs, you also need to add <pre> %buildsys_name Your name %buildsys_email Your email </pre> in ~/.rpmmacros == User Experience == N/A Packager experience change only == Dependencies == The change is a spin-off of https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macro… Therefore, it depends on the success of that other change and will probably need rebasing if the code in this other change evolves during the redhat-rpm-config merge. It also depends on mock / copr/ koji buy-in and changes, that may add their own requirements. == Contingency Plan == There is no contingency plan because the change will happen or not at all. == Documentation == There is as much documentation as the average redhat-rpm-config change (ie comments in the macro files themselves) == Release Notes == N/A Packager productivity change only -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis

12 23

Schedule for Thursday's FPC Meeting (2019-08-15 16:00 UTC)
by James Antill 18 Jul '21

18 Jul '21

Following is the list of topics that will be discussed in the FPC meeting Thursday at 2019-08-15 16:00 UTC in #fedora-meeting-1 on irc.freenode.net. Local time information (via. uitime): ================= Day: Thursday ================== 2019-08-15 09:00 PDT US/Pacific 2019-08-15 12:00 EDT --> US/Eastern <-- 2019-08-15 16:00 UTC UTC 2019-08-15 17:00 BST Europe/London 2019-08-15 18:00 CEST Europe/Berlin 2019-08-15 18:00 CEST Europe/Paris 2019-08-15 21:30 IST Asia/Calcutta ---------------- New Day: Friday ----------------- 2019-08-16 00:00 HKT Asia/Hong_Kong 2019-08-16 00:00 +08 Asia/Singapore 2019-08-16 01:00 JST Asia/Tokyo 2019-08-16 02:00 AEST Australia/Brisbane Links to all tickets below can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting = Followups = #topic #902 Cleanup & enhance spec files .fpc 902 https://pagure.io/packaging-committee/issue/902 #topic #904 Caret versioning .fpc 904 https://pagure.io/packaging-committee/issue/904 #topic #907 Which %__foo macros for executables are acceptable? .fpc 907 https://pagure.io/packaging-committee/issue/907 #topic #909 Suggest that linting/measuring-coverage is not for %check .fpc 909 https://pagure.io/packaging-committee/issue/909 #topic #914 Automatic R runtime dependencies .fpc 914 https://pagure.io/packaging-committee/issue/914 = Open Floor = For more complete details, please visit each individual ticket. The report of the agenda items can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting If you would like to add something to this agenda, you can: * Reply to this e-mail * File a new ticket at: https://pagure.io/packaging-committee * E-mail me directly * Bring it up at the end of the meeting, during the open floor topic. Note that added topics may be deferred until the following meeting.

3 2

How to pass runtime argument on RPM installation?
by Mohankumar S 01 Jul '20

01 Jul '20

Hi, I want to pass a runtime argument while installing a rpm . say like rpm -ivh sample. rpm "hello". I want to read the argument and make use of it while installing it. Any help would be appreciated. Regards, Mohan

6 11

Patches in Forge macros - Auto macros - Detached rpm changelogs - Fedora 33 System-Wide Change proposal
by Ben Cotton 01 Jul '20

01 Jul '20

https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_macro… == Summary == redhat-rpm-config will be updated to add patching support to forge macros, a plug-able framework to register macros to execute in specific sections, and rpm changelogs in detached files. == Owner == * Name: [[User:nim| Nicolas Mailhot]] * Email: <nicolas.mailhot at laposte.net> == Detailed Description == This is a system-wide change because all packages build with redhat-rpm-config, but it only concerns packages that opted to use this part of redhat-rpm-config (users of forge, fonts and go macros). It was driven first, by the need to make the underlying macro infrastructure robust enough to package Go modules, and second, by an unfortunate rpm 4.15 regression that proved it was foolish to depend on rpmbuild to parse Tags in anything except canonical order. === Forge === * forge macro now process patches, including in multi-source spec files, in a natural way * all dependencies on source/patch numbering were eradicated, you can write a whole multi-source/multi-patch spec without worrying about source or patch numbers * zero suffix is no longer special (à la Source/Source0 way), you can declare forge blocks starting at 42 if that‘s your preference === Fully automated packaging === A framework was added so macro subsystems can register execution blocks in specific parts or the spec file. Execution blocks are orchestrated (using KISS rules) so for example the forge part of %prep is executed before the go parts that depend on forge archives being unpacked and patched, and macros that want to create srpm headers are executed before macros that want to create subpackage headers. Such a framework is a requirement to control the generation order within the spec file and make sure rpm maintainers are not cross with you. That means a spec with no special custom processing is reduced to a set of %global control variables that activate specific execution blocks, and everything bellow those control variable is short and unchanging boilerplate. A packager that needs custom processing can add custom code above or bellow the various `%auto_foo` calls, and check with `rpmspec -P` that the result does what he wants it to do. For obvious reliability reasons injecting custom code in the middle of an `%auto_foo` sequence is not allowed. <pre> %global source_name … %global source_release … %global source_post_release … %global forge_url0 … %global forge_commit0 … %global forge_url1 … %global forge_tag1 … %global go_module33 … %global go_description33 … %global font_family22 … %global font_conf22 … %auto_init %auto_pkg %sourcelist %auto_sources %patchlist %auto_patches %prep %auto_prep %generate_buildrequires %auto_generate_buildrequires %build %auto_build %install %auto_install %check %auto_check %auto_files %changelog %auto_changelog </pre> === Detached changelogs === This framework was used to implement detached rpm changelogs in a reliable way. === Generic -doc creation === This framework was used to implement automated -doc subpackage creation, because creating them by hand gets annoying after the nth upstream that wants you do distribute heavy PDF documentation files. === Huge refactoring and fleshing out of the lua library === Writing high-level features like the above required defining a library of lua routines like an expand that expands fully, an unset that actually undefines, a read that tells you if a variable exists or is set to "", a `fedora.echo()` wrapper around `rpm.expand("%{echo:%{expand:" .. text .. "}}")`, etc. Those are now available for others to use should they want to. My coding skills are not up to navigating the upstream low level rpm lua API without blowing up on the landmines it is littered with. Therefore, I abstracted landmine avoidance in a single place. === Drawbacks === Nothing is free, and a higher level of automation required using rigid naming for control variables. Because software is a lot less tolerant of fuzzy naming than human beings. So, all forge control variables are renamed, fonts control variables have been renamed too, and go control variables will need renaming (in that last case, that’s not a problem because moving to go modules requires reworking variables anyway, so it will be done as part of the module effort in F34). To ease the transition a compatibility layer was added to forge macros so old variables and new variables are aliased both ways (this will eventually go away because it’s quite a lot of compatibility code to maintain). Mixing syntaxes (old and new) is not supported, you need to convert your spec file to new forge variables or not at all (if not at all, do not try to use new features like patching). == Benefit to Fedora == Spec files that do more with less manual expensive to maintain spec code. Without this productivity win, complex efforts like converting Fedora Go packages to Go modules, or draining the Font packages swamp given that legacy formats are no longer supported by apps, are not possible with the current level of Fedora manpower. == Scope == * Proposal owners: The core of the feature is done and tested (and retested). It may evolve during the redhat-rpm-config merge process. https://src.fedoraproject.org/rpms/redhat-rpm-config/pull-request/95 * Other developers: The way current forge macros call forge macros will need a little patching once the change lands. For other packagers, there should be no change except a warning in rpm build logs to switch to the new syntax before the compatibility layer is removed. * Release engineering: https://pagure.io/releng/issue/9565 * FPC: https://pagure.io/packaging-committee/issue/997 * Policies and guidelines: Forge guidelines will need some rework (mostly simplification, because the new syntax is both more powerful and more regular). For the average packager, the new syntax is the same old syntax with little naming adjustments (for example, %{forgeurl} becomes %{forge_url}, %forgemeta is subsumed into %auto_init, etc) * Trademark approval: N/A (not needed for this Change)  == Upgrade/compatibility impact == This is a pure build tooling update, it changes how things are built not what is built. == How To Test == A redhat-rpm-config packages with the changes and some example packages are available in https://copr.fedorainfracloud.org/coprs/nim/refactoring-forge-patches-auto-… == User Experience == N/A Packager experience change only == Dependencies == The change depends on a redhat-rpm-config merge by redhat-rpm-config maintainers == Contingency Plan == There is no contingency plan because the redhat-rpm-config merge will happen or not. If it does not happen, i18n, fonts and Go Changes that are/were envisioned for F33 or F34 will be postponed indefinitely. == Documentation == There is as much documentation as the average redhat-rpm-config change (ie comments in the macro files themselves) == Release Notes == N/A Packager productivity change only -- Ben Cotton He / Him / His Senior Program Manager, Fedora & CentOS Stream Red Hat TZ=America/Indiana/Indianapolis

3 3

Two PRs to look at as we haven't had quorum for a couple of weeks
by James Antill 25 Jun '20

25 Jun '20

We missed quorum again this week, and a couple of people have been asking about their PRs so I wondered if we might be able to get them moving a bit over email: #814 Add SELinux Independent Policy Guidelines https://pagure.io/packaging-committee/pull-request/814 #938 Add Package Review Process page https://pagure.io/packaging-committee/pull-request/938

2 1

Schedule for Thursday's FPC Meeting (2020-06-18 16:00 UTC)
by James Antill 18 Jun '20

18 Jun '20

Following is the list of topics that will be discussed in the FPC meeting Thursday at 2020-06-18 16:00 UTC in #fedora-meeting-1 on irc.freenode.net. Local time information (via. uitime): ================= Day: Thursday ================== 2020-06-18 09:00 PDT US/Pacific 2020-06-18 12:00 EDT --> US/Eastern <-- 2020-06-18 16:00 UTC UTC 2020-06-18 17:00 BST Europe/London 2020-06-18 18:00 CEST Europe/Berlin 2020-06-18 18:00 CEST Europe/Paris 2020-06-18 21:30 IST Asia/Calcutta ---------------- New Day: Friday ----------------- 2020-06-19 00:00 HKT Asia/Hong_Kong 2020-06-19 00:00 +08 Asia/Singapore 2020-06-19 01:00 JST Asia/Tokyo 2020-06-19 02:00 AEST Australia/Brisbane Links to all tickets below can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting = Followup Issues = #topic #907 Which %__foo macros for executables are acceptable? .fpc 907 https://pagure.io/packaging-committee/issue/907 #topic #909 Suggest that linting/measuring-coverage is not for %check .fpc 909 https://pagure.io/packaging-committee/issue/909 #topic #977 Get new members? .fpc 977 https://pagure.io/packaging-committee/issue/977 = Followup Pull Requests = #topic #pr-814 Add SELinux Independent Policy Guidelines. https://pagure.io/packaging-committee/pull-request/814 #topic #pr-938 Add Package Review Process page. https://pagure.io/packaging-committee/pull-request/938 #topic #pr-947 Deprecate Old Style Dependency Generators. https://pagure.io/packaging-committee/pull-request/947 #topic #pr-954 Prohibit use of `rpm` command from specfile. https://pagure.io/packaging-committee/pull-request/954 = New Pull Requests = #topic #pr-942 Recommend storing changelog entries in separate file. https://pagure.io/packaging-committee/pull-request/942 = Open Floor = For more complete details, please visit each individual ticket. The report of the agenda items can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting If you would like to add something to this agenda, you can: * Reply to this e-mail * File a new ticket at: https://pagure.io/packaging-committee * E-mail me directly * Bring it up at the end of the meeting, during the open floor topic. Note that added topics may be deferred until the following meeting.

1 0

Re: Highlights from the latest Copr release 2020-06-10
by Pavel Raiskup 16 Jun '20

16 Jun '20

On Tuesday, June 16, 2020 8:48:32 PM CEST Ian McInerney wrote: > ...snip... > > > > - There's now more fair build scheduler. Previously, no matter whether the > > "background" attribute was set or not for the build - once builder was > > allocated for a concrete copr user - copr never terminated the builder > > as long as the user kept filling the build queue with new tasks (and it > > blocked the quota for others). Newly, there's a limit of at most > > eight consecutive builds or 30 minutes for one user (sandbox) on one > > builder and the builder is immediately terminated - which gives a chance > > to assign new builders to others' tasks (which have a higher priority at > > that point). > > > > I am confused by this scheduler part. Does this mean that any build that > takes longer than 30 minutes will be cancelled? What about software > packages that are larger and require more build time? This deserved better spelling => any builder that was assigned to one user for _more than 30 minutes_ is - after the build _is finished_ - terminated. The build timeout is a different configuration option (currently around 24h, and the timeout concept should change soon so it is more flexible, https://pagure.io/copr/copr/issue/1303 ). Pavel > -Ian >

1 0

Highlights from the latest Copr release 2020-06-10
by Pavel Raiskup 11 Jun '20

11 Jun '20

Hello! On Jun 10, 2020, a new Copr release landed production. Related client tooling updates are here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-390cce74d6 https://bodhi.fedoraproject.org/updates/FEDORA-2020-f90873700f https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-3b8c2778f1 https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-feca133c66 https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2020-8449e5d605 Here is the list of visible changes, and new features: - Increased build task throughput. The VM spawner is more flexible now. First, we don't allocate that many workers if they are not actually needed, but more importantly the upper limit for concurrently running workers has risen to 140 (including all architectures), previously we had ~70. We'll see how things go from the backend perspective, but it is expected that we'll go even higher (waiting for a new HW in the new Fedora lab, cheaper VM instances in the future, etc.). So the numbers will likely change; this is to make clear the current state of things. - Copr project "runtime" dependencies were implemented. Newly you can specify set of repositories your project depends on. Such repositories will be installed together with the copr project repo file (e.g., by 'dnf copr enable YOU/YOUR_PROEJCT'). Those repositories can be other project in Copr or some 3rd party repository. This is very similar to build-time dependencies implemented long time ago. Take a look at blog post: https://fedora-copr.github.io/posts/runtime-dependencies - There's now more fair build scheduler. Previously, no matter whether the "background" attribute was set or not for the build - once builder was allocated for a concrete copr user - copr never terminated the builder as long as the user kept filling the build queue with new tasks (and it blocked the quota for others). Newly, there's a limit of at most eight consecutive builds or 30 minutes for one user (sandbox) on one builder and the builder is immediately terminated - which gives a chance to assign new builders to others' tasks (which have a higher priority at that point). - Copr-cli supports batch build delete feature: $ copr-cli delete build_id [build_id ...] Please use this instead of requesting removal of each build_id separately - it will be much faster because you will save the copr backend useless createrepo_c runs after each request. - We also implemented a priority queue algorithm for Action tasks (non-build tasks, e.g., forking, build/project deleting, etc.). This should solve several issues when the action queue gets temporarily too large to process immediately; namely the delayed "initial createrepo" action problem that previously caused an ugly build failures in freshly created copr projects. - Cancel build feature was fixed, and the "cancel" request should reliably release all occupied builder machines (allowing them to be re-used, or terminated). - Users are not allowed to disable chroot in a project that has some running builds against the disabled chroot. The running builds need to be canceled first. This change as done to avoid inconsistencies between frontend/backend, and wasted builder quota on useless tasks. - More space for /var/cache/mock directory, so repeated builds on the same machine won't face the ENOSPC error on mock caches. Considering that each builder can only do at most eight builds (then terminated), this problem should be finally fixed. Happy building! Pavel

2 2

Schedule for Thursday's FPC Meeting (2020-06-11 16:00 UTC)
by James Antill 10 Jun '20

10 Jun '20

Following is the list of topics that will be discussed in the FPC meeting Thursday at 2020-06-11 16:00 UTC in #fedora-meeting-1 on irc.freenode.net. Local time information (via. uitime): ================= Day: Thursday ================== 2020-06-11 09:00 PDT US/Pacific 2020-06-11 12:00 EDT --> US/Eastern <-- 2020-06-11 16:00 UTC UTC 2020-06-11 17:00 BST Europe/London 2020-06-11 18:00 CEST Europe/Berlin 2020-06-11 18:00 CEST Europe/Paris 2020-06-11 21:30 IST Asia/Calcutta ---------------- New Day: Friday ----------------- 2020-06-12 00:00 HKT Asia/Hong_Kong 2020-06-12 00:00 +08 Asia/Singapore 2020-06-12 01:00 JST Asia/Tokyo 2020-06-12 02:00 AEST Australia/Brisbane Links to all tickets below can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting = Followup Issues = #topic #907 Which %__foo macros for executables are acceptable? .fpc 907 https://pagure.io/packaging-committee/issue/907 #topic #909 Suggest that linting/measuring-coverage is not for %check .fpc 909 https://pagure.io/packaging-committee/issue/909 #topic #963 Blanket Bootstrap Exception for building Mono .fpc 963 https://pagure.io/packaging-committee/issue/963 #topic #977 Get new members? .fpc 977 https://pagure.io/packaging-committee/issue/977 = Followup Pull Requests = #topic #pr-814 Add SELinux Independent Policy Guidelines https://pagure.io/packaging-committee/pull-request/814 #topic #pr-938 Add Package Review Process page https://pagure.io/packaging-committee/pull-request/938 #topic #pr-#947 Deprecate Old Style Dependency Generators https://pagure.io/packaging-committee/pull-request/947 #topic #pr-#954 Prohibit use of `rpm` command from specfile. https://pagure.io/packaging-committee/pull-request/954 = New Pull Requests = #topic #pr-#942 Recommend storing changelog entries in separate file. https://pagure.io/packaging-committee/pull-request/942 = Open Floor = For more complete details, please visit each individual ticket. The report of the agenda items can be found at: https://pagure.io/packaging-committee/issues?status=Open&tags=meeting If you would like to add something to this agenda, you can: * Reply to this e-mail * File a new ticket at: https://pagure.io/packaging-committee * E-mail me directly * Bring it up at the end of the meeting, during the open floor topic. Note that added topics may be deferred until the following meeting.

1 0

Package tests?
by Miroslav Suchý 10 Jun '20

10 Jun '20

Hi. On my system: du -ch /usr/lib/python3.8/site-packages/*/tests/ 18 MB in total I did not see anything in Guidelines about packaging tests. Do we really need them in final package? Are there any objection to propose in Guidelines something like: %exclude %{python3_sitelib}/YOUR-MODULE/tests -- Miroslav Suchy, RHCA Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys

4 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

packaging June 2020