Automatic virtual provides for RPM macros?
by Miro Hrončok
Hello,
today at Nest, somebody said "unfortunately, there is no way to tell what
package to install to get a particular RPM macro".
I think that having an RPM provides generator for "rpm-macro(__python3)" or
similar should be a fairly simple exercise.
Would you folks consider that useful?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
1 week, 6 days
RPM-level auto release and changelog bumping - Fedora 33 System-Wide Change proposal
by Ben Cotton
https://fedoraproject.org/wiki/Changes/rpm_level_auto_release_and_changel...
== Summary ==
redhat-rpm-config will be updated so users of the auto framework get
automated release and changelog bumping.
== Owner ==
* Name: [[User:nim| Nicolas Mailhot]]
* Email: <nicolas.mailhot at laposte.net>
== Detailed Description ==
This is a system-wide change because all packages build with
redhat-rpm-config, but it only concerns packages that opted to use
this part of redhat-rpm-config (auto framework).
The change will make those packages auto-bump and auto-changelog at
the rpm level, in an infrastructure-independent way.
== Benefit to Fedora ==
Autobumping removes a huge packager shore and makes timestamping in
changelogs more reliable.
== Scope ==
* Proposal owners: The feature is coded and works at the rpm level.
Unfortunately, mock filters away the srpms containing the bump state,
so it does not work in upper layers.
* Other developers: The feature requires buy-in by mock developers
(and probably koji developers) to lift the restrictions that block it
above the rpm level. Also, it requires a mechanism to pass the user
name and email that will be used in bumped changelogs (defining two
variables in ~/.rpmmacros is sufficient at rpm level)
* Mock issue: https://github.com/rpm-software-management/mock/issues/599
* Release engineering: https://pagure.io/releng/issue/9567
* Policies and guidelines: maybe eventually if things work out on the
technical level
* FPC issue: https://pagure.io/packaging-committee/issue/998
* Trademark approval: N/A (not needed for this Change)
== Upgrade/compatibility impact ==
This is a pure build tooling update, it changes how things are built
not what is built.
== How To Test ==
A redhat-rpm-config packages with the changes and some example
packages are available in
https://copr.fedorainfracloud.org/coprs/nim/refactoring-forge-patches-aut...
Since the mock/copr layer is currently blocking the feature, you need
to install the redhat-rpm-config and forge macro packages available in
this repo locally. Afterwards you can take any of the example packages
in the repo and rebuild them with rpmbuild -ba to your heart content,
and see the releases bump and the changelogs being updated
accordingly.
To get beautiful changelogs, you also need to add
<pre>
%buildsys_name Your name
%buildsys_email Your email
</pre>
in ~/.rpmmacros
== User Experience ==
N/A Packager experience change only
== Dependencies ==
The change is a spin-off of
https://fedoraproject.org/wiki/Changes/Patches_in_Forge_macros_-_Auto_mac...
Therefore, it depends on the success of that other change and will
probably need rebasing if the code in this other change evolves during
the redhat-rpm-config merge.
It also depends on mock / copr/ koji buy-in and changes, that may add
their own requirements.
== Contingency Plan ==
There is no contingency plan because the change will happen or not at all.
== Documentation ==
There is as much documentation as the average redhat-rpm-config change
(ie comments in the macro files themselves)
== Release Notes ==
N/A Packager productivity change only
--
Ben Cotton
He / Him / His
Senior Program Manager, Fedora & CentOS Stream
Red Hat
TZ=America/Indiana/Indianapolis
7 months, 1 week
What way of opting out form a particular Python shebang flag do you prefer?
by Miro Hrončok
Hello Pythonistas, packagers.
In the context of this change:
https://fedoraproject.org/wiki/Changes/PythonSafePath
Python shebangs will have be:
#! /usr/bin/python3 -sP
In order to remove certain flags, packagers have the following tool:
# Unset -s on python shebang - ensure that extensions installed with pip
# to user locations are seen and properly loaded
%global py3_shebang_flags %(echo %py3_shebang_flags | sed s/s//)
Or:
# Don't add -P to Python shebang
# This package only works when /usr/bin is in sys.path
%global py3_shebang_flags %(echo %py3_shebang_flags | sed s/P//)
In the implementation PR, Maxwell suggested a different approach:
https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/141#com...
Basically, packagers would do something like this:
# Unset -s on python shebang - ensure that extensions installed with pip
# to user locations are seen and properly loaded
%global _python3_shebang_nousersite %{nil}
Or:
# Don't add -P to Python shebang
# This package only works when /usr/bin is in sys.path
%global _python3_shebang_safepath %{nil}
The macro names are not set in stone, it could even be %_python3_shebang_s and
%_python3_shebang_P.
The previous sed-based way would still work and packages that already use it
would not need to change immediately.
Do you consider the macro based approach better (worth it)? And if so, do you
prefer actual flag letters in the macro names, or the verbose names?
Thanks for your input.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
8 months
license of the binary policy
by Jilayne Lovejoy
Hi Fedora legal and packaging,
I'm cross-posting this, as I think it's relevant to both groups.
The current policy for filling out the license field of the spec file
(as described at
https://docs.fedoraproject.org/en-US/packaging-guidelines/LicensingGuidel...
) states, "The License: field refers to the licenses of the contents of
the binary rpm. When in doubt, ask."
As we consider how to improve documentation related to Fedora licensing,
it would be helpful to hear people's thoughts on the following:
1) how do you (package maintainers) interpret this policy in practice?
2) what further information/documentation about this policy would be
helpful?
3) should this policy be different, and if so, how?
4) any other related thoughts or observations
Thanks!
Jilayne
8 months, 1 week
Shell Completions
by Maxwell G
Hi everyone,
Currently, packaging shell completions is a bit of a hassle. You have to
memorize a bunch of long directory names (`%{_datadir}/bash-completion/
completions`, `%{_datadir}/fish/vendor_completions.d`, and `%{_datadir}/zsh/
site-functions`), and repeat them multiple times in the specfile. For some
reason, the bash completion directories are owned by filesystem, but zsh and
fish's shell completions directories are not. Therefore, every package needs
to copy
```
%files
[...]
%{_datadir}/bash-completion/completions/%{name}
%dir %{_datadir}/fish
%dir %{_datadir}/fish/vendor_completions.d
%{_datadir}/fish/vendor_completions.d/%{name}.fish
%dir %{_datadir}/zsh
%dir %{_datadir}/zsh/site-functions
%{_datadir}/zsh/site-functions/_%{name}
```
or own each shell's respective root directory and not bother with `%dir`
and explicit paths. I don't like doing the latter, as I prefer to be explicit
and follow the spirit of https://docs.fedoraproject.org/en-US/packaging-guidelines/#_explicit_lists.
I was thinking of creating an sh-completions-packaging package (name
suggestions welcome) that provides a subpackage containing macros for each of
those directories that would be part of the default buildroot and a filesystem
subpackage that owns these directories so packages can Require it and not have
to copy this long boiler plate.
I would appreciate any feedback you have, particularly regarding naming of the
packages and macros. For the macros, would `%{_bashcompdir}`,
`%{_zshcompdir}`, and `%{_fishcompdir}` be appropriate macro names? I'm a bit
unclear about macro namespacing conventions.
Even if you don't like this idea, I think the guidelines could at least use
some clarification in this area. I have seen or dealt with multiple packages
that install shell completions to the wrong directories, don't properly own the
directories, or have other problems.
--
Thanks,
Maxwell G (@gotmax23)
Pronouns: He/Him/His
8 months, 3 weeks
Re: [RFC] Build tag in RPM: from NVR to NVRB
by Miro Hrončok
On 23. 06. 22 14:24, Aleksandra Fedorova wrote:
>> 3) If every Fedora packager can rebuild anything without a commit, what do we
>> do prevent accidental builds?
> I think each rebuild should be treated as a new package, thus it would
> require a new bodhi update, testing, and signing. Which means it will
> be less likely to accidentally ship it.
>
> But as I mentioned in the post, right now we'd like to propose the
> refactoring, and not a change of the development workflow. Thus I
> would initially restrict the rebuild possibility to the admin group
> which handles mass-rebuilds and other admin tasks. Then I would
> gradually open it up case by case, each case through a separate
> conversation.
>
> Saying that, probably the first case, which I would consider is: is
> there a problem of an accidental rebuild of a merged code for Fedora
> Rawhide? What would be the reasons for us to_not_ allow it?
A specific example is this workflow:
Imagine 200 packages need to be rebuilt with boost 1.99.
1) I build boost 1.99 in a side tag
2) I commit a bump to 200 packages
3) I submit a side tag build for 200 packages
4) I repeat (3) until it seems futile
This solves the dependency order issues quite well. I also don't need to think
about that much and scripting it is trivial. If the build previously succeeded,
it won't do anything. If it failed, it will try again.
I realize this isn't a very clever workflow. In fact, that isn't my exact
workflow. Really, in (4) I exclude the packages that already rebuilt
successfully. But I don't need to be that careful when checking. Is my list of
finished builds slightly out of date? No problem. Are some builds still running
and I try to submit them again? No problem. Nothing happens.
With the proposed build ID think, every workflow like this would either need to
be extremely more robust, or we would end up with 20 useless concurrent
libreoffice builds very soon.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
9 months
Re: [RFC] Build tag in RPM: from NVR to NVRB
by Miro Hrončok
On 23. 06. 22 14:24, Aleksandra Fedorova wrote:
>> 2) A packager rebuilds packages from Fedora dist-git in a Copr repo, what will
>> be the value of the build tag? How will the Copr build sort over the official
>> Fedora builds? (This is essentially the same question but the answer might differ.)
> Similar as above, the questions would be how does it work now, do we
> want a change or do we want to keep the current setup?
Examples from before the Python 3.11 side tag was merged.
Fedora rawhide: python3-tomli-2.0.1-2.fc37 (built with Python 3.10)
Python 3.11 copr: python3-tomli-2.0.1-2.fc37 (built with Python 3.11)
The NEVRs are identical. Other builds in the Python 3.11 copr install the 3.11
build.
Now when we managed to not pick up a certain bump:
Fedora rawhide: python3-tomli-2.0.1-2.fc37 (built with Python 3.10)
Python 3.11 copr: python3-tomli-2.0.1-1.fc37 (built with Python 3.11)
The rawhide's NEVR is higher. Other builds in the Python 3.11 copr install try
to install the 3.10 build and that conflicts with other stuff in the copr and
the dependency resolution fails.
We certainly don't want to regress in that regard.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
9 months
Re: [RFC] Build tag in RPM: from NVR to NVRB
by Miroslav Suchý
Dne 23. 06. 22 v 14:24 Aleksandra Fedorova napsal(a):
>> 1) An user rebuilds a package from Fedora dist-git in local mock, what will be
>> the value of the build tag? How will the local build sort over the official
>> Fedora builds?
> Afaik currently, if you do a local build, you need to bump an NVR to
> ensure the upgrade path is set. Then if the base distribution upgrades
> or rebuilds the package, you would need to bump your version again if
> you like to preserve the upgrade path.
>
> This approach would work exactly the same way after the change, as
> bumping the Release tag in the sources makes build-related versioning
> irrelevant.
>
> We can agree that by default Build tag in mock environment is unset.
> Local builds have no build id, and to make clean upgrades bump of
> Release tag is required.
>
> But there are some new possibilities:
>
> If you want to create a local rebuild without Release bump, you would
> be able to pass the build tag value manually and set to whatever you
> want. For example you can add 1 to existing build tag. Or you could
would be able => most people will not do that.
> set it to 0. Or to 99999999.
Any final guidelines should recommend the semantics of the number. At least.
It can be monotonous sequence (1, 2, 3...) or a date used in SOA in DNS (2022062301). Or anything else. Just document it.
> The latter is less likely though.
>
> If I were to create local builds which are always higher or always
> lower than builds from the base system, I would probably play with the
> dist-tag, not build id:
>
> $ rpmdev-vercmp 0:1.2.3-12.fc37.15 0:1.2.3-12.localfc37.1
> 0:1.2.3-12.fc37.15 < 0:1.2.3-12.localfc37.1
>
> There is also a possibility to set default local build tag to be the
> local timestamp. But I am not sure if it is useful for a local
> development
>
>> 2) A packager rebuilds packages from Fedora dist-git in a Copr repo, what will
>> be the value of the build tag? How will the Copr build sort over the official
>> Fedora builds? (This is essentially the same question but the answer might differ.)
> Similar as above, the questions would be how does it work now, do we
> want a change or do we want to keep the current setup?
While for local build users may set the tag to some value, Copr should set it. And Copr developers needs to know to what
value it will be.
I, personally, do not participate in this thread as I am in neutral position. At the end, I would appreciate issue
created against Mock and Copr where will be clear guidance what and how it should be set.
Miroslav
9 months
Re: [RFC] Build tag in RPM: from NVR to NVRB
by Neal Gompa
On Thu, Jun 23, 2022 at 8:25 AM Aleksandra Fedorova <alpha(a)bookwar.info> wrote:
>
> Hi, Miro,
>
> On Thu, Jun 23, 2022 at 12:15 PM Miro Hrončok <mhroncok(a)redhat.com> wrote:
> >
> > On 18. 06. 22 13:05, Aleksandra Fedorova wrote:
> > > Hi, all,
> > >
> > > I'd like to discuss how we can add Build tag in the RPM.
> > >
> > > As one of the key points is to turn it into a common standard for rpm
> > > packages across the ecosystem, the conversation is currently opened
> > > upstream [1] and in RHEL Engineering. And I'd like to get Fedora
> > > community on board.
> > >
> > > This is not a finalized proposal and I think it is not ready yet to be
> > > a Fedora Change.
> > > But I want to start a conversation and ask for opinions. There are
> > > also some open questions which need help, especially the requirements
> > > around build reason. And alternative suggestions are welcome as well.
> > >
> > > I've posted long version at
> > > https://discussion.fedoraproject.org/t/rfc-build-tag-in-rpms/39954
> > >
> > > You can comment there (simply login with your FAS id) or here on the
> > > mailing list.
> > > And I am going to update that post with the new feedback.
> > >
> > > Shorter version:
> > > ================================================
> > > We'd like to introduce Build Number/Tag/Id in the rpm metadata.
> > >
> > > By this change we would like to:
> > > * Provide a possibility to change build environment and rebuild rpm
> > > packages without changing their content: neither sources nor spec
> > > files.
> > > * Set a common standard for the RPM-based ecosystem, which can be used
> > > not just within Fedora, but also by Remixes, downstreams, SIGs and
> > > other distributions.
> > >
> > > The most visible impact of the proposal would be the filename change:
> > >
> > > Current: dnf-4.9.0-12.fc36.noarch.rpm
> > > Proposed: dnf-4.9.0-12.fc36.34897715.noarch.rpm
> > >
> > > It can be implemented in two steps:
> > >
> > > 1) Now → Compatibility mode
> > >
> > > * Introduce Build tag in the rpm metadata
> > > * Introduce “build reason” to be added to rpm changelog as a top entry
> > > * Enable passing Build tag value to the build in Koji. The value of
> > > the Build tag will be set from Koji build id.
> > > * Introduce macro in Release field of the rpm spec files, which adds
> > > Build tag after the usual disttag
> > > Release: 12.%{?dist}%{?build:.%{build}}
> > > * Introduce option to pass “build reason” to a Koji build via koji cli
> > > and fedpkg/centpkg tooling.
> > >
> > > 2) Compatibility mode → Final
> > >
> > > * Implement support for the upgrade path on the rpm side in a
> > > compatible way. So that NV(R’=R+B) and NVRB are treated the same.
> > > * Remove %{build} part from Release tag and use independent Build tag
> > > for versioning.
> > > ================================================
> > >
> > > [1] https://github.com/rpm-software-management/rpm/discussions/2031
> >
> > I have couple concerns/questions.
> >
> > When modularity was introduced, local builds (outside of MSB) were side tracked
> > not to be part of the MVP. Implementing this later proved out to be rather
> > complicated. So let's focus on the following gotchas from the start here
> > instead please:
>
> I think these are very valid questions. I didn't think about them at
> first, but then added at the last moment under the name:
> "Cross-distribution upgrade paths" in the doc. But I like how you
> mention Copr and local builds as especially important cases of this,
> so I will adjust the wording.
>
> I don't have a ready to go answer, because I am not sure about the
> goal. How do we want this to work?
>
> Here are just some considerations.
>
> >
> > 1) An user rebuilds a package from Fedora dist-git in local mock, what will be
> > the value of the build tag? How will the local build sort over the official
> > Fedora builds?
>
> Afaik currently, if you do a local build, you need to bump an NVR to
> ensure the upgrade path is set. Then if the base distribution upgrades
> or rebuilds the package, you would need to bump your version again if
> you like to preserve the upgrade path.
>
> This approach would work exactly the same way after the change, as
> bumping the Release tag in the sources makes build-related versioning
> irrelevant.
>
> We can agree that by default Build tag in mock environment is unset.
> Local builds have no build id, and to make clean upgrades bump of
> Release tag is required.
>
> But there are some new possibilities:
>
> If you want to create a local rebuild without Release bump, you would
> be able to pass the build tag value manually and set to whatever you
> want. For example you can add 1 to existing build tag. Or you could
> set it to 0. Or to 99999999.
>
> The latter is less likely though.
>
> If I were to create local builds which are always higher or always
> lower than builds from the base system, I would probably play with the
> dist-tag, not build id:
>
> $ rpmdev-vercmp 0:1.2.3-12.fc37.15 0:1.2.3-12.localfc37.1
> 0:1.2.3-12.fc37.15 < 0:1.2.3-12.localfc37.1
>
> There is also a possibility to set default local build tag to be the
> local timestamp. But I am not sure if it is useful for a local
> development.
>
I think it makes sense to just set it to zero by default.
> > 2) A packager rebuilds packages from Fedora dist-git in a Copr repo, what will
> > be the value of the build tag? How will the Copr build sort over the official
> > Fedora builds? (This is essentially the same question but the answer might differ.)
>
> Similar as above, the questions would be how does it work now, do we
> want a change or do we want to keep the current setup?
>
> CC @Miroslav Suchý
>
I think we'll want to introduce sticky vendors to the Fedora ecosystem
to deal with this. Otherwise we'll have EVRB games between sources.
Users generally want packages coming from a particular source and
following that track makes sense.
DNF has the capability to do this if you set
"allow_vendor_change=False" in dnf.conf.
See: https://dnf.readthedocs.io/en/latest/conf_ref.html#allow-vendor-change-label
> > 3) If every Fedora packager can rebuild anything without a commit, what do we
> > do prevent accidental builds?
>
> I think each rebuild should be treated as a new package, thus it would
> require a new bodhi update, testing, and signing. Which means it will
> be less likely to accidentally ship it.
>
> But as I mentioned in the post, right now we'd like to propose the
> refactoring, and not a change of the development workflow. Thus I
> would initially restrict the rebuild possibility to the admin group
> which handles mass-rebuilds and other admin tasks. Then I would
> gradually open it up case by case, each case through a separate
> conversation.
>
> Saying that, probably the first case, which I would consider is: is
> there a problem of an accidental rebuild of a merged code for Fedora
> Rawhide? What would be the reasons for us to _not_ allow it?
>
The most compelling reason would be storage. Koji keeps the binaries
of every successful build. We need some policy changes to keep this
from turning into even more of a problem than it is now.
> >
> > As for your open questions, I think that build reason is useful as a changelog
> > message (and should be exposed to users).
> >
> > I also agree with others in this thread that exposing the build ID so
> > user-visibly isn't great.
> >
>
> I am not convinced about it. Technically we can configure filename to
> not show build-id at all.
> But then we will have a confusion, that upgrade path is not visible
> from a filename. And as it was mentioned elsewhere [1] we already have
> Epoch tag which works this way, and users hate it for this exact
> reason.
>
> I think that cleaner tags and filenames is a good goal.
> But I think the requirement "filename should contain all information
> which is required to calculate the upgrade path between two builds" is
> valid and probably should be included in the definition of that
> "cleanliness".
>
> [1] https://github.com/rpm-software-management/rpm/discussions/2031#discussio...
>
It's important for a non-obvious reason: it's pretty much the only way
to ensure when repodata is made, that multiple builds can live safely
in the same repository.
--
真実はいつも一つ！/ Always, there's only one truth!
9 months
Re: [RFC] Build tag in RPM: from NVR to NVRB
by Miro Hrončok
On 18. 06. 22 13:05, Aleksandra Fedorova wrote:
> Hi, all,
>
> I'd like to discuss how we can add Build tag in the RPM.
>
> As one of the key points is to turn it into a common standard for rpm
> packages across the ecosystem, the conversation is currently opened
> upstream [1] and in RHEL Engineering. And I'd like to get Fedora
> community on board.
>
> This is not a finalized proposal and I think it is not ready yet to be
> a Fedora Change.
> But I want to start a conversation and ask for opinions. There are
> also some open questions which need help, especially the requirements
> around build reason. And alternative suggestions are welcome as well.
>
> I've posted long version at
> https://discussion.fedoraproject.org/t/rfc-build-tag-in-rpms/39954
>
> You can comment there (simply login with your FAS id) or here on the
> mailing list.
> And I am going to update that post with the new feedback.
>
> Shorter version:
> ================================================
> We'd like to introduce Build Number/Tag/Id in the rpm metadata.
>
> By this change we would like to:
> * Provide a possibility to change build environment and rebuild rpm
> packages without changing their content: neither sources nor spec
> files.
> * Set a common standard for the RPM-based ecosystem, which can be used
> not just within Fedora, but also by Remixes, downstreams, SIGs and
> other distributions.
>
> The most visible impact of the proposal would be the filename change:
>
> Current: dnf-4.9.0-12.fc36.noarch.rpm
> Proposed: dnf-4.9.0-12.fc36.34897715.noarch.rpm
>
> It can be implemented in two steps:
>
> 1) Now → Compatibility mode
>
> * Introduce Build tag in the rpm metadata
> * Introduce “build reason” to be added to rpm changelog as a top entry
> * Enable passing Build tag value to the build in Koji. The value of
> the Build tag will be set from Koji build id.
> * Introduce macro in Release field of the rpm spec files, which adds
> Build tag after the usual disttag
> Release: 12.%{?dist}%{?build:.%{build}}
> * Introduce option to pass “build reason” to a Koji build via koji cli
> and fedpkg/centpkg tooling.
>
> 2) Compatibility mode → Final
>
> * Implement support for the upgrade path on the rpm side in a
> compatible way. So that NV(R’=R+B) and NVRB are treated the same.
> * Remove %{build} part from Release tag and use independent Build tag
> for versioning.
> ================================================
>
> [1] https://github.com/rpm-software-management/rpm/discussions/2031
I have couple concerns/questions.
When modularity was introduced, local builds (outside of MSB) were side tracked
not to be part of the MVP. Implementing this later proved out to be rather
complicated. So let's focus on the following gotchas from the start here
instead please:
1) An user rebuilds a package from Fedora dist-git in local mock, what will be
the value of the build tag? How will the local build sort over the official
Fedora builds?
2) A packager rebuilds packages from Fedora dist-git in a Copr repo, what will
be the value of the build tag? How will the Copr build sort over the official
Fedora builds? (This is essentially the same question but the answer might differ.)
3) If every Fedora packager can rebuild anything without a commit, what do we
do prevent accidental builds?
As for your open questions, I think that build reason is useful as a changelog
message (and should be exposed to users).
I also agree with others in this thread that exposing the build ID so
user-visibly isn't great.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
9 months
