Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html
I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras for what that means.)
Into something like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description ...
%install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them.
(Obviously, the macro name is open up for discussion.)
----------------
An API would be required to exclude extras:
- that are not useful for other packages (for example build/development requirements, commonly named dev, doc or test) - that have requirements that are not packaged in Fedora
For example (mimicking the API of %pyproject_check_import):
%pyproject_generate_extras_subpkgs -n python3-xxx -e test -e 'nonfree*'
----------------
However, extras are also currently manually passed to %pyproject_buildrequires:
%generate_buildrequires %pyproject_buildrequires -x extra1 -x extra2 -x test
It should already be possible to implement automatic extras discovery in %pyproject_buildrequires with older RPM versions and allow it to be used this way:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS> can either be -x '*' (if we start treating -x values as globs, which is backwards compatible and probably generally useful), or a single-letter switch such as -a (but honestly we are running out of meaningful letters).
(When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied. However, an explicit form needs to exist for packages that don't need to exclude any extras at all.)
Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default, once RPM 4.19 is omnipresent.
----------------
Combined, this would mean that the packager needs to:
1. specify extras that are not supposed to be used as BRs 2. specify extras that are not supposed to be packaged
In the ideal word (2) is a superset of (1).
Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from %pyproject_buildrequires?
When a package has extra1, extra2, nonfree1, nonfree2 and test extras, one could do:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
...
%pyproject_install ... %pyproject_generate_extras_subpkgs -X test
That would mean:
- extra1 is BRed and packaged - extra2 is BRed and packaged - test is BRed but not packaged - nonfree1 is neither - nonfree2 is neither
----------------
Alternatively the information could be supplied by %globals:
%global _python_ignored_extras nonfree* %global _python_unpackaged_extras test
However, I somehow dislike this approach.
----------------
I'd appreciate your thoughts on the matter.
On 30. 03. 23 23:55, Miro Hrončok wrote:
Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html
I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras for what that means.)
Into something like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description ...
%install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them.
One challenge: When an extra is removed upstream, the packager might not notice that it needs to be provided/obsoleted/etc.
Hi Miro,
On March 31, 2023 10:06:14 AM UTC, "Miro Hrončok" mhroncok@redhat.com wrote:
On 30. 03. 23 23:55, Miro Hrončok wrote:
Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html
I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras for what that means.)
Into something like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description ...
%install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them.
One challenge: When an extra is removed upstream, the packager might not notice that it needs to be provided/obsole>
That is indeed a problem. What if we made the macro require the list of subpackages (e.g. as a comma separated string) as an argument? The if the packager wouldn't notice the removal, they'd get a build error.
Cheers,
Dan
On 04. 04. 23 19:24, Dan Čermák wrote:
Hi Miro,
On March 31, 2023 10:06:14 AM UTC, "Miro Hrončok" mhroncok@redhat.com wrote:
On 30. 03. 23 23:55, Miro Hrončok wrote:
Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html
I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras for what that means.)
Into something like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description ...
%install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them.
One challenge: When an extra is removed upstream, the packager might not notice that it needs to be provided/obsole>
That is indeed a problem. What if we made the macro require the list of subpackages (e.g. as a comma separated string) as an argument? The if the packager wouldn't notice the removal, they'd get a build error.
We already have that today without dynamic subpackages, because when you know the list ahead, you can "generate" subpackages already.
$ rpm --eval '%pyproject_extras_subpkg -n python3-... extra1 extra2' %package -n python3-...+extra1 Summary: Metapackage for python3-...: extra1 extras Requires: python3-... = %{version}-%{release} %description -n python3-...+extra1 This is a metapackage bringing in extra1 extras requires for python3-.... It makes sure the dependencies are installed.
%files -n python3-...+extra1 -f .../BUILD/%{name}-%{version}-%{release}.x86_64-pyproject-ghost-distinfo
%package -n python3-...+extra2 Summary: Metapackage for python3-...: extra2 extras Requires: python3-... = %{version}-%{release} %description -n python3-...+extra2 This is a metapackage bringing in extra2 extras requires for python3-.... It makes sure the dependencies are installed.
%files -n python3-...+extra2 -f .../%{name}-%{version}-%{release}.x86_64-pyproject-ghost-distinfo
On Thu, Mar 30, 2023 at 11:56 PM Miro Hrončok mhroncok@redhat.com wrote:
Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html
I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras for what that means.)
Into something like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description ...
%install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them.
(Obviously, the macro name is open up for discussion.)
Overall this changes the concept of manually opting-in to extras to manually opting-out. I have a slight preference over the current status quo, as the packager needs to be aware of the package's extras and modify the SPEC accordingly, whereas in the event of implementing an opt-out approach, things could easily go in the way of enabling everything till something breaks. At the moment, when working on a package, I am choosing which extras to enable, leading me to also search about requirements, purpose of each extra etc.
Being able to enable everything and exclude what I don't need, would lead to the path of least resistance which is basically to enable everything and not think about potential issues (as there wouldn't be any at this point). Now I can imagine "fail to install" bugzilla's over various extras with the extra caveat that in order to fix a package we would require to carry the Obsoletes tag in the SPEC for two releases. Coupled with the fact that an auto-generated list of extras makes difficult to figure out if an extra has been removed in order to fix it on the package level makes this approach less than ideal in my head.
Of course not all packages have extras and one can reasonably argue that packagers who are utilizing the %pyproject_extras_subpkg macro would be well aware of the implications if they change to the new macro.
While this change would definitely make things easier for packagers who'd like a cleaner SPEC (and continuing on the path that pyproject macros have set to align the upstream metadata with the rpm generated metadata) I believe it can be more error prone than the current approach and the potential risks outweigh the benefits. I'm happy to be proven wrong of course.
----------------
An API would be required to exclude extras:
- that are not useful for other packages
(for example build/development requirements, commonly named dev, doc or test)
- that have requirements that are not packaged in Fedora
For example (mimicking the API of %pyproject_check_import):
%pyproject_generate_extras_subpkgs -n python3-xxx -e test -e 'nonfree*'
However, extras are also currently manually passed to %pyproject_buildrequires:
%generate_buildrequires %pyproject_buildrequires -x extra1 -x extra2 -x test
It should already be possible to implement automatic extras discovery in %pyproject_buildrequires with older RPM versions and allow it to be used this way:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS> can either be -x '*' (if we start treating -x values as globs, which is backwards compatible and probably generally useful), or a single-letter switch such as -a (but honestly we are running out of meaningful letters).
(When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied. However, an explicit form needs to exist for packages that don't need to exclude any extras at all.)
Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default, once RPM 4.19 is omnipresent.
Combined, this would mean that the packager needs to:
- specify extras that are not supposed to be used as BRs
- specify extras that are not supposed to be packaged
In the ideal word (2) is a superset of (1).
Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from %pyproject_buildrequires?
When a package has extra1, extra2, nonfree1, nonfree2 and test extras, one could do:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
...
%pyproject_install ... %pyproject_generate_extras_subpkgs -X test
That would mean:
- extra1 is BRed and packaged
- extra2 is BRed and packaged
- test is BRed but not packaged
- nonfree1 is neither
- nonfree2 is neither
Your thoughts on the implementation are quite sound and consistent, I don't really have any comments on that as you have taken all the possibilities into account and provided examples. If those thoughts move forward with an implementation I'll provide more comprehensive feedback on that.
Also %pyproject_generate_extras_subpkgs somehow inheriting the excluded extras from %pyproject_buildrequires is reasonable, I dislike the archaic %global approach as well.
----------------
Alternatively the information could be supplied by %globals:
%global _python_ignored_extras nonfree* %global _python_unpackaged_extras test
However, I somehow dislike this approach.
I'd appreciate your thoughts on the matter.
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On 5/15/23 19:33, Charalampos Stratakis wrote:
On Thu, Mar 30, 2023 at 11:56 PM Miro Hrončok <mhroncok@redhat.com mailto:mhroncok@redhat.com> wrote:
Hello Python packagers. RPM 4.19 introduces this feature: https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html <https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html> I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this: %package -n python3-... Summary: %{summary} %description -n python3-... %_description %pyproject_extras_subpkg -n python3-xxx extra1 extra2 (See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras <https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras> for what that means.) Into something like this: %package -n python3-... Summary: %{summary} %description -n python3-... %_description ... %install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them. (Obviously, the macro name is open up for discussion.)Overall this changes the concept of manually opting-in to extras to manually opting-out. I have a slight preference over the current status quo, as the packager needs to be aware of the package's extras and modify the SPEC accordingly, whereas in the event of implementing an opt-out approach, things could easily go in the way of enabling everything till something breaks. At the moment, when working on a package, I am choosing which extras to enable, leading me to also search about requirements, purpose of each extra etc.
Being able to enable everything and exclude what I don't need, would lead to the path of least resistance which is basically to enable everything and not think about potential issues (as there wouldn't be any at this point). Now I can imagine "fail to install" bugzilla's over various extras with the extra caveat that in order to fix a package we would require to carry the Obsoletes tag in the SPEC for two releases. Coupled with the fact that an auto-generated list of extras makes difficult to figure out if an extra has been removed in order to fix it on the package level makes this approach less than ideal in my head.
I like the gain of automation and I have similar worries that it would be too easy to let packages slip from our radar. We could employ automation to report to us when an extra disappears.
Currently, a missing extra is reported via `lostpayload` and `subpackages` inspections of rpminspect run by Zuul CI which end up in the VERIFY state. (example: https://fedora.softwarefactory-project.io/logs//27/27/7967d00bcc29ea4dcb8520...)
Maybe rpminspect could extend the `subpackages` inspection to behave differently: - report OK if the subpackage is properly Obsoleted - report BAD if not - fail the entire rpminspect status
This however requires to run rpminspect/Zuul on the package update by opening a pull request. We enable Zuul CI it for all packages added to the python-packagers-sig, but many updates land in the branches via direct push.
Of course not all packages have extras and one can reasonably argue that packagers who are utilizing the %pyproject_extras_subpkg macro would be well aware of the implications if they change to the new macro.
While this change would definitely make things easier for packagers who'd like a cleaner SPEC (and continuing on the path that pyproject macros have set to align the upstream metadata with the rpm generated metadata) I believe it can be more error prone than the current approach and the potential risks outweigh the benefits. I'm happy to be proven wrong of course.
An API would be required to exclude extras: - that are not useful for other packages (for example build/development requirements, commonly named dev, doc or test) - that have requirements that are not packaged in Fedora For example (mimicking the API of %pyproject_check_import): %pyproject_generate_extras_subpkgs -n python3-xxx -e test -e 'nonfree*' ---------------- However, extras are also currently manually passed to %pyproject_buildrequires: %generate_buildrequires %pyproject_buildrequires -x extra1 -x extra2 -x test It should already be possible to implement automatic extras discovery in %pyproject_buildrequires with older RPM versions and allow it to be used this way: %generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*' RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS> can either be -x '*' (if we start treating -x values as globs, which is backwards compatible and probably generally useful), or a single-letter switch such as -a (but honestly we are running out of meaningful letters). (When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied. However, an explicit form needs to exist for packages that don't need to exclude any extras at all.) Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default, once RPM 4.19 is omnipresent. ---------------- Combined, this would mean that the packager needs to: 1. specify extras that are not supposed to be used as BRs 2. specify extras that are not supposed to be packaged In the ideal word (2) is a superset of (1). Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from %pyproject_buildrequires? When a package has extra1, extra2, nonfree1, nonfree2 and test extras, one could do: %generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*' ... %pyproject_install ... %pyproject_generate_extras_subpkgs -X test That would mean: - extra1 is BRed and packaged - extra2 is BRed and packaged - test is BRed but not packaged - nonfree1 is neither - nonfree2 is neitherYour thoughts on the implementation are quite sound and consistent, I don't really have any comments on that as you have taken all the possibilities into account and provided examples. If those thoughts move forward with an implementation I'll provide more comprehensive feedback on that. Also %pyproject_generate_extras_subpkgs somehow inheriting the excluded extras from %pyproject_buildrequires is reasonable, I dislike the archaic %global approach as well.
---------------- Alternatively the information could be supplied by %globals: %global _python_ignored_extras nonfree* %global _python_unpackaged_extras test However, I somehow dislike this approach. ---------------- I'd appreciate your thoughts on the matter. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org <mailto:packaging@lists.fedoraproject.org> To unsubscribe send an email to packaging-leave@lists.fedoraproject.org <mailto:packaging-leave@lists.fedoraproject.org> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ <https://docs.fedoraproject.org/en-US/project/code-of-conduct/> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines <https://fedoraproject.org/wiki/Mailing_list_guidelines> List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.org <https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.org> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue <https://pagure.io/fedora-infrastructure/new_issue>-- Regards,
Charalampos Stratakis Senior Software Engineer Python Maintenance Team, Red Hat
packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
On Thu, Mar 30, 2023 at 11:56 PM Miro Hrončok mhroncok@redhat.com wrote:
Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html
I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this:
Is there a reason not to write a brp script so that users won't have to do anything manually at all? Basically scan the sitelibdir for the dist-info and create necessary parts from there?
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras for what that means.)
Into something like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description ...
%install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them.
(Obviously, the macro name is open up for discussion.)
An API would be required to exclude extras:
- that are not useful for other packages
(for example build/development requirements, commonly named dev, doc or test)
- that have requirements that are not packaged in Fedora
For example (mimicking the API of %pyproject_check_import):
%pyproject_generate_extras_subpkgs -n python3-xxx -e test -e 'nonfree*'
However, extras are also currently manually passed to %pyproject_buildrequires:
%generate_buildrequires %pyproject_buildrequires -x extra1 -x extra2 -x test
It should already be possible to implement automatic extras discovery in %pyproject_buildrequires with older RPM versions and allow it to be used this way:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS> can either be -x '*' (if we start treating -x values as globs, which is backwards compatible and probably generally useful), or a single-letter switch such as -a (but honestly we are running out of meaningful letters).
(When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied. However, an explicit form needs to exist for packages that don't need to exclude any extras at all.)
Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default, once RPM 4.19 is omnipresent.
Combined, this would mean that the packager needs to:
- specify extras that are not supposed to be used as BRs
- specify extras that are not supposed to be packaged
In the ideal word (2) is a superset of (1).
Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from %pyproject_buildrequires?
When a package has extra1, extra2, nonfree1, nonfree2 and test extras, one could do:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
...
%pyproject_install ... %pyproject_generate_extras_subpkgs -X test
That would mean:
- extra1 is BRed and packaged
- extra2 is BRed and packaged
- test is BRed but not packaged
- nonfree1 is neither
- nonfree2 is neither
Alternatively the information could be supplied by %globals:
%global _python_ignored_extras nonfree* %global _python_unpackaged_extras test
However, I somehow dislike this approach.
I'd appreciate your thoughts on the matter.
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Hi Igor,
On May 15, 2023 6:24:07 PM UTC, Igor Raits igor.raits@gmail.com wrote:
On Thu, Mar 30, 2023 at 11:56 PM Miro Hrončok mhroncok@redhat.com wrote:
Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html
I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this:
Is there a reason not to write a brp script so that users won't have to do anything manually at all? Basically scan the sitelibdir for the dist-info and create necessary parts from there?
Don't the brp-scripts run after the build? I though the dynamic subpackage creation has to occur beforehand.
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras for what that means.)
Into something like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description ...
%install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them.
(Obviously, the macro name is open up for discussion.)
An API would be required to exclude extras:
- that are not useful for other packages
(for example build/development requirements, commonly named dev, doc or test)
- that have requirements that are not packaged in Fedora
For example (mimicking the API of %pyproject_check_import):
%pyproject_generate_extras_subpkgs -n python3-xxx -e test -e 'nonfree*'
However, extras are also currently manually passed to %pyproject_buildrequires:
%generate_buildrequires %pyproject_buildrequires -x extra1 -x extra2 -x test
It should already be possible to implement automatic extras discovery in %pyproject_buildrequires with older RPM versions and allow it to be used this way:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS> can either be -x '*' (if we start treating -x values as globs, which is backwards compatible and probably generally useful), or a single-letter switch such as -a (but honestly we are running out of meaningful letters).
(When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied. However, an explicit form needs to exist for packages that don't need to exclude any extras at all.)
Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default, once RPM 4.19 is omnipresent.
Combined, this would mean that the packager needs to:
- specify extras that are not supposed to be used as BRs
- specify extras that are not supposed to be packaged
In the ideal word (2) is a superset of (1).
Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from %pyproject_buildrequires?
When a package has extra1, extra2, nonfree1, nonfree2 and test extras, one could do:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
...
%pyproject_install ... %pyproject_generate_extras_subpkgs -X test
That would mean:
- extra1 is BRed and packaged
- extra2 is BRed and packaged
- test is BRed but not packaged
- nonfree1 is neither
- nonfree2 is neither
Alternatively the information could be supplied by %globals:
%global _python_ignored_extras nonfree* %global _python_unpackaged_extras test
However, I somehow dislike this approach.
I'd appreciate your thoughts on the matter.
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject.... Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue
Hey,
On Mon, May 15, 2023, 22:04 Dan Čermák dan.cermak@cgc-instruments.de wrote:
Hi Igor,
On May 15, 2023 6:24:07 PM UTC, Igor Raits igor.raits@gmail.com wrote:
On Thu, Mar 30, 2023 at 11:56 PM Miro Hrončok mhroncok@redhat.com
wrote:
Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html
I decided to write this email to gather my thoughts. I believe that
with this,
we can turn manual Python extras subpackages like this:
Is there a reason not to write a brp script so that users won't have to do anything manually at all? Basically scan the sitelibdir for the dist-info and create necessary parts from there?
Don't the brp-scripts run after the build? I though the dynamic subpackage creation has to occur beforehand.
I think brp script runs exactly before subpackages are created.
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See
https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras
for what that means.)
Into something like this:
%package -n python3-... Summary: %{summary}
%description -n python3-... %_description ...
%install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them.
(Obviously, the macro name is open up for discussion.)
An API would be required to exclude extras:
- that are not useful for other packages
(for example build/development requirements, commonly named dev, doc
or test)
- that have requirements that are not packaged in Fedora
For example (mimicking the API of %pyproject_check_import):
%pyproject_generate_extras_subpkgs -n python3-xxx -e test -e
'nonfree*'
However, extras are also currently manually passed to
%pyproject_buildrequires:
%generate_buildrequires %pyproject_buildrequires -x extra1 -x extra2 -x test
It should already be possible to implement automatic extras discovery in %pyproject_buildrequires with older RPM versions and allow it to be
used this way:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS>
can
either be -x '*' (if we start treating -x values as globs, which is
backwards
compatible and probably generally useful), or a single-letter switch
such as -a
(but honestly we are running out of meaningful letters).
(When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied.
However,
an explicit form needs to exist for packages that don't need to exclude
any
extras at all.)
Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default,
once RPM
4.19 is omnipresent.
Combined, this would mean that the packager needs to:
- specify extras that are not supposed to be used as BRs
- specify extras that are not supposed to be packaged
In the ideal word (2) is a superset of (1).
Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from %pyproject_buildrequires?
When a package has extra1, extra2, nonfree1, nonfree2 and test extras,
one
could do:
%generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
...
%pyproject_install ... %pyproject_generate_extras_subpkgs -X test
That would mean:
- extra1 is BRed and packaged
- extra2 is BRed and packaged
- test is BRed but not packaged
- nonfree1 is neither
- nonfree2 is neither
Alternatively the information could be supplied by %globals:
%global _python_ignored_extras nonfree* %global _python_unpackaged_extras test
However, I somehow dislike this approach.
I'd appreciate your thoughts on the matter.
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok _______________________________________________ packaging mailing list -- packaging@lists.fedoraproject.org To unsubscribe send an email to packaging-leave@lists.fedoraproject.org Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:
https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject....
Do not reply to spam, report it:
packaging@lists.fedoraproject.org
-
Charalampos Stratakis -
Dan Čermák -
Igor Raits -
Karolina Surma -
Miro Hrončok