python-devel May 2023

python-devel@lists.fedoraproject.org

11 participants
10 discussions

Guidance for a python package update
by Troy Curtis Jr 30 May '23

30 May '23

I'm currently working on the update for python-Levenshtein [1] and I have a question about the best way to proceed. In the intervening time since the last release we have packaged, a new maintainer has taken over maintenance, and built the updates around a stack of other packages (rapidfuzz-cpp, python-rapidfuzz, scikit-build, etc). Now that all these dependencies are available I can look to actually do the update. As part of the updates, the current maintainer has deprecated the original pypi name of "python-Levenshtein" in favor of the more typical name without the "python-" prefix. He is continuing to maintain the "python-Levenshtein" pypi package for compatibility, but now it is just an empty package which defines "Levenshtein" as a dependency [2]. This means the upstream for the Fedora python-Levenshtein package must point to the new Levenshtein package in order to get the actual package contents. However, this means the metadata for the package indicates the name "Levenshtein" which translate to a provide for "python3.11dist(levenshtein)", but that is different than the currently provided "python3.11dist(python-levenshtein)", which current dependencies require. One option would be to simply manually add a "python3.11dist(python-levenshtein)" provide, but it seems this seems specifically prohibited in the Python packaging guidelines [3]. I think there are 2 options: - Manually provide the legacy name outside of the python package meta-data - Rebuild all users of the provide to use the new name The main downside to the second approach is that nothing would provide the python-Levenshtein name, which is still present and usable via pypi and perfectly compatible with the current library implementation. So what is the appropriate course of action here? Thanks for the help, Troy 1: https://bugzilla.redhat.com/show_bug.cgi?id=1036607 2: https://github.com/maxbachmann/python-Levenshtein/blob/main/setup.cfg 3: https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Machine-r…

2 3

Bypassing setuptools_scm
by Orion Poplawski 28 May '23

28 May '23

Is there a canonical way to bypass setuptools_scm? Sometimes one would like to build from a github tarball (e.g. for test data) but then setuptool_scm fails to detect a version. -- Orion Poplawski he/him/his - surely the least important thing about me IT Systems Manager 720-772-5637 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane orion(a)nwra.com Boulder, CO 80301 https://www.nwra.com/

2 1

Re: [Fedora-packaging] Re: RPM 4.19: Dynamic subpackages with Python extras
by Igor Raits 23 May '23

23 May '23

Hey, On Mon, May 15, 2023, 22:04 Dan Čermák <dan.cermak(a)cgc-instruments.de> wrote: > Hi Igor, > > On May 15, 2023 6:24:07 PM UTC, Igor Raits <igor.raits(a)gmail.com> wrote: > >On Thu, Mar 30, 2023 at 11:56 PM Miro Hrončok <mhroncok(a)redhat.com> > wrote: > >> > >> Hello Python packagers. > >> > >> RPM 4.19 introduces this feature: > >> > >> https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html > >> > >> I decided to write this email to gather my thoughts. I believe that > with this, > >> we can turn manual Python extras subpackages like this: > > > >Is there a reason not to write a brp script so that users won't have > >to do anything manually at all? Basically scan the sitelibdir for the > >dist-info and create necessary parts from there? > > Don't the brp-scripts run after the build? I though the dynamic subpackage > creation has to occur beforehand. > I think brp script runs exactly before subpackages are created. > >> > >> %package -n python3-... > >> Summary: %{summary} > >> > >> %description -n python3-... %_description > >> > >> %pyproject_extras_subpkg -n python3-xxx extra1 extra2 > >> > >> (See > https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras > >> for what that means.) > >> > >> Into something like this: > >> > >> %package -n python3-... > >> Summary: %{summary} > >> > >> %description -n python3-... %_description > >> ... > >> > >> %install > >> %pyproject_install > >> ... > >> %pyproject_generate_extras_subpkgs -n python3-xxx > >> > >> > >> The %pyproject_generate_extras_subpkgs macro would parse the installed > >> .dist-info directory to find out what extras are available and generate > >> subpackages for all of them. > >> > >> (Obviously, the macro name is open up for discussion.) > >> > >> ---------------- > >> > >> An API would be required to exclude extras: > >> > >> - that are not useful for other packages > >> (for example build/development requirements, commonly named dev, doc > or test) > >> - that have requirements that are not packaged in Fedora > >> > >> For example (mimicking the API of %pyproject_check_import): > >> > >> %pyproject_generate_extras_subpkgs -n python3-xxx -e test -e > 'nonfree*' > >> > >> ---------------- > >> > >> However, extras are also currently manually passed to > %pyproject_buildrequires: > >> > >> %generate_buildrequires > >> %pyproject_buildrequires -x extra1 -x extra2 -x test > >> > >> It should already be possible to implement automatic extras discovery in > >> %pyproject_buildrequires with older RPM versions and allow it to be > used this way: > >> > >> %generate_buildrequires > >> %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*' > >> > >> RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS> > can > >> either be -x '*' (if we start treating -x values as globs, which is > backwards > >> compatible and probably generally useful), or a single-letter switch > such as -a > >> (but honestly we are running out of meaningful letters). > >> > >> (When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied. > However, > >> an explicit form needs to exist for packages that don't need to exclude > any > >> extras at all.) > >> > >> > >> Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default, > once RPM > >> 4.19 is omnipresent. > >> > >> ---------------- > >> > >> Combined, this would mean that the packager needs to: > >> > >> 1. specify extras that are not supposed to be used as BRs > >> 2. specify extras that are not supposed to be packaged > >> > >> In the ideal word (2) is a superset of (1). > >> > >> Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from > >> %pyproject_buildrequires? > >> > >> When a package has extra1, extra2, nonfree1, nonfree2 and test extras, > one > >> could do: > >> > >> %generate_buildrequires > >> %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*' > >> > >> ... > >> > >> %pyproject_install > >> ... > >> %pyproject_generate_extras_subpkgs -X test > >> > >> That would mean: > >> > >> - extra1 is BRed and packaged > >> - extra2 is BRed and packaged > >> - test is BRed but not packaged > >> - nonfree1 is neither > >> - nonfree2 is neither > >> > >> ---------------- > >> > >> Alternatively the information could be supplied by %globals: > >> > >> %global _python_ignored_extras nonfree* > >> %global _python_unpackaged_extras test > >> > >> However, I somehow dislike this approach. > >> > >> ---------------- > >> > >> I'd appreciate your thoughts on the matter. > >> > >> -- > >> Miro Hrončok > >> -- > >> Phone: +420777974800 > >> IRC: mhroncok > >> _______________________________________________ > >> packaging mailing list -- packaging(a)lists.fedoraproject.org > >> To unsubscribe send an email to packaging-leave(a)lists.fedoraproject.org > >> Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > >> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > >> List Archives: > https://lists.fedoraproject.org/archives/list/packaging@lists.fedoraproject… > >> Do not reply to spam, report it: > https://pagure.io/fedora-infrastructure/new_issue > > > > > > >

1 0

Bringing PyTorch to Fedora
by Kaitlyn Abdo 18 May '23

18 May '23

Hello everyone, I hope this email finds everyone well. I’m here to introduce myself but also ask for anyone who would be interested in helping me out on a fun side project that me and my colleague, Teng, have planned. I’m Kaitlyn Abdo and I’m an Associate Technical Marketing Manager at Red Hat, specifically for the AI/ML products and offerings. Teng is a Principal Technical Product Manager at Red Hat as well, but I’ll let Teng formally introduce himself :). To give the TLDR, we want to package PyTorch for Fedora. We are looking for individuals in the community who are interested in making this happen with us. We will be hosting most of the discussion moving forward on the AI/ML discussion page (https://discussion.fedoraproject.org/tag/ai-ml-sig) on Fedora Discussion, but we will still send out major communication through the mailing lists. We are aware of several existing SIGs (ML SIG-https://fedoraproject.org/wiki/SIGs/ML, Heterogeneous Computing Special Interest SIG-https://fedoraproject.org/wiki/SIGs/HC, Big Data SIG-https://fedoraproject.org/wiki/SIGs/bigdata) and would like to help reignite activity in a coordinated way. I talked to the leadership of the ML SIG about revamping and possibly merging other less active SIGs to create a broader AI/ML SIG. The ML SIG leadership agreed that this could be beneficial for communication and project purposes, so I’m open to input from both leadership and individuals in other SIGs. Please reach out to either me at kabdo(a)redhat.com or Teng at tema(a)redhat.com if you are interested. We are in the very early stages of this and want to build a team of anyone who would be interested. Thank you for your time and we look forward to hearing from you! Best, Kaitlyn Abdo

1 0

RPM 4.19: Dynamic subpackages with Python extras
by Miro Hrončok 17 May '23

17 May '23

Hello Python packagers. RPM 4.19 introduces this feature: https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this: %package -n python3-... Summary: %{summary} %description -n python3-... %_description %pyproject_extras_subpkg -n python3-xxx extra1 extra2 (See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras for what that means.) Into something like this: %package -n python3-... Summary: %{summary} %description -n python3-... %_description ... %install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them. (Obviously, the macro name is open up for discussion.) ---------------- An API would be required to exclude extras: - that are not useful for other packages (for example build/development requirements, commonly named dev, doc or test) - that have requirements that are not packaged in Fedora For example (mimicking the API of %pyproject_check_import): %pyproject_generate_extras_subpkgs -n python3-xxx -e test -e 'nonfree*' ---------------- However, extras are also currently manually passed to %pyproject_buildrequires: %generate_buildrequires %pyproject_buildrequires -x extra1 -x extra2 -x test It should already be possible to implement automatic extras discovery in %pyproject_buildrequires with older RPM versions and allow it to be used this way: %generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*' RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS> can either be -x '*' (if we start treating -x values as globs, which is backwards compatible and probably generally useful), or a single-letter switch such as -a (but honestly we are running out of meaningful letters). (When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied. However, an explicit form needs to exist for packages that don't need to exclude any extras at all.) Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default, once RPM 4.19 is omnipresent. ---------------- Combined, this would mean that the packager needs to: 1. specify extras that are not supposed to be used as BRs 2. specify extras that are not supposed to be packaged In the ideal word (2) is a superset of (1). Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from %pyproject_buildrequires? When a package has extra1, extra2, nonfree1, nonfree2 and test extras, one could do: %generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*' ... %pyproject_install ... %pyproject_generate_extras_subpkgs -X test That would mean: - extra1 is BRed and packaged - extra2 is BRed and packaged - test is BRed but not packaged - nonfree1 is neither - nonfree2 is neither ---------------- Alternatively the information could be supplied by %globals: %global _python_ignored_extras nonfree* %global _python_unpackaged_extras test However, I somehow dislike this approach. ---------------- I'd appreciate your thoughts on the matter. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

4 4

python3.11-rpm to EPEL 9
by Miro Hrončok 17 May '23

17 May '23

Hello, I'm working on adding python3.11-rpm to EPEL 9 and EPEL 9 Next. See https://src.fedoraproject.org/rpms/python3-rpm/pull-request/3 and 4. I decided to reuse the python3-rpm component (currently epel7 only). Let me know if I should not. If there is a significant demand, I can try add this (and python39-rpm) to EPEL 8 as well. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 4

pyproject-rpm-macros 1.8.0: self-referential extras, %{pyproject_build_lib} deprecated
by Miro Hrončok 16 May '23

16 May '23

Hello Pythonistas, I've built pyproject-rpm-macros-1.8.0-1.fc39 in Rawhide and updates are available for older Fedoras. Later on I plan to sync this to c9s as well. https://bodhi.fedoraproject.org/updates/pyproject-rpm-macros The release brings a bugfix/enhancement (depending on how you look at it) and a deprecation. ## %pyproject_buildrequires now supports self-referential extras requirements That is, if you package e.g. attrs and use: %pyproject_buildrequires -x tests It correctly handles the requirement on attrs[tests-no-zope] as defined here: https://github.com/python-attrs/attrs/blob/23.1.0/pyproject.toml#L45 When a self-referential dependency on extras is found, the required extra is added to the list of desired extras, and all the previously discarded "alien" requirements are reevaluated using the new extras list. Example use: https://src.fedoraproject.org/rpms/python-fiona/pull-request/2 ## The provisional %{pyproject_build_lib} macro was deprecated There is no scheduled removal, but it may break entirely in the future. See https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproj… for motivation. Packagers are encouraged to use $PWD/build/lib.%{python3_platform}-cpython-%{python3_version_nodots} instead if they need to (works on all current Fedoras but not on RHEL 9's Python 3.9). The following Fedora Rawhide packages use it: asv buku ocrmypdf pytest python-Automat python-constantly python-cppy python-cypari2 python-cysignals python-elasticsearch python-graphviz python-hyperframe python-hyperlink python-h2 python-libdiscid python-mistune python-mplcursors python-nb2plots python-networkx python-nipy python-numcodecs python-opentracing python-pbr python-persistent python-pikepdf python-pplpy python-primecountpy python-priority python-pybtex python-pyedflib python-pytest-regressions python-service-identity python-simplejson python-stestr python-transforms3d python-twisted -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Package naming for vkbasalt-cli
by Sandro 16 May '23

16 May '23

Hi, I need to package vkbasalt-cli [1] as a dependency for Bottles. Bottles will use the Python library, but vkbasalt-cli also provides a command for independent/direct usage. My plan is to name the Fedora package 'vkbasalt-cli' in line with upstream and add a virtual provide for 'python3-vkbasalt-cli'. Could someone confirm that this is the right approach for vkbasalt-cli? Thanks in advance, -- Sandro FAS: gui1ty IRC: Penguinpee Elsewhere: [Pp]enguinpee

4 12

How to drop 32bit support from the scientific Python stack
by Miro Hrončok 04 May '23

04 May '23

Hello folks, Couple months ago, we discussed with @psimovec if it's possible to ExcludeArch i686 from scipy. This Python discussion brought the topic back: https://discuss.python.org/t/dropping-32-bit-packages/5476/9 I've tried to see how many packages would be affected and the short answer is: all of them, unless we break the dependency chains somewhere. There are many deep transitive build dependency chains on scipy, but the perhaps most interesting one is: (everything) <- rpm <- rust-rpm-sequoia <- rust-packaging <- pytest <- python-hypothesis <- python-pandas <- scipy Another interesting fact is that the majority of the dependency chains I found traverses through a path of noarch Python packages (such as pytest). If scipy drops i686 we would need to conditionalize the test-related BuildRequires of such noarch packages -- unfortunately that would mean that based on randomness (noarch packages are built on random archotcture), not all tests would always run and we could potentially ship packages that are broken / untested properly. And if we don't conditionalize the dependencies, the packages would randomly fail to build. An alternative is to make all the related packages archful with noarch python3-xxx subpackages -- which would be quite tedious and wasteful resource-wise. The best way forward for this use case (and many others that will show up sooner or later) would be to stop building noarch packages on i686. That way, only archful packages that (Build)Require scipy would need to be changed. Would that be possible? -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Numpy version pinned despite numpy>=1.22.3
by Sandro 01 May '23

01 May '23

Hi, I'm trying to build/update a package using mesonpy [1]. While going through the build process, exploring how the existing macros behave together with mesonpy, I came across a weird build error, for which I have not yet found a cause or explanation. The build fails with [2]: No matching package to install: 'python3dist(numpy) = 1.23.2' The version in rawhide is 1.24.3, yet the requirement in pyproject.toml [3] is: "numpy>=1.22.3" So, why is it insisting on using 1.23.2 while the existing 1.24.3 satisfies the requirement? There's one thing I noticed [4], which I believe is unrelated. But I asked upstream for clarification anyway. The spec file is currently work in progress. I simply triggered a Copr build, so I could provide links here and verify that this is not a local issue in my build environment. [1] So fresh it's still warm: https://src.fedoraproject.org/rpms/python-meson-python [2] https://download.copr.fedorainfracloud.org/results/gui1ty/neuro-sig/fedora-… [3] https://github.com/has2k1/scikit-misc/blob/8f829888ecf0fa96eb18aeb3f723fc98… [4] https://github.com/has2k1/scikit-misc/issues/24 Cheers, -- Sandro FAS: gui1ty IRC: Penguinpee Elsewhere: [Pp]enguinpee

3 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel May 2023