python-devel March 2023

python-devel@lists.fedoraproject.org

6 participants
5 discussions

RPM 4.19: Dynamic subpackages with Python extras
by Miro Hrončok 17 May '23

17 May '23

Hello Python packagers. RPM 4.19 introduces this feature: https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html I decided to write this email to gather my thoughts. I believe that with this, we can turn manual Python extras subpackages like this: %package -n python3-... Summary: %{summary} %description -n python3-... %_description %pyproject_extras_subpkg -n python3-xxx extra1 extra2 (See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras for what that means.) Into something like this: %package -n python3-... Summary: %{summary} %description -n python3-... %_description ... %install %pyproject_install ... %pyproject_generate_extras_subpkgs -n python3-xxx The %pyproject_generate_extras_subpkgs macro would parse the installed .dist-info directory to find out what extras are available and generate subpackages for all of them. (Obviously, the macro name is open up for discussion.) ---------------- An API would be required to exclude extras: - that are not useful for other packages (for example build/development requirements, commonly named dev, doc or test) - that have requirements that are not packaged in Fedora For example (mimicking the API of %pyproject_check_import): %pyproject_generate_extras_subpkgs -n python3-xxx -e test -e 'nonfree*' ---------------- However, extras are also currently manually passed to %pyproject_buildrequires: %generate_buildrequires %pyproject_buildrequires -x extra1 -x extra2 -x test It should already be possible to implement automatic extras discovery in %pyproject_buildrequires with older RPM versions and allow it to be used this way: %generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*' RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS> can either be -x '*' (if we start treating -x values as globs, which is backwards compatible and probably generally useful), or a single-letter switch such as -a (but honestly we are running out of meaningful letters). (When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied. However, an explicit form needs to exist for packages that don't need to exclude any extras at all.) Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default, once RPM 4.19 is omnipresent. ---------------- Combined, this would mean that the packager needs to: 1. specify extras that are not supposed to be used as BRs 2. specify extras that are not supposed to be packaged In the ideal word (2) is a superset of (1). Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from %pyproject_buildrequires? When a package has extra1, extra2, nonfree1, nonfree2 and test extras, one could do: %generate_buildrequires %pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*' ... %pyproject_install ... %pyproject_generate_extras_subpkgs -X test That would mean: - extra1 is BRed and packaged - extra2 is BRed and packaged - test is BRed but not packaged - nonfree1 is neither - nonfree2 is neither ---------------- Alternatively the information could be supplied by %globals: %global _python_ignored_extras nonfree* %global _python_unpackaged_extras test However, I somehow dislike this approach. ---------------- I'd appreciate your thoughts on the matter. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

4 4

Let's deprecate the %{pyproject_build_lib} macro... ?
by Miro Hrončok 18 Apr '23

18 Apr '23

Hello. The %{pyproject_build_lib} macro is a *provisional* macro from pyproject-rpm-macros. It was added to solve the problem of out-of-tree pip builds. From the pyproject-rpm-macros README: """ Sometimes, it is desired to be able to import the just-built extension modules in the %build section, e.g. to build the documentation with Sphinx. %build %pyproject_wheel ... build the docs here ... With pure Python packages, it might be possible to set PYTHONPATH=${PWD} or PYTHONPATH=${PWD}/src. However, it is a bit more complicated with extension modules. The location of just-built modules might differ depending on Python version, architecture, pip version, etc. Hence, the macro %{pyproject_build_lib} exists to be used like this: %build %pyproject_wheel PYTHONPATH=%{pyproject_build_lib} ... build the docs here ... """ When this macro was introduced, the built extension module lived in a folder like /builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-xxxxxxxx/build/lib.linux-x86_64-3.10, hence it made sense to include in the pyproject-rpm-macros and call it %pyproject_something. ------------------------------------------------------------ Today, on Rawhide, pip default to in-tree-builds and the value for extension modules will most likely be: $PWD/build/lib.%{python3_platform}-cpython-%{python3_version_nodots} And for pure Python: $PWD/build/lib And it turns out, the value is setuptools-specific, see https://bugzilla.redhat.com/show_bug.cgi?id=2176393 Other build backends might store the built extension modules elsewhere or simply pack the pure Python files into wheel directly from the source tree. ------------------------------------------------------------ In Rawhide, 41 packages use PYTHONPATH="%{pyproject_build_lib}" and 2 packages have a comment with %%{pyproject_build_lib} in them. Considering the macro - is not build-backend-agnostic, and - is not so much needed as it once was I propose we deprecate it with no further fixes going in, but no scheduled removal. The 41 packages can be fixed once Fedora 36 goes EOL by expanding the macro to the values above, or if desired, we could macronize this in pythohn3-setuptools as %{setuptools_build_lib}. One problem is that the macro is unfortunately still needed on EL 9. Thoughts? -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 2

Anyone interested in maintaining Spyder?
by Mukundan Ragavan 30 Mar '23

30 Mar '23

I have been maintaining spyder in Fedora for sometime now. Since the time I took up maintaining spyder, the dependencies have expanded considerably (I have packaged several of these dependencies in the past). Unfortunately, I am not able to devote as much as time as needed to maintain this package (at least in working condition) anymore. I should point out that there are specific version dependencies for certain packages (e.g. ipython) that are not in sync with Fedora. I have relaxed the dependencies in the Fedora package (so that spyder continues to work, albeit with errors). I did start updating spyder to 5.4.x series. Two more packages need to be packaged. See COPR - https://copr.fedorainfracloud.org/coprs/nonamedotc/spyder/builds/ All that said, is anyone or the python SIG interested in maintaining this package? I will be happy to continue on as co-maintainer and try to help out when I can. Thanks in advance. Mukundan. -- GPG Key: E5C8BC67

1 0

Update python-prometheus_client
by Kai A. Hiller 29 Mar '23

29 Mar '23

Hello, I need a newer version of prometheus_client in Fedora, but its maintainer jkaluza is unresponsive. Can you please update it? Preferably for f37 and f38, too. I created a PR a while ago[1]. Thank you and kind regards Kai [1] https://src.fedoraproject.org/rpms/python-prometheus_client/pull-request/5

1 0

JupyterLab finally in Fedora and the future of Jupyter Notebook
by Lumír Balhar 06 Mar '23

06 Mar '23

Hi. There were some attempts to package JupyterLab (the first one in 2018) but none of them were successful. But now, after packaging 20 new Python and Rust packages, JupyterLab finally landed into rawhide. The main motivation for this effort is the ongoing plan to base the next version of Jupyter Notebook (version 7) on top of the components of JupyterLab. Take a look at JEP 79 is you want to know more about this - https://jupyter.org/enhancement-proposals/79-notebook-v7/notebook-v7.html So, how the future of Jupyter Notebook looks like in Fedora? In Fedora 37, there is still the old standalone Jupyter Notebook version 6.4 and nothing will change about it. Fedora 38 contains a slightly newer version of Jupyter Notebook (6.5 - the last release in 6). Everything looks the same as before but behind the curtains, the standard components Notebook is based on moved to a new package named nbclassic. This is gonna be important for Fedora 39. Fedora 39 now contains JupyterLab 4.0.0 alpha 34 which allows me to update Jupyter Notebook to 7.0.0 alpha 14. So, there will be three options in rawhide you can choose from when working with ipynb files: * JupyterLab - new development environment with text editor, consoles, files browser and more. * Jupyter Notebook - document-centric UI you already know but looking a little bit different because it'll based on components from Jupyter lab. * Jupyter nbclassic - The good old Jupyter Notebook as you know it for years. This package should be supported upstream for 2 years after the stable version of Notebook v7 is released. Both JupyterLab and Notebook are currently alpha but the plan of upstream developers is to release stable releases before JupyterCon in May so we should have plenty of time to update the packages in rawhide before the beta freeze in August. If you want to use the new notebook v7 before I update it in rawhide, use this COPR: https://copr.fedorainfracloud.org/coprs/lbalhar/notebook/ Any help testing the prereleases is appreciated. Have a nice day. Lumír

3 4

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel March 2023