RPM 4.19: Dynamic subpackages with Python extras
by Miro Hrončok
Hello Python packagers.
RPM 4.19 introduces this feature:
https://rpm-software-management.github.io/rpm/manual/dynamic_specs.html
I decided to write this email to gather my thoughts. I believe that with this,
we can turn manual Python extras subpackages like this:
%package -n python3-...
Summary: %{summary}
%description -n python3-... %_description
%pyproject_extras_subpkg -n python3-xxx extra1 extra2
(See https://docs.fedoraproject.org/en-US/packaging-guidelines/Python/#Extras
for what that means.)
Into something like this:
%package -n python3-...
Summary: %{summary}
%description -n python3-... %_description
...
%install
%pyproject_install
...
%pyproject_generate_extras_subpkgs -n python3-xxx
The %pyproject_generate_extras_subpkgs macro would parse the installed
.dist-info directory to find out what extras are available and generate
subpackages for all of them.
(Obviously, the macro name is open up for discussion.)
----------------
An API would be required to exclude extras:
- that are not useful for other packages
(for example build/development requirements, commonly named dev, doc or test)
- that have requirements that are not packaged in Fedora
For example (mimicking the API of %pyproject_check_import):
%pyproject_generate_extras_subpkgs -n python3-xxx -e test -e 'nonfree*'
----------------
However, extras are also currently manually passed to %pyproject_buildrequires:
%generate_buildrequires
%pyproject_buildrequires -x extra1 -x extra2 -x test
It should already be possible to implement automatic extras discovery in
%pyproject_buildrequires with older RPM versions and allow it to be used this way:
%generate_buildrequires
%pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
RPM macros can only accept short flags, so <FLAG_TO_ENABLE_ALL_EXTRAS> can
either be -x '*' (if we start treating -x values as globs, which is backwards
compatible and probably generally useful), or a single-letter switch such as -a
(but honestly we are running out of meaningful letters).
(When -X is used, <FLAG_TO_ENABLE_ALL_EXTRAS> can probably be implied. However,
an explicit form needs to exist for packages that don't need to exclude any
extras at all.)
Eventually, I'd like to make <FLAG_TO_ENABLE_ALL_EXTRAS> the default, once RPM
4.19 is omnipresent.
----------------
Combined, this would mean that the packager needs to:
1. specify extras that are not supposed to be used as BRs
2. specify extras that are not supposed to be packaged
In the ideal word (2) is a superset of (1).
Should %pyproject_generate_extras_subpkgs somehow inherit the -Xes from
%pyproject_buildrequires?
When a package has extra1, extra2, nonfree1, nonfree2 and test extras, one
could do:
%generate_buildrequires
%pyproject_buildrequires <FLAG_TO_ENABLE_ALL_EXTRAS> -X 'nonfree*'
...
%pyproject_install
...
%pyproject_generate_extras_subpkgs -X test
That would mean:
- extra1 is BRed and packaged
- extra2 is BRed and packaged
- test is BRed but not packaged
- nonfree1 is neither
- nonfree2 is neither
----------------
Alternatively the information could be supplied by %globals:
%global _python_ignored_extras nonfree*
%global _python_unpackaged_extras test
However, I somehow dislike this approach.
----------------
I'd appreciate your thoughts on the matter.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
1 week, 4 days
Let's deprecate the %{pyproject_build_lib} macro... ?
by Miro Hrončok
Hello.
The %{pyproject_build_lib} macro is a *provisional* macro from
pyproject-rpm-macros.
It was added to solve the problem of out-of-tree pip builds. From the
pyproject-rpm-macros README:
"""
Sometimes, it is desired to be able to import the just-built extension modules
in the %build section, e.g. to build the documentation with Sphinx.
%build
%pyproject_wheel
... build the docs here ...
With pure Python packages, it might be possible to set PYTHONPATH=${PWD} or
PYTHONPATH=${PWD}/src. However, it is a bit more complicated with extension
modules.
The location of just-built modules might differ depending on Python version,
architecture, pip version, etc. Hence, the macro %{pyproject_build_lib} exists
to be used like this:
%build
%pyproject_wheel
PYTHONPATH=%{pyproject_build_lib} ... build the docs here ...
"""
When this macro was introduced, the built extension module lived in a folder
like
/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-xxxxxxxx/build/lib.linux-x86_64-3.10,
hence it made sense to include in the pyproject-rpm-macros and call it
%pyproject_something.
------------------------------------------------------------
Today, on Rawhide, pip default to in-tree-builds and the value for extension
modules will most likely be:
$PWD/build/lib.%{python3_platform}-cpython-%{python3_version_nodots}
And for pure Python:
$PWD/build/lib
And it turns out, the value is setuptools-specific, see
https://bugzilla.redhat.com/show_bug.cgi?id=2176393
Other build backends might store the built extension modules elsewhere or
simply pack the pure Python files into wheel directly from the source tree.
------------------------------------------------------------
In Rawhide, 41 packages use PYTHONPATH="%{pyproject_build_lib}" and 2 packages
have a comment with %%{pyproject_build_lib} in them.
Considering the macro
- is not build-backend-agnostic, and
- is not so much needed as it once was
I propose we deprecate it with no further fixes going in, but no scheduled removal.
The 41 packages can be fixed once Fedora 36 goes EOL by expanding the macro to
the values above, or if desired, we could macronize this in pythohn3-setuptools
as %{setuptools_build_lib}.
One problem is that the macro is unfortunately still needed on EL 9.
Thoughts?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
1 month, 1 week
Anyone interested in maintaining Spyder?
by Mukundan Ragavan
I have been maintaining spyder in Fedora for sometime now. Since the
time I took up maintaining spyder, the dependencies have expanded
considerably (I have packaged several of these dependencies in the
past). Unfortunately, I am not able to devote as much as time as needed
to maintain this package (at least in working condition) anymore.
I should point out that there are specific version dependencies for
certain packages (e.g. ipython) that are not in sync with Fedora. I have
relaxed the dependencies in the Fedora package (so that spyder continues
to work, albeit with errors). I did start updating spyder to 5.4.x
series. Two more packages need to be packaged. See COPR -
https://copr.fedorainfracloud.org/coprs/nonamedotc/spyder/builds/
All that said, is anyone or the python SIG interested in maintaining
this package? I will be happy to continue on as co-maintainer and try to
help out when I can. Thanks in advance.
Mukundan.
--
GPG Key: E5C8BC67
1 month, 4 weeks
JupyterLab finally in Fedora and the future of Jupyter Notebook
by Lumír Balhar
Hi.
There were some attempts to package JupyterLab (the first one in 2018)
but none of them were successful. But now, after packaging 20 new Python
and Rust packages, JupyterLab finally landed into rawhide. The main
motivation for this effort is the ongoing plan to base the next version
of Jupyter Notebook (version 7) on top of the components of JupyterLab.
Take a look at JEP 79 is you want to know more about this -
https://jupyter.org/enhancement-proposals/79-notebook-v7/notebook-v7.html
So, how the future of Jupyter Notebook looks like in Fedora?
In Fedora 37, there is still the old standalone Jupyter Notebook version
6.4 and nothing will change about it.
Fedora 38 contains a slightly newer version of Jupyter Notebook (6.5 -
the last release in 6). Everything looks the same as before but behind
the curtains, the standard components Notebook is based on moved to a
new package named nbclassic. This is gonna be important for Fedora 39.
Fedora 39 now contains JupyterLab 4.0.0 alpha 34 which allows me to
update Jupyter Notebook to 7.0.0 alpha 14. So, there will be three
options in rawhide you can choose from when working with ipynb files:
* JupyterLab - new development environment with text editor, consoles,
files browser and more.
* Jupyter Notebook - document-centric UI you already know but looking a
little bit different because it'll based on components from Jupyter lab.
* Jupyter nbclassic - The good old Jupyter Notebook as you know it for
years. This package should be supported upstream for 2 years after the
stable version of Notebook v7 is released.
Both JupyterLab and Notebook are currently alpha but the plan of
upstream developers is to release stable releases before JupyterCon in
May so we should have plenty of time to update the packages in rawhide
before the beta freeze in August.
If you want to use the new notebook v7 before I update it in rawhide,
use this COPR: https://copr.fedorainfracloud.org/coprs/lbalhar/notebook/
Any help testing the prereleases is appreciated.
Have a nice day.
Lumír
2 months, 3 weeks
