Packaging python3-*-devel subpackages
by Miro Hrončok
Hello Pythonistats and packaging folks.
Tomáš (CCed) approached me today with an interesting question.
A Python package he is packaging into RPM (python3-rapidfuzz) installs some
development files (.h and .pxd).
Tomáš queried upstream about those files to figure out if they should be
installed or if they are only needed to build the package itself. Upstream
responded:
> rapidfuzz.h and __init__.pxd provide a capi for rapidfuzz, which allows users
> to write their own similarity metrics in C/C++/Cython, which can be called by
> rapidfuzz in a more performant way through this C-API.
> For this reason the header file should be part of the installation as well.
OK, we want to ship them. Our packaging guidelines however say:
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_devel_packages
> Specifically, -devel packages must be used to contain files which are
> intended solely for development or needed only at build-time. This is done to
> minimize the install footprint for users. There are some types of files which
> almost always belong in a -devel package:
> - Header files...
So from our guidelines perspective, the files would go to
python3-rapidfuzz-devel. The way we packaged Python in 201x, this would be the
end of it.
However, in this decade with automatic Python BuildRequires, we could easily
end up in this situation:
1. a Python package lists rapidfuzz as a build requirement
2. %pyproject_buildrequires generates a dependency on python3dist(rapidfuzz)
3. only python3-rapidfuzz is pulled
4. %build wants to use rapidfuzz.h
5. packager needs to manually BuildRequire python3-rapidfuzz-devel
To avoid (5), my suggestion was to add the following requirement to
python3-rapidfuzz:
Requires: (python3-rapidfuzz-devel%{?_isa} =
%{?epoch:%{epoch}:}%{version}-%{release} if python3-devel)
That way, application packages that pull in pytohn3-rapidfuzz as a runtime
dependency won't get it (we are minimizing the install footprint for users),
but users who build stuff (including RPM packages) will get it (we make
automatically generated dependencies work as intended).
(Of course, users who have python3-devel installed for reasons other than
rapidfuxx will still get the files, but the assumption here is that users who
install -devel packages intent to /generally/ build stuff.)
Is that a good suggestion? And if so, should it be a general recommendation for
such cases?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
3 months, 1 week
Fwd: [pypy-dev] Dropping py3.8, working on py3.10
by Miro Hrončok
Considering this, I intent to retire pypy3.8 from Fedora 38+ before the beta
freeze.
-------- Forwarded Message --------
Subject: [pypy-dev] Dropping py3.8, working on py3.10
Date: Tue, 10 Jan 2023 12:01:58 +0200
From: Matti Picus <matti.picus(a)gmail.com>
To: pypy-dev(a)python.org <pypy-dev(a)python.org>
We have begun work on py3.10. I changed the buildbot configuration to build
default (py2.7), py3.9, py3.10 instead of default (py2.7), py3.8, py3.9. Builds
for py3.8 (or any branch) can be kicked off manually. Unless there is
significant push-back, or some latent urgent bug appears on py3.8, I do not
plan on making another py3.8 release.
Help is always welcome, the py3.10 work is at a stage of "death by 1000 paper
cuts" where there are many small details that need to be changed. One thing
people could do is to peruse the lib-python failures (testing py3.10 on the
upstream stdlib tests) for obvious [2] problems. You can see the log file by
looking at the latest builds [0] -> clicking on the pypy-c-jit-linux-x86-64
link -> shell_16 lib-python test failed -> stdio. There are some hints for
parsing that output on the wiki [1].
Matti
[0] https://buildbot.pypy.org/summary?branch=py3.10
[1] https://foss.heptapod.net/pypy/pypy/-/wikis/Analysing-lib-python-errors
[2] what may be obvious to you may be gibberish to someone else, which is why
we need everyone's expertise
3 months, 1 week
News in pyproject-rpm-macros 1.5.x and 1.6.x
by Miro Hrončok
Hello Pythonistas.
We have released a new version of pyproject-rpm-macros 1.6.2.
The last announcement on this list was for 1.4.0, so let me summarize what's
new since then:
The 1.5.x versions are not very significant. 1.5.0+ now uses the
%py3_test_envvars macro (available on Fedora 38+) in %tox. This is mostly an
internal change, but it has subtle differences for packages using %tox. The
noticeable difference is that CFLAGS/LDFLAGS are now set to
${RPM_OPT_FLAGS}/${RPM_LD_FLAGS}, so if the tests run gcc etc. in them they use
the RPM build flags.
1.5.1 adjusted tests for tox 4+, which is also available in Fedora 38+.
------------------------------------------------------------------------------
Version 1.6.0 introduced a new pyproject-srpm-macros subpackage (installed in
the default buildroot) with a minimal implementation of the
%pyproject_buildrequires macro.
Packages using %pyproject_buildrequires now technically don't need to manually
BuildRequire neither pyproject-rpm-macros nor python3-devel.
With that change, the python3-devel package in ELN/RHEL10+ no longer requires
pyproject-rpm-macros. ELN packages that use pyproject-rpm-macros and don't use
%pyproject_buildrequires might need to be adjusted. As always, I note that
using pyproject-rpm-macros without %pyproject_buildrequires works fine but has
no backwards compatibility guarantees -- packagers who prefer to list all
BuildRequires manually might need to adjust the list even on "minor"
pyproject-rpm-macros updates.
This change was mostly motivated by the need to avoid pulling
pyproject-rpm-macros into the AppStream RHEL10+ repository and is not impacting
Fedora builds in any negative way. The python3-devel package in Fedora still
requires pyproject-rpm-macros.
------------------------------------------------------------------------------
Version 1.6.1 fixed a subtle bug in %pyproject_buildrequires. When the build
backend (mostly a setup.py scrip) was shelling out to subprocesses its stdout
was falsely part of the generated BuildRequires. This is now fixed.
See http://bugzilla.redhat.com/2166888 for details.
------------------------------------------------------------------------------
Version 1.6.2 now considers more paths as language files.
Previously, only files like .../locale/xx/LC_MESSAGES/*.mo were marked as
%lang(xx). Currently the /locale/ part no longer needs to be there.
We decided to treat this as a bugfix.
See https://bugzilla.redhat.com/2166295 for details.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
3 months, 1 week
