The great Mailman 3 / Hyperkitty upgrade: bumping flufl-lock and
mistune?
by Michel Alexandre Salim
Hi all,
Neal Gompa and I have been reviving the effort to get our mailing list
server infrastructure (currently running on RHEL 7 with missing packages
provided in an unofficial repo) hostable on RHEL 9 + EPEL.
Pagure issue: https://pagure.io/fedora-infrastructure/issue/8455
Bugzilla tracker: https://bugzilla.redhat.com/show_bug.cgi?id=2030061
Status: https://hackmd.io/Pb9otlVGQHe1r9BIC5bi7w (not fully updated yet)
We're currently stuck on the following:
https://bugzilla.redhat.com/show_bug.cgi?id=2032607
Your package (python-hyperkitty) Fails To Install in Fedora 36:
can't install hyperkitty:
- nothing provides python3.10dist(flufl-lock) >= 4 needed by hyperkitty-1.3.5-1.fc36.noarch
- nothing provides python3.10dist(mistune) >= 2~rc1 needed by hyperkitty-1.3.5-1.fc36.noarch
I have PRs attached to the upgrade requests for mistune: https://bugzilla.redhat.com/show_bug.cgi?id=1782288
and flufl-lock: https://bugzilla.redhat.com/show_bug.cgi?id=1852603
but both have breaking changes I detailed in the above Bz entries; if
you're a maintainer cc:ed on this email please check the relevant bz:
❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-flufl-lock
mailman3-0:3.3.4-5.fc35.noarch
mailman3-0:3.3.4-5.fc35.src
mailman3-0:3.3.4-5.fc36.noarch
mailman3-0:3.3.4-5.fc36.src
odcs-0:0.3.4-6.fc35.noarch
odcs-0:0.3.4-6.fc35.src
odcs-0:0.3.4-6.fc36.noarch
odcs-0:0.3.4-6.fc36.src
python-cartopy-0:0.20.0-1.fc35.src
python-cartopy-0:0.20.1-2.fc36.src
❯ sudo dnf repoquery --enablerepo=fedora-source,rawhide,rawhide-source --whatrequires python3-mistune
python-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.src
python-nbconvert-0:6.1.0-2.fc35.src
python-nbconvert-0:6.1.0-3.fc36.src
python3-m2r-0:0.2.1-5.20190604git66f4a5a.fc35.noarch
python3-nbconvert-0:6.1.0-2.fc35.noarch
python3-nbconvert-0:6.1.0-3.fc36.noarch
In particular, python-nbconvert specifically requires mistune < 2, and
upstream doesn't seem to have a newer release yet. python-cartopy oddly
only requires flufl-lock in its SRPM, not the built RPM.
PRs:
https://src.fedoraproject.org/rpms/python-flufl-lock/pull-request/1
https://src.fedoraproject.org/rpms/python-mistune/pull-request/5
(the packages are not in side tags yet because the PRs are not merged
yet, but if it helps I can build them in a COPR for F35)
We should probably bump the packages in Rawhide anyway, but also to
note:
- both of these packages are not co-maintained by the Python SIG
- most of the recent updates have been done by non-maintainers
Would it make sense to get the following groups officially added to the
package ACLs?
- infra-sig (admin), to ease maintaining the dependencies for Mailman
and Hyperkitty
- python-sig (commit or admin), for fixing issues e.g. with newer Python
versions
- epel-packagers-sig (collaborator, epel* branches) for helping to
bootstrap on new EL releases
Thanks,
--
Michel Alexandre Salim
profile: https://keyoxide.org/michel@michel-slm.name
2 years, 4 months
Provisional %{pyproject_build_lib} macro
by Miro Hrončok
Hello Pythonistas,
the pyproject-rpm-macros-0-51 update (available in Rawhide+ELN and updates
ready for 33 and 34) introduces a new provisional %{pyproject_build_lib} macro.
The intended use case will remain the same, but it might yet change its
behavior slightly. Here is the snippet from pyproject-rpm-macros README, better
viewed at
https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/RE...
PROVISIONAL: Importing just-built (extension) modules in %build
---------------------------------------------------------------
Sometimes, it is desired to be able to import the *just-built* extension modules
in the `%build` section, e.g. to build the documentation with Sphinx.
%build
%pyproject_wheel
... build the docs here ...
With pure Python packages, it might be possible to set `PYTHONPATH=${PWD}` or
`PYTHONPATH=${PWD}/src`.
However, it is a bit more complicated with extension modules.
The location of just-built modules might differ depending on Python version,
architecture, pip version.
Hence, the macro `%{pyproject_build_lib}` exists to be used like this:
%build
%pyproject_wheel
PYTHONPATH=%{pyproject_build_lib} ... build the docs here ...
This macro is currently **provisional** and the behavior might change.
The `%{pyproject_build_lib}` macro expands to an Shell `$(...)` expression and
does not work when put into single quotes (`'`).
Depending on the pip version, the expanded value will differ:
### New pip 21.3+ with in-tree-build (Fedora 36+)
Always use the macro from the same directory where you called
`%pyproject_wheel` from.
The value will expand to something like:
* `/builddir/build/BUILD/%{name}-%{version}/build/lib.linux-x86_64-3.10` for
wheels with extension modules
* `/builddir/build/BUILD/%{name}-%{version}/build/lib` for pure Python wheels
If multiple wheels were built from the same directory,
some pure Python and some with extension modules,
the expanded value will be combined with `:`:
*
`/builddir/build/BUILD/%{name}-%{version}/build/lib.linux-x86_64-3.10:/builddir/build/BUILD/%{name}-%{version}/build/lib`
If multiple wheels were built from different directories,
the value will differ depending on the current directory.
### Older pip with out-of-tree-build (Fedora 34, 35, and EL 9)
The value will expand to something like:
*
`/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-xxxxxxxx/build/lib.linux-x86_64-3.10`
for wheels with extension modules
*
`/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-xxxxxxxx/build/lib`
for pure Python wheels
Note that the exact value is **not stable** between builds
(the `xxxxxxxx` part is randomly generated,
neither you should consider the `.pyproject-builddir` directory to remain stable).
If multiple wheels are built,
the expanded value will always be combined with `:` regardless of the current
directory, e.g.:
*
`/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-xxxxxxxx/build/lib.linux-x86_64-3.10:/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-yyyyyyyy/build/lib.linux-x86_64-3.10:/builddir/build/BUILD/%{name}-%{version}/.pyproject-builddir/pip-req-build-zzzzzzzz/build/lib`
**Note:** If you manage to build some wheels with in-tree-build and some with
out-of-tree-build option,
the expanded value will contain all relevant directories.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
2 years, 4 months
Automatically generated Obsoletes tags?
by Tomáš Orsava
Hi,
I'm working on a way to automatically generate Obsoletes tags for Python
packages. I.e. for each `python3-foo` package, the automatic generator
would create a tag `Obsoletes: python3.10-foo < X-Y` (assuming python3
version is 3.10).
Currently we automatically generate only Provides tags, but not
Obsoletes, and when updating to a newer "main" Python version, a user
might get file conflicts during the dnf transaction (e.g. files from
python3.10-requests would conflict with files from python3-requests). If
we have the automatic Obsoletes, `python3-foo` packages would always
take precedence over any older `python3.X-foo`, as they should.
To be frank, we don't truly need this in Fedora right now: We only ship
alternative Python interpreters (e.g. python3.8, python3.9), but we do
not ship additional packages for these alternative Pythons (e.g. we have
no python3.9-requests).
However, this will be useful for downstream distributors (e.g. CentOS,
RHEL, ...), or in cases where somebody builds custom packages for
alternative stacks in Fedora.
The downside is that we will have a few thousand (est. 3624 [1])
additional Obsoletes tags in the Fedora repos that are mostly useless.
Does anyone see a problem with this? Given the amount of tags already
present (e.g. 336 thousand provides [2], 80 thousand requires [3] and
7.5 thousand obsoletes [4]), I think it won't negatively affect
anything, but I might be mistaken.
I think it would be useful to have the generator enabled by default, but
of course to account for corner cases, I'd implement an opt-out macro
(similar to `%python_disable_dependency_generator`). And to make the
automatic generator consistent with the manual macros, I would modify
the `%py_provides` macro to also generate Obsoletes tags.
What do you think?
[1] repoquery --repo=rawhide --provides -a | grep '^python3\.10-' | wc -l
[2] repoquery --repo=rawhide --provides -a | wc -l
[3] repoquery --repo=rawhide --requires -a | wc -l
[4] repoquery --repo=rawhide --obsoletes -a | wc -l
All the best,
Tomas Orsava
2 years, 4 months
pyproject_buildrequires choked on mistune
by Michel Alexandre Salim
I'm packaging python-mistune as a dependency for hyperkitty (so we can
finally pull off a Mailman3/Hyperkitty migration from the current
RHEL7+custom repo setup - tracked in
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2030061)
python-mistune review: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=2031262
One weird thing is, pyproject_buildrequires fails:
```
Executing(%generate_buildrequires): /bin/sh -e /var/tmp/rpm-tmp.KZ4hKk
+ umask 022 + cd /builddir/build/BUILD
+ cd mistune-2.0.0
+ echo python3-devel
+ echo 'python3dist(pip) >= 19'
+ echo 'python3dist(packaging)'
+ '[' -f pyproject.toml ']'
+ echo 'python3dist(toml)'
+ rm -rfv '*.dist-info/'
+ '[' -f /usr/bin/python3 ']'
+ RPM_TOXENV=py310
+ HOSTNAME=rpmbuild
+ /usr/bin/python3 -s /usr/lib/rpm/redhat/pyproject_buildrequires.py --generate-extras --python3_pkgver
sion 3
Handling setuptools from build-system.requires
Requirement satisfied: setuptools
(installed: setuptools 58.5.3)
Handling wheel from build-system.requires
Requirement not satisfied: wheel
Traceback (most recent call last):
File "/usr/lib/rpm/redhat/pyproject_buildrequires.py", line 421, in main
generate_requires(
File "/usr/lib/rpm/redhat/pyproject_buildrequires.py", line 354, in generate_requires
backend = get_backend(requirements)
File "/usr/lib/rpm/redhat/pyproject_buildrequires.py", line 219, in get_backend
raise FileNotFoundError('File "setup.py" not found for legacy project.')
FileNotFoundError: File "setup.py" not found for legacy project.
```
so I had to manually add the BRs in https://salimma.fedorapeople.org/specs/python/python-mistune.spec
the `pyproject.toml` is super simple:
```
$ cat python/mistune-2.0.0/pyproject.toml
[build-system]
requires = [ "setuptools", "wheel" ]
```
because the project really didn't have any dependency (beyond pytest for
running tests).
Is this a problem with our tooling or with upstream's pyproject.toml?
Happy to follow up with them if they need a more fleshed-out project
definition.
Thanks,
--
Michel Alexandre Salim
profile: https://keyoxide.org/michel@michel-slm.name
2 years, 4 months
Adding python3.10dist(pp) to python-ppft
by Ankur Sinha
Hi folks,
I was looking to unretire the unmaintained python-pp because some neuro
packages still use it but then was reminded by @music that we already
provide python-ppft which is a maintained fork and drop-in replacement
for pp.
https://github.com/uqfoundation/ppft
Currently python3-ppft provides these:
$ rpm -q --provides python3-ppft
python-ppft = 1.6.6.4-2.fc35
python3-ppft = 1.6.6.4-2.fc35
python3.10-ppft = 1.6.6.4-2.fc35
python3.10dist(ppft) = 1.6.6.4
python3dist(ppft) = 1.6.6.4
Can we also provide all these for `pp` so that packages in Fedora that
require pp automatically use ppft?
I tried adding `%py_provides pp` to python3-ppft, and while that does
add a few of these, it does not add the python3.10dist(pp) or
python3dist(pp) ones (which are autogenerated from the metadata?):
$ rpm -q --provides -p ./results_python-ppft/1.6.6.4/3.fc36/python3-ppft-1.6.6.4-3.fc36.noarch.rpm
python-pp = 1.6.6.4-3.fc36
python-ppft = 1.6.6.4-3.fc36
python3-pp = 1.6.6.4-3.fc36
python3-ppft = 1.6.6.4-3.fc36
python3.10-pp = 1.6.6.4-3.fc36
python3.10-ppft = 1.6.6.4-3.fc36
python3.10dist(ppft) = 1.6.6.4
python3dist(ppft) = 1.6.6.4
Would there be a way to include these machine readable provides too
since these forms are used by other packages in the automatically
generated requires etc. (if this is the right thing to do)?
I guess the alternative would be that every package that requires pp be
patched to require ppft, but tweaking a single python-ppft package would
be less work. :)
--
Thanks,
Regards,
Ankur Sinha "FranciscoD" (He / Him / His) | https://fedoraproject.org/wiki/User:Ankursinha
Time zone: Europe/London
2 years, 4 months