Hello,
recently, we were suggested an improvement for %pyproject_buildrequires
-r/-x.
We could read the project's runtime dependencies (if they're not marked
as `dynamic`) from pyproject.toml [project] table directly, without
calling prepare_metadata_for_build_wheel or building the wheel to read
the dependency metadata from it.
Reading the metadata via prepare_metadata_for_build_wheel is already
quite fast, however implementing that hook is optional for the build
backends. When the hook is not available, we need to build the wheel,
which can be potentially very slow and resource-hungry (imagine building
the entire scipy project).
Metadata in pyproject.toml is standardized (PEP 621), however there is
no way of telling ahead of time if the build backend already supports
that standard.
See the details and discussion in bugzilla:
https://bugzilla.redhat.com/2261939
Maxwell has raised some valid concerns there:
- Other tools (build frontends, e.g. pip/build) call
prepare_metadata_for_build_wheel, our macros would diverge in
functionality compared to the rest of the landscape.
- pyproject.toml's [project] may not be the source of metadata that the
build backend uses. A project could switch to a build backend without
PEP 621 support (e.g. poetry-core) and forget to remove the [project] table.
- There can be potential differences between BuildRequires and Requires
generators when one generates dependencies based on the pyproject.toml
[project] table while the other on the packaged dist-info metadata.
- Using macros to build on older systems: e.g. RHEL 9's old setuptools
version silently ignores the pyproject.toml [project] table -
%pyproject_buildrequires could still pull the dependency information
from it, but the resulting package would be broken because it did not
include those in the packaged dist-info metadata.
One way to mitigate would be to make the proposed behavior opt-in only,
with the possibility to either build wheel with -w option (already
existing) or e.g. -p (now-proposed: reading from pyproject.toml) in case
backend doesn't have prepare_metadata_for_build_wheel. However, this
adds a layer of complexity for packagers and macros maintainers.
The questions we'd love your input for:
- Should %pyproject_buildrequires try to read dependencies from
pyproject.toml first and fall back to calling hooks only if that fails?
- Should %pyproject_buildrequires call the hook and try to fall
back to reading dependencies from pyproject.toml when the hook is not
availbale?
- Should this behavior exist but not be the default (explicit flag
would be required to opt-in)?
- Can you think of a better alternative than the ones described here?
Cheers,
Karolina
Hello Pythonistas,
Python 3.13 has an experimental JIT compiler:
https://docs.python.org/dev/whatsnew/3.13.html#experimental-jit-compiler
Enabling it is a configure (hence build-time) option.
How do we handle this in Fedora?
- We can keep it disabled, as it is experimental.
- We can enable it, but be ready to revert if it causes problems.
- We can add yet another build variant, but we already have 4 of those
(regular, debug, freethreading, freethreading-debug), so I'd rather not make it
6 (or 8, if we include freethreading+jit combinations). I don't know yet if it
would be co-installable.
Opinions?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
On 10. 06. 24 15:07, Tom Stellard wrote:
> On 5/31/24 01:55, Karolina Surma wrote:
>> Hello,
>>
>> To deliver Python 3.13 with Fedora Linux 41, we will run a coordinated
>> rebuild in a side tag.
>>
>> https://fedoraproject.org/wiki/Changes/Python3.13
>>
>> Python 3.13.0b2 is scheduled for Tuesday, Jun 4th 2024.
>> We hope to start the mass rebuild shortly after it's available.
>>
>> TL;DR: If you can, for the period of the mass rebuild just don't build your
>> packages in rawhide.
>> We will let you know when the side tag rebuild actually starts and when it is
>> merged and it's safe to build in rawhide with Python 3.13.
>>
>> Details:
>> If you see a "Rebuilt for Python 3.13" (or similar) commit in your package,
>> please don't rebuild it in regular rawhide or another rawhide side tag.
>> If you need to, please let us know, so we can coordinate.
>>
>> If you'd like to build a package after we already rebuilt it, you should be
>> able to build it in the side tag via:
>>
>> on branch rawhide:
>> $ fedpkg build --target=f41-python
>> $ koji wait-repo f41-python --build <nvr>
>>
>
> I'm in the middle of updating the LLVM packages in my own side tag,
> would it work if I tag python3.13.0b2 into my LLVM side-tag and
> rebuild the LLVM packages there?
It works if you merge your side tag later than ours.
If you merge it sooner, it breaks the world unless you untag python first
(which would presumably break the Python packages built in your side tag).
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
Hi,
While looking into a package upgrade I came across pynose [1]. It
appears to be a drop-in replacement for nose [2], which has been
deprecated in Fedora 32 [3].
There are still quite a few packages depending on nose, that have not
yet moved away from it. The official successor to nose is nose2 [4]. But
that package does not support all of the behavior of nose.
Therefore, I'm thinking of introducing pynose as a drop in replacement
of deprecated nose. Pynose uses the same namespace as nose, but provides
python3dist(pynose). Thus adding Provides: for nose would make it a
drop-in replacement for packages currently depending on nose.
As a preliminary smoke test I built all packages depending on nose in
Copr against pynose [5]. Packages failing to build against pynose I also
built against nose [6]. I haven't checked all packages in detail yet,
but the amount packages failing appears manageable (17).
Going by [3] I suppose this would also need to go through the changes
process. If so, I'd be willing to drive that. But I could use some
help/support since this would be my first change proposal.
In a way this e-mail is kind of an informal general rehearsal.
With that said, please shoot! 🔫
[1] https://github.com/mdmintz/pynose
[2] https://github.com/nose-devs/nose
[3] https://fedoraproject.org/wiki/Changes/DeprecateNose
[4] https://github.com/nose-devs/nose2
[5] https://copr.fedorainfracloud.org/coprs/gui1ty/pynose
[6] https://copr.fedorainfracloud.org/coprs/gui1ty/nose
Cheers,
--
Sandro
FAS: gui1ty
Matrix: Penguinpee
Elsewhere: [Pp]enguinpee
Hello,
To deliver Python 3.13 with Fedora Linux 41, we will run a coordinated
rebuild in a side tag.
https://fedoraproject.org/wiki/Changes/Python3.13
Python 3.13.0b2 is scheduled for Tuesday, Jun 4th 2024.
We hope to start the mass rebuild shortly after it's available.
TL;DR: If you can, for the period of the mass rebuild just don't build
your packages in rawhide.
We will let you know when the side tag rebuild actually starts and when
it is merged and it's safe to build in rawhide with Python 3.13.
Details:
If you see a "Rebuilt for Python 3.13" (or similar) commit in your package,
please don't rebuild it in regular rawhide or another rawhide side tag.
If you need to, please let us know, so we can coordinate.
If you'd like to build a package after we already rebuilt it, you should
be able to build it in the side tag via:
on branch rawhide:
$ fedpkg build --target=f41-python
$ koji wait-repo f41-python --build <nvr>
It takes time to build all the essential packages,
so don't expect all your dependencies to be available right away.
Any attempts to build your packages in the side tag before we do will
likely fail due to missing dependencies.
When in trouble, ask here or on Fedora's Matrix - Fedora Python room
(https://matrix.to/#/#python:fedoraproject.org)
Ping me (ksurma) or Miro (mhroncok) if you need to talk to us.
Builds will appear here:
https://koji.fedoraproject.org/koji/builds?latest=0&tagID=f41-python&order=…
Please avoid any potentially disturbing or major changes in Python
packages until the rebuild is over.
Thanks!
Karolina