Replacing pytest -n auto with pytest -n %{_smp_build_ncpus}
by Miro Hrončok
Hello Pythonistats, packagers,
A handful of Fedora Python packages uses pytest-xdist to run tests in parallel
like this:
%pytest -n auto
-n auto means pytest will spawn a number of workers processes equal to the
number of available CPUs.
In the spirit of other packaging guidelines, I believe we should use this instead:
%pytest -n %{_smp_build_ncpus}
This means the same thing in most of the ceases, but will limit the number of
workers depending on other constraints in the spec or in the environment.
Should I do this in a mass change? Not so many packages use pytest -n auto in
the spec:
$ rg -l -- '(-n|--numprocesses)(\s*|=)auto(\s|$)'
ansible-bender.spec
azure-cli.spec
ocrmypdf.spec
python-cartopy.spec
python-GridDataFormats.spec
python-hypothesis.spec
python-matplotlib.spec
python-mplcairo.spec
python-rpmautospec.spec
python-sqlalchemy.spec
python-tox.spec
python-xarray.spec
python-zstandard.spec
pythran.spec
scipy.spec
(Other packages have that in tox.ini or pytest config file, but I am not aiming
at changing that here.)
And, considering many other packages might want to benefit from that, should
this be:
1) encouraged in the Python packaging guidelines
2) macronized (I was thinking %pytest_parallel, but TBD)
?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
7 months
What way of opting out form a particular Python shebang flag do you
prefer?
by Miro Hrončok
Hello Pythonistas, packagers.
In the context of this change:
https://fedoraproject.org/wiki/Changes/PythonSafePath
Python shebangs will have be:
#! /usr/bin/python3 -sP
In order to remove certain flags, packagers have the following tool:
# Unset -s on python shebang - ensure that extensions installed with pip
# to user locations are seen and properly loaded
%global py3_shebang_flags %(echo %py3_shebang_flags | sed s/s//)
Or:
# Don't add -P to Python shebang
# This package only works when /usr/bin is in sys.path
%global py3_shebang_flags %(echo %py3_shebang_flags | sed s/P//)
In the implementation PR, Maxwell suggested a different approach:
https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/141#com...
Basically, packagers would do something like this:
# Unset -s on python shebang - ensure that extensions installed with pip
# to user locations are seen and properly loaded
%global _python3_shebang_nousersite %{nil}
Or:
# Don't add -P to Python shebang
# This package only works when /usr/bin is in sys.path
%global _python3_shebang_safepath %{nil}
The macro names are not set in stone, it could even be %_python3_shebang_s and
%_python3_shebang_P.
The previous sed-based way would still work and packages that already use it
would not need to change immediately.
Do you consider the macro based approach better (worth it)? And if so, do you
prefer actual flag letters in the macro names, or the verbose names?
Thanks for your input.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10 months, 1 week
[HEADS UP] Python 3.11.0b4 in rawhide
by Miro Hrončok
Hello,
I've just updated Python from 3.11.0b3 to 3.11.0b4 in rawhide.
Let me know if you encounter any problems.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10 months, 2 weeks
Python 3.11 final release might be delayed to December
by Miro Hrončok
Hello,
forwarding this message to Fedora.
Will know more by the end of this week -- we might need to consider reverting
back to Python 3.10 if we don't want to ship Fedora 37 GA with a beta version
of Python :(
Fedora 37 mass rebuild is planned for 2022-07-20 -- I will make sure we know
what to do by then.
Too bad we haven't known this before we started the 3.11 mass rebuild :/
-------- Forwarded Message --------
Subject: [Python-Dev] [Release] Status of Python 3.11 release
Date: Mon, 4 Jul 2022 23:36:39 +0100
From: Pablo Galindo Salgado <pablogsal(a)gmail.com>
To: Python Dev <python-dev(a)python.org>, python-committers
<python-committers(a)python.org>
Hi everyone,
# TLDR
We may be pushing the final release until December if the stability of Python
3.11 doesn't improve.
# Long Explanation
Unfortunately, we cannot still release the next Python 3.11 beta release
(3.11.0b4) because we still have a bunch
of pending release blockers. Unfortunately, this is still after several
preexisting release blockers have been fixed,
some of them being discovered after I sent my last update. These are the
current release blockers:
https://github.com/python/cpython/issues?q=is%3Aissue+is%3Aopen+sort%3Aup...
<https://github.com/python/cpython/issues?q=is%3Aissue+is%3Aopen+sort%3Aup...>
We also have some deferred blockers (some of them should actually be release
blockers):
https://github.com/python/cpython/issues?q=is%3Aissue+is%3Aopen+sort%3Aup...
<https://github.com/python/cpython/issues?q=is%3Aissue+is%3Aopen+sort%3Aup...>
Due to this and the fact that we are already 3 weeks delayed in the release
schedule, the current stability of Python 3.11 is not as good
as it is supposed to be at this stage of the release schedule and more testing
from end-users and library authors is required. After
discussing with the Steering Council, we are considering delaying the final
release until December to allow for two more beta releases.
This is how we are going to proceed:
* If the current release blockers are not fixed by the end of this week, two
more betas will be released (1 month per beta) and
we will *definitely* delay the final release until December.
* If the current release blockers are fixed we will proceed to release Python
3.11.0b4 on Monday. We will target the current release
date (Monday, 2022-10-03) but if more release blockers that affect fundamental
parts of the Python interpreter or the standard libraries
are raised, the release team will still consider adding two more betas nd
pushing the final release to December.
One of the goals that we are going to try to achieve from the release team is
that no substantial code changes are added between the last
beta and the first release candidate. This is so all the fixes that affect
fundamental parts of the interpreter or the standard library can be
tested by end-users before the first release candidate is released (and not
with it). This is also partially because once we release the first release
candidate, the ABI will be frozen and certain kinds of fixes will be more
complicated.
Hopefully, this addresses some of you that have reached out with concerns over
the stability of Python 3.11 and the release schedule.
I understand that delaying the release until December will complicate things
for some Linux distributions and will affect end users and redistributors
targeting the original release, but please understand that our
responsibility in the release team after all is to guarantee a stable final release
above all and unfortunately, we don't currently have the confidence that we
would like given the current state of the release process.
Please do not hesitate in reaching out if you have any questions or concerns.
Thanks, everyone for your help and understanding and thanks a lot to all of you
for your great work!
Cheers from cloudy London,
Pablo Galindo Salgado
10 months, 2 weeks
planning to orphan python-django (and a couple of dependent packages)
by Matthias Runge
Hello there,
you may or may not know, I have been maintaining python-django for quite
some time in the past, some time as part of my job. My role changed and
I really can not dedicate Django the time it deserves. I am looking for
someone or persons willing to take
- python-django
- python-django-angular
- python-django-appconf
- python-django-authority
- python-django-compressor
- python-django-contact-form
- python-django-debug-toolbar
- python-django-haystack
- python-django-mptt
- python-django-nose
- python-django-notification
- python-django-pagination
- python-django-pipeline
- python-django-piston
- python-django-pyscss
- python-django-rest-framework
- python-django-reversion
- python-django-robots
- python-django-tables2
- python-django-tagging
Matthias
10 months, 3 weeks
Re: Python 3.11 final release might be delayed to December
by Miro Hrončok
On 05. 07. 22 16:02, Richard W.M. Jones wrote:
> On Tue, Jul 05, 2022 at 03:46:47PM +0200, Miro Hrončok wrote:
>> On 05. 07. 22 15:05, Richard W.M. Jones wrote:
>>> On Tue, Jul 05, 2022 at 01:17:39PM +0200, Miro Hrončok wrote:
>>>> Hello,
>>>> forwarding this message to Fedora.
>>>>
>>>> Will know more by the end of this week -- we might need to consider
>>>> reverting back to Python 3.10 if we don't want to ship Fedora 37 GA
>>>> with a beta version of Python:(
>>> Is there a reason why shipping a beta version of Python would be bad?
>>> I've been using it on my main development machine for a few weeks and
>>> for the (fairly limited) Python stuff I do it seems to be fine.
>>>
>>> It'd be a problem if it was causing bugs.
>>
>> I suppose it could amplify the "Fedora is the bleeding edge" message.
>
> Which is good :)
>
> So one issue is if Python is intending to add new features between now
> and the final release of 3.11, and then other software might (in
> future) check for the presence of some new feature by using a version
> number test. Since our beta Python would look sufficiently like 3.11,
> it might report as having the new feature.
That is not intended. In fact, it is forbidden by Python policies. Python 3.11
is feature complete, features can only be reverted at this point.
> Is it possible to update Python to 3.11 after Fedora 37 GA is released?
Maybe. We can do it if it is ABI compatible with the beta we shipped at GA.
Upstream only guarantees ABI compatibility at RC :/
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10 months, 3 weeks
Re: Python 3.11 final release might be delayed to December
by Richard Shaw
On Tue, Jul 5, 2022 at 8:07 AM Richard W.M. Jones <rjones(a)redhat.com> wrote:
> On Tue, Jul 05, 2022 at 01:17:39PM +0200, Miro Hrončok wrote:
> > Hello,
> > forwarding this message to Fedora.
> >
> > Will know more by the end of this week -- we might need to consider
> > reverting back to Python 3.10 if we don't want to ship Fedora 37 GA
> > with a beta version of Python :(
>
> Is there a reason why shipping a beta version of Python would be bad?
> I've been using it on my main development machine for a few weeks and
> for the (fairly limited) Python stuff I do it seems to be fine.
>
> It'd be a problem if it was causing bugs.
>
Would it not make sense then to review the blocker bugs and then determine
if they are blockers for us?
Thanks,
Richard
10 months, 3 weeks
Re: Python 3.11 final release might be delayed to December
by Miro Hrončok
On 05. 07. 22 15:05, Richard W.M. Jones wrote:
> On Tue, Jul 05, 2022 at 01:17:39PM +0200, Miro Hrončok wrote:
>> Hello,
>> forwarding this message to Fedora.
>>
>> Will know more by the end of this week -- we might need to consider
>> reverting back to Python 3.10 if we don't want to ship Fedora 37 GA
>> with a beta version of Python:(
> Is there a reason why shipping a beta version of Python would be bad?
> I've been using it on my main development machine for a few weeks and
> for the (fairly limited) Python stuff I do it seems to be fine.
>
> It'd be a problem if it was causing bugs.
I suppose it could amplify the "Fedora is the bleeding edge" message.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
10 months, 3 weeks
