python-devel August 2024

python-devel@lists.fedoraproject.org

6 participants
6 discussions

Proposal: Let's update virtualenv in Rawhide (drops Python 3.6 support)
by Miro Hrončok 26 Aug '24

26 Aug '24

Hello Pythonistas. The virtualenv version in Fedora is currently stuck at 20.21.1 (April 2023). This is because version 20.22+ dropped support for Python 2.7 and 3.6 virtual environments. Now when Python 2.7 was retired via https://fedoraproject.org/wiki/Changes/RetirePython2.7 the only difference is Python 3.6. If we upgrade virtualenv, developers targeting Python 3.6 (e.g. RHEL 8 platform Python) can use `python3.6 -m venv` to create Python 3.6 virtual environments. Developers using tox can follow https://tox.wiki/en/4.18.0/faq.html#testing-end-of-life-python-versions I propose we bite the bullet and update virtualenv to the latest version in Fedora 42+. -- Miro Hrončok -- Phone: +420777974800 Fedora Matrix: mhroncok

3 3

setuptools 72 removes setup.py test, your package(s) will break
by Miro Hrončok 16 Aug '24

16 Aug '24

Hello. We will eventually update python-setuptools in Fedora to version 72 and that version removes support for the `setup.py test` command. We will probably use the change process to propose the upgrade, so don't expect it to land soon. However, if your package still uses the `setup.py test` command (deprecated for 5 years), please update to %pytest, %tox, %{python3} -m unittest, etc. You can use this opportunity to update your package to the current Python packaging guidelines, see e.g. https://src.fedoraproject.org/rpms/python-cachetools/pull-request/4 Affected 199 packages by maintainers based on a simple specfile grep for ^[^-#].+\bsetup\.py\s+test\b abbra python-whoosh abompard python-flufl-i18n python-jinja2_pluralize python-pyramid_sawing python-pysocks python-translationstring python-zope-deprecation python-zope-testing adamwill python-mwclient aekoroglu python-django-prometheus python-repoze-lru python-serpy python-xunitparser ajax binwalk alexl pycairo setools amigadave isrcsubmit python3-mallard-ducktype amoralej python-rsd-lib python-yaql ankursinha python-klusta python-num2words python-pyfim apevec python-appdirs python-distroinfo python-oslo-context python-testresources python-testscenarios python-wsgi_intercept asn cepces athoscr python-booleanoperations python-epub python-fontMath python-glyphsLib python-mutatormath python-pyclipper atim kitty aviso python-enlighten besser82 python-jcconv caolanm pycairo setools carlwgeorge python-testresources chandankumar python-oslo-context python-tosca-parser python-wsgi_intercept chedi python-jinja2-time python-poyo python-pytest-flakes python-tree-format chkr calibre churchyard python-appdirs python-cachetools python-genty python-launchpadlib python-lazr-uri python-scripttest python-simplegeneric python-wadllib cicku python-apsw ckalina ddiskit cqi python-django-contrib-comments dcallagh python-phonenumbers python-www-authenticate dcavalca tpm2-pkcs11 dchen cepces defolos osc dfateyev python-apsw dherrera python-pytest-runner dmach osc python-productmd dminer python-zstd dshea python-jinja2_pluralize dwalsh setools echevemaster python-pydispatcher transifex-client eeickmeyer python-jack-client esyr ddiskit fab ioc-writer past-time python-cx-oracle python-dfdatetime python-geojson python-gmqtt python-imagehash python-ipgetter2 python-jsonpath-ng python-jsonpath-rw python-shodan python-slacker python-textparser python-zc-customdoctests fale python-decorator fedepell python-jinja2-time filiperosset hddfancontrol fjanus PyGreSQL frostyx python-flask-wtf-decorators fschwarz python-digitalocean python-genshi girst mopidy mopidy-mpd gsauthof python-aioresponses gui1ty python-moksha-hub hguemar python-appdirs hhorak PyGreSQL hiwkby python-oslo-metrics hobbes1069 python-collada hvad python-events ignatenkobrain python-pytest-flakes python-simplegeneric jamatos python-markups python-smartypants jcapitao python-cliff python-distroinfo python-timeout-decorator jcaratzas python-repoze-lru python-simplegeneric python-singledispatch python-sphinxcontrib-trio python-zc-lockfile jcline python-ipdb jeckersb python-cachetools jjelen tpm2-pkcs11 jmontleon python-dictdiffer jonathanspw kitty python-freeipa python-pystray python-pytest-runner python-zstd jonny python-libusb1 jorti python-pysrt jpena python-murano-pkg-check python-tosca-parser jruzicka python-tosca-parser jstanek udiskie jujens python-CommonMark python-hgapi jussilehtola fprettify kalev pycairo kevin calibre python-chai python-decorator python-fedmsg-meta-fedora-infrastructure python-html5-parser python-pyramid-tm python-pysocks python-pytest-fixture-config python-svg-path python-tgscheduler kni python-ouimeaux python-pysignals ktdreyer python-testscenarios laxathom python-gammu lbalhar python-boolean.py python-feedgen python-hypothesis-fspaths lbazan python-neurosynth transifex-client lberk python-rmtest lbrabec python-xunitparser lcons python-auth-credential python-dirq python-messaging python-simplevisor limb pyicu python-gattlib python-scour python-svg python-tubes python-typing-inspect lnie libvirt-test-API lsedlar python-productmd lyessaadi python-yapsy maha python-sphinxcontrib-autoprogram melmorabity grammalecte mfabian nototools mohanboddu python-openidc-client mrunge python-case python-pytimeparse python-roman python-singledispatch python-testresources python-testscenarios msuchy osc nathans python-rmtest nb python-digitalocean ngompa osc python-case python-dkimpy python-freeipa python-pystray python-tubes python-zstd omos python-r128gain ondrejj python-chameleon python-nine python-xlrd opohorel python-launchpadlib python-lazr-uri python-wadllib orion python-iptools python-xlib python-zope-testing orphan python-gerritlib transtats-cli osloup PyGreSQL pbrobinson adapt gtts gtts-token pyjokes python-can python-casttube tpm2-pkcs11 pcmoore setools pghmcfc python-crypto pingou python-binaryornot python-case python-chai python-flask-xml-rpc python-vobject python-xlrd pkubat PyGreSQL plautrba setools pnemade ansible-inventory-grapher python-bracex python-cpuinfo poros ddiskit praiskup PyGreSQL puiterwijk python-cccolutils pwhalen pyjokes pyowm pwu nototools qulogic python-json-minify python-xmp-toolkit radez python-zc-lockfile python-zope-testing rajeeshknambiar python-ofxparse raphgro python-pdfkit rathann lazygal rcritten cepces python-pyasn1 rdieter python-mwclient rebus python-colorclass python-oletools python-pcodedmp python-roman rhughes pycairo setools robert python-oletools python-pcodedmp rstrode pycairo setools sagitter petsc salimma kitty python-lupa python-pytest-runner python-volatile python-zstd sergiomb python-gammu python-xlib sgallagh python-whoosh sharkcz python-sql python-vobject shoracek tpm2-pkcs11 slinabery python-jsonpath-rw solopasha kitty spichugi python-pyasn1 suanand transtats-cli suraia python-precis_i18n suve python-ssdeep swt2c binwalk tdecacqu python-statsd terjeros hgsvn python-cwcwidth python-httpsig-cffi python-pyte python-tidy tflink python-xunitparser thebeanogamer classification-banner thm python-sphinxcontrib-trio python-textile python3-lxc thofmann btrfs-sxbackup tomspur python-simplegeneric trawets fros trb143 python-pysword troycurtisjr python-ipdb ttorling python-pysocks vascom python-pymediainfo vmojzis sepolicy_analysis setools vtrefny python-pytimeparse ykarel python-os-service-types zaneb python-autopage zawertun kitty zbyszek calibre zsun ddiskit -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 1

Re: setuptools 72 removes setup.py test, your package(s) will break
by Miro Hrončok 14 Aug '24

14 Aug '24

On 14. 08. 24 18:21, Paul Howarth wrote: >> You can use this opportunity to update your package to the current >> Python packaging guidelines, see e.g. >> https://src.fedoraproject.org/rpms/python-cachetools/pull-request/4 >> >> Affected 199 packages by maintainers based on a simple specfile grep >> for ^[^-#].+\bsetup\.py\s+test\b > There are probably quite a few packages not caught by this, for example > python-pycryptodomex, which has: > > %check > PYTHONPATH=$RPM_BUILD_ROOT%{python3_sitearch}/ %{__python3} %{py_setup} test $ rg 'setup\} test\b' python-journal-brief.spec 64:%{__python3} %{py_setup} test python-pycryptodomex.spec 107:PYTHONPATH=$RPM_BUILD_ROOT%{python3_sitearch}/ %{__python3} %{py_setup} test > There might also be packages using tox with a tox.ini like this one: > https://github.com/pycrypto/pycrypto/blob/master/tox.ini Yeah, invocations hidden in tox.ini, Makefile, etc. are harder to grep for. I might end up building everything in copr and grepping the logs for the error messages. -- Miro Hrončok -- Phone: +420777974800 Fedora Matrix: mhroncok

3 2

The -O3 flag leaked into sysconfig CFLAGS, should we keep it?
by Miro Hrončok 14 Aug '24

14 Aug '24

Hello Pythonistas. For years, the CFLAGS embedded in Python sysconfig contained -O2 in Fedora. This was how Python was built and by default, all flags used to build Python were embedded. Later, the flag was removed in Fedora 39 via this change: https://fedoraproject.org/wiki/Changes/Python_Extension_Flags_Reduction We wanted to remove as many flags as possible, with this motivation: """ Python developers will get more upstream-like experience when building Python extension modules and also closer to building vanilla C programs. """ Note that removing -O2 specifically was not the primary motivation, but the removal was intentional at that time. -------- Now we build Python 3.13 with -O3 via this change: https://fedoraproject.org/wiki/Changes/Python_built_with_gcc_O3 The change proposal said: """ Other Python extension modules will remain bulidng as before, e.g. in RPM packages, they will still be built with -O2... """ However, I made a mistake and the -O3 flag leaked into sysconfig CFLAGS. The good news is this does not seem to affect RPM packages, they are still being built with -O2, like this: <flags embedded in sysconfig> <$CFLAGS from automatic %set_build_flags> E.g. ... -fcf-protection -fexceptions -O3 -O2 -flto=auto ... The latter flag wins. ---- OTOH users building their own extension modules will get -O3. This is not what was intended. However, "Python developers will get more upstream-like experience" is more true now, because upstream Python builds use -O3: $ podman run --rm -ti python:3.12 /usr/bin/bash # python >>> import sysconfig >>> sysconfig.get_config_var('CFLAGS') '-fno-strict-overflow -Wsign-compare -DNDEBUG -g -O3 -Wall' So the question is: Do we keep -O3 for user-built extension modules for speed and upstream-like experience? (I would update the -O3 change proposal.) Or do we loose the flag, as currently documented? Alternatively, do something else entirely (e.g. embed -O2, or other flag...)? -- Miro Hrončok -- Phone: +420777974800 Fedora Matrix: mhroncok

2 6

The RPM pyproject Declarative Buildsystem is available in Rawhide
by Miro Hrončok 13 Aug '24

13 Aug '24

Hello Pythonistas, pyproject-rpm-macros 1.14.0 landed in Fedora Rawhide and ELN. It defines a provisional Declarative Buildsystem for RPM 4.20+. It allows you to write Python specfiles like this: --------------------------------- Name: python-pello Version: 1.0.4 Release: %autorelease Summary: Example Python library License: MIT-0 URL: https://github.com/fedora-python/Pello Source: %{url}/archive/v%{version}/Pello-%{version}.tar.gz BuildSystem: pyproject BuildOption(install): -l pello BuildArch: noarch %description ... %package -n python3-pello Summary: %{summary} %description -n python3-pello ... %files -n python3-pello -f %{pyproject_files} %doc README.md %{_bindir}/pello_greeting --------------------------------- It will fill-in %prep, %generate_buildrequires, %build, %install, and %check sections for you. Yet you can configure/extend/override the sections if needed. Read about all the possibilities at https://src.fedoraproject.org/rpms/pyproject-rpm-macros/blob/rawhide/f/READ… in the *Provisional: Declarative Buildsystem (RPM 4.20+)* section. This is still *provisional* and the API might change in the future. Beware that RPM in Fedora 39/40 and EPELs does not understand this and RPM < 4.20 will not parse such spec files, so this is inconvenient to use unless you run Rawhide -- fedpkg prep, rpmbuild -bs, spectool -g, rpmdev-bumpspec won't work properly... If you don't use Rawhide, you can build a SRPM like this: mock -r fedora-rawhide-x86_64 --enablerepo=local --resultdir=./ --buildsrpm --spec python-pello.spec --source Pello-1.0.4.tar.gz Or use fedpkg with --srpm-mock. However, even some infra tools that don't run on Rawhide will likely fail with a package like this (e.g. Copr or Zuul). Use this at your own risk. Koji builds SRPMs for Rawhide on Rawhide, so it works there. Please experiment with this and report back how does it work for you. Happy packaging. -- Miro Hrončok -- Phone: +420777974800 Fedora Matrix: mhroncok

1 1

Default BuildOption(install) for the RPM pyproject declarative buildsystem
by Miro Hrončok 13 Aug '24

13 Aug '24

Hello. I am working on the RPM pyproject declarative buildsystem. t;dr it turns this: BuildSystem: pyproject into this: %prep %autosetup -p1 -C %generate_buildrequires %pyproject_buildrequires %build %pyproject_wheel %install %pyproject_install %pyproject_save_files ... %check %pyproject_check_import It allows to override options to sections with BuildOption. For detailed documentation, see https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/455 (it has README changes in it). --------------------------------- What should be the default for BuildOption(install)? That is, what should be passed to %pyproject_save_files if the packager does not override it? 1. The safe+hard default: nothing ================================= The %pyproject_save_files macro currently has no default. It requires at least one argument. This is to avoid accidentally packaging something we don't want. Explicit is better than implicit. For this reason, users of the pyproject declarative buildsystem would be required to always set BuildOption(install) explicitly. Why I don't like this option: The declarative buildsystem was invented to make spec files simpler. Making BuildOption(install) mandatory defeats the purpose. 2. The easy+dangerous default: '*' ================================== When we pass '*' to %pyproject_save_files by default, we ensure all packages work without BuildOption(install). This is the easy way. Everything is automagic, no boilerplate necessary. Passing '*' to %pyproject_save_files is forbidden by the Fedora packaging guidelines because it is dangerous. If upstream removes a Python module or (perhaps accidentally) adds a new one that conflicts with other packages, the Fedora packager should notice. Why I don't like this option: The default is forbidden in Fedora and hence it is only useful elsewhere (e.g. Copr). Defaults should encourage the desired pattern, defaults should not be forbidden. Packagers will use the default and the rule will be ignored as not practical. 3. Guess the module name: python-foo -> foo =========================================== When the package is named somehow, we like the Python modules to be named identical-ish. We can create a transformation of %{name} and make that the default. Something like this: 1. Strip python- prefix if found. 2. Strip everything after a dot (including the dot) if found. 3. Convert dashes, dots etc. to underscores. Some nice examples: python-requests -> requests python-zope.interface -> zope python-flit-core -> flit_core tomcli -> tomcli Examples that would need manual BuildOption(install): python-zope-interface -> zope_interface (we need zope) python-djangorestframework -> djangorestframework (we need rest_framework) pytest -> pytest (we need pytest _pytest py) This would produce a reasonable default that is easy and safe for sane packages but it is not dangerous for the weird ones. Option 3a) is to do this in the pyproject declarative buildsytsem macros implementation. Explicit callers of %pyproject_save_files would not be able to use this, which only has downsides. Option 3b) is to make this the behavior of %pyproject_save_files when it has no arguments. If additional module globs are necessary, all module globs need to be passed explicitly. Option 3c) is to make this the behavior of `%pyproject_save_files -n` and make -n the default BuildOption(install). Users of %pyproject_save_files could combine -n with explicit module glob list (e.g. `%pyproject_save_files -n _pytest py` for pytest). Why I don't like this: Its' heuristics. It smells like guesswork. I prefer things explicit. 4. Default to %{pypi_name} ========================== Similarly to option 3, we prefer the PyPI name and the module name to match. Hence, we could default to %{pypi_name} or %{srcname} like %{pypi_source} does. There would still need to be some transformations applied, because %{pypi_name} can contain dashes and dots, etc... Why I don't like this: %{pypi_name} is not a cardinal part of a package. It's not part of the Python packaging guidelines and does not have a clearly defined semantics (is it the project name or the name part of the source tarball?). It encourages packagers to define the macro and use it everywhere in the spec, making it harder to read. Moreover, we need to apply text transformations to it anyway, hence we might as well do them on %{name} directly (which does not have clear semantics either, but is always there). --------------------------------- If I had to pick, I'd go with 3c. IMHO it is the most sane option. What are your thoughts? Do you see other options? -- Miro Hrončok -- Phone: +420777974800 Fedora Matrix: mhroncok

4 7

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel August 2024