python-devel June 2021

python-devel@lists.fedoraproject.org

15 participants
18 discussions

package request: PyWaffle
by Matthew Miller 20 Dec '21

20 Dec '21

Hi! I'm using this https://github.com/gyli/PyWaffle for some visualizations for Fedora Project stats. I'm kind of out of the loop on the state of the art of python packaging, and wondered if some kind Python SIG person would like to take it on for me.

4 12

Macro to smoke-test-import a Python module in %check
by Miro Hrončok 04 Aug '21

04 Aug '21

Hello Python RPM packagers, based on some discussion in the "F35 Change: Python Packaging Guidelines overhaul (System-Wide Change proposal)" thread [0], I've drafted a macro that can help to test-import a Python module in %check when no other tests exist or are when they cannot be executed during build [1]. The semantics is quite simple: %check %py3_check_import mymodule mymodule.submodule When all listed modules are successfully imported, "nothing happens", when at lest one fails to import, the entire build fails. The above line is translated very-roughly to `python3 -c 'import mymodule, mymodule.submodule'` (see the implementation [0] for more accurate representation). Given the Python semantics, it is possible to use commas as module separators as well (but no parentheses). The %buildroot's %pythn3_site{lib,arch} is added to PYTHONPATH. The runtime dependencies are obviously needed for this to work, so they need to be manually BuildRequired or even better, generated in %generate_buildrequires via `%pyproject_buildrequires -r` to use this macro. The macro can be used repeatedly when importing multiple modules at once is undesired (e.g. when only one module can be imported from the same Python interpreter): %check %py3_check_import mymodule.either %py3_check_import mymodule.or Before I merge this and backport to all Fedoras and EPELs, I'd like to know: - Do you have better ideas for the macro name? - Do you have better ideas for the macro semantics? [0] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… [1] https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/99 -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

4 6

An unsuccessful case study: Using pyproject-rpm-macros with PyQt-builder and sip 5
by Miro Hrončok 22 Jul '21

22 Jul '21

Hello, since Scott is working on update to sip 5 (thank you!) I've figured this is a good time check if we can use the PyQt-builder Python build backend with our pyproject-rpm-macros. tl;dr It is broken in various ways and I am unsure where to attempt to fix things. ------------- This was my case study: I've used the python-pyqtchart package as a test subject, as it is fairly simple and has an open pull request to port it to sip5 [1]. I've pretty much reconstructed the spec file as I would normally do: %generate_buildrequires %pyproject_buildrequires %build %pyproject_wheel %install %pyproject_install To ease things up, I've kept the non-Python-packages BuilRequires intact, so the package still BuildRequires python3-qt5-devel, qt5-qtcharts-devel and qt5-qtbase-private-devel. I've removed the BuildRequires on python3dist(sip) and python3dist(pyqt-builder) as I've expected those to be generated. The simplified pyproject.toml file looks like this: [build-system] requires = ["sip >=5.3, <7", "PyQt-builder >=1.6, <2"] build-backend = "sipbuild.api" [tool.sip.metadata] name = "PyQtChart" version = "5.15.2" ... requires-dist = "PyQt5 (>=5.15)" [tool.sip] project-factory = "pyqtbuild:PyQtProject" [tool.sip.project] tag-prefix = "QtChart" [tool.sip.bindings.QtChart] qmake-QT = ["charts"] I've also tried with version 5.15.4, but not much is different. I've used the copr with sip5 [2] to build the package. ------------- %pyproject_buildrequires works and generates requirements on python3dist(sip) and python3dist(pyqt-builder) \o/ %pyproject_buildrequires -r doesn't work. It says: ValueError: build backend cannot provide build metadata (incl. runtime requirements) before buld Except for the typo [3], this is the expected behavior for a build backend that does not support the prepare_metadata_for_build_wheel hook. Bummer, but fair. We can RFE PyQt-builder to add that hook. ------------- %pyproject_wheel fails very badly. The traceback is unparseable by humans. It ends with: pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 1: ... Check the logs for full command output. I have no idea where are the logs. We might want to improve this in pip. After some digging, I've realized the problem is that PyQt-builder calls "qmake" and that command was not available. I've looked into the code and the "qmake" name is hardcoded, but searched on $PATH. The "original" non-pyproject spec has: sip-build ... --qmake="%{_qt5_qmake}" But there is no (known) way to pass such options to the backend. I was able to make it work by exporting PATH="%{_qt5_bindir}:$PATH". I was wondering whether the Fedora's sip5 (or PyQt-builder) package might change the default to make this work, but maybe there is no reasonable default here, as PyQt-builder is Qt version agnostic. Hence I don't know how to make it work out of the box. Not sure where to improve this experience. Anyway, workaround exists. ------------- During %pyproject_wheel, "make" is called internally by the backend. The build is not parallelized. I was able to workaround this by using MAKEFLAGS='%{?_smp_mflags}'. We might want to set this environment variable to '%{_make_output_sync} %{?_smp_mflags} %{_make_verbose}' in the %pyproject_wheel macro, so any backed that utilizes make gets the default arguments Fedora packages should use. Anyway, this is not a blocker to build the package. ------------- Processing files: python-pyqtchart-debugsource-5.15.4-1.fc35.x86_64 RPM build errors: error: Empty %files file /builddir/build/BUILD/PyQtChart-5.15.4 /debugsourcefiles.list This is a problem I get next. It likely means the build flags are not propagated. Our macros set the CFLAGS and LDFLAGS environment variables, but the build does not respect them. I've tried exporting CXXFLAGS as well, but no dice. Apparently, the build backend does not pass the variables to the build. I've tried a workaround like this: export MAKEFLAGS='%{?_smp_mflags} CPPFLAGS="%{build_cxxflags}"' But it makes the internal make call just dump the targets instead of actually building, so I guess I am doing it wrong. Either way, I think we need to fix PyQt-builder/sip to respect the flags from the environment variables. Funnily enbough, when I break make like this, it creates the wheel anyway, but with no dist-info metadata in it. I guess if users break make by setting wrong MAKEFLAGS, the wheel should not be created at all :/ To move forward, I've used %global debug_package %{nil}, but I consider not using the Fedora's flag a blocker before we recommend the macros for packages that use PyQt-builder. ------------- The next problem I got was: error: File not found: /builddir/build/BUILDROOT/python-pyqtchart-5.15.4-1.fc35.x86_64/usr/share/qt5/qsci/api/python/PyQtChart.api The "original" non-pyproject specfile had: sip-build ... --api-dir=%{_qt5_datadir}/qsci/api/python And as said previously, I don't know a way to pass such options to the backend. I am not skilled enough in Qt and SIP, so I have no idea if we need to ship the .api file or not. But if we need or want to, we should be able to. When the option is not used, the api file is not installed anywhere. I was unable to figure this out yet. ------------- That's it. I don't really know where to fix things, and I'd appreciate some pointers. This is not a very high priority for me right now, but once the new Python guidelines [4] are in place, I think we should make this work, possibly with some reasonable worarounds. [1] https://src.fedoraproject.org/rpms/python-pyqtchart/pull-request/6 [2] https://copr.fedorainfracloud.org/coprs/swt2c/pyqt5-sip5/ [3] https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/167 [4] https://fedoraproject.org/wiki/Changes/PythonPackagingGuidelines202x -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

4 4

PSA: /usr/lib/rpm/redhat/brp-python-hardlink: No such file or directory [and how to solve it]
by Miro Hrončok 30 Jun '21

30 Jun '21

If you see the following error in a Rawhide mock build: ... + /usr/lib/rpm/redhat/brp-python-hardlink /var/tmp/rpm-tmp.WFxggi: line 65: /usr/lib/rpm/redhat/brp-python-hardlink: No such file or directory error: Bad exit status from /var/tmp/rpm-tmp.WFxggi (%install) Update your mock chroot from the "local" repository: $ mock -r fedora-rawhide-x86_64 --enablerepo=local --update And don't clean it before running the build. If you see the error in Koji, please let me know ASAP. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 1

Intent to retire python2-setuptools
by Miro Hrončok 29 Jun '21

29 Jun '21

Hello Fedorans and especially Pythonistas. I'd like to retire python2-setuptools from Fedora 35+. It contains utilities to execute code downloaded from the internet and it has not been updated for 2 years. It has a potential of a security hole, although I am not currently aware of any. Unfortunately, 6 packages still (Build)Require python2-setuptools in Fedora 35: https://fedora.portingdb.xyz/pkg/python2-setuptools/ I've split them into 3 categories: 1) Trac plugins: trac-(customfieldadmin|monotone|watchlist)-plugin Note: trac-monotone-plugin is the only package that Requires python2-setuptools in Fedora 35 on runtime. Trac has been already updated to Python 3 in Fedora 34+ The plugins don't (and cannot) work with Python 2 when Trac runs on Python 3. Bugzillas exists for years: https://bugzilla.redhat.com/show_bug.cgi?id=1739034 https://bugzilla.redhat.com/show_bug.cgi?id=1739024 https://bugzilla.redhat.com/show_bug.cgi?id=1739042 They were updated this February to point out the uselessness of the packages, but nothign happened. I have a clear conscience if the packages stop building (and installing in case of trac-monotone-plugin). 2) Packages that BuildRequire python2-setuptools but don't use it: NFStest Pull request exists to remove the unused dependency: https://src.fedoraproject.org/rpms/NFStest/pull-request/2 3) Packages that BuildRequire python2-setuptools but don't have to: python2-cairo, python-psutil The two packages use this concept in setup.py: try: from setuptools import setup except ImportError: from distutils.core import setup technically, they don't need setuptools to build, the dependency is optional. Unfortunately they used setuptools to build in the past and if we remove it, the will change the .egg-info metadata directory to a file. Hence they will hit a well known limitation in RPM: https://docs.fedoraproject.org/en-US/packaging-guidelines/Directory_Replace… To compensate a rather ugly scriptlet is needed. The changes were proposed: https://src.fedoraproject.org/rpms/python-psutil/pull-request/10 https://src.fedoraproject.org/rpms/python2-cairo/pull-request/1 However, I'd rather maintain the 2 scriptlets than python2-setuptools. Please let me know if you see a problem. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 2

Heads-up: python-pyrsistent 0.18.0 coming to Rawhide with a minor API change
by Ben Beasley 29 Jun '21

29 Jun '21

In one week, or slightly later, I will update python-pyrsistent to 0.18.0 in Rawhide (https://bugzilla.redhat.com/show_bug.cgi?id=1977038) This includes one minor API change; please see the release notes at https://raw.githubusercontent.com/tobgu/pyrsistent/1b8e85ed0164a8f9908bee05… for the full list of changes. The breaking change is as follows: > Fix #209 Update freeze recurse into pyrsistent data structures and thaw to recurse into lists and dicts, > Thanks @phil-arh for this! > NB! This is a backwards incompatible change! To keep the old behaviour pass `strict=False` to freeze and thaw. The following packages depend directly on this package (maintainers in parentheses): • python-jsonschema (apevec, @openstack-sig) • python-poetry-core (thrnciar, churchyard, @python-sig) Maintainers of potentially-affected packages should have received this email directly. I have performed test rebuilds of both directly-dependent packages in a COPR (https://copr.fedorainfracloud.org/coprs/music/pyrsistent-0.18/) without incident, so I hope no changes to dependent packages will be required.

1 0

package review request
by Mukundan Ragavan 28 Jun '21

28 Jun '21

Hi, Can someone please review this relatively straightforward package? This is new dependency for spyder. https://bugzilla.redhat.com/show_bug.cgi?id=1977120 Thank you. Mukundan. -- GPG Key: E5C8BC67

1 0

Re: [Fedora-packaging] Macro to smoke-test-import a Python module in %check
by Miro Hrončok 28 Jun '21

28 Jun '21

On 28. 06. 21 22:25, Dan Čermák wrote: > Hi Miro, > > Miro Hrončok <mhroncok(a)redhat.com> writes: > >> Hello Python RPM packagers, >> >> based on some discussion in the "F35 Change: Python Packaging Guidelines >> overhaul (System-Wide Change proposal)" thread [0], I've drafted a macro that >> can help to test-import a Python module in %check when no other tests exist or >> are when they cannot be executed during build [1]. >> >> The semantics is quite simple: >> >> >> %check >> %py3_check_import mymodule mymodule.submodule >> >> ... > > This looks pretty good imho. Would it somehow be possible for the macro > to automatically try to import the module name from the generated > python3.Ydist() provides? That would make the macro even more convenient > to use and reduce any potential user error further. Honestly, I'd rather not do that. People already confuse "module names" (that's what you import) with "Python package names" (that's what you install from PyPI or what's provided as python3.Ydist()). And while this would make the macro slightly easier to use for the optimal case when those names are identical, it would only create more confusion when they are not. In the future, we could extend it to automatically import modules found in the %buidlroot, but now I'd rather provide a simple tool where the packager needs to be explicit, than a very complex tool that does heuristics. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

orphaning some python packages
by Mukundan Ragavan 27 Jun '21

27 Jun '21

I am orphaning the following packages. They should not be needed for anything else in Fedora (acc. to dnf repoquery --whatprovides). python-jsonrpcserver (there is another package of similar name that is needed by spyder) All these are required by the package above. python-aiozmq python-flask-socketio python-socketio python-engineio Feel free to take any of the packages. Mukundan. -- GPG Key: E5C8BC67

1 0

fedora python virtualbox
by Szymon Gawłowski 23 Jun '21

23 Jun '21

Hello there! I've installed Fedora-python on the Virtualbox and need to enter login & password, but I don't know where it is? What is login&password? -- *Szymon Gawłowski*

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel June 2021