python-devel September 2019

python-devel@lists.fedoraproject.org

9 participants
9 discussions

Building the python executable statically with libpython*.a for better performance
by Miro Hrončok 04 Nov '19

04 Nov '19

Hello, we've been recently approached by a colleague from Red Hat working on performance (CCed). According to their testing, Fedora Python performance could be improved by ~15% by building /usr/bin/python* statically with libpython*.a. That sounds like a worthy thing to do. https://bugzilla.redhat.com/show_bug.cgi?id=1749479 Since Python 3.8 Python extension modules are no longer linked to libpython.so and we can do the following: * build /usr/bin/python3(.8) statically with libpython*.a * build and ship libpython3.8.so.1.0 for packages that "embed" Python The change in the python3 package is trivial: https://src.fedoraproject.org/rpms/python3/pull-request/133 However it can have serious impact on Python extension modules that are linked to libpython3.8.so.1.0 by various "nonstandard" build mechanisms or by compiling code for Python extension module and code that embeds Python into one file. We will likely propose a Fedora 32 Change for this, however I'm opening this topic for discussion before we do so. Testing the proposed Pull Request with your code is also helpful. Let me know how can we make that easier (e.g. if you want a Copr or a Fedora 30/31 python38 package with this change). WDYT? -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

4 6

cython issues on EL8
by Orion Poplawski 27 Sep '19

27 Sep '19

I could use some help with some cython issues building pandas for EL8. First attempt: https://koji.fedoraproject.org/koji/taskinfo?taskID=37887295 running build_ext pandas._libs.algos: -> [['pandas/_libs/algos.c']] Traceback (most recent call last): File "setup.py", line 838, in <module> **setuptools_kwargs File "/usr/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup return distutils.core.setup(**attrs) File "/usr/lib64/python3.6/distutils/core.py", line 148, in setup dist.run_commands() File "/usr/lib64/python3.6/distutils/dist.py", line 955, in run_commands self.run_command(cmd) File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/usr/lib64/python3.6/distutils/command/build.py", line 135, in run self.run_command(cmd_name) File "/usr/lib64/python3.6/distutils/cmd.py", line 313, in run_command self.distribution.run_command(command) File "/usr/lib64/python3.6/distutils/dist.py", line 974, in run_command cmd_obj.run() File "/usr/lib64/python3.6/distutils/command/build_ext.py", line 339, in run self.build_extensions() File "setup.py", line 407, in build_extensions self.check_cython_extensions(self.extensions) File "setup.py", line 402, in check_cython_extensions src=src Exception: Cython-generated file 'pandas/_libs/algos.c' not found. Cython is required to compile pandas from a development branch. Please install Cython or download a release package of pandas. Compared to a rawhide build, it seems like the cython c files are not being automatically regenerated at the start of the build. What component is supposed to handle this? Second attempt to run cython by hand generates another error: https://koji.fedoraproject.org/koji/taskinfo?taskID=37887587 + cython ./pandas/_libs/missing.pyx ./pandas/_libs/testing.pyx ./pandas/_libs/interval.pyx ./pandas/_libs/window.pyx ./pandas/_libs/internals.pyx ./pandas/_libs/join.pyx ./pandas/_libs/index.pyx ./pandas/_libs/writers.pyx ./pandas/_libs/groupby.pyx ./pandas/_libs/reduction.pyx ./pandas/_libs/hashtable.pyx ./pandas/_libs/parsers.pyx ./pandas/_libs/tslibs/conversion.pyx ./pandas/_libs/tslibs/np_datetime.pyx ./pandas/_libs/tslibs/period.pyx ./pandas/_libs/tslibs/timedeltas.pyx ./pandas/_libs/tslibs/ccalendar.pyx ./pandas/_libs/tslibs/strptime.pyx ./pandas/_libs/tslibs/resolution.pyx ./pandas/_libs/tslibs/fields.pyx ./pandas/_libs/tslibs/c_timestamp.pyx ./pandas/_libs/tslibs/offsets.pyx ./pandas/_libs/tslibs/timestamps.pyx ./pandas/_libs/tslibs/parsing.pyx ./pandas/_libs/tslibs/tzconversion.pyx ./pandas/_libs/tslibs/timezones.pyx ./pandas/_libs/tslibs/frequencies.pyx ./pandas/_libs/tslibs/nattype.pyx ./pandas/_libs/hashing.pyx ./pandas/_libs/tslib.pyx ./pandas/_libs/indexing.pyx ./pandas/_libs/sparse.pyx ./pandas/_libs/algos.pyx ./pandas/_libs/skiplist.pyx ./pandas/_libs/ops.pyx ./pandas/_libs/reshape.pyx ./pandas/_libs/lib.pyx ./pandas/_libs/properties.pyx ./pandas/io/sas/sas.pyx ./pandas/io/msgpack/_packer.pyx ./pandas/io/msgpack/_unpacker.pyx Error compiling Cython file: ------------------------------------------------------------ ... cdef int status with nogil: status = tokenize_nrows(self.parser, nrows) if self.parser.warn_msg != NULL: print(self.parser.warn_msg, file=sys.stderr) ^ ------------------------------------------------------------ pandas/_libs/parsers.pyx:932:44: Expected ')', found '=' It this because cython is too old in EL8? TIA, Orion -- Orion Poplawski Manager of NWRA Technical Systems 720-772-5637 NWRA, Boulder/CoRA Office FAX: 303-415-9702 3380 Mitchell Lane orion(a)nwra.com Boulder, CO 80301 https://www.nwra.com/

2 2

Python 3.8 porting help
by Richard Shaw 26 Sep '19

26 Sep '19

I maintain FreeCAD for Fedora and there appears to have been a change in the API for how Python handles thread management in 3.8, specifically PyInterpreterState which is giving the the following error when trying to build FreeCAD: In file included from /builddir/build/BUILD/FreeCAD-0.18.3/src/Base/swigpyrun.cpp:41: /builddir/build/BUILD/FreeCAD-0.18.3/src/Base/swigpyrun.inl: In function 'void Swig_python::cleanupSWIG_T(const char*)': /builddir/build/BUILD/FreeCAD-0.18.3/src/Base/swigpyrun.inl:75:31: error: invalid use of incomplete type 'PyInterpreterState' {aka 'struct _is'} 75 | PyObject *modules = interp->modules; | ^~ In file included from /usr/include/python3.8/genobject.h:11, from /usr/include/python3.8/Python.h:121, from /builddir/build/BUILD/FreeCAD-0.18.3/src/Base/PyExport.h:42, from /builddir/build/BUILD/FreeCAD-0.18.3/src/Base/swigpyrun.cpp:25: /usr/include/python3.8/pystate.h:20:8: note: forward declaration of 'PyInterpreterState' {aka 'struct _is'} 20 | struct _is; | ^~~ Any tips? I can provide a full build log or setup a COPR but it's somewhat difficult since I'm building a whole new stack of some dependencies: Coin3->Coin4 (in review) - Which SoQt, SIMVoleon, OpenSceneGraph and Pivy need to be ported to. Pyside->PySide2 (which implies Qt4->Qt5) Thanks, Richard

3 5

libselinux-python3 for EPEL 7
by Carl George 20 Sep '19

20 Sep '19

Red Hat added python3 to RHEL 7.7, but at least for now has declined to turn on the python3 subpackage in the libselinux spec file. https://bugzilla.redhat.com/show_bug.cgi?id=1719978 I have attempted to to extract the python3 subpackage into it's own spec file. It seems to work as best I can tell. I have submitted it for review as an EPEL7-only package. https://bugzilla.redhat.com/show_bug.cgi?id=1754039 I would love feedback on this if anyone has time to take a look.

1 0

package review request: python-wurlitzer
by Mukundan Ragavan 15 Sep '19

15 Sep '19

Can someone review this package for me? This should be fairly straightforward. https://bugzilla.redhat.com/show_bug.cgi?id=1752298 Thanks, Mukundan.

1 0

[HEADS UP] Retiring python2 and introducing python27 later this week
by Miro Hrončok 11 Sep '19

11 Sep '19

We are retiring python2 and introducing python27 later this week. Rawhide only. As for now, nothing should break, except python2-debug will exist no more. Packages (build)requiring python2 or python2-devel should continue to work for now. If not, let us know. If you plan to keep a Python 2 package in Fedora 32+, talk to me, I can help you draft an exception request. Will post a reply here once it is done. The following packages build subpackages for python2-debug (and should stop doing it in Fedora 32+): - gcc-python-plugin (FTBFS) - python-mysql (PR exists) - python-psycopg2 (notified in an existing bugzilla) -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

4 11

Please help proofread the Fedora Loves Python brochure
by Miro Hrončok 10 Sep '19

10 Sep '19

Hello Pythonistas, the new Fedora Loves Python brochure is getting finalized. I'd appreciate if you could help proofreading it (check both facts and grammar/typos). Fedora Design ticket: https://pagure.io/design/issue/541 Link to the PDF: https://pagure.io/design/issue/541#comment-596434 If you do, please share your concerns in the ticket. Thanks. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

nbconvert 5.6.0
by Mukundan Ragavan 03 Sep '19

03 Sep '19

I am unable to build nbconvert because of build failures. Can someone please take a look at the build log and help me update this? Build log - https://koji.fedoraproject.org/koji/taskinfo?taskID=37430133 Thanks, Mukundan.

2 4

Python 3.8.0b4 ready in rawhide and updates-testing
by Miro Hrončok 03 Sep '19

03 Sep '19

Python 3.8.0b4 is ready in rawhide (as python3). The python38 package was updated in F31, F30 and F29: https://bodhi.fedoraproject.org/updates/FEDORA-2019-d11594bf0a https://bodhi.fedoraproject.org/updates/FEDORA-2019-4954d8773c https://bodhi.fedoraproject.org/updates/FEDORA-2019-d58eb75449 Please test your code with Python 3.8.0b4 extensively and report issues. -------- Forwarded Message -------- Subject: [Python-Dev] [RELEASE] ACTION REQUIRED: Python 3.8.0b4 now available for testing Date: Fri, 30 Aug 2019 11:09:37 +0200 From: Łukasz Langa <lukasz(a)langa.pl> To: Python Committers <python-committers(a)python.org>, Python-Dev <python-dev(a)python.org>, python-list(a)python.org, python-announce(a)python.org It's time for the *last beta release* of Python 3.8. Go find it at: https://www.python.org/downloads/release/python-380b4/ This release is the *last* of four planned beta release previews. Beta release previews are intended to give the wider community the opportunity to test new features and bug fixes and to prepare their projects to support the new feature release. The next pre-release of Python 3.8 will be 3.8.0c1, the first release candidate, currently scheduled for 2019-09-30. *Call to action* We strongly encourage maintainers of third-party Python projects to test with 3.8 during the beta phase and report issues found to the Python bug tracker as soon as possible. Please note this is the last beta release, there is not much time left to identify and fix issues before the release of 3.8.0. If you were hesitating trying it out before, now is the time. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (2019-09-30). Our goal is have no ABI changes after beta 3 and no code changes after 3.8.0c1, the release candidate. *To achieve that, it will be extremely important to get as much exposure for 3.8 as possible during the beta phase. That beta phase is coming to an end. Please test now.* Please keep in mind that this is a preview release and its use is not recommended for production environments. *Acknowledgments* Many developers worked hard for the past four weeks to squash remaining bugs, some requiring non-obvious decisions. Many thanks to the most active, namely Raymond Hettinger, Steve Dower, Victor Stinner, Terry Jan Reedy, Serhiy Storchaka, Pablo Galindo Salgado, Tal Einat, Zackery Spytz, Ronald Oussoren, Neil Schemenauer, Inada Naoki, Christian Heimes, and Andrew Svetlov. 3.8.0 would not reach the Last Beta without you. Thank you! - Ł

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel September 2019