python-devel May 2016

python-devel@lists.fedoraproject.org

13 participants
10 discussions

[Python-SIG] Self-Introduction: Tim Orling
by Tim Orling 15 Aug '16

15 Aug '16

Greetings, I recently became a co-maintainer for pystatgrab [1] and am a contributor to upstream [2]. I am a co-founder and co-maintainer of the meta-python [3] and meta-maker [4] layers for OpenEmbedded. I am also a contributor to IronPython [5] and RobotFramework [6]. In my professional life, I work for the Intel Open Technology Center on the Yocto Project, which depends heavily on Python for its tools and infrastructure. [1] https://admin.fedoraproject.org/pkgdb/package/rpms/pystatgrab/ [2] https://github.com/i-scream/pystatgrab/graphs/contributors [3] https://layers.openembedded.org/layerindex/branch/master/layer/meta-python/ [4] https://layers.openembedded.org/layerindex/branch/master/layer/meta-maker/ [5] https://github.com/orgs/IronLanguages/people [6] https://github.com/robotframework/robotframework/graphs/contributors

5 12

Reg- binclock porting from python 2 to python 3
by Rajeshkumar Pothiappan 15 Jun '16

15 Jun '16

Hi Fedora Team, I would like to contribute in porting binclock from python 2 to python 3. I found http://fedora.portingdb.xyz/pkg/binclock/ https://admin.fedoraproject.org/pkgdb/package/rpms/binclock/ Please guide me where to start with or whom to contact and how to . by Rajeshkumar P

5 7

krop and python3
by Raphael Groner 14 Jun '16

14 Jun '16

Hi, I need some help to fix a bug ¹ with python3 and site-packages. > cd /usr/lib/python3.4/site-packages/krop/ > python2 -c 'import krop ; krop.main()' → dialog > python3 -c 'import krop ; krop.main()' Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/lib/python3.4/site-packages/krop/krop.py", line 25, in main formatter_class=RawTextHelpFormatter) TypeError: __init__() got an unexpected keyword argument 'version' Besides that error, the start script in %{_bindir}/krop is b0rken and should just work for users clicking on the desktop icon. ¹ https://bugzilla.redhat.com/show_bug.cgi?id=1321376

2 5

Alternatives for python versions
by Avram Lubkin 30 May '16

30 May '16

I wasn't able to find much recent discussion on this, but I'm a bit frustrated that there is so much manual work being done creating symlinks when a system already exists for that. It seem many projects which support both python2 and python3 are depending on a hardcoded shebang, install order, or symlink creation in the spec file to set which version of python to use. It also seems the py*_build macros are hardcoded to use a versioned python interpreter for shebang lines. Frankly I feel like I should be able to use /usr/bin/python or /usr/bin/env python and that should point to a non version-specifc python interpreter. If I have definite dependencies on a specific interpreter, then it is my responsibility to set my shebang appropriately. Otherwise, if my project has an unversioned executable, I have to change it based on my will and the Fedora version, rather than the will of the end user. Maybe they prefer python2, python3, or some other distribution of python becomes popular later (pypy?). If I don't have hard dependencies, why should I force a version? And why should I need to have conditionals on the Fedora version or maintain two spec files? Assuming an end user never touches the default configuration, dependencies should already be taken care of with python_provides since unversioned dependencies should default to the release default. Whether /usr/bin/python is symlinked using alternatives or another method, the py*_build macros should have a mechanism to overwrite the executable value. Avram

1 0

Problems with scripts in a common spec file
by John Dennis 30 May '16

30 May '16

With reference to the guidelines for Python packaging found here: https://fedoraproject.org/wiki/Packaging:Python Specifically the material concerning executables in /usr/bin. In the "Example common spec file" section is this comment. # Must do the python2 install first because the scripts in /usr/bin are # overwritten with every setup.py install, and in general we want the # python3 version to be the default. But this does not work if you only install the python2-XXX package instead of both the python2-XXX and python3-XXX packages. Here is why. distutils will adjust the shbang interpreter line in installed scripts to be interpreter specific, e.g. you get one of: #!/usr/bin/python2 or #!/usr/bin/python3 But the guidelines require the py3 version of the script in the py2 package. Thus if you install just the python2-XXX package you'll end up with a script that runs the /usr/bin/python3 interpreter whose sys.path only includes library locations for py3! Thus the script cannot execute because it's missing its libraries (because the libraries were installed in the py2 locations). How is this supposed to be addressed? One possible workaround is to add a requirement on the py3 package in the py2 package. But that is silly because it completely negates everything in the py2 package (everything will run as if it were py3 after installing the py2 package and the py2 files will never be referenced). Perhaps the issue is better rephrased as "How does one package Python when there are no version specific differences/requirements?" If this is covered in the guidelines it's not obvious and needs clarification. I assume this means using the generic %{_python} macros instead of dual use of the version specific macros so that everything corresponds to the system default version of Python. Correct? -- John

2 5

New PyPI URL format for tarballs/sources
by Tomas Orsava 26 May '16

26 May '16

Hi! The Python Package Index (PyPI) has decided to change the format of the URLs to download tarballs/sources. The new format is not predictable, because part of the URL is a hash of the contents of the file. [0] This means that for the vast majority of Python packages (those using PyPI to download sources) the Source0 link needs to be updated. The maintainer can either go to PyPI with each version update and copy the URL containing the hash, or use the new URL redirector provided by PyPI with a predictable format (like the previous one was): https://files.pythonhosted.org/packages/source/p/positional/positional-1.1.… The URL redirector will have longterm support [1], so I believe it's the better choice. Maybe it would be beneficial to also work the change into the RPM rebase-helper [2] which does automatic scratch builds when a new version of software is detected upstream? Does anyone have experience with this project? [0] https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-p… [1] https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-p… [2] https://github.com/phracek/rebase-helper Great day to all, Tomas Orsava

1 0

Applying for Python SIG
by Ardian Haxha 21 May '16

21 May '16

Hi there, I am Ardian a student from Kosovo. I have been contributing to fedora for quite some time as an Ambassador, I am almost done with my studies so there is more free time, I would like to contribute in the coding/packaging different parts of the project. Python being one of my first and favorite programming languages is what I would like to get started. I have already build an RPM with the help of Robert Scheck (I am very thankful). I would like to join the python interest group. Kindly, Ardian.

1 0

Python-SIG introduction
by Charalampos Stratakis 19 May '16

19 May '16

Hello, I am a member of Red Hat's Python Maintenance team and I have applied for the FAS group as well as the mailing list. I am interested in Python in Fedora, upstream and well, everything python related. Regards, Charalampos Stratakis

1 0

Introduction (cydrobolt)
by Chaoyi Zha 18 May '16

18 May '16

Hey python-devel, I'm Chaoyi, a Fedora user and contributor. I've been using Python for a couple years now, and it has been a great experience. I write my apps in Python whenever I can because of its readability and versatility. I found out about the python-sig through a Trac ticket. I love Python and would love to be involved in the Python community at Fedora. I mostly contribute to Fedora on the infrastructure team, but I also work with the ambassadors and web teams, among others. My FAS and IRC username is "cydrobolt" Thanks, Chaoyi

1 0

Problems with script installation in RPM's
by John Dennis 13 May '16

13 May '16

I've been following the guidelines for Python packaging found here: https://fedoraproject.org/wiki/Packaging:Python in particular the cookbook for supporting both Py2 and Py3. I've discovered two places where things fail to work as expected with respect to script installation. Both of these problems took an inordinately long time to fully diagnose. The first problem I believe is a bug in the python rpm macros and needs to be fixed (filed as bug #1335203). The second problem can be fixed via a workaound in the spec file, but the cookbook recipe will need to be updated to call attention to it. Problem 1: ---------- If the script includes an interpreter argument in it's shebang line the installed script cannot execute and aborts. For example if the source script has a shebang line like this: #!/usr/bin/python -E The installed script will have a shebang line like this (using Py2 as an example) #!/usr/bin/python2 -s -E The problem is how the shebang line is parsed to build parameters for exec. Unfortunately there is no standard. Some operating systems split at whitespace and treat the first part as the path to the interpreter and the rest as individual arguments. Some operating systems split at the first whitespace and treat the front part as the path to the interpeter and the rest as a single argument (this is what happens in Linux). Thus when the Python interpreter is run it sees argv[1] as "-s -E" which it doesn't understand as a single argument and the result is: Unknown option: - usage: /usr/bin/python [option] ... [-c cmd | -m mod | file | -] [arg] ... Try `python -h' for more information. The cause of the problem is in /usr/lib/rpm/macros.d/macros.python2 and /usr/lib/rpm/macros.d/macros.python3. I'll show the Py2 version but the Py3 version is equivalent. %py2_shbang_opts -s %py2_build() %{expand:\ CFLAGS="%{optflags}" %{__python2} %{py_setup} %{?py_setup_args} build --executable="%{__python2} %{py2_shbang_opts}" %{?1}\ } Then in distutils/command/build_scripts.py it uses the first_line_re regular expression to split the shebang line into 2 parts, the interpreter and the interpreter arguments. It then replaces the shebang line with the value of --executable followed by a space and interpreter arguments the regexp found. But in our case the interpreter is *not* the path to an interpreter, rather it's the path to the interpreter *plus* the %py2_shbang_opts. Thus you end up with multiple arguments to the interpreter, but the interpreter doesn't see it as multiple arguments, instead it sees 1 argument, a single string containing "-s -E". I think the fundamental problem is that macros.python{2,3} is abusing the --executable argument, it's meant to be the path to an executable *only*. It clearly does not expect extra arguments. If build_scripts.py had logic to merge arguments together, e.g. -s -E would become -sE then all would be good, but it doesn't. Problem 2: ---------- The Py2 version of the script (which forces /usr/bin/python2 in it's shebang line) is installed in the python3 package. Thus if you install python3-NVR and it has a script in /usr/bin you'll end up executing python2 instead of python3. The cookbook shows this snippet: %install # Must do the python2 install first because the scripts in /usr/bin are # overwritten with every setup.py install, and in general we want the # python3 version to be the default. %py2_install %py3_install The problem is this is exactly the *opposite* of what actually happens if builds are fast. %py2_install runs first and copies the Py2 version of the script into $BUILDROOT/usr/bin. Next %py3_install runs and an attempt is made to install the Py3 version into the exact same location $BUILDROOT/usr/bin. However distutils has logic which avoids copying over an existing installed file unless the src is newer. This occurs in distutils/file_util.py which has this code in copy_file(): if update and not newer(src, dst): if verbose >= 1: log.debug("not copying %s (output up-to-date)", src) return dst, 0 The documentation for newer() in distutils/dep_util.py acknowledges the fundamental problem: Note that this test is not very accurate: files created in the same second will have the same "age". This is exactly what was happening. %py2_build and %py3_build each created the script with the same timestamp and the logic in copy_file() concluded it shouldn't replace the Py2 script with the Py3 script. The result is the python3-package ended up with a script with a Py2 interpreter shebang line, clearly this is wrong. The workaround I came up with is to delay the execution of %py3_build by at least 1 second by inserting a sleep in-between the %py2_build and %py3_build macros in the spec file, like this: %py2_build sleep 1 %py3_build Actually I think sleeping 2 seconds might be safer given the 1 second resolution on the timestamps. BTW, the fact the file was not updated is silent in the output of the setup.py command. Note the message is contained in a log.debug, not a log.info. I could find no way to enable debug log output using the command line arguments. I had to resort to adding the following in my setup.py script in order to see the message: import distutils.log distutils.log.set_verbosity(distutils.log.DEBUG) # Set DEBUG level -- John

6 10

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel May 2016