python-devel August 2018

python-devel@lists.fedoraproject.org

17 participants
16 discussions

self-introduction mythcat
by Cătălin George Feștilă 09 Aug '18

09 Aug '18

Dear team. My name is Cătălin George Feștilă known as mythcat into Fedora area. I want to join to this group because I used python : https://python-catalin.blogspot.com/ I'm born 1976 - Romania Thank you. Best regards.

2 1

Why are we shipping debug builds of pythons?
by Miro Hrončok 09 Aug '18

09 Aug '18

Hi, an interesting discussion came up in the Python Maint team recently, about not shipping python3-debug and python2-debug. On the Chesterton's fence principle [0], I'd would like to know why are we building and shipping them before we have a discussion about their removal to save build time and remove packaging cruft. Anyone has an idea? Those packages are meant to debug Python, yet all people I know who do that, build they own Python for that purpose (often from the master branch). I tracked down the introduction of the python-debug package in this commit [1] by David Malcolm (CCed) @ 8 years ago, added in Fedora 14 shortly before upgrade to 2.7. Yet the commit message lacks rationale. [0] https://en.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence [1] https://src.fedoraproject.org/rpms/python/c/f020abd35954981b383884105dad425… Thanks, -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

6 7

Python Bundled Wheels package
by Miro Hrončok 08 Aug '18

08 Aug '18

Hi Pythonistas and Fedora packagers. Recently I've realized we bundle too much wheels with our pythons + virtualenv package. That is unfortunate: * we don't build those. stricly seeking, it's just a zip with python files in it, yet this is not permitted in Fedora * we only sometimes list it as Provides: bundled(...) and when we do, it is tedious * we never list bundled deps in those bundled wheels (pip bundles a lot) * we never adapt the license tag to include license of bundled wheels (and bundled deps in those) - it would be even more tedious, as pip License tag can be very complicated * we duplicate those across packages I went ahead and prepared a concept in: https://bugzilla.redhat.com/show_bug.cgi?id=1605156 This is one package that builds all the required wheels. It might be a bit weird that it's only one package, but I think it can lower the maintenance burden. Also, we won't update any wheel package, we only add or remove them, so there is no "life cycle". Later we can decide that there are simply too many thing sin one package and split it). This package makes sure the license tag is right and all the virtual bundled provides are in place. Even as one package, I think it's a big improvement comparing to current state of things. Could you please review the decisions made in the spec? Namely: * naming (main package, subpackages) * virtual provides * that the spec is generated by a script and how that script works * the method of usage described in the package review request I've also decided not to run tests, as for them to mean something, we would need to run them against all relevant Python versions. Also, it would complicate the package a lot. The package is approved thanks to Robert-André Mauchin, yet I won't request the repo until it's settled that we want this. Also, once we start using this, maybe we can stop doing rewheel and just use wheels from here in the python3 package as well? Thanks. -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 3

HEADS UP: /usr/bin/cython(ize) is now in python3-Cython
by Miro Hrončok 08 Aug '18

08 Aug '18

Cython-0.28.4-3.fc29 has landed in rawhide. It moves all stuff in /usr/bin to pytyon3 package only, because the functionality is the same on whatever Python version. This might break your package if you BR Cython. Here are some tips that should be Fedora/EPEL version agnostic: If you need to use /usr/bin/cython (or /usr/bin/cythonize) during build, BR /usr/bin/cython (or /usr/bin/cythonize). If you need to use Cython from Python (from cython import...), BR pytnonX-Cython or pythonXdist(cython) (for X={2,3} as needed). If you would like to use Cython from the command line and explicitly use a Python version (e.g. not to drag an unnecessary dependency on the other Python version), you can use %{__pythonX} -m cython or pythonX -m cython. I'll give it some time to settle and later I will scratch build all packages that BR Cython and see if we broke them, will apply fixes. Packages that use Cython at build time: Maintainers by package: CheMPS2 talcite PyYAML jeckersb autowrap sagitter blueman pwalter borgbackup bpereto fschwarz brltty jskarvad limb olysonek bzr hno pstodulk vvitek cassandra acaringi hhorak trepik ceph branto dachary dmick ke4qqq kkeithle ktdreyer steve stingray compizconfig-python jskarvad raveit65 dionaea rebus eclipse-pydev akurtakov jjohnstn espresso deji junghans tomspur git-fame fale ignatenkobrain h5py stevetraylen terjeros ipython churchyard cstratak dcantrel ignatenkobrain lbalhar mrunge salimma tomspur libfreenect jkastner kwizart rmattes libmacaroons bbockelm libplist pbrobinson netcdf4-python orion numpy cstratak jspaleta limb orion rdieter tomspur ttomecek openfst jjames mikep openms sagitter petsc4py sagitter pocketsphinx jjames mikep printrun churchyard pygame jkaluza jskarvad limb pygrib jdekloe pyliblo fab pymilia sergiopr pyproj jdekloe pystatgrab fab heliocastro potty slankes ttorling python-MDAnalysis rathann python-admesh churchyard python-afl mcyprian orphan python-aiohttp fab ignatenkobrain python-assimulo sagitter python-astropy orion sergiopr python-astropy-healpix lupinix python-astroscrappy lupinix python-basemap jspaleta limb python-bintrees rkuska python-cartopy qulogic python-cassandra-driver filabrazilska lbalhar lkundrak praiskup python-cffi brouhaha jdulaney python-compreffor athoscr python-cradox jpena python-cypari2 pcpa python-cysignals pcpa python-djvulibre bstinson python-falcon abompard carlwgeorge jamielennox python-fiona qulogic python-fpylll pcpa python-gensim besser82 python-giacpy sagitter python-gmpy2 jjames orion python-gsd rathann python-gssapi rharwood simo sross python-healpy lupinix python-hidapi besser82 python-httptools ignatenkobrain python-hug hguemar python-jnius raphgro python-kmod grover python-libdiscid ohaessler python-line_profiler jacksonisaac python-lupa dhanesh95 python-lxml fale ignatenkobrain mizdebsk python-meliae lmacken tadej python-mistune lupinix python-msgpack dfateyev fab ktdreyer pjp sundaram python-multidict athmane ignatenkobrain python-nipy ignatenkobrain python-numpy-stl churchyard python-orderedset jjames python-pandas kushal orion sergiopr wakko666 python-peewee cstratak mstuchli vkrizan python-photutils sergiopr python-plyvel dcallagh python-pybloomfiltermmap athmane python-pyclipper athoscr python-pymssql melmorabity python-pyopengl swt2c python-pytaglib ignatenkobrain python-pywt ignatenkobrain sergiopr python-rasterio qulogic python-rencode jgu orion patches python-scikit-image orion sergiopr python-scikit-learn besser82 ignatenkobrain lupinix sergiopr python-shapely jcp volter python-slixmpp fantom python-smartcols ignatenkobrain python-ssh2-python ignatenkobrain python-statsmodels sergiopr python-tables tnorth zbyszek python-theano ignatenkobrain jjames python-thriftpy ignatenkobrain python-tinycss brouhaha python-uvloop ignatenkobrain python-yarl fab ignatenkobrain python-zmq ralph tomspur python2-astropy lupinix python2-ipython lbalhar python3-cangjie bochecha sagemath pcpa sphinxbase jjames mikep sphinxtrain jjames mikep sympy cbm jjames jussilehtola orion urh jskarvad xpra jgu sagitter sergiomb Packages by maintainer: abompard python-falcon acaringi cassandra akurtakov eclipse-pydev athmane python-multidict python-pybloomfiltermmap athoscr python-compreffor python-pyclipper bbockelm libmacaroons besser82 python-gensim python-hidapi python-scikit-learn bochecha python3-cangjie bpereto borgbackup branto ceph brouhaha python-cffi python-tinycss bstinson python-djvulibre carlwgeorge python-falcon cbm sympy churchyard ipython printrun python-admesh python-numpy-stl cstratak ipython numpy python-peewee dachary ceph dcallagh python-plyvel dcantrel ipython deji espresso dfateyev python-msgpack dhanesh95 python-lupa dmick ceph fab pyliblo pystatgrab python-aiohttp python-msgpack python-yarl fale git-fame python-lxml fantom python-slixmpp filabrazilska python-cassandra-driver fschwarz borgbackup grover python-kmod heliocastro pystatgrab hguemar python-hug hhorak cassandra hno bzr ignatenkobrain git-fame ipython python-aiohttp python-httptools python-lxml python-multidict python-nipy python-pytaglib python-pywt python-scikit-learn python-smartcols python-ssh2-python python-theano python-thriftpy python-uvloop python-yarl jacksonisaac python-line_profiler jamielennox python-falcon jcp python-shapely jdekloe pygrib pyproj jdulaney python-cffi jeckersb PyYAML jgu python-rencode xpra jjames openfst pocketsphinx python-gmpy2 python-orderedset python-theano sphinxbase sphinxtrain sympy jjohnstn eclipse-pydev jkaluza pygame jkastner libfreenect jpena python-cradox jskarvad brltty compizconfig-python pygame urh jspaleta numpy python-basemap junghans espresso jussilehtola sympy ke4qqq ceph kkeithle ceph ktdreyer ceph python-msgpack kushal python-pandas kwizart libfreenect lbalhar ipython python-cassandra-driver python2-ipython limb brltty numpy pygame python-basemap lkundrak python-cassandra-driver lmacken python-meliae lupinix python-astropy-healpix python-astroscrappy python-healpy python-mistune python-scikit-learn python2-astropy mcyprian python-afl melmorabity python-pymssql mikep openfst pocketsphinx sphinxbase sphinxtrain mizdebsk python-lxml mrunge ipython mstuchli python-peewee ohaessler python-libdiscid olysonek brltty orion netcdf4-python numpy python-astropy python-gmpy2 python-pandas python-rencode python-scikit-image sympy orphan python-afl patches python-rencode pbrobinson libplist pcpa python-cypari2 python-cysignals python-fpylll sagemath pjp python-msgpack potty pystatgrab praiskup python-cassandra-driver pstodulk bzr pwalter blueman qulogic python-cartopy python-fiona python-rasterio ralph python-zmq raphgro python-jnius rathann python-MDAnalysis python-gsd raveit65 compizconfig-python rdieter numpy rebus dionaea rharwood python-gssapi rkuska python-bintrees rmattes libfreenect sagitter autowrap openms petsc4py python-assimulo python-giacpy xpra salimma ipython sergiomb xpra sergiopr pymilia python-astropy python-pandas python-photutils python-pywt python-scikit-image python-scikit-learn python-statsmodels simo python-gssapi slankes pystatgrab sross python-gssapi steve ceph stevetraylen h5py stingray ceph sundaram python-msgpack swt2c python-pyopengl tadej python-meliae talcite CheMPS2 terjeros h5py tnorth python-tables tomspur espresso ipython numpy python-zmq trepik cassandra ttomecek numpy ttorling pystatgrab vkrizan python-peewee volter python-shapely vvitek bzr wakko666 python-pandas zbyszek python-tables -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

1 0

Re: Why are we shipping debug builds of pythons?
by David Malcolm 06 Aug '18

06 Aug '18

On Mon, 2018-08-06 at 13:15 +0200, Jan Kratochvil wrote: > On Mon, 06 Aug 2018 10:57:31 +0200, Petr Viktorin wrote: > > On 08/05/18 14:01, Jan Kratochvil wrote: > > > That is all together messy. This is why Microsoft has their debug > > > build of > > > their whole OS - Windows - called Checked Build: > > > https://docs.microsoft.com/en-us/windows-hardware/drivers/devte > > > st/checked-and-free-build-differences > > > > The main issue with --with-pydebug is that it changes the ABI, i.e. > > all > > extensions need to be re-built specifically for it (and debug > > extensions > > outside the standard library aren't usually packaged in Fedora). > > That makes > > it much less useful than if it just used less optimizations. Petr: IMHO, it's much *more* useful: it checks for leaked objects, which is the most painful issue to deal with when debugging Python extensions. That was my motivation for adding it. > > That confirms the whole OS "Checked Build" variant would solve even > this > problem. Jan: In my view, it's not a problem, it's a feature. An OS-wide change wouldn't turn on "--with-pydebug" for Python, unless it was directly coded into the specfile. The change to the ABI is due to two new fields with the base struct for python objects. Dave

1 0

Re: Why are we shipping debug builds of pythons?
by Petr Viktorin 06 Aug '18

06 Aug '18

On 08/05/18 14:01, Jan Kratochvil wrote: > On Sun, 05 Aug 2018 13:39:58 +0200, Charalampos Stratakis wrote: >> Here we are referring to the python2-debug or python3-debug builds, which >> are just extra python builds that are compiled with the --with-pydebug flag > > Not just --with-pydebug: > > ------------------------------------------------------------------------------ > %if %{with debug_build} > BuildPython debug \ > "--without-ensurepip --with-pydebug" \ > "-O0" > %endif # with debug_build > > BuildPython optimized \ > "--without-ensurepip %{optimizations_flag}" \ > "" > ------------------------------------------------------------------------------ > > Many packages have their *-debug variants. And then some packages (like > kernel) switch even their standard builds to debug mode but only in Rawhide. > Despite all that the *-debug package one occasionally needs is then missing > and one still has to build it locally. > > That is all together messy. This is why Microsoft has their debug build of > their whole OS - Windows - called Checked Build: > https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/checked-a… The main issue with --with-pydebug is that it changes the ABI, i.e. all extensions need to be re-built specifically for it (and debug extensions outside the standard library aren't usually packaged in Fedora). That makes it much less useful than if it just used less optimizations.

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel August 2018