Hi Pythonians,
I have a favor to ask. I'm trying to package scikit-misc as a dependency
for plotnine (for neuro-sig).
The build fails with:
fatal error: longintrepr.h: No such file or directory
I'm aware of the fact that the file has been moved in Python 3.11, but
was under the impression this change was implemented in Cython 0.29.5.
I opened an issue upstream, but I would appreciate a second pair of eyes
on the build. The file that tries to include the header file is
generated, so nothing I could work around with using a patch (with my
limited knowledge that is).
Build:
https://copr.fedorainfracloud.org/coprs/gui1ty/neuro-sig/build/4869979/
Issue: https://github.com/has2k1/scikit-misc/issues/17
-- Sandro
Hey all,
I'm building python-uswid as a rpm as it's going to be needed by the fwupd-efi package at build time in the near future. I'm also the upstream maintainer, so I'm not against changing upstream and then tagging a new release if there's something that needs to be fixed to build a Fedora package. I'm no python expert, so advice very welcome.
I've uploaded a srpm here and I'd appreciate some early 40,000ft checks before I submit a Fedora package review: https://people.freedesktop.org/~hughsient/temp/python-uswid-0.4.1-1.fc37.sr… -- the upstream is https://github.com/hughsie/python-uswid/ for the curious.
Thanks!
Richard.
> Successfully waited 0:00 for python-tox-4.2.6-1.fc38 to appear in the
f38-build repo.
If you hit unexpected issues with %tox, let us know here or in Bugzilla.
Thanks.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
Hello folks,
this recent Fedora change:
https://fedoraproject.org/wiki/Changes/Add_FORTIFY_SOURCE%3D3_to_distributi…
Made me think:
Which compiler flags we need to store in Python and which can we omit?
In order to make Python extension modules binary compatible with Python, Python
saves the compiler flags at compile-time and reuses them when building
extension modules.
Historically, we had troubles with this approach because some of the flags are
unusable without redhat-rpm-config, annobin etc.
As a result, there are now 2 compiler flags macros available for RPM:
%{build_cflags} and %{extension_cflags} (same for ldflags etc.).
While Python itself is built with %{build_cflags}, it saves %{extension_cflags}
in sysconfig.
The flags differ like this:
$ diff -u <(rpm --eval '%build_cflags' | tr ' ' '\n') <(rpm --eval
'%extension_cflags' | tr ' ' '\n') | grep ^-
--flto=auto
--ffat-lto-objects
--specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
--specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
$ diff -u <(rpm --eval '%build_ldflags' | tr ' ' '\n') <(rpm --eval
'%extension_ldflags' | tr ' ' '\n') | grep ^-
--specs=/usr/lib/rpm/redhat/redhat-hardened-ld
--specs=/usr/lib/rpm/redhat/redhat-annobin-cc1
(There are also some other differences wrt
https://fedoraproject.org/wiki/Changes/Package_information_on_ELF_objects but
those are apparently harder to get from outside of a real build.)
The current set of flags from Python can be obtained by:
>>> sysconfig.get_config_var('CFLAGS')
'-Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -fexceptions -g
-grecord-gcc-switches -pipe -Wall -Werror=format-security -U_FORTIFY_SOURCE
-Wp,-U_FORTIFY_SOURCE -Wp,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS
-fstack-protector-strong -m64 -mtune=generic -fasynchronous-unwind-tables
-fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2
-fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security
-U_FORTIFY_SOURCE -Wp,-U_FORTIFY_SOURCE -Wp,-D_FORTIFY_SOURCE=3
-Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
-D_GNU_SOURCE -fPIC -fwrapv -O2 -fexceptions -g -grecord-gcc-switches -pipe
-Wall -Werror=format-security -U_FORTIFY_SOURCE -Wp,-U_FORTIFY_SOURCE
-Wp,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong
-m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection
-fcf-protection -D_GNU_SOURCE -fPIC -fwrapv'
>>> sysconfig.get_config_var('LDFLAGS')
'-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g
-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -Wl,--build-id=sha1 -g'
I wonder if other flags should be removed as well.
Isn't Python built e.g. with -Werror=format-security or -Wsign-compare binary
compatible with extension modules built without it?
What about FORTIFY_SOURCE and others?
Is there a compiler flags expert here who could help me determine what flags
could (or even should) be removed from %{extension_*flags}?
Thanks,
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
Hello.
Since Python 2.0 (1994), Python provided a useful tool pathfix.py that
we use in Python RPM macros for fixing shebangs of Python modules and
some RPM packages use it as well directly in their specfiles for similar
purposes. The script will no longer be part of CPython source code and
python3-devel RPM package. Because we think it's useful, we have decided
to create a new upstream project for it
(https://github.com/fedora-python/pathfix) on Github and include it in
the python3-rpm-macros package. The change means the script will move
from /usr/bin/pathfix.py to /usr/lib/rpm/redhat/pathfix.py.
For users of %py_shebang_fix and %py3_shebang_fix no action is needed.
The macros will soon use the new location of the tool.
If you use the tool directly in your specfile a change is needed before
Python 3.12 become the main one in Fedora 38. We have a list of affected
packages and will open PRs for them soon.
Packages requiring /usr/bin/pathfix.py:
$ repoquery -q --repo=rawhide{,-source} --whatrequires /usr/bin/pathfix.py
Mayavi-0:4.8.1-1.fc38.src
PySolFC-0:2.18.0-1.fc38.src
TeXmacs-0:2.1.2-2.fc38.src
autodownloader-0:0.5.0-6.fc37.src
cp2k-0:9.1-3.fc37.src
dblatex-0:0.3.12-7.fc37.src
diskimage-builder-0:3.20.1-2.fc37.src
dnsperf-0:2.9.0-3.fc37.src
gimp-layer-via-copy-cut-0:1.6-22.fc37.src
gimp-resynthesizer-0:2.0.3-9.20190428gitadfa25a.fc37.src
global-0:6.6.5-7.fc38.src
gtkpod-0:2.1.5-22.fc38.src
guitarix-0:0.44.1-1.fc37.src
kernel-tools-0:6.1.0-0.rc2.git0.1.fc38.src
koan-0:3.0.1-4.fc37.src
mediawiki-0:1.38.2-1.fc38.src
monsterz-0:0.7.1-31.fc37.src
new-session-manager-0:1.3.2-7.fc37.src
noggin-0:1.6.1-4.fc38.src
nut-0:2.8.0-5.fc38.src
nx-libs-0:3.5.99.26-5.fc37.src
octave-miscellaneous-0:1.3.0-10.fc37.src
pcsc-lite-0:1.9.9-1.fc38.src
percona-xtrabackup-0:8.0.25_17-1.fc35.src
pynag-0:1.1.2-11.fc37.src
python-confluent-kafka-0:1.6.1-1.fc38.src
python-cram-0:0.7-13.fc37.src
python-pyside2-1:5.15.2.1-7.fc37.src
rabbitvcs-0:0.18-8.fc37.src
roca-detect-0:1.2.12-21.fc37.src
scipy-0:1.8.1-6.fc37.src
swift-lang-0:5.7-3.fc38.src
syslog-ng-0:3.37.1-1.fc38.src
systemtap-0:4.8~pre16650659g6a096a7d-1.fc38.src
wine-mono-0:7.3.0-2.fc37.src
specfiles containing "pathfix.py":
ceph.spec
blktrace.spec
aws.spec
avogadro2-libs.spec
cobbler.spec
crawl.spec
partio.spec
dblatex.spec
dnsperf.spec
diskimage-builder.spec
dlib.spec
cp2k.spec
fio.spec
future.spec
fedora-review.spec
fedora-easy-karma.spec
GConf2.spec
gimp-layer-via-copy-cut.spec
gimp-resynthesizer.spec
llvm10.spec
global.spec
gnome-weather.spec
clang10.spec
gtkpod.spec
netbox.spec
htop.spec
guitarix.spec
gtk2.spec
kernel-tools.spec
kernel.spec
monsterz.spec
intel-cm-compiler.spec
koan.spec
libabigail.spec
libevdev.spec
libevent.spec
libinput.spec
libsbml.spec
libtdb.spec
Mayavi.spec
mesos.spec
mftrace.spec
micropython.spec
mercurial.spec
mediawiki.spec
nodejs.spec
llvm-test-suite.spec
llvm11.0.spec
nx-libs.spec
octave-miscellaneous.spec
offlineimap.spec
openclonk.spec
openscap.spec
nut.spec
pcsc-lite.spec
percona-xtrabackup.spec
clang11.spec
petsc.spec
noggin.spec
policycoreutils.spec
pyhoca-cli.spec
pynag.spec
python3.10.spec
python3.8.spec
python3.6.spec
python3.7.spec
python2.7.spec
python-biopython.spec
python-cram.spec
python-confluent-kafka.spec
python-bluepy.spec
PySolFC.spec
python3.9.spec
pyhoca-gui.spec
python-nbconvert.spec
python-nb2plots.spec
python-nitrate.spec
python-os-testr.spec
python-igor.spec
python-exabgp.spec
python-pyside2.spec
python-reportlab.spec
python-rpm-macros.spec
python-scss.spec
pytz.spec
rocminfo.spec
roca-detect.spec
rabbitvcs.spec
python-zbase32.spec
python-mistune08.spec
llvm9.0.spec
llvm7.0.spec
libtalloc.spec
scipy.spec
solarwolf.spec
TeXmacs.spec
llvm8.0.spec
swift-lang.spec
python3.11.spec
llvm11.spec
telepathy-logger.spec
syslog-ng.spec
systemtap.spec
scons.spec
tuna.spec
vips.spec
wavextract.spec
xpra.spec
andriller.spec
tuptime.spec
clang12.spec
llvm12.spec
new-session-manager.spec
workrave.spec
wine-mono.spec
wabt.spec
vdrift.spec
inksmoto.spec
inkscape.spec
autojump.spec
autodownloader.spec
apbs.spec
Have a nice day.
Lumír
tl;dr: Python 3.12 should be built with no-omit-frame-pointer if
upstream recommends it.
Hello,
You might be aware of a Fedora change proposal [0] (discussed on
fedora-devel [1] and FESCo [2]) are discussing turning on C compiler
flags that help with performance *measurement*, but might hurt
performance itself: `-fno-omit-frame-pointer` and
`-mno-omit-leaf-frame-pointer`.
Apparently there are some benchmarks that make Python look extra slow
when the flags are turned on -- which I don't quite understand.
Meanwhile, on the upstream side, Python 3.12 (due next year, main Python
for Fedora 39 [3]) has support for `perf`. Upstream plans to recommend
compiling with these flags when measuring performance [4], and AFAIK,
the plan is to recommend *always* compiling with them.
The idea is that possible speedups from "allowing anyone to
profile/optimize their workflow" are worth the initial slowdown.
I'm not much of a performance expert myself, but I do get drawn into the
relevant discussions on the CPython side.
As far as I can see, performance geeks are enthusiastic for `perf`
support, and I'd like to get them to (continue to) use Fedora builds.
If CPython upstream does recommend these flags (or makes them default),
I'm considering to turn the no-omit options on for Python 3.12 even if
Fedora as a whole doesn't.
Note that even a 2% slowdown will likely won back by general performance
improvements – the Faster CPython team is targeting a 20% average
speedup for pure-Python code in 3.12, on top of the ~25% for 3.11. And
the people responsible for this speedup have a say in the upstream
recommendations.
Technically there are three separate places where the flags can be set,
I think we should turn them on everywhere:
- CPython itself & its standard library
- The debug build (/usr/bin/python3-debug)
- Default for libraries in Fedora (RPM macros)
- Befault for libraries built by users (sysconfig settings)
[0]: https://fedoraproject.org/wiki/Changes/fno-omit-frame-pointer
[1]:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org…
[2]: https://pagure.io/fesco/issue/2817
[3]: https://fedoraproject.org/wiki/Changes/Python3.12
[4]:
https://docs.python.org/3.12/howto/perf_profiling.html#how-to-obtain-the-be…
Hi Folks,
I am looking to update one of my packages in EPEL 8 and Python3.6 was
dropped.
Can anyone point me in the right direction as to how to edit my spec
file `buildrequires` and the macros to enforce Python 3.7 (or higher)
instead of Python 3.6 anto do it "the right way", not a kludge?
Thank you,
Mark E. Fuller, Ph.D.
fuller(a)fedoraproject.org
@fuller:one.ems.host
https://mefuller.dev
PGP Fingerprint: 73F1 A30C BDF4 DB4B C75F FD0F D599 E76C FFCA BF60