12:09 a.m.
glibc upstream, during development of the 2.24 release, introduced new
symbol versions recvmsg(a)GLIBC_2.24, sendmsg(a)GLIBC_2.24 (and
recvmmsg(a)GLIBC_2.24, sendmmsg(a)GLIBC_2.24 on 64-bit architectures), in
order to fix some minor POSIX compliance issue. (POSIX and the Linux
kernel disagree about the width of some fields in struct msghdr.) These
changes landed in rawhide as part of glibc-2.23.90-19.fc25.
This change caused quite a few issues (chrony stopped building, Address
Sanitizer interception of these functions was affected, probably more).
Considering that the deviation from POSIX was really minor, this was
considered a poor trade-off, and the patch and ABI change was eventually
reverted upstream.
We cannot implement the ABI reversal immediately in rawhide because that
would break existing binaries, and we don't want to do a bootstrap or
mass rebuild for this. Therefore, glibc-2.23.90-21.fc25 turns the new
symbols into compatibility symbols. As a result, new binaries will be
linked against the old symbol versions, as before, and old binaries
continue to work.
(glibc-2.23.90-22.fc25 adds some polishing, but this version is
functionally equivalent to the previous one.)
Based on Sunday's compose, the list of packages which need to rebuild is
fairly small:
source
----------------------------------------
NetworkManager-1.2.2-2.fc25
OpenIPMI-2.0.22-1.fc25
OpenImageIO-1.6.14-1.fc25
boost-1.60.0-7.fc25
chrony-2.4-1.fc25
collectd-5.5.1-12.fc25
cryptsetup-1.7.2-1.fc25
firefox-47.0-4.fc25
freerdp-2.0.0-9.git.aa15327.fc25
gdb-7.11.1-75.fc25
glib2-2.49.1-2.fc25
glibc-2.23.90-20.fc25
gnutls-3.4.13-1.fc25
haproxy-1.6.5-2.fc25
hostapd-2.5-4.fc25
java-1.8.0-openjdk-1.8.0.92-1.b14.fc25
libguestfs-1.33.35-1.fc25
libnice-0.1.13-6.fc25
libvirt-1.3.5-1.fc25
lxc-2.0.1-1.fc25
mono-4.4.0-5.fc25
nftables-0.6-1.fc25
ntp-4.2.6p5-41.fc25
openssh-7.2p2-7.fc25
php-5.6.23-0.1.RC1.fc25
python-twisted-16.1.1-3.fc25
qt5-qtbase-5.6.1-2.fc25
qt5-qtwebengine-5.6.1-1.fc25
qt5-qtwebkit-5.6.1-1.b889f46git.fc25
root-6.06.04-2.fc25
samba-4.4.4-1.fc25
thunderbird-45.1.1-2.fc25
tigervnc-1.6.0-6.fc25
weechat-1.5-1.fc25
wine-1.9.11-1.fc25
(35 rows)
I'll wait some time to see how many of those pick up the old ABI due to
regular development. Probably around mid-July, I will start pinging
developers if rebuilds are missing. We must remove the compatibility
symbol before the release, so I'll have to enlist help of provenpackages
eventually to trigger missing rebuilds.
Thanks,
Florian
8:26 a.m.
On Mon, Jun 13, 2016 at 07:09:33AM +0200, Florian Weimer wrote:
glibc upstream, during development of the 2.24 release, introduced
new symbol versions recvmsg(a)GLIBC_2.24, sendmsg(a)GLIBC_2.24 (and
recvmmsg(a)GLIBC_2.24, sendmmsg(a)GLIBC_2.24 on 64-bit architectures),
in order to fix some minor POSIX compliance issue. (POSIX and the
Linux kernel disagree about the width of some fields in struct
msghdr.) These changes landed in rawhide as part of
glibc-2.23.90-19.fc25.
This may explain why some old code we had that sent file descriptors
around using SCM_RIGHTS suddenly broke last week in Rawhide. I was
going to investigate it, but once I had a look at our code, I ended up
completely rewriting it instead, and that appears to have fixed it,
but perhaps that was just as a side effect of relinking (the old code
needed to be rewritten anyway).
Here was the change I made, in case it's interesting:
https://github.com/libguestfs/libguestfs/commit/69e5410e6264f58082883beb5...
Libvirt also broke with some kind of file descriptor passing error:
https://bugzilla.redhat.com/show_bug.cgi?id=1344016
No one has really looked at the libvirt problem, but Cole pointed me
at this thread.
I tried just installing glibc 2.23.90-22, but even after a reboot that
did not appear to fix libvirt.
Recompiling libvirt won't be a problem.
libguestfs-1.33.35-1.fc25
libvirt-1.3.5-1.fc25
I will do these two.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
8:52 a.m.
On Mon, Jun 13, 2016 at 07:09:33AM +0200, Florian Weimer wrote:
glibc upstream, during development of the 2.24 release, introduced
new
symbol versions recvmsg(a)GLIBC_2.24, sendmsg(a)GLIBC_2.24 (and
recvmmsg(a)GLIBC_2.24, sendmmsg(a)GLIBC_2.24 on 64-bit architectures), in order
to fix some minor POSIX compliance issue. (POSIX and the Linux kernel
disagree about the width of some fields in struct msghdr.) These changes
landed in rawhide as part of glibc-2.23.90-19.fc25.
This change caused quite a few issues (chrony stopped building, Address
Sanitizer interception of these functions was affected, probably more).
Considering that the deviation from POSIX was really minor, this was
considered a poor trade-off, and the patch and ABI change was eventually
reverted upstream.
Do you have a pointer to the glibc change, or specific details about
what exactly changed. It looks like the change broke libvirt usage
of SCM_RIGHTS, and despite fact that you're reverting it in glibc,
I'm wondering if there is non-standards compliant usage in libvirt
that we should be fixing regardless.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
8:57 a.m.
On Mon, Jun 13, 2016 at 03:37:26PM +0200, Florian Weimer wrote:
On 06/13/2016 03:26 PM, Richard W.M. Jones wrote:
>I tried just installing glibc 2.23.90-22, but even after a reboot that
>did not appear to fix libvirt.
If you don't rebuild, the code will use the GLIBC_2.24 symbols,
which have the POSIX conformance enhancement.
Thanks - in any case I have rebuilt libvirt against glibc -22 now.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-builder quickly builds VMs from scratch
http://libguestfs.org/virt-builder.1.html
9:29 a.m.
On 06/13/2016 03:52 PM, Daniel P. Berrange wrote:
On Mon, Jun 13, 2016 at 07:09:33AM +0200, Florian Weimer wrote:
> glibc upstream, during development of the 2.24 release, introduced new
> symbol versions recvmsg(a)GLIBC_2.24, sendmsg(a)GLIBC_2.24 (and
> recvmmsg(a)GLIBC_2.24, sendmmsg(a)GLIBC_2.24 on 64-bit architectures), in order
> to fix some minor POSIX compliance issue. (POSIX and the Linux kernel
> disagree about the width of some fields in struct msghdr.) These changes
> landed in rawhide as part of glibc-2.23.90-19.fc25.
>
> This change caused quite a few issues (chrony stopped building, Address
> Sanitizer interception of these functions was affected, probably more).
> Considering that the deviation from POSIX was really minor, this was
> considered a poor trade-off, and the patch and ABI change was eventually
> reverted upstream.
Do you have a pointer to the glibc change, or specific details about
what exactly changed. It looks like the change broke libvirt usage
of SCM_RIGHTS, and despite fact that you're reverting it in glibc,
I'm wondering if there is non-standards compliant usage in libvirt
that we should be fixing regardless.
The upstream commit with the revert is here:
https://sourceware.org/git/?p=glibc.git;a=commit;h=78880cc185dc521855a580...
I assume the error comes from this code at gnulib/lib/passfd.c:
/* fake errno: at end the file is not available */
errno = len ? EACCES : ENOTCONN;
So the ENOTCONN isn't real.
It looks the most likely explanation is that glibc temporarily had the
struct cmsghdr change in the header file, but no compensating rewriting
code in the library. The padding field could be random garbage and
passed to the kernel, which observes an impossible length value as a result.
Florian
10:10 a.m.
New subject: Second stage of glibc recvmsg/sendmsg ABI revert landed in rawhide
On 06/13/2016 07:09 AM, Florian Weimer wrote:
glibc upstream, during development of the 2.24 release, introduced
new
symbol versions recvmsg(a)GLIBC_2.24, sendmsg(a)GLIBC_2.24 (and
recvmmsg(a)GLIBC_2.24, sendmmsg(a)GLIBC_2.24 on 64-bit architectures), in
order to fix some minor POSIX compliance issue. (POSIX and the Linux
kernel disagree about the width of some fields in struct msghdr.) These
changes landed in rawhide as part of glibc-2.23.90-19.fc25.
This change caused quite a few issues (chrony stopped building, Address
Sanitizer interception of these functions was affected, probably more).
Considering that the deviation from POSIX was really minor, this was
considered a poor trade-off, and the patch and ABI change was eventually
reverted upstream.
We cannot implement the ABI reversal immediately in rawhide because that
would break existing binaries, and we don't want to do a bootstrap or
mass rebuild for this. Therefore, glibc-2.23.90-21.fc25 turns the new
symbols into compatibility symbols. As a result, new binaries will be
linked against the old symbol versions, as before, and old binaries
continue to work.
I have removed the compatibility symbols from rawhide, after all the
necessary rebuilds happened on primary. I double-checked aarch64,
ppc64, ppc64le, s390x as well and asked for additional rebuilds where
appropriate. (Some packages were built against different glibc versions
than on primary.) As a result, except on ppc64/ppc64le, all compat
symbol references are gone from the build root. ppc64/ppc64le current
has unrelated issues and falls farther and farther beyond primary, which
is why I didn't want to delay the compat removal (and alignment with the
glibc upstream ABI) until ppc64/ppc64le catches up eventually. I do not
expect any problems as long as rebuilds happen roughly in the order they
happened on primary.
Symptoms of an issue related to the compat symbol removal include error
messages about undefined GLIBC_2.24 symbols, particularly sendmsg,
sendmmsg, recvmsg, recvmmsg. RPM dependencies do not express this
information because they are not sufficiently fine-grained.
Florian
2838
days inactive
2879
days old
6 comments
3 participants
participants (3)
-
Daniel P. Berrange -
Florian Weimer -
Richard W.M. Jones