12:39 p.m.
One of my packages failed the mass rebuild, but only on ppc64le and
aarch64. The error they both hit is this:
Error: package or namespace load failed for 'BiocParallel' in
dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object
'/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so':
/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so:
undefined symbol: shm_open
Error: loading failed
Here's the linker invocation:
g++ -m64 -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-z,relro
-Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-o BiocParallel.so ipcmutex.o -L/usr/lib64/R/lib -lR
Now, google says this might be due to a lack of -lrt, but what's not
clear to me is why it only fails on these two architectures (I've
confirmed that none of the others have -lrt either).
Full build logs are here:
https://koji.fedoraproject.org/koji/taskinfo?taskID=32581444
Thanks in advance,
~tom
3:41 p.m.
On Wednesday, 06 February 2019 at 19:39, Tom Callaway wrote:
One of my packages failed the mass rebuild, but only on ppc64le and
aarch64. The error they both hit is this:
Error: package or namespace load failed for 'BiocParallel' in
dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object
'/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so':
/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so:
undefined symbol: shm_open
Error: loading failed
Here's the linker invocation:
g++ -m64 -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-z,relro
-Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-o BiocParallel.so ipcmutex.o -L/usr/lib64/R/lib -lR
Now, google says this might be due to a lack of -lrt, but what's not
clear to me is why it only fails on these two architectures (I've
confirmed that none of the others have -lrt either).
shm_open(3) concurs:
[...]
Link with -lrt.
[...]
So it's rather curious why it's succeeding on all the other arches.
Regards,
Dominik
--
Fedora https://getfedora.org | RPM Fusion http://rpmfusion.org
There should be a science of discontent. People need hard times and
oppression to develop psychic muscles.
-- from "Collected Sayings of Muad'Dib" by the Princess Irulan
7:28 p.m.
Tom Callaway wrote on 2019/02/07 3:39:
One of my packages failed the mass rebuild, but only on ppc64le and
aarch64. The error they both hit is this:
Error: package or namespace load failed for 'BiocParallel' in
dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object
'/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so':
/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so:
undefined symbol: shm_open
Error: loading failed
Here's the linker invocation:
g++ -m64 -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-z,relro
-Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-o BiocParallel.so ipcmutex.o -L/usr/lib64/R/lib -lR
Now, google says this might be due to a lack of -lrt, but what's not
clear to me is why it only fails on these two architectures (I've
confirmed that none of the others have -lrt either).
Full build logs are here:
https://koji.fedoraproject.org/koji/taskinfo?taskID=32581444
Thanks in advance,
~tom
I don't know well about R, however that is probably because R-core (-3.5.3-4.fc30)
package already
requires librt.so on x86_64, i686, etc, while on aarch64 and ppc64le, it does not, which
probably indicates
that on x86_64, i686, etc R binary is already linked with librt.so , while on aarch64 and
ppc64le
it is not.
So probably on x86_64, i686 when dlopen()ing BiocParallel.so symbol for shm_open is
already resolved
(because R is linked against librt.so) while on aarch64 and ppc64le it is not.
I guess this is because there is some configuration difference on R.spec on aarch64 and
ppc64le.
Regards,
Mamoru
9:49 a.m.
On 2/6/19 8:28 PM, Mamoru TASAKA wrote:
I don't know well about R, however that is probably because
R-core
(-3.5.3-4.fc30) package already
requires librt.so on x86_64, i686, etc, while on aarch64 and ppc64le, it
does not, which probably indicates
that on x86_64, i686, etc R binary is already linked with librt.so ,
while on aarch64 and ppc64le
it is not.
So probably on x86_64, i686 when dlopen()ing BiocParallel.so symbol for
shm_open is already resolved
(because R is linked against librt.so) while on aarch64 and ppc64le it
is not.
I guess this is because there is some configuration difference on R.spec
on aarch64 and
ppc64le.
So, R links with rt if this configure check succeeds:
AC_CHECK_LIB(rt, clock_gettime)
Sure enough, on aarch64 and ppc64le, there is no clock_gettime in
librt.so.1. I'm not sure _why_, but there is probably a good reason.
I now understand why these builds are failing and have several paths to
workaround it.
Thanks!
~tom
10:34 a.m.
* Tom Callaway:
So, R links with rt if this configure check succeeds:
AC_CHECK_LIB(rt, clock_gettime)
Sure enough, on aarch64 and ppc64le, there is no clock_gettime in
librt.so.1. I'm not sure _why_, but there is probably a good reason.
Hah!
We moved clock_gettime to libc.so.6 in glibc 2.17, so that it is
available without -lrt as well. But we forgot to turn the legacy
definitions into compatibility symbols, so they are still being picked
up with -lrt. This should be easy enough to fix.
Thanks,
Florian
10:41 a.m.
On Thursday, 07 February 2019 at 17:34, Florian Weimer wrote:
* Tom Callaway:
> So, R links with rt if this configure check succeeds:
>
> AC_CHECK_LIB(rt, clock_gettime)
>
> Sure enough, on aarch64 and ppc64le, there is no clock_gettime in
> librt.so.1. I'm not sure _why_, but there is probably a good reason.
Hah!
We moved clock_gettime to libc.so.6 in glibc 2.17, so that it is
available without -lrt as well. But we forgot to turn the legacy
definitions into compatibility symbols, so they are still being picked
up with -lrt. This should be easy enough to fix.
clock_getres(2) confirms:
Link with -lrt (only for glibc versions before 2.17).
Regards,
Dominik
--
Fedora https://getfedora.org | RPM Fusion http://rpmfusion.org
There should be a science of discontent. People need hard times and
oppression to develop psychic muscles.
-- from "Collected Sayings of Muad'Dib" by the Princess Irulan
10:06 p.m.
| One of my packages failed the mass rebuild, but only on ppc64le and
aarch64. The error they both hit is this:
Error: package or namespace load failed for 'BiocParallel' in
dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object
'/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so':
/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so:
undefined symbol: shm_open
Error: loading failed
On ububtu (ARM) ...
$ for i in /lib/aarch64-linux-gnu/*.so ; do echo $i;nm -D $i | grep
shm_open; done | more
...
/lib/aarch64-linux-gnu/librt-2.27.so
0000000000003f28 T shm_open
It looks like you have to link -lrt
so long
MUFTI
2:29 a.m.
* Tom Callaway:
One of my packages failed the mass rebuild, but only on ppc64le and
aarch64. The error they both hit is this:
Error: package or namespace load failed for 'BiocParallel' in
dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object
'/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so':
/builddir/build/BUILDROOT/R-BiocParallel-1.16.5-1.fc30.ppc64le/usr/lib64/R/library/BiocParallel/libs/BiocParallel.so:
undefined symbol: shm_open
Error: loading failed
Is this an error message from the R interpreter?
On x86-64, the R interpreter already loads librt:
7f0f2635a000-7f0f2635c000 r--p 00000000 fd:01 402963247
/usr/lib64/librt-2.29.so
7f0f2635c000-7f0f26360000 r-xp 00002000 fd:01 402963247
/usr/lib64/librt-2.29.so
7f0f26360000-7f0f26362000 r--p 00006000 fd:01 402963247
/usr/lib64/librt-2.29.so
7f0f26362000-7f0f26363000 r--p 00007000 fd:01 402963247
/usr/lib64/librt-2.29.so
7f0f26363000-7f0f26364000 rw-p 00008000 fd:01 402963247
/usr/lib64/librt-2.29.so
That's because libR loads it:
$ readelf -d /usr/lib64/R/lib/libR.so
…
0x0000000000000001 (NEEDED) Shared library: [librt.so.1]
…
The way dynamic linking works is that it searches for symbols in the
caller's namespace, and not just what's referenced by DT_NEEDED.
Here's the linker invocation:
g++ -m64 -std=gnu++11 -shared -L/usr/lib64/R/lib -Wl,-z,relro
-Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-o BiocParallel.so ipcmutex.o -L/usr/lib64/R/lib -lR
Now, google says this might be due to a lack of -lrt, but what's not
clear to me is why it only fails on these two architectures (I've
confirmed that none of the others have -lrt either).
In general, underlinking can be very problematic because it causes the
base symbol version to be used at run time, even if your program
actually requires a newer symbol version (for example, due to the
headers used at build time). Here it does not matter (today) because we
only have one symbol version for shm_open. But you do get interesting
effects with timer_create.
Thanks,
Florian
1907
days inactive
1908
days old
7 comments
5 participants
participants (5)
-
Dominik 'Rathann' Mierzejewski -
Florian Weimer -
J. Scheurich -
Mamoru TASAKA -
Tom Callaway