Re: [fedora-arm] Broken sha512sum in coreutils
by Gordan Bobic
Gordan Bobic wrote:
> Andy Green wrote:
>> On 01/07/11 16:15, Somebody in the thread at some point said:
>>
>>> [root@sheeva ~]# cat /proc/cpu/alignment
>>> User: 12230853
>>> System: 1
>>> Skipped: 0
>>> Half: 0
>>> Word: 0
>>> DWord: 1
>>> Multi: 0
>>> User faults: 0 (ignored)
>>
>> I wrote about the alignment issue here on ARM:
>>
>> http://warmcat.com/_wp/2007/05/25/the-alignment-monster/
>>
>> If I understood it, you have a bazillion alignment faults in your user
>> code that aren't being fixed up, which will cause exactly this kind of
>> symptom with sha512sum.
>>
>> Try
>>
>> echo 2 > /proc/cpu/alignment
>>
>> (take care to keep the spaces as shown) and try the tests again.
>
> Yup, that fixes it! :)
>
> So what is the correct/permanent solution to avoid problems like this?
>
> *goes to RTFA* ;)
>
> Gordan
13 years, 3 months
Re: [fedora-arm] Broken sha512sum in coreutils
by Gordan Bobic
Gordan Bobic wrote:
> Andy Green wrote:
>> On 01/07/11 16:15, Somebody in the thread at some point said:
>>
>>> [root@sheeva ~]# cat /proc/cpu/alignment
>>> User: 12230853
>>> System: 1
>>> Skipped: 0
>>> Half: 0
>>> Word: 0
>>> DWord: 1
>>> Multi: 0
>>> User faults: 0 (ignored)
>>
>> I wrote about the alignment issue here on ARM:
>>
>> http://warmcat.com/_wp/2007/05/25/the-alignment-monster/
>
> Aha, I know what you are talking about now, and I am aware of such
> issues. Here is a similar article a friend of mine wrote:
>
> http://www.altechnative.net/?p=48
Does changing the /proc/cpu/alignment setting from 0 (ignored) to 2
(fixup) come with a noticeable performance penalty?
Gordan
13 years, 3 months
Alignment - proposed rc.sysinit patch and sysconfig file
by Gordan Bobic
Since we've been discussing it recently, how does this look?
Attached are:
patch against F13 /etc/rc.sysinit
/etc/sysconfig/alignment
It is the first thing that runs in rc.sysinit. Provided that bash, echo,
uname and grep aren't having alignment issues, it should be OK. And if
those were buggy we'd have much bigger problems.
Comments?
Gordan
# Default alignment setting for ARM CPUs.
# 0 - ignore unaligned accesses silently (kernel's default policy)
# 1 - warn on unaligned accesses in kernel log
# 2 - fixup unaligned accesses silently
# 3 - fixup+warn on unaligned accesses
# 4 - signal on unaligned accesses (core dump the offending program)
# 5 - signal+warn on unaligned accesses
ALIGNMENT=3
13 years, 3 months
Alignment and dietlibc
by Gordan Bobic
Guys, it is as I suspected:
Before building dietlibc (with my patches (posted to bugzilla), without
which it won't build at all):
User: 0
System: 1
Skipped: 0
Half: 0
Word: 0
DWord: 1
Multi: 0
User faults: 3 (fixup+warn)
After building dietlibc and running it's test suite:
User: 1012877
System: 1
Skipped: 0
Half: 0
Word: 0
DWord: 1
Multi: 0
User faults: 3 (fixup+warn)
I don't think there can be any doubt any more about where the previos
astronomically high number of misalignments came from. This also means
that the chances of properly running dietlibc on ARM, at least with
decent performance without hardware alignment fixing are pretty low.
Gordan
13 years, 3 months
Re: [fedora-arm] dietlibc port?
by Rich Mattes
On 1/6/2011 11:56 AM, Gordan Bobic wrote:
> Deliberately off-list?
>
No, sorry, more of a brainfart. Too many email clients...
> Rich Mattes wrote:
>> On Thu, Jan 6, 2011 at 11:33 AM, Gordan Bobic <gordan(a)bobich.net
>> <mailto:gordan@bobich.net>> wrote:
>>
>> Andy Green wrote:
>> > On 01/06/11 16:17, Somebody in the thread at some point said:
>> >
>> > Hi -
>> >
>> >> Digging a bit more, it seems that bin-arm/diet segfaults when
>> executed
>> >> without any parameters, so it seems that this is the bit that
>> >> miscompiled somewhere.
>> >
>> > Did you try running it in gdb and get a clue about what blows?
>>
>> Not yet, no.
>>
>> >> It turns out that the problem is actually to do witha GCC bug.
>> When it
>> >> is building bin-arm/diet with -Ox (x = {s,2,3}) , the resulting
>> binary
>> >> segfaults. When the binary is built with -O1 or -O0, it works fine.
>> >
>> > Is there a particular gcc bug you traced it down to? Because it does
>> > not follow that all changes to program behaviour with different
>> > optimization options are necessarily gcc bugs; it can show up
>> > broken-ness in code that happens to not show symptoms according
>> to how
>> > it is compiled.
>>
>> That's fair, but an awful lot seems to break. If dietlibc really
>> is that
>> broken, I have to question how it ever worked for anyone
>> reliably. I'm
>> also wondering if it might be some of the Fedora patches that are
>> breaking it. I'll try it without any post-0.32 patches.
>>
>> You should also probably try the F13 gcc. Bugs reported against the
>> F12 gcc are going to be ignored, since nobody is working on any F12
>> stuff anymore.
>
> I am slowly shifting things over to F13, but there is a lot to DIY
> build. For example, things like Firefox don't work because of an API
> version mismatch with xulrunner. And it helps to have a relatively
> usable system to do development on. :)
>
> Besides, the reason I need dietlibc is because util-vserver requires
> it and I want to work on F13 stuff in a vserver chroot in order to
> ensure that the host system remains operational even if I badly break
> important low-level things.
>
Ah, I see. I've just been using a plain chroot into the f13 rootfs with
/proc bind mounted to the rootfs /proc, but it looks like vserver has
some more features.
>> If it helps, you should be able to override the CFLAGS and CXXFLAGS
>> in the dietlibc specfile before calling make in order to lower the
>> optimization level.
>
> The problem with that is two-fold:
>
> 1) It seems a lot (most?) of spec files don't actually set CC, CXX,
> CPP, CFLAGS, CXXFLAGS, etc. accordoing to the rpmrc. I'm currently in
> the process of going through most of the packages in RHEL6 (building
> for x86 using ICC for a bit of extra speed) and F12/F13 (to build the
> missing ones for the ARM port), and it seems that this happens all
> over the place, and inconsistently to boot. For example, bzip2 sets
> the variables according to rpmrc, but gzip doesn't. It is really
> frustrating to find that the package management functionality is
> ignored so frequently.
>
> 2) Ironically, having said 1), it seems that dietlibc introduces it's
> optimization flags of -Os/-O2 _regardless_ of what the CFLAGS in the
> environment says. I haven't checked the patches in the src.rpm yet,
> but it is _probably_ coming from the upstream tar ball. Which is not
> good, because it means chasing such dubious Makefile behaviour
> upstream with developers who think "they know better" because "it
> works for them".
>
> Gordan
The %configure macro is where the flags are set. You can run rpm --eval
%configure to see what's going on there. It looks like the CPPFLAGS
exported in gzip are in addition to the default CPPFLAGS, which are
appended when %configure is called. As for the bzip2 spec, it looks
like there isn't a configure script, so the maintainer had to manually
set all the variables that %configure would have taken care of. I guess
bzip2 is actually the anomaly here.
As for -O2 and -Os slipping in, you'll have to patch or sed them out. A
lot of the time upstream will hard code flags, but since -O2 is the
Fedora default nobody really removes them when they pop up. -O2 only
appears in about 7 files, so it shouldn't be too hard to eliminate.
Rich
13 years, 3 months
Fwd: Re: Broken sha512sum in coreutils
by Andy Green
-------- Original Message --------
Subject: Re: [fedora-arm] Broken sha512sum in coreutils
Date: Fri, 07 Jan 2011 16:46:25 +0000
From: Gordan Bobic <gordan(a)bobich.net>
To: andy(a)warmcat.com
Andy Green wrote:
> On 01/07/11 16:15, Somebody in the thread at some point said:
>
>> [root@sheeva ~]# cat /proc/cpu/alignment
>> User: 12230853
>> System: 1
>> Skipped: 0
>> Half: 0
>> Word: 0
>> DWord: 1
>> Multi: 0
>> User faults: 0 (ignored)
>
> I wrote about the alignment issue here on ARM:
>
> http://warmcat.com/_wp/2007/05/25/the-alignment-monster/
Aha, I know what you are talking about now, and I am aware of such
issues. Here is a similar article a friend of mine wrote:
http://www.altechnative.net/?p=48
What I wasn't aware of was that you can change alignment via
/proc/cpu/alignment. What, exactly does that do?
Gordan
13 years, 3 months
Fwd: Re: Broken sha512sum in coreutils
by Andy Green
-------- Original Message --------
Subject: Re: [fedora-arm] Broken sha512sum in coreutils
Date: Fri, 07 Jan 2011 16:36:37 +0000
From: Gordan Bobic <gordan(a)bobich.net>
To: andy(a)warmcat.com
Andy Green wrote:
> On 01/07/11 16:15, Somebody in the thread at some point said:
>
>> [root@sheeva ~]# cat /proc/cpu/alignment
>> User: 12230853
>> System: 1
>> Skipped: 0
>> Half: 0
>> Word: 0
>> DWord: 1
>> Multi: 0
>> User faults: 0 (ignored)
>
> I wrote about the alignment issue here on ARM:
>
> http://warmcat.com/_wp/2007/05/25/the-alignment-monster/
>
> If I understood it, you have a bazillion alignment faults in your user
> code that aren't being fixed up, which will cause exactly this kind of
> symptom with sha512sum.
>
> Try
>
> echo 2 > /proc/cpu/alignment
>
> (take care to keep the spaces as shown) and try the tests again.
Yup, that fixes it! :)
So what is the correct/permanent solution to avoid problems like this?
*goes to RTFA* ;)
Gordan
P.S.
Deliberately off-list?
13 years, 3 months
Fwd: Re: Broken sha512sum in coreutils
by Andy Green
Failed to send it to the list
-------- Original Message --------
Subject: Re: [fedora-arm] Broken sha512sum in coreutils
Date: Fri, 07 Jan 2011 16:29:26 +0000
From: Andy Green <andy(a)warmcat.com>
Reply-To: andy(a)warmcat.com
To: Gordan Bobic <gordan(a)bobich.net>
On 01/07/11 16:15, Somebody in the thread at some point said:
> [root@sheeva ~]# cat /proc/cpu/alignment
> User: 12230853
> System: 1
> Skipped: 0
> Half: 0
> Word: 0
> DWord: 1
> Multi: 0
> User faults: 0 (ignored)
I wrote about the alignment issue here on ARM:
http://warmcat.com/_wp/2007/05/25/the-alignment-monster/
If I understood it, you have a bazillion alignment faults in your user
code that aren't being fixed up, which will cause exactly this kind of
symptom with sha512sum.
Try
echo 2 > /proc/cpu/alignment
(take care to keep the spaces as shown) and try the tests again.
-Andy
13 years, 3 months
Broken sha512sum in coreutils
by Gordan Bobic
It would appear that sha512sum is broken, at least in the F12 distro
(coreutils-7.6-5.fc12.fa1.armv5tel). It is producing a different hash
for the same file compared to what my x86 machines produce. This is
quite worrying and a potentially dangerous crypto-security issue.
The only thing that comes to mind as a potential cause (other than a
bug) is that I am using gcc/libgcc from F13 with F12 coreutils. I just
updated coreutils to the F13 package (coreutils-8.4-5.fc13.armv5tel),
and that produces the correct hashes.
Can anybody with a clean F12 vanilla copy check if they can reproduce
the problem?
Thanks.
Gordan
13 years, 3 months
