5:48 a.m.
Hi.
After applying yesterday's rawhide to my iBook, the system is quite
thoroughly toasted. After yum was finished, ssh'ing into the
machine no longer worke, and neither did logging in at the local
console. Rebooting the machine hangs after the initrd is processed,
seemingly failing to execute /sbin/init.
My suspicion is that a very low level library is botched (glibc or
selinux). Trying to chroot() into the system from an old YDL 3.0
install CD (the only CD what would boot on PPC I found on short
notice) fails, just producing segfaults, but YDL 3.0 is quite
old, of course, and still running 2.4. Will try and get a
FC4 install CD today and try again.
PS: The system does not use selinux.
5:58 a.m.
Hi,
After applying yesterday's rawhide to my iBook, the system is
quite
thoroughly toasted.
I think Jeff said it best - "Rawhide eats babies"
Sorry to hear about the problems though.
TTFN
Paul
--
"Some people will do anything for a woman in uniform" - The Doctor -
Unregenerate (Big Finish audio)
6:11 a.m.
On Thu, Jul 28, 2005 at 11:58:15AM +0100, Paul F. Johnson wrote:
> After applying yesterday's rawhide to my iBook, the system
is quite
> thoroughly toasted.
I think Jeff said it best - "Rawhide eats babies"
I know, and I am not complaining. That was just a head up warning
for other people that the danger level is slightly higher than
normal :)
7:03 a.m.
On 7/28/05, Ralf Ertzinger <fedora(a)camperquake.de> wrote:
Hi.
After applying yesterday's rawhide to my iBook, the system is quite
thoroughly toasted. After yum was finished, ssh'ing into the
machine no longer worke, and neither did logging in at the local
console. Rebooting the machine hangs after the initrd is processed,
seemingly failing to execute /sbin/init.
You don't have a list of the packages you updated yesterday do you?
Looking over yesterdays updates 20050727 I'm not seeing any thing from
yesterday that I would suspect that could have caused this. yum
couldn't update the rpm package because of a broken dep.. and
everything seems to be particularly minor in terms of system
operation.
-jef
7:31 a.m.
On Thu, Jul 28, 2005 at 08:03:10AM -0400, Jeff Spaleta wrote:
You don't have a list of the packages you updated yesterday do
you?
IIRC yum writes such a list to /var/log after each update, so
it ought to be there, yes.
Looking over yesterdays updates 20050727 I'm not seeing any thing
from
yesterday that I would suspect that could have caused this.
The system was not on a "the day before yesterday" state before
upgrading, so the cause could be further back in time. As said,
I suspect glibc or selinux, both of which were updated.
Sorry for the confusion.
8 a.m.
On 7/28/05, Ralf Ertzinger <fedora(a)camperquake.de> wrote:
On Thu, Jul 28, 2005 at 08:03:10AM -0400, Jeff Spaleta wrote:
The system was not on a "the day before yesterday" state before
upgrading, so the cause could be further back in time. As said,
I suspect glibc or selinux, both of which were updated.
Sorry for the confusion.
If you can get access to that yum.log we'll have a better idea.
Unless you are the only ppc user tracking rawhide at all.. i would
have expected someone out there to be tracking rawhide daily on ppc
and would have expected them to scream about it before today if this
was caused by an 'older' update. The fact that you are the first
person to scream about it.. doesn't bode well for re-producibility.
Have you dived into the zilla yet to see if a ppc meltdown has been
reported in the last week?
-jef
11:13 a.m.
On Thu, 2005-07-28 at 12:48 +0200, Ralf Ertzinger wrote:
After applying yesterday's rawhide to my iBook, the system is
quite
thoroughly toasted. After yum was finished, ssh'ing into the
machine no longer worke, and neither did logging in at the local
console. Rebooting the machine hangs after the initrd is processed,
seemingly failing to execute /sbin/init.
I just updated the Pegasos II box to rawhide, and it seems to be working
fine here. My first suspicion would be a failure to upgrade glibc
correctly.
--
dwmw2
11:46 a.m.
Hi.
David Woodhouse <dwmw2(a)infradead.org> wrote:
I just updated the Pegasos II box to rawhide, and it seems to be
working
fine here. My first suspicion would be a failure to upgrade glibc
correctly.
I have gotten the system up with a FC4-Install-CD, and there is some confusion
in the package db indeed. For example, I have currently 2 glibc packages
installed. Reinstalling the new packages from the rescue cd now.
Yay for rpm --root (and the fact that yum does not immediately delete
package files after upgrade)
--
white mouse (n.) -- an animal which if killed in sufficient numbers
under controlled conditions produces a PhD.
12:11 p.m.
On 7/28/05, Ralf Ertzinger <fedora(a)camperquake.de> wrote:
Hi.
David Woodhouse <dwmw2(a)infradead.org> wrote:
> I just updated the Pegasos II box to rawhide, and it seems to be working
> fine here. My first suspicion would be a failure to upgrade glibc
> correctly.
I have gotten the system up with a FC4-Install-CD, and there is some confusion
in the package db indeed. For example, I have currently 2 glibc packages
installed.
If you have 2 versions installed.. and the latest version was an
upgrade attempt via yum.. it sounds like there was a problem that
prevented the cleanup portion of the rpm transaction from completing
correctly. Was this an automated run of yum while you were afk.. or
did you happen to watch the output scroll? If there is a scriptlet
error or some other type of problem that rpm notices, there might be a
message to stdout or stderr. The error messages rpm throws don't show
up in the yum log however so you'd need the actually stdout and stderr
for the yum session you ran.
You might want to look for other duplicate packages which might have
also failed to cleanup correctly.
alias dupes='rpm -aq --queryformat "%{NAME}\n" | sort | uniq -c | grep
-v -E " *1 "'
-jef
12:30 p.m.
On Thu, 2005-07-28 at 13:11 -0400, Jeff Spaleta wrote:
If you have 2 versions installed.. and the latest version was an
upgrade attempt via yum.. it sounds like there was a problem that
prevented the cleanup portion of the rpm transaction from completing
correctly.
I see this kind of thing quite often. It doesn't help that yum will
often leave quite a lot of time between installing the new version of a
given package and uninstalling the old one.
--
dwmw2
12:37 p.m.
On Thu, 2005-07-28 at 18:30 +0100, David Woodhouse wrote:
On Thu, 2005-07-28 at 13:11 -0400, Jeff Spaleta wrote:
> If you have 2 versions installed.. and the latest version was an
> upgrade attempt via yum.. it sounds like there was a problem that
> prevented the cleanup portion of the rpm transaction from completing
> correctly.
I see this kind of thing quite often. It doesn't help that yum will
often leave quite a lot of time between installing the new version of a
given package and uninstalling the old one.
yum does a ts.order() - it's using rpm's transaction ordering - how
would you like for yum to speed up that time?
-sv
12:39 p.m.
From: "seth vidal" <skvidal(a)phy.duke.edu>
yum does a ts.order() - it's using rpm's transaction ordering
- how
would you like for yum to speed up that time?
Speaking of which, if things are left behind in case of a problem,
this is not very transactional, is it?
Obviously, this is not yum's fault, but rather RPM's -- shouldn't
RPM transactions be a bit more ACID?
--
Dimi Paun <dimi(a)lattica.com>
Lattica, Inc.
12:44 p.m.
On Thu, 2005-07-28 at 13:39 -0400, Dimi Paun wrote:
From: "seth vidal" <skvidal(a)phy.duke.edu>
> yum does a ts.order() - it's using rpm's transaction ordering - how
> would you like for yum to speed up that time?
Speaking of which, if things are left behind in case of a problem,
this is not very transactional, is it?
Obviously, this is not yum's fault, but rather RPM's -- shouldn't
RPM transactions be a bit more ACID?
a lot easier said than done.
it's not just a db transaction - it's also a fs and script transaction.
Often time lots of the %scriptlets can't be reversed.
-sv
12:41 p.m.
On Thu, Jul 28, 2005 at 01:37:40PM -0400, seth vidal wrote:
On Thu, 2005-07-28 at 18:30 +0100, David Woodhouse wrote:
> On Thu, 2005-07-28 at 13:11 -0400, Jeff Spaleta wrote:
> > If you have 2 versions installed.. and the latest version was an
> > upgrade attempt via yum.. it sounds like there was a problem that
> > prevented the cleanup portion of the rpm transaction from completing
> > correctly.
>
> I see this kind of thing quite often. It doesn't help that yum will
> often leave quite a lot of time between installing the new version of a
> given package and uninstalling the old one.
>
yum does a ts.order() - it's using rpm's transaction ordering - how
would you like for yum to speed up that time?
What about the cleanup scripts or removing old files? Looks to me
like all that is only done at the very end right now. Maybe completing
individual package updates in one step before updating the next
package should be an option for users.
greetings,
Florian La Roche
5:54 a.m.
On Thu, Jul 28, 2005 at 07:41:23PM +0200, Florian La Roche wrote:
What about the cleanup scripts or removing old files? Looks to me
like all that is only done at the very end right now. Maybe completing
individual package updates in one step before updating the next
package should be an option for users.
Please talk to your rpm maintainer, this was an intentional change
in rpm-4.
Cheers,
Michael.
--
Michael Schroeder mls(a)suse.de
main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}
1:41 a.m.
What about the cleanup scripts or removing old files? Looks to me
like all that is only done at the very end right now. Maybe completing
individual package updates in one step before updating the next
package should be an option for users.
again - once this is handed off to ts.run() yum only reports what rpm
does. so if this is going to be an option (which I think is a bit silly,
tbh) it would need to be supported in rpm, first.
-sv
12:42 p.m.
On 7/28/05, David Woodhouse <dwmw2(a)infradead.org> wrote:
I see this kind of thing quite often. It doesn't help that yum
will
often leave quite a lot of time between installing the new version of a
given package and uninstalling the old one.
A lot of time... would mean you are doing a lot of updates. You'd
rather yum do install/clean up one package at a time? I think that
leads to a whole other nest of problems. I don't even think cli rpm
does updates that way when you give it a big list of packages to
update.
I'm personally more concerned about rpm learning how to pass messages
back through its python bindings so yum and other tools can actually
log rpm level messages.. instead of having to screenscrape to know
what rpm was spitting out to stdout and stderr. I'm pretty sure
scriptlet failures have to be screenscraped as well as messages of
rpmnew and rpmsave files.
-jef
5:52 a.m.
On Thu, Jul 28, 2005 at 01:42:52PM -0400, Jeff Spaleta wrote:
A lot of time... would mean you are doing a lot of updates.
You'd
rather yum do install/clean up one package at a time? I think that
leads to a whole other nest of problems. I don't even think cli rpm
does updates that way when you give it a big list of packages to
update.
Well, old rpm versions used to do the uninstall right after the install.
I don't know why it was changed, the new way has some big disadvantages,
for example you get lots of duplicate packages if rpm exits in the
middle of the update.
Cheers,
Michael.
--
Michael Schroeder mls(a)suse.de
main(_){while(_=~getchar())putchar(~_-1/(~(_|32)/13*2-11)*13);}
12:39 p.m.
Hi.
Jeff Spaleta <jspaleta(a)gmail.com> wrote:
If you have 2 versions installed.. and the latest version was an
upgrade attempt via yum.. it sounds like there was a problem that
prevented the cleanup portion of the rpm transaction from completing
correctly. Was this an automated run of yum while you were afk.. or
did you happen to watch the output scroll?
I semi-watched it. There were some failed scripts, but nothing glibc
related as far as I recall.
You might want to look for other duplicate packages which might have
also failed to cleanup correctly.
There were some of those, but the main cause was glibc. After
reinstalling that I could chroot() into the system again. The
rest will be reinstalled after rebooting.
12:48 p.m.
On 7/28/05, Ralf Ertzinger <fedora(a)camperquake.de> wrote:
There were some of those, but the main cause was glibc.
And my point is... if glibc failed to cleanup or install correctly...
then there could very well have been a scriptlet error for glibc. The
postinstall scriptlet for glibc is rather.... unique.. as far as
scriptlets go. People with ppc hardware need to try to reproduce
this. If there is a pedantic situational bug in
/usr/sbin/glibc_post_upgrade.* its in your best interest to look at
this more closely.
-jef
1:06 p.m.
Hi.
Jeff Spaleta <jspaleta(a)gmail.com> wrote:
And my point is... if glibc failed to cleanup or install
correctly...
then there could very well have been a scriptlet error for glibc. The
postinstall scriptlet for glibc is rather.... unique.. as far as
scriptlets go. People with ppc hardware need to try to reproduce
this. If there is a pedantic situational bug in
/usr/sbin/glibc_post_upgrade.* its in your best interest to look at
this more closely.
I think yum rums %postinstall in the first phase of the upgrade
(Upgrading <....>). The glibc script restarts the sshd, and this
is noticeable in the console output. I remeber seeing this yesterday,
and there were no visible errors on the console at this stage.
We'll see if the next upgrade goes well.
--
Disloyalty: There comes a time when every team must learn to make
individual sacrifices. -- Despair Inc. calendar, September 2001
1:08 p.m.
On Thu, 2005-07-28 at 13:48 -0400, Jeff Spaleta wrote:
On 7/28/05, Ralf Ertzinger <fedora(a)camperquake.de> wrote:
> There were some of those, but the main cause was glibc.
And my point is... if glibc failed to cleanup or install correctly...
then there could very well have been a scriptlet error for glibc. The
postinstall scriptlet for glibc is rather.... unique.. as far as
scriptlets go. People with ppc hardware need to try to reproduce
this. If there is a pedantic situational bug in
/usr/sbin/glibc_post_upgrade.* its in your best interest to look at
this more closely.
Or a trigger failure - redhat-lsb has caused this in the past
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=160585
Without reproducibility and logs it's impossible to say.
Paul
1:17 p.m.
Hi.
Paul Nasrat <pnasrat(a)redhat.com> wrote:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=160585
Without reproducibility and logs it's impossible to say.
All I can offer you is the list of files that yum upgraded.
Maybe yum ought to log script errors, too.
--
Q. How many people does it take to change a lightbulb on Usenet?
A. 1 to say the light's out, 100 to say "Yes, I noticed that", and 1,000
to post requests for FTP sites with free software that replaces
lightbulbs. Meanwhile, it's still dark.
1:18 p.m.
On 7/28/05, Paul Nasrat <pnasrat(a)redhat.com> wrote:
Or a trigger failure - redhat-lsb has caused this in the past
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=160585
Without reproducibility and logs it's impossible to say.
blasted triggers..... does the ppc redhat-lsb package have something
ppc specific?
-jef
1:25 p.m.
On Thu, 2005-07-28 at 14:18 -0400, Jeff Spaleta wrote:
On 7/28/05, Paul Nasrat <pnasrat(a)redhat.com> wrote:
> Or a trigger failure - redhat-lsb has caused this in the past
>
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=160585
>
> Without reproducibility and logs it's impossible to say.
blasted triggers..... does the ppc redhat-lsb package have something
ppc specific?
No it has a trigger on glibc
Paul
6:11 a.m.
On Thu, 2005-07-28 at 17:13 +0100, David Woodhouse wrote:
On Thu, 2005-07-28 at 12:48 +0200, Ralf Ertzinger wrote:
> After applying yesterday's rawhide to my iBook, the system is quite
> thoroughly toasted. After yum was finished, ssh'ing into the
> machine no longer worke, and neither did logging in at the local
> console. Rebooting the machine hangs after the initrd is processed,
> seemingly failing to execute /sbin/init.
I just updated the Pegasos II box to rawhide, and it seems to be
working fine here. My first suspicion would be a failure to upgrade
glibc correctly.
It was fine immediately after the upgrade but it died overnight, when
prelink happened. Un-prelinking /lib/ld-2.3.90.so makes it all work
again.
--
dwmw2
4:41 p.m.
On Thursday, July 28, 2005 3:48 am, Ralf Ertzinger wrote:
Hi.
After applying yesterday's rawhide to my iBook, the system is quite
thoroughly toasted. After yum was finished, ssh'ing into the
machine no longer worke, and neither did logging in at the local
console. Rebooting the machine hangs after the initrd is processed,
seemingly failing to execute /sbin/init.
My suspicion is that a very low level library is botched (glibc or
selinux). Trying to chroot() into the system from an old YDL 3.0
install CD (the only CD what would boot on PPC I found on short
notice) fails, just producing segfaults, but YDL 3.0 is quite
old, of course, and still running 2.4. Will try and get a
FC4 install CD today and try again.
PS: The system does not use selinux.
I noticed the same thing when I updated my PowerBook yesterday. The
machine wouldn't even reboot after the 'yum update' was complete; after
'starting nash' I didn't see any output. I had to boot from an old FC4
CD and recover.
What I found was that glibc was apparently busted (everything
segfaulted), along with ld.so, libk5crypto (fixing those fixed the
'Illegal instructions' I was getting) and maybe one or two others. I
had to copy them from my boot image, install a new kernel and initrd,
and reboot to downgrade to the older glibc. I didn't see any errors
during the upgrade either, so I'm not sure what went wrong (though the
logs are probably still there).
Jesse
6842
days inactive
6846
days old
26 comments
10 participants
participants (10)
-
David Woodhouse -
Dimi Paun -
Florian La Roche -
Jef Spaleta -
Jesse Barnes -
Michael Schroeder -
Paul F. Johnson -
Paul Nasrat -
Ralf Ertzinger -
seth vidal