rsyslogd suddenly went crazy (120% cpu)
- First Post
- Replies
- Stats
-
Go to
- ----- 2024 -----
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2009 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2008 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2007 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2006 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2005 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2004 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2003 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2002 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2001 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2000 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1999 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1998 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1997 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1996 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1995 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1994 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1993 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1992 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1991 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1990 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1989 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1988 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1987 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1986 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1985 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1984 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1983 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1982 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1981 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 1980 -----
- December
- November
Hi,
I am running Fedora 19 + updates-testing and suddenly rsyslogd went crazy,
and is now consuming constantly 120% cpu (and is also constantly expanding
allocated memory).
I first thought it could have something to do with updates I've applied
recently, so I rolled back to an older btrfs-snapshot - which however
didn't improve the situation.
I've recorded some stack-traces: http://pastebin.com/ikwC3h4g
When executing "tail -f /var/log/messages" I get the same messages from 16.
and 17. May over and over again.
Any ideas what could cause these problems?
Thank you in advance, Clemens
Attachments:
- attachment.html (text/html — 698 bytes)
Am 14.06.2013 23:10, schrieb Clemens Eisserer:
I am running Fedora 19 + updates-testing and suddenly rsyslogd went
crazy, and is now consuming constantly 120% cpu
(and is also constantly expanding allocated memory).
I first thought it could have something to do with updates I've applied recently, so
I rolled back to an older
btrfs-snapshot - which however didn't improve the situation.
I've recorded some stack-traces: http://pastebin.com/ikwC3h4g
When executing "tail -f /var/log/messages" I get the same messages from 16. and
17. May over and over again.
Any ideas what could cause these problems?
wrong mailing-list because F19 is *not* released now and
so your question belongs to the testing-list, but however
-------- Original-Nachricht --------
Betreff: Re: rsyslog-7.4.0-1 is bad
Datum: Fri, 14 Jun 2013 00:24:24 -0700
Von: Adam Williamson <awilliam(a)redhat.com>
Antwort an: Development discussions related to Fedora
<devel(a)lists.fedoraproject.org>
Organisation: Red Hat
An: For testing and quality assurance of Fedora releases
<test(a)lists.fedoraproject.org>
Kopie (CC): devel(a)lists.fedoraproject.org
On Thu, 2013-06-13 at 23:48 -0700, Adam Williamson wrote:
Just a heads up for F19 / F20 users - rsyslog-7.4.0-1 is a bad
update,
many users (inc. me on two systems) are seeing it
spamming /var/log/messages with old logs and sucking up 100% CPU time.
You might not notice immediately if you're not running something that
spins its fans up. I recommend downgrading to 7.2.6-1 ASAP. Bug is
https://bugzilla.redhat.com/show_bug.cgi?id=974335 .
Better bug link:
https://bugzilla.redhat.com/show_bug.cgi?id=974132
Turns out this is ultimately a journald bug (and explains why journalctl
has been busted for me lately). But still, if you're hit by it, you're
going to want to nerf rsyslogd ASAP, because until you do, it will
spam /var/log/messages until your disk is full. Mine had reached 8GB by
the time I noticed. Some other poor bugger's got to 30GB.
Hi Harald,
Better bug link:
https://bugzilla.redhat.com/show_bug.cgi?id=974132
Turns out this is ultimately a journald bug (and explains why journalctl
has been busted for me lately). But still, if you're hit by it, you're
going to want to nerf rsyslogd ASAP, because until you do, it will
spam /var/log/messages until your disk is full. Mine had reached 8GB by
the time I noticed. Some other poor bugger's got to 30GB
Thanks for the hint & pointer - by deleting the journal I was able to solve
the effects to my system.
Thanks, Clemens
Kevin Fenzi writes:
Did I read this right? /var/log/messages have been replaced by binary goo?
On Fri, 14 Jun 2013 18:51:53 -0400
Sam Varshavchik wrote:
Did I read this right? /var/log/messages have been replaced by binary
goo?
Nah, just the systemd crap which no one wants to read anyway
(I think). I certainly still have a /var/log/messages with lots
of junk in it (in fact, lots of systemd junk as well).
P.S. A good trick here is to add this line to /etc/yum.conf
exclude=rsyslog-7.4.0-1.fc19.x86_64
That way, yum won't wind up reinstalling the crazy version
of rsyslog.
On Fri, 14 Jun 2013 18:51:53 -0400
Sam Varshavchik <mrsam(a)courier-mta.com> wrote:
Kevin Fenzi writes:
> http://www.happyassassin.net/2013/06/14/fedora-1920-logfile-explosions/
Did I read this right? /var/log/messages have been replaced by binary
goo?
No, you didn't read it right.
rsyslog now has the ability to pull from journald and log (as it always
has) in text to /var/log/messages. Nothing changes aside from more
information in /var/log/messages from journald.
kevin
On Fri, 14 Jun 2013 23:10:08 +0200
Clemens Eisserer <linuxhippy(a)gmail.com> wrote:
Hi,
I am running Fedora 19 + updates-testing and suddenly rsyslogd went
crazy, and is now consuming constantly 120% cpu (and is also
constantly expanding allocated memory).
I first thought it could have something to do with updates I've
applied recently, so I rolled back to an older btrfs-snapshot - which
however didn't improve the situation.
I've recorded some stack-traces: http://pastebin.com/ikwC3h4g
When executing "tail -f /var/log/messages" I get the same messages
from 16. and 17. May over and over again.
Any ideas what could cause these problems?
Thank you in advance, Clemens
http://www.happyassassin.net/2013/06/14/fedora-1920-logfile-explosions/
kevin
3981
days inactive
3982
days old
6 comments
5 participants
tags (0)
participants (5)
-
Clemens Eisserer -
Kevin Fenzi -
Reindl Harald -
Sam Varshavchik -
Tom Horsley