1:04 p.m.
Current kernel: 5.7.12-200.fc32.x86_64
gnome desktop
PC: Thinkpad X280, intel core i7, 8th gen
After some time, usually hours, the following four tasks in top are
running at 100%:
sadc
kworker/6:1+events_freezable
dmesg
systemd-journal
sometimes there are only three of these at 100% but usually four, never
more.
I have not been able to come up with a sequence of steps to cause it.
At this point the only way to shut down is to use the power switch, it
will run forever in a normal shutdown as the timeout limits increase.
The next start up will always be strange:
Things will proceed apparently normally up to the login opportunity:
If I check the upper right corner I will see that the wired and
wireless networks are both up.
After I login the wireless will not be up and nothing I try will
bring it up.
There will also be notifications about "problems", up to 35 occurrences
of "system problem" and "kerneloops" none of which is
"reportable"
This login process will have to be repeated probably twice before a
normal environment with wireless network operating.
This also occurred prior to the last one or two updates.
At this point I think I will do a clean install.
I have assumed that if this happened to others it would have appeared
here already, but just in case....
I have been using Fedora forever as a software system development platform
screen shot of "top" available if anyone wants it.
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells(a)leidos.com
1:29 p.m.
On Thu, Aug 13, 2020 at 12:04 PM Wells, Roger K. via devel
<devel(a)lists.fedoraproject.org> wrote:
Current kernel: 5.7.12-200.fc32.x86_64
gnome desktop
PC: Thinkpad X280, intel core i7, 8th gen
After some time, usually hours, the following four tasks in top are
running at 100%:
sadc
What is this? I don't have it running or installed on my system, and
dnf search returns nothing.
cat /proc/<pid>/stack
To get a stack trace for that sadc process, and I guess file a bug
against it, whatever it is.
And also a complete dmesg might reveal something to go on.
--
Chris Murphy
3:05 p.m.
On 8/13/20 2:04 PM, Wells, Roger K. via devel wrote:
After some time, usually hours, the following four tasks in top are
running at 100%:
sadc
kworker/6:1+events_freezable
dmesg
systemd-journal
So sadc is not part of current Fedora. It may be some artifact from
older Fedoras (e.g. sysstat-11.5.7-4.fc27.x86_64 has /usr/lib64/sa/sadc)
or some custom system activity data collection software that is locally
installed at your site.
My guess is it goes bonkers and causes frantic kernel worker thread
activity, and generates tons of messges, keeping dmesg and journal busy.
Please look into that, and let us know what you find out.
3:12 p.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/13/20 4:05 PM, Przemek Klosowski via devel wrote:
On 8/13/20 2:04 PM, Wells, Roger K. via devel wrote:
> After some time, usually hours, the following four tasks in top are
> running at 100%:
> sadc
> kworker/6:1+events_freezable
> dmesg
> systemd-journal
So sadc is not part of current Fedora. It may be some artifact from
older Fedoras (e.g. sysstat-11.5.7-4.fc27.x86_64 has
/usr/lib64/sa/sadc) or some custom system activity data collection
software that is locally installed at your site.
My guess is it goes bonkers and causes frantic kernel worker thread
activity, and generates tons of messges, keeping dmesg and journal
busy. Please look into that, and let us know what you find out.
Thanks. I'll do
something to disable it. FWIW, this computer has never
run anything but Fedora since day1.
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells(a)leidos.com
3:22 p.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/13/20 4:12 PM, Wells, Roger K. via devel wrote:
> So sadc is not part of current Fedora. It may be some artifact
from
> older Fedoras (e.g. sysstat-11.5.7-4.fc27.x86_64 has
> /usr/lib64/sa/sadc) or some custom system activity data collection
> software that is locally installed at your site.
Thanks. I'll do something to disable it. FWIW, this computer has never
run anything but Fedora since day1.
Did you install F32 from scratch, or was it an upgrade from earlier
versions? If you had F27 on it, upgraded to 28->29->30->31->32, it would
explain how sadc got on it.
Also, I have a Fedora box but my work installs third party management
and monitoring software on it that is not part of official Fedora, so
that would be another avenue for something called sadc to get on your
Fedora box.
3:42 p.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/13/20 4:23 PM, Przemek Klosowski via devel wrote:
On 8/13/20 4:12 PM, Wells, Roger K. via devel wrote:
So sadc is not part of current Fedora. It may be some artifact from
older Fedoras (e.g. sysstat-11.5.7-4.fc27.x86_64 has
/usr/lib64/sa/sadc) or some custom system activity data collection
software that is locally installed at your site.
Thanks. I'll do something to disable it. FWIW, this computer has never
run anything but Fedora since day1.
Did you install F32 from scratch, or was it an upgrade from earlier versions? If you had
F27 on it, upgraded to 28->29->30->31->32, it would explain how sadc got on
it.
Also, I have a Fedora box but my work installs third party management and monitoring
software on it that is not part of official Fedora, so that would be another avenue for
something called sadc to get on your Fedora box.
No, it has had several fedora releases and many updates. I don't recall where it
started. F27 is possible though.
That's why I was going to do a fresh install (mentioned in the original)
I am the only one who puts anything on this machine though, so no third party management
or monitoring.
I just thought that it might be worth understanding what's going on here in case
someone else is affected now or later.
Thanks for your interest
_______________________________________________
devel mailing list --
devel@lists.fedoraproject.org<mailto:devel@lists.fedoraproject.org>
To unsubscribe send an email to
devel-leave@lists.fedoraproject.org<mailto:devel-leave@lists.fedoraproject.org>
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells@leidos.com<mailto:roger.k.wells@leidos.com>
3:27 p.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/13/20 4:12 PM, Wells, Roger K. via devel wrote:
I'll do something to disable it.
Oh, just thougth I'd mention---what I'd do would be
locate sadc <- hopefully this would return the location of the sadc
binary, perhaps /var/lib64/sa/sadc
rpm -qf /var/lib64/sa/sadc <- this will report which package is sadc a
part of, perhaps sysstat
yum erase sysstat <- deletes the offending package once and for all.
3:44 p.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/13/20 4:28 PM, Przemek Klosowski via devel wrote:
On 8/13/20 4:12 PM, Wells, Roger K. via devel wrote:
> I'll do something to disable it.
Oh, just thougth I'd mention---what I'd do would be
locate sadc <- hopefully this would return the location of the
sadc binary, perhaps /var/lib64/sa/sadc
rpm -qf /var/lib64/sa/sadc <- this will report which package is sadc
a part of, perhaps sysstat
yum erase sysstat <- deletes the offending package once and for all.
Thanks for
the instructions. I'll report back
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells(a)leidos.com
7:33 a.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/13/20 4:44 PM, Wells, Roger K. via devel wrote:
On 8/13/20 4:28 PM, Przemek Klosowski via devel wrote:
> On 8/13/20 4:12 PM, Wells, Roger K. via devel wrote:
>> I'll do something to disable it.
> Oh, just thougth I'd mention---what I'd do would be
>
> locate sadc <- hopefully this would return the location of the
> sadc binary, perhaps /var/lib64/sa/sadc
>
> rpm -qf /var/lib64/sa/sadc <- this will report which package is sadc
> a part of, perhaps sysstat
>
> yum erase sysstat <- deletes the offending package once and for all.
Thanks for the instructions. I'll report back
That was not the cause.
Now when it happens I have only three tasks running at 100% (same ones
as reported earlier).
Everything else, kerneloops, shutdown via power switch, etc, is as before.
> _______________________________________________
> devel mailing list -- devel(a)lists.fedoraproject.org
> To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells(a)leidos.com
7:45 a.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/14/20 8:33 AM, Wells, Roger K. via devel wrote:
That was not the cause.
Now when it happens I have only three tasks running at 100% (same ones
as reported earlier).
Everything else, kerneloops, shutdown via power switch, etc, is as before.
Could you repost to the list with more info? I think you're saying that
after removing sadc you still are 100% CPU with
kworker/6:1+events_freezable
dmesg
systemd-journal
running full tilt. THis probably means that it's the kernel worker threads are doing
something that causes lots of errors, and all the other processes (including sadc that is
now gone) are just busy writing those errors.
So, what is kernel doing? what does your system log say? type 'dmesg' or
'journalctl -e' and look at the end of the data.
Also, the perf toolkit might help as explained in
https://askubuntu.com/questions/33640/kworker-what-is-it-and-why-is-it-ho...
1.
Install |perf|:
|sudo apt-get install linux-tools-common linux-tools-3.11.0-15-generic |
(The second package must match your kernel version. You can first
install just |linux-tools-common| and call |perf| to let it tell you
which package it needs.)
2.
Record some 10 seconds of backtraces on all your CPUs:
|sudo perf record -g -a sleep 10 |
3.
Analyse your recording:
|sudo perf report |
8:14 a.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/14/20 8:45 AM, Przemek Klosowski via devel wrote:
On 8/14/20 8:33 AM, Wells, Roger K. via devel wrote:
That was not the cause.
Now when it happens I have only three tasks running at 100% (same ones
as reported earlier).
Everything else, kerneloops, shutdown via power switch, etc, is as before.
Could you repost to the list with more info? I think you're saying that after removing
sadc you still are 100% CPU with
kworker/6:1+events_freezable
dmesg
systemd-journal
That is correct. The "6:1" in the kworker case varies.
running full tilt. THis probably means that it's the kernel worker threads are doing
something that causes lots of errors, and all the other processes (including sadc that is
now gone) are just busy writing those errors.
So, what is kernel doing? what does your system log say? type 'dmesg' or
'journalctl -e' and look at the end of the data.
The following repeats at the end of journalctl:
In the last second of the journal there are 14688 individual entries.
Aug 13 18:23:51 rwells-x280 kernel: Hardware name: LENOVO 20KFCTO1WW/20KFCTO1WW, BIOS
N20ET55W (1.40 ) 06/01/2020
Aug 13 18:23:51 rwells-x280 kernel: Workqueue: events_freezable ieee80211_restart_work
[mac80211]
Aug 13 18:23:51 rwells-x280 kernel: RIP: 0010:drv_sta_state+0x26e/0x3e0 [mac80211]
Aug 13 18:23:51 rwells-x280 kernel: Code: 0b 45 31 ed e9 44 fe ff ff 48 8b 83 78 04 00 00
48 8d b3 98 04 00 00 48 c7 c7 80 46 f2 c0 48 85 c0 48 0f 45 f0 e8 c9 bf 23 d2 <0f>
0b 41 bd fb ff ff ff e9 1b fe ff ff 65 8b 05 be fc 16 3f 89 c0
Aug 13 18:23:51 rwells-x280 systemd-journald[928]: Missed 44 kernel messages
Aug 13 18:23:51 rwells-x280 kernel: iTCO_wdt mac80211 iTCO_vendor_support snd_hda_intel
irqbypass intel_rapl_msr snd_intel_dspcfg rapl snd_hda_codec libarc4 snd_hda_core uvcvideo
intel_cstate iwlwifi snd_hwdep btusb videobuf2_vmalloc snd_seq intel_uncore
videobuf2_memops btrtl videobuf2_v4l2 btbcm snd_seq_device btintel snd_pcm pcspkr cfg80211
mei_me wmi_bmof intel_wmi_thunderbolt videobuf2_common i2c_i801 snd_timer thunderbolt
videodev bluetooth mei intel_pch_thermal mc thinkpad_acpi ucsi_acpi joydev typec_ucsi
processor_thermal_device ecdh_generic intel_rapl_common ledtrig_audio ecc
intel_xhci_usb_role_switch roles intel_soc_dts_iosf typec snd soundcore rfkill
int3403_thermal int340x_thermal_zone int3400_thermal acpi_thermal_rel acpi_pad ip_tables
dm_crypt i915 i2c_algo_bit cec crct10dif_pclmul crc32_pclmul crc32c_intel drm_kms_helper
ghash_clmulni_intel nvme drm serio_raw e1000e uas nvme_core usb_storage wmi video
hid_multitouch fuse
Aug 13 18:23:51 rwells-x280 kernel: CPU: 0 PID: 5 Comm: kworker/0:0 Tainted: G W
5.7.12-200.fc32.x86_64 #1
Also, the perf toolkit might help as explained in
https://askubuntu.com/questions/33640/kworker-what-is-it-and-why-is-it-ho...
1. Install perf:
sudo apt-get install linux-tools-common linux-tools-3.11.0-15-generic
(The second package must match your kernel version. You can first install just
linux-tools-common and call perf to let it tell you which package it needs.)
2. Record some 10 seconds of backtraces on all your CPUs:
sudo perf record -g -a sleep 10
3. Analyse your recording:
sudo perf report
I'll give this a try
thanks
1.
_______________________________________________
devel mailing list --
devel@lists.fedoraproject.org<mailto:devel@lists.fedoraproject.org>
To unsubscribe send an email to
devel-leave@lists.fedoraproject.org<mailto:devel-leave@lists.fedoraproject.org>
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells@leidos.com<mailto:roger.k.wells@leidos.com>
11:24 a.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 20-08-14 09:14:51, Wells, Roger K. via devel wrote:
On 8/14/20 8:45 AM, Przemek Klosowski via devel wrote:
> So, what is kernel doing? what does your system log say? type
> 'dmesg' or 'journalctl -e' and look at the end of the data.
The following repeats at the end of journalctl:
In the last second of the journal there are 14688 individual entries.
Aug 13 18:23:51 rwells-x280 kernel: Hardware name: LENOVO
20KFCTO1WW/20KFCTO1WW, BIOS N20ET55W (1.40 ) 06/01/2020
Aug 13 18:23:51 rwells-x280 kernel: Workqueue: events_freezable
ieee80211_restart_work [mac80211]
Aug 13 18:23:51 rwells-x280 kernel: RIP:
0010:drv_sta_state+0x26e/0x3e0 [mac80211]
Aug 13 18:23:51 rwells-x280 kernel: Code: 0b 45 31 ed e9 44 fe ff ff
48 8b 83 78 04 00 00 48 8d b3 98 04 00 00 48 c7 c7 80 46 f2 c0 48 85
c0 48 0f 45 f0 e8 c9 bf 23 d2 <0f> 0b 41 bd fb ff ff ff e9 1b fe ff
ff 65 8b 05 be fc 16 3f 89 c0
Aug 13 18:23:51 rwells-x280 systemd-journald[928]: Missed 44 kernel
messages
Aug 13 18:23:51 rwells-x280 kernel: iTCO_wdt mac80211
iTCO_vendor_support snd_hda_intel irqbypass intel_rapl_msr
snd_intel_dspcfg rapl snd_hda_codec libarc4 snd_hda_core uvcvideo
intel_cstate iwlwifi snd_hwdep btusb videobuf2_vmalloc snd_seq
intel_uncore videobuf2_memops btrtl videobuf2_v4l2 btbcm
snd_seq_device btintel snd_pcm pcspkr cfg80211 mei_me wmi_bmof
intel_wmi_thunderbolt videobuf2_common i2c_i801 snd_timer thunderbolt
videodev bluetooth mei intel_pch_thermal mc thinkpad_acpi ucsi_acpi
joydev typec_ucsi processor_thermal_device ecdh_generic
intel_rapl_common ledtrig_audio ecc intel_xhci_usb_role_switch roles
intel_soc_dts_iosf typec snd soundcore rfkill int3403_thermal
int340x_thermal_zone int3400_thermal acpi_thermal_rel acpi_pad
ip_tables dm_crypt i915 i2c_algo_bit cec crct10dif_pclmul
crc32_pclmul crc32c_intel drm_kms_helper ghash_clmulni_intel nvme drm
serio_raw e1000e uas nvme_core usb_storage wmi video hid_multitouch
fuse
Aug 13 18:23:51 rwells-x280 kernel: CPU: 0 PID: 5 Comm: kworker/0:0
Tainted: G W 5.7.12-200.fc32.x86_64 #1
There is a problem with your WIFI adapter (what mac80211 is referring
to). I don't think you can unplug and replug the adapter, which would
likely help, so don't just reset the computer, but power-cycle it.
Sometimes dmesg has messages that don't make it into the log due to
rate-limiting. Not sure how to see dmesg when things are jammed up.
Do you know if the problem depends on a particular kernel? Ususally
an older kernel will work, even if it isn't for the current version of
Fedora. If that helps, try to narrow it down to a particular kernel.
If you find a boundary where it starts happening, report it on the
kernel bugzilla. You won't be able to properly report the bug, since
your kernel is tainted, but reporting it improperly may still be worth
something to kernel developers.
I suspect the hardware, but I'm often wrong.
--
____________________________________________________________________
TonyN.:' <mailto:tonynelson@georgeanelson.com>
' <http://www.georgeanelson.com/>
9:36 a.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/14/20 12:29 PM, Tony Nelson wrote:
On 20-08-14 09:14:51, Wells, Roger K. via devel wrote:
> On 8/14/20 8:45 AM, Przemek Klosowski via devel wrote:
>> So, what is kernel doing? what does your system log say? type
>> 'dmesg' or 'journalctl -e' and look at the end of the data.
>
> The following repeats at the end of journalctl:
> In the last second of the journal there are 14688 individual entries.
>
> Aug 13 18:23:51 rwells-x280 kernel: Hardware name: LENOVO
> 20KFCTO1WW/20KFCTO1WW, BIOS N20ET55W (1.40 ) 06/01/2020
> Aug 13 18:23:51 rwells-x280 kernel: Workqueue: events_freezable
> ieee80211_restart_work [mac80211]
> Aug 13 18:23:51 rwells-x280 kernel: RIP:
> 0010:drv_sta_state+0x26e/0x3e0 [mac80211]
> Aug 13 18:23:51 rwells-x280 kernel: Code: 0b 45 31 ed e9 44 fe ff ff
> 48 8b 83 78 04 00 00 48 8d b3 98 04 00 00 48 c7 c7 80 46 f2 c0 48 85
> c0 48 0f 45 f0 e8 c9 bf 23 d2 <0f> 0b 41 bd fb ff ff ff e9 1b fe ff
> ff 65 8b 05 be fc 16 3f 89 c0
> Aug 13 18:23:51 rwells-x280 systemd-journald[928]: Missed 44 kernel
> messages
> Aug 13 18:23:51 rwells-x280 kernel: iTCO_wdt mac80211
> iTCO_vendor_support snd_hda_intel irqbypass intel_rapl_msr
> snd_intel_dspcfg rapl snd_hda_codec libarc4 snd_hda_core uvcvideo
> intel_cstate iwlwifi snd_hwdep btusb videobuf2_vmalloc snd_seq
> intel_uncore videobuf2_memops btrtl videobuf2_v4l2 btbcm
> snd_seq_device btintel snd_pcm pcspkr cfg80211 mei_me wmi_bmof
> intel_wmi_thunderbolt videobuf2_common i2c_i801 snd_timer thunderbolt
> videodev bluetooth mei intel_pch_thermal mc thinkpad_acpi ucsi_acpi
> joydev typec_ucsi processor_thermal_device ecdh_generic
> intel_rapl_common ledtrig_audio ecc intel_xhci_usb_role_switch roles
> intel_soc_dts_iosf typec snd soundcore rfkill int3403_thermal
> int340x_thermal_zone int3400_thermal acpi_thermal_rel acpi_pad
> ip_tables dm_crypt i915 i2c_algo_bit cec crct10dif_pclmul
> crc32_pclmul crc32c_intel drm_kms_helper ghash_clmulni_intel nvme drm
> serio_raw e1000e uas nvme_core usb_storage wmi video hid_multitouch fuse
> Aug 13 18:23:51 rwells-x280 kernel: CPU: 0 PID: 5 Comm: kworker/0:0
> Tainted: G W 5.7.12-200.fc32.x86_64 #1
There is a problem with your WIFI adapter (what mac80211 is referring
to). I don't think you can unplug and replug the adapter, which would
likely help, so don't just reset the computer, but power-cycle it.
Sometimes dmesg has messages that don't make it into the log due to
rate-limiting. Not sure how to see dmesg when things are jammed up.
Do you know if the problem depends on a particular kernel? Ususally
an older kernel will work, even if it isn't for the current version of
Fedora. If that helps, try to narrow it down to a particular kernel.
If you find a boundary where it starts happening, report it on the
kernel bugzilla. You won't be able to properly report the bug, since
your kernel is tainted, but reporting it improperly may still be worth
something to kernel developers.
I suspect the hardware, but I'm often wrong.
Thanks (finally)
Yes it seems to be associated with wifi. Disable wifi (nmcli radio wifi
off) and problem leaves.
I threw in the towel and did a fresh install of F32 to no avail.
A little research reveals similar/same problem in mac80211 beginning
around kernel 5.6.9 IIRC.
In any case wifi is to important to me so I will need to revert to
CentOS until this is fixed or
if it is a hardware issue the kernel can deal with it (report and move
on). I am suspicious about a hardware problem
since sometimes I can run connected through wifi for hours, other times
the problem appears quickly.
I'll keep it up to date and if/when the issue is resolved I'll be back.
Let me know if I can provide any info or tests.
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells(a)leidos.com
5:45 p.m.
New subject: EXTERNAL: Re: hundred percent cpu load
As tedious as it is, my suggestion is kernel bisect.
Various strategies abound. I'd probably go for course to fine
granularity: start with Fedora Live ISOs, and see clearly where it
does and doesn't happen. It sounds like the problem manifests quickly
so you don't need to reinstall or compile. Once you've got a good vs
bad kernel based on ISOs, you probably will need to compile because
koji kernels aren't kept around more than (eek! I don't know but maybe
a couple months?). I suggest not bothering with stable updates. Just
work on the mainline kernel git. 5.3.0 good, 5.7.0 bad.
make localmodconfig will significantly reduce the cost of compile time
If it's not fixed already in 5.8 or 5.9, easy to test because both are
in koji, I wouldn't assume it's just going to get fixed somehow.
Someone would have to do a bisect and report it. In very rare cases if
it's not fixed soonish (a kernel release or two) insist on lkml that
the change that caused the regression is reverted. Of course, if it's
hardware related, you won't likely find a good kernel.
--
Chris Murphy
8:09 a.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/20/20 6:47 PM, Chris Murphy wrote:
As tedious as it is, my suggestion is kernel bisect.
Various strategies abound. I'd probably go for course to fine
granularity: start with Fedora Live ISOs, and see clearly where it
does and doesn't happen. It sounds like the problem manifests quickly
so you don't need to reinstall or compile. Once you've got a good vs
bad kernel based on ISOs, you probably will need to compile because
koji kernels aren't kept around more than (eek! I don't know but maybe
a couple months?). I suggest not bothering with stable updates. Just
work on the mainline kernel git. 5.3.0 good, 5.7.0 bad.
make localmodconfig will significantly reduce the cost of compile time
If it's not fixed already in 5.8 or 5.9, easy to test because both are
in koji, I wouldn't assume it's just going to get fixed somehow.
Someone would have to do a bisect and report it. In very rare cases if
it's not fixed soonish (a kernel release or two) insist on lkml that
the change that caused the regression is reverted. Of course, if it's
hardware related, you won't likely find a good kernel.
--
Chris Murphy
Thanks for your time & suggestions.
I have already started trying older live ISO's just don't have but one
right now but I'll find some more
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells(a)leidos.com
12:44 p.m.
New subject: EXTERNAL: Re: hundred percent cpu load
On 8/14/20 9:15 AM, Wells, Roger K. via devel wrote:
On 8/14/20 8:45 AM, Przemek Klosowski via devel wrote:
On 8/14/20 8:33 AM, Wells, Roger K. via devel wrote:
That was not the cause.
Now when it happens I have only three tasks running at 100% (same ones
as reported earlier).
Everything else, kerneloops, shutdown via power switch, etc, is as before.
Could you repost to the list with more info? I think you're saying that after removing
sadc you still are 100% CPU with
kworker/6:1+events_freezable
dmesg
systemd-journal
That is correct. The "6:1" in the kworker case varies.
running full tilt. THis probably means that it's the kernel worker threads are doing
something that causes lots of errors, and all the other processes (including sadc that is
now gone) are just busy writing those errors.
So, what is kernel doing? what does your system log say? type 'dmesg' or
'journalctl -e' and look at the end of the data.
The following repeats at the end of journalctl:
In the last second of the journal there are 14688 individual entries.
Aug 13 18:23:51 rwells-x280 kernel: Hardware name: LENOVO 20KFCTO1WW/20KFCTO1WW, BIOS
N20ET55W (1.40 ) 06/01/2020
Aug 13 18:23:51 rwells-x280 kernel: Workqueue: events_freezable ieee80211_restart_work
[mac80211]
Aug 13 18:23:51 rwells-x280 kernel: RIP: 0010:drv_sta_state+0x26e/0x3e0 [mac80211]
Aug 13 18:23:51 rwells-x280 kernel: Code: 0b 45 31 ed e9 44 fe ff ff 48 8b 83 78 04 00 00
48 8d b3 98 04 00 00 48 c7 c7 80 46 f2 c0 48 85 c0 48 0f 45 f0 e8 c9 bf 23 d2 <0f>
0b 41 bd fb ff ff ff e9 1b fe ff ff 65 8b 05 be fc 16 3f 89 c0
Aug 13 18:23:51 rwells-x280 systemd-journald[928]: Missed 44 kernel messages
Aug 13 18:23:51 rwells-x280 kernel: iTCO_wdt mac80211 iTCO_vendor_support snd_hda_intel
irqbypass intel_rapl_msr snd_intel_dspcfg rapl snd_hda_codec libarc4 snd_hda_core uvcvideo
intel_cstate iwlwifi snd_hwdep btusb videobuf2_vmalloc snd_seq intel_uncore
videobuf2_memops btrtl videobuf2_v4l2 btbcm snd_seq_device btintel snd_pcm pcspkr cfg80211
mei_me wmi_bmof intel_wmi_thunderbolt videobuf2_common i2c_i801 snd_timer thunderbolt
videodev bluetooth mei intel_pch_thermal mc thinkpad_acpi ucsi_acpi joydev typec_ucsi
processor_thermal_device ecdh_generic intel_rapl_common ledtrig_audio ecc
intel_xhci_usb_role_switch roles intel_soc_dts_iosf typec snd soundcore rfkill
int3403_thermal int340x_thermal_zone int3400_thermal acpi_thermal_rel acpi_pad ip_tables
dm_crypt i915 i2c_algo_bit cec crct10dif_pclmul crc32_pclmul crc32c_intel drm_kms_helper
ghash_clmulni_intel nvme drm serio_raw e1000e uas nvme_core usb_storage wmi video
hid_multitouch fuse
Aug 13 18:23:51 rwells-x280 kernel: CPU: 0 PID: 5 Comm: kworker/0:0 Tainted: G W
5.7.12-200.fc32.x86_64 #1
Also, the perf toolkit might help as explained in
https://askubuntu.com/questions/33640/kworker-what-is-it-and-why-is-it-ho...
1. Install perf:
sudo apt-get install linux-tools-common linux-tools-3.11.0-15-generic
(The second package must match your kernel version. You can first install just
linux-tools-common and call perf to let it tell you which package it needs.)
2. Record some 10 seconds of backtraces on all your CPUs:
sudo perf record -g -a sleep 10
3. Analyse your recording:
sudo perf report
I'll give this a try
thanks
I did it but I should have remembered earlier:
When this happens all sudo prefaced terminal input just locks up and never (apparently)
executes.
Maybe there is a clue here?
1.
_______________________________________________
devel mailing list --
devel@lists.fedoraproject.org<mailto:devel@lists.fedoraproject.org>
To unsubscribe send an email to
devel-leave@lists.fedoraproject.org<mailto:devel-leave@lists.fedoraproject.org>
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells@leidos.com<mailto:roger.k.wells@leidos.com>
_______________________________________________
devel mailing list --
devel@lists.fedoraproject.org<mailto:devel@lists.fedoraproject.org>
To unsubscribe send an email to
devel-leave@lists.fedoraproject.org<mailto:devel-leave@lists.fedoraproject.org>
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
--
Roger Wells, P.E.
leidos
221 Third St
Newport, RI 02840
401-847-4210 (voice)
401-849-1585 (fax)
roger.k.wells@leidos.com<mailto:roger.k.wells@leidos.com>
1394
days inactive
1402
days old
15 comments
4 participants
participants (4)
-
Chris Murphy -
Przemek Klosowski -
Tony Nelson -
Wells, Roger K.