12:46 p.m.
Hi,
my ThinkPad X220 runs Fedora 23 with automatic updates, Chromium and
Spotify Copr. Up until Friday night, it worked just fine.
Friday afternoon the new Kernel 4.4.5 was installed. It was only loaded
on Saturday when I started the machine in the afternoon. I suspended the
laptop to RAM and found that it did not woke up. I only got this strange
light show:
https://www.youtube.com/watch?v=ajoWfCtk7yE
The error (sadly) is 100% reproducible. I then went on and started with
the earlier kernels, 4.4.4 and 4.4.3, which worked fine before Friday.
(Thanks for keeping three kernels by default!) The error is just the
same, it does not wake up.
Then I also started a Kubuntu 15.04 from USB and had a slightly
different error. The machine would start the fan and the power light
would turn on, but the wifi was still off and the screen was black.
Using Arch Linux 2016.03.01 I was able to reproduce the error with
Kernel 4.4.1. Therefore I think it is not about the software any more.
This might be somewhere in the hardware or UEFI firmware.
In the discussion about systemd mounting efiwars read-write I read about
some bricked laptops by deleting the efivars. Is it possible that some
recent change (like Friday or Saturday) did something that could
potentially interfere with the suspend-to-RAM? A frind of mine who wants
to switch to Fedora wants to wait a bit as he has the same laptops and
fears that his might break as well.
The laptop is four years old and has been used all the time, I use it as
my desktop on a docking-station. Warranty has experied a year ago, so I
have to fix it myself with help. What can I do to get the system to wake
up normally again?
Regards,
Martin Ueding
2:03 p.m.
On Sun, Mar 20, 2016 at 11:46 AM, Martin Ueding <lists(a)martin-ueding.de> wrote:
In the discussion about systemd mounting efiwars read-write I read
about
some bricked laptops by deleting the efivars.
Yours still boots. Bricked laptops don't boot. So I don't think it's related.
Is it possible that some
recent change (like Friday or Saturday) did something that could
potentially interfere with the suspend-to-RAM?
Seems unlikely but you can look at the changelog to see if anything
relates to your hardware. The x220 is sufficiently common in Linux
land that if there's a software bug that's causing this, it won't take
much longer for people to start going crazy and this shows up on
reddit or hn.
https://cdn.kernel.org/pub/linux/kernel/v4.x/ChangeLog-4.4.5
Looks like there's a new firmware for this laptop model. Might make it
better, scant chance it makes things worse but then you can probably
extract a a few hundred bucks from Lenovo off a new laptop if it does.
http://support.lenovo.com/us/en/products/laptops-and-netbooks/thinkpad-x-...
--
Chris Murphy
3 p.m.
Hi,
my ThinkPad X220 runs Fedora 23 with automatic updates, Chromium and
Spotify Copr. Up until Friday night, it worked just fine.
Friday afternoon the new Kernel 4.4.5 was installed. It was only loaded
on Saturday when I started the machine in the afternoon. I suspended the
laptop to RAM and found that it did not woke up. I only got this strange
light show:
https://www.youtube.com/watch?v=ajoWfCtk7yE
The error (sadly) is 100% reproducible. I then went on and started with
the earlier kernels, 4.4.4 and 4.4.3, which worked fine before Friday.
(Thanks for keeping three kernels by default!) The error is just the
same, it does not wake up.
There are more kernels available here:
http://koji.fedoraproject.org/koji/packageinfo?packageID=8
You should also try kernel-4.4.4-300.fc23 and kernel-4.4.4-301.fc23 to best find the point
of regression. Then please file a bug into bugzilla against kernel, and describe your
problem. If you have localized your issue well between two minor updates, there's a
high chance the kernel team can find the regression and fix it (also feel free to poke
someone at #fedora-kernel if you see no feedback on that bug report).
However, I have to say that I also have X220 at home, I tested it with kernel 4.4.5-300
and found no issues with suspend. Your hardware doesn't have to be exactly the same as
mine, though. It just means that not all X220 laptops are affected.
2:35 p.m.
If you also have an X220, this seems like I have an hardware issue. As
there are new kernels every week and I boot my machine once a day, the
chance of booting with a new kernel is not that small.
In the UEFI I noticed that I cannot save anything any more. Therefore I
fear that the UEFI chip has some sort of hardware malfunction. A UEFI
upgrade did not really change anything.
Screenshots are a little down in the question:
http://unix.stackexchange.com/q/271045
What kind of CPU do you have in your X220? I have the Intel Core i5-2520M.
2:54 p.m.
On Wed, Mar 23, 2016 at 1:35 PM, Martin Ueding <lists(a)martin-ueding.de> wrote:
If you also have an X220, this seems like I have an hardware issue.
As
there are new kernels every week and I boot my machine once a day, the
chance of booting with a new kernel is not that small.
In the UEFI I noticed that I cannot save anything any more. Therefore I
fear that the UEFI chip has some sort of hardware malfunction. A UEFI
upgrade did not really change anything.
Screenshots are a little down in the question:
http://unix.stackexchange.com/q/271045
What kind of CPU do you have in your X220? I have the Intel Core i5-2520M.
What do you get for 'efibootmgr -v' and 'ls -l
/sys/firmware/efi/efivars/'
I'd sooner think there's NVRAM corruption. It's so common Apple has
had a keyboard shortcut to blow away NVRAM contents for something like
20+ years across every Mac hardware model they've made.
--
Chris Murphy
5:59 a.m.
Am 23.03.2016 um 20:54 schrieb Chris Murphy:
What do you get for 'efibootmgr -v' and 'ls -l
/sys/firmware/efi/efivars/'
I attached the output. I am not sure how to read that, it looks like
there is actual content.
I'd sooner think there's NVRAM corruption. It's so common
Apple has
had a keyboard shortcut to blow away NVRAM contents for something like
20+ years across every Mac hardware model they've made.
That would also explain why I cannot save anything in the UEFI any more.
Is this something fixable?
2:42 p.m.
On Sun, Apr 3, 2016 at 4:59 AM, Martin Ueding <lists(a)martin-ueding.de> wrote:
Am 23.03.2016 um 20:54 schrieb Chris Murphy:
> What do you get for 'efibootmgr -v' and 'ls -l
/sys/firmware/efi/efivars/'
I attached the output. I am not sure how to read that, it looks like
there is actual content.
> I'd sooner think there's NVRAM corruption. It's so common Apple has
> had a keyboard shortcut to blow away NVRAM contents for something like
> 20+ years across every Mac hardware model they've made.
That would also explain why I cannot save anything in the UEFI any more.
Is this something fixable?
Yep it looks full of superfluous boot entries. There were some
combination of firmware and shim.efi that caused this problem.
You can delete Boot0019 through Boot0129. Use either 'efibootmgr -b
XXXX -B' or rm from efivars/ but I wouldn't try deleting 0000 through
0018. Basically don't delete anything you don't know exactly what it
does. But any boot entry that's pointing to shim.efi can be removed.
Next make sure you have a current shim.efi. I can't tell you if the
Fedora shim package overwrites EFI/BOOT/ so I would just remove
whatever is in there, and then 'dnf reinstall shim grub2-efi' and
that'll repopulate both EFI/BOOT/ and EFI/fedora/ with the right
things. While it shouldn't matter, you could top it all off with
'grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg' to get it back to a
stock state.
# sha256sum /boot/efi/EFI/BOOT/BOOTX64.EFI
475552c7476ad45e42344eee8b30d44c264d200ac2468428aa86fc8795fb6e34
/boot/efi/EFI/BOOT/BOOTX64.EFI
What you should have now at reboot is a one time fallback to
BOOT/BOOTX64.EFI which won't find a proper boot entry, and it'll
create one and one only. And you won't even notice that happen
probably. So reboot a few times and check 'efibootmgr -v' to see if
you start getting accumulating boot entries again or not.
And if you haven't done it already make certain the firmware is up to date.
--
Chris Murphy
11:11 a.m.
Hi Chris,
Am 03.04.2016 um 21:42 schrieb Chris Murphy:
You can delete Boot0019 through Boot0129. Use either 'efibootmgr
-b
XXXX -B'
Funny enough, the first delete failed due to lack of memory! So there
was some hard limit reached.
or rm from efivars/ but I wouldn't try deleting 0000 through
0018.
Deleting a couple through `efivars` also got `efibootmgr -b XXXX -B`
back to work.
Basically don't delete anything you don't know exactly what
it
does. But any boot entry that's pointing to shim.efi can be removed.
I deleted the kubuntu and all the fedora boot entries using a little
script that I just wrote to clean up entries.
Next make sure you have a current shim.efi. I can't tell you if
the
Fedora shim package overwrites EFI/BOOT/ so I would just remove
whatever is in there, and then 'dnf reinstall shim grub2-efi' and
that'll repopulate both EFI/BOOT/ and EFI/fedora/ with the right
things. While it shouldn't matter, you could top it all off with
'grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg' to get it back to a
stock state.
Done all that.
# sha256sum /boot/efi/EFI/BOOT/BOOTX64.EFI
475552c7476ad45e42344eee8b30d44c264d200ac2468428aa86fc8795fb6e34
/boot/efi/EFI/BOOT/BOOTX64.EFI
This checksum matched before and after the process.
What you should have now at reboot is a one time fallback to
BOOT/BOOTX64.EFI which won't find a proper boot entry, and it'll
create one and one only. And you won't even notice that happen
probably.
The first time, there was some message there, but it was too quick to
read fully.
So reboot a few times and check 'efibootmgr -v' to see if
you start getting accumulating boot entries again or not.
Sadly, they keep accumulating. After the first boot I had 32 Fedora
entries, then a couple of rounds got me to 35 and now I am at 37.
And if you haven't done it already make certain the firmware is
up to date.
I tried to update the UEFI firmware using Windows (as UEFI does not boot
a 16-bit DOS from USB-Stick), but that was not the very latest version.
But still newer than the one that was previously installed.
All in all I can make changes in the UEFI again, they actually work and
the machine reboot. Then I dared to to a suspend-to-ram and the laptop
actually woke up. So thank you very, very much! Do you want a box of
beer or something? :-)
The core problem (accumulating boot entries) apparently persists, but I
can now use my script at startup as a workaround to clean up those
entries again.
What can I do to fix the core problem?
Regards
Martin
1:48 p.m.
On Mon, Apr 4, 2016 at 10:11 AM, Martin Ueding <lists(a)martin-ueding.de> wrote:
Hi Chris,
Am 03.04.2016 um 21:42 schrieb Chris Murphy:
> So reboot a few times and check 'efibootmgr -v' to see
if
> you start getting accumulating boot entries again or not.
Sadly, they keep accumulating. After the first boot I had 32 Fedora
entries, then a couple of rounds got me to 35 and now I am at 37.
I wonder if this is an NVRAM garbage collection problem. That is, you
deleted all Fedora entries, rebooted, and after reboot you found 32
Fedora entries? I'm willing to bet they just re-appeared due to
improper (or slow) garbage collection. It might just be delayed, so
try deleting them all again, and waiting a while (?no idea 1 minute?
10 minutes?). I'd also try a power off cold boot rather than a reboot
and see if that makes a difference.
That you keep seeing more entries being added is a bug. I'm just not
sure whether it's a bug in shim or the firmware. Off hand I'd say it
sounds like the firmware itself isn't honoring boot order, otherwise
it wouldn't be falling back to EFI/BOOT/BOOTX64.efi, it would use the
path to shim given in the first boot order entry, and therefore an
additional entry wouldn't be added to NVRAM. It maybe be worth filing
an upstream bug with all the details (make/model, firmware version,
shim version)
https://github.com/rhinstaller/shim/issues
I'd suggest trying to understand the problem better and get it fixed,
either with an even newer firmware update, or if upstream has any
ideas. Failing that, if the hardware doesn't support Secure Boot or
you aren't using it, it should be possible to remove EFI/BOOT/* and
copy EFI/fedora/grubx64.efi to EFI/BOOT/BOOTX64.efi along with its
grub.cfg, and now the firmware will use GRUB directly, since it seems
to only depend on the fallback bootloader no matter what's in NVRAM.
That's kinda ugly and it might break, I don't actually know where
grubx64.efi is looking for grub.cfg, I think it looks in the same
directory as the bootloader. A gotcha is you'll probably want to
create a new link from /etc/grub2-efi.cfg to this new location,
otherwise any kernel updates will have grubby looking to modify the
grub.cfg in the old location, and you won't see new kernels in the
GRUB menu.
--
Chris Murphy
2956
days inactive
2971
days old
8 comments
3 participants
participants (3)
-
Chris Murphy -
Kamil Paral -
Martin Ueding