I'd love to make logging during early boot not require everyone to happen to get the `console=` entries ordered correctly on the kernel command line. Inevitably we always get it wrong or the ordering of the `console=` lines on a pre-baked works great for some people but not for others. What if there was a way to get those early startup systemd unit messages on all `console=` entries? Maybe there is.
I've got some code that I think could help with this. We basically configure the journal in the initramfs to log all messages at 'info' level and above to kmsg. We also have to make the kernel not ratelimit messages during this time. Check out the proposed change and see what you think:
On 08/17/2018 08:55 AM, Dusty Mabe wrote:
I'd love to make logging during early boot not require everyone to happen to get the `console=` entries ordered correctly on the kernel command line. Inevitably we always get it wrong or the ordering of the `console=` lines on a pre-baked works great for some people but not for others. What if there was a way to get those early startup systemd unit messages on all `console=` entries? Maybe there is.
I've got some code that I think could help with this. We basically configure the journal in the initramfs to log all messages at 'info' level and above to kmsg. We also have to make the kernel not ratelimit messages during this time. Check out the proposed change and see what you think:
Probably obvious question but can you clarify what the exact problem is with logging in initramfs right now? It sounds like the issue is that if something goes wrong early it doesn't get logged to the journal so you want to output it to the console?
Thanks, Laura
On 08/17/2018 01:20 PM, Laura Abbott wrote:
On 08/17/2018 08:55 AM, Dusty Mabe wrote:
I'd love to make logging during early boot not require everyone to happen to get the `console=` entries ordered correctly on the kernel command line. Inevitably we always get it wrong or the ordering of the `console=` lines on a pre-baked works great for some people but not for others. What if there was a way to get those early startup systemd unit messages on all `console=` entries? Maybe there is.
I've got some code that I think could help with this. We basically configure the journal in the initramfs to log all messages at 'info' level and above to kmsg. We also have to make the kernel not ratelimit messages during this time. Check out the proposed change and see what you think:
Probably obvious question but can you clarify what the exact problem is with logging in initramfs right now? It sounds like the issue is that if something goes wrong early it doesn't get logged to the journal so you want to output it to the console?
Right now journal output will go to the primary console of the machine, which is the last `console=` entry on the kernel command line. It's been like this forever. It's expected.
The problem is that often enough a user either doesn't know which device is the primary console or somehow doesn't have access to the output of the device that is associated with the primary console, thus they must play a guessing game. If somehow they don't have access to that device then they are out of luck.
I'm proposing that we log journal output to kmsg during the initramfs (after switch root that won't be the case because the journald.conf settings from the root filesystem would take effect), which will mean that output goes to all `console=` devices and not just the primary one.
The reason for this request is because we are starting to use ignition [1] a lot more and debugging troubles in the initramfs is much easier if you don't have to hunt for the right output device where the magical messages are. Of course in Fedora CoreOS we could just apply these settings just for CoreOS, but this is a problem I've had myself for a long time and I'd actually like to see it applied to all of Fedora.
Dusty
On 08/17/2018 01:57 PM, Dusty Mabe wrote:
On 08/17/2018 01:20 PM, Laura Abbott wrote:
On 08/17/2018 08:55 AM, Dusty Mabe wrote:
I'd love to make logging during early boot not require everyone to happen to get the `console=` entries ordered correctly on the kernel command line. Inevitably we always get it wrong or the ordering of the `console=` lines on a pre-baked works great for some people but not for others. What if there was a way to get those early startup systemd unit messages on all `console=` entries? Maybe there is.
I've got some code that I think could help with this. We basically configure the journal in the initramfs to log all messages at 'info' level and above to kmsg. We also have to make the kernel not ratelimit messages during this time. Check out the proposed change and see what you think:
Probably obvious question but can you clarify what the exact problem is with logging in initramfs right now? It sounds like the issue is that if something goes wrong early it doesn't get logged to the journal so you want to output it to the console?
Right now journal output will go to the primary console of the machine, which is the last `console=` entry on the kernel command line. It's been like this forever. It's expected.
The problem is that often enough a user either doesn't know which device is the primary console or somehow doesn't have access to the output of the device that is associated with the primary console, thus they must play a guessing game. If somehow they don't have access to that device then they are out of luck.
I'm proposing that we log journal output to kmsg during the initramfs (after switch root that won't be the case because the journald.conf settings from the root filesystem would take effect), which will mean that output goes to all `console=` devices and not just the primary one.
The reason for this request is because we are starting to use ignition [1] a lot more and debugging troubles in the initramfs is much easier if you don't have to hunt for the right output device where the magical messages are. Of course in Fedora CoreOS we could just apply these settings just for CoreOS, but this is a problem I've had myself for a long time and I'd actually like to see it applied to all of Fedora.
Dusty
bumping this.. i would like some input if anyone has any, whether it be good or bad.
Dusty
-
Dusty Mabe -
Laura Abbott