On Wed, Jun 9, 2021 at 2:47 AM Hari Bathini <hbathini(a)linux.ibm.com> wrote:
On 07/06/21 2:24 pm, Kairui Song wrote:
> Hi Hari,
Hi Kairui,
>
> On Tue, May 25, 2021 at 5:19 PM Hari Bathini <hbathini(a)linux.ibm.com> wrote:
>>
>> In case of fadump, the initramfs image has to be built to boot into
>> the production environment as well as to offload the active crash dump
>> to the specified dump target (for boot after crash). As the same image
>> would be used for both boot scenarios, it could not be built optimally
>> while accommodating both cases.
>>
>> Since dracut commit 8f7c332e488f ("refactor(squash): structure in a
>> cleaner way"), the squash'ed image is a complete environment that can
>> be activated for a given boot scenario. Use --include to include the
>> squash'ed initramfs image built to offload active crash dump to the
>> specified dump target.
>>
>> Also, introduce a new out-of-tree dracut module (99zz-fadumpinit) that
>> installs a customized init program while moving the default /init to
>> /init.dracut.
>>
>> Thus, the customized init program is leveraged to isolate fadump image
>> within the default initramfs image by kicking off default boot process
>> (exec /init.dracut) for regular boot scenario and activating squash'ed
>> image only when an active crash dump would be available.
>>
>> Signed-off-by: Hari Bathini <hbathini(a)linux.ibm.com>
>> ---
>>
>> * Resending the patch with '--quiet' option added to dracut while
building the
>> initramfs image with capture image isolated.
>>
>> Changes in V2:
>> * Moved kernel modules installation from install() to installkernel() for
>> zz-fadumpinit dracut module.
>> * Updated kexec-tools.spec to install zz-fadumpinit module files appropriately.
>> * Fixed error check for capture initramfs image isolation with dracut.
>>
>>
>> dracut-fadump-init-fadump.sh | 40 ++++++++++++++++++++++++++++++++++++++++
>> dracut-fadump-module-setup.sh | 23 +++++++++++++++++++++++
>> dracut-module-setup.sh | 2 +-
>> kdumpctl | 12 +++++++++++-
>> kexec-tools.spec | 11 +++++++++++
>> 5 files changed, 86 insertions(+), 2 deletions(-)
>> create mode 100644 dracut-fadump-init-fadump.sh
>> create mode 100644 dracut-fadump-module-setup.sh
>>
>> diff --git a/dracut-fadump-init-fadump.sh b/dracut-fadump-init-fadump.sh
>> new file mode 100644
>> index 0000000..5a81cbe
>> --- /dev/null
>> +++ b/dracut-fadump-init-fadump.sh
>> @@ -0,0 +1,40 @@
>> +#!/bin/sh
>> +PATH=/bin:/sbin
>> +
>> +[ -e /proc/self/mounts ] \
>> + || (mkdir -p /proc && mount -t proc -o nosuid,noexec,nodev proc
/proc)
>> +
>> +grep -q '^sysfs /sys sysfs' /proc/self/mounts \
>> + || (mkdir -p /sys && mount -t sysfs -o nosuid,noexec,nodev sysfs
/sys)
>> +
>> +grep -q '^devtmpfs /dev devtmpfs' /proc/self/mounts \
>> + || (mkdir -p /dev && mount -t devtmpfs -o
mode=755,noexec,nosuid,strictatime devtmpfs /dev)
>> +
>> +grep -q '^tmpfs /run tmpfs' /proc/self/mounts \
>> + || (mkdir -p /run && mount -t tmpfs -o
mode=755,noexec,nosuid,strictatime tmpfs /run)
>> +
>> +if [ ! -f /proc/device-tree/rtas/ibm,kernel-dump ] && [ ! -f
/proc/device-tree/ibm,opal/dump/mpipl-boot ]; then
>> + exec /init.dracut
>> + exit
>> +fi
>> +
>> +# Load required modules
>> +modprobe loop
>> +modprobe squashfs
>> +modprobe overlay
>> +
>> +# Mount the squash image
>> +mkdir -p /squash
>> +mount -t ramfs ramfs /squash
>> +mkdir -p /squash/root /squash/overlay/upper /squash/overlay/work
>> +mount -t squashfs -o ro,loop /fadump-squash.img /squash/root
>> +
>> +# Setup new root overlay
>> +mkdir -p /newroot
>> +mount -t overlay overlay -o
lowerdir=/squash/root,upperdir=/squash/overlay/upper,workdir=/squash/overlay/work/
/newroot/
>> +
>> +# Move all mount points to new root to prepare chroot
>> +mount --move /squash /newroot/squash
>> +
>> +# Jump to new root and clean setup files
>> +SYSTEMD_IN_INITRD=lenient exec switch_root /newroot /init
>> diff --git a/dracut-fadump-module-setup.sh b/dracut-fadump-module-setup.sh
>> new file mode 100644
>> index 0000000..479154a
>> --- /dev/null
>> +++ b/dracut-fadump-module-setup.sh
>> @@ -0,0 +1,23 @@
>> +#!/bin/bash
>> +
>> +check() {
>> + return 255
>> +}
>> +
>> +depends() {
>> + return 0
>> +}
>> +
>> +installkernel() {
>> + # Install required kernel modules for the init script (init-fadump.sh)
>> + hostonly="" instmods "loop" "squashfs"
"overlay"
>> +}
>> +
>> +install() {
>> + mv -f "$initdir/init" "$initdir/init.dracut"
>> + inst_simple "$moddir/init-fadump.sh" /init
>> + chmod 0755 "$initdir/init"
>> +
>> + # Install required binaries for the init script (init-fadump.sh)
>> + inst_multiple sh mount modprobe mkdir switch_root grep
>> +}
>> diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh
>> index a99a0ea..ab48e79 100755
>> --- a/dracut-module-setup.sh
>> +++ b/dracut-module-setup.sh
>> @@ -37,7 +37,7 @@ depends() {
>> done
>> }
>>
>> - if is_squash_available && ! is_fadump_capable; then
>> + if is_squash_available; then
>> add_opt_module squash
>> else
>> dwarning "Required modules to build a squashed kdump image is
missing!"
>> diff --git a/kdumpctl b/kdumpctl
>> index 978dae5..7e617e8 100755
>> --- a/kdumpctl
>> +++ b/kdumpctl
>> @@ -18,6 +18,7 @@ DEFAULT_INITRD_BAK=""
>> KDUMP_INITRD=""
>> TARGET_INITRD=""
>> FADUMP_REGISTER_SYS_NODE="/sys/kernel/fadump_registered"
>> +FADUMP_SQUASH_IMG=fadump-squash.img
>> #kdump shall be the default dump mode
>> DEFAULT_DUMP_MODE="kdump"
>> image_time=0
>> @@ -105,6 +106,7 @@ save_core()
>> rebuild_fadump_initrd()
>> {
>> local target_initrd_tmp
>> + local _dracut_isolate_args
>>
>> # this file tells the initrd is fadump enabled
>> touch /tmp/fadump.initramfs
>> @@ -119,6 +121,14 @@ rebuild_fadump_initrd()
>> fi
>> rm -f /tmp/fadump.initramfs
>>
>> + lsinitrd $target_initrd_tmp /squash-root.img >
/tmp/$FADUMP_SQUASH_IMG
>> + _dracut_isolate_args=" --rebuild $DEFAULT_INITRD_BAK --add
zz-fadumpinit -i /tmp/$FADUMP_SQUASH_IMG /$FADUMP_SQUASH_IMG"
>> + if [ ! -s /tmp/$FADUMP_SQUASH_IMG ] \
>> + || ! dracut --force --quiet $_dracut_isolate_args
$target_initrd_tmp; then
>> + dwarn "WARNING: Could not isolate the dump capture
initramfs image."
>> + fi
>> + rm -f /tmp/$FADUMP_SQUASH_IMG
>
> I think we need to use a bash shell `trap` here, so if the script exit
> unexpectedly, /tmp/$FADUMP_SQUASH_IMG is also cleaned.
>
You mean something like `trap "rm -f /tmp/$FADUMP_SQUASH_IMG" EXIT`?
>> +
>> # updating fadump initrd
>> ddebug "updating fadump initrd: $target_initrd_tmp
$TARGET_INITRD"
>> mv $target_initrd_tmp $TARGET_INITRD
>> @@ -612,7 +622,7 @@ check_rebuild()
>> #in case of fadump mode, check whether the default/target
>> #initrd is already built with dump capture capability
>> if [ "$DEFAULT_DUMP_MODE" == "fadump" ];
then
>> - capture_capable_initrd=$(lsinitrd -f
$DRACUT_MODULES_FILE $TARGET_INITRD | grep ^kdumpbase$ | wc -l)
>> + capture_capable_initrd=$(lsinitrd -f
$DRACUT_MODULES_FILE $TARGET_INITRD | grep -e ^kdumpbase$ -e ^zz-fadumpinit$ | wc -l)
>> fi
>> fi
>>
>> diff --git a/kexec-tools.spec b/kexec-tools.spec
>> index 11c64cb..a7cbf93 100644
>> --- a/kexec-tools.spec
>> +++ b/kexec-tools.spec
>> @@ -55,6 +55,9 @@ Source107: dracut-kdump-emergency.target
>> Source108: dracut-early-kdump.sh
>> Source109: dracut-early-kdump-module-setup.sh
>>
>> +Source200: dracut-fadump-init-fadump.sh
>> +Source201: dracut-fadump-module-setup.sh
>> +
>> Requires(post): systemd-units
>> Requires(preun): systemd-units
>> Requires(postun): systemd-units
>> @@ -212,6 +215,7 @@ install -m 644 makedumpfile-%{mkdf_ver}/eppic_scripts/*
$RPM_BUILD_ROOT/usr/shar
>>
>> %define remove_dracut_prefix() %(echo -n %1|sed 's/.*dracut-//g')
>> %define remove_dracut_early_kdump_prefix() %(echo -n %1|sed
's/.*dracut-early-kdump-//g')
>> +%define remove_dracut_fadump_prefix() %(echo -n %1|sed
's/.*dracut-fadump-//g')
>>
>> # deal with dracut modules
>> mkdir -p -m755
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99kdumpbase
>> @@ -230,6 +234,13 @@ cp %{SOURCE108}
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99earlyk
>> cp %{SOURCE109}
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99earlykdump/%{remove_dracut_early_kdump_prefix
%{SOURCE109}}
>> chmod 755
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99earlykdump/%{remove_dracut_prefix
%{SOURCE108}}
>> chmod 755
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99earlykdump/%{remove_dracut_early_kdump_prefix
%{SOURCE109}}
>> +%ifarch ppc64 ppc64le
>> +mkdir -p -m755
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99zz-fadumpinit
>> +cp %{SOURCE200}
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99zz-fadumpinit/%{remove_dracut_fadump_prefix
%{SOURCE200}}
>> +cp %{SOURCE201}
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99zz-fadumpinit/%{remove_dracut_fadump_prefix
%{SOURCE201}}
>> +chmod 755
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99zz-fadumpinit/%{remove_dracut_fadump_prefix
%{SOURCE200}}
>> +chmod 755
$RPM_BUILD_ROOT/etc/kdump-adv-conf/kdump_dracut_modules/99zz-fadumpinit/%{remove_dracut_fadump_prefix
%{SOURCE201}}
>> +%endif
>>
>>
>> %define dracutlibdir %{_prefix}/lib/dracut
>>
>>
> This patch looks good to me. Just one concern,
> dracut-fadump-module-setup.sh just reimplements squash init loader,
> and it is almost identical to init-squash.sh, if we can reuse the
> init-squash.sh directly, it will make the maintenance easier.
I could think of a few changes to squash module to avoid code duplication.
Firstly, putting init-squash.sh in some commonly accessible place like
/usr/lib/squash/init-squash.sh
instead of $moddir/init-squash.sh, for zz-fadumpinit module to reuse it.
Secondly, the below change in init-squash.sh to have the flexibility to
use a different image instead
of /squash-root.img:
--- init-squash.sh.orig 2021-06-08 16:37:36.173878604 +0530
+++ init-squash.sh 2021-06-08 16:39:07.942122234 +0530
@@ -18,10 +18,14 @@
modprobe squashfs
modprobe overlay
+if [ -z "$SQUASH_IMG" ]; then
+ SQUASH_IMG="/squash-root.img"
+fi
+
# Mount the squash image
mount -t ramfs ramfs /squash
mkdir -p /squash/root /squash/overlay/upper /squash/overlay/work
-mount -t squashfs -o ro,loop /squash-root.img /squash/root
+mount -t squashfs -o ro,loop "$SQUASH_IMG" /squash/root
# Setup new root overlay
mkdir /newroot
With the above two changes, the init script for fadump could simply be:
#!/usr/bin/sh
PATH=/bin:/sbin
[ -e /proc/self/mounts ] \
|| (mkdir -p /proc && mount -t proc -o nosuid,noexec,nodev proc
/proc)
grep -q '^sysfs /sys sysfs' /proc/self/mounts \
|| (mkdir -p /sys && mount -t sysfs -o nosuid,noexec,nodev sysfs
/sys)
# Check if a fadump is active
if [ -f /proc/device-tree/rtas/ibm,kernel-dump ] || [ -f
/proc/device-tree/ibm,opal/dump/mpipl-boot ]; then
export SQUASH_IMG="/fadump-squash.img"
exec /init.fadump
else
exec /init.dracut
fi
with /usr/lib/squash/init-squash.sh installed as /init.fadump
The above changes alone are not sufficient though, as new dependencies
in init-squash.sh will break
zz-fadumpinit module. So, something like /usr/lib/squash/setup-squash.sh
is also needed, I guess..
# cat /usr/lib/squash/setup-squash.sh
#!/bin/bash
squash_dep_bins() {
echo "sh mount modprobe mkdir switch_root grep"
}
squash_dep_kmods() {
echo "loop squashfs overlay"
}
where these functions can be used to setup the dependent programs/kernel
modules appropriately. But this still comes
with a few more assumptions that the image is build a certain way and
extracted certain way though.
>
> Maybe we can figure out a way to simplify it later, like just extract
> the whole kdump initramfs into a subdirectory, like /kdumproot, and
> let init-fadump.sh chroot into.
Yeah. I do agree that an approach that has no dependency with squash
module is much better.
I am experimenting on it. Attached the WIP patch for your inputs. While
it does work, there are a
couple of open questions.
One, whether it is the optimal approach?
Two, dracut could build image with different compression methods. So,
should init-fadump.sh be made to
work for all such image types or enforce an image type and only support
it. The former method makes
maintaining init-fadump.sh tougher while the later method might make
kdump service take much
longer for fadump case (image build followed by extraction, then a
supported compression and another image build..)
Hi Hari,
For the second approach (chroot into /fadumproot), I didn't mean
extract the img file on system boot, I mean to extract the kdump
initramfs on image build, and include that directory, something like:
cd /tmp/kdumpimg
lsinitrd --unpack /boot/initramfs-kdump.img
dracut --add zz-fadumpinit -i /tmp/kdumpimg /fadumproot /boot/initramfs.img
This should work I guess? And we got rid of all the dependency issues.
And dracut has a hardlink optimization, so redundant files introduced
by including that directory should get merged and the initramfs size
should not grow too large.
>
> But the dracut squash module should be in a stable state for now, I'm
> also fine if we just stick with the current implementation.
I will wait for your inputs on the no squash dependency approach I just
purposed to make my opinion on this.
Thanks
Hari
--
Best Regards,
Kairui Song