On Wed, Aug 12, 2020 at 2:33 PM Lianbo Jiang <lijiang(a)redhat.com> wrote:
Sometimes, debugging the kdump service failure becomes very challenging
because there is no additional debugging information, which requires
modification of the scripts like kdumpctl, mkdumprd, etc to collect the
information for troubleshooting.
In view of this, add 'set -x' in the scripts in order to get the complete
debugging information.
Signed-off-by: Lianbo Jiang <lijiang(a)redhat.com>
---
kdumpctl | 10 ++++++++++
mkdumprd | 10 ++++++++++
2 files changed, 20 insertions(+)
diff --git a/kdumpctl b/kdumpctl
index 122e065b8c69..4138f8798d5e 100755
--- a/kdumpctl
+++ b/kdumpctl
@@ -5,6 +5,7 @@ KDUMP_KERNELVER=""
KDUMP_COMMANDLINE=""
KEXEC_ARGS=""
KDUMP_CONFIG_FILE="/etc/kdump.conf"
+KDUMP_LOG_PATH="/var/log"
MKDUMPRD="/sbin/mkdumprd -f"
DRACUT_MODULES_FILE="/usr/lib/dracut/modules.txt"
SAVE_PATH=/var/crash
@@ -32,6 +33,15 @@ if [ -f /etc/sysconfig/kdump ]; then
. /etc/sysconfig/kdump
fi
+if [ ! -d $KDUMP_LOG_PATH ]; then
+ mkdir -p $KDUMP_LOG_PATH
+fi
Hi Lianbo
If KDUMP_LOG_PATH is always "/var/log", I think there is no need to
check if it exists.
If /var/log is deleted, the system will be so broken and unusable.
Better just error out.
+
+#redirect stderr and apend debug messages to a file
+exec 2>> $KDUMP_LOG_PATH/kdump.log
If we just redirect the stderr, then some simple error message from
kexec or other command will be omitted and redirect to the log file,
like error message from kexec that kernel key not available or
reserved memory not enough.
I think that will make it harder to debug most failures.
Maybe we can use some trick here like:
exec >&1 2> >(tee -a $KDUMP_LOG_PATH/kdump.log | grep -v ^+KDUMP_DEBUG
>&2)
export PS4='+KDUMP_DEBUG $(date "+%Y-%m-%d %H:%M:%S")
${BASH_SOURCE}@${LINENO}: '
set -x
This way any simple error message from kexec will be shown in
terminal, and debug logs starting with KDUMP_DEBUG will be redirected
to the log file and not spawn the console.
> +export PS4='+ $(date "+%Y-%m-%d %H:%M:%S") ${BASH_SOURCE}@${LINENO}:
'
> +set -x
> +
> single_instance_lock()
> {
> local rc timeout=5
> diff --git a/mkdumprd b/mkdumprd
> index 91ce3d5a0686..f352a026b3a7 100644
> --- a/mkdumprd
> +++ b/mkdumprd
> @@ -15,6 +15,16 @@ conf_file="/etc/kdump.conf"
> SSH_KEY_LOCATION="/root/.ssh/kdump_id_rsa"
> SAVE_PATH=$(get_save_path)
> OVERRIDE_RESETTABLE=0
> +KDUMP_LOG_PATH="/var/log"
> +
> +if [ ! -d $KDUMP_LOG_PATH ]; then
> + mkdir -p $KDUMP_LOG_PATH
> +fi
+
+#redirect stderr and apend debug messages to a file
+exec 2>> $KDUMP_LOG_PATH/kdump.log
Maybe it's better to use a different log file, like kdump-mkdumprd.log
or just mkdumprd.log
+export PS4='+ $(date "+%Y-%m-%d %H:%M:%S")
${BASH_SOURCE}@${LINENO}: '
+set -x
extra_modules=""
dracut_args="--quiet --hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode
strict -o \"plymouth dash resume ifcfg earlykdump\""
--
2.17.1
And as DaveY mentioned in another reply, set -x may have performance
issue, I can have an option to enable/disable the debug log.
--
Best Regards,
Kairui Song