zstd has better compression ratio and time consumption balance than zlib/lzo/snappy. When no customized compression method specified in kdump.conf, we will use zstd as the default compression method.
**The test method:
I installed kexec-tools with and without the patch, executing the following command for 4 times, and calculate the averange time:
$ rm -f /boot/initramfs-*kdump.img && time kdumpctl rebuild && \ ls -ail /boot/initramfs-*kdump.img
**The test result:
dracut with squash module | dracut without sqaush module zlib zstd | zlib zstd real 19.981 23.5385 | 19.38475 23.5575 user 16.04475 16.84475 | 18.0245 9.28875 sys 1.61875 1.818 | 1.617 1.50475 | size of | kdump.img 38099968 35432448 | 21445168 19006000
**discussion
As for the file size of kdump.img, zstd makes a smaller size, which is benificial for kdump kernel limited memory.
As for the time consumption of making kdump.img, in dracut with squash module (squash-root.img included) case, zstd doesn't have a clear advance. In dracut without sqaush module (no squash-root.img included), zstd have a shorter user time than zlib.
Signed-off-by: Tao Liu ltao@redhat.com --- kdump-lib.sh | 6 ++++++ mkdumprd | 4 ++++ mkfadumprd | 4 ++++ 3 files changed, 14 insertions(+)
diff --git a/kdump-lib.sh b/kdump-lib.sh index e435498..e374c71 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -443,6 +443,12 @@ is_wdt_active() return 1 }
+have_compression_in_dracut_args() +{ + [[ " $(kdump_get_conf_val dracut_args)" =~ \ + .*[[:space:]]--(gzip|bzip2|lzma|xz|lzo|lz4|zstd|no-compress|compress)([[:space:]]|$) ]] +} + # If "dracut_args" contains "--mount" information, use it # directly without any check(users are expected to ensure # its correctness). diff --git a/mkdumprd b/mkdumprd index d87d588..9c26ecc 100644 --- a/mkdumprd +++ b/mkdumprd @@ -431,6 +431,10 @@ done <<< "$(kdump_read_conf)"
handle_default_dump_target
+if ! have_compression_in_dracut_args; then + add_dracut_arg "--compress" "zstd" +fi + if [[ -n $extra_modules ]]; then add_dracut_arg "--add-drivers" "$extra_modules" fi diff --git a/mkfadumprd b/mkfadumprd index b890f83..16fdacc 100644 --- a/mkfadumprd +++ b/mkfadumprd @@ -62,6 +62,10 @@ if is_squash_available; then _dracut_isolate_args+=(--add squash) fi
+if ! have_compression_in_dracut_args; then + _dracut_isolate_args+=(--compress zstd) +fi + if ! dracut --force --quiet "${_dracut_isolate_args[@]}" "$@" "$TARGET_INITRD"; then perror_exit "mkfadumprd: failed to setup '$TARGET_INITRD' with dump capture capability" fi
Hi Tao,
Thanks for providing the benchmark data.
On Fri, Dec 24, 2021 at 05:02:11PM +0800, Tao Liu wrote:
zstd has better compression ratio and time consumption balance than zlib/lzo/snappy. When no customized compression method specified in kdump.conf, we will use zstd as the default compression method.
**The test method:
I installed kexec-tools with and without the patch, executing the following command for 4 times, and calculate the averange time:
$ rm -f /boot/initramfs-*kdump.img && time kdumpctl rebuild && \ ls -ail /boot/initramfs-*kdump.img
**The test result:
dracut with squash module | dracut without sqaush module zlib zstd | zlib zstd real 19.981 23.5385 | 19.38475 23.5575 user 16.04475 16.84475 | 18.0245 9.28875
I'm curious why the real time is much larger than the user time for zstd. Could the process spend sometime in sleeping state because of waiting for IO? How about saving the initramfs to ramfs?
sys 1.61875 1.818 | 1.617 1.50475 | size of | kdump.img 38099968 35432448 | 21445168 19006000
**discussion
As for the file size of kdump.img, zstd makes a smaller size, which is benificial for kdump kernel limited memory.
As for the time consumption of making kdump.img, in dracut with squash module (squash-root.img included) case, zstd doesn't have a clear advance. In dracut without sqaush module (no squash-root.img included), zstd have a shorter user time than zlib.
Signed-off-by: Tao Liu ltao@redhat.com
kdump-lib.sh | 6 ++++++ mkdumprd | 4 ++++ mkfadumprd | 4 ++++ 3 files changed, 14 insertions(+)
diff --git a/kdump-lib.sh b/kdump-lib.sh index e435498..e374c71 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -443,6 +443,12 @@ is_wdt_active() return 1 }
+have_compression_in_dracut_args() +{
- [[ " $(kdump_get_conf_val dracut_args)" =~ \
.*[[:space:]]--(gzip|bzip2|lzma|xz|lzo|lz4|zstd|no-compress|compress)([[:space:]]|$) ]]
Would it be better to use the following regex?
+ [[ "$(kdump_get_conf_val dracut_args)" =~ \ + (^|[[:space:]])--(gzip|bzip2|lzma|xz|lzo|lz4|zstd|no-compress|compress)([[:space:]]|$) ]]
+}
# If "dracut_args" contains "--mount" information, use it # directly without any check(users are expected to ensure # its correctness). diff --git a/mkdumprd b/mkdumprd index d87d588..9c26ecc 100644 --- a/mkdumprd +++ b/mkdumprd @@ -431,6 +431,10 @@ done <<< "$(kdump_read_conf)"
handle_default_dump_target
+if ! have_compression_in_dracut_args; then
- add_dracut_arg "--compress" "zstd"
+fi
if [[ -n $extra_modules ]]; then add_dracut_arg "--add-drivers" "$extra_modules" fi diff --git a/mkfadumprd b/mkfadumprd index b890f83..16fdacc 100644 --- a/mkfadumprd +++ b/mkfadumprd @@ -62,6 +62,10 @@ if is_squash_available; then _dracut_isolate_args+=(--add squash) fi
+if ! have_compression_in_dracut_args; then
- _dracut_isolate_args+=(--compress zstd)
+fi
if ! dracut --force --quiet "${_dracut_isolate_args[@]}" "$@" "$TARGET_INITRD"; then perror_exit "mkfadumprd: failed to setup '$TARGET_INITRD' with dump capture capability" fi -- 2.33.1
Hi Coiby,
On Mon, Dec 27, 2021 at 9:55 AM Coiby Xu coxu@redhat.com wrote:
Hi Tao,
Thanks for providing the benchmark data.
On Fri, Dec 24, 2021 at 05:02:11PM +0800, Tao Liu wrote:
zstd has better compression ratio and time consumption balance than zlib/lzo/snappy. When no customized compression method specified in kdump.conf, we will use zstd as the default compression method.
**The test method:
I installed kexec-tools with and without the patch, executing the following command for 4 times, and calculate the averange time:
$ rm -f /boot/initramfs-*kdump.img && time kdumpctl rebuild && \ ls -ail /boot/initramfs-*kdump.img
**The test result:
dracut with squash module | dracut without sqaush module zlib zstd | zlib zstd
real 19.981 23.5385 | 19.38475 23.5575 user 16.04475 16.84475 | 18.0245 9.28875
I'm curious why the real time is much larger than the user time for zstd. Could the process spend sometime in sleeping state because of waiting for IO? How about saving the initramfs to ramfs?
I guess saving the initramfs to ramfs won't help, because here we only have 38M ~ 20M data to write into disk. It will only take maybe several microseconds when comparing to a few seconds of real vs user. Anyway, I'm trying to reserve the same machine to do some analysis. Please wait for a while.
sys 1.61875 1.818 | 1.617 1.50475 | size of | kdump.img 38099968 35432448 | 21445168 19006000
**discussion
As for the file size of kdump.img, zstd makes a smaller size, which is benificial for kdump kernel limited memory.
As for the time consumption of making kdump.img, in dracut with squash module (squash-root.img included) case, zstd doesn't have a clear advance. In dracut without sqaush module (no squash-root.img included), zstd have a shorter user time than zlib.
Signed-off-by: Tao Liu ltao@redhat.com
kdump-lib.sh | 6 ++++++ mkdumprd | 4 ++++ mkfadumprd | 4 ++++ 3 files changed, 14 insertions(+)
diff --git a/kdump-lib.sh b/kdump-lib.sh index e435498..e374c71 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -443,6 +443,12 @@ is_wdt_active() return 1 }
+have_compression_in_dracut_args() +{
[[ " $(kdump_get_conf_val dracut_args)" =~ \
.*[[:space:]]--(gzip|bzip2|lzma|xz|lzo|lz4|zstd|no-compress|compress)([[:space:]]|$) ]]
Would it be better to use the following regex?
[[ "$(kdump_get_conf_val dracut_args)" =~ \
(^|[[:space:]])--(gzip|bzip2|lzma|xz|lzo|lz4|zstd|no-compress|compress)([[:space:]]|$) ]]
Thanks, I agree with your modification.
Thanks, Tao Liu
+}
# If "dracut_args" contains "--mount" information, use it # directly without any check(users are expected to ensure # its correctness). diff --git a/mkdumprd b/mkdumprd index d87d588..9c26ecc 100644 --- a/mkdumprd +++ b/mkdumprd @@ -431,6 +431,10 @@ done <<< "$(kdump_read_conf)"
handle_default_dump_target
+if ! have_compression_in_dracut_args; then
add_dracut_arg "--compress" "zstd"
+fi
if [[ -n $extra_modules ]]; then add_dracut_arg "--add-drivers" "$extra_modules" fi diff --git a/mkfadumprd b/mkfadumprd index b890f83..16fdacc 100644 --- a/mkfadumprd +++ b/mkfadumprd @@ -62,6 +62,10 @@ if is_squash_available; then _dracut_isolate_args+=(--add squash) fi
+if ! have_compression_in_dracut_args; then
_dracut_isolate_args+=(--compress zstd)
+fi
if ! dracut --force --quiet "${_dracut_isolate_args[@]}" "$@" "$TARGET_INITRD"; then perror_exit "mkfadumprd: failed to setup '$TARGET_INITRD' with dump capture capability" fi -- 2.33.1
-- Best regards, Coiby